LSDS cache clear fails in the cloud

Andy Knipp

It is as if OT does not understand cloud computing. I suspect they do not.

So before (LSDS 7/8) there was a java bean that said how long the cache clear file would stay (typically about 20 minutes). This is great in a cloud deployment, because you push to a location that is read by some random number of servers (depending on load, there could be 2, there could be 7). The TS server or OD receiver do not know the hostnames of the servers reading the data.

The new behavior in 16.3 (specifically not a bug), is that one instance sees the cache clear file, renames it to processing, clears the cache and deletes the file. Since all servers share the same cache clear, that means one is correct and the remainder are wrong.

OT has offered some convoluted suggestions that will not work in an autoscaling environment (which is why I doubt they understand the cloud at all).. Our choices are ugly and uglier.

Right now for anyone who is planning to use LSDS, you can just dedicate a single larger box and not autoscale or you can write some convoluted code to get the cache clear file to a instance specific location for processing. We are doing the latter.