Tomcat/tcServer session manager with attribute replication
I’d like to think that my programming projects don’t suffer from philosophical schizophrenia but that I simply move from point A to point B so fast it just looks that way. Unfortunately, sometimes I have to come face-to-face with this and accept it for what it is: my programming projects suffer from philosophical schizophrenia.
I say this because I’ve been hacking on some changes to the virtual/hybrid cloud Tomcat and tcServer session manager and I went through several stages while I was trying to solve some basic problems.
For one, how do I get a serialized object in the cloud when I don’t know where it is? RabbitMQ comes to my rescue here because I think I’ve finally settled on the most efficient way to answer this question: bind a queue to a topic exchange using the object’s key or identifier (in this case, a session ID). Then any messages for that object automagically get routed to the right VM (the one that has that object in memory). I’m thinking this idea can be extended to create an asynchronous, RabbitMQ-backed, Map implementation.
Now that I have the object, how do I keep it in sync with the “master”? In my case, I send a replication message out whenever a session’s (set|remove)Attribute methods are called and the value objects differ. One notable problem that I don’t see being easily overcome (but thankfully, doesn’t apply to my scenario) is if there are listeners on sessions. I don’t have RabbitMQ wired into the internal Catalina session event mechanism. I could add that at some point, but for the moment, I think this kind of dumb saving/loading via RabbitMQ messages will work for what we’re doing.
I’ve now switched back to a ONEFORALL type operation mode which means there is only one real session object that resides in an internal map on the server who first created it. Whenever another server sees a request for this session, it will send a load message to this queue every time it needs it. That last part is important: it loads a session object every time it needs it. When code sets an attribute on server TC3, that attribute is replicated back to the original server (TC1) so subsequent session loads get that updated object. I’m still trying to wrap my head around how I want to handle replication in case of node failures. No good answer on that one, yet.
REPLICATED mode is my next task. In simplifying this code, I focussed on getting the ONEFORALL mode working right to begin with. Now I can go back and make it more performant by cranking out a little extra code to handle replication events.
Initial smoke tests seem to indicate this works pretty well. Session load times in my testing were around 20-60 milliseconds. Depending on the location of your RabbitMQ server and your network topology, you might experience different results. I’m sanding off a few rough edges now and I’ll be testing this on a new set of cloud servers we’re setting up as part of a new vSphere/vCenter-managed cloud.
As always, the code is available on GitHub: