This week I have been onsite with Gary Tully, one of my colleagues from our Dublin office. We were visiting a customer having a very simple sounding requirement for Active MQ. His company contributes contents to various web pages and portals all over the internet and
he has a Tomcat infrastructure in place for providing the content. The next step is to note
whether a visitor to a site has recently visited other pages where he also provided some content.
The asynchronus link between the web application and the database is realized using the Fuse release of Active MQ 5.2. One requirement that sounded very odd in the beginning was to use persistent messages, but at the same time allow to drop messages in cases of heavy load. Translated that basically means to gather as much statistical data as possible, but it doesn't matter if we can't keep up with the load entirely.
Now what has this to do with messaging and ActiveMQ? - Well, we have a quite typical slow consumer problem here. We are producing messages much faster than we can consume them, because creating a message is more or less a string concatenation while consuming the message involves a database insert. As a result any buffer - on disk or in memory - might be filled up during longer periods of high traffic. The normal messaging behavior - also with ActiveMQ - is to slow down the message producer so that the consumers get a chance to catch up and ensure that no message is lost. In our case that would mean that the response times for delivering the content would decline, which should not happen. The solution we are looking for is to give the message broker a well defined amount of space for buffering messages and simply throw back an exception when the buffer is full.
That sounds easy enough and we were thinking along the lines of putting in place a proper system usage section in the broker configuration. We started looking at the default activemq.xml that is provided with an ActiveMQ download and in there we found the following section:
The memory usage is easy enough to understand and from Gary I learned, that the temporary storage is used for spooling out messages that have been sent
NON_PERSISTENTwhile the storage is used for those messages that have been sent
PERSISTENT. As we were sending only persistent messages to keep as much of the statistical data as possible even in case of broker restarts we simply modified the storage section.
To turn the behavior in case of reaching the buffer limits, the systemUsage xbean understands a boolean property
sendFailIfNoSpace. Setting this to true should simply throw a
JMSException instead of blocking the send call.
As a result, it is important to explicitly specify a persistence adapter within the configuration file and reference it from the broker and the storage definition as shown in the configuration below.
One caveat in the set up is to make sure that the
maximumFileLengthdefined in the persistence adapter is less than half the maximum storage size. Otherwise only one data file is used by the underlying store and that will never get cleaned up. Having maximumFileLength set correctly ensures that we will have at least 2 data files.
When we finally were on our way home we decided that we need to get our findings in writing so that we could reference it in the future. I am working with Gary to get the test cases we produced and the configuration files we created into the ActiveMQ project.