OpenAMQ crashes FAQ

Any registered Wikidot user can edit or create a FAQ.

What to do if amq_server crashes

If the broker crashes, the formal logs (daily/alert) files won't say anything useful. Most users run the broker with stdout and stderr redirected to a console log, and this will contain the assertion failure or runtime error that caused the crash.

A backtrace is essential to diagnosing the issue in most cases, so please enable core dumps ("ulimit -c unlimited" on Linux) and get a back trace. For this to be useful, build the software in debug mode (BOOM_MODEL=debug,mt).

When you get a crash, create an issue on www.openamq.org, with the backtrace, and the last section of the console log. Also, explain what version you are running, your OS, and how you reproduce the problem, if possible.

Finally, run the most recent releases so that you are not hitting a problem that has been fixed.

amq_server crashes, in Direct Mode

Direct Mode can be quite greedy on sockets. 1.3c5 and previous versions did not show a clear message when the process ran out of sockets. Later versions are more explicit. In any case if you use Direct Mode you need to configure your per-process limits to allow 2+N sockets per client where N is the number of exchanges published to, and the number of queues consumed from by each client.

amq_server crashes, while running

Please try to run in debug mode if you are not in very high traffic situations, and enable core dumps on all servers (on Linux, "ulimit -c unlimited"). Then, when OpenAMQ crashes (which is rare but not impossible), you will get a core dump that helps us try to see what happened. Load the core dump using gdb (for other platforms, use the appropriate debugger), and ask for a backtrace using the command 'bt'. For example:

gdb amq_server core.11234
bt

Then send us the back trace. Note that this is only useful if you built the server in debug mode.

After I rebuilt OpenAMQ, it crashes very horribly

You may have a mixture of builds in your executable, with structures that don't match between libraries and calling programs. Do a clean build.

Server consumes all system memory, leading to 100% CPU usage

The most usual cause for this problem is that a publisher is sending messages to a queue, but no-one is reading them off the queue, or the consumers are just too slow. OpenAMQ has some built-in limits but these may be too high, they are designed for big servers.

First, run the server with —dump-trace 1 and observe the console output. It will say something like this:

2007-08-10 17:29:39: I: cnn=2 msg=32947 mem=32948K/168081K exc=8 que=1 csm=1

Here it's pretty clear, the server is being overloaded with messages (32k of them) in one queue, with one consumer that is (presumably) too slow.

Configure your queues to have a lower limit (the default is 50k messages). Create a file called amq_server.cfg that contains this:

<?xml?>
<config>
    <queue_profile name = "private">
        <limit name = "warn" value = "1000" />
        <limit name = "trim" value = "5000" />
    </queue_profile>

    <queue_profile name = "shared">
        <limit name = "warn" value = "1000" />
        <limit name = "kill" value = "5000" />
    </queue_profile>
</config>

Comments

Add a New Comment

Edit | Files | Tags | Source | Print

rating: 0+x

Author

Pieter Hintjens <moc.xitami|hp#moc.xitami|hp>

All FAQs

OpenAMQ licensing FAQ: Read this if you have questions about licensing OpenAMQ, including GPL concerns.

OpenAMQ languages FAQ: Read this if you have questions about whether you can use OpenAMQ from your particular programming language, before you ask the openamq-dev mailing list.

OpenAMQ freezes FAQ: Read this if your OpenAMQ server seems to freeze, before you ask the openamq-dev mailing list.

OpenAMQ crashes FAQ: Read this if your OpenAMQ server crashes, before you ask the openamq-dev mailing list.

OpenAMQ build FAQ: Read this if you have questions about building OpenAMQ, before you ask the openamq-dev mailing list.