Background to DMP and OpenAMQ's Direct Mode

Summary

Default AMQP messaging (Basic.Publish, Basic.Deliver) is difficult to optimize. It is faster than traditional messaging but slower than performance-focused products like ØMQ. We designed and implemented the Direct Message Protocol (DMP), which solves the performance issues in AMQP. OpenAMQ's Direct Mode is the first implementation of the gro.pqma.ikiw|PMD-4#gro.pqma.ikiw|PMD-4 protocol which is described at http://wiki.amqp.org/spec:4.

With default AMQP messaging, an OpenAMQ client application can send and receive about 20,000 messages per second, and a broker can handle about 120,000 messages per second. With Direct Mode the same application can send or receive 130,000 messages per second, and brokers can handle 590,000 messages per second. Latencies are also lower (185 vs 250 usec), and more stable. Client applications further benefit from reduced CPU usage at the same message loads. Applications need no changes to use Direct Mode, which is invisibly implemented by the OpenAMQ WireAPI client stack.

As well as performance gains, which may not be systematic across other AMQP products, DMP makes it cheaper to manage AMQP infrastructures. With default AMQP messaging, slow or blocked clients create message backlogs on the server. This easily causes servers to run out of memory, and die. DMP puts the burden of message backlog on the client. This solves the problem of message backlogs on the server, and removes the main cause for operator intervention in large-scale AMQP infrastructures.

We developed DMP for three reasons. First, to give our community access to faster messaging, and do it in an interoperable form. Second, to make life easier for our clients who today spend too much on managing their AMQP infrastructures. Third, to explore techniques for optimized high-performance message transfer, which is part of our research work into AMQP/1.0.

To promote interoperability, we wrote DMP as a free and open specification that AMQP developer could add to their stack. We estimate the cost of adding DMP support to a client or server stack as 2-5 days, and this document is intended to make that process faster. DMP is hosted on the http://wiki.amqp.org site and governed by the policies and licenses of that site.

Origins of DMP

We will look at the theoretical basis for DMP as part of the AMQP evolutionary process. The specification addresses two basic areas of AMQP that can be improved, and by addressing these, shows us how to do simpler, faster message transfer.

These two areas are:

1. The use of server-held queues, where we think the AMQP design is overly and dangerously centralized.
2. The use of a binary framing for the AMQP protocol, which we think creates avoidable costs.

The Private Queue Problem

Every AMQP message must always be routed via a queue in the server. This is in our analysis redundant for the majority of use cases. Consider the three basic messaging scenarios:

• Point-to-point, where one application sends a message to another. The sender publishes a message to an exchange, which routes it to a private queue. The recipient consumes messages off its private queue and processes them.

• Data distribution, where one application sends a message to many others. The sender publishes a message to an exchange, which routes it to multiple private queues. The recipients each consume messages off their respective private queues, and process them.

• Workload distribution, where one application requests a service implemented by a group of applications. The sender publishes a message to an exchange, which routes it to a single shared queue. The recipients all consume off this shared queue, and get messages in a round-robin fashion.

What is missing from the above explanation is that each recipient has a further, hidden queue that is implemented by the client-side AMQP API. This hidden queue is invisible to the protocol but it is necessary because AMQP delivers messages to recipients asynchronously, except in simplistic single-threaded client stacks. AMQP does have mechanisms to send messages synchronously, but these are not meaningful in any performance-sensitive scenario.

In the point-to-point and data distribution scenarios, which are the heavy lifters in messaging, the server-side private queue, and the hidden client-side private queue do exactly the same work. It may seem obvious to hold private queues on a central server, and this feature has been in AMQP since its inception. But this design adds latency, allows misbehaving clients to put stress on the server, and makes things more complex than they need to be by asking that applications create and manage extra entities.

We can review the costs and risks of storing private queues on the server:

• It allows slow consumers to create memory problems on the server, which need to be managed by setting queue limits, discarding messages, etc. This means that unless explicitly managed, servers are vulnerable in times of excess traffic, while we want the opposite: stability under stress.
• It creates extra latency, as messages are queued onto the server-side private queue and then dequeued onto the client-side private queue. This extra latency penalizes applications which need dependably low latency.
• It demands the concept of "consumer" to regulate the flow of messages between these two private queues.
• It excludes distributed peer-to-peer architectures, which are needed for ultra low-latency messaging.

Of these issues, the first is most visible to today's users of AMQP, and the main immediate benefit of using DMP. Misbehaving applications are a very common problem, and managing their misbehaviour is usually the main work of system administrators who look after AMQP infrastructures. DMP resolves this problem by moving private queues from the server to the client. When private queues overflow, it's the client that crashes, not the server. This is a good solution: it forces poorly written applications to improve, and leaves servers safe.

So the first change that underpins DMP is to bypass private queues on the server, and to send messages directly to the hidden client-side private queue. This diagram shows the classic AMQP flow from sender to recipient:

[S] --> [X ==> Q] --> [q --> R]

Where 'S' is the sender, 'X' is an exchange, 'Q' is a private queue, 'q' is an invisible queue, and 'R' is the recipient. Each bracketed set represents a network node, and "—>" arrows represent message flows. The "==>" arrow represents message flows pulled by a binding. When we bypass the private queue, we get this:

[S] --> [X] ==> [q --> R]

In fact, we can generalize this model so that every message recipient (exchanges as well as applications) have invisible private queues for incoming messages, and these queues can be fed by bindings from other points of the network. This more generalized architecture is not needed for DMP but will come back in future protocol designs.

We can therefore remove private queues entirely from our schema, they are at every recipient edge:

[S] --> [X] ==> [R]

This model is also compatible with federation (multi-broker) and distributed peer-to-peer (zero-broker) architectures, but that is a pattern to explore later, elsewhere. The knowledge gained from making DMP makes this exploration easier.

Note that the binding (==>) now also replaces the consumer. When we use direct messaging, we do not need consumers. Applications control the flow of messages into their private queues by creating and destroying bindings.

There is one potential problem with direct messaging. If the network link to a client is slow, data will back-up into the server and the exchange will not be able to deliver data. This may be cause to queue data being sent, but in our experience the ideal response to saturation is to drop data (a "fail fast" approach that will helpfully break poorly-written applications) rather than to push the problem upstream where it will cause damage to innocent parts of the architecture.

The Binary Protocol Problem

AMQP is a binary protocol. This means that methods like Queue.Create are defined as binary frames in which every field sits in a well-defined place, encoded carefully to make the best use of space. Binary encoding of numbers is more compact than string encoding. "123" needs one octet in binary and three as a string. Because AMQP is a binary protocol, it is fast to parse. Strings are safe, since there is no text parsing to do. AMQP's binary encoding means AMQP software is not at risk of buffer overflow attacks. And it's easy to generate the encoding/decoding routines, since AMQP is defined as easy-to-parse XML.

These sound like strong advantages but they come at a cost. Furthermore, we can show that while the costs are real, the advantages are largely worthless. The main justification for the binary encoding was performance. Yet AMQP is slower than, for example, FTP.

AMQP's basic assumption that binary encoding is needed can be broken into more detailed assumptions, each wrong:

• That it is necessary to optimise control commands like Queue.Create. The assumption is that such commands are relevant to performance. However, they form an insignificant part of the messaging activity, the almost total mass being message transfer, not control.

• That control commands need to occupy the same network connection as messages. The assumption is that a logical package of control commands and message data must travel on the same physical path. In fact they can travel on very different paths, even across different parts of the network.

• That the encoding for control commands and for message transfer need to be the same. In fact, there is no reason for trying to use a single encoding model, and we can get benefits from allowing each part of the protocol use the best encoding, whatever that is.

A simplifying change to AMQP would be to separate control from data, and then use the simplest possible encoding form for commands, and the simplest possible encoding form for messages. It worth emphasising the importance of simplicity, especially in young protocols that must support growth.

Separating control from data makes the dialogue between client and server much simpler. One of the big barriers to writing AMQP clients is that they have to understand a complex set of exchanges that mix control and data. It turns out that splitting these apart makes the two separate dialogues significantly simpler than the single combined dialogue.

The natural semantic for control commands is pessimistic synchronous dialogue in which every request is acknowledged by a success / failure response. This is the model is used in most Internet protocols. It assumes that the risk of failure for any step is high, and that performance is not an issue. The client therefore gets an explicit response to every request and it checks this response before continuing with the next request. The synchronous semantic is slow because it involves a round trip for each step.

The natural semantic for data transfer is optimistic asynchronous monologue in which one party shoves data as fast as possible to another, not waiting for any response whatsoever. This is the model used in every performance-oriented Internet protocol. It maps to the concept of "streaming". It assumes that the risk of failure is low, and therefore the client can send data without waiting for responses. Data can be batched, and network actions minimized. Data loss is ignored, or recovered by resending.

AMQP does allow both synchronous and asynchronous dialogues but it's not tied to the natural semantics of control and data. The natural semantics are weakly bounded, insufficiently inevitable. And these weak boundaries are fully exploited as people experiment with asynchronous control and synchronous data, creating unnatural semantics.

One reason FTP is both widely implemented, and very fast, is that it separates control commands from message transfer, and uses the natural semantics for both.

Having separated the two types of work, we're free to choose the simplest encoding for each. The simplest possible encoding for commands is in the form of text, with (for example) the 'Header: value' syntax that is well known from HTTP, SMTP, etc. This is trivial to parse using regular expressions. Attacks on this kind of encoding are well understood and they are easy to deal with. There are no data types, everything is a string.

Using a text encoding for commands would simplify many aspects of AMQP:

• It becomes easier to understand.
• It becomes easier to keep compatibility.
• It becomes easier to write clients.
• It becomes easier to make test cases.

Does text parsing create a performance penalty? Yes, but it is irrelevant to overall performance, because control commands are used only for a few seconds in conversations that can last for days or weeks.

What then is the simplest possible encoding for messages? AMQP defines a non-trivial envelope around messages (around 60-100 octets), which may be fine for large messages, but is inefficient for small messages and high throughput rates.

When we developed ØMQ, we wondered just how small the message envelope could get. The answer is, we can reduce it to a single octet. The simplest message encoding has a 1-octet header that encodes a 7-bit size and a 1-bit continuation indicator:

[octet][0 to 127 bytes of binary data]

Empirical tests also show that this is the most efficient encoding for random message sizes. We can of course define other encodings, each with their own costs and benefits. For routing, for example, we need an envelope with properties such as a routing key.

The next question is, "how do we mix those simple text-based control commands with that simple message encoding?" There are several answers:

• We can wrap binary messages in textual envelopes. This is how BEEP and HTTP work. This single-connection design looks simpler but in fact becomes complex to understand and implement, and it is inefficient.

• We can use distinct connections for control commands and for messages, like FTP. This is simple but means we need to manage multiple ports. It also creates problems with firewalls. It feels wrong: it should be possible to do everything on the single AMQP port 5672.

• We can start with a simple text-based control model and switch to simple binary message encoding if we decide to start message transfer. This is analogous to how TLS switches from an insecure to an encrypted connection.

The last option looks best because it also separates control and data, thus lets us keep the two natural semantics clearly separated. Mixing them, as AMQP does today, creates some extraordinarily delicate problems, such as how to handle errors that can hit both synchronous and asynchronous dialogues. AMQP's exception handling is an elegant solution but it solves a problem we would rather see disappear. By splitting control and data into two separate dialogues, error handling also becomes much more conventional.

DMP does not address AMQP's control protocol, we'll examine this later, elsewhere. But it does present a solution for the message protocol.

How DMP works

DMP aims to be abstracted enough to be independent of the specifics of AMQP as it exists today. DMP connects a client application to a "sink" or "feed" on a server application. In a contemporary AMQP architecture a sink is an exchange, and a feed is a server-held private queue.

The client initiates a DMP connection by requesting a lease to send messages to a sink, or receive messages from a feed. It does this on an authenticated session (the main AMQP channel). The server responds by providing a lease, or raising an exception. The client then opens a fresh connection to the server and provides the lease. The server validates the lease and then switches the connection to DMP message transfers. DMP currently operates over TCP/IP.

Many clients can connect to the same sink, but feeds are exclusive to single clients. This is coherent with the mapping of sinks to exchanges, and feeds to server-side private queues.

DMP is aimed at typical high-volume applications that publish to a very small number of exchanges or consume from a very small number of private queues. Since DMP increases the number of client-to-server connections, it can be problematic for servers that have to work with very large (thousands) of clients at once.

Clients can open and close DMP connections at any time. They are independent of each other and when active, may be independent of the main AMQP connection.

The current DMP specification defines a fixed message envelope which encodes all data in the AMQP message envelope. This envelope uses 12 octets plus header fields. The envelope is designed to make message batching very easy. DMP may offer thinner envelopes in the future for high-performance market data style scenarios.

DMP integration with AMQP works through an extension class called "Direct". The Direct class has two methods, Direct.Put and Direct.Get, each with a response, respectively Direct.Put-Ok and Direct.Get-Ok. Direct.Put requests a lease for a sink (an exchange), and Direct.Get requests a lease for a feed (a private queue). Having received a lease, the client application opens a new connection and initiates the DMP dialogue.

The technical details are documented in the DMP specifications on http://wiki.amqp.org/spec:4.

OpenAMQ Direct Mode

OpenAMQ's implementation of DMP is called Direct Mode. It demonstrates how to implement DMP with minimal cost to existing applications.

Overall, Direct Mode is handled inside OpenAMQ's WireAPI client stack, and is invisible to client applications, though applications can switch Direct Mode on and off as needed. The AMQP wiring is unchanged: we still create private queues on the server, and bind these queues to exchanges.

When a private queue has been leased to a DMP connection, the server bypasses it: the exchange then sends messages directly to that DMP connection, rather than queuing them on the private queue. Leased private queues never holds any messages, so the configurable queue limits have no effect. However, WireAPI provides configurable queue limits for client-side queues.

DMP connections are broken when the main AMQP connection is closed, and leased private queues are automatically deleted as they would be in default AMQP messaging.

By default, and for compatibility with older client stacks, Direct Mode is not enabled. Applications can choose Direct Mode explicitly by setting a connection property, or via their configuration. DMP also allows, and OpenAMQ implements, a server property that lets administrators enable Direct Mode from the server configuration, with zero changes to client configurations or applications.

Direct Mode performance is significantly better than default AMQP messaging, boosting message throughput by around six times, and reducing best and worst-case latency.

These improvements come from five specific aspects of DMP:

1. The message flow through the server is faster, which cuts latency.
2. The DMP message transfer protocol is simpler, so can be processed with fewer steps.
3. The DMP framing lets OpenAMQ read and write messages with fewer network stack transitions.
4. OpenAMQ is able to batch DMP messages both at write and read.
5. DMP messages are smaller, so cross the network faster.

Not all AMQP products will have the same gains. Some of these gains can be achieved without DMP. For example, bypassing private queues, and batching, are both doable in default AMQP messaging, but with less ease.

Direct Mode batching is opportunistic. Both client and server stacks use dedicated threads to do DMP writing. These threads are fed by internal event queues. When a thread writes a DMP message, it will collect as many waiting messages off the event queue as are available and will fit in the configured buffer. Since network i/o is usually slower than internal eventing, and since most of the cost of message transfer is in the network stack, most DMP messages will be written in batches of 2-5, giving a free and proportionate performance boost.

When OpenAMQ reads a message in Direct Mode it will read as much data as the network stack can provide. This typically means that OpenAMQ will read 2-3 messages at once, also boosting performance. The size of the output and input buffers is configurable, but we find that above a certain size the benefits tail off.

Interoperability

Applications that wish to use DMP require a compatible server and a compatible client library. Currently, only OpenAMQ's server and WireAPI library implement Direct Mode, and with other clients the OpenAMQ server will work in normal AMQP mode.

Users can freely mix applications using Direct Mode and applications using normal AMQP message transfer. In any case, shared queues do not work with DMP (a DMP feed must correspond to an exclusive queue). Messages are fully interoperable between these two protocols - i.e. a publisher could use Direct Mode, while a client does not, or vice-versa.

Conclusions

DMP is a short and simple specification that addresses some basic shortcomings in AMQP, and has already delivered compelling value to users of the OpenAMQ product. The main advantages are less work to manage servers, and improved capacity of clients, and servers. The main costs are more connections to servers, which may create problems for some architectures. Support for DMP is today limited to iMatix's products but by documenting the specification as a free and open standard, and by clearly explaining its implementation, we hope that other AMQP implementors will also adopt DMP.

Moving beyond the current generation of AMQP, DMP demonstrates how a simple, focussed message transfer protocol can deliver performance and stability, and we hope that AMQP/1.0 will include mechanisms that are similar to, or at least inspired by, DMP.

Comments

Edit | Files | Tags | Source | Print

Author

Pieter Hintjens <moc.xitami|hp#moc.xitami|hp>

Installing and using OpenAMQ

Introduction to OpenAMQ: This document is an introduction to the concept of business messaging in general, and to OpenAMQ in particular. It is intended for new OpenAMQ users who wish to understand the problems that OpenAMQ solves, and how OpenAMQ can be useful in software applications.

Basic use of OpenAMQ: This document explains how to get OpenAMQ running on your system. It explains how to download the software, how to unpack and build it (if you are using a source package), and how to run basic tests on the resulting software.

Advanced use of OpenAMQ: This guide is for people who need to configure and manage OpenAMQ servers. We explain how to configure and tune an OpenAMQ server, covering these topics: logging, monitoring, high-availability failover, and joining OpenAMQ servers into wide-area federations.

Writing applications

Programming WireAPI: This is the main guide for developers who wish to use OpenAMQ in their applications. We describe WireAPI, the C/C++ API that OpenAMQ provides for accessing AMQP. Expert WireAPI users may wish to read the iMatix iCL guide, but this document is otherwise self-complete.

Programming PAL: This guide is for OpenAMQ developers who need a quick way to write test cases and simple scenarios. We explain the PAL language, an XML scripting tool that gives you a fast way to construct AMQP applications to test routing models, performance and stability tests, and other test cases.

Programming the Console: This document explains how to write applications that automate management of OpenAMQ servers via console automation. The OpenAMQ console automation architecture offers developers different ways of accessing the functionality of the console API and integrating it with their own preferred tools and management facilities.

Technical library

Developer's Guide to ASL: This is a technical guide for protocol developers who wish to use the iMatix ASL framework for the development of connected client-server protocols. ASL is a generic framework that uses a protocol modeling language to construct the whole infrastructure for a given protocol. ASL was built primarily to support AMQP.

Developer's Guide to iCL: This is a technical guide for developers who wish to understand how the iMatix iCL framework works. iCL is a class-oriented modelling language for C applications and is one of the basic frameworks used in iMatix applications such as OpenAMQ.

Developer's Guide to MOP: This is a technical guide for developers who wish to understand how the iMatix code generation frameworks are constructed. We explain the principles of model oriented programming, and the basics of code generation using the iMatix GSL language. This provides essential basic knowledge for anyone intending to modify the OpenAMQ software.

Developer's Guide to SMT: This is a technical guide for developers who wish to understand how the iMatix SMT framework works. To use this guide the reader should be familiar with the iMatix iCL framework, and the iMatix Model Oriented Programming principles.

RFCs

The CML Request for Comments: We describe a generic technique for managing AMQP servers from a remote client application. This technique consists of a standard transport mechanism built over AMQP, and a standard XML language used to exchange information between a management component built-in to the server, and a management application. This is a request for comments, it is not a standard.