ryah ([info]four) wrote,

amq protocol

The concepts in the AMQP specification are beautiful. Certainly abstract enough to handle most situations. But have you looked at the protocol? It's extremely complex. A simpler protocol could express AMQ concepts sufficiently, ensure more support and better interoperability. Worse is better, you know? Someone should try again and limit themselves to a 5 page specification.

Ideas
  • Only use TCP.
  • Use ASCII. Binary protocols are harder.
  • Eliminate multiple exchanges per virtual server. server = exchange.
  • Eliminate multiple types of exchanges. Only have the "topic" exchange (which is the one which allows matching on simple regular expressions).
  • Eliminate message priorities. (Handle this in the application. e.g. append .emergency to the routing key.)
  • Eliminate message meta-data. Mime-types? give me a break. The application can handle that.
  • Eliminate transactions. Just a simple response once the message has been received by the exchange. (Applications can implement transactions themselves on top of the protocol.)
  • Eliminate resuming message uploads. (Applications can split large pieces of data into multiple messages themselves.) Might even have an explicit maximum message size, say, 500kb.
  • Keep the various queue settings (durable, exclusive, auto-delete).
  • Eliminate possibility of durable messages. Queues too? Disk access = death.
  • Do not include timestamps in the protocol. Allow applications to handle that themselves. (Although it would be nice for a consumer to know how many seconds the message has been sitting in queue.)
  • Eliminate complex bindings. Only use pattern matching. One or more bindings per queue - the bindings are deleted when the queue is deleted. Bindings are created with queue creation and afterwards immutable.
  • I'm torn on the issue of access control. AMQP itself doesn't specify much. user/password logins without SSL are arguably more harm than benefit - and it certainly complicates clients. Maybe just leave access control out of the specification and if it is needed use SSL. Systems like memcached have been wildly successful without ACL.
  • Possibly split the server across two ports. One for producers one for consumers. This would simplify SSL settings.


So the protocol might look like this (client commands in red, server responses in black)
  1. Producers only produce messages. Nothing else. They connect and send messages like this:
    MESSAGE irc.ruby_lang 11 hello world\r\n

    The string irc.ruby_lang is the routing key. 11 is the content-length. The server responds with
    MESSAGE_CREATED 1\r\n
    meaning it got the message and it was put into one queue.

  2. Consumers deal with the bulk of the processing. First the they can create queues and bindings
    CREATE_QUEUE irc.ruby_lang\r\n
    CREATE_BINDING irc.ruby_lang irc.ruby*\r\n
    The server would respond with
    QUEUE_CREATED\r\n
    BINDING_CREATED\r\n
    Then they can subscribe to a queue
    SUBSCRIBE irc.ruby_lang\r\n
    And the server will start sending them messages which will look like this
    MESSAGE (queue) (message-id) (message-routing-key) (seconds-in-queue) (content-length) (data)
    For example
    MESSAGE irc.ruby_lang 1234554321 irc.ruby_lang 0 11 hello world\r\n
    After each message received by the consumer it should send an acknowledgment.
    MESSAGE_RECEIVED 1234554321\r\n

    Then there will be the various other commands
    UNSUBSCRIBE irc.ruby_lang\r\n
    DELETE_BINDING irc.ruby_lang irc.ruby*\r\n
    DELETE_QUEUE irc.ruby_lang\r\n
    To which the server responds
    UNSUBSCRIBED irc.ruby_lang\r\n
    BINDING_DELETED\r\n
    QUEUE_DELETED irc.ruby_lang\r\n


  3. Producers, consumers, and servers should all respond to PING with PONG.

  • Post a new comment

    Error

    Comments allowed for friends only

    Anonymous comments are disabled in this journal

    Your reply will be screened

  • 0 comments
Create an Account
Forgot your login or password?
Facebook Twitter More login options
English • Español • Deutsch • Русский…