I recently wrapped a Ruby on Rails project in which we were leveraging RabbitMQ as the message broker. We wanted a quick and easy solution to manage the configuration of the exchanges and queues for deployment. Since the developers on the team had the occasional need to leverage RabbitMQ locally it made sense to combine those solutions into a rake task.
The task leverages the RabbitMQ command line interface (CLI) which gives you the ability to accomplish most anything. The problem is that most of the team were not well versed in RabbitMQ and didn't need to be, so to make it simple I first created a 'rake rabbit:setup' task that would pull down the CLI script and place it in the proper path for execution of the remaining tasks. This only needed to be executed once and they were good to go.
Execution
To configure the system with the CLI is pretty straightforward, you can simply export the rabbit configuration to a JSON file using the 'rabbitmqadmin export rabbit.config' command, make your changes and import the configuration file using 'rabbitmqadmin import rabbit.config'. The problem is that if you change an existing exchange or queue those changes are ignored. To work around that problem, I added that ability to delete exchanges and queues so that a fresh configuration would allow for all configurations to be the latest and greatest. You are probably thinking, what about existing messages, if you delete a queue with messages in it you are going to lose data, what kind of solution is that? Well, relax if you call the delete queues or delete all, which deletes queues and exchanges, it will only remove those queues that are empty and warn you that some queues have messages. So you can then act on those queues with messages and repeat the delete action to ensure all is well. If you are not worried about existing messages and data loss you can easily delete the queues containing messages by running 'rake rabbit:delete:queues_by_force'.
In the long run, I am confident that the task can be improved upon, but it is a great starting point to incorporating a configuration based management of RabbitMQ.
The task leverages the RabbitMQ command line interface (CLI) which gives you the ability to accomplish most anything. The problem is that most of the team were not well versed in RabbitMQ and didn't need to be, so to make it simple I first created a 'rake rabbit:setup' task that would pull down the CLI script and place it in the proper path for execution of the remaining tasks. This only needed to be executed once and they were good to go.
Configuration
To start, I wanted to make the configurations quite simple and familiar to the team so the rabbitmq.yml file was born. The rabbitmq.yml file allows for configuration of exchanges, queues and bindings between them. As you will see in the example gist below, you specify the user and password for RabbitMQ. Exchanges can define all available types, identify an alternate exchange and whether the exchange is internal or not. Queues allow for simple definition of just the name to including the expiration time and dead letter settings. Bindings tell an exchange and queue about one another and their relationship. You can also add a host configuration entry and modify the base command in the rake task if you are interested in configuring a remote RabbitMQ server.
Execution
To configure the system with the CLI is pretty straightforward, you can simply export the rabbit configuration to a JSON file using the 'rabbitmqadmin export rabbit.config' command, make your changes and import the configuration file using 'rabbitmqadmin import rabbit.config'. The problem is that if you change an existing exchange or queue those changes are ignored. To work around that problem, I added that ability to delete exchanges and queues so that a fresh configuration would allow for all configurations to be the latest and greatest. You are probably thinking, what about existing messages, if you delete a queue with messages in it you are going to lose data, what kind of solution is that? Well, relax if you call the delete queues or delete all, which deletes queues and exchanges, it will only remove those queues that are empty and warn you that some queues have messages. So you can then act on those queues with messages and repeat the delete action to ensure all is well. If you are not worried about existing messages and data loss you can easily delete the queues containing messages by running 'rake rabbit:delete:queues_by_force'.
In the long run, I am confident that the task can be improved upon, but it is a great starting point to incorporating a configuration based management of RabbitMQ.
To use this with your Rails project, you will want to download the gist (https://gist.github.com/4233924) and place the "rabbit.rake" file in your lib/tasks directory. The tasks expect a yml file, also part of the gist which should be placed in config/rabbitmq.yml.
Rake Tasks
A quick review of the tasks available and their usage.
rake rabbit:setup# Prepares the current system to be able to run the rabbit rake tasks. It must be run before you can execute any of the other tasks.
rake rabbit:configure# Configures Exchanges, Queues and Bindings based on config/rabbitmq.yml
rake rabbit:delete:all
# Deletes all Exchanges and empty Queues within RabbitMQ
rake rabbit:delete:exchanges
# Deletes all Exchanges within RabbitMQ
rake rabbit:delete:queues
# Deletes all empty Queues within RabbitMQ
rake rabbit:delete:queues_by_force
# Deletes all Queues (and messages) within RabbitMQ
Configuration
The rabbitmq.yml file allows for configuration of exchanges, queues and bindings between them. As you will see below, you specify the user and password for RabbitMQ. Exchanges can define all available types, identify an alternate exchange and whether the exchange is internal or not. Queues allow for simple definition of just the name to including the expiration time and dead letter settings. Bindings tell an exchange and queue about one another and their relationship.
The gist
The gist
No comments:
Post a Comment