Message Queues with Grails 3 and RabbitMQ

Learn how to use message queues with Grails 3 and RabbitMQ-Native plugin

Authors: Ben Rhine,Sergio del Amo

Grails Version: 3.3.2

1 Training

Grails Training - Developed and delivered by the folks who created and actively maintain the Grails framework!.

2 Getting Started

In this guide we will show you how to setup and use RabbitMQ with a Grails 3 application. We will be using the rabbitmq-native plugin

2.1 What you will need

To complete this guide, you will need the following:

Some time on your hands
A decent text editor or IDE
JDK 1.7 or greater installed with JAVA_HOME configured appropriately

2.2 How to complete the guide

To get started do the following:

Download and unzip the source

Clone the Git repository:
git clone https://github.com/grails-guides/grails-rabbitmq.git

The Grails guides repositories contain three folders:

initial Initial project. Often a simple Grails app with some additional code to give you a head-start.
complete
complete-analytics

In this guide you are going to create two Grails Applications. Both complete and complete-analytics apps are a completed example. It is the result of working through the steps presented by the guide and applying those changes to the initial folder.

To complete the guide, go to the initial folder

cd into grails-guides/grails-rabbitmq/initial

and follow the instructions in the next sections.

You can go right to the completed example if you cd into grails-guides/grails-rabbitmq/complete and grails-guides/grails-rabbitmq/complete-analytics.

3 Application Overview

In this guide, we setup a message queue to work across two different applications. In this guide, we have an app which lists books and book’s detail. We want to keep track of the number of times each book is viewed. We add a separate analytics app that keeps track of the number of times each one is viewed.

4 Message Queues - RabbitMQ

The construct of message queues provides for asynchronous communications, meaning that the sender and receiver do not need to interact with the message queue concurrently. When a message is sent it will be added to the queue and stored until the recipient retrieves them. Message queues can be used internally in an app or externally with multiple applications. A number of different implementations of message queues exist but one of the more popular and the one we will use for this guide is RabbitMQ.

4.1 Setup RabbitMQ service

To get RabbitMQ up and running on your system there are multiple different ways to get setup and going. You may choose to follow the directions from RabbitMQ themselves located here, but we would recommend following the way we set it up for parity.

To have a similar to real world setup we have chosen to setup our RabbitMQ using Docker as its quick, simple, and easy to tear down and setup again if you run into any issues. Docker is a container engine which allows for quick and simple installs of many different frameworks and products and helps avoid differences in local systems to provide a guaranteed clean instance of the product you are tying to run. We will assume you already have Docker installed on your system moving forward but if you do not you can install it from here.

Make sure Docker is running on your local system (You should see a little whale)

Once you know Docker is up and running from your terminal run the following command.

$ docker run -d --hostname my-rabbit --name some-rabbit -p 15672:15672 -p 4369:4369 -p 5671:5671 -p 5672:5672 -p 15671:15671 -p 25672:25672 rabbitmq:3-management

The above command does a lot for you, it downloads and runs a dockerized instance of RabbitMQ, sets the hostname, gives the instance a name, configures the ports you will need to successfully access RabbitMQ, and includes the management interface so we can have more insight to how the message queue works. Once the command completes you will see output similar to the following.

In addition to Docker you may like a more visual interface to see your RabbitMQ container in action. If thats the case go ahead and download "Kitematic" which is a GUI interface for your underlying Docker service. Docker will even help you get Kitematic from its menu.

If you installed Kitematic you should now see your RabbitMQ instance running on the lefthand side.

As mentioned above we included the RabbitMQ admin interface so we can see some details about our message queues. You can access the admin interface located at localhost:15672.

To login use the username / pass of guest / guest. After a successful login you will see the following page

For more information on this specific docker container please view the documentation here.

5 Books Application

We have already added the necessary assets (book cover images) to the initial/grails-app/assets/images folder for you.

Moving forward we need to add additional functionality to our app since we will no longer be using just a single application. To start we first need to add a Book domain.

grails-app/domain/demo/Book.groovy

package demo

import grails.compiler.GrailsCompileStatic

@GrailsCompileStatic
class Book {
    String image
    String title
    String author
    String about
    String href
    static mapping = {
        image nullable: false
        title nullable: false
        author nullable: false
        about nullable: false
        href nullable: false
        about type: 'text'
    }
}

We will be using a custom findAll to return our list of books and for that we will need to use an additional BookImage object. First create your BookImage in the src/main/groovy directory

src/main/groovy/demo/BookImage.groovy

package demo

import groovy.transform.CompileStatic

@CompileStatic
class BookImage {
    Long id
    String image
}

Create default CRUD actions for Book leveraging GORM data services.

grails-app/services/demo/BookDataService.groovy

package demo

import grails.gorm.services.Service
import grails.gorm.transactions.ReadOnly
import groovy.transform.CompileDynamic
import groovy.transform.CompileStatic
import org.grails.datastore.mapping.query.api.BuildableCriteria
import org.hibernate.transform.Transformers

interface IBookDataService {
    Book save(String title, String author, String about, String href, String image)
    Number count()
    Book findById(Long id)
}

@Service(Book)
abstract class BookDataService implements IBookDataService {

    @CompileDynamic
    @ReadOnly
    List<BookImage> findAll() {
        BuildableCriteria c = Book.createCriteria()
        c.list {
            resultTransformer(Transformers.aliasToBean(BookImage))
            projections {
                property('id', 'id')
                property('image', 'image')
            }
        }
    }
}

Next connect our new controller up to the services that we just created. Our index will leverage our custom findAll to return a complete list of books while our show will make use of the data services findById.

grails-app/controllers/demo/BookController.groovy

package demo

import groovy.transform.CompileStatic

@CompileStatic
class BookController {

    static allowedMethods = [index: 'GET', show: 'GET']

    BookDataService bookDataService

    def index() {
        [bookList: bookDataService.findAll()]
    }

    def show(Long id) {
        [bookInstance: bookDataService.findById(id)]
    }

}

Then we need to actually create the book data with our Bootstrap.groovy

grails-app/init/demo/Bootstrap.groovy

package demo

import groovy.transform.CompileStatic

@CompileStatic
class BootStrap {

    public final static List< Map<String, String> > GRAILS_BOOKS = [
            [
                    title : 'Grails 3 - Step by Step',
                    author: 'Cristian Olaru',
                    href: 'https://grailsthreebook.com/',
                    about : 'Learn how a complete greenfield application can be implemented quickly and efficiently with Grails 3 using profiles and plugins. Use the sample application that accompanies the book as an example.',
                    image: 'grails_3_step_by_step.png',
            ],
            [
                    title : 'Practical Grails 3',
                    author: ' Eric Helgeson',
                    href  : 'https://www.grails3book.com/',
                    about : 'Learn the fundamental concepts behind building Grails applications with the first book dedicated to Grails 3. Real, up-to-date code examples are provided, so you can easily follow along.',
                    image: 'pratical-grails-3-book-cover.png',
            ],
            [
                    title : 'Falando de Grails',
                    author: 'Henrique Lobo Weissmann',
                    href  : 'http://www.casadocodigo.com.br/products/livro-grails',
                    about : 'This is the best reference on Grails 2.5 and 3.0 written in Portuguese. It&#39;s a great guide to the framework, dealing with details that many users tend to ignore.',
                    image: 'grails_weissmann.png',
            ],
            [
                    title : 'Grails Goodness Notebook',
                    author: 'Hubert A. Klein Ikkink',
                    href  : 'https://leanpub.com/grails-goodness-notebook',
                    about : 'Experience the Grails framework through code snippets. Discover (hidden) Grails features through code examples and short articles. The articles and code will get you started quickly and provide deeper insight into Grails.',
                    image: 'grailsgood.png',
            ],
            [
                    title : 'The Definitive Guide to Grails 2',
                    author: 'Jeff Scott Brown and Graeme Rocher',
                    href  : 'http://www.apress.com/9781430243779',
                    about : 'As the title states, this is the definitive reference on the Grails framework, authored by core members of the development team.',
                    image: 'grocher_jbrown_cover.jpg',
            ],
            [
                    title : 'Grails in Action',
                    author: 'Glen Smith and Peter Ledbrook',
                    href  : 'http://www.manning.com/gsmith2/',
                    about : 'The second edition of Grails in Action is a comprehensive introduction to Grails 2 focused on helping you become super-productive fast.',
                    image: 'gsmith2_cover150.jpg',
            ],
            [
                    title : 'Grails 2: A Quick-Start Guide',
                    author: 'Dave Klein and Ben Klein',
                    href  : 'http://www.amazon.com/gp/product/1937785777?tag=misa09-20',
                    about : 'This revised and updated edition shows you how to use Grails by iteratively building a unique, working application.',
                    image : 'bklein_cover.jpg',
            ],
            [
                    title : 'Programming Grails',
                    author: 'Burt Beckwith',
                    href  : 'http://shop.oreilly.com/product/0636920024750.do',
                    about : 'Dig deeper into Grails architecture and discover how this application framework works its magic.',
                    image: 'bbeckwith_cover.gif'
            ]
    ] as List< Map<String, String> >

    public final static List< Map<String, String> > GROOVY_BOOKS = [
            [
                    title: 'Making Java Groovy',
                    author: 'Ken Kousen',
                    href: 'http://www.manning.com/kousen/',
                    about: 'Make Java development easier by adding Groovy. Each chapter focuses on a task Java developers do, like building, testing, or working with databases or restful web services, and shows ways Groovy can make those tasks easier.',
                    image: 'Kousen-MJG.png',
            ],
            [
                    title: 'Groovy in Action, 2nd Edition',
                    author: 'Dierk König, Guillaume Laforge, Paul King, Cédric Champeau, Hamlet D\'Arcy, Erik Pragt, and Jon Skeet',
                    href: 'http://www.manning.com/koenig2/',
                    about: 'This is the undisputed, definitive reference on the Groovy language, authored by core members of the development team.',
                    image: 'regina.png',
            ],
            [
                    title: 'Groovy for Domain-Specific Languages',
                    author: 'Fergal Dearle',
                    href: 'http://www.packtpub.com/groovy-for-domain-specific-languages-dsl/book',
                    about: 'Learn how Groovy can help Java developers easily build domain-specific languages into their applications.',
                    image: 'gdsl.jpg',
            ],
            [
                    title: 'Groovy 2 Cookbook',
                    author: 'Andrey Adamovitch, Luciano Fiandeso',
                    href: 'http://www.packtpub.com/groovy-2-cookbook/book',
                    about: 'This book contains more than 90 recipes that use the powerful features of Groovy 2 to develop solutions to everyday programming challenges.',
                    image: 'g2cook.jpg',
            ],
            [
                    title: 'Programming Groovy 2',
                    author: 'Venkat Subramaniam',
                    href: 'http://pragprog.com/book/vslg2/programming-groovy-2',
                    about: 'This book helps experienced Java developers learn to use Groovy 2, from the basics of the language to its latest advances.',
                    image: 'vslg2.jpg'
            ],
    ] as List< Map<String, String> >

    BookDataService bookDataService

    def init = { servletContext ->
        for (Map<String, String> bookInfo : (GRAILS_BOOKS + GROOVY_BOOKS)) {
            bookDataService.save(bookInfo.title, bookInfo.author, bookInfo.about, bookInfo.href, bookInfo.image)
        }
    }

    def destroy = {
    }
}

Lastly we update our url mapping so that our default url will display our list of books

grails-app/controllers/demo/UrlMappings.groovy

package demo

class UrlMappings {

    static mappings = {
        "/$controller/$action?/$id?(.$format)?"{
            constraints {
                // apply constraints here
            }
        }

        "/"(controller: "book") (1)
        "500"(view:'/error')
        "404"(view:'/notFound')
    }
}

1	Updated default URL

Run the app

$ ./gradlew bootRun

5.1 RabbitMQ publisher

Add RabbitMQ Native plugin to the initial app.

build.gradle

compile "org.grails.plugins:rabbitmq-native:3.4.4"

Once we have added the dependency we can go ahead and add the necessary config to connect to RabbitMQ.

grails-app/conf/application.yml

rabbitmq:
    connections:
      - name: main (1)
        host: localhost (2)
        port: 5672
        username: guest  (3)
        password: guest  (4)
    queues:
      - name: bookQueue  (5)

1	All connections require a name but you can name it whatever you want
2	Where RabbitMQ is located. In this case on our local system but this can also be a web url or ip address
3	Default username `guest`
4	Default password `guest`
5	Creation of a `bookqueue`

When a book’s details is viewed we publish a message to RabbitMQ. In order to do that we create an interceptor.

Create a new interceptor in the grails-app/controllers/demo called BookShowInterceptor.groovy.

grails-app/controllers/demo/BookShowInterceptor.groovy

package demo

import com.budjb.rabbitmq.publisher.RabbitMessagePublisher
import groovy.transform.CompileStatic

@CompileStatic
class BookShowInterceptor {
    RabbitMessagePublisher rabbitMessagePublisher  (1)

    BookShowInterceptor() { (2)
        match(controller:"book", action:"show")
    }

    boolean after() { (3)
        final Book book = (Book) model.bookInstance (4)

        rabbitMessagePublisher.send { (5)
            routingKey = "bookQueue"
            body = [id: book.id, title: book.title]
        }
        true
    }
}

1	Inject the rabbitMessagePublisher
2	Intercept when the book controller show method is called
3	`after` method executes after the action but prior to view rendering. It returns true to continue processing to the view.
4	Model is available in the `after` method to retrieve data about the current object
5	Send specified message using a specific queue. When sending a message using send no return message is expected.

6 Building Analytics app

Create a new Grails 3 application for this additional app. For example by using Grails Application Forge or the command line:

$ grails create-app initial-analytics --profile=rest-api

For the multi app part of this guide we will need to be able to run both apps simultaneously. To avoid a running port conflict update your initial-analytics app application.yml to include the following.

grails-app/conf/application.yml

server:
  port: 8090

Create a Domain class BookPageView which will keep track of how many times a book has been viewed.

grails-app/domain/demo/BookPageView.groovy

package demo

import grails.compiler.GrailsCompileStatic
import grails.rest.Resource

@Resource
@GrailsCompileStatic
class BookPageView {
    Long bookId
    String bookName
    Integer views

    static constraints = {
        bookId nullable: false, unique: true
        bookName nullable: false
        views nullable: false, min: 0
    }
}

As before create default actions for our BookPageView leveraging data services.

grails-app/services/demo/BookPageViewDataService.groovy

package demo

import grails.gorm.services.Query
import grails.gorm.services.Service
import grails.gorm.transactions.Transactional
import groovy.transform.CompileStatic

@CompileStatic
interface IBookPageViewDataService {
    void delete(Serializable id)

    BookPageView save(Long bookId, String bookName, Integer views)

    BookPageView findByBookId(Long bookId)

    @Query("select $b.views from ${BookPageView b} where $b.bookId = $bookIdParam") (1)
    List<Integer> findViewsByBookId(Long bookIdParam)

    @Query("update ${BookPageView b} set ${b.views} = ${b.views} + 1 where $b.bookId = $bookIdParam") (2)
    Number updateViews(Long bookIdParam)
}

@Service(BookPageView)
abstract class BookPageViewDataService implements IBookPageViewDataService {

    @Transactional
    void increment(Long bookId, String bookName) {
        List<Integer> views = findViewsByBookId(bookId)
        if (!views) {
            save(bookId, bookName, 1)
        } else {
            updateViews(bookId)
        }
    }
}

1	Use JPA-QL query to perform a projection
2	Implement update operations using JPA-QL

Add an integration test:

complete-analytics/src/integration-test/groovy/demo/BookPageViewDataServiceSpec.groovy

package demo

import grails.testing.mixin.integration.Integration
import spock.lang.IgnoreIf
import spock.lang.Specification

@IgnoreIf( { System.getenv('TRAVIS') as boolean } )
@Integration
class BookPageViewDataServiceSpec extends Specification {

    BookPageViewDataService bookPageViewDataService

    def "test increments"() {
        expect:
        !bookPageViewDataService.findByBookId(2)

        when:
        bookPageViewDataService.increment(2, 'Practical Grails 3')
        BookPageView bookPageView = bookPageViewDataService.findByBookId(2)

        then:
        bookPageView
        bookPageViewDataService.findByBookId(2).views == 1

        when:
        bookPageViewDataService.increment(2, 'Practical Grails 3')
        bookPageView = bookPageViewDataService.findByBookId(2)

        then:
        bookPageView
        bookPageViewDataService.findByBookId(2).views == 2

        cleanup:
        bookPageViewDataService.delete(bookPageView.id)
    }
}

6.1 RabbitMQ consumer

We need to create a new consumer, we will create the consumer in the initial-analytics application. First navigate to the analytics application

$ cd ~/yourProjectLocation/initial-analytics

Then as before create a new consumer

$ ./grailsw create-consumer demo.BookPageViewConsumer

Edit the generated consumer to look as follows

grails-app/rabbit-consumers/demo/BookPageViewConsumer.groovy

package demo

import com.budjb.rabbitmq.consumer.MessageContext
import groovy.transform.CompileStatic
import groovy.util.logging.Slf4j

@Slf4j
@CompileStatic
class BookPageViewConsumer {

    BookPageViewDataService bookPageViewDataService (1)

    static rabbitConfig = [ (2)
            queue: "bookQueue"
    ]

    /**
     * Handle an incoming RabbitMQ message.
     *
     * @param body    The converted body of the incoming message.
     * @param context Properties of the incoming message.
     * @return
     */
    def handleMessage(Map body, MessageContext messageContext) {
        log.debug '{}', body.toString()
        bookPageViewDataService.increment((Long) body.id, (String) body.title) (3)
    }
}

1	Add our page view service
2	Tell our consumer which queue to connect to
3	Tell our consumer what to do when it receives a message. Call our custom increment action.

7 Running the apps

To see our two apps communicate asynchronously using RabbitMQ we have to run both apps simultaneously. To do that open two separate terminals. In terminal one …

$ cd ~/yourProjectLocation/initial

then

$ ./gradlew bootRun

Next in terminal two …

$ cd ~/yourProjectLocation/initial-analytics

then

$ ./gradlew bootRun

Now with both apps running open two browser tabs. In tab one navigate to http://localhost:8080 and you should see a list of books available.

then in your second tab navigate to http://localhost:8090/BookPageView and you should see an empty array.

Select a book to click and after clicking you should see the book description

And if you go refresh the analytics page you will now see that book 1 has been viewed 1 time.

Try this on several books, and go back and revisit a previous book and then refresh your analytics page to see how many views each book has.

Additionally if you view the terminal where you are running the analytics app you can see the messages it received.

8 Rabbit Management

As mentioned earlier in section 3.1 [setupRabbit] we have access to the rabbit management console located at http://localhost:15672. Go ahead and navigate to the console and login. Once logged in select Queues from the menu.

from here you can see the two queues which we have created with our applications. Now select the `Overview tab and return to the management home page. When looking at the home page note that there are 0 messages in the queue.

At this point stop your analytics app from the terminal with Ctrl-C and then go back to your still running app and go click on and view a few more books. Once you have viewed a few more books, return to your admin home and you will see that new messages have been generated and are waiting in the queue.

Go ahead and restart your analytics app now from your terminal

$ ./gradlew bootRun

Once your analytics app has restarted you will see it process the waiting messages and when you return to the rabbit admin you will see the message queue has returned to 0.

9 Next Steps

To further your understanding read through the plugin documentation found here. Additionally feel free to read the RabbitMQ documentation here, and take a look at there many examples available in different languages here.

10 Do you need help with Grails?

Object Computing, Inc. (OCI) sponsored the creation of this Guide. A variety of consulting and support services are available.

OCI is Home to Grails

Meet the Team