Message Broker for Microservices
Microservices uses asynchronous communication commonly through message broker. Each Service handles and collaborates with other services in order to perform on a given request coming from the Client via inter-communication among them. A broker ensures communication between different microservices is reliable and stable, that the messages are managed and monitored within the system and that messages don’t get lost. There are a few message brokers you can choose from, varying in scale and data capabilities. This post will compare the three most popular brokers:
What is Microservices Communication?
There are two common ways microservices communicate with each other: Synchronous and Asynchronous.
In Synchronous communication, The requester/consumer/caller waits for a response before sending the next message - preferably operates as REST over HTTP. In Asynchronous communication, the messages are sent without waiting for a response. This is suitable for a distributed system and usually requires a message broker to manage the messages.
Styles of Communication
Notifications — a sender sends a message to a recipient but does not expect any/nor even get any reply.
Request/response — a service sends a request message to a recipient and expects to receive a reply of the message promptly.
Request/asynchronous response — a service sends a request message to a recipient and expects to receive a reply of the message eventually.
Publish/subscribe — a service publishes a message to zero or more recipients
Publish/asynchronous response — a service publishes a request to one or recipients, some of whom send back a reply but some don’t.
Choosing the right type of communication involves on various parameters :
Structure of microservice,
infrastructure that in place,
purpose of communication
Advantages of Asynchronous Communication
They are non-blocking by definition.
It supports scale better than synchronous operations
Provides recovery mechanism on microservice event crashes
Don't really know about services involved each other
Better decoupling of services
Increases the capability of central discovery, monitoring, load-balancing, and even policy enforcer
Enhances flexibility and scalability
Asynchronous is usually managed through message broker. Choosing the right the message broker should have below considerations
Scaling need of the Broker - Number of messages sent per second
Data Persistence - Message recoverability on failure
Consumer Capability - Managing the toplogies of the consumer structure. i.e.., One - to - one and/or one-to-many consumers
RabbitMQ expects some performance issues when in the persistent mode.
Where to use: complex routing and support when the required rate is not high.
Where to use : Large amounts of Data. It is a high throughput distributed queue and ideal to use for one- to -many use cases where persistene is requires
Redis is differnet from the other message brokers. It is an in-memory data store that can be used as either a high-performance key-value store or a message broker. It has no persistency and it just dumps the data memory into a Disk/Db. It is suited well for real -time processing.
Where to use: Largely suited for short-lived messages. It is the perfect candidate for short retention messages where persistence isn’t so important and you can tolerate some loss
Any message queue that allows publishing messages decoupled from consuming them is effectively acting as a storage system for the in-flight messages. What is different about Kafka is that it is a very good storage system. It provides data persistency and stores streams of records that render it capable of exchanging quality