There is a very long history of messaging in software systems. We can easily see an event-driven solution and events in the context of messaging systems and messages. However, there are different characteristics that are worth considering:
Messaging: Messages transport a payload and messages are persisted until consumed. Message consumers are typically directly targeted and related to the producer who cares that the message has been delivered and processed.
Events: Events are persisted as a replayable stream history. Event consumers are not tied to the producer. An event is a record of something that has happened and so can’t be changed. (You can’t change history.)
Request messages representing a task to be completed on an application or system.
Messaging and Event Streaming Use Cases
Within an organization, there are many different types of messages flowing, for instance:
Response messages that communicate the completion of a request
Logging that informs us of the processing of a request
Monitoring that informs us of the state of a system
Many additional examples can be provided and they will fall into two types. Messages that are focused on:
Operations: Messages that represent current or future processing. For instance, a retailer may send an order request to their payment provider for processing. This message is targeted to the payment processing system with an explicit intent to be actioned.
Events: Messages that represent the state of the system. For instance, a retailer may emit an event for each order completion. These events represent facts and are emitted with no knowledge of how they will be used.
Considering these two types, what use cases does a solution require to support these types of messaging?
Although we could list hundreds of use cases, we will focus on three key aspects, to focus on the differences.
Transient Data – data is only stored until a consumer has processed the message, or it expires. The data does not need to be persisted longer than required, and it is actually beneficial from a system resource point of view that it does not.
Request / Reply– operations can be fire and forget, however often a request/reply interaction is desirable. Therefore, any solution should inherently support this pattern of interaction.
Targeted Reliable Delivery – when a message is sent, it is deliberately targeted to the entity that will process the request or receive the response. This targeting can use logical addressing and, in many instances, it is preferable. Messages should also be delivered using a suitable level of reliability. Different messages will have different levels, but often once-and-once-only, assured and transactional behavior is critical.
Stream history: when retrieving events, often consumers are interested in historical events, not only the most recent. There are many instances when this is valuable, such as retrieving the historical trends of the availability of a system. Therefore, every event needs to be appended and made available to consumers, and depending on the configuration, a certain number or volume of events will be stored prior to removal.
Scalable Consumption – multiple events are likely to be emitted for a single operation as it passes through the network. Therefore, the scaling requirements to handle events are likely to be orders of magnitude larger than those required to support operational messages. The scalability requirement is not only limited to the producers creating messages, but also to consumption. A single event should be available to be consumed by many consumers with limited impact as the number of consumers is increased.
Immutable Data– when an event is emitted, it is a statement of fact and can be considered immutable data. This enables consumers to rely on the assumption of consistent replay and reduces the impact in replicating the data from a consistent point of view.
Immediately you can see that the capabilities required to handle event and operation message types are significantly different. Therefore, it should not be surprising that different products exist for handling these categories of data. For instance, event streaming technology specializes in handling events, while brokered messaging focuses on reliably handling operation messages. This is not saying that either technology is limited to a particular use case, however, it is useful to understand the use cases each technology has been specifically built to support in order to match the right solution to your requirements.
I hope you got something out of this blog. If you found this post illuminating at all, consider sharing this post and following for more upcoming content.