Problem
How to reliably/atomically update the database and send messages/events?
Forces
- 2PC is not an option
- If the database transaction commits messages must be sent. Conversely, if the database rolls back, the messages must not be sent
- Messages must be sent to the message broker in the order they were sent by the service. This ordering must be preserved across multiple service instances that update the same aggregate.
Solution
A service that uses a relational database inserts messages/events into
an outbox table
(e.g. MESSAGE) as part of the local transaction.
A service that uses a NoSQL database appends the messages/events to the attribute of the record (e.g. document or item) being updated.
A separate Message Relay process publishes the events inserted into the database to a message broker.
Result context
This pattern has the following benefits:
- 2PC is not used
- Messages are guaranteed to be sent if and only if the database transaction commits
- Messages are sent to the message broker in the order they were sent by the application
- Potentially error prone since the developer might forget to publish the message/event after updating the database.
- The Message Relay might publish a message more than once. It might, for example, crash after publishing a message but before recording the fact that it has done so. When it restarts, it will then publish the message again. As a result, a message consumer must be idempotent, perhaps by tracking the IDs of the messages that it has already processed. Fortunately, since Message Consumers usually need to be idempotent (because a message broker can deliver messages more than once) this is typically not a problem.
