Never lose an event: Outbox Pattern with DynamoDb + Eventbridge Pipes
[Summary Generated with AI] This article addresses the issue of data inconsistency in distributed, event-driven systems due to dual writes, where a database update and event publication might not succeed simultaneously. It presents a fictional car workshop scenario to highlight this problem and introduces the Outbox Pattern as a solution. By using AWS services like DynamoDB Transactions and EventBridge Pipes, the Outbox Pattern ensures reliable messaging and data consistency. The system's resilience and maintainability are improved by atomically handling database and outbox writes and reliably delivering events using serverless components. For complete code samples, refer to the linked GitHub repository.
Introduction
When dealing with distributed systems, failures may happen in the communication gaps between systems. One common issue in event-driven systems is the “dual write”: update a database and then publish an event. A disruption (timeouts, retries, crashes, etc) in a part of the system will make operation results diverge, resulting in incorrect state and issues with data consistency and integrity
This post showcases a fictional use case for a Car Workshop which may run into this problem by using a naive approach. It shows how to introduce more confidence in the system by implementing the “Outbox Pattern” architecture pattern and shows how to implement such pattern using AWS Services such as DynamoDb Transactions and EventBridge pipes
For full code implementation, please Check the GitHub Repository
Use Case
As a fictional example, a car workshop does repair orders for cars that come to the building. Once a repair is finished, the system must update a database with the status as COMPLETED and trigger an asynchronous process to send an Invoice to the customer and a notification to another section of the building
Component diagram of the car workshop use case “Repair Completed”. Own Creation
While the above architecture for the use case may work most times, we can see the system relies on a fragile two-part writes: One to DynamoDb to update the repair status, and the other write to publish the event to EventBridge bus. The following is an example code snippet inside the Lambda Function that showcases this:
await ddb.update({ pk: `REPAIR#${id}`, sk: 'META', set: { status: 'COMPLETED' }});
await eventBridge.putEvents({ detail: { eventType: 'RepairCompleted', repairId: id }});
return 200;
Some potential problems from this code:
Db updates successfully but the event bridge “putEvents” times out.
- This results in database with the Repair as completed but no event was sent for invoicing and notification
DB updates successfully and event bridge “putEvents” also works, but Lambda crashes before responding.
- This will trigger a lambda retry with the same input, which will send a duplicated event for the invoicing and notification
Outbox Pattern
In System Design, Reliable messaging is an essential part. Specially in distributed systems where multiple services are connecting and integrating with each other in a choreography-style.
The Outbox Pattern aims to ensure a reliable messaging between different services and components. It is specially important in scenarios where data consistency and data integrity are critical. It does so by defining a table or entity as an “Outbox”, which will be separated from the main data entity. It temporarily stores messaged that need to be sent to other systems.
By leveraging databases ACID Transactions, we can join together two or more writes in a single atomic operation, ensuring that the data is consistent and having the “Outbox” entity be in charge of the delivery of the events
By using this pattern, we are effectively updating the needed business entities and setting the message to be sent in one go. A separate process can then send the messages to the message broker.
Outbox Pattern Diagram. Taken from https://microservices.io/patterns/data/transactional-outbox.html
Serverless Outbox Pattern
By leveraging Serverless services in AWS, we can propose a better architecture that gives more reliability on event delivery and data integrity. Using DynamoDb Transactions, DynamoDb Streams and EventBridge Pipes we can give more confidence to the system:
Component diagram of the car workshop use case “Repair Completed” implementing the “Outbox Pattern”. Own Creation
As we’re now using the “Outbox” pattern, the lambda updates the status AND writes a new entity called “Outbox” in one go, using a DynamoDb Transaction. This ensures that both writes to the database are encapsulated atomically, resulting in either both writes succeeding or none. The following is a code snippet of a sample “DynamoDb Transaction” that implements the Outbox Pattern:
const params = {
TransactItems: [
{
Update: {
TableName: table,
Key: {
pk: marshall(`REPAIR#${repairId}`),
sk: marshall('META'),
},
ConditionExpression: 'attribute_not_exists(#status) OR #status <> :done',
UpdateExpression: 'SET #status = :done, completeAt = :now',
ExpressionAttributeNames: { '#status': 'status' },
ExpressionAttributeValues: marshall({ ':done': 'COMPLETED', ':now': now }),
}
},
{
Put: {
TableName: table,
Item: marshall({
pk: `OUTBOX#${eventId}`,
sk: `REPAIR#${repairId}`,
entityType: 'OUTBOX',
eventType: 'RepairCompleted',
occurredAt: now,
eventId,
repairId,
}),
ConditionExpression: 'attribute_not_exists(pk)'
}
}
]
};
const command = new DynamoDb.TransactWriteItemsCommand(params);
const response = await dynamoDbClient.send(command);
By setting DynamoDb Streams, we can setup a stream on the dynamodb table that triggers on specific database writes. Using EventBridge Pipes, we can connect the DynamoDb events with an EventBridge Bus. EventBridge pipes allows us to set up filtering, which we can use to filter by “entity type” being “OUTBOX”
EventBridge Pipe showing the event filtering by “OUTBOX” events. Own Creation
The event delivered to the EventBridge Bus can have the following structure, which is filtered and mapped using EventBridge Pipes:
{
"detail-type": "OutboxEvent",
"source": "carworkshop.outbox",
"detail": {
"eventId": "c8f2d7e6-1e2f-4b7b-8c41-0b2a2f1c9f92",
"eventType": "RepairCompleted",
"repairId": "R-123",
"occurredAt": "2025-11-12T14:00:00Z",
"source": "fleetfix.outbox"
}
}
For a complete code sample implementation, please Check the GitHub Repository
Conclusion
One of the challenges when dealing with distributed systems is ensuring data consistency and integrity across asynchronous processes. One common issue is the “dual-write” approach where two separate writes to two separate systems may result in data discrepancies, events not sent or duplicated events
By implementing the “Outbox Pattern”, this article shows how to handle event-delivery that is consistent with the business data updates. By leveraging DynamoDb Transactions and EventBridge pipes, the pattern was implemented for a fictional “car workshop” use case where a repair is done and asynchronous process needs to be sent to handle invoicing and notifications
Incorporating the “Outbox pattern” in our architecture allows for systems that more resilient, handle failures gracefully, which ensures a more predictable and maintainable system overall
