Event Sourcing in Distributed Systems: A Practical Guide for Software Developers
Learn how Event Sourcing models business change as an ordered history of events. This tutorial explains event streams, commands, state reconstruction, event stores, domain events, aggregates, .NET implementation, concurrency, snapshots, transactions, projections, read models, failures, eventual consistency, versioning, and schema evolution. It closes by examining real technologies and when Event Sourcing is an appropriate design choice.
Lesson 1: Why Event Sourcing Exists
Event Sourcing becomes much easier to understand when we first look at the problem that led to the idea. Most applications store the latest version of their data, which works well for many situations, but this approach can hide the sequence of changes that produced the current result. In this lesson, we will start with a simple bank account example and see why preserving business history can sometimes be as important as preserving the current state.
The Traditional Way of Storing Data
Imagine that we are building a banking application. A customer deposits $500 into an account and later withdraws $100. In a traditional application, we usually store the current state of the account in a database. The database might contain a row like this:
CREATE TABLE Accounts (
Id INT PRIMARY KEY,
CustomerId INT,
Balance DECIMAL(18,2)
);
INSERT INTO Accounts VALUES (1, 101, 400.00);
The application knows that the customer's current balance is $400, and for many systems, this is exactly what we need. When the customer opens the application, we simply execute a query such as SELECT Balance FROM Accounts WHERE Id = 1. This approach is simple, familiar, and is the foundation of most CRUD applications.
What Happens to the History?
The problem becomes visible when we ask a different question: How did the account reach $400? The database only tells us the current result. It does not necessarily tell us that the customer first deposited $500 and then withdrew $100.
For example, suppose the account went through these operations:
Initial balance: $0
Deposit $500
Withdraw $100
Current balance: $400
If we only store the final state, the previous business actions are no longer part of the account's primary data. We could add separate transaction tables or audit logs, but now we are maintaining additional mechanisms to reconstruct what happened.
Why History Can Matter
This historical information can be extremely valuable in real systems. Imagine that a customer contacts the bank and says, "I do not recognize this withdrawal." The current balance of $400 does not explain what happened. We need to know which operation changed the balance, when it happened, and potentially why it was accepted.
The same problem appears outside banking. In an order system, storing only Status = Shipped does not tell us that the order was first created, then paid, then packed, and finally shipped. In an airline system, storing only the current flight status as Departed does not preserve the sequence of operational changes that produced that state.
The Problem with Treating State as the Whole Truth
Traditional state-based persistence is excellent when our primary question is, "What is the current state?" However, some business systems also need to answer, "What happened?", "When did it happen?", and "How did we arrive at the current state?"
We can solve this by storing additional audit records, but those records are often treated as secondary information. The main database still contains the current state, while the history exists somewhere else. This can eventually create consistency and maintenance challenges because the current state and historical records must remain synchronized.
The Event Sourcing Idea
Event Sourcing approaches the problem from a different direction. Instead of making the current state the primary source of truth, we store the business events that changed the state. The current state can then be calculated by replaying those events.
For our bank account, we could store:
AccountOpened
Deposited $500
Withdrew $100
The current balance is no longer something that must exist as the fundamental historical record. It is the result of applying those events:
0
+ 500
- 100
= 400
The important idea is that an event represents something that already happened. MoneyDeposited means money was deposited. MoneyWithdrawn means money was withdrawn. These events are normally treated as immutable historical facts rather than records that we continuously overwrite.
A Simple Comparison
With traditional state-based persistence, we might have:
Database:
AccountId = 1
Balance = 400
With Event Sourcing, we might instead have:
Event 1: AccountOpened
Event 2: MoneyDeposited(500)
Event 3: MoneyWithdrawn(100)
The second approach gives us something the first approach does not naturally provide: a complete history of the changes that produced the current state. This is the fundamental reason Event Sourcing exists.
The Trade-Off
Event Sourcing is not simply a better database design. It solves a particular problem by introducing another set of complexities. Reconstructing state, handling event versions, managing concurrency, building projections, and dealing with large event streams all require additional design decisions.
Therefore, Event Sourcing should not be introduced merely because "keeping history is useful." Traditional CRUD plus an audit table may be simpler and completely sufficient for many applications. Event Sourcing becomes particularly interesting when the history of business decisions and state changes is itself important to the system.
In the next lesson, we will move from this problem to the core Event Sourcing model and see exactly how storing events as the source of truth changes the way an application persists and retrieves data.