Event-Driven Microservices Architecture: A Practical Guide for Software Developers

To work effectively with a messaging system, we need to understand the basic building blocks that control how messages move through it. This lesson introduces producers, consumers, queues, topics, subscriptions, and related concepts, using RabbitMQ and Kafka to show how the same event can reach multiple independent services.

Producers and Consumers

In an event-driven system, a producer is the application component that creates and sends a message, while a consumer is the component that receives and processes it. For example, when a customer successfully places an order, the Order Service can act as the producer and publish an OrderPlaced event. The Payment Service, Inventory Service, and Notification Service can act as consumers because they react to that event.

Order Service
   Producer
      |
      | OrderPlaced
      v
Messaging Infrastructure
      |
      +----> Payment Service
      +----> Inventory Service
      +----> Notification Service

The important point is that the producer and consumers do not need to communicate directly. The messaging infrastructure sits between them and handles the delivery of messages. This allows consumers to be added or removed without requiring the producer to change its business logic.

Queues in RabbitMQ

In RabbitMQ, a queue is a place where messages wait until consumers process them. A consumer typically receives messages from a queue and acknowledges them after successful processing.

For example, the Payment Service might consume messages from a payment-orders queue.

Order Service
      |
      v
   RabbitMQ
      |
      v
payment-orders queue
      |
      v
Payment Service

RabbitMQ also uses exchanges to route messages to queues. A producer normally publishes a message to an exchange rather than directly to a queue. The exchange then uses bindings and routing rules to determine which queues should receive the message.

Order Service
      |
      v
   Exchange
    /  |  \
   v   v   v
Queue Queue Queue
  |     |     |
Payment Inventory Notification

This routing model allows one published event to reach multiple independent consumers.

Topics and Subscriptions

Kafka uses a different terminology. Instead of RabbitMQ queues and exchanges, Kafka organizes events into topics. A producer publishes records to a topic, and consumers read those records from the topic.

For example:

Order Service
      |
      | OrderPlaced
      v
Kafka: orders topic
      |
      +----> Payment Consumer
      +----> Inventory Consumer
      +----> Notification Consumer

Kafka consumers normally belong to consumer groups. Consumers in the same group cooperate to process a topic, while different consumer groups can independently consume the same events.

For example, the Payment Service can belong to one consumer group and the Inventory Service to another.

             orders topic
                  |
        +---------+---------+
        |                   |
 Payment Group        Inventory Group
        |                   |
 Payment Service      Inventory Service

Both groups can process the same OrderPlaced event independently. This is an important difference from a simple queue where multiple consumers of the same queue commonly compete for messages.

Multiple Consumers and One Event

Suppose the Order Service publishes one OrderPlaced event. The business requires three things to happen: payment must be processed, inventory must be reserved, and the customer must receive a notification. These are independent responsibilities, so we do not want the Order Service to call each service directly.

With RabbitMQ, separate queues can receive the event through an exchange. With Kafka, separate consumer groups can read the same topic. In both cases, the architectural result is similar: multiple services can independently react to the same business event.

                 OrderPlaced
                      |
                Messaging System
                 /      |      \
                /       |       \
          Payment    Inventory   Notification

This is one of the main strengths of event-driven architecture. Publishing an event can trigger several independent business processes without the producer needing to know about all of them.

A .NET Example

Using MassTransit with RabbitMQ, a producer can publish an event with code similar to this:

await publishEndpoint.Publish(
    new OrderPlaced(
        order.Id,
        customer.Id,
        order.TotalAmount));

A consumer can then react to the event:

public async Task Consume(
    ConsumeContext<OrderPlaced> context)
{
    var order = context.Message;

    await ProcessPayment(order);
}

The messaging library handles much of the underlying broker communication, allowing the application code to focus on the business operation.

The Mental Model

A useful way to remember these concepts is that the producer creates the message, the messaging infrastructure transports and organizes it, and the consumer processes it. RabbitMQ commonly uses exchanges and queues to route messages, while Kafka uses topics and consumer groups to organize and consume event streams.

Once these concepts are understood, the next question becomes more important: what exactly should we put inside an event? In the next lesson, we will learn how to design event contracts, including event names, payloads, identifiers, timestamps, and metadata.