Skip to main content

Level 3-4: Async Events & Saga

5 min read
AI documentation guide

Intermediate to advanced complexity patterns covering event-driven architecture and distributed transactions.

Level 3: Async Event-Driven MSA​

Characteristics:

  • Event bus (Kafka, RabbitMQ, EventBridge)
  • Eventually Consistent data model
  • Domain event publish/subscribe
  • Asynchronous communication, loose coupling

AIDLC Application:

Figure 1. AIDLC application flow for Level 3 async event-driven MSA

Expand the diagram to read labels and follow connections.
Diagram

Ontology Level​

Full Ontology: Entities + relationships + event schemas + invariants

  • Explicit event contracts (schema registry)
  • Define event ordering/dependencies

Example Ontology (Event Schema):

# ontology/order-events.yaml
events:
  OrderCreated:
    schema:
      eventId: string
      orderId: string
      sequence: long
      userId: string
      items: list<OrderItem>
      createdAt: timestamp
    producers:
      - OrderService
    consumers:
      - InventoryService (deduct stock)
      - NotificationService (send notification)
    idempotencyKey: eventId
    partitionKey: orderId
    ordering: monotonic sequence per partition (orderId)

  OrderConfirmed:
    schema:
      eventId: string
      orderId: string
      sequence: long
      confirmedAt: timestamp
    producers:
      - PaymentService
    consumers:
      - ShippingService
    idempotencyKey: eventId
    partitionKey: orderId

invariants:
  - OrderCreated must precede OrderConfirmed
  - OrderCancelled cannot follow OrderShipped

Harness Checklist​

  • Include: Event schema verification (Avro, Protobuf)
  • Include: Idempotency harness (duplicate event handling)
  • Include: Event ordering verification
  • Include: Eventually Consistent tests (eventual state verification)
  • Include: Dead Letter Queue handling

Application Strategy​

  • Define events through Event Storming
  • Event schema ontology required
  • Idempotency harness (handle duplicate events)
  • Integrate with Schema Registry
  • Automate Eventually Consistency testing

Level 4: Saga + Compensating Transactions​

Characteristics:

  • Distributed transactions (Saga pattern)
  • Compensating transactions
  • Orchestration Saga or Choreography Saga
  • Complex failure scenarios

AIDLC Application:

Figure 2. AIDLC application flow for Level 4 Saga + compensating transactions

Expand the diagram to read labels and follow connections.
Diagram

Ontology Level​

Full Ontology + Saga Spec: Entities + events + Saga steps + compensation logic

  • Define Saga state transitions per step
  • Explicit compensation logic (rollback scenarios)

Example Ontology (Saga):

# ontology/travel-booking-saga.yaml
saga:
  name: TravelBookingSaga
  type: orchestration
  orchestrator: BookingService

  steps:
    - name: ReserveFlight
      service: FlightService
      action: reserveFlight
      compensation: cancelFlightReservation
      timeout: 10s
      retryPolicy: exponentialBackoff(3)

    - name: ReserveHotel
      service: HotelService
      action: reserveHotel
      compensation: cancelHotelReservation
      timeout: 10s
      retryPolicy: exponentialBackoff(3)

    - name: ChargePayment
      service: PaymentService
      action: chargeCard
      compensation: refundPayment
      timeout: 5s
      retryPolicy: none
      onUnknownOutcome: reconcile

  failureScenarios:
    - scenario: FlightReservationFailed
      compensations:
        - (none, first step failure)

    - scenario: HotelReservationFailed
      compensations:
        - cancelFlightReservation

    - scenario: PaymentFailed
      compensations:
        - cancelHotelReservation
        - cancelFlightReservation

    - scenario: PaymentUnknown
      description: Charge outcome is uncertain (e.g. timeout). Reconcile by querying payment status, then decide to complete or compensate.
      action:
        - queryPaymentStatus
        - if confirmed success, complete; if confirmed failure, cancelHotelReservation + cancelFlightReservation

  invariants:
    - All compensations must be idempotent
    - Compensation order is reverse of execution order
    - Saga timeout = sum of (step timeout x max retries) + compensation time + buffer

Harness Checklist​

  • Include: Saga step-by-step verification
  • Include: Compensating transaction verification (rollback scenarios)
  • Include: Timeout harness (prevent infinite wait)
  • Include: Retry policy verification
  • Include: Circuit breaker
  • Include: Distributed tracing (OpenTelemetry)

Harness Implementation Example​

Compensating Transaction Harness​

# harness/saga_compensation_test.py
def test_saga_compensation():
    """Verify that compensation logic works correctly on Saga failure"""
    saga = TravelBookingSaga()

    # 1. Flight reservation success
    saga.execute_step("ReserveFlight")
    assert flight_service.is_reserved("flight123")

    # 2. Hotel reservation success
    saga.execute_step("ReserveHotel")
    assert hotel_service.is_reserved("hotel456")

    # 3. On payment failure the saga triggers compensation automatically
    with pytest.raises(PaymentFailedException):
        saga.execute_step("ChargePayment")

    # 4. Verify the compensation the orchestrator ran on failure
    assert saga.status == "compensated"
    assert not hotel_service.is_reserved("hotel456")  # cancelled
    assert not flight_service.is_reserved("flight123")  # cancelled

Application Strategy​

  • Saga design required (orchestration vs choreography)
  • Explicit compensation logic in ontology
  • Add compensating transaction verification to harness
  • Test all failure scenarios (Chaos Engineering)
  • Expert review required

Next Steps​

Highest complexity Event Sourcing pattern: