Implementing Microservices with .NET Core and Angular

 Using (CQRS • Saga • Azure Service Bus • Circuit Breaker • Adapter • Patterns & Best Practices) 

TL;DR (What you'll build)

A cloud-ready microservices system where each microservice:

• Is a .NET Core Web API (REST) exposing bounded-capability endpoints.

• Uses CQRS: separate read model and write model (commands vs queries).

• Coordinates complex business workflows using SAGA (via Azure Service Bus).

• Communicates async via Azure Service Bus (topics/subscriptions or queues).

• Uses Circuit Breaker (Polly) + retry/backoff to improve resilience.

• Uses Adapter pattern for external systems and for infrastructure abstraction.

• Front-end is an Angular SPA calling an API Gateway (or BFF) that routes to microservices.

---

Architecture overview (high level)

1. API Gateway / BFF (optional): Single public endpoint, routing, auth, aggregation.

2. Microservices (multiple): each with its own database (DB-per-service), own bounded context.

   • Example services: Orders, Payments, Inventory, Customers, Shipping.

3. Azure Service Bus: topics for publish/subscribe events, queues for commands/tasks.

4. CQRS: Commands handled by write-side; queries served by optimized read DB (e.g., read replica or denormalized store like CosmosDB/SQL).

5. Sagas: Orchestrate long-running transactions (compensations) via events/messages.

6. Polly: Circuit Breaker + Retry + Timeout for outbound HTTP/Bus calls.

7. Monitoring & Logging: Application Insights, centralized logs (ELK/Serilog sink).

8. CI/CD & Hosting: Build pipeline -> container images -> AKS / App Service.

Diagram (textual):

Angular SPA -> API Gateway -> Microservice A (Commands) -> Azure Service Bus topic
                                        -> Microservice B subscribes (Updates read DB)
Microservice A -> publishes events -> Azure Service Bus -> Saga Orchestrator -> other services

---

Core design principles

• Single Responsibility & Bounded Contexts — keep microservices small and focused.

• DB-per-service — avoid shared DB to reduce coupling.

• Event-driven — communicate asynchronously where eventual consistency is acceptable.

• Idempotency — message handlers must be idempotent.

• Id-based correlation — include correlation IDs in messages for tracing.

• Resilience — use bulkheads, circuit breakers, retries, timeouts.

• Observability — structured logs, distributed tracing (W3C TraceContext).

---

Key patterns & how to apply them

1. CQRS (Command Query Responsibility Segregation)

• Write side: handles commands (POST /orders → CreateOrderCommand). Validations and state changes happen here.

• Read side: optimized projection used for queries (GET /orders/123). Read DB is updated by event subscribers.

• Implementation:

  • Commands go to controllers → Command Handlers (MediatR is common).

  • After state change, publish domain event (to Azure Service Bus).

  • Event consumers update read model (denormalized tables or NoSQL).

Example files/folders:

/OrdersService
  /Commands
    CreateOrderCommand.cs
    CreateOrderCommandHandler.cs
  /Domain
    Order.cs
    OrderAggregate.cs
  /Events
    OrderCreatedEvent.cs
  /ReadModel
    OrderReadModel.cs
    OrderProjectionHandler.cs

2. Saga (long-running transactions)

• Use Saga to coordinate multi-step flows (e.g., order -> reserve inventory -> charge payment -> ship).

• Two approaches:

  • Choreography: each service reacts to events. Simple but can be hard to reason about for complex flows.

  • Orchestration: Saga Orchestrator service sends commands to participants and listens for replies/events. Easier to visualize and control compensations.

• Implementation with Azure Service Bus:

  • Orchestrator publishes ReserveInventoryCommand to a queue.

  • Inventory service replies InventoryReservedEvent or InventoryReserveFailedEvent.

  • Orchestrator then issues ChargePaymentCommand or CompensateOrderCommand.

• Always plan for compensation (rollback-like steps) when a later step fails.

3. Azure Service Bus (Messaging)

• Use Topics + Subscriptions for publish/subscribe; Queues for point-to-point commands.

• Message schema:

  • MessageId, CorrelationId, CausationId, MessageType, Payload.

• Use dead-letter queues for poison messages and implement retries & DLQ handling.

4. Circuit Breaker & Resilience (Polly)

• Wrap all outbound network calls (HTTP, DB, Service Bus operations) with retry + jittered backoff + circuit breaker.

• Typical policy: Retry (3 attempts, exponential backoff) + Circuit Breaker (open after 5 failures for 30s) + Timeout (5s).

• Example: use HttpClientFactory with Polly policies.

5. Adapter Pattern

• Use adapters to isolate external/specific client libs (e.g., Azure Service Bus SDK, Payment Gateway SDK) from your domain.

• Define an interface (e.g., IPaymentGateway) and implement StripeAdapter/DummyAdapter. This supports testability and swapping providers.

6. Repository + Unit of Work (when needed)

• Use repository for aggregate persistence; Unit of Work if multiple repositories within same DB transaction (but prefer small transactions).

7. Anti-Corruption Layer

• When integrating with legacy systems, use an adapter/anti-corruption layer to map incompatible models.

---

Concrete implementation steps

1. Set up projects

Solution example:

/MyEshop
  /src
    /ApiGateway (aspnet core)
    /Services
      /Orders.Api
      /Inventory.Api
      /Payments.Api
      /Saga.Orchestrator
    /Shared
      /Messaging (contracts)
      /Common (logging, exceptions)
  /ui
    /angular-spa

2. Shared contracts & DTOs

Create a Shared/Messaging project with message types (events/commands) used across services. Strongly-typed messages reduce coupling.

Example OrderCreatedEvent:

public record OrderCreatedEvent(Guid OrderId, Guid CustomerId, decimal Total, IEnumerable<OrderLineDto> Lines);

3. Implement CQRS in a service (Orders example)

• Use MediatR for intra-service command/handler pattern.

• Controller:

[HttpPost]
public async Task<IActionResult> Create([FromBody] CreateOrderRequest req)
{
    var cmd = new CreateOrderCommand(req.CustomerId, req.Lines);
    var result = await _mediator.Send(cmd);
    return CreatedAtAction(..., result.OrderId);
}

• CommandHandler:

public class CreateOrderCommandHandler : IRequestHandler<CreateOrderCommand, CreateOrderResult>
{
    private readonly IOrderRepository _repo;
    private readonly IMessageBus _bus; // adapter over Azure Service Bus

    public async Task<CreateOrderResult> Handle(CreateOrderCommand request, CancellationToken token)
    {
        var order = Order.Create(request.CustomerId, request.Lines);
        await _repo.AddAsync(order);
        await _repo.UnitOfWork.SaveChangesAsync();

        // publish event to bus
        await _bus.Publish(new OrderCreatedEvent(order.Id, order.CustomerId, order.Total, order.Lines));

        return new CreateOrderResult(order.Id);
    }
}

4. Publish/Subscribe (Azure Service Bus)

• Implement IMessageBus adapter that wraps Azure Service Bus ServiceBusClient.

• For publishing events:

public async Task Publish<T>(T @event) where T : class
{
    var message = JsonSerializer.Serialize(@event);
    await _sender.SendMessageAsync(new ServiceBusMessage(message) { MessageId = Guid.NewGuid().ToString() });
}

• For subscriptions, create background services (hosted services) to process messages and project read model.

5. Saga Orchestration (example flow)

• Orders publishes OrderCreatedEvent.

• Saga.Orchestrator subscribes. On OrderCreated:

  1. Send ReserveInventoryCommand to Inventory queue.

  2. Wait for InventoryReservedEvent or InventoryReserveFailedEvent.

  3. If reserved, send ChargePaymentCommand to Payments.

  4. If payment succeeds, send ShipOrderCommand to Shipping.

  5. If any step fails, run compensating actions (e.g., ReleaseInventoryCommand, RefundPaymentCommand) and mark order Failed.

Orchestrator pattern: implement state persistence for saga instances (e.g., in CosmosDB/SQL). Store Saga state keyed by CorrelationId.

6. Circuit Breaker with HttpClientFactory + Polly

Register in Program.cs:

services.AddHttpClient("payments", client => { client.BaseAddress = new Uri(config["Payments:BaseUrl"]); })
  .AddPolicyHandler(GetRetryPolicy())
  .AddPolicyHandler(GetCircuitBreakerPolicy());

IAsyncPolicy<HttpResponseMessage> GetRetryPolicy() =>
  HttpPolicyExtensions.HandleTransientHttpError()
    .WaitAndRetryAsync(3, retryAttempt => TimeSpan.FromSeconds(Math.Pow(2, retryAttempt)));

IAsyncPolicy<HttpResponseMessage> GetCircuitBreakerPolicy() =>
  HttpPolicyExtensions.HandleTransientHttpError()
    .CircuitBreakerAsync(5, TimeSpan.FromSeconds(30));

7. Angular front-end (simple BFF pattern)

• Angular calls API Gateway/BFF endpoints which route to microservices.

• Use typed services, interceptors for auth (JWT) and correlation headers.

Example Angular service:

@Injectable({ providedIn: 'root' })
export class OrdersService {
  constructor(private http: HttpClient) {}
  createOrder(order: CreateOrderDto) {
    return this.http.post<ApiResponse<OrderView>>('/api/orders', order);
  }
}

Add an HTTP interceptor to send x-correlation-id header and Authorization bearer token.

8. Idempotency & Message deduplication

• Keep ProcessedMessage table with MessageId to ignore duplicate message deliveries.

• Use Service Bus message-id + dedupe logic.

9. Testing

• Unit test command handlers and domain logic.

• Integration tests for message flows (use in-memory or test instance of Service Bus or use the Azure Service Bus emulator).

• Contract tests for message contracts.

10. Observability, Monitoring & Health

• Use Application Insights (or OpenTelemetry) for distributed tracing and metrics.

• Add Health Checks endpoints and Kubernetes liveness/readiness probes.

• Structured logging with Serilog, sink to central store.

---

Deployment & CI/CD (short)

• Dockerize each service.

• Build pipeline: build -> test -> image -> push to ACR.

• Deploy pipeline: pull image -> AKS (Kubernetes manifests) or Azure App Service.

• Use Helm charts for Kubernetes; include config via ConfigMaps and secrets via Azure Key Vault.

• Use canary / blue-green with ingress controller and feature flags (LaunchDarkly or open-source).

---

Practical tips & pitfalls

• Do not share a single database across services — this creates high coupling.

• Prefer eventual consistency; design UIs to show pending states (e.g., "Payment Processing").

• Compensation is not rollback — design compensating actions carefully.

• Keep messages small and versionable — maintain backward compatibility.

• Start with choreography for simple flows; move to orchestrator for complex flows requiring centralized control.

• Use feature toggles while deploying new message flows.

---

Minimal code sample: publish event to Service Bus (adapter)

public interface IMessageBus
{
    Task PublishAsync<T>(T @event, string topicName);
}

public class AzureServiceBusAdapter : IMessageBus
{
    private readonly ServiceBusClient _client;
    public AzureServiceBusAdapter(ServiceBusClient client) => _client = client;

    public async Task PublishAsync<T>(T @event, string topicName)
    {
        var sender = _client.CreateSender(topicName);
        var body = JsonSerializer.Serialize(@event);
        var msg = new ServiceBusMessage(body) { MessageId = Guid.NewGuid().ToString() };
        msg.ApplicationProperties["MessageType"] = typeof(T).FullName;
        await sender.SendMessageAsync(msg);
    }
}

---

Folder structure suggestion (service-level)

/Orders.Api
  /Controllers
  /Commands
  /CommandHandlers
  /Domain
  /Events
  /Projections
  /Infrastructure
     /Persistence
     /Messaging (ServiceBus adapter)
     /HttpClients (adapters)
  /Services (business services)
  /Tests

---

Checklist before go-live

• Message contracts versioned & documented.

• Idempotency & DLQ handled.

• Circuit breakers instrumented.

• Health checks + readiness gates.

• Logging and distributed tracing configured.

• Saga persistence & compensations tested.

• Secure secrets (Key Vault) and communication (TLS, JWT).

---

Suggested next steps (practical small milestones)

1. Create a Shared.Messaging project with event/command contracts.

2. Scaffold Orders service with basic create order command and publisher (no Saga yet).

3. Create Inventory service subscriber to update read model.

4. Add Azure Service Bus adapter and test end-to-end message flow locally (or in dev subscription).

5. Implement a simple saga orchestrator for a 3-step flow (inventory → payment → shipping).

6. Add Polly policies and instrument logging/tracing.

7. Dockerize and deploy to a dev cluster.

Design Patterns Explained: GoF & Modern Patterns with Real-World Examples

In software development, Design Patterns act as proven solutions to common problems. Instead of reinventing the wheel, developers can apply these reusable patterns to build scalable, maintainable, and robust applications.

This article explains Gang of Four (GoF) design patterns and also highlights modern design patterns widely used in cloud, microservices, and enterprise systems.

---

📌 What Are Design Patterns?

Design patterns are general, reusable solutions to recurring software problems. They are not code snippets but guidelines on how to solve design challenges effectively.

Benefits of using design patterns:

• Improve code reusability & readability

• Promote best practices & standardization

• Reduce development & maintenance time

• Help in team communication (common vocabulary)

---

🏛 Gang of Four (GoF) Design Patterns

The GoF introduced 23 design patterns in their famous book "Design Patterns: Elements of Reusable Object-Oriented Software".

They are categorized into three groups:

1. Creational Patterns – Object creation

Help in creating objects without exposing the creation logic.

• Singleton → Ensures only one instance exists.

  public sealed class Logger
  {
      private static readonly Logger _instance = new Logger();
      private Logger() { }
      public static Logger Instance => _instance;
  }
  

  ✅ Used in logging, caching, config management.

• Factory Method → Subclasses decide object creation.

• Abstract Factory → Create families of related objects.

• Builder → Step-by-step object construction.

• Prototype → Clone objects without depending on their class.

---

2. Structural Patterns – Class & object composition

Define how objects are composed for flexibility.

• Adapter → Converts one interface into another.
  ✅ Example: Wrapping an old payment gateway API to fit into a new e-commerce app.

• Decorator → Add responsibilities dynamically.

• Facade → Provide a simple interface to a complex system.

• Composite → Treat individual & group objects uniformly (tree structure).

• Proxy → Control access (like lazy loading, security).

---

3. Behavioral Patterns – Object communication

Define how objects interact & responsibilities are distributed.

• Observer → One-to-many dependency.

  public interface IObserver { void Update(string msg); }
  public class User : IObserver
  {
      private string _name;
      public User(string name) => _name = name;
      public void Update(string msg) => 
  Console.WriteLine($"{_name} got update: {msg}");
  }
  public class NotificationService
  {
      private List<IObserver> observers = new();
      public void Subscribe(IObserver o) => observers.Add(o);
      public void Notify(string msg)
      {
          foreach (var o in observers) o.Update(msg);
      }
  }
  

  ✅ Used in push notifications, event-driven systems.

• Strategy → Choose algorithm at runtime.

• Command → Encapsulate actions as objects.

• State → Change behavior based on object state.

• Mediator, Memento, Interpreter, Visitor, Template Method, Chain of Responsibility, Iterator.

---

🚀 Modern / New Design Patterns (Beyond GoF)

Today’s software systems (Cloud, Microservices, Big Data, AI) need patterns beyond GoF. Here are some important ones:

1. Dependency Injection (DI)

• Promotes loose coupling by injecting dependencies instead of creating them inside a class.

• Widely used in .NET Core, Spring Boot, Angular.

  public class OrderService
  {
      private readonly IPaymentGateway _paymentGateway;
      public OrderService(IPaymentGateway paymentGateway)
   => _paymentGateway = paymentGateway;
      public void PlaceOrder() => _paymentGateway.Process();
  }
  

---

2. Repository Pattern

• Abstracts database access to keep business logic independent.

• Used in DDD (Domain-Driven Design) and enterprise apps.

---

3. CQRS (Command Query Responsibility Segregation)

• Separates read and write operations for better scalability.

• Common in microservices + event sourcing.

---

4. Saga Pattern (Microservices)

• Handles distributed transactions across microservices.

• Uses event-driven orchestration or choreography to maintain consistency.

---

5. Event Sourcing

• State changes are stored as events (not just final values).

• Used in banking, e-commerce order history, blockchain-like systems.

---

6. Circuit Breaker Pattern

• Prevents cascading failures by stopping requests to a failing service.

• Implemented in Polly (.NET), Resilience4j (Java).

---

7. API Gateway Pattern

• Acts as a single entry point for microservices.

• Handles routing, load balancing, authentication, caching.

---

8. Strangler Fig Pattern

• Gradually replaces legacy systems by routing new features to modern services while
   old ones still run.

---

🔑 Conclusion

• GoF Patterns are still foundational for object-oriented design.

• Modern patterns are essential for cloud, distributed systems, and microservices.

• By combining both, developers can build scalable, reliable, and future-proof applications.

How to Implement MicroServices in .Net Core Application- Point-to-point to explanation

 

1) Service boundaries & contracts

• One business capability per service (e.g., Orders, Payments, Catalog). Each owns its own database.
• API surface: REST (JSON) or gRPC for synchronous calls; events (Kafka/RabbitMQ/Azure Service Bus) for async integration.
• Version your APIs (e.g., /v1), keep backward compatible changes, use OpenAPI/Swagger and contract tests (Pact).

2) Resilience patterns (timeouts, retries, circuit breaker, bulkhead)

Use HttpClientFactory + Polly on every outbound call:

builder.Services.AddHttpClient("CatalogClient", c =>

{

    c.BaseAddress = new Uri(config["CatalogUrl"]);

    c.Timeout = TimeSpan.FromSeconds(3);

})

.AddTransientHttpErrorPolicy(p => p.WaitAndRetryAsync(

    retryCount: 3,

    sleepDurationProvider: attempt => TimeSpan.FromMilliseconds(200 * Math.Pow(2, attempt)), // expo backoff

    onRetry: (outcome, ts, attempt, ctx) =>

        logger.LogWarning(outcome.Exception, "Retry {Attempt}", attempt)))

.AddTransientHttpErrorPolicy(p => p.CircuitBreakerAsync(

    handledEventsAllowedBeforeBreaking: 5,

    durationOfBreak: TimeSpan.FromSeconds(30)))

.AddPolicyHandler(Policy.BulkheadAsync<HttpResponseMessage>(maxParallelization: 50, maxQueuingActions: 100));

Rules of thumb

• Always set timeouts on I/O.
• Prefer idempotent operations; send Idempotency-Key headers for POSTs where possible.
• Use cancellation tokens end-to-end.

3) Circuit breaker semantics

• Trip when a dependency is unhealthy; fail fast and return a cached/partial response or friendly error.
• Expose breaker state via metrics and health checks (see §5).

4) Data & database handling (consistency without distributed transactions)

• Database per service (EF Core or Dapper). No cross-service joins.
• Propagate data between services using domain events—not shared tables.
• Ensure atomicity with the Outbox pattern:
1. Write your entity and an OutboxMessage row in the same DB transaction.
2. A background Outbox Publisher reads unpublished rows and emits them to the broker, then marks them sent.

// Inside your OrderService command handler

await using var tx = db.Database.BeginTransactionAsync();

db.Orders.Add(order);

db.Outbox.Add(new OutboxMessage {

    Type = "OrderPlaced",

    Payload = JsonSerializer.Serialize(new { order.Id, order.Total, order.CustomerId }),

    OccurredOn = DateTimeOffset.UtcNow

});

await db.SaveChangesAsync();

await tx.CommitAsync();

// BackgroundService publisher

protected override async Task ExecuteAsync(CancellationToken ct)

{

    while (!ct.IsCancellationRequested)

    {

        var messages = await db.Outbox

            .Where(m => m.SentOn == null)

            .OrderBy(m => m.OccurredOn)

            .Take(100).ToListAsync(ct);

 

        foreach (var msg in messages)

        {

            await broker.PublishAsync(msg.Type, msg.Payload, ct);

            msg.SentOn = DateTimeOffset.UtcNow;

        }

        await db.SaveChangesAsync(ct);

        await Task.Delay(TimeSpan.FromSeconds(1), ct);

    }

}

• Updating “relative/related” databases: consume those events in other services to update local read models (CQRS). Use Sagas (orchestration or choreography) for multi-step processes (e.g., Order → Reserve Inventory → Process Payment → Arrange Shipping), with timeouts + compensation.

5) Health checks, readiness, and monitoring

Add liveness (process alive) and readiness (deps OK) endpoints:

builder.Services.AddHealthChecks()

    .AddSqlServer(config.GetConnectionString("OrdersDb"))

    .AddRabbitMQ(config["RabbitMq:Connection"])

    .AddUrlGroup(new Uri(config["CatalogUrl"] + "/health"), name: "catalog");

 

app.MapHealthChecks("/health/live", new HealthCheckOptions { Predicate = _ => false });

app.MapHealthChecks("/health/ready", new HealthCheckOptions {

    Predicate = _ => true,

    ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse

});

• Kubernetes/Containers probe /health/live and gate traffic on /health/ready.
• Add self-diagnostics (thread pool starvation, GC info) via EventCounters exported by OpenTelemetry.

6) Observability (logs, traces, metrics)

Unified, structured telemetry makes bugs easy to fix.

OpenTelemetry (logs + traces + metrics) → backends like Grafana Tempo/Prometheus/Loki, Jaeger, Elastic, or Azure Application Insights.

builder.Services.AddOpenTelemetry()

 .WithTracing(t => t

     .AddAspNetCoreInstrumentation()

     .AddHttpClientInstrumentation()

     .AddSqlClientInstrumentation()

     .AddSource("OrderService")

     .AddOtlpExporter()) // or Azure Monitor exporter

 .WithMetrics(m => m

     .AddAspNetCoreInstrumentation()

     .AddRuntimeInstrumentation()

     .AddHttpClientInstrumentation()

     .AddPrometheusExporter());

 

app.MapPrometheusScrapingEndpoint(); // /metrics

Structured logging (e.g., Serilog → console/JSON + sink of choice):

Log.Logger = new LoggerConfiguration()

    .Enrich.FromLogContext()

    .Enrich.WithCorrelationId()

    .WriteTo.Console(new JsonFormatter())

    .WriteTo.Seq(config["SeqUrl"]) // or Elastic/Azure

    .CreateLogger();

 

builder.Host.UseSerilog();

• Include CorrelationId/TraceId in every log; propagate with middleware:

app.Use(async (ctx, next) =>

{

    var cid = ctx.Request.Headers["X-Correlation-ID"].FirstOrDefault() ?? Guid.NewGuid().ToString();

    ctx.Items["CorrelationId"] = cid;

    ctx.Response.Headers["X-Correlation-ID"] = cid;

    using (LogContext.PushProperty("CorrelationId", cid))

        await next();

});

• Emit domain events and key business metrics (orders placed, failures, latencies, queue depth).

7) API gateway / proxy servers

• Use a Gateway for routing, TLS, auth/ratelimiting, request shaping:
• YARP (in-process .NET reverse proxy) or Ocelot; at the edge you can also use NGINX/Envoy.
• Centralize AuthN/Z (OAuth2/OIDC via Azure AD, Auth0, or Duende IdentityServer), rate limits, request/response size limits, CORS, and header normalization at the gateway.
• Prefer service discovery (K8s DNS) and mTLS in cluster; consider a service mesh (Istio/Linkerd) for uniform retries/mtls/traffic shifting and better telemetry without code changes.

8) Security & configuration

• Never share DBs across services. Grant least privilege per service.
• Secrets via Azure Key Vault / K8s secrets; bind with IOptions<T> and validate on startup.
• Enforce TLS everywhere, JWT validation, rotating keys, and input validation (FluentValidation).

9) Deployment & runtime

• Containerize each service (Docker), build once, promote the same image through Dev → QA → Prod.
• Orchestrate with Kubernetes (AKS) or Azure App Services for simpler cases.
• Blue/Green or Canary deployments; mesh/gateway can split traffic gradually.
• Horizontal scaling via HPA (CPU/RPS/latency). Keep services stateless; externalize state (DB/cache).
• Caching: local memory for tiny TTLs, Redis for shared cache; always set cache keys + expirations.

10) CI/CD and quality gates

• GitHub Actions / Azure DevOps Pipelines:
• Build → unit/integration/contract tests → SCA & SAST → container scan → publish image → deploy via Helm/Bicep/Terraform.
• Block merges without green tests and security checks.
• Spin ephemeral test environments per PR when feasible.

11) Diagnostics & “easy bug fixing”

• Global exception handler → consistent error envelopes with traceId:

app.UseExceptionHandler(builder =>

{

    builder.Run(async ctx =>

    {

        var traceId = Activity.Current?.Id ?? ctx.TraceIdentifier;

        var problem = new { traceId, message = "Unexpected error." };

        ctx.Response.StatusCode = 500;

        await ctx.Response.WriteAsJsonAsync(problem);

    });

});

• Feature flags (Azure App Configuration) for safe rollouts; dark-launch code paths.
• Replayable messages (don’t auto-ack until processed). Keep dead-letter queues and dashboards.
• Synthetic checks (probes that place a tiny test order in lower envs and alert on end-to-end failure).

12) Database migrations & uptime

• Apply EF Core migrations at startup (carefully) or via migration jobs;
• Prefer expand-and-contract: add new columns/tables (expand), release code that writes both, backfill, then remove old fields (contract).
• For read models, permit rebuild from event streams or upstream APIs if corrupted.

13) Monitoring & alerts you’ll actually use

• Golden signals per service: Latency, Traffic, Errors, Saturation.
• Alert on SLIs/SLOs (e.g., p95 < 300ms, error rate < 1%), queue lag, breaker open rate, DB CPU, connection pool exhaustion.
• Route alerts to Teams/Slack with runbooks linked (how to diagnose/rollback).

14) Local/dev ergonomics

• Docker-compose for local broker + DB + dependencies.
• Seed data + test fixtures; make services start with sensible defaults.
• Makefile/PS scripts for common tasks; one-command bootstrap.

---

Minimal reference architecture (quick checklist)

• ASP.NET Core services (+ gRPC if needed)
• HttpClientFactory + Polly (timeouts/retries/circuit breaker/bulkhead)
• Outbox + BackgroundPublisher; events over Kafka/RabbitMQ/Azure Service Bus
• DB per service (EF Core/Dapper) + migrations (expand/contract)
• OpenTelemetry (traces/metrics/logs) → Prometheus/Grafana/Tempo or App Insights
• Serilog structured logging + correlation IDs
• Health checks: /health/live, /health/ready
• API Gateway: YARP/Ocelot; edge NGINX/Envoy; OAuth2/OIDC
• Containerized; AKS/App Service; blue/green or canary
• CI/CD with tests, scans, infrastructure as code
• Feature flags; DLQs; synthetic probes; runbooks