# System Architecture This document provides a comprehensive overview of the Go Platform architecture, including system components, their relationships, and how modules integrate with the core platform. ## Table of Contents - [High-Level Architecture](#high-level-architecture) - [Layered Architecture](#layered-architecture) - [Module System Architecture](#module-system-architecture) - [Component Relationships](#component-relationships) - [Data Flow](#data-flow) - [Deployment Architecture](#deployment-architecture) ## High-Level Architecture The Go Platform follows a **microservices architecture** where each service is independently deployable from day one: - **Core Kernel**: Infrastructure only (config, logger, DI, health, metrics, observability) - no business logic - **Core Services**: Auth Service, Identity Service, Authz Service, Audit Service - separate microservices - **API Gateway**: Single entry point for all external traffic, handles routing and authentication - **Feature Services**: Blog, Billing, Analytics, etc. - independent services - **Infrastructure Adapters**: Cache, Event Bus, Scheduler, etc. - shared infrastructure All services communicate via gRPC (primary) or HTTP (fallback) through service client interfaces, with service discovery via a service registry. Each service has its own database connection pool and schema. Services share infrastructure (PostgreSQL instance, Redis, Kafka) but are independently deployable and scalable. ```mermaid graph TB subgraph "API Gateway" Gateway[API Gateway
:8080] end subgraph "Core Services" AuthSvc[Auth Service
:8081] IdentitySvc[Identity Service
:8082] AuthzSvc[Authz Service
:8083] AuditSvc[Audit Service
:8084] end subgraph "Feature Services" BlogSvc[Blog Service
:8091] BillingSvc[Billing Service
:8092] AnalyticsSvc[Analytics Service
:8093] end subgraph "Core Kernel" Kernel[Core Kernel
Infrastructure Only] end subgraph "Infrastructure" DB[(PostgreSQL)] Cache[(Redis)] Queue[Kafka/Event Bus] Storage[S3/Blob Storage] Registry[Service Registry] end subgraph "External Services" OIDC[OIDC Provider] Email[Email Service] Sentry[Sentry] end Gateway --> AuthSvc Gateway --> IdentitySvc Gateway --> AuthzSvc Gateway --> BlogSvc Gateway --> BillingSvc AuthSvc --> IdentitySvc AuthSvc --> Registry AuthzSvc --> IdentitySvc AuthzSvc --> Cache AuthzSvc --> AuditSvc BlogSvc --> AuthzSvc BlogSvc --> IdentitySvc BlogSvc --> Registry AuthSvc --> DB IdentitySvc --> DB AuthzSvc --> DB AuditSvc --> DB BlogSvc --> DB BillingSvc --> DB AuthSvc --> Cache AuthzSvc --> Cache BlogSvc --> Cache BillingSvc --> Cache BlogSvc --> Queue BillingSvc --> Queue AnalyticsSvc --> Queue Kernel --> DB Kernel --> Cache Kernel --> Queue Kernel --> Registry AuthSvc --> OIDC IdentitySvc --> Email AuditSvc --> Sentry style Gateway fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff style Kernel fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff style AuthSvc fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff style IdentitySvc fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff style BlogSvc fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff style BillingSvc fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff ``` ## Layered Architecture The platform follows a **hexagonal architecture** with clear separation of concerns across layers. ```mermaid graph TD subgraph "Presentation Layer" HTTP[HTTP/REST API] GraphQL[GraphQL API] CLI[CLI Interface] end subgraph "Application Layer" AuthMiddleware[Auth Middleware] AuthzMiddleware[Authorization Middleware] RateLimit[Rate Limiting] Handlers[Request Handlers] end subgraph "Domain Layer" Services[Domain Services] Entities[Domain Entities] Policies[Business Policies] end subgraph "Infrastructure Layer" Repos[Repositories] CacheAdapter[Cache Adapter] EventBus[Event Bus] Jobs[Scheduler/Jobs] end subgraph "Core Kernel (Infrastructure Only)" DI[DI Container] Config[Config Manager] Logger[Logger] Metrics[Metrics] Health[Health Checks] Tracer[OpenTelemetry Tracer] end HTTP --> AuthMiddleware GraphQL --> AuthMiddleware CLI --> AuthMiddleware AuthMiddleware --> AuthzMiddleware AuthzMiddleware --> RateLimit RateLimit --> Handlers Handlers --> Services Services --> Entities Services --> Policies Services --> Repos Services --> CacheAdapter Services --> EventBus Services --> Jobs Repos --> DB[(Database)] CacheAdapter --> Cache[(Redis)] EventBus --> Queue[(Kafka)] Services --> DI Repos --> DI Handlers --> DI DI --> Config DI --> Logger DI --> Metrics DI --> Health style Core fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff style Services fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff style Repos fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff ``` ## Module System Architecture Modules are the building blocks of the platform. Each module can register services, routes, permissions, and background jobs. ```mermaid graph TB subgraph Lifecycle["Module Lifecycle"] Discover["1. Discover Modules"] Load["2. Load Module"] Validate["3. Validate Dependencies"] Init["4. Initialize Module"] Start["5. Start Module"] end subgraph Registration["Module Registration"] Static["Static Registration
via init()"] Dynamic["Dynamic Loading
via .so files"] end subgraph Components["Module Components"] Routes["HTTP Routes"] Services["Services"] Repos["Repositories"] Perms["Permissions"] Jobs["Background Jobs"] Migrations["Database Migrations"] end Discover --> Load Load --> Static Load --> Dynamic Static --> Validate Dynamic --> Validate Validate --> Init Init --> Routes Init --> Services Init --> Repos Init --> Perms Init --> Jobs Init --> Migrations Routes --> Start Services --> Start Repos --> Start Perms --> Start Jobs --> Start Migrations --> Start classDef lifecycle fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff classDef registration fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff classDef components fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff classDef start fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff class Discover,Load,Validate,Init lifecycle class Static,Dynamic registration class Routes,Services,Repos,Perms,Jobs,Migrations components class Start start ``` ### Module Initialization Sequence ```mermaid sequenceDiagram participant Main participant Loader participant Registry participant Module participant DI participant Router participant DB Main->>Loader: LoadModules() Loader->>Registry: Discover modules Registry-->>Loader: List of modules loop For each module Loader->>Module: Load module Module->>Registry: Register(module) Registry->>Registry: Validate dependencies end Main->>Registry: GetAllModules() Registry-->>Main: Ordered module list Main->>DI: Create container loop For each module Main->>Module: Init() Module->>DI: Provide services Module->>Router: Register routes Module->>DB: Register migrations end Main->>DB: Run migrations Main->>Router: Start HTTP server ``` ## Component Relationships This diagram shows how core components interact with each other and with modules. ```mermaid graph TB subgraph "Core Kernel Components (Infrastructure)" ConfigMgr[Config Manager] LoggerService[Logger Service] DI[DI Container] ModuleLoader[Module Loader] HealthRegistry[Health Registry] MetricsRegistry[Metrics Registry] ErrorBus[Error Bus] Tracer[OpenTelemetry Tracer] ServiceRegistry[Service Registry] end subgraph "Core Services (Separate Microservices)" AuthService[Auth Service
:8081] IdentityService[Identity Service
:8082] AuthzService[Authz Service
:8083] AuditService[Audit Service
:8084] end subgraph "Infrastructure Adapters" EventBus[Event Bus
Kafka] CacheService[Cache Service
Redis] end subgraph "Infrastructure Components" DBClient[Database Client] CacheClient[Cache Client] Scheduler[Scheduler] Notifier[Notifier] end subgraph "External Services" Sentry[Sentry
Error Reporting] end subgraph "Module Components" ModuleRoutes[Module Routes] ModuleServices[Module Services] ModuleRepos[Module Repositories] end DI --> ConfigMgr DI --> LoggerService DI --> ModuleLoader DI --> HealthRegistry DI --> MetricsRegistry DI --> ErrorBus DI --> Tracer DI --> ServiceRegistry ModuleServices -->|gRPC| AuthService ModuleServices -->|gRPC| IdentityService ModuleServices -->|gRPC| AuthzService ModuleServices -->|gRPC| AuditService ModuleServices --> EventBus ModuleServices --> CacheService ModuleServices --> DBClient ModuleRepos --> DBClient AuthService --> DBClient IdentityService --> DBClient AuthzService --> DBClient AuditService --> DBClient AuthService --> ServiceRegistry IdentityService --> ServiceRegistry AuthzService --> ServiceRegistry AuditService --> ServiceRegistry ErrorBus --> LoggerService ErrorBus --> Sentry style DI fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff style AuthService fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff style IdentityService fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff style ModuleServices fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff ``` ## Data Flow ### Request Processing Flow ```mermaid sequenceDiagram participant Client participant Gateway[API Gateway] participant AuthSvc[Auth Service] participant AuthzSvc[Authz Service] participant Service[Feature Service] participant IdentitySvc[Identity Service] participant AuditSvc[Audit Service] participant Repo participant DB participant Cache participant EventBus Client->>Gateway: HTTP Request Gateway->>Gateway: Rate limiting Gateway->>AuthSvc: Validate JWT token (gRPC) AuthSvc->>AuthSvc: Verify token AuthSvc-->>Gateway: Token valid + user info Gateway->>AuthzSvc: Check permissions (gRPC) AuthzSvc->>AuthzSvc: Resolve permissions AuthzSvc-->>Gateway: Authorized Gateway->>Service: Route to service (gRPC/HTTP) Service->>Cache: Check cache Cache-->>Service: Cache miss Service->>Repo: Query data Repo->>DB: Execute query DB-->>Repo: Return data Repo-->>Service: Domain entity Service->>Cache: Store in cache Service->>IdentitySvc: Get user info (gRPC, if needed) IdentitySvc-->>Service: User data Service->>EventBus: Publish event Service->>AuditSvc: Record action (gRPC) Service-->>Gateway: Response data Gateway-->>Client: JSON response ``` ### Module Event Flow ```mermaid graph LR subgraph "Module A" AService[Service A] AHandler[Handler A] end subgraph "Event Bus" Bus[Event Bus] end subgraph "Module B" BService[Service B] BHandler[Handler B] end subgraph "Module C" CService[Service C] end AHandler --> AService AService -->|Publish Event| Bus Bus -->|Subscribe| BService Bus -->|Subscribe| CService BService --> BHandler CService --> CService ``` ## Deployment Architecture ### Development Deployment ```mermaid graph TB subgraph "Developer Machine" IDE[IDE/Editor] Go[Go Runtime] Docker[Docker] end subgraph "Local Services" Gateway[API Gateway
:8080] AuthSvc[Auth Service
:8081] IdentitySvc[Identity Service
:8082] AuthzSvc[Authz Service
:8083] AuditSvc[Audit Service
:8084] BlogSvc[Blog Service
:8091] DB[(PostgreSQL
:5432)] Redis[(Redis
:6379)] Kafka[Kafka
:9092] Consul[Consul
:8500] end IDE --> Go Go --> Gateway Go --> AuthSvc Go --> IdentitySvc Go --> AuthzSvc Go --> AuditSvc Go --> BlogSvc Gateway --> AuthSvc Gateway --> IdentitySvc Gateway --> BlogSvc AuthSvc --> DB IdentitySvc --> DB AuthzSvc --> DB AuditSvc --> DB BlogSvc --> DB AuthSvc --> Redis AuthzSvc --> Redis BlogSvc --> Redis BlogSvc --> Kafka AuthSvc --> Consul IdentitySvc --> Consul AuthzSvc --> Consul AuditSvc --> Consul BlogSvc --> Consul Docker --> DB Docker --> Redis Docker --> Kafka Docker --> Consul style Gateway fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff style AuthSvc fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff style IdentitySvc fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff ``` ### Production Deployment ```mermaid graph TB subgraph "Load Balancer" LB[Load Balancer
HTTPS] end subgraph "Service Instances" Gateway1[API Gateway 1] Gateway2[API Gateway 2] AuthSvc1[Auth Service 1] AuthSvc2[Auth Service 2] IdentitySvc1[Identity Service 1] IdentitySvc2[Identity Service 2] BlogSvc1[Blog Service 1] BlogSvc2[Blog Service 2] end subgraph "Database Cluster" Primary[(PostgreSQL
Primary)] Replica[(PostgreSQL
Replica)] end subgraph "Cache Cluster" Redis1[(Redis
Master)] Redis2[(Redis
Replica)] end subgraph "Message Queue" Kafka1[Kafka Broker 1] Kafka2[Kafka Broker 2] Kafka3[Kafka Broker 3] end subgraph "Observability" Prometheus[Prometheus] Grafana[Grafana] Jaeger[Jaeger] Loki[Loki] end subgraph "External Services" Sentry[Sentry] S3[S3 Storage] end LB --> Gateway1 LB --> Gateway2 Gateway1 --> AuthSvc1 Gateway1 --> AuthSvc2 Gateway1 --> IdentitySvc1 Gateway1 --> IdentitySvc2 Gateway1 --> BlogSvc1 Gateway1 --> BlogSvc2 Gateway2 --> AuthSvc1 Gateway2 --> AuthSvc2 Gateway2 --> IdentitySvc1 Gateway2 --> IdentitySvc2 Gateway2 --> BlogSvc1 Gateway2 --> BlogSvc2 AuthSvc1 --> Primary AuthSvc2 --> Primary IdentitySvc1 --> Primary IdentitySvc2 --> Primary BlogSvc1 --> Primary BlogSvc2 --> Primary AuthSvc1 --> Replica AuthSvc2 --> Replica IdentitySvc1 --> Replica IdentitySvc2 --> Replica BlogSvc1 --> Replica BlogSvc2 --> Replica AuthSvc1 --> Redis1 AuthSvc2 --> Redis1 IdentitySvc1 --> Redis1 IdentitySvc2 --> Redis1 BlogSvc1 --> Redis1 BlogSvc2 --> Redis1 BlogSvc1 --> Kafka1 BlogSvc2 --> Kafka2 AuthSvc1 --> Prometheus AuthSvc2 --> Prometheus IdentitySvc1 --> Prometheus IdentitySvc2 --> Prometheus BlogSvc1 --> Prometheus BlogSvc2 --> Prometheus Prometheus --> Grafana AuthSvc1 --> Jaeger AuthSvc2 --> Jaeger IdentitySvc1 --> Jaeger IdentitySvc2 --> Jaeger BlogSvc1 --> Jaeger BlogSvc2 --> Jaeger AuthSvc1 --> Loki AuthSvc2 --> Loki IdentitySvc1 --> Loki IdentitySvc2 --> Loki BlogSvc1 --> Loki BlogSvc2 --> Loki AuthSvc1 --> Sentry AuthSvc2 --> Sentry IdentitySvc1 --> Sentry IdentitySvc2 --> Sentry BlogSvc1 --> Sentry BlogSvc2 --> Sentry BlogSvc1 --> S3 BlogSvc2 --> S3 style LB fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff style Gateway1 fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff style Gateway2 fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff style Primary fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff style Redis1 fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff ``` ## Core Kernel Components The core kernel provides **infrastructure only** - no business logic. It is the foundation that all services depend on. Business logic resides in separate services (Auth, Identity, Authz, Audit). ### Component Responsibilities ```mermaid mindmap root((Core Kernel)) Configuration Load configs Environment vars Secret management Dependency Injection Service registration Lifecycle management Module wiring Logging Structured logs Request correlation Log levels Observability Metrics Tracing Health checks Service Discovery Service registry Service registration Health checking Module System Module discovery Module loading Dependency resolution ``` ## Module Integration Points Modules integrate with the core through well-defined interfaces: ```mermaid graph TB subgraph "Core Kernel Interfaces" IConfig[ConfigProvider] ILogger[Logger] ITracer[Tracer] IMetrics[Metrics] IHealth[Health] end subgraph "Service Client Interfaces" IAuthClient[AuthServiceClient] IIdentityClient[IdentityServiceClient] IAuthzClient[AuthzServiceClient] IAuditClient[AuditServiceClient] end subgraph "Infrastructure Interfaces" IEventBus[EventBus] ICache[Cache] IBlobStore[BlobStore] IScheduler[Scheduler] INotifier[Notifier] end subgraph "Feature Service Implementation" Module[Feature Service] ModuleServices[Service Layer] ModuleRoutes[HTTP/gRPC Routes] end Module --> IConfig Module --> ILogger Module --> ITracer Module --> IMetrics Module --> IHealth ModuleServices -->|gRPC| IAuthClient ModuleServices -->|gRPC| IIdentityClient ModuleServices -->|gRPC| IAuthzClient ModuleServices -->|gRPC| IAuditClient ModuleServices --> IEventBus ModuleServices --> ICache ModuleServices --> IBlobStore ModuleServices --> IScheduler ModuleServices --> INotifier ModuleRoutes -->|gRPC| IAuthzClient style IConfig fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff style Module fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff ``` ## Microservices Architecture The platform is designed as **microservices from day one**, with each module being an independent service. ### Service Architecture ```mermaid graph TB subgraph "API Gateway" Gateway[API Gateway
Routing & Auth] end subgraph "Core Services" AuthSvc[Auth Service
:8081] IdentitySvc[Identity Service
:8082] AuthzSvc[Authz Service
:8083] AuditSvc[Audit Service
:8084] end subgraph "Feature Services" BlogSvc[Blog Service
:8091] BillingSvc[Billing Service
:8092] AnalyticsSvc[Analytics Service
:8093] end subgraph "Infrastructure" DB[(PostgreSQL)] Cache[(Redis)] Queue[Kafka] Registry[Service Registry] end Gateway --> AuthSvc Gateway --> IdentitySvc Gateway --> BlogSvc Gateway --> BillingSvc AuthSvc --> IdentitySvc AuthSvc --> Registry BlogSvc --> AuthzSvc BlogSvc --> IdentitySvc BlogSvc --> Registry BillingSvc --> IdentitySvc BillingSvc --> Registry AuthSvc --> DB IdentitySvc --> DB BlogSvc --> DB BillingSvc --> DB AuthSvc --> Cache BlogSvc --> Cache BillingSvc --> Cache BlogSvc --> Queue BillingSvc --> Queue AnalyticsSvc --> Queue style Gateway fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff style Registry fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff ``` ### Service Communication All inter-service communication uses service client interfaces: ```mermaid graph TB subgraph "Service Client Interface" Interface[Service Interface
pkg/services/] end subgraph "Implementations" GRPC[gRPC Client
Primary] HTTP[HTTP Client
Fallback] end subgraph "Service Registry" Registry[Service Registry
Discovery & Resolution] end Interface --> GRPC Interface --> HTTP Registry --> GRPC Registry --> HTTP style Interface fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff style Registry fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff ``` ### Service Communication Patterns The platform uses three communication patterns: 1. **Synchronous Service Calls** (via Service Clients): - gRPC calls (primary) - type-safe, efficient - HTTP/REST calls (fallback) - for external integration - All calls go through service client interfaces - Service discovery via registry 2. **Asynchronous Events** (via Event Bus): - Distributed via Kafka - Preferred for cross-service communication - Event-driven architecture for loose coupling 3. **Shared Infrastructure** (For state): - Redis for cache and distributed state - PostgreSQL for persistent data - Kafka for events ### Service Registry The service registry enables service discovery and resolution: ```mermaid graph TB subgraph "Service Registry" Registry[Service Registry Interface] Consul[Consul Registry] K8s[K8s Service Discovery] Etcd[etcd Registry] end subgraph "Services" AuthSvc[Auth Service] IdentitySvc[Identity Service] BlogSvc[Blog Service] end Registry --> Consul Registry --> K8s Registry --> Etcd Consul --> AuthSvc K8s --> IdentitySvc Etcd --> BlogSvc style Registry fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff ``` ### Scaling Strategy #### Independent Service Scaling - Scale individual services based on load - Independent resource allocation - Independent deployment - Better resource utilization - Team autonomy #### Development Mode - For local development, services can run in the same repository/monorepo - Services still communicate via gRPC/HTTP through service clients (no direct in-process calls) - Each service has its own process and entry point - Docker Compose for easy local setup with all services - Maintains microservices architecture even in development - Services can be started individually for debugging ## Next Steps - [Module Architecture](./architecture-modules.md) - Detailed module architecture and design - [Module Requirements](./module-requirements.md) - Requirements for each module - [Component Relationships](./component-relationships.md) - Detailed component interactions - [ADRs](../adr/README.md) - Architecture Decision Records - [ADR-0029: Microservices Architecture](../adr/0029-microservices-architecture.md) - Microservices strategy - [ADR-0030: Service Communication](../adr/0030-service-communication-strategy.md) - Communication patterns