# System Architecture
This document provides a comprehensive overview of the Go Platform architecture, including system components, their relationships, and how modules integrate with the core platform.
## Table of Contents
- [High-Level Architecture](#high-level-architecture)
- [Layered Architecture](#layered-architecture)
- [Module System Architecture](#module-system-architecture)
- [Component Relationships](#component-relationships)
- [Data Flow](#data-flow)
- [Deployment Architecture](#deployment-architecture)
## High-Level Architecture
The Go Platform follows a **microservices architecture** where each service is independently deployable from day one:
- **Core Kernel**: Infrastructure only (config, logger, DI, health, metrics, observability) - no business logic
- **Core Services**: Auth Service, Identity Service, Authz Service, Audit Service - separate microservices
- **API Gateway**: Single entry point for all external traffic, handles routing and authentication
- **Feature Services**: Blog, Billing, Analytics, etc. - independent services
- **Infrastructure Adapters**: Cache, Event Bus, Scheduler, etc. - shared infrastructure
All services communicate via gRPC (primary) or HTTP (fallback) through service client interfaces, with service discovery via a service registry. Each service has its own database connection pool and schema. Services share infrastructure (PostgreSQL instance, Redis, Kafka) but are independently deployable and scalable.
```mermaid
graph TB
subgraph "API Gateway"
Gateway[API Gateway
:8080]
end
subgraph "Core Services"
AuthSvc[Auth Service
:8081]
IdentitySvc[Identity Service
:8082]
AuthzSvc[Authz Service
:8083]
AuditSvc[Audit Service
:8084]
end
subgraph "Feature Services"
BlogSvc[Blog Service
:8091]
BillingSvc[Billing Service
:8092]
AnalyticsSvc[Analytics Service
:8093]
end
subgraph "Core Kernel"
Kernel[Core Kernel
Infrastructure Only]
end
subgraph "Infrastructure"
DB[(PostgreSQL)]
Cache[(Redis)]
Queue[Kafka/Event Bus]
Storage[S3/Blob Storage]
Registry[Service Registry]
end
subgraph "External Services"
OIDC[OIDC Provider]
Email[Email Service]
Sentry[Sentry]
end
Gateway --> AuthSvc
Gateway --> IdentitySvc
Gateway --> AuthzSvc
Gateway --> BlogSvc
Gateway --> BillingSvc
AuthSvc --> IdentitySvc
AuthSvc --> Registry
AuthzSvc --> IdentitySvc
AuthzSvc --> Cache
AuthzSvc --> AuditSvc
BlogSvc --> AuthzSvc
BlogSvc --> IdentitySvc
BlogSvc --> Registry
AuthSvc --> DB
IdentitySvc --> DB
AuthzSvc --> DB
AuditSvc --> DB
BlogSvc --> DB
BillingSvc --> DB
AuthSvc --> Cache
AuthzSvc --> Cache
BlogSvc --> Cache
BillingSvc --> Cache
BlogSvc --> Queue
BillingSvc --> Queue
AnalyticsSvc --> Queue
Kernel --> DB
Kernel --> Cache
Kernel --> Queue
Kernel --> Registry
AuthSvc --> OIDC
IdentitySvc --> Email
AuditSvc --> Sentry
style Gateway fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style Kernel fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style AuthSvc fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
style IdentitySvc fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
style BlogSvc fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
style BillingSvc fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
```
## Layered Architecture
The platform follows a **hexagonal architecture** with clear separation of concerns across layers.
```mermaid
graph TD
subgraph "Presentation Layer"
HTTP[HTTP/REST API]
GraphQL[GraphQL API]
CLI[CLI Interface]
end
subgraph "Application Layer"
AuthMiddleware[Auth Middleware]
AuthzMiddleware[Authorization Middleware]
RateLimit[Rate Limiting]
Handlers[Request Handlers]
end
subgraph "Domain Layer"
Services[Domain Services]
Entities[Domain Entities]
Policies[Business Policies]
end
subgraph "Infrastructure Layer"
Repos[Repositories]
CacheAdapter[Cache Adapter]
EventBus[Event Bus]
Jobs[Scheduler/Jobs]
end
subgraph "Core Kernel (Infrastructure Only)"
DI[DI Container]
Config[Config Manager]
Logger[Logger]
Metrics[Metrics]
Health[Health Checks]
Tracer[OpenTelemetry Tracer]
end
HTTP --> AuthMiddleware
GraphQL --> AuthMiddleware
CLI --> AuthMiddleware
AuthMiddleware --> AuthzMiddleware
AuthzMiddleware --> RateLimit
RateLimit --> Handlers
Handlers --> Services
Services --> Entities
Services --> Policies
Services --> Repos
Services --> CacheAdapter
Services --> EventBus
Services --> Jobs
Repos --> DB[(Database)]
CacheAdapter --> Cache[(Redis)]
EventBus --> Queue[(Kafka)]
Services --> DI
Repos --> DI
Handlers --> DI
DI --> Config
DI --> Logger
DI --> Metrics
DI --> Health
style Core fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style Services fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style Repos fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
```
## Module System Architecture
Modules are the building blocks of the platform. Each module can register services, routes, permissions, and background jobs.
```mermaid
graph TB
subgraph Lifecycle["Module Lifecycle"]
Discover["1. Discover Modules"]
Load["2. Load Module"]
Validate["3. Validate Dependencies"]
Init["4. Initialize Module"]
Start["5. Start Module"]
end
subgraph Registration["Module Registration"]
Static["Static Registration
via init()"]
Dynamic["Dynamic Loading
via .so files"]
end
subgraph Components["Module Components"]
Routes["HTTP Routes"]
Services["Services"]
Repos["Repositories"]
Perms["Permissions"]
Jobs["Background Jobs"]
Migrations["Database Migrations"]
end
Discover --> Load
Load --> Static
Load --> Dynamic
Static --> Validate
Dynamic --> Validate
Validate --> Init
Init --> Routes
Init --> Services
Init --> Repos
Init --> Perms
Init --> Jobs
Init --> Migrations
Routes --> Start
Services --> Start
Repos --> Start
Perms --> Start
Jobs --> Start
Migrations --> Start
classDef lifecycle fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
classDef registration fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
classDef components fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
classDef start fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
class Discover,Load,Validate,Init lifecycle
class Static,Dynamic registration
class Routes,Services,Repos,Perms,Jobs,Migrations components
class Start start
```
### Module Initialization Sequence
```mermaid
sequenceDiagram
participant Main
participant Loader
participant Registry
participant Module
participant DI
participant Router
participant DB
Main->>Loader: LoadModules()
Loader->>Registry: Discover modules
Registry-->>Loader: List of modules
loop For each module
Loader->>Module: Load module
Module->>Registry: Register(module)
Registry->>Registry: Validate dependencies
end
Main->>Registry: GetAllModules()
Registry-->>Main: Ordered module list
Main->>DI: Create container
loop For each module
Main->>Module: Init()
Module->>DI: Provide services
Module->>Router: Register routes
Module->>DB: Register migrations
end
Main->>DB: Run migrations
Main->>Router: Start HTTP server
```
## Component Relationships
This diagram shows how core components interact with each other and with modules.
```mermaid
graph TB
subgraph "Core Kernel Components (Infrastructure)"
ConfigMgr[Config Manager]
LoggerService[Logger Service]
DI[DI Container]
ModuleLoader[Module Loader]
HealthRegistry[Health Registry]
MetricsRegistry[Metrics Registry]
ErrorBus[Error Bus]
Tracer[OpenTelemetry Tracer]
ServiceRegistry[Service Registry]
end
subgraph "Core Services (Separate Microservices)"
AuthService[Auth Service
:8081]
IdentityService[Identity Service
:8082]
AuthzService[Authz Service
:8083]
AuditService[Audit Service
:8084]
end
subgraph "Infrastructure Adapters"
EventBus[Event Bus
Kafka]
CacheService[Cache Service
Redis]
end
subgraph "Infrastructure Components"
DBClient[Database Client]
CacheClient[Cache Client]
Scheduler[Scheduler]
Notifier[Notifier]
end
subgraph "External Services"
Sentry[Sentry
Error Reporting]
end
subgraph "Module Components"
ModuleRoutes[Module Routes]
ModuleServices[Module Services]
ModuleRepos[Module Repositories]
end
DI --> ConfigMgr
DI --> LoggerService
DI --> ModuleLoader
DI --> HealthRegistry
DI --> MetricsRegistry
DI --> ErrorBus
DI --> Tracer
DI --> ServiceRegistry
ModuleServices -->|gRPC| AuthService
ModuleServices -->|gRPC| IdentityService
ModuleServices -->|gRPC| AuthzService
ModuleServices -->|gRPC| AuditService
ModuleServices --> EventBus
ModuleServices --> CacheService
ModuleServices --> DBClient
ModuleRepos --> DBClient
AuthService --> DBClient
IdentityService --> DBClient
AuthzService --> DBClient
AuditService --> DBClient
AuthService --> ServiceRegistry
IdentityService --> ServiceRegistry
AuthzService --> ServiceRegistry
AuditService --> ServiceRegistry
ErrorBus --> LoggerService
ErrorBus --> Sentry
style DI fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style AuthService fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
style IdentityService fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
style ModuleServices fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
```
## Data Flow
### Request Processing Flow
```mermaid
sequenceDiagram
participant Client
participant Gateway[API Gateway]
participant AuthSvc[Auth Service]
participant AuthzSvc[Authz Service]
participant Service[Feature Service]
participant IdentitySvc[Identity Service]
participant AuditSvc[Audit Service]
participant Repo
participant DB
participant Cache
participant EventBus
Client->>Gateway: HTTP Request
Gateway->>Gateway: Rate limiting
Gateway->>AuthSvc: Validate JWT token (gRPC)
AuthSvc->>AuthSvc: Verify token
AuthSvc-->>Gateway: Token valid + user info
Gateway->>AuthzSvc: Check permissions (gRPC)
AuthzSvc->>AuthzSvc: Resolve permissions
AuthzSvc-->>Gateway: Authorized
Gateway->>Service: Route to service (gRPC/HTTP)
Service->>Cache: Check cache
Cache-->>Service: Cache miss
Service->>Repo: Query data
Repo->>DB: Execute query
DB-->>Repo: Return data
Repo-->>Service: Domain entity
Service->>Cache: Store in cache
Service->>IdentitySvc: Get user info (gRPC, if needed)
IdentitySvc-->>Service: User data
Service->>EventBus: Publish event
Service->>AuditSvc: Record action (gRPC)
Service-->>Gateway: Response data
Gateway-->>Client: JSON response
```
### Module Event Flow
```mermaid
graph LR
subgraph "Module A"
AService[Service A]
AHandler[Handler A]
end
subgraph "Event Bus"
Bus[Event Bus]
end
subgraph "Module B"
BService[Service B]
BHandler[Handler B]
end
subgraph "Module C"
CService[Service C]
end
AHandler --> AService
AService -->|Publish Event| Bus
Bus -->|Subscribe| BService
Bus -->|Subscribe| CService
BService --> BHandler
CService --> CService
```
## Deployment Architecture
### Development Deployment
```mermaid
graph TB
subgraph "Developer Machine"
IDE[IDE/Editor]
Go[Go Runtime]
Docker[Docker]
end
subgraph "Local Services"
Gateway[API Gateway
:8080]
AuthSvc[Auth Service
:8081]
IdentitySvc[Identity Service
:8082]
AuthzSvc[Authz Service
:8083]
AuditSvc[Audit Service
:8084]
BlogSvc[Blog Service
:8091]
DB[(PostgreSQL
:5432)]
Redis[(Redis
:6379)]
Kafka[Kafka
:9092]
Consul[Consul
:8500]
end
IDE --> Go
Go --> Gateway
Go --> AuthSvc
Go --> IdentitySvc
Go --> AuthzSvc
Go --> AuditSvc
Go --> BlogSvc
Gateway --> AuthSvc
Gateway --> IdentitySvc
Gateway --> BlogSvc
AuthSvc --> DB
IdentitySvc --> DB
AuthzSvc --> DB
AuditSvc --> DB
BlogSvc --> DB
AuthSvc --> Redis
AuthzSvc --> Redis
BlogSvc --> Redis
BlogSvc --> Kafka
AuthSvc --> Consul
IdentitySvc --> Consul
AuthzSvc --> Consul
AuditSvc --> Consul
BlogSvc --> Consul
Docker --> DB
Docker --> Redis
Docker --> Kafka
Docker --> Consul
style Gateway fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
style AuthSvc fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
style IdentitySvc fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
```
### Production Deployment
```mermaid
graph TB
subgraph "Load Balancer"
LB[Load Balancer
HTTPS]
end
subgraph "Service Instances"
Gateway1[API Gateway 1]
Gateway2[API Gateway 2]
AuthSvc1[Auth Service 1]
AuthSvc2[Auth Service 2]
IdentitySvc1[Identity Service 1]
IdentitySvc2[Identity Service 2]
BlogSvc1[Blog Service 1]
BlogSvc2[Blog Service 2]
end
subgraph "Database Cluster"
Primary[(PostgreSQL
Primary)]
Replica[(PostgreSQL
Replica)]
end
subgraph "Cache Cluster"
Redis1[(Redis
Master)]
Redis2[(Redis
Replica)]
end
subgraph "Message Queue"
Kafka1[Kafka Broker 1]
Kafka2[Kafka Broker 2]
Kafka3[Kafka Broker 3]
end
subgraph "Observability"
Prometheus[Prometheus]
Grafana[Grafana]
Jaeger[Jaeger]
Loki[Loki]
end
subgraph "External Services"
Sentry[Sentry]
S3[S3 Storage]
end
LB --> Gateway1
LB --> Gateway2
Gateway1 --> AuthSvc1
Gateway1 --> AuthSvc2
Gateway1 --> IdentitySvc1
Gateway1 --> IdentitySvc2
Gateway1 --> BlogSvc1
Gateway1 --> BlogSvc2
Gateway2 --> AuthSvc1
Gateway2 --> AuthSvc2
Gateway2 --> IdentitySvc1
Gateway2 --> IdentitySvc2
Gateway2 --> BlogSvc1
Gateway2 --> BlogSvc2
AuthSvc1 --> Primary
AuthSvc2 --> Primary
IdentitySvc1 --> Primary
IdentitySvc2 --> Primary
BlogSvc1 --> Primary
BlogSvc2 --> Primary
AuthSvc1 --> Replica
AuthSvc2 --> Replica
IdentitySvc1 --> Replica
IdentitySvc2 --> Replica
BlogSvc1 --> Replica
BlogSvc2 --> Replica
AuthSvc1 --> Redis1
AuthSvc2 --> Redis1
IdentitySvc1 --> Redis1
IdentitySvc2 --> Redis1
BlogSvc1 --> Redis1
BlogSvc2 --> Redis1
BlogSvc1 --> Kafka1
BlogSvc2 --> Kafka2
AuthSvc1 --> Prometheus
AuthSvc2 --> Prometheus
IdentitySvc1 --> Prometheus
IdentitySvc2 --> Prometheus
BlogSvc1 --> Prometheus
BlogSvc2 --> Prometheus
Prometheus --> Grafana
AuthSvc1 --> Jaeger
AuthSvc2 --> Jaeger
IdentitySvc1 --> Jaeger
IdentitySvc2 --> Jaeger
BlogSvc1 --> Jaeger
BlogSvc2 --> Jaeger
AuthSvc1 --> Loki
AuthSvc2 --> Loki
IdentitySvc1 --> Loki
IdentitySvc2 --> Loki
BlogSvc1 --> Loki
BlogSvc2 --> Loki
AuthSvc1 --> Sentry
AuthSvc2 --> Sentry
IdentitySvc1 --> Sentry
IdentitySvc2 --> Sentry
BlogSvc1 --> Sentry
BlogSvc2 --> Sentry
BlogSvc1 --> S3
BlogSvc2 --> S3
style LB fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style Gateway1 fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
style Gateway2 fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
style Primary fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style Redis1 fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
```
## Core Kernel Components
The core kernel provides **infrastructure only** - no business logic. It is the foundation that all services depend on. Business logic resides in separate services (Auth, Identity, Authz, Audit).
### Component Responsibilities
```mermaid
mindmap
root((Core Kernel))
Configuration
Load configs
Environment vars
Secret management
Dependency Injection
Service registration
Lifecycle management
Module wiring
Logging
Structured logs
Request correlation
Log levels
Observability
Metrics
Tracing
Health checks
Service Discovery
Service registry
Service registration
Health checking
Module System
Module discovery
Module loading
Dependency resolution
```
## Module Integration Points
Modules integrate with the core through well-defined interfaces:
```mermaid
graph TB
subgraph "Core Kernel Interfaces"
IConfig[ConfigProvider]
ILogger[Logger]
ITracer[Tracer]
IMetrics[Metrics]
IHealth[Health]
end
subgraph "Service Client Interfaces"
IAuthClient[AuthServiceClient]
IIdentityClient[IdentityServiceClient]
IAuthzClient[AuthzServiceClient]
IAuditClient[AuditServiceClient]
end
subgraph "Infrastructure Interfaces"
IEventBus[EventBus]
ICache[Cache]
IBlobStore[BlobStore]
IScheduler[Scheduler]
INotifier[Notifier]
end
subgraph "Feature Service Implementation"
Module[Feature Service]
ModuleServices[Service Layer]
ModuleRoutes[HTTP/gRPC Routes]
end
Module --> IConfig
Module --> ILogger
Module --> ITracer
Module --> IMetrics
Module --> IHealth
ModuleServices -->|gRPC| IAuthClient
ModuleServices -->|gRPC| IIdentityClient
ModuleServices -->|gRPC| IAuthzClient
ModuleServices -->|gRPC| IAuditClient
ModuleServices --> IEventBus
ModuleServices --> ICache
ModuleServices --> IBlobStore
ModuleServices --> IScheduler
ModuleServices --> INotifier
ModuleRoutes -->|gRPC| IAuthzClient
style IConfig fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
style Module fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
```
## Microservices Architecture
The platform is designed as **microservices from day one**, with each module being an independent service.
### Service Architecture
```mermaid
graph TB
subgraph "API Gateway"
Gateway[API Gateway
Routing & Auth]
end
subgraph "Core Services"
AuthSvc[Auth Service
:8081]
IdentitySvc[Identity Service
:8082]
AuthzSvc[Authz Service
:8083]
AuditSvc[Audit Service
:8084]
end
subgraph "Feature Services"
BlogSvc[Blog Service
:8091]
BillingSvc[Billing Service
:8092]
AnalyticsSvc[Analytics Service
:8093]
end
subgraph "Infrastructure"
DB[(PostgreSQL)]
Cache[(Redis)]
Queue[Kafka]
Registry[Service Registry]
end
Gateway --> AuthSvc
Gateway --> IdentitySvc
Gateway --> BlogSvc
Gateway --> BillingSvc
AuthSvc --> IdentitySvc
AuthSvc --> Registry
BlogSvc --> AuthzSvc
BlogSvc --> IdentitySvc
BlogSvc --> Registry
BillingSvc --> IdentitySvc
BillingSvc --> Registry
AuthSvc --> DB
IdentitySvc --> DB
BlogSvc --> DB
BillingSvc --> DB
AuthSvc --> Cache
BlogSvc --> Cache
BillingSvc --> Cache
BlogSvc --> Queue
BillingSvc --> Queue
AnalyticsSvc --> Queue
style Gateway fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style Registry fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
```
### Service Communication
All inter-service communication uses service client interfaces:
```mermaid
graph TB
subgraph "Service Client Interface"
Interface[Service Interface
pkg/services/]
end
subgraph "Implementations"
GRPC[gRPC Client
Primary]
HTTP[HTTP Client
Fallback]
end
subgraph "Service Registry"
Registry[Service Registry
Discovery & Resolution]
end
Interface --> GRPC
Interface --> HTTP
Registry --> GRPC
Registry --> HTTP
style Interface fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style Registry fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
```
### Service Communication Patterns
The platform uses three communication patterns:
1. **Synchronous Service Calls** (via Service Clients):
- gRPC calls (primary) - type-safe, efficient
- HTTP/REST calls (fallback) - for external integration
- All calls go through service client interfaces
- Service discovery via registry
2. **Asynchronous Events** (via Event Bus):
- Distributed via Kafka
- Preferred for cross-service communication
- Event-driven architecture for loose coupling
3. **Shared Infrastructure** (For state):
- Redis for cache and distributed state
- PostgreSQL for persistent data
- Kafka for events
### Service Registry
The service registry enables service discovery and resolution:
```mermaid
graph TB
subgraph "Service Registry"
Registry[Service Registry Interface]
Consul[Consul Registry]
K8s[K8s Service Discovery]
Etcd[etcd Registry]
end
subgraph "Services"
AuthSvc[Auth Service]
IdentitySvc[Identity Service]
BlogSvc[Blog Service]
end
Registry --> Consul
Registry --> K8s
Registry --> Etcd
Consul --> AuthSvc
K8s --> IdentitySvc
Etcd --> BlogSvc
style Registry fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
```
### Scaling Strategy
#### Independent Service Scaling
- Scale individual services based on load
- Independent resource allocation
- Independent deployment
- Better resource utilization
- Team autonomy
#### Development Mode
- For local development, services can run in the same repository/monorepo
- Services still communicate via gRPC/HTTP through service clients (no direct in-process calls)
- Each service has its own process and entry point
- Docker Compose for easy local setup with all services
- Maintains microservices architecture even in development
- Services can be started individually for debugging
## Next Steps
- [Module Architecture](./architecture-modules.md) - Detailed module architecture and design
- [Module Requirements](./module-requirements.md) - Requirements for each module
- [Component Relationships](./component-relationships.md) - Detailed component interactions
- [ADRs](../adr/README.md) - Architecture Decision Records
- [ADR-0029: Microservices Architecture](../adr/0029-microservices-architecture.md) - Microservices strategy
- [ADR-0030: Service Communication](../adr/0030-service-communication-strategy.md) - Communication patterns