toolz/goplt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: Align documentation with true microservices architecture

Browse Source 
Transform all documentation from modular monolith to true microservices
architecture where core services are independently deployable.

Key Changes:
- Core Kernel: Infrastructure only (no business logic)
- Core Services: Auth, Identity, Authz, Audit as separate microservices
  - Each service has own entry point (cmd/{service}/)
  - Each service has own gRPC server and database schema
  - Services register with Consul for service discovery
- API Gateway: Moved from Epic 8 to Epic 1 as core infrastructure
  - Single entry point for all external traffic
  - Handles routing, JWT validation, rate limiting, CORS
- Service Discovery: Consul as primary mechanism (ADR-0033)
- Database Pattern: Per-service connections with schema isolation

Documentation Updates:
- Updated all 9 architecture documents
- Updated 4 ADRs and created 2 new ADRs (API Gateway, Service Discovery)
- Rewrote Epic 1: Core Kernel & Infrastructure (infrastructure only)
- Rewrote Epic 2: Core Services (Auth, Identity, Authz, Audit as services)
- Updated Epic 3-8 stories for service architecture
- Updated plan.md, playbook.md, requirements.md, index.md
- Updated all epic READMEs and story files

New ADRs:
- ADR-0032: API Gateway Strategy
- ADR-0033: Service Discovery Implementation (Consul)

New Stories:
- Epic 1.7: Service Client Interfaces
- Epic 1.8: API Gateway Implementation
This commit is contained in:
0x1d
2025-11-06 08:47:27 +01:00
parent cab7cadf9e
commit 38a251968c
47 changed files with 3190 additions and 1613 deletions
Download Patch File Download Diff File Expand all files Collapse all files

414
docs/content/architecture/architecture.md
View File

@@ -13,20 +13,36 @@ This document provides a comprehensive overview of the Go Platform architecture,
## High-Level Architecture
The Go Platform follows a **microservices architecture** where each module is an independent service:
- **Core Services**: Authentication, Identity, Authorization, Audit, etc.
- **Feature Services**: Blog, Billing, Analytics, etc. (modules)
- **Infrastructure Services**: Cache, Event Bus, Scheduler, etc.
The Go Platform follows a **microservices architecture** where each service is independently deployable from day one:
- **Core Kernel**: Infrastructure only (config, logger, DI, health, metrics, observability) - no business logic
- **Core Services**: Auth Service, Identity Service, Authz Service, Audit Service - separate microservices
- **API Gateway**: Single entry point for all external traffic, handles routing and authentication
- **Feature Services**: Blog, Billing, Analytics, etc. - independent services
- **Infrastructure Adapters**: Cache, Event Bus, Scheduler, etc. - shared infrastructure
All services communicate via gRPC (primary) or HTTP (fallback), with service discovery via a service registry. Services share infrastructure (PostgreSQL, Redis, Kafka) but are independently deployable and scalable.
All services communicate via gRPC (primary) or HTTP (fallback) through service client interfaces, with service discovery via a service registry. Each service has its own database connection pool and schema. Services share infrastructure (PostgreSQL instance, Redis, Kafka) but are independently deployable and scalable.
```mermaid
graph TB
subgraph "Go Platform"
Core[Core Kernel]
Module1[Module 1<br/>Blog]
Module2[Module 2<br/>Billing]
Module3[Module N<br/>Custom]
subgraph "API Gateway"
Gateway[API Gateway<br/>:8080]
end
subgraph "Core Services"
AuthSvc[Auth Service<br/>:8081]
IdentitySvc[Identity Service<br/>:8082]
AuthzSvc[Authz Service<br/>:8083]
AuditSvc[Audit Service<br/>:8084]
end
subgraph "Feature Services"
BlogSvc[Blog Service<br/>:8091]
BillingSvc[Billing Service<br/>:8092]
AnalyticsSvc[Analytics Service<br/>:8093]
end
subgraph "Core Kernel"
Kernel[Core Kernel<br/>Infrastructure Only]
end
subgraph "Infrastructure"
@@ -34,6 +50,7 @@ graph TB
Cache[(Redis)]
Queue[Kafka/Event Bus]
Storage[S3/Blob Storage]
Registry[Service Registry]
end
subgraph "External Services"
@@ -42,29 +59,52 @@ graph TB
Sentry[Sentry]
end
Core --> DB
Core --> Cache
Core --> Queue
Core --> Storage
Core --> OIDC
Core --> Email
Core --> Sentry
Gateway --> AuthSvc
Gateway --> IdentitySvc
Gateway --> AuthzSvc
Gateway --> BlogSvc
Gateway --> BillingSvc
Module1 --> Core
Module2 --> Core
Module3 --> Core
AuthSvc --> IdentitySvc
AuthSvc --> Registry
AuthzSvc --> IdentitySvc
AuthzSvc --> Cache
AuthzSvc --> AuditSvc
BlogSvc --> AuthzSvc
BlogSvc --> IdentitySvc
BlogSvc --> Registry
Module1 --> DB
Module2 --> DB
Module3 --> DB
AuthSvc --> DB
IdentitySvc --> DB
AuthzSvc --> DB
AuditSvc --> DB
BlogSvc --> DB
BillingSvc --> DB
Module1 --> Queue
Module2 --> Queue
AuthSvc --> Cache
AuthzSvc --> Cache
BlogSvc --> Cache
BillingSvc --> Cache
style Core fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style Module1 fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
style Module2 fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
style Module3 fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
BlogSvc --> Queue
BillingSvc --> Queue
AnalyticsSvc --> Queue
Kernel --> DB
Kernel --> Cache
Kernel --> Queue
Kernel --> Registry
AuthSvc --> OIDC
IdentitySvc --> Email
AuditSvc --> Sentry
style Gateway fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style Kernel fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style AuthSvc fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
style IdentitySvc fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
style BlogSvc fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
style BillingSvc fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
```
## Layered Architecture
@@ -99,12 +139,13 @@ graph TD
Jobs[Scheduler/Jobs]
end
subgraph "Core Kernel"
subgraph "Core Kernel (Infrastructure Only)"
DI[DI Container]
Config[Config Manager]
Logger[Logger]
Metrics[Metrics]
Health[Health Checks]
Tracer[OpenTelemetry Tracer]
end
HTTP --> AuthMiddleware
@@ -244,7 +285,7 @@ This diagram shows how core components interact with each other and with modules
```mermaid
graph TB
subgraph "Core Kernel Components"
subgraph "Core Kernel Components (Infrastructure)"
ConfigMgr[Config Manager]
LoggerService[Logger Service]
DI[DI Container]
@@ -252,15 +293,20 @@ graph TB
HealthRegistry[Health Registry]
MetricsRegistry[Metrics Registry]
ErrorBus[Error Bus]
EventBus[Event Bus]
Tracer[OpenTelemetry Tracer]
ServiceRegistry[Service Registry]
end
subgraph "Security Components"
AuthService[Auth Service]
AuthzService[Authorization Service]
TokenProvider[Token Provider]
PermissionResolver[Permission Resolver]
AuditService[Audit Service]
subgraph "Core Services (Separate Microservices)"
AuthService[Auth Service<br/>:8081]
IdentityService[Identity Service<br/>:8082]
AuthzService[Authz Service<br/>:8083]
AuditService[Audit Service<br/>:8084]
end
subgraph "Infrastructure Adapters"
EventBus[Event Bus<br/>Kafka]
CacheService[Cache Service<br/>Redis]
end
subgraph "Infrastructure Components"
@@ -282,36 +328,36 @@ graph TB
DI --> HealthRegistry
DI --> MetricsRegistry
DI --> ErrorBus
DI --> EventBus
DI --> AuthService
DI --> AuthzService
DI --> DBClient
DI --> CacheClient
DI --> Scheduler
DI --> Notifier
DI --> Tracer
DI --> ServiceRegistry
AuthService --> TokenProvider
AuthzService --> PermissionResolver
AuthzService --> AuditService
ModuleServices -->|gRPC| AuthService
ModuleServices -->|gRPC| IdentityService
ModuleServices -->|gRPC| AuthzService
ModuleServices -->|gRPC| AuditService
ModuleServices --> EventBus
ModuleServices --> CacheService
ModuleServices --> DBClient
ModuleServices --> CacheClient
ModuleServices --> EventBus
ModuleServices --> AuthzService
ModuleRepos --> DBClient
ModuleRoutes --> AuthzService
Scheduler --> CacheClient
Notifier --> EventBus
AuthService --> DBClient
IdentityService --> DBClient
AuthzService --> DBClient
AuditService --> DBClient
AuthService --> ServiceRegistry
IdentityService --> ServiceRegistry
AuthzService --> ServiceRegistry
AuditService --> ServiceRegistry
ErrorBus --> LoggerService
ErrorBus --> Sentry
style DI fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style AuthService fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style AuthService fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
style IdentityService fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
style ModuleServices fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
```
## Data Flow
@@ -320,34 +366,28 @@ graph TB
```mermaid
sequenceDiagram
participant Client
participant Router
participant AuthMW[Auth Middleware]
participant AuthzMW[Authz Middleware]
participant RateLimit[Rate Limiter]
participant Handler
participant Service
participant Gateway[API Gateway]
participant AuthSvc[Auth Service]
participant AuthzSvc[Authz Service]
participant Service[Feature Service]
participant IdentitySvc[Identity Service]
participant AuditSvc[Audit Service]
participant Repo
participant DB
participant Cache
participant EventBus
participant Audit
Client->>Router: HTTP Request
Router->>AuthMW: Extract JWT
AuthMW->>AuthMW: Validate token
AuthMW->>Router: Add user to context
Client->>Gateway: HTTP Request
Gateway->>Gateway: Rate limiting
Gateway->>AuthSvc: Validate JWT token (gRPC)
AuthSvc->>AuthSvc: Verify token
AuthSvc-->>Gateway: Token valid + user info
Router->>AuthzMW: Check permissions
AuthzMW->>AuthzMW: Resolve permissions
AuthzMW->>Router: Authorized
Gateway->>AuthzSvc: Check permissions (gRPC)
AuthzSvc->>AuthzSvc: Resolve permissions
AuthzSvc-->>Gateway: Authorized
Router->>RateLimit: Check rate limits
RateLimit->>Cache: Get rate limit state
Cache-->>RateLimit: Rate limit status
RateLimit->>Router: Within limits
Router->>Handler: Process request
Handler->>Service: Business logic
Gateway->>Service: Route to service (gRPC/HTTP)
Service->>Cache: Check cache
Cache-->>Service: Cache miss
@@ -357,12 +397,14 @@ sequenceDiagram
Repo-->>Service: Domain entity
Service->>Cache: Store in cache
Service->>EventBus: Publish event
Service->>Audit: Record action
Service->>IdentitySvc: Get user info (gRPC, if needed)
IdentitySvc-->>Service: User data
Service-->>Handler: Response data
Handler-->>Router: HTTP response
Router-->>Client: JSON response
Service->>EventBus: Publish event
Service->>AuditSvc: Record action (gRPC)
Service-->>Gateway: Response data
Gateway-->>Client: JSON response
```
### Module Event Flow
@@ -408,21 +450,56 @@ graph TB
end
subgraph "Local Services"
App[Platform App<br/>:8080]
Gateway[API Gateway<br/>:8080]
AuthSvc[Auth Service<br/>:8081]
IdentitySvc[Identity Service<br/>:8082]
AuthzSvc[Authz Service<br/>:8083]
AuditSvc[Audit Service<br/>:8084]
BlogSvc[Blog Service<br/>:8091]
DB[(PostgreSQL<br/>:5432)]
Redis[(Redis<br/>:6379)]
Kafka[Kafka<br/>:9092]
Consul[Consul<br/>:8500]
end
IDE --> Go
Go --> App
App --> DB
App --> Redis
App --> Kafka
Go --> Gateway
Go --> AuthSvc
Go --> IdentitySvc
Go --> AuthzSvc
Go --> AuditSvc
Go --> BlogSvc
Gateway --> AuthSvc
Gateway --> IdentitySvc
Gateway --> BlogSvc
AuthSvc --> DB
IdentitySvc --> DB
AuthzSvc --> DB
AuditSvc --> DB
BlogSvc --> DB
AuthSvc --> Redis
AuthzSvc --> Redis
BlogSvc --> Redis
BlogSvc --> Kafka
AuthSvc --> Consul
IdentitySvc --> Consul
AuthzSvc --> Consul
AuditSvc --> Consul
BlogSvc --> Consul
Docker --> DB
Docker --> Redis
Docker --> Kafka
Docker --> Consul
style Gateway fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
style AuthSvc fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
style IdentitySvc fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
```
### Production Deployment
@@ -433,10 +510,15 @@ graph TB
LB[Load Balancer<br/>HTTPS]
end
subgraph "Platform Instances"
App1[Platform Instance 1]
App2[Platform Instance 2]
App3[Platform Instance N]
subgraph "Service Instances"
Gateway1[API Gateway 1]
Gateway2[API Gateway 2]
AuthSvc1[Auth Service 1]
AuthSvc2[Auth Service 2]
IdentitySvc1[Identity Service 1]
IdentitySvc2[Identity Service 2]
BlogSvc1[Blog Service 1]
BlogSvc2[Blog Service 2]
end
subgraph "Database Cluster"
@@ -467,53 +549,86 @@ graph TB
S3[S3 Storage]
end
LB --> App1
LB --> App2
LB --> App3
LB --> Gateway1
LB --> Gateway2
App1 --> Primary
App2 --> Primary
App3 --> Primary
App1 --> Replica
App2 --> Replica
App3 --> Replica
Gateway1 --> AuthSvc1
Gateway1 --> AuthSvc2
Gateway1 --> IdentitySvc1
Gateway1 --> IdentitySvc2
Gateway1 --> BlogSvc1
Gateway1 --> BlogSvc2
Gateway2 --> AuthSvc1
Gateway2 --> AuthSvc2
Gateway2 --> IdentitySvc1
Gateway2 --> IdentitySvc2
Gateway2 --> BlogSvc1
Gateway2 --> BlogSvc2
App1 --> Redis1
App2 --> Redis1
App3 --> Redis1
AuthSvc1 --> Primary
AuthSvc2 --> Primary
IdentitySvc1 --> Primary
IdentitySvc2 --> Primary
BlogSvc1 --> Primary
BlogSvc2 --> Primary
App1 --> Kafka1
App2 --> Kafka2
App3 --> Kafka3
AuthSvc1 --> Replica
AuthSvc2 --> Replica
IdentitySvc1 --> Replica
IdentitySvc2 --> Replica
BlogSvc1 --> Replica
BlogSvc2 --> Replica
App1 --> Prometheus
App2 --> Prometheus
App3 --> Prometheus
AuthSvc1 --> Redis1
AuthSvc2 --> Redis1
IdentitySvc1 --> Redis1
IdentitySvc2 --> Redis1
BlogSvc1 --> Redis1
BlogSvc2 --> Redis1
BlogSvc1 --> Kafka1
BlogSvc2 --> Kafka2
AuthSvc1 --> Prometheus
AuthSvc2 --> Prometheus
IdentitySvc1 --> Prometheus
IdentitySvc2 --> Prometheus
BlogSvc1 --> Prometheus
BlogSvc2 --> Prometheus
Prometheus --> Grafana
App1 --> Jaeger
App2 --> Jaeger
App3 --> Jaeger
App1 --> Loki
App2 --> Loki
App3 --> Loki
AuthSvc1 --> Jaeger
AuthSvc2 --> Jaeger
IdentitySvc1 --> Jaeger
IdentitySvc2 --> Jaeger
BlogSvc1 --> Jaeger
BlogSvc2 --> Jaeger
AuthSvc1 --> Loki
AuthSvc2 --> Loki
IdentitySvc1 --> Loki
IdentitySvc2 --> Loki
BlogSvc1 --> Loki
BlogSvc2 --> Loki
App1 --> Sentry
App2 --> Sentry
App3 --> Sentry
AuthSvc1 --> Sentry
AuthSvc2 --> Sentry
IdentitySvc1 --> Sentry
IdentitySvc2 --> Sentry
BlogSvc1 --> Sentry
BlogSvc2 --> Sentry
App1 --> S3
App2 --> S3
App3 --> S3
BlogSvc1 --> S3
BlogSvc2 --> S3
style LB fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style Gateway1 fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
style Gateway2 fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
style Primary fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style Redis1 fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
```
## Core Kernel Components
The core kernel provides the foundation for all modules. Each component has specific responsibilities:
The core kernel provides **infrastructure only** - no business logic. It is the foundation that all services depend on. Business logic resides in separate services (Auth, Identity, Authz, Audit).
### Component Responsibilities
@@ -536,10 +651,10 @@ mindmap
Metrics
Tracing
Health checks
Security
Authentication
Authorization
Audit logging
Service Discovery
Service registry
Service registration
Health checking
Module System
Module discovery
Module loading
@@ -555,8 +670,19 @@ graph TB
subgraph "Core Kernel Interfaces"
IConfig[ConfigProvider]
ILogger[Logger]
IAuth[Authenticator]
IAuthz[Authorizer]
ITracer[Tracer]
IMetrics[Metrics]
IHealth[Health]
end
subgraph "Service Client Interfaces"
IAuthClient[AuthServiceClient]
IIdentityClient[IdentityServiceClient]
IAuthzClient[AuthzServiceClient]
IAuditClient[AuditServiceClient]
end
subgraph "Infrastructure Interfaces"
IEventBus[EventBus]
ICache[Cache]
IBlobStore[BlobStore]
@@ -564,23 +690,29 @@ graph TB
INotifier[Notifier]
end
subgraph "Module Implementation"
Module[Feature Module]
ModuleServices[Module Services]
ModuleRoutes[Module Routes]
subgraph "Feature Service Implementation"
Module[Feature Service]
ModuleServices[Service Layer]
ModuleRoutes[HTTP/gRPC Routes]
end
Module --> IConfig
Module --> ILogger
ModuleServices --> IAuth
ModuleServices --> IAuthz
Module --> ITracer
Module --> IMetrics
Module --> IHealth
ModuleServices -->|gRPC| IAuthClient
ModuleServices -->|gRPC| IIdentityClient
ModuleServices -->|gRPC| IAuthzClient
ModuleServices -->|gRPC| IAuditClient
ModuleServices --> IEventBus
ModuleServices --> ICache
ModuleServices --> IBlobStore
ModuleServices --> IScheduler
ModuleServices --> INotifier
ModuleRoutes --> IAuthz
ModuleRoutes -->|gRPC| IAuthzClient
style IConfig fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
style Module fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
@@ -737,10 +869,12 @@ graph TB
- Team autonomy
#### Development Mode
- For local development, multiple services can run in the same process
- Services still communicate via gRPC/HTTP (no direct calls)
- Docker Compose for easy local setup
- For local development, services can run in the same repository/monorepo
- Services still communicate via gRPC/HTTP through service clients (no direct in-process calls)
- Each service has its own process and entry point
- Docker Compose for easy local setup with all services
- Maintains microservices architecture even in development
- Services can be started individually for debugging
## Next Steps