toolz/goplt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: Align documentation with true microservices architecture

Browse Source 
Transform all documentation from modular monolith to true microservices
architecture where core services are independently deployable.

Key Changes:
- Core Kernel: Infrastructure only (no business logic)
- Core Services: Auth, Identity, Authz, Audit as separate microservices
  - Each service has own entry point (cmd/{service}/)
  - Each service has own gRPC server and database schema
  - Services register with Consul for service discovery
- API Gateway: Moved from Epic 8 to Epic 1 as core infrastructure
  - Single entry point for all external traffic
  - Handles routing, JWT validation, rate limiting, CORS
- Service Discovery: Consul as primary mechanism (ADR-0033)
- Database Pattern: Per-service connections with schema isolation

Documentation Updates:
- Updated all 9 architecture documents
- Updated 4 ADRs and created 2 new ADRs (API Gateway, Service Discovery)
- Rewrote Epic 1: Core Kernel & Infrastructure (infrastructure only)
- Rewrote Epic 2: Core Services (Auth, Identity, Authz, Audit as services)
- Updated Epic 3-8 stories for service architecture
- Updated plan.md, playbook.md, requirements.md, index.md
- Updated all epic READMEs and story files

New ADRs:
- ADR-0032: API Gateway Strategy
- ADR-0033: Service Discovery Implementation (Consul)

New Stories:
- Epic 1.7: Service Client Interfaces
- Epic 1.8: API Gateway Implementation
This commit is contained in:
0x1d
2025-11-06 08:47:27 +01:00
parent cab7cadf9e
commit 38a251968c
47 changed files with 3190 additions and 1613 deletions
Download Patch File Download Diff File Expand all files Collapse all files

52
docs/content/architecture/architecture-modules.md
View File

@@ -1,6 +1,6 @@
# Module Architecture
This document details the architecture of modules, how they are structured, how they interact with the core platform, and how multiple modules work together.
This document details the architecture of modules (feature services), how they are structured as independent services, how they interact with core services, and how multiple services work together in the microservices architecture.
## Table of Contents
@@ -52,28 +52,33 @@ graph TD
style Service fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
```
### Module Directory Structure
### Service Directory Structure
Each module is an independent service with its own entry point:
```
modules/blog/
├── go.mod # Module dependencies
├── module.yaml # Module manifest
├── pkg/
│ └── module.go # IModule implementation
cmd/blog-service/
└── main.go # Service entry point
services/blog/
├── go.mod # Service dependencies
├── module.yaml # Service manifest
├── api/
│ └── blog.proto # gRPC service definition
├── internal/
│ ├── api/
│ │ └── handler.go # HTTP handlers
│ │ └── handler.go # gRPC handlers
│ ├── domain/
│ │ ├── post.go # Domain entities
│ │ └── post_repo.go # Repository interface
│ ├── service/
│ │ └── post_service.go # Business logic
│ └── ent/
│ ├── schema/
│ │ └── post.go # Ent schema
│ └── migrate/ # Migrations
└── tests/
└── integration_test.go
│ └── database/
│ └── client.go # Database connection
└── ent/
├── schema/
│ └── post.go # Ent schema
└── migrate/ # Migrations
```
## Module Interface
@@ -262,9 +267,9 @@ graph LR
style Step1 fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
```
## Module Communication
## Service Communication
Modules (services) communicate through service client interfaces. All inter-service communication uses gRPC (primary) or HTTP (fallback).
Modules are implemented as independent services that communicate through service client interfaces. All inter-service communication uses gRPC (primary) or HTTP (fallback) via service clients. Services discover each other through the service registry (Consul).
### Communication Patterns
@@ -301,14 +306,27 @@ graph TB
BlogService -->|gRPC| AuthClient
BlogService -->|gRPC| IdentityClient
BlogService -->|gRPC| AuthzClient
BlogService -->|gRPC| AuditClient
BlogService -->|Publish| EventBus
EventBus -->|Subscribe| AnalyticsService
AuthClient -->|Discover| Registry
IdentityClient -->|Discover| Registry
AuthzClient -->|Discover| Registry
AuditClient -->|Discover| Registry
Registry --> AuthService
Registry --> IdentityService
Registry --> AuthzService
Registry --> AuditService
AuthClient --> AuthService
IdentityClient --> IdentityService
AuthzClient --> IdentityService
AuthzClient --> AuthzService
AuditClient --> AuditService
style EventBus fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style Registry fill:#50c878,stroke:#2e7d4e,stroke-width:3px,color:#fff
style BlogService fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
style AnalyticsService fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
style ServiceClients fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff

414
docs/content/architecture/architecture.md
View File

@@ -13,20 +13,36 @@ This document provides a comprehensive overview of the Go Platform architecture,
## High-Level Architecture
The Go Platform follows a **microservices architecture** where each module is an independent service:
- **Core Services**: Authentication, Identity, Authorization, Audit, etc.
- **Feature Services**: Blog, Billing, Analytics, etc. (modules)
- **Infrastructure Services**: Cache, Event Bus, Scheduler, etc.
The Go Platform follows a **microservices architecture** where each service is independently deployable from day one:
- **Core Kernel**: Infrastructure only (config, logger, DI, health, metrics, observability) - no business logic
- **Core Services**: Auth Service, Identity Service, Authz Service, Audit Service - separate microservices
- **API Gateway**: Single entry point for all external traffic, handles routing and authentication
- **Feature Services**: Blog, Billing, Analytics, etc. - independent services
- **Infrastructure Adapters**: Cache, Event Bus, Scheduler, etc. - shared infrastructure
All services communicate via gRPC (primary) or HTTP (fallback), with service discovery via a service registry. Services share infrastructure (PostgreSQL, Redis, Kafka) but are independently deployable and scalable.
All services communicate via gRPC (primary) or HTTP (fallback) through service client interfaces, with service discovery via a service registry. Each service has its own database connection pool and schema. Services share infrastructure (PostgreSQL instance, Redis, Kafka) but are independently deployable and scalable.
```mermaid
graph TB
subgraph "Go Platform"
Core[Core Kernel]
Module1[Module 1<br/>Blog]
Module2[Module 2<br/>Billing]
Module3[Module N<br/>Custom]
subgraph "API Gateway"
Gateway[API Gateway<br/>:8080]
end
subgraph "Core Services"
AuthSvc[Auth Service<br/>:8081]
IdentitySvc[Identity Service<br/>:8082]
AuthzSvc[Authz Service<br/>:8083]
AuditSvc[Audit Service<br/>:8084]
end
subgraph "Feature Services"
BlogSvc[Blog Service<br/>:8091]
BillingSvc[Billing Service<br/>:8092]
AnalyticsSvc[Analytics Service<br/>:8093]
end
subgraph "Core Kernel"
Kernel[Core Kernel<br/>Infrastructure Only]
end
subgraph "Infrastructure"
@@ -34,6 +50,7 @@ graph TB
Cache[(Redis)]
Queue[Kafka/Event Bus]
Storage[S3/Blob Storage]
Registry[Service Registry]
end
subgraph "External Services"
@@ -42,29 +59,52 @@ graph TB
Sentry[Sentry]
end
Core --> DB
Core --> Cache
Core --> Queue
Core --> Storage
Core --> OIDC
Core --> Email
Core --> Sentry
Gateway --> AuthSvc
Gateway --> IdentitySvc
Gateway --> AuthzSvc
Gateway --> BlogSvc
Gateway --> BillingSvc
Module1 --> Core
Module2 --> Core
Module3 --> Core
AuthSvc --> IdentitySvc
AuthSvc --> Registry
AuthzSvc --> IdentitySvc
AuthzSvc --> Cache
AuthzSvc --> AuditSvc
BlogSvc --> AuthzSvc
BlogSvc --> IdentitySvc
BlogSvc --> Registry
Module1 --> DB
Module2 --> DB
Module3 --> DB
AuthSvc --> DB
IdentitySvc --> DB
AuthzSvc --> DB
AuditSvc --> DB
BlogSvc --> DB
BillingSvc --> DB
Module1 --> Queue
Module2 --> Queue
AuthSvc --> Cache
AuthzSvc --> Cache
BlogSvc --> Cache
BillingSvc --> Cache
style Core fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style Module1 fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
style Module2 fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
style Module3 fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
BlogSvc --> Queue
BillingSvc --> Queue
AnalyticsSvc --> Queue
Kernel --> DB
Kernel --> Cache
Kernel --> Queue
Kernel --> Registry
AuthSvc --> OIDC
IdentitySvc --> Email
AuditSvc --> Sentry
style Gateway fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style Kernel fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style AuthSvc fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
style IdentitySvc fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
style BlogSvc fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
style BillingSvc fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
```
## Layered Architecture
@@ -99,12 +139,13 @@ graph TD
Jobs[Scheduler/Jobs]
end
subgraph "Core Kernel"
subgraph "Core Kernel (Infrastructure Only)"
DI[DI Container]
Config[Config Manager]
Logger[Logger]
Metrics[Metrics]
Health[Health Checks]
Tracer[OpenTelemetry Tracer]
end
HTTP --> AuthMiddleware
@@ -244,7 +285,7 @@ This diagram shows how core components interact with each other and with modules
```mermaid
graph TB
subgraph "Core Kernel Components"
subgraph "Core Kernel Components (Infrastructure)"
ConfigMgr[Config Manager]
LoggerService[Logger Service]
DI[DI Container]
@@ -252,15 +293,20 @@ graph TB
HealthRegistry[Health Registry]
MetricsRegistry[Metrics Registry]
ErrorBus[Error Bus]
EventBus[Event Bus]
Tracer[OpenTelemetry Tracer]
ServiceRegistry[Service Registry]
end
subgraph "Security Components"
AuthService[Auth Service]
AuthzService[Authorization Service]
TokenProvider[Token Provider]
PermissionResolver[Permission Resolver]
AuditService[Audit Service]
subgraph "Core Services (Separate Microservices)"
AuthService[Auth Service<br/>:8081]
IdentityService[Identity Service<br/>:8082]
AuthzService[Authz Service<br/>:8083]
AuditService[Audit Service<br/>:8084]
end
subgraph "Infrastructure Adapters"
EventBus[Event Bus<br/>Kafka]
CacheService[Cache Service<br/>Redis]
end
subgraph "Infrastructure Components"
@@ -282,36 +328,36 @@ graph TB
DI --> HealthRegistry
DI --> MetricsRegistry
DI --> ErrorBus
DI --> EventBus
DI --> AuthService
DI --> AuthzService
DI --> DBClient
DI --> CacheClient
DI --> Scheduler
DI --> Notifier
DI --> Tracer
DI --> ServiceRegistry
AuthService --> TokenProvider
AuthzService --> PermissionResolver
AuthzService --> AuditService
ModuleServices -->|gRPC| AuthService
ModuleServices -->|gRPC| IdentityService
ModuleServices -->|gRPC| AuthzService
ModuleServices -->|gRPC| AuditService
ModuleServices --> EventBus
ModuleServices --> CacheService
ModuleServices --> DBClient
ModuleServices --> CacheClient
ModuleServices --> EventBus
ModuleServices --> AuthzService
ModuleRepos --> DBClient
ModuleRoutes --> AuthzService
Scheduler --> CacheClient
Notifier --> EventBus
AuthService --> DBClient
IdentityService --> DBClient
AuthzService --> DBClient
AuditService --> DBClient
AuthService --> ServiceRegistry
IdentityService --> ServiceRegistry
AuthzService --> ServiceRegistry
AuditService --> ServiceRegistry
ErrorBus --> LoggerService
ErrorBus --> Sentry
style DI fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style AuthService fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style AuthService fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
style IdentityService fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
style ModuleServices fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
```
## Data Flow
@@ -320,34 +366,28 @@ graph TB
```mermaid
sequenceDiagram
participant Client
participant Router
participant AuthMW[Auth Middleware]
participant AuthzMW[Authz Middleware]
participant RateLimit[Rate Limiter]
participant Handler
participant Service
participant Gateway[API Gateway]
participant AuthSvc[Auth Service]
participant AuthzSvc[Authz Service]
participant Service[Feature Service]
participant IdentitySvc[Identity Service]
participant AuditSvc[Audit Service]
participant Repo
participant DB
participant Cache
participant EventBus
participant Audit
Client->>Router: HTTP Request
Router->>AuthMW: Extract JWT
AuthMW->>AuthMW: Validate token
AuthMW->>Router: Add user to context
Client->>Gateway: HTTP Request
Gateway->>Gateway: Rate limiting
Gateway->>AuthSvc: Validate JWT token (gRPC)
AuthSvc->>AuthSvc: Verify token
AuthSvc-->>Gateway: Token valid + user info
Router->>AuthzMW: Check permissions
AuthzMW->>AuthzMW: Resolve permissions
AuthzMW->>Router: Authorized
Gateway->>AuthzSvc: Check permissions (gRPC)
AuthzSvc->>AuthzSvc: Resolve permissions
AuthzSvc-->>Gateway: Authorized
Router->>RateLimit: Check rate limits
RateLimit->>Cache: Get rate limit state
Cache-->>RateLimit: Rate limit status
RateLimit->>Router: Within limits
Router->>Handler: Process request
Handler->>Service: Business logic
Gateway->>Service: Route to service (gRPC/HTTP)
Service->>Cache: Check cache
Cache-->>Service: Cache miss
@@ -357,12 +397,14 @@ sequenceDiagram
Repo-->>Service: Domain entity
Service->>Cache: Store in cache
Service->>EventBus: Publish event
Service->>Audit: Record action
Service->>IdentitySvc: Get user info (gRPC, if needed)
IdentitySvc-->>Service: User data
Service-->>Handler: Response data
Handler-->>Router: HTTP response
Router-->>Client: JSON response
Service->>EventBus: Publish event
Service->>AuditSvc: Record action (gRPC)
Service-->>Gateway: Response data
Gateway-->>Client: JSON response
```
### Module Event Flow
@@ -408,21 +450,56 @@ graph TB
end
subgraph "Local Services"
App[Platform App<br/>:8080]
Gateway[API Gateway<br/>:8080]
AuthSvc[Auth Service<br/>:8081]
IdentitySvc[Identity Service<br/>:8082]
AuthzSvc[Authz Service<br/>:8083]
AuditSvc[Audit Service<br/>:8084]
BlogSvc[Blog Service<br/>:8091]
DB[(PostgreSQL<br/>:5432)]
Redis[(Redis<br/>:6379)]
Kafka[Kafka<br/>:9092]
Consul[Consul<br/>:8500]
end
IDE --> Go
Go --> App
App --> DB
App --> Redis
App --> Kafka
Go --> Gateway
Go --> AuthSvc
Go --> IdentitySvc
Go --> AuthzSvc
Go --> AuditSvc
Go --> BlogSvc
Gateway --> AuthSvc
Gateway --> IdentitySvc
Gateway --> BlogSvc
AuthSvc --> DB
IdentitySvc --> DB
AuthzSvc --> DB
AuditSvc --> DB
BlogSvc --> DB
AuthSvc --> Redis
AuthzSvc --> Redis
BlogSvc --> Redis
BlogSvc --> Kafka
AuthSvc --> Consul
IdentitySvc --> Consul
AuthzSvc --> Consul
AuditSvc --> Consul
BlogSvc --> Consul
Docker --> DB
Docker --> Redis
Docker --> Kafka
Docker --> Consul
style Gateway fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
style AuthSvc fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
style IdentitySvc fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
```
### Production Deployment
@@ -433,10 +510,15 @@ graph TB
LB[Load Balancer<br/>HTTPS]
end
subgraph "Platform Instances"
App1[Platform Instance 1]
App2[Platform Instance 2]
App3[Platform Instance N]
subgraph "Service Instances"
Gateway1[API Gateway 1]
Gateway2[API Gateway 2]
AuthSvc1[Auth Service 1]
AuthSvc2[Auth Service 2]
IdentitySvc1[Identity Service 1]
IdentitySvc2[Identity Service 2]
BlogSvc1[Blog Service 1]
BlogSvc2[Blog Service 2]
end
subgraph "Database Cluster"
@@ -467,53 +549,86 @@ graph TB
S3[S3 Storage]
end
LB --> App1
LB --> App2
LB --> App3
LB --> Gateway1
LB --> Gateway2
App1 --> Primary
App2 --> Primary
App3 --> Primary
App1 --> Replica
App2 --> Replica
App3 --> Replica
Gateway1 --> AuthSvc1
Gateway1 --> AuthSvc2
Gateway1 --> IdentitySvc1
Gateway1 --> IdentitySvc2
Gateway1 --> BlogSvc1
Gateway1 --> BlogSvc2
Gateway2 --> AuthSvc1
Gateway2 --> AuthSvc2
Gateway2 --> IdentitySvc1
Gateway2 --> IdentitySvc2
Gateway2 --> BlogSvc1
Gateway2 --> BlogSvc2
App1 --> Redis1
App2 --> Redis1
App3 --> Redis1
AuthSvc1 --> Primary
AuthSvc2 --> Primary
IdentitySvc1 --> Primary
IdentitySvc2 --> Primary
BlogSvc1 --> Primary
BlogSvc2 --> Primary
App1 --> Kafka1
App2 --> Kafka2
App3 --> Kafka3
AuthSvc1 --> Replica
AuthSvc2 --> Replica
IdentitySvc1 --> Replica
IdentitySvc2 --> Replica
BlogSvc1 --> Replica
BlogSvc2 --> Replica
App1 --> Prometheus
App2 --> Prometheus
App3 --> Prometheus
AuthSvc1 --> Redis1
AuthSvc2 --> Redis1
IdentitySvc1 --> Redis1
IdentitySvc2 --> Redis1
BlogSvc1 --> Redis1
BlogSvc2 --> Redis1
BlogSvc1 --> Kafka1
BlogSvc2 --> Kafka2
AuthSvc1 --> Prometheus
AuthSvc2 --> Prometheus
IdentitySvc1 --> Prometheus
IdentitySvc2 --> Prometheus
BlogSvc1 --> Prometheus
BlogSvc2 --> Prometheus
Prometheus --> Grafana
App1 --> Jaeger
App2 --> Jaeger
App3 --> Jaeger
App1 --> Loki
App2 --> Loki
App3 --> Loki
AuthSvc1 --> Jaeger
AuthSvc2 --> Jaeger
IdentitySvc1 --> Jaeger
IdentitySvc2 --> Jaeger
BlogSvc1 --> Jaeger
BlogSvc2 --> Jaeger
AuthSvc1 --> Loki
AuthSvc2 --> Loki
IdentitySvc1 --> Loki
IdentitySvc2 --> Loki
BlogSvc1 --> Loki
BlogSvc2 --> Loki
App1 --> Sentry
App2 --> Sentry
App3 --> Sentry
AuthSvc1 --> Sentry
AuthSvc2 --> Sentry
IdentitySvc1 --> Sentry
IdentitySvc2 --> Sentry
BlogSvc1 --> Sentry
BlogSvc2 --> Sentry
App1 --> S3
App2 --> S3
App3 --> S3
BlogSvc1 --> S3
BlogSvc2 --> S3
style LB fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style Gateway1 fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
style Gateway2 fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
style Primary fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style Redis1 fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
```
## Core Kernel Components
The core kernel provides the foundation for all modules. Each component has specific responsibilities:
The core kernel provides **infrastructure only** - no business logic. It is the foundation that all services depend on. Business logic resides in separate services (Auth, Identity, Authz, Audit).
### Component Responsibilities
@@ -536,10 +651,10 @@ mindmap
Metrics
Tracing
Health checks
Security
Authentication
Authorization
Audit logging
Service Discovery
Service registry
Service registration
Health checking
Module System
Module discovery
Module loading
@@ -555,8 +670,19 @@ graph TB
subgraph "Core Kernel Interfaces"
IConfig[ConfigProvider]
ILogger[Logger]
IAuth[Authenticator]
IAuthz[Authorizer]
ITracer[Tracer]
IMetrics[Metrics]
IHealth[Health]
end
subgraph "Service Client Interfaces"
IAuthClient[AuthServiceClient]
IIdentityClient[IdentityServiceClient]
IAuthzClient[AuthzServiceClient]
IAuditClient[AuditServiceClient]
end
subgraph "Infrastructure Interfaces"
IEventBus[EventBus]
ICache[Cache]
IBlobStore[BlobStore]
@@ -564,23 +690,29 @@ graph TB
INotifier[Notifier]
end
subgraph "Module Implementation"
Module[Feature Module]
ModuleServices[Module Services]
ModuleRoutes[Module Routes]
subgraph "Feature Service Implementation"
Module[Feature Service]
ModuleServices[Service Layer]
ModuleRoutes[HTTP/gRPC Routes]
end
Module --> IConfig
Module --> ILogger
ModuleServices --> IAuth
ModuleServices --> IAuthz
Module --> ITracer
Module --> IMetrics
Module --> IHealth
ModuleServices -->|gRPC| IAuthClient
ModuleServices -->|gRPC| IIdentityClient
ModuleServices -->|gRPC| IAuthzClient
ModuleServices -->|gRPC| IAuditClient
ModuleServices --> IEventBus
ModuleServices --> ICache
ModuleServices --> IBlobStore
ModuleServices --> IScheduler
ModuleServices --> INotifier
ModuleRoutes --> IAuthz
ModuleRoutes -->|gRPC| IAuthzClient
style IConfig fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
style Module fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
@@ -737,10 +869,12 @@ graph TB
- Team autonomy
#### Development Mode
- For local development, multiple services can run in the same process
- Services still communicate via gRPC/HTTP (no direct calls)
- Docker Compose for easy local setup
- For local development, services can run in the same repository/monorepo
- Services still communicate via gRPC/HTTP through service clients (no direct in-process calls)
- Each service has its own process and entry point
- Docker Compose for easy local setup with all services
- Maintains microservices architecture even in development
- Services can be started individually for debugging
## Next Steps

46
docs/content/architecture/component-relationships.md
View File

@@ -32,10 +32,8 @@ graph TD
Scheduler[Scheduler]
end
subgraph "Security Layer"
Auth[Auth Service]
Authz[Authz Service]
Audit[Audit Service]
subgraph "Service Registry"
Registry[Service Registry<br/>Consul]
end
subgraph "Observability Layer"
@@ -52,17 +50,14 @@ graph TD
DI --> Cache
DI --> EventBus
DI --> Scheduler
DI --> Auth
DI --> Authz
DI --> Audit
DI --> Metrics
DI --> Health
DI --> Tracer
DI --> Registry
Auth --> DB
Authz --> DB
Authz --> Cache
Audit --> DB
Registry --> Auth
Registry --> Authz
Registry --> Audit
DB --> Tracer
Cache --> Tracer
@@ -73,9 +68,9 @@ graph TD
style Auth fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
```
## Module to Core Integration
## Service to Service Integration
Modules (services) integrate with core services through service client interfaces. All communication uses gRPC or HTTP.
Feature services integrate with core services through service client interfaces. All communication uses gRPC (primary) or HTTP (fallback). Services discover each other via the service registry (Consul).
```mermaid
graph LR
@@ -86,10 +81,14 @@ graph LR
end
subgraph "Service Clients"
AuthClient[Auth Service Client]
AuthzClient[Authz Service Client]
IdentityClient[Identity Service Client]
AuditClient[Audit Service Client]
AuthClient[Auth Service Client<br/>gRPC]
AuthzClient[Authz Service Client<br/>gRPC]
IdentityClient[Identity Service Client<br/>gRPC]
AuditClient[Audit Service Client<br/>gRPC]
end
subgraph "Service Registry"
Registry[Consul<br/>Service Discovery]
end
subgraph "Core Services"
@@ -117,6 +116,16 @@ graph LR
ModuleService --> EventBusService
ModuleService --> CacheService
AuthClient -->|Discover| Registry
AuthzClient -->|Discover| Registry
IdentityClient -->|Discover| Registry
AuditClient -->|Discover| Registry
Registry --> AuthService
Registry --> AuthzService
Registry --> IdentityService
Registry --> AuditService
AuthClient --> AuthService
AuthzClient --> AuthzService
IdentityClient --> IdentityService
@@ -127,7 +136,8 @@ graph LR
EventBusService --> QueueClient
style ModuleService fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
style AuthService fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
style Registry fill:#50c878,stroke:#2e7d4e,stroke-width:3px,color:#fff
style AuthService fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
style DBClient fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style ServiceClients fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
```

59
docs/content/architecture/data-flow-patterns.md
View File

@@ -19,18 +19,22 @@ Data flows through the platform in multiple patterns depending on the type of op
### Standard HTTP Request Flow
Complete data flow from HTTP request to response.
Complete data flow from HTTP request through API Gateway to backend service and response.
```mermaid
graph TD
Start[HTTP Request] --> Auth[Authentication]
Auth -->|Valid| Authz[Authorization]
Auth -->|Invalid| Error1[401 Response]
Start[HTTP Request] --> Gateway[API Gateway]
Gateway --> RateLimit{Rate Limit Check}
RateLimit -->|Allowed| Auth[Validate JWT via Auth Service]
RateLimit -->|Exceeded| Error0[429 Too Many Requests]
Authz -->|Authorized| Handler[Request Handler]
Authz -->|Unauthorized| Error2[403 Response]
Auth -->|Valid| Authz[Check Permission via Authz Service]
Auth -->|Invalid| Error1[401 Unauthorized]
Handler --> Service[Domain Service]
Authz -->|Authorized| Route[Route to Backend Service]
Authz -->|Unauthorized| Error2[403 Forbidden]
Route --> Service[Backend Service]
Service --> Cache{Cache Check}
Cache -->|Hit| CacheData[Return Cached Data]
@@ -42,17 +46,19 @@ graph TD
Service --> CacheStore[Update Cache]
Service --> EventBus[Publish Events]
Service --> Audit[Audit Log]
Service --> AuditSvc[Audit Service<br/>gRPC]
Service --> Metrics[Update Metrics]
Service --> Handler
Handler --> Response[HTTP Response]
CacheData --> Response
Service --> Gateway
Gateway --> Response[HTTP Response]
CacheData --> Gateway
Error0 --> Response
Error1 --> Response
Error2 --> Response
Response --> Client[Client]
style Gateway fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style Auth fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
style Service fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style Cache fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
@@ -60,22 +66,30 @@ graph TD
### Request Data Transformation
How request data is transformed as it flows through the system.
How request data is transformed as it flows through API Gateway to backend service.
```mermaid
sequenceDiagram
participant Client
participant Handler
participant Gateway
participant BackendService
participant Service
participant Repo
participant DB
Client->>Handler: HTTP Request (JSON)
Handler->>Handler: Parse JSON
Handler->>Handler: Validate request
Handler->>Handler: Convert to DTO
Client->>Gateway: HTTP Request (JSON)
Gateway->>Gateway: Rate limiting
Gateway->>Gateway: Validate JWT (via Auth Service)
Gateway->>Gateway: Check permission (via Authz Service)
Gateway->>Gateway: Route to service (via service discovery)
Gateway->>Gateway: Forward request (gRPC/HTTP)
Handler->>Service: Business DTO
Gateway->>BackendService: Request (gRPC/HTTP)
BackendService->>BackendService: Parse request
BackendService->>BackendService: Validate request
BackendService->>BackendService: Convert to DTO
BackendService->>Service: Business DTO
Service->>Service: Business logic
Service->>Service: Domain entity
@@ -89,10 +103,13 @@ sequenceDiagram
Service->>Service: Business logic
Service->>Service: Response DTO
Service-->>Handler: Response DTO
Service-->>BackendService: Response DTO
Handler->>Handler: Convert to JSON
Handler-->>Client: HTTP Response (JSON)
BackendService->>BackendService: Convert to response format
BackendService-->>Gateway: Response (gRPC/HTTP)
Gateway->>Gateway: Transform response (if needed)
Gateway-->>Client: HTTP Response (JSON)
```
## Event Data Flow

4
docs/content/architecture/module-integration-patterns.md
View File

@@ -6,7 +6,7 @@ This document explains how modules integrate with the core platform, focusing on
## Overview
Modules are independent services that extend the platform's functionality. They integrate with the core platform through well-defined interfaces, service clients, and a standardized initialization process. Each module operates as an independent service while leveraging core platform capabilities.
Modules are implemented as independent services that extend the platform's functionality. They integrate with core services through service client interfaces, service discovery (Consul), and a standardized initialization process. Each module operates as an independent service with its own entry point, database connection, and deployment while leveraging core platform capabilities.
## Key Concepts
@@ -94,7 +94,7 @@ sequenceDiagram
DB-->>Module: Migrations registered
Module->>ServiceRegistry: Register service
ServiceRegistry->>ServiceRegistry: Register with registry
ServiceRegistry->>ServiceRegistry: Register with Consul
ServiceRegistry-->>Module: Service registered
Module->>Module: OnStart hook (optional)

35
docs/content/architecture/operational-scenarios.md
View File

@@ -25,16 +25,22 @@ Complete flow of user logging in and receiving authentication tokens.
sequenceDiagram
participant User
participant Client
participant Gateway[API Gateway]
participant AuthService
participant IdentityService
participant DB
participant TokenProvider
participant AuditService
participant Registry[Consul]
User->>Client: Enter credentials
Client->>AuthService: POST /api/v1/auth/login
Client->>Gateway: POST /api/v1/auth/login
Gateway->>Gateway: Rate limiting check
Gateway->>AuthService: Login request (gRPC)
AuthService->>AuthService: Validate request format
AuthService->>IdentityService: Verify credentials
AuthService->>Registry: Discover Identity Service
Registry-->>AuthService: Identity Service endpoint
AuthService->>IdentityService: Verify credentials (gRPC)
IdentityService->>DB: Query user by email
DB-->>IdentityService: User data
IdentityService->>IdentityService: Verify password hash
@@ -49,11 +55,14 @@ sequenceDiagram
DB-->>TokenProvider: Token stored
TokenProvider-->>AuthService: Refresh token
AuthService->>AuditService: Log login
AuthService->>Registry: Discover Audit Service
Registry-->>AuthService: Audit Service endpoint
AuthService->>AuditService: Log login (gRPC)
AuditService->>DB: Store audit log
AuditService-->>AuthService: Logged
AuthService-->>Client: Access + Refresh tokens
AuthService-->>Gateway: Access + Refresh tokens
Gateway-->>Client: Access + Refresh tokens
Client-->>User: Authentication successful
```
@@ -63,23 +72,25 @@ How the system checks if a user has permission to perform an action.
```mermaid
sequenceDiagram
participant Gateway[API Gateway]
participant Handler
participant AuthzMiddleware
participant AuthzService
participant PermissionResolver
participant Cache
participant DB
participant IdentityService
participant Registry[Consul]
Handler->>AuthzMiddleware: Check permission
AuthzMiddleware->>AuthzMiddleware: Extract user from context
AuthzMiddleware->>AuthzService: Authorize(user, permission)
Gateway->>Handler: Request with user context
Handler->>AuthzService: Authorize(user, permission) (gRPC)
AuthzService->>Registry: Discover Identity Service
Registry-->>AuthzService: Identity Service endpoint
AuthzService->>Cache: Check permission cache
Cache-->>AuthzService: Cache miss
AuthzService->>PermissionResolver: Resolve permissions
PermissionResolver->>IdentityService: Get user roles
PermissionResolver->>IdentityService: Get user roles (gRPC)
IdentityService->>DB: Query user roles
DB-->>IdentityService: User roles
IdentityService-->>PermissionResolver: Roles list
@@ -91,12 +102,12 @@ sequenceDiagram
AuthzService->>AuthzService: Check permission in list
AuthzService->>Cache: Store in cache
AuthzService-->>AuthzMiddleware: Authorized/Unauthorized
AuthzService-->>Handler: Authorized/Unauthorized
alt Authorized
AuthzMiddleware-->>Handler: Continue
Handler-->>Gateway: Continue request
else Unauthorized
AuthzMiddleware-->>Handler: 403 Forbidden
Handler-->>Gateway: 403 Forbidden
end
```

8
docs/content/architecture/service-orchestration.md
View File

@@ -6,12 +6,12 @@ This document explains how services work together in the Go Platform's microserv
## Overview
The Go Platform consists of multiple independent services that communicate via service clients (gRPC/HTTP) and share infrastructure components. Services are discovered and registered through a service registry, enabling dynamic service location and health monitoring.
The Go Platform consists of multiple independent services that communicate via service clients (gRPC/HTTP) and share infrastructure components. Services are discovered and registered through a service registry (Consul), enabling dynamic service location and health monitoring.
## Key Concepts
- **Service**: Independent process providing specific functionality
- **Service Registry**: Central registry for service discovery (Consul, Kubernetes, etcd)
- **Service Registry**: Central registry for service discovery (Consul - primary, Kubernetes as alternative)
- **Service Client**: Abstraction for inter-service communication
- **Service Discovery**: Process of locating services by name
- **Service Health**: Health status of a service (healthy, unhealthy, degraded)
@@ -59,7 +59,7 @@ Services automatically register themselves with the service registry on startup
sequenceDiagram
participant Service
participant ServiceRegistry
participant Registry[Consul/K8s]
participant Registry[Consul<br/>Service Registry]
participant Client
Service->>ServiceRegistry: Register(serviceInfo)
@@ -93,7 +93,7 @@ sequenceDiagram
1. **Service Startup**: Service initializes and loads configuration
2. **Service Info Creation**: Create service info with name, version, address, protocol
3. **Registry Registration**: Register service with Consul/Kubernetes/etc
3. **Registry Registration**: Register service with Consul (primary) or Kubernetes service discovery (alternative)
4. **Health Check Setup**: Start health check endpoint
5. **Health Status Updates**: Periodically update health status in registry
6. **Service Discovery**: Clients query registry for service endpoints

140
docs/content/architecture/system-behavior.md
View File

@@ -6,7 +6,7 @@ This document provides a high-level explanation of how the Go Platform behaves e
## Overview
The Go Platform is a microservices-based system where each module operates as an independent service. Services communicate via gRPC (primary) or HTTP (fallback), share infrastructure components (PostgreSQL, Redis, Kafka), and are orchestrated through service discovery and dependency injection.
The Go Platform is a microservices-based system where each service is independently deployable from day one. Services communicate via gRPC (primary) or HTTP (fallback) through service clients, share infrastructure components (PostgreSQL instance, Redis, Kafka), and are orchestrated through service discovery and dependency injection. All external traffic enters through the API Gateway.
## Key Concepts
@@ -16,9 +16,11 @@ The Go Platform is a microservices-based system where each module operates as an
- **Event Bus**: Asynchronous communication channel for events
- **DI Container**: Dependency injection container managing service lifecycle
## Application Bootstrap Sequence
## Service Bootstrap Sequence
The platform follows a well-defined startup sequence that ensures all services are properly initialized and registered.
Each service (API Gateway, Auth, Identity, Authz, Audit, and feature services) follows a well-defined startup sequence. Services bootstrap independently.
### Individual Service Startup
```mermaid
sequenceDiagram
@@ -26,9 +28,9 @@ sequenceDiagram
participant Config
participant Logger
participant DI
participant Registry
participant ModuleLoader
participant ServiceImpl
participant ServiceRegistry
participant DB
participant HTTP
participant gRPC
@@ -39,63 +41,121 @@ sequenceDiagram
Logger-->>Main: Logger ready
Main->>DI: Create DI container
DI->>DI: Register core services
DI->>DI: Register core kernel services
DI-->>Main: DI container ready
Main->>ModuleLoader: Discover modules
ModuleLoader->>ModuleLoader: Scan module directories
ModuleLoader->>ModuleLoader: Load module.yaml files
ModuleLoader-->>Main: Module list
Main->>ServiceImpl: Register service implementation
ServiceImpl->>DI: Register service dependencies
ServiceImpl->>DB: Connect to database
DB-->>ServiceImpl: Connection ready
Main->>Registry: Register modules
Registry->>Registry: Resolve dependencies
Registry->>Registry: Order modules
Registry-->>Main: Ordered modules
loop For each module
Main->>Module: Initialize module
Module->>DI: Register services
Module->>Registry: Register routes
Module->>Registry: Register migrations
end
Main->>Registry: Run migrations
Registry->>Registry: Execute in dependency order
Main->>DB: Run migrations
DB-->>Main: Migrations complete
Main->>ServiceRegistry: Register service
ServiceRegistry->>ServiceRegistry: Register with Consul/K8s
ServiceRegistry-->>Main: Service registered
Main->>gRPC: Start gRPC server
Main->>HTTP: Start HTTP server
HTTP-->>Main: Server ready
gRPC-->>Main: Server ready
Main->>HTTP: Start HTTP server (if needed)
HTTP-->>Main: HTTP server ready
gRPC-->>Main: gRPC server ready
Main->>DI: Start lifecycle
DI->>DI: Execute OnStart hooks
DI-->>Main: All services started
DI-->>Main: Service started
```
### Bootstrap Phases
### Platform Startup (All Services)
```mermaid
sequenceDiagram
participant Docker
participant Gateway
participant AuthSvc
participant IdentitySvc
participant AuthzSvc
participant AuditSvc
participant BlogSvc
participant Registry
participant DB
Docker->>DB: Start PostgreSQL
Docker->>Registry: Start Consul
DB-->>Docker: Database ready
Registry-->>Docker: Registry ready
par Service Startup (in parallel)
Docker->>Gateway: Start API Gateway
Gateway->>Registry: Register
Gateway->>Gateway: Start HTTP server
Gateway-->>Docker: Gateway ready
and
Docker->>AuthSvc: Start Auth Service
AuthSvc->>DB: Connect
AuthSvc->>Registry: Register
AuthSvc->>AuthSvc: Start gRPC server
AuthSvc-->>Docker: Auth Service ready
and
Docker->>IdentitySvc: Start Identity Service
IdentitySvc->>DB: Connect
IdentitySvc->>Registry: Register
IdentitySvc->>IdentitySvc: Start gRPC server
IdentitySvc-->>Docker: Identity Service ready
and
Docker->>AuthzSvc: Start Authz Service
AuthzSvc->>DB: Connect
AuthzSvc->>Registry: Register
AuthzSvc->>AuthzSvc: Start gRPC server
AuthzSvc-->>Docker: Authz Service ready
and
Docker->>AuditSvc: Start Audit Service
AuditSvc->>DB: Connect
AuditSvc->>Registry: Register
AuditSvc->>AuditSvc: Start gRPC server
AuditSvc-->>Docker: Audit Service ready
and
Docker->>BlogSvc: Start Blog Service
BlogSvc->>DB: Connect
BlogSvc->>Registry: Register
BlogSvc->>BlogSvc: Start gRPC server
BlogSvc-->>Docker: Blog Service ready
end
Docker->>Docker: All services ready
```
### Service Bootstrap Phases (Per Service)
1. **Configuration Loading**: Load YAML files, environment variables, and secrets
2. **Foundation Services**: Initialize logger, config provider, DI container
3. **Module Discovery**: Scan and load module manifests
4. **Dependency Resolution**: Build dependency graph and order modules
5. **Module Initialization**: Initialize each module in dependency order
6. **Database Migrations**: Run migrations in dependency order
7. **Service Registration**: Register service with service registry
8. **Server Startup**: Start HTTP and gRPC servers
9. **Lifecycle Hooks**: Execute OnStart hooks for all services
2. **Foundation Services**: Initialize core kernel (logger, config, DI container)
3. **Database Connection**: Connect to database with own connection pool
4. **Service Implementation**: Register service-specific implementations
5. **Database Migrations**: Run service-specific migrations
6. **Service Registration**: Register service with service registry
7. **Server Startup**: Start gRPC server (and HTTP if needed)
8. **Lifecycle Hooks**: Execute OnStart hooks
### Platform Startup Order
1. **Infrastructure**: Start PostgreSQL, Redis, Kafka, Consul
2. **Core Services**: Start Auth, Identity, Authz, Audit services (can start in parallel)
3. **API Gateway**: Start API Gateway (depends on service registry)
4. **Feature Services**: Start Blog, Billing, etc. (can start in parallel)
5. **Health Checks**: All services report healthy to registry
## Request Processing Pipeline
Every HTTP request flows through a standardized pipeline that ensures security, observability, and proper error handling.
Every HTTP request flows through API Gateway first, then to backend services. The pipeline ensures security, observability, and proper error handling.
```mermaid
graph TD
Start([HTTP Request]) --> Auth[Authentication Middleware]
Auth -->|Valid Token| Authz[Authorization Middleware]
Start([HTTP Request]) --> Gateway[API Gateway]
Gateway --> RateLimit[Rate Limiting]
RateLimit -->|Allowed| Auth[Validate JWT via Auth Service]
RateLimit -->|Exceeded| Error0[429 Too Many Requests]
Auth -->|Valid Token| Authz[Check Permission via Authz Service]
Auth -->|Invalid Token| Error1[401 Unauthorized]
Authz -->|Authorized| RateLimit[Rate Limiting]