Files
goplt/docs/content/plan.md
0x1d 38a251968c docs: Align documentation with true microservices architecture
Transform all documentation from modular monolith to true microservices
architecture where core services are independently deployable.

Key Changes:
- Core Kernel: Infrastructure only (no business logic)
- Core Services: Auth, Identity, Authz, Audit as separate microservices
  - Each service has own entry point (cmd/{service}/)
  - Each service has own gRPC server and database schema
  - Services register with Consul for service discovery
- API Gateway: Moved from Epic 8 to Epic 1 as core infrastructure
  - Single entry point for all external traffic
  - Handles routing, JWT validation, rate limiting, CORS
- Service Discovery: Consul as primary mechanism (ADR-0033)
- Database Pattern: Per-service connections with schema isolation

Documentation Updates:
- Updated all 9 architecture documents
- Updated 4 ADRs and created 2 new ADRs (API Gateway, Service Discovery)
- Rewrote Epic 1: Core Kernel & Infrastructure (infrastructure only)
- Rewrote Epic 2: Core Services (Auth, Identity, Authz, Audit as services)
- Updated Epic 3-8 stories for service architecture
- Updated plan.md, playbook.md, requirements.md, index.md
- Updated all epic READMEs and story files

New ADRs:
- ADR-0032: API Gateway Strategy
- ADR-0033: Service Discovery Implementation (Consul)

New Stories:
- Epic 1.7: Service Client Interfaces
- Epic 1.8: API Gateway Implementation
2025-11-06 08:54:19 +01:00

1733 lines
60 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Go Platform Implementation Plan
**"Pluginfriendly SaaS/Enterprise Platform Go Edition"**
> This document outlines a complete, epic-based implementation plan for building the Go platform boilerplate based on the requirements from `playbook.md`.
---
## Executive Summary
This plan breaks down the implementation into **8 epics**, each with specific deliverables and acceptance criteria. The approach prioritizes building a solid foundation (core kernel) before adding feature modules and advanced capabilities.
**Total Estimated Timeline:** 8-12 weeks (depending on team size and parallelization)
**Key Principles:**
- **Hexagonal Architecture** with clear separation between `pkg/` (interfaces) and `internal/` (implementations)
- **Dependency Injection** using `uber-go/fx` for lifecycle management
- **microMicroservices Architecture** - each service is independently deployable from day one
- Core Kernel: Infrastructure only (config, logger, DI, health, metrics, observability)
- Core Services: Auth, Identity, Authz, Audit as separate microservices
- API Gateway: Single entry point for all external traffic (Epic 1)
- **Service Client Interfaces** - all inter-service communication via gRPC/HTTP
- **Service Discovery** - all services register and discover via service registry
- **Database Isolation** - each service has its own database connection pool and schema
- **Plugin-first** architecture supporting both static and dynamic module loading
- **Security-by-Design** with JWT auth, RBAC/ABAC, and audit logging
- **Observability** via OpenTelemetry, Prometheus, and structured logging
---
## Epic 0: Project Setup & Foundation (Week 1)
### Objectives
- Initialize repository structure with proper Go project layout
- Implement configuration management system
- Establish structured logging system
- Set up CI/CD pipeline and development tooling
- Bootstrap dependency injection and application entry point
### Stories
#### 0.1 Project Initialization and Repository Structure
**Goal:** Establish a properly structured Go project with all necessary directories, configuration files, and documentation.
**Deliverables:**
- Initialize Go module with correct module path
- Create complete directory structure (`cmd/`, `internal/`, `pkg/`, `modules/`, `config/`, etc.)
- Add `.gitignore` for Go projects
- Create comprehensive `README.md` with project overview, setup instructions, and architecture overview
- Set up basic project documentation structure
**Acceptance Criteria:**
- `go mod init` creates module with correct path
- All directories are in place following Go best practices
- `.gitignore` excludes build artifacts, dependencies, and IDE files
- `README.md` provides clear project overview and setup instructions
- Project structure matches architecture documentation
#### 0.2 Configuration Management System
**Goal:** Implement a flexible configuration system that loads settings from YAML files, environment variables, and supports type-safe access.
**Deliverables:**
- `ConfigProvider` interface in `pkg/config/` for abstraction
- Viper-based implementation in `internal/config/` that:
- Loads `config/default.yaml` as baseline
- Merges environment-specific YAML files (development/production)
- Applies environment variable overrides
- Supports nested configuration keys
- Provides type-safe getters (GetString, GetInt, GetBool, etc.)
- Supports unmarshaling into structs
- Configuration files (`config/default.yaml`, `config/development.yaml`, `config/production.yaml`)
- Configuration loader with validation
**Acceptance Criteria:**
- Configuration loads from YAML files successfully
- Environment variables override YAML values
- Type-safe getters work correctly
- Configuration can be unmarshaled into structs
- Config system is injectable via DI container
- All modules can access configuration through interface
#### 0.3 Structured Logging System
**Goal:** Implement a production-ready logging system with structured JSON output, request correlation, and configurable log levels.
**Deliverables:**
- `Logger` interface in `pkg/logger/` with methods for all log levels
- Zap-based implementation in `internal/logger/` with:
- Structured JSON logging for production
- Human-readable logging for development
- Configurable log levels (debug, info, warn, error)
- Request-scoped fields support
- Context-aware logging
- Request ID middleware for Gin that:
- Generates unique request IDs
- Adds request ID to all logs within request context
- Returns request ID in response headers
- Global logger export via `pkg/logger` package
**Acceptance Criteria:**
- Logs are structured JSON in production mode
- Log levels are configurable and respected
- Request IDs are generated and included in all logs
- Logger can be injected via DI container
- All modules can use logger through interface
- Request correlation works across service boundaries
#### 0.4 CI/CD Pipeline and Development Tooling
**Goal:** Establish automated testing, linting, and build processes with a developer-friendly Makefile.
**Deliverables:**
- GitHub Actions workflow (`.github/workflows/ci.yml`) that:
- Sets up Go environment
- Caches Go modules
- Runs linters (golangci-lint or staticcheck)
- Runs unit tests
- Builds binary artifacts
- Validates code formatting
- Comprehensive `Makefile` with commands:
- `make test` - run all tests
- `make lint` - run linters
- `make build` - build platform binary
- `make docker-build` - build Docker image
- `make clean` - clean build artifacts
- `make fmt` - format code
- `make generate` - run code generation
**Acceptance Criteria:**
- CI pipeline runs on every push and PR
- All linting checks pass
- Tests run successfully (even if empty initially)
- Binary builds successfully
- Docker image builds successfully
- Makefile commands work as expected
- CI pipeline fails fast on errors
#### 0.5 Dependency Injection and Application Bootstrap
**Goal:** Set up dependency injection container using Uber FX and create the application entry point that initializes the platform.
**Deliverables:**
- DI container in `internal/di/container.go` that:
- Initializes Uber FX container
- Registers Config and Logger providers
- Provides lifecycle management hooks
- Supports service overrides for testing
- Application entry point in `cmd/platform/main.go` that:
- Loads configuration
- Initializes DI container
- Sets up basic application lifecycle
- Starts minimal HTTP server (placeholder for Epic 1)
- Handles graceful shutdown
**Acceptance Criteria:**
- DI container initializes successfully
- Config and Logger are provided via DI
- Application starts and shuts down gracefully
- Lifecycle hooks work correctly
- Services can be overridden for testing
- Application compiles and runs successfully
### Deliverables
- ✅ Repository structure in place
- ✅ Configuration system loads YAML files and env vars
- ✅ Structured logging works
- ✅ CI pipeline runs linting and builds binary
- ✅ Basic DI container initialized
### Acceptance Criteria
- `go build ./cmd/platform` succeeds
- `go test ./...` runs (even if tests are empty)
- CI pipeline passes on empty commit
- Config loads from `config/default.yaml`
---
## Epic 1: Core Kernel & Infrastructure (Week 2-3)
### Objectives
- Extend DI container to support core kernel infrastructure only (no business services)
- Implement API Gateway as core infrastructure component
- Create service client interfaces for microservices architecture
- Build health monitoring and metrics system
- Create error handling and error bus
- Establish HTTP/gRPC server foundation for services
- Integrate OpenTelemetry for distributed tracing
- Basic service registry implementation
**Note:** This epic focuses on infrastructure only. Business services (Auth, Identity, Authz, Audit) are implemented in Epic 2 as separate microservices.
### Stories
#### 1.1 Enhanced Dependency Injection Container
**Goal:** Extend the DI container to provide core kernel infrastructure services only (no business logic services).
**Deliverables:**
- Extended `internal/di/container.go` with:
- Registration of core kernel services only
- Lifecycle management via FX
- Service override support for testing
- `internal/di/providers.go` with provider functions:
- `ProvideConfig()` - configuration provider
- `ProvideLogger()` - logger service
- `ProvideHealthCheckers()` - health check registry
- `ProvideMetrics()` - Prometheus metrics registry
- `ProvideErrorBus()` - error bus service
- `ProvideTracer()` - OpenTelemetry tracer
- `ProvideServiceRegistry()` - service registry interface
- `internal/di/core_module.go` exporting `CoreModule` fx.Option that provides all core kernel services
**Note:** Database, Auth, Identity, Authz, Audit are NOT in core kernel - they are separate services implemented in Epic 2.
**Acceptance Criteria:**
- All core kernel services are provided via DI container
- Services are initialized in correct dependency order
- Lifecycle hooks work for all services
- Services can be overridden for testing
- DI container compiles without errors
- No business logic services in core kernel
#### 1.2 Database Client Foundation
**Goal:** Set up database client foundation for services. Each service will have its own database connection and schema.
**Deliverables:**
- Database client wrapper in `internal/infra/database/client.go`:
- `NewEntClient(dsn string, schema string) (*ent.Client, error)` - supports schema isolation
- Connection pooling configuration (max connections, idle timeout)
- Migration runner wrapper
- Database health check integration
- Per-service connection pool management
- Database configuration in `config/default.yaml` with:
- Connection string (DSN)
- Connection pool settings per service
- Schema isolation configuration
- Database client factory for creating service-specific clients
**Note:** Core domain entities (User, Role, Permission, AuditLog) are implemented in Epic 2 as part of their respective services (Identity, Authz, Audit).
**Acceptance Criteria:**
- Database client connects to PostgreSQL with schema support
- Connection pooling is configured correctly
- Database health check works
- Multiple services can connect to same database instance with different schemas
- Each service manages its own connection pool
#### 1.3 Health Monitoring and Metrics System
**Goal:** Implement comprehensive health checks and Prometheus metrics for monitoring platform health and performance.
**Deliverables:**
- Health check system:
- `HealthChecker` interface in `pkg/health/health.go`
- Health check registry in `internal/health/registry.go`:
- Register multiple health checkers
- `GET /healthz` endpoint (liveness probe)
- `GET /ready` endpoint (readiness probe with database check)
- Individual component health checks
- Prometheus metrics system:
- Metrics registry setup
- HTTP request duration histogram
- HTTP request counter (by method, path, status)
- Database query duration histogram (via Ent interceptor)
- Error counter (by type)
- `GET /metrics` endpoint (Prometheus format)
- Integration with HTTP server and DI container
**Acceptance Criteria:**
- `/healthz` returns 200 when service is alive
- `/ready` checks database connectivity and returns appropriate status
- `/metrics` exposes Prometheus metrics in correct format
- All HTTP requests are measured
- Database queries are instrumented
- Metrics are registered in DI container
- Health checks can be extended by modules
#### 1.4 Error Handling and Error Bus
**Goal:** Implement centralized error handling with an error bus that captures, logs, and optionally reports all application errors.
**Deliverables:**
- Error bus interface in `pkg/errorbus/errorbus.go`:
- `ErrorPublisher` interface with `Publish(err error)` method
- Channel-based error bus implementation in `internal/errorbus/channel_bus.go`:
- Buffered channel for error publishing
- Background goroutine consumes errors
- Logs all errors with context
- Optional: Sentry integration (Epic 6 placeholder)
- Panic recovery middleware:
- Recovers from panics in HTTP handlers
- Publishes panics to error bus
- Returns appropriate HTTP error responses
- Integration with DI container and HTTP middleware stack
**Acceptance Criteria:**
- Errors are captured and logged via error bus
- Panics are recovered and logged
- HTTP handlers return proper error responses
- Error bus is injectable via DI
- Error context (request ID, user ID) is preserved
- Background error consumer works correctly
#### 1.5 HTTP/gRPC Server Foundation
**Goal:** Create HTTP and gRPC server foundation that services can use. Each service will have its own server instance.
**Deliverables:**
- HTTP server foundation in `internal/server/http.go`:
- Gin router initialization helper
- Common middleware stack:
- Request ID generator (unique per request)
- Structured logging middleware (logs all requests)
- Panic recovery → error bus
- Prometheus metrics collection
- CORS support (configurable)
- Request timeout handling
- Response compression
- Server lifecycle management
- gRPC server foundation in `internal/server/grpc.go`:
- gRPC server initialization
- Interceptor support (logging, tracing, metrics)
- Lifecycle management
- FX lifecycle integration for both HTTP and gRPC servers
**Note:** Services (Auth, Identity, etc.) will use these foundations to create their own server instances in Epic 2.
**Acceptance Criteria:**
- HTTP server foundation is reusable by services
- gRPC server foundation is reusable by services
- All middleware executes in correct order
- Request IDs are generated and logged
- Metrics are collected for all requests
- Graceful shutdown works correctly
- Servers are configurable via config system
#### 1.6 OpenTelemetry Distributed Tracing
**Goal:** Integrate OpenTelemetry for distributed tracing across all services to enable observability in production.
**Deliverables:**
- OpenTelemetry setup in `internal/observability/tracer.go`:
- TracerProvider initialization
- Export to stdout (development mode)
- Export to OTLP collector (production mode)
- Trace context propagation
- HTTP instrumentation middleware:
- Automatic span creation for HTTP requests
- Trace context propagation via headers
- Span attributes (method, path, status code, etc.)
- gRPC instrumentation:
- gRPC interceptor for automatic span creation
- Trace context propagation via gRPC metadata
- Database instrumentation:
- Ent interceptor for database queries
- Query spans with timing and parameters
- Integration with logger (include trace ID in logs)
**Acceptance Criteria:**
- HTTP requests create OpenTelemetry spans
- gRPC calls create OpenTelemetry spans
- Database queries are traced
- Trace context propagates across service boundaries
- Trace IDs are included in logs
- Traces export correctly to configured backend
- Tracing works in both development and production modes
- Tracing has minimal performance impact
#### 1.7 Service Client Interfaces
**Goal:** Create service client interfaces for all core services to enable microservices communication.
**Deliverables:**
- Service client interfaces in `pkg/services/` for all core services:
- `IdentityServiceClient` - User and identity operations
- `AuthServiceClient` - Authentication operations
- `AuthzServiceClient` - Authorization operations
- `AuditServiceClient` - Audit logging
- Service client factory in `internal/services/factory.go`:
- Create gRPC clients (primary)
- Create HTTP clients (fallback)
- Support service registry integration
- Handle client lifecycle and connection pooling
- Configuration for service protocol selection
**Acceptance Criteria:**
- Service client interfaces are defined for all core services
- Service factory creates gRPC clients
- Service factory creates HTTP clients (fallback)
- Service clients are injectable via DI
- Configuration supports protocol selection
- All inter-service communication goes through service clients
#### 1.8 API Gateway Implementation
**Goal:** Implement API Gateway as core infrastructure component that routes all external traffic to backend services.
**Deliverables:**
- API Gateway service entry point: `cmd/api-gateway/main.go`
- Gateway implementation in `services/gateway/internal/`:
- Request routing to backend services via service discovery
- JWT token validation via Auth Service client
- Permission checking via Authz Service client (optional, for route-level auth)
- Rate limiting middleware (per-user and per-IP)
- CORS support
- Request/response transformation
- Load balancing across service instances
- Gateway configuration in `config/default.yaml`:
- Route definitions (path → service mapping)
- Rate limiting configuration
- CORS configuration
- Integration with service registry for service discovery
- Health check endpoint for gateway
**Acceptance Criteria:**
- API Gateway routes requests to backend services correctly
- JWT validation works via Auth Service
- Rate limiting works correctly
- CORS is configurable and works
- Service discovery integration works
- Gateway is independently deployable
- Gateway has health check endpoint
- All external traffic goes through gateway
### Deliverables
- ✅ DI container with core kernel services only
- ✅ Database client foundation (per-service connections)
- ✅ Health and metrics endpoints functional
- ✅ Error bus captures and logs errors
- ✅ HTTP/gRPC server foundation for services
- ✅ Basic observability with OpenTelemetry
- ✅ Service client interfaces for microservices
- ✅ API Gateway service (core infrastructure)
- ✅ Basic service registry implementation
### Acceptance Criteria
- `GET /healthz` returns 200 for all services
- `GET /ready` checks service health
- `GET /metrics` exposes Prometheus metrics
- Panic recovery logs errors via error bus
- HTTP/gRPC requests are traced with OpenTelemetry
- API Gateway routes requests to backend services
- Service client interfaces are defined
- No business logic services in Epic 1
---
## Epic 2: Core Services (Authentication & Authorization) (Week 3-5)
### Objectives
- Separate Auth, Identity, Authz, and Audit into independent microservices
- Each service has its own entry point, database connection, and gRPC server
- Implement complete JWT-based authentication system (Auth Service)
- Build comprehensive identity management with user lifecycle (Identity Service)
- Create role-based access control (RBAC) system (Authz Service)
- Implement audit logging system (Audit Service)
- All services communicate via service clients (gRPC/HTTP)
- All services register with service registry
**Note:** This epic transforms the monolithic core into separate, independently deployable services.
- Add comprehensive audit logging for security compliance
- Provide database seeding for initial setup
### Stories
#### 2.1 Auth Service - JWT Authentication
**Goal:** Implement Auth Service as an independent microservice with complete JWT-based authentication system, access tokens, refresh tokens, and secure token management.
**Deliverables:**
- Auth Service entry point: `cmd/auth-service/main.go`
- Service implementation in `services/auth/internal/`:
- gRPC server for Auth Service
- HTTP endpoints (optional, for compatibility)
- Authentication interfaces in `pkg/services/auth.go`:
- `AuthServiceClient` interface for authentication operations
- Service client implementation (gRPC/HTTP)
- JWT implementation in `services/auth/internal/jwt_auth.go`:
- Generate short-lived access tokens (15 minutes)
- Generate long-lived refresh tokens (7 days)
- Token signature verification
- Token expiration validation
- Claims extraction
- Authentication middleware in `internal/auth/middleware.go`:
- Extract JWT from `Authorization: Bearer <token>` header
- Verify token validity
- Inject authenticated user into request context
- Helper function: `auth.FromContext(ctx) *User`
- gRPC service definition: `services/auth/api/auth.proto`
- Authentication endpoints (gRPC):
- `Login(email, password)` - Authenticate user and return tokens
- `RefreshToken(refresh_token)` - Refresh access token using refresh token
- `ValidateToken(token)` - Validate JWT token (used by API Gateway)
- Password validation against stored hashes (via Identity Service)
- Database connection: Own connection pool and schema (`auth_schema`)
- Service registration: Register with service registry
- Integration with Identity Service: Use `IdentityServiceClient` for user lookup
**Acceptance Criteria:**
- Auth Service is independently deployable
- Service has own entry point (`cmd/auth-service/`)
- gRPC server starts and serves authentication requests
- Users can login via gRPC and receive access and refresh tokens
- Access tokens expire after configured duration
- Refresh tokens can be used to obtain new access tokens
- Token validation works (used by API Gateway)
- Invalid tokens are rejected with appropriate errors
- Service registers with service registry
- Service uses Identity Service client for user lookup
- Service has own database connection and schema
#### 2.2 Identity Service - User Management
**Goal:** Implement Identity Service as an independent microservice with complete user identity management, registration, email verification, password management, and user CRUD operations.
**Deliverables:**
- Identity Service entry point: `cmd/identity-service/main.go`
- Service implementation in `services/identity/internal/`:
- gRPC server for Identity Service
- HTTP endpoints (optional, for compatibility)
- Identity interfaces in `pkg/services/identity.go`:
- `IdentityServiceClient` interface for user operations
- Service client implementation (gRPC/HTTP)
- User repository implementation in `services/identity/internal/user_repo.go`:
- CRUD operations using Ent
- Password hashing (bcrypt or argon2)
- Email uniqueness validation
- User lookup by ID and email
- User service implementation in `internal/identity/user_service.go`:
- User registration with email verification token generation
- Email verification flow
- Password reset flow (token-based, time-limited)
- Password change with old password verification
- User profile updates
- gRPC service definition: `services/identity/api/identity.proto`
- User management endpoints (gRPC):
- `CreateUser(user)` - Register new user
- `GetUser(id)` - Get user profile
- `UpdateUser(id, user)` - Update user profile
- `DeleteUser(id)` - Delete user (admin only)
- `GetUserByEmail(email)` - Get user by email
- `VerifyEmail(token)` - Verify email with token
- `RequestPasswordReset(email)` - Request password reset
- `ResetPassword(token, new_password)` - Reset password
- `ChangePassword(user_id, old_password, new_password)` - Change password
- Database connection: Own connection pool and schema (`identity_schema`)
- Ent schema: User entity in `services/identity/ent/schema/user.go`
- Service registration: Register with service registry
- Integration with email notification system (Epic 5) via event bus
**Acceptance Criteria:**
- Identity Service is independently deployable
- Service has own entry point (`cmd/identity-service/`)
- gRPC server starts and serves user management requests
- Users can register via gRPC with email and password
- Passwords are securely hashed
- Email verification tokens are generated and validated
- Password reset flow works end-to-end
- Users can update their profiles via gRPC
- Service registers with service registry
- Service has own database connection and schema
- User entity is properly defined in Ent schema
#### 2.3 Role-Based Access Control (RBAC) System
**Goal:** Implement a complete RBAC system with permissions, role management, and authorization middleware.
**Deliverables:**
- Permission system in `pkg/perm/perm.go`:
- `Permission` type (string format: "module.resource.action")
- Core permission constants (system, user, role permissions)
- Permission validation utilities
- Permission resolver interface in `pkg/perm/resolver.go`:
- `HasPermission(ctx, userID, perm)` method
- `GetUserPermissions(ctx, userID)` method
- Permission resolver implementation in `internal/perm/in_memory_resolver.go`:
- Load user roles from database
- Load role permissions from database
- Check user permissions (with caching)
- Permission inheritance via roles
- Authorization interface in `pkg/auth/authz.go`:
- `Authorizer` interface with `Authorize(ctx, perm)` method
- RBAC authorizer implementation in `internal/auth/rbac_authorizer.go`:
- Extract user from context
- Resolve user permissions
- Check permission against required permission
- Return authorization errors
- Authorization middleware:
- `RequirePermission(perm Permission) gin.HandlerFunc` decorator
- Integration with route registration
- Proper error responses for unauthorized access
**Acceptance Criteria:**
- Permissions are defined and can be checked
- Users inherit permissions through roles
- Authorization middleware protects routes
- Unauthorized requests return 403 errors
- Permission checks are cached for performance
- Permission system is extensible by modules
#### 2.4 Role Management API
**Goal:** Provide complete API for managing roles, assigning permissions to roles, and assigning roles to users.
**Deliverables:**
- Role repository in `internal/identity/role_repo.go`:
- CRUD operations for roles
- Assign permissions to roles (many-to-many)
- Assign roles to users (many-to-many)
- List roles with permissions
- List users with roles
- Role management API endpoints:
- `POST /api/v1/roles` - Create new role
- `GET /api/v1/roles` - List all roles (with pagination)
- `GET /api/v1/roles/:id` - Get role details with permissions
- `PUT /api/v1/roles/:id` - Update role
- `DELETE /api/v1/roles/:id` - Delete role
- `POST /api/v1/roles/:id/permissions` - Assign permissions to role
- `DELETE /api/v1/roles/:id/permissions/:permId` - Remove permission from role
- `POST /api/v1/users/:id/roles` - Assign roles to user
- `DELETE /api/v1/users/:id/roles/:roleId` - Remove role from user
- Authorization on all endpoints (admin only)
- Validation and error handling
**Acceptance Criteria:**
- Admin users can create and manage roles
- Permissions can be assigned to roles
- Roles can be assigned to users
- Role changes affect user permissions immediately
- All role operations are audited
- API endpoints are protected with proper permissions
#### 2.5 Audit Logging System
**Goal:** Implement comprehensive audit logging that records all security-sensitive actions for compliance and security monitoring.
**Deliverables:**
- Audit interface in `pkg/audit/audit.go`:
- `Auditor` interface with `Record(ctx, action)` method
- `AuditAction` struct with actor, action, target, metadata
- Audit implementation in `internal/audit/ent_auditor.go`:
- Write audit logs to `audit_log` table
- Capture actor from request context
- Include request metadata (ID, IP, user agent, timestamp)
- Store action details and target information
- Support JSON metadata for flexible logging
- Audit middleware:
- Intercept all authenticated requests
- Record action (HTTP method + path)
- Extract user and request context
- Store audit log entry
- Integration with authentication endpoints:
- Log login attempts (success and failure)
- Log password changes
- Log role assignments and removals
- Log permission changes
- Log user registration
- Audit log query API (admin only):
- `GET /api/v1/audit-logs` - Query audit logs with filters
**Acceptance Criteria:**
- All authenticated actions are logged
- Audit logs include complete context (actor, action, target, metadata)
- Audit logs are immutable (no updates/deletes)
- Audit logs can be queried and filtered
- Audit logging has minimal performance impact
- Audit logs are stored securely
#### 2.6 Database Seeding and Initialization
**Goal:** Provide database seeding functionality to create initial admin user, default roles, and core permissions.
**Deliverables:**
- Seed script in `internal/seed/seed.go`:
- Create default admin user (if doesn't exist)
- Create default roles (admin, user, guest)
- Assign core permissions to roles
- Set up initial role hierarchy
- Seed command in `cmd/seed/main.go`:
- Command-line interface for seeding
- Idempotent seeding (safe to run multiple times)
- Configuration via environment variables
- Integration with application startup (optional):
- Auto-seed on first startup in development
- Manual seeding in production
**Acceptance Criteria:**
- Seed script creates admin user successfully
- Default roles are created with proper permissions
- Seeding is idempotent (can run multiple times safely)
- Seed script can be run via CLI
- Admin user can login and manage system
### Deliverables
- ✅ JWT authentication with access/refresh tokens
- ✅ User CRUD with email verification
- ✅ Role and permission management
- ✅ Authorization middleware
- ✅ Audit logging for all actions
- ✅ Seed script for initial data
### Acceptance Criteria
- User can register and login
- JWT tokens are validated on protected routes
- Users without permission get 403
- All actions are logged in audit table
- Admin can create roles and assign permissions
- Integration test: user without permission cannot access protected resource
---
## Epic 3: Module Framework (Week 4-5)
### Objectives
- Design and implement complete module system interface
- Build module registry with dependency resolution
- Create permission code generation from module manifests
- Implement module loader supporting static and dynamic loading
- Add module lifecycle management and initialization
- Provide CLI tooling for module management
- Implement service registry and discovery for microservices
### Stories
#### 3.1 Module System Interface and Registry
**Goal:** Design and implement the complete module system interface with registration, dependency resolution, and lifecycle management.
**Deliverables:**
- Module interface in `pkg/module/module.go`:
- `IModule` interface with Name(), Version(), Dependencies(), Init(), Migrations()
- Optional lifecycle hooks: OnStart() and OnStop()
- Module manifest in `pkg/module/manifest.go`:
- `Manifest` struct with Name, Version, Dependencies, Permissions, Routes
- `module.yaml` schema definition
- Module registry in `internal/registry/registry.go`:
- Thread-safe module map
- Register(), All(), Get() functions
- Dependency validation (check dependencies are satisfied)
- Duplicate name detection
- Version compatibility checking
**Acceptance Criteria:**
- Modules can register via `registry.Register()`
- Registry validates dependencies
- Registry prevents duplicate registrations
- Module interface is extensible
#### 3.2 Permission Code Generation System
**Goal:** Create automated permission code generation from module manifests to ensure type-safe permission constants.
**Deliverables:**
- Permission generation script in `scripts/generate-permissions.go`:
- Scan all `modules/*/module.yaml` files
- Extract permissions from manifests
- Generate `pkg/perm/generated.go` with Permission constants
- Support for multiple modules
- Go generate integration:
- `//go:generate` directive in `pkg/perm/perm.go`
- `make generate` command in Makefile
- Automatic generation on build
**Acceptance Criteria:**
- Permission constants are generated from `module.yaml`
- Generated code is type-safe
- Code generation runs automatically
- Permissions follow naming convention
#### 3.3 Module Loader and Initialization
**Goal:** Implement module loading (static and dynamic) with dependency resolution and automatic initialization.
**Deliverables:**
- Module loader in `internal/pluginloader/loader.go`:
- Support static registration (preferred method)
- Optional: Go plugin loading (`.so` files)
- Module discovery from `modules/*/module.yaml`
- Static loader in `internal/pluginloader/static_loader.go`:
- Import modules via side-effect imports
- Collect all registered modules
- Optional plugin loader in `internal/pluginloader/plugin_loader.go`:
- Scan `./plugins/*.so` files
- Load via `plugin.Open()`
- Extract and validate module symbols
- Module initializer in `internal/module/initializer.go`:
- Collect all registered modules
- Resolve dependency order (topological sort)
- Initialize each module's `Init()` fx.Option
- Merge all options into main fx container
- Run migrations in dependency order
- Handle errors gracefully
- FX lifecycle integration:
- Call `OnStart()` during app startup
- Call `OnStop()` during graceful shutdown
**Acceptance Criteria:**
- Modules load in correct dependency order
- Module migrations run automatically
- Module initialization integrates with FX
- Lifecycle hooks work correctly
- Dependency resolution handles cycles
#### 3.4 Module Management CLI Tool
**Goal:** Provide CLI tooling for managing modules, validating dependencies, and testing module loading.
**Deliverables:**
- CLI tool in `cmd/platformctl/main.go`:
- `platformctl modules list` - List all loaded modules with versions
- `platformctl modules validate` - Validate module dependencies
- `platformctl modules test <module>` - Test module loading
- `platformctl modules info <module>` - Show module details
- Makefile integration:
- `make install-cli` - Install CLI tool
- `make cli` - Build CLI tool
**Acceptance Criteria:**
- CLI tool lists all modules
- Dependency validation works
- Module testing works
- CLI is installable and usable
#### 3.5 Service Registry and Discovery
**Goal:** Implement a service registry that enables service discovery for microservices.
**Deliverables:**
- Service registry interface in `pkg/services/registry.go`:
- `ServiceRegistry` interface with Register, Discover, GetService methods
- `ServiceInfo` struct with service metadata
- Consul registry in `internal/services/registry/consul.go`:
- Consul integration for service discovery
- Service registration and discovery
- Health status tracking
- Kubernetes service discovery in `internal/services/registry/kubernetes.go`:
- Kubernetes service discovery
- Service health checking
- Service auto-registration on startup
- Configuration for registry type selection
**Acceptance Criteria:**
- Service registry interface is defined
- Consul registry works correctly
- Kubernetes registry works correctly
- Services are auto-registered on startup
- Service discovery works
- Health checking works
- Registry is configurable
### Deliverables
- ✅ Module interface and registration system
- ✅ Static module registry working
- ✅ Permission code generation tool
- ✅ Module loader with dependency resolution
- ✅ Module initialization in main app
- ✅ CLI tool for module management
- ✅ Service registry for discovery
### Acceptance Criteria
- Modules can register via `registry.Register()`
- Permission constants are generated from `module.yaml`
- Modules load in correct dependency order
- Module migrations run on startup
- `platformctl modules list` shows all modules
- Integration test: load multiple modules and verify initialization
---
## Epic 4: Sample Feature Module (Blog) (Week 5-6)
### Objectives
- Create a complete sample module (Blog) to demonstrate the framework
- Show how to add routes, permissions, database entities, and services
- Provide reference implementation for future developers
### Tasks
#### 4.1 Blog Module Structure
- [ ] Create `modules/blog/` directory:
```
modules/blog/
├── go.mod
├── module.yaml
├── internal/
│ ├── api/
│ │ └── handler.go
│ ├── domain/
│ │ ├── post.go
│ │ └── post_repo.go
│ └── service/
│ └── post_service.go
└── pkg/
└── module.go
```
- [ ] Initialize `go.mod`:
```bash
cd modules/blog
go mod init github.com/yourorg/blog
```
#### 4.2 Module Manifest
- [ ] Create `modules/blog/module.yaml`:
```yaml
name: blog
version: 0.1.0
dependencies:
- core >= 1.0.0
permissions:
- blog.post.create
- blog.post.read
- blog.post.update
- blog.post.delete
routes:
- method: POST
path: /api/v1/blog/posts
permission: blog.post.create
- method: GET
path: /api/v1/blog/posts/:id
permission: blog.post.read
- method: PUT
path: /api/v1/blog/posts/:id
permission: blog.post.update
- method: DELETE
path: /api/v1/blog/posts/:id
permission: blog.post.delete
- method: GET
path: /api/v1/blog/posts
permission: blog.post.read
```
#### 4.3 Blog Domain Model
- [ ] Create `modules/blog/internal/domain/post.go`:
```go
type Post struct {
ID string
Title string
Content string
AuthorID string
CreatedAt time.Time
UpdatedAt time.Time
}
```
- [ ] Create Ent schema `modules/blog/internal/ent/schema/post.go`:
- Fields: title, content, author_id (FK to user)
- Indexes: author_id, created_at
- [ ] Generate Ent code for blog module
#### 4.4 Blog Repository
- [ ] Create `modules/blog/internal/domain/post_repo.go`:
```go
type PostRepository interface {
Create(ctx context.Context, p *Post) (*Post, error)
FindByID(ctx context.Context, id string) (*Post, error)
FindByAuthor(ctx context.Context, authorID string) ([]*Post, error)
Update(ctx context.Context, p *Post) error
Delete(ctx context.Context, id string) error
}
```
- [ ] Implement using Ent client (shared from core)
#### 4.5 Blog Service
- [ ] Create `modules/blog/internal/service/post_service.go`:
- Business logic for creating/updating posts
- Validation (title length, content requirements)
- Authorization checks (author can only update own posts)
- Integration with audit system
#### 4.6 Blog API Handlers
- [ ] Create `modules/blog/internal/api/handler.go`:
- `POST /api/v1/blog/posts` - Create post
- `GET /api/v1/blog/posts/:id` - Get post
- `GET /api/v1/blog/posts` - List posts (with pagination)
- `PUT /api/v1/blog/posts/:id` - Update post
- `DELETE /api/v1/blog/posts/:id` - Delete post
- [ ] Use authorization middleware:
```go
grp.Use(auth.RequirePermission(perm.BlogPostCreate))
```
- [ ] Register handlers in module's `Init()`
#### 4.7 Blog Module Implementation
- [ ] Create `modules/blog/pkg/module.go`:
```go
type BlogModule struct{}
func (b BlogModule) Name() string { return "blog" }
func (b BlogModule) Version() string { return "0.1.0" }
func (b BlogModule) Dependencies() []string { return nil }
func (b BlogModule) Init() fx.Option {
return fx.Options(
fx.Provide(NewPostRepo),
fx.Provide(NewPostService),
fx.Invoke(RegisterHandlers),
)
}
func (b BlogModule) Migrations() []func(*ent.Client) error {
return []func(*ent.Client) error{
func(c *ent.Client) error {
return c.Schema.Create(context.Background())
},
}
}
var Module BlogModule
func init() {
registry.Register(Module)
}
```
#### 4.8 Integration
- [ ] Update main `go.mod` to include blog module:
```go
replace github.com/yourorg/blog => ./modules/blog
```
- [ ] Import blog module in `cmd/platform/main.go`:
```go
import _ "github.com/yourorg/blog/pkg"
```
- [ ] Run permission generation: `make generate`
- [ ] Verify blog permissions are generated
#### 4.9 Tests
- [ ] Create integration test `modules/blog/internal/api/handler_test.go`:
- Test creating post with valid permission
- Test creating post without permission (403)
- Test updating own post vs other's post
- Test pagination
- [ ] Add unit tests for service and repository
### Deliverables
- ✅ Complete Blog module with CRUD operations
- ✅ Module registered and loaded by core
- ✅ Permissions generated and used
- ✅ Routes protected with authorization
- ✅ Database migrations run
- ✅ Integration tests passing
### Acceptance Criteria
- Blog module loads on platform startup
- `POST /api/v1/blog/posts` requires `blog.post.create` permission
- User can create, read, update, delete posts
- Authorization enforced (users can only edit own posts)
- Integration test: full CRUD flow works
- Audit logs record all blog actions
---
## Epic 5: Infrastructure Adapters (Week 6-7)
### Objectives
- Implement infrastructure adapters (cache, queue, blob storage, email)
- Make adapters swappable via interfaces
- Add scheduler/background jobs system
- Implement event bus (in-process and Kafka)
- Add gRPC service definitions and clients for microservices communication
### Tasks
#### 5.1 Cache (Redis)
- [ ] Install `github.com/redis/go-redis/v9`
- [ ] Create `pkg/infra/cache/cache.go` interface:
```go
type Cache interface {
Get(ctx context.Context, key string) ([]byte, error)
Set(ctx context.Context, key string, value []byte, ttl time.Duration) error
Delete(ctx context.Context, key string) error
}
```
- [ ] Implement `internal/infra/cache/redis_cache.go`
- [ ] Add Redis config to `config/default.yaml`
- [ ] Register in DI container
- [ ] Add cache middleware for selected routes (optional)
#### 5.2 Event Bus
- [ ] Create `pkg/eventbus/eventbus.go` interface:
```go
type EventBus interface {
Publish(ctx context.Context, topic string, event Event) error
Subscribe(topic string, handler EventHandler) error
}
```
- [ ] Implement `internal/infra/bus/inprocess_bus.go`:
- Channel-based in-process bus
- Used for testing and development
- [ ] Implement `internal/infra/bus/kafka_bus.go`:
- Install `github.com/segmentio/kafka-go`
- Producer for publishing
- Consumer groups for subscribing
- Error handling and retries
- [ ] Add Kafka config to `config/default.yaml`
- [ ] Register bus in DI container (switchable via config)
- [ ] Add core events:
- `platform.user.created`
- `platform.user.updated`
- `platform.role.assigned`
- `platform.permission.granted`
#### 5.3 Blob Storage
- [ ] Install `github.com/aws/aws-sdk-go-v2/service/s3`
- [ ] Create `pkg/infra/blob/blob.go` interface:
```go
type BlobStore interface {
Upload(ctx context.Context, key string, data []byte) error
Download(ctx context.Context, key string) ([]byte, error)
Delete(ctx context.Context, key string) error
GetSignedURL(ctx context.Context, key string, ttl time.Duration) (string, error)
}
```
- [ ] Implement `internal/infra/blob/s3_store.go`
- [ ] Add S3 config to `config/default.yaml`
- [ ] Register in DI container
- [ ] Add file upload endpoint: `POST /api/v1/files/upload`
#### 5.4 Email Notification
- [ ] Install `github.com/go-mail/mail`
- [ ] Create `pkg/notification/notification.go` interface:
```go
type Notifier interface {
SendEmail(ctx context.Context, to, subject, body string) error
SendSMS(ctx context.Context, to, message string) error
}
```
- [ ] Implement `internal/infra/email/smtp_notifier.go`:
- SMTP configuration
- HTML email support
- Templates for common emails (verification, password reset)
- [ ] Add email config to `config/default.yaml`
- [ ] Integrate with identity service:
- Send verification email on registration
- Send password reset email
- [ ] Register in DI container
#### 5.5 Scheduler & Background Jobs
- [ ] Install `github.com/robfig/cron/v3` and `github.com/hibiken/asynq`
- [ ] Create `pkg/scheduler/scheduler.go` interface:
```go
type Scheduler interface {
Cron(spec string, job JobFunc) error
Enqueue(queue string, payload any) error
}
```
- [ ] Implement `internal/infra/scheduler/asynq_scheduler.go`:
- Redis-backed job queue
- Cron jobs for periodic tasks
- Job retries and backoff
- Job status tracking
- [ ] Create `internal/infra/scheduler/job_registry.go`:
- Register jobs from modules
- Start job processor on app startup
- [ ] Add example jobs:
- Cleanup expired tokens (daily)
- Send digest emails (weekly)
- [ ] Add job monitoring endpoint: `GET /api/v1/jobs/status`
#### 5.6 Secret Store Integration
- [ ] Create `pkg/infra/secret/secret.go` interface:
```go
type SecretStore interface {
GetSecret(ctx context.Context, key string) (string, error)
}
```
- [ ] Implement `internal/infra/secret/vault_store.go` (HashiCorp Vault):
- Install `github.com/hashicorp/vault/api`
- Support KV v2 secrets
- [ ] Implement `internal/infra/secret/aws_secrets.go` (AWS Secrets Manager):
- Install `github.com/aws/aws-sdk-go-v2/service/secretsmanager`
- [ ] Integrate with config loader:
- Overlay secrets on top of file/env config
- Load secrets lazily (cache)
- [ ] Register in DI container (optional, via config)
#### 5.7 gRPC Service Definitions and Clients
**Goal:** Implement gRPC service definitions and clients to enable microservices communication, allowing modules to be extracted as independent services.
**Deliverables:**
- gRPC service definitions in `api/proto/`:
- Protocol Buffer files for core services (identity, auth, authz, permission, audit)
- Service and message definitions
- Proper versioning
- gRPC server implementations in `internal/services/grpc/server/`:
- Server implementations wrapping existing services
- Error handling and validation
- Request/response conversion
- gRPC client implementations in `internal/services/grpc/client/`:
- Clients that satisfy service client interfaces
- Connection pooling and retry logic
- Circuit breaker support
- Timeout handling
- gRPC server setup and integration
- Code generation from protobuf files
- Configuration for enabling gRPC services
**Acceptance Criteria:**
- gRPC service definitions are created
- gRPC servers are implemented
- gRPC clients implement service interfaces
- Service factory can create gRPC clients
- gRPC services can be enabled via configuration
- Code generation works
### Deliverables
- ✅ Cache adapter (Redis) working
- ✅ Event bus (in-process and Kafka) functional
- ✅ Blob storage (S3) adapter
- ✅ Email notification system
- ✅ Scheduler and background jobs
- ✅ Secret store integration (optional)
- ✅ gRPC service definitions and clients
### Acceptance Criteria
- Cache stores and retrieves data correctly
- Events are published and consumed
- Files can be uploaded and downloaded
- Email notifications are sent
- Background jobs run on schedule
- Integration test: full infrastructure stack works
---
## Epic 6: Observability & Production Readiness (Week 7-8)
### Objectives
- Enhance observability with full OpenTelemetry integration
- Add comprehensive error reporting (Sentry)
- Create Grafana dashboards
- Improve logging with request correlation
- Add rate limiting and security hardening
### Tasks
#### 6.1 OpenTelemetry Enhancement
- [ ] Complete OpenTelemetry setup:
- Export traces to Jaeger/OTLP collector
- Add database instrumentation (Ent interceptor)
- Add Kafka instrumentation
- Add Redis instrumentation
- [ ] Create custom spans:
- Module initialization spans
- Background job spans
- Event publishing spans
- [ ] Add trace context propagation:
- Include trace ID in logs
- Propagate across HTTP calls
- Include in error reports
#### 6.2 Error Reporting (Sentry)
- [ ] Install `github.com/getsentry/sentry-go`
- [ ] Integrate with error bus:
- Send errors to Sentry
- Include trace ID in Sentry events
- Add user context (user ID, email)
- Add module context (module name)
- [ ] Add Sentry middleware:
- Capture panics
- Capture HTTP errors (4xx, 5xx)
- [ ] Configure Sentry DSN via config
#### 6.3 Logging Enhancements
- [ ] Add request correlation:
- Generate unique request ID per request
- Include in all logs
- Return in response headers (`X-Request-ID`)
- [ ] Add structured fields:
- `user_id` from context
- `tenant_id` from context
- `module` name for module logs
- `trace_id` from OpenTelemetry
- [ ] Create log aggregation config:
- JSON format for production
- Human-readable for development
- Support for Loki/CloudWatch/ELK
#### 6.4 Prometheus Metrics Expansion
- [ ] Add more metrics:
- Database connection pool stats
- Cache hit/miss ratio
- Event bus publish/consume rates
- Background job execution times
- Module-specific metrics (via module interface)
- [ ] Create metric labels:
- `module` label for module metrics
- `tenant_id` label (if multi-tenant)
- `status` label for error rates
#### 6.5 Grafana Dashboards
- [ ] Create `ops/grafana/dashboards/`:
- `platform-overview.json` - Overall health
- `http-metrics.json` - HTTP request metrics
- `database-metrics.json` - Database performance
- `module-metrics.json` - Per-module metrics
- `error-rates.json` - Error tracking
- [ ] Document dashboard setup in `docs/operations.md`
#### 6.6 Rate Limiting
- [ ] Install `github.com/ulule/limiter/v3`
- [ ] Create rate limit middleware:
- Per-user rate limiting
- Per-IP rate limiting
- Configurable limits per endpoint
- [ ] Add rate limit config:
```yaml
rate_limiting:
enabled: true
per_user: 100/minute
per_ip: 1000/minute
```
- [ ] Return `X-RateLimit-*` headers
#### 6.7 Security Hardening
- [ ] Add security headers middleware:
- `X-Content-Type-Options: nosniff`
- `X-Frame-Options: DENY`
- `X-XSS-Protection: 1; mode=block`
- `Strict-Transport-Security` (if HTTPS)
- `Content-Security-Policy`
- [ ] Add request size limits:
- Max body size (10MB default)
- Max header size
- [ ] Add input validation:
- Use `github.com/go-playground/validator`
- Validate all request bodies
- Sanitize user inputs
- [ ] Add SQL injection protection:
- Use parameterized queries (Ent already does this)
- Add linter rule to prevent raw SQL
#### 6.8 Performance Optimization
- [ ] Add database connection pooling:
- Configure max connections
- Configure idle timeout
- Monitor pool stats
- [ ] Add query optimization:
- Add indexes for common queries
- Use database query logging (development)
- Add slow query detection
- [ ] Add response compression:
- Gzip middleware for large responses
- [ ] Add caching strategy:
- Cache frequently accessed data (user permissions, roles)
### Deliverables
- ✅ Full OpenTelemetry integration
- ✅ Sentry error reporting
- ✅ Enhanced logging with correlation
- ✅ Comprehensive Prometheus metrics
- ✅ Grafana dashboards
- ✅ Rate limiting
- ✅ Security hardening
- ✅ Performance optimizations
### Acceptance Criteria
- Traces are exported and visible in Jaeger
- Errors are reported to Sentry with context
- Logs include request IDs and trace IDs
- Metrics are exposed and scraped by Prometheus
- Rate limiting prevents abuse
- Security headers are present
- Performance meets SLA (< 100ms p95 for auth endpoints)
---
## Epic 7: Testing, Documentation & CI/CD (Week 8-9)
### Objectives
- Comprehensive test coverage (unit, integration, contract)
- Complete documentation
- Production-ready CI/CD pipeline
- Docker images and deployment guides
### Tasks
#### 7.1 Unit Tests
- [ ] Achieve >80% code coverage for core modules:
- Config loader
- Logger
- Auth service
- Permission resolver
- Module registry
- [ ] Use `github.com/stretchr/testify` for assertions
- [ ] Use `github.com/golang/mock` or `mockery` for mocks
- [ ] Add test helpers:
- `testutil.NewTestDB()` - In-memory SQLite for tests
- `testutil.NewTestUser()` - Create test user
- `testutil.NewTestContext()` - Context with user
#### 7.2 Integration Tests
- [ ] Install `github.com/testcontainers/testcontainers-go`
- [ ] Create integration test suite:
- Full HTTP request flow
- Database operations
- Event bus publishing/consuming
- Background job execution
- [ ] Test scenarios:
- User registration → login → API access
- Role assignment → permission check
- Module loading and initialization
- Multi-module interaction
- [ ] Create `docker-compose.test.yml`:
- PostgreSQL
- Redis
- Kafka (optional)
- [ ] Add test tags: `//go:build integration`
#### 7.3 Contract Tests
- [ ] Install `github.com/pact-foundation/pact-go` (optional)
- [ ] Create API contract tests:
- Verify API responses match OpenAPI spec
- Test backward compatibility
- [ ] Use OpenAPI validator:
- Install `github.com/getkin/kin-openapi`
- Validate request/response against OpenAPI spec
- Generate OpenAPI spec from code annotations
#### 7.4 Load Testing
- [ ] Create `perf/` directory with k6 scripts:
- `perf/auth-load.js` - Login endpoint load test
- `perf/api-load.js` - General API load test
- [ ] Document performance benchmarks:
- Request latency (p50, p95, p99)
- Throughput (requests/second)
- Resource usage (CPU, memory)
#### 7.5 Documentation
- [ ] Create `README.md`:
- Quick start guide
- Architecture overview
- Installation instructions
- Development setup
- [ ] Create `docs/architecture.md`:
- System architecture diagram
- Module system explanation
- Extension points
- [ ] Create `docs/extension-points.md`:
- How to create a module
- Permission system
- Event bus usage
- Background jobs
- [ ] Create `docs/api.md`:
- API endpoints documentation
- Authentication flow
- Error codes
- [ ] Create `docs/operations.md`:
- Deployment guide
- Monitoring setup
- Troubleshooting
- Grafana dashboards
- [ ] Add code examples:
- `examples/` directory with sample modules
- Code comments and godoc
#### 7.6 CI/CD Pipeline Enhancement
- [ ] Update `.github/workflows/ci.yml`:
- Run unit tests with coverage
- Run integration tests (with testcontainers)
- Run linters (golangci-lint, gosec)
- Generate coverage report
- Upload artifacts
- [ ] Add release workflow:
- Semantic versioning
- Tag releases
- Build and push Docker images
- Generate changelog
- [ ] Add security scanning:
- `gosec` for security issues
- Dependabot for dependency updates
- Trivy for container scanning
#### 7.7 Docker Images
- [ ] Create multi-stage `Dockerfile`:
```dockerfile
# Build stage
FROM golang:1.22-alpine AS builder
# ... build commands
# Runtime stage
FROM gcr.io/distroless/static-debian12
# ... copy binary
```
- [ ] Create `docker-compose.yml` for development:
- Platform service
- PostgreSQL
- Redis
- Kafka (optional)
- [ ] Create `docker-compose.prod.yml` for production
- [ ] Add health checks to Dockerfile
- [ ] Document Docker usage in `docs/deployment.md`
#### 7.8 Deployment Guides
- [ ] Create `docs/deployment/kubernetes.md`:
- Kubernetes manifests
- Helm chart (optional)
- Service definitions
- ConfigMap and Secret management
- [ ] Create `docs/deployment/docker.md`:
- Docker Compose deployment
- Environment variables
- Volume mounts
- [ ] Create `docs/deployment/cloud.md`:
- AWS/GCP/Azure deployment notes
- Managed service integration
- Load balancer configuration
#### 7.9 Developer Experience
- [ ] Create `Makefile` with common tasks:
```makefile
make dev # Start dev environment
make test # Run tests
make lint # Run linters
make generate # Generate code
make docker-build # Build Docker image
make migrate # Run migrations
```
- [ ] Add development scripts:
- `scripts/dev.sh` - Start all services
- `scripts/test.sh` - Run test suite
- `scripts/seed.sh` - Seed test data
- [ ] Create `.env.example` with all config variables
- [ ] Add pre-commit hooks (optional):
- Run linter
- Run tests
- Check formatting
### Deliverables
- ✅ >80% test coverage
- ✅ Integration test suite
- ✅ Complete documentation
- ✅ Production CI/CD pipeline
- ✅ Docker images and deployment guides
- ✅ Developer tooling and scripts
### Acceptance Criteria
- All tests pass in CI
- Code coverage >80%
- Documentation is complete and accurate
- Docker images build and run successfully
- Deployment guides are tested
- New developers can set up environment in <30 minutes
---
## Epic 8: Advanced Features & Polish (Week 9-10, Optional)
### Objectives
- Add advanced features (OIDC, GraphQL)
- Performance optimization
- Additional sample feature services
- Final polish and bug fixes
**Note:** API Gateway is now in Epic 1 (Story 1.8) as core infrastructure, not an advanced feature.
### Tasks
#### 8.1 OpenID Connect (OIDC) Support
- [ ] Install `github.com/coreos/go-oidc`
- [ ] Implement OIDC provider:
- Discovery endpoint
- JWKS endpoint
- Token endpoint
- UserInfo endpoint
- [ ] Add OIDC client support:
- Validate tokens from external IdP
- Map claims to internal user
- [ ] Document OIDC setup in `docs/auth.md`
#### 8.2 GraphQL API (Optional)
- [ ] Install `github.com/99designs/gqlgen`
- [ ] Create GraphQL schema:
- User queries
- Blog queries
- Mutations
- [ ] Implement resolvers:
- Use existing services
- Add authorization checks
- [ ] Add GraphQL endpoint: `POST /graphql`
#### 8.3 Additional Sample Feature Services
- [ ] Create Notification Service (`cmd/notification-service/`):
- Service entry point, gRPC server
- Email templates
- Notification preferences
- Notification history
- [ ] Create Analytics Service (`cmd/analytics-service/`):
- Service entry point, gRPC server
- Event tracking
- Analytics dashboard API
- Export functionality
#### 8.4 Performance Optimization
- [ ] Add database query caching
- [ ] Optimize N+1 queries
- [ ] Add response caching (Redis)
- [ ] Implement connection pooling optimizations
- [ ] Add database read replicas support
#### 8.5 Internationalization (i18n)
- [ ] Install i18n library
- [ ] Add locale detection:
- From Accept-Language header
- From user preferences
- [ ] Create message catalogs
- [ ] Add translation support for error messages
#### 8.6 Final Polish
- [ ] Code review and refactoring
- [ ] Bug fixes
- [ ] Performance profiling
- [ ] Security audit
- [ ] Documentation review
### Deliverables
- ✅ OIDC support (optional)
- ✅ GraphQL API (optional)
- ✅ Additional sample feature services (Notification, Analytics)
- ✅ Performance optimizations
- ✅ Final polish
---
## Implementation Checklist Summary
### Epic 0: Setup ✅
- [ ] Repository structure
- [ ] Configuration system
- [ ] Logging foundation
- [ ] Basic CI/CD
- [ ] DI setup
### Epic 1: Core Kernel ✅
- [ ] DI container
- [ ] Database (Ent)
- [ ] Health & metrics
- [ ] Error bus
- [ ] HTTP server
- [ ] OpenTelemetry
### Epic 2: Auth & Authorization ✅
- [ ] JWT authentication
- [ ] Identity management
- [ ] Roles & permissions
- [ ] Authorization middleware
- [ ] Audit logging
### Epic 3: Module Framework ✅
- [ ] Module interface
- [ ] Static registry
- [ ] Permission generation
- [ ] Module loader
- [ ] Module initialization
### Epic 4: Sample Module (Blog) ✅
- [ ] Blog module structure
- [ ] Domain model
- [ ] Repository & service
- [ ] API handlers
- [ ] Integration tests
### Epic 5: Infrastructure ✅
- [ ] Cache (Redis)
- [ ] Event bus
- [ ] Blob storage
- [ ] Email notification
- [ ] Scheduler/jobs
- [ ] Multi-tenancy (optional)
### Epic 6: Observability ✅
- [ ] OpenTelemetry
- [ ] Sentry integration
- [ ] Enhanced logging
- [ ] Prometheus metrics
- [ ] Grafana dashboards
- [ ] Rate limiting
- [ ] Security hardening
### Epic 7: Testing & Docs ✅
- [ ] Unit tests (>80% coverage)
- [ ] Integration tests
- [ ] Documentation
- [ ] CI/CD pipeline
- [ ] Docker images
- [ ] Deployment guides
### Epic 8: Advanced Features (Optional) ✅
- [ ] OIDC support
- [ ] GraphQL API
- [ ] Additional modules
- [ ] Performance optimization
---
## Risk Mitigation
### Technical Risks
| Risk | Impact | Mitigation |
|------|--------|------------|
| **Circular import issues** | High | Strict separation: interfaces in `pkg/`, implementations in `internal/` |
| **Plugin version mismatch** | Medium | Prefer static registration; document version requirements |
| **Database migration conflicts** | Medium | Central migration orchestrator, dependency ordering |
| **Performance bottlenecks** | Low | Load testing in Epic 7, profiling, caching strategy |
| **Security vulnerabilities** | High | Security audit, gosec scanning, input validation |
### Process Risks
| Risk | Impact | Mitigation |
|------|--------|------------|
| **Scope creep** | Medium | Stick to epic-based approach, defer optional features to Epic 8 |
| **Incomplete documentation** | Medium | Documentation as part of each epic, not afterthought |
| **Testing gaps** | High | Test coverage requirements, integration tests early |
---
## Success Criteria
The platform is considered complete when:
1. ✅ All core modules are implemented and tested
2. ✅ Blog module serves as working reference
3. ✅ Test coverage >80%
4. ✅ Documentation is complete
5. ✅ CI/CD pipeline is production-ready
6. ✅ Docker images build and run
7. ✅ Integration tests pass
8. ✅ Security audit passes
9. ✅ Performance meets SLA (<100ms p95 for auth)
10. New developer can set up in <30 minutes
---
## Next Steps After Implementation
1. **Gather Feedback**: Share with team, collect requirements
2. **Iterate**: Add features based on feedback
3. **Scale**: Optimize for production load
4. **Extend**: Add more modules as needed
5. **Community**: Open source (if applicable), gather contributors
---
## References
- [playbook.md](./playbook.md) - General platform playbook
- [Go Modules Documentation](https://go.dev/doc/modules)
- [Ent Documentation](https://entgo.io/docs/getting-started)
- [Uber FX Documentation](https://github.com/uber-go/fx)
- [OpenTelemetry Go](https://opentelemetry.io/docs/instrumentation/go/)
---
**Document Version:** 1.0
**Status:** Ready for Implementation