Compare commits

..

123 Commits

Author SHA1 Message Date
3a98b72ffd Merge pull request 'feature/epic2-core-services' (#6) from feature/epic2-core-services into main
All checks were successful
CI / Test (push) Successful in 3m38s
CI / Lint (push) Successful in 54s
CI / Build (push) Successful in 45s
CI / Format Check (push) Successful in 2s
Reviewed-on: #6
2025-11-07 10:23:19 +01:00
a785cd73de perf(ci): pre-install all tools in pre-commit Docker image
All checks were successful
CI / Test (pull_request) Successful in 53s
CI / Lint (pull_request) Successful in 26s
CI / Build (pull_request) Successful in 38s
CI / Format Check (pull_request) Successful in 2s
- Move Go 1.25.3 installation to Dockerfile
- Pre-install protobuf plugins and golangci-lint in image
- Set environment variables in Dockerfile
- Remove runtime installation steps from pre-commit script
- Significantly improves pre-commit check performance
2025-11-07 10:16:12 +01:00
75c5293c8c fix(lint): add bounds checking for integer conversions to fix gosec warnings
All checks were successful
CI / Test (pull_request) Successful in 57s
CI / Lint (pull_request) Successful in 27s
CI / Build (pull_request) Successful in 39s
CI / Format Check (pull_request) Successful in 2s
- Add bounds checking for Limit and Offset conversions in audit_client.go
- Add bounds checking for t, m, and p conversions in password.go
- Add nolint comments with explanations for safe conversions
2025-11-07 10:04:14 +01:00
9712e50f6c fix(scripts): disable VCS stamping in pre-commit container 2025-11-07 09:55:24 +01:00
8eda9af769 fix(scripts): ensure working directory is correct for make commands 2025-11-07 09:54:00 +01:00
4b33ed522d fix(scripts): install make in pre-commit container 2025-11-07 09:52:45 +01:00
bc740f7b1f fix(scripts): use make fmt-check instead of go fmt 2025-11-07 09:52:17 +01:00
e98e4d3099 fix(scripts): use make fmt-check in pre-commit script 2025-11-07 09:51:23 +01:00
93623e6865 fix(scripts): install Go 1.25.3 in pre-commit container and fix formatting 2025-11-07 09:47:50 +01:00
b531f92436 feat(ci): add pre-commit check script using wirelos/pre-commit image 2025-11-07 09:46:35 +01:00
31e8ca7ce9 fix(lint): use explicit safe type conversions for gosec
Some checks failed
CI / Test (pull_request) Successful in 51s
CI / Lint (pull_request) Failing after 26s
CI / Build (pull_request) Successful in 38s
CI / Format Check (pull_request) Failing after 2s
Use separate variables with explicit else branches to make type
conversions safe and satisfy gosec integer overflow checks.
2025-11-07 09:37:53 +01:00
e673fcae6f fix(lint): fix all linting errors
Some checks failed
CI / Test (pull_request) Successful in 53s
CI / Lint (pull_request) Failing after 26s
CI / Build (pull_request) Successful in 39s
CI / Format Check (pull_request) Failing after 2s
- Check BindEnv return values in config.go
- Add bounds checks for int->int32/uint32 conversions to prevent overflow
- Remove unused test helper functions
2025-11-07 09:34:38 +01:00
131e44f3d4 fix(ci): install latest golangci-lint using official script
Some checks failed
CI / Test (pull_request) Successful in 52s
CI / Lint (pull_request) Failing after 26s
CI / Build (pull_request) Successful in 38s
CI / Format Check (pull_request) Successful in 2s
2025-11-07 09:29:49 +01:00
0d6094267a fix(ci): install golangci-lint v2.4.0 for Go 1.25 and config v2 support 2025-11-07 09:27:27 +01:00
4b8536e34a fix(ci): use official install script for golangci-lint 2025-11-07 09:27:17 +01:00
e509faea25 fix(ci): install golangci-lint v2 for config compatibility
Some checks failed
CI / Test (pull_request) Successful in 51s
CI / Lint (pull_request) Failing after 5s
CI / Build (pull_request) Successful in 37s
CI / Format Check (pull_request) Successful in 3s
2025-11-07 09:27:02 +01:00
355008a3a2 fix(ci): build golangci-lint from source for Go 1.25 support
Some checks failed
CI / Test (pull_request) Successful in 51s
CI / Lint (pull_request) Failing after 5s
CI / Build (pull_request) Successful in 37s
CI / Format Check (pull_request) Successful in 2s
2025-11-07 09:24:58 +01:00
c8d944e9ea fix(ci): let golangci-lint-action auto-select compatible version
Some checks failed
CI / Test (pull_request) Successful in 52s
CI / Lint (pull_request) Failing after 7s
CI / Build (pull_request) Successful in 37s
CI / Format Check (pull_request) Successful in 2s
2025-11-07 09:22:52 +01:00
d24bb96d62 fix(ci): use golangci-lint v1.64.0 2025-11-07 09:22:33 +01:00
61d614690f fix(ci): use golangci-lint v1.65.0 for Go 1.25 support
Some checks failed
CI / Test (pull_request) Successful in 52s
CI / Lint (pull_request) Failing after 6s
CI / Build (pull_request) Successful in 37s
CI / Format Check (pull_request) Successful in 2s
2025-11-07 09:19:54 +01:00
5f2e1104f2 fix(ci): use golangci-lint latest version
Some checks failed
CI / Test (pull_request) Successful in 52s
CI / Lint (pull_request) Failing after 6s
CI / Build (pull_request) Successful in 37s
CI / Format Check (pull_request) Successful in 2s
2025-11-07 09:17:22 +01:00
7c0aefb7f4 fix(fmt): formatting 2025-11-07 09:10:22 +01:00
8cfdfbc951 fix(schema): remove duplicate auditlog.go schema 2025-11-07 09:04:05 +01:00
0912f0f81b feat(schema): restore complete Ent schema files 2025-11-07 09:03:43 +01:00
1f8c2626dc fix(gitignore): allow ent/schema/ directory 2025-11-07 08:42:06 +01:00
fb10051443 fix(ci): generate Ent from ent/schema and copy to internal/ent 2025-11-07 08:41:03 +01:00
8bb36b5735 fix(ci): correctly copy Ent files excluding schema directory 2025-11-07 08:39:43 +01:00
837b04b433 fix(ci): use find to copy all Ent generated files 2025-11-07 08:39:26 +01:00
868649d6d2 fix(ci): add debug output to Ent generation step 2025-11-07 08:38:22 +01:00
13da884a21 fix(ci): exclude generate.go from Ent code copy 2025-11-07 08:21:40 +01:00
c2e2ab01f2 fix(ci): generate Ent code and copy to internal/ent 2025-11-07 08:21:31 +01:00
8c10c3dba9 fix(ci): use full module path for Ent target directory 2025-11-07 08:20:56 +01:00
b6eb8d75bb fix(ci): generate Ent code to internal/ent directory 2025-11-07 08:17:32 +01:00
4c62817cff fix(ci): revert to use apk for Alpine runner 2025-11-06 22:49:45 +01:00
0edeb67075 test: add comprehensive tests and fix CI build 2025-11-06 22:49:13 +01:00
b3c8f68989 fix(ci): update to use Makefile commands 2025-11-06 22:42:17 +01:00
3f18163313 fix(gitignore): exclude generated protobuf and Ent files 2025-11-06 22:39:55 +01:00
d42b1cd5f1 fix(proto): fix protobuf generation and update gateway tests 2025-11-06 22:39:43 +01:00
471a057d25 fix(ci): fix CI build and update Makefile to build all services 2025-11-06 22:34:49 +01:00
ad4ecaed1f fix(ci): update to use Alpine package manager (apk) 2025-11-06 22:30:39 +01:00
6ce1007f73 fix(ci): update CI to generate protobuf and Ent ORM files 2025-11-06 22:26:54 +01:00
4e6db9995f fix(gitignore): remove generated files from git tracking 2025-11-06 22:15:08 +01:00
dbe29bfb82 fix(consul): fix gRPC health checks and add API Gateway Consul registration 2025-11-06 22:04:55 +01:00
04022b835e feat(auth): Complete Auth Service implementation and fix Consul health checks
- Add VerifyPassword RPC to Identity Service
  - Added to proto file and generated code
  - Implemented in Identity Service gRPC server
  - Added to Identity Service client interface and gRPC client

- Complete RefreshToken implementation
  - Store refresh tokens in database using RefreshToken entity
  - Validate refresh tokens with expiration checking
  - Revoke refresh tokens on logout and token rotation

- Integrate Authz Service for role retrieval
  - Added AuthzServiceClient to Auth Service
  - Get user roles during login and token refresh
  - Gracefully handle Authz Service failures

- Require JWT secret in configuration
  - Removed default secret fallback
  - Service fails to start if JWT secret is not configured

- Fix Consul health checks for Docker
  - Services now register with Docker service names (e.g., audit-service)
  - Allows Consul (in Docker) to reach services via Docker DNS
  - Health checks use gRPC service names instead of localhost

This completes all TODOs in auth_service_fx.go and fixes the Consul
health check failures in Docker environments.
2025-11-06 21:26:34 +01:00
b02c1d44c8 fix(consul): Fix health checks for gRPC services in Docker
- Add gRPC health check support to Consul registry
  - Services are gRPC-only, not HTTP
  - Consul was trying HTTP health checks which failed
  - Now uses gRPC health checks via grpc.health.v1.Health service

- Update HealthCheckConfig to support both HTTP and gRPC
  - Add GRPC field for gRPC service name
  - Add UseGRPC flag to choose health check type
  - Default to gRPC for services (use_grpc: true in config)

- Fix service address registration in Docker
  - Services now register with Docker service name (e.g., auth-service)
  - Allows Consul to reach services via Docker network DNS
  - Falls back to localhost for local development

- Update default.yaml to enable gRPC health checks
  - Set use_grpc: true
  - Set grpc: grpc.health.v1.Health

This fixes services being deregistered from Consul due to failed
HTTP health checks. Services will now pass gRPC health checks.
2025-11-06 21:17:33 +01:00
54e1866997 fix(config): Fix environment variable mapping for Docker
- Add SetEnvKeyReplacer to convert underscores to dots
- Explicitly bind DATABASE_DSN, REGISTRY_CONSUL_ADDRESS, REGISTRY_TYPE
- Fixes database connection issues in Docker where services couldn't
  read DATABASE_DSN environment variable
- Services in Docker can now connect to postgres:5432 instead of localhost
2025-11-06 21:09:47 +01:00
cf4bf9505a fix(docs): Fix service run commands to include all package files
- Change from 'go run ./cmd/{service}/main.go' to 'go run ./cmd/{service}/*.go'
  - go run with single file doesn't include other files in the package
  - Service implementations are in separate _fx.go files
  - Using wildcard includes all .go files in the package

- Update README.md and SUMMARY.md with correct commands
- Fixes 'undefined: provideXService' errors when running services
2025-11-06 21:04:03 +01:00
a2990f02ba fix(gitignore): Only ignore API Gateway binary, not directory 2025-11-06 21:03:16 +01:00
01603a0722 feat(docker): Add API Gateway Dockerfile 2025-11-06 21:03:09 +01:00
cba2096adf feat(docker): Add API Gateway to Docker Compose
- Create Dockerfile for API Gateway
  - Multi-stage build using golang:1.25-alpine
  - Minimal runtime image using alpine:latest
  - Exposes port 8080

- Add API Gateway service to docker-compose.yml
  - Depends on Consul and all core services
  - Environment variables for gateway configuration
  - Port 8080 exposed

- Update SUMMARY.md
  - Add API Gateway to service list
  - Add API Gateway to Docker build instructions
  - Update file structure to include API Gateway Dockerfile
2025-11-06 21:02:54 +01:00
4cac2b2592 fix(services): Fix duplicate health registry provider
- Change from fx.Provide to fx.Invoke for health registry registration
  - CoreModule() already provides *health.Registry
  - Services should register their database checkers with the existing registry
  - Use fx.Invoke to register database health checkers instead of providing new registry

- Fixes duplicate provider error for *health.Registry
- All services now build and should start successfully
2025-11-06 21:00:52 +01:00
dfe460cb03 fix(services): Fix service startup failures
- Remove duplicate CoreModule() calls from all service main.go files
  - NewContainer() already includes CoreModule() automatically
  - This was causing duplicate ConfigProvider provider errors

- Update all _fx.go files to use *database.Client instead of *ent.Client
  - database.Client embeds *ent.Client, so it can be used directly
  - This fixes type mismatches between providers and consumers
  - Keep ent import for constants like ent.Desc

- All services now build and should start successfully
2025-11-06 20:56:37 +01:00
2f2a14f2c5 docs: Update README.md with current implementation and quick start
- Add Core Services section highlighting Epic 2 completion
- Update directory structure to include all service entry points
- Add comprehensive Quick Start guide with:
  - Prerequisites including NixOS support
  - Installation steps with code generation
  - Two deployment options (development vs full Docker)
  - Service endpoints and ports
  - Testing examples with grpcurl
- Update Architecture section with Core Services details
- Add Implementation Status section showing completed epics
- Update Configuration section with service-specific settings
- Add links to Epic 2 documentation
2025-11-06 20:49:19 +01:00
ff330e510d fix(gitignore): Only ignore service binaries, not directories 2025-11-06 20:47:04 +01:00
3191ae9444 feat(docker): Add Dockerfiles for all services 2025-11-06 20:46:58 +01:00
031a90eca0 feat(docker): Add Docker support for all services
- Create Dockerfiles for all four services (auth, identity, authz, audit)
  - Multi-stage builds using golang:1.25-alpine
  - Minimal runtime images using alpine:latest
  - Copy config files to runtime image

- Create docker-compose.dev.yml for development
  - Only PostgreSQL and Consul
  - Use when running services locally with 'go run'

- Update docker-compose.yml for full deployment
  - All services + infrastructure
  - Services build from Dockerfiles
  - Health checks and dependencies configured
  - Environment variables for service configuration

- Add .dockerignore to optimize build context
  - Excludes docs, tests, IDE files, build artifacts

- Update SUMMARY.md
  - Document both docker-compose files
  - Add Docker deployment section
  - Update file structure to include Dockerfiles
2025-11-06 20:46:43 +01:00
33339f19cb docs: Fix duplicate heading in SUMMARY.md 2025-11-06 20:08:50 +01:00
6d6e07e09a docs: Add Consul verification steps to SUMMARY.md 2025-11-06 20:08:44 +01:00
3ac8983e98 feat(docker): Add Consul to docker-compose and update documentation
- Add Consul service to docker-compose.yml
  - Running in dev mode on port 8500
  - Health checks configured
  - Persistent volume for data
  - Web UI available at http://localhost:8500/ui

- Update SUMMARY.md
  - Document Consul setup in docker-compose
  - Add Consul verification steps
  - Update prerequisites to include Docker Compose
  - Add note about Consul Web UI

- Remove obsolete version field from docker-compose.yml
2025-11-06 20:08:37 +01:00
cb28a120ed chore: Update .gitignore to exclude auth-service and authz-service binaries 2025-11-06 20:07:45 +01:00
b1b895e818 feat(epic2): Implement core authentication and authorization services
- Implement Audit Service (2.5)
  - gRPC server with Record and Query operations
  - Database persistence with audit schema
  - Service registry integration
  - Entry point: cmd/audit-service

- Implement Identity Service (2.2)
  - User CRUD operations
  - Password hashing with argon2id
  - Email verification and password reset flows
  - Entry point: cmd/identity-service
  - Fix package naming conflicts in user_service.go

- Implement Auth Service (2.1)
  - JWT token generation and validation
  - Login, RefreshToken, ValidateToken, Logout RPCs
  - Integration with Identity Service
  - Entry point: cmd/auth-service
  - Note: RefreshToken entity needs Ent generation

- Implement Authz Service (2.3, 2.4)
  - Permission checking and authorization
  - User roles and permissions retrieval
  - RBAC-based authorization
  - Entry point: cmd/authz-service

- Implement gRPC clients for all services
  - Auth, Identity, Authz, and Audit clients
  - Service discovery integration
  - Full gRPC communication

- Add service configurations to config/default.yaml
- Create SUMMARY.md with implementation details and testing instructions
- Fix compilation errors in Identity Service (password package conflicts)
- All services build successfully and tests pass
2025-11-06 20:07:20 +01:00
da7a4e3703 Merge pull request 'feature/microservice-architecture' (#5) from feature/microservice-architecture into main
All checks were successful
CI / Test (push) Successful in 26s
CI / Lint (push) Successful in 21s
CI / Build (push) Successful in 17s
CI / Format Check (push) Successful in 2s
Reviewed-on: #5
2025-11-06 13:47:17 +01:00
f9170bb00b fix(docs): diagrams
All checks were successful
CI / Test (pull_request) Successful in 26s
CI / Lint (pull_request) Successful in 21s
CI / Build (pull_request) Successful in 16s
CI / Format Check (pull_request) Successful in 2s
2025-11-06 13:29:32 +01:00
b4b918cba8 docs: ensure newline before lists across docs for MkDocs rendering
All checks were successful
CI / Test (pull_request) Successful in 27s
CI / Lint (pull_request) Successful in 20s
CI / Build (pull_request) Successful in 16s
CI / Format Check (pull_request) Successful in 2s
2025-11-06 10:56:50 +01:00
a1586cb302 fix(fmt): format code
All checks were successful
CI / Test (pull_request) Successful in 26s
CI / Lint (pull_request) Successful in 20s
CI / Build (pull_request) Successful in 16s
CI / Format Check (pull_request) Successful in 3s
2025-11-06 10:35:20 +01:00
a9b8df06f3 fix(lint): remove unused grpc imports from auth_client after commenting out connectToService
Some checks failed
CI / Test (pull_request) Successful in 26s
CI / Lint (pull_request) Successful in 20s
CI / Build (pull_request) Successful in 17s
CI / Format Check (pull_request) Failing after 2s
2025-11-06 10:33:06 +01:00
767654f257 fix(lint): resolve golangci-lint errors
Some checks failed
CI / Test (pull_request) Failing after 22s
CI / Lint (pull_request) Failing after 19s
CI / Build (pull_request) Failing after 6s
CI / Format Check (pull_request) Successful in 2s
- Fix errcheck: explicitly ignore tx.Rollback() error in defer
  - When transaction commits successfully, Rollback() returns an error (expected)
  - Use defer func() with explicit error assignment to satisfy linter

- Remove unused connectToService function
  - Function is not currently used (proto files not yet generated)
  - Commented out with TODO for future implementation
  - Prevents unused function lint error
2025-11-06 10:28:48 +01:00
cd57fe7c14 fix(ci): align golangci-lint config with v2.6 schema (remove 'issues.exclude')
Some checks failed
CI / Test (pull_request) Successful in 25s
CI / Lint (pull_request) Failing after 21s
CI / Build (pull_request) Successful in 16s
CI / Format Check (pull_request) Successful in 2s
2025-11-06 10:16:55 +01:00
b56b3c8c93 fix(ci): update golangci-lint config for v2.6 compatibility
Some checks failed
CI / Test (pull_request) Successful in 26s
CI / Lint (pull_request) Failing after 5s
CI / Build (pull_request) Successful in 16s
CI / Format Check (pull_request) Successful in 2s
- Change version from number to string: version: "2"
- Remove deprecated exclude-use-default option
- Change exclude-rules to exclude (new format in v2.6)
- Remove deprecated output section (print-issued-lines, print-linter-name)
- Remove linters-settings (not allowed in v2.6 schema validation)

Fixes CI validation errors with golangci-lint v2.6.1:
- version type validation error
- exclude-use-default and exclude-rules not allowed
- output options not allowed
- linters-settings not allowed at root level
2025-11-06 10:14:13 +01:00
c05038ccf2 fix: resolve test race conditions and update golangci-lint action
Some checks failed
CI / Test (pull_request) Successful in 24s
CI / Lint (pull_request) Failing after 18s
CI / Build (pull_request) Successful in 15s
CI / Format Check (pull_request) Successful in 1s
- Fix race condition in gateway tests by using TestMain to set Gin mode once
  - Remove duplicate gin.SetMode(gin.TestMode) calls from individual tests
  - Add TestMain function to initialize test environment before all tests
  - Prevents race conditions when tests run in parallel with -race flag

- Update golangci-lint-action from v6 to v7
  - v6 doesn't support golangci-lint v2.x versions
  - v7 supports golangci-lint v2.x and automatically selects compatible version
  - Change version from v2.6.0 to latest for automatic compatibility

All tests now pass with race detector enabled.
2025-11-06 09:57:58 +01:00
557e6a009e fix(ci): update golangci-lint to support Go 1.25.3
Some checks failed
CI / Test (pull_request) Failing after 23s
CI / Lint (pull_request) Failing after 18s
CI / Build (pull_request) Successful in 14s
CI / Format Check (pull_request) Successful in 2s
- Replace manual golangci-lint v2.1.6 installation (built with Go 1.24)
- Use official golangci/golangci-lint-action@v6 GitHub Action
- Set version to v2.6.0 which supports Go 1.25+
- Action automatically handles Go version compatibility

Fixes CI error: 'the Go language version (go1.24) used to build
golangci-lint is lower than the targeted Go version (1.25.3)'
2025-11-06 09:53:39 +01:00
260bc07114 test: add comprehensive tests for API Gateway implementation
- Add unit tests for gateway service (services/gateway/gateway_test.go)
  - Test gateway creation, route setup, service discovery, and error handling
  - Achieve 67.9% code coverage for gateway service
  - Test all HTTP methods are properly handled
  - Test route matching and 404 handling

- Add tests for API Gateway main entry point (cmd/api-gateway/main_test.go)
  - Test DI container setup and structure
  - Test service instance creation logic
  - Test lifecycle hooks registration

- Add testify dependency for assertions (go.mod)

All tests pass successfully. Proxy forwarding tests are noted for integration
test suite as they require real HTTP connections (per ADR-0028 testing strategy).
2025-11-06 09:52:16 +01:00
fbc3fc37e4 feat: upgrade to Go 1.25.3 and update build to include api-gateway
Some checks failed
CI / Test (pull_request) Successful in 36s
CI / Lint (pull_request) Failing after 3s
CI / Build (pull_request) Successful in 16s
CI / Format Check (pull_request) Successful in 2s
- Create ADR-0034 documenting Go version upgrade to 1.25.3
- Mark ADR-0002 as superseded by ADR-0034
- Update CI workflow to use Go 1.25.3
- Update Makefile to build both platform and api-gateway binaries
- Update CI workflow to build and upload both binaries
- Update documentation (ADR README and mkdocs.yml) to include ADR-0034
2025-11-06 09:41:47 +01:00
8ac85f9b1c chore: Remove duplicate gateway.go file from internal directory
Some checks failed
CI / Test (pull_request) Failing after 1m20s
CI / Lint (pull_request) Failing after 3s
CI / Build (pull_request) Successful in 53s
CI / Format Check (pull_request) Successful in 3s
2025-11-06 09:25:26 +01:00
9c21ece585 chore: Clean up duplicate files and update .gitignore
- Remove duplicate services/gateway/internal/gateway.go
- Add api-gateway binary to .gitignore
2025-11-06 09:24:19 +01:00
16731fc1d1 refactor: Align Epic 0 & Epic 1 with true microservices architecture
Refactor core kernel and infrastructure to support true microservices
architecture where services are independently deployable.

Phase 1: Core Kernel Cleanup
- Remove database provider from CoreModule (services create their own)
- Update ProvideHealthRegistry to not depend on database
- Add schema support to database client (NewClientWithSchema)
- Update main entry point to remove database dependency
- Core kernel now provides only: config, logger, error bus, health, metrics, tracer, service registry

Phase 2: Service Registry Implementation
- Create ServiceRegistry interface (pkg/registry/registry.go)
- Implement Consul registry (internal/registry/consul/consul.go)
- Add Consul dependency (github.com/hashicorp/consul/api)
- Add registry configuration to config/default.yaml
- Add ProvideServiceRegistry() to DI container

Phase 3: Service Client Interfaces
- Create service client interfaces:
  - pkg/services/auth.go - AuthServiceClient
  - pkg/services/identity.go - IdentityServiceClient
  - pkg/services/authz.go - AuthzServiceClient
  - pkg/services/audit.go - AuditServiceClient
- Create ServiceClientFactory (internal/client/factory.go)
- Create stub gRPC client implementations (internal/client/grpc/)
- Add ProvideServiceClientFactory() to DI container

Phase 4: gRPC Service Definitions
- Create proto files for all core services:
  - api/proto/auth.proto
  - api/proto/identity.proto
  - api/proto/authz.proto
  - api/proto/audit.proto
- Add generate-proto target to Makefile

Phase 5: API Gateway Implementation
- Create API Gateway service entry point (cmd/api-gateway/main.go)
- Create Gateway implementation (services/gateway/gateway.go)
- Add gateway configuration to config/default.yaml
- Gateway registers with Consul and routes requests to backend services

All code compiles successfully. Core services (Auth, Identity, Authz, Audit)
will be implemented in Epic 2 using these foundations.
2025-11-06 09:23:36 +01:00
38a251968c docs: Align documentation with true microservices architecture
Transform all documentation from modular monolith to true microservices
architecture where core services are independently deployable.

Key Changes:
- Core Kernel: Infrastructure only (no business logic)
- Core Services: Auth, Identity, Authz, Audit as separate microservices
  - Each service has own entry point (cmd/{service}/)
  - Each service has own gRPC server and database schema
  - Services register with Consul for service discovery
- API Gateway: Moved from Epic 8 to Epic 1 as core infrastructure
  - Single entry point for all external traffic
  - Handles routing, JWT validation, rate limiting, CORS
- Service Discovery: Consul as primary mechanism (ADR-0033)
- Database Pattern: Per-service connections with schema isolation

Documentation Updates:
- Updated all 9 architecture documents
- Updated 4 ADRs and created 2 new ADRs (API Gateway, Service Discovery)
- Rewrote Epic 1: Core Kernel & Infrastructure (infrastructure only)
- Rewrote Epic 2: Core Services (Auth, Identity, Authz, Audit as services)
- Updated Epic 3-8 stories for service architecture
- Updated plan.md, playbook.md, requirements.md, index.md
- Updated all epic READMEs and story files

New ADRs:
- ADR-0032: API Gateway Strategy
- ADR-0033: Service Discovery Implementation (Consul)

New Stories:
- Epic 1.7: Service Client Interfaces
- Epic 1.8: API Gateway Implementation
2025-11-06 08:54:19 +01:00
cab7cadf9e docs: update readme
All checks were successful
CI / Test (push) Successful in 21s
CI / Lint (push) Successful in 18s
CI / Build (push) Successful in 12s
CI / Format Check (push) Successful in 2s
2025-11-05 21:45:56 +01:00
b01d5bdeea Merge pull request 'feature/epic1-core-infrastructure' (#2) from feature/epic1-core-infrastructure into main
All checks were successful
CI / Test (push) Successful in 21s
CI / Lint (push) Successful in 17s
CI / Build (push) Successful in 12s
CI / Format Check (push) Successful in 2s
Reviewed-on: #2
2025-11-05 21:36:54 +01:00
8c90075086 fix: correct Mermaid graph syntax for endpoint labels with slashes
All checks were successful
CI / Test (pull_request) Successful in 22s
CI / Lint (pull_request) Successful in 18s
CI / Build (pull_request) Successful in 12s
CI / Format Check (pull_request) Successful in 2s
Mermaid graphs require node labels with special characters like forward
slashes to be quoted. Changed /healthz and /metrics from square bracket
format to quoted string format to fix the lexical error.
2025-11-05 21:26:17 +01:00
9b33c1528a fix: correct Mermaid sequence diagram syntax for permissions list
Mermaid sequence diagrams don't support YAML-style lists with dashes
in message content. Changed the multi-line permission list to a single
comma-separated line to fix the parse error.
2025-11-05 21:24:55 +01:00
0d6c62ab03 fix: remove t.Parallel() from metrics tests to fix race conditions
All checks were successful
CI / Test (pull_request) Successful in 21s
CI / Lint (pull_request) Successful in 17s
CI / Build (pull_request) Successful in 12s
CI / Format Check (pull_request) Successful in 2s
The Gin framework uses a global mode setting (gin.SetMode()) which is not
thread-safe when tests run in parallel. Removing t.Parallel() from metrics
tests that use gin.SetMode() prevents data races when running tests with
the race detector enabled.

All tests now pass with 'make test' which includes -race flag.
2025-11-05 21:20:29 +01:00
3bc37dd48c fix: resolve all linting errors
Some checks failed
CI / Test (pull_request) Failing after 18s
CI / Lint (pull_request) Successful in 17s
CI / Build (pull_request) Successful in 13s
CI / Format Check (pull_request) Successful in 2s
- Use typed context key instead of string in errorbus test to avoid collisions
- Remove unused imports (health.HealthChecker, trace.TracerProvider) from test files
- Simplify interface verification checks (removed unnecessary type assertions)

All linting errors resolved. make lint now passes.
2025-11-05 21:14:00 +01:00
7ffacb6620 fix: add mutex to mockLogger in errorbus tests to prevent race conditions
Some checks failed
CI / Test (pull_request) Successful in 22s
CI / Lint (pull_request) Failing after 17s
CI / Build (pull_request) Successful in 12s
CI / Format Check (pull_request) Successful in 2s
The mockLogger's errors slice was being accessed concurrently from
multiple goroutines (the error bus consumer and the test goroutine),
causing race conditions when running tests with the race detector.

Added sync.Mutex to protect the errors slice and proper locking when
accessing it in test assertions.
2025-11-05 21:11:46 +01:00
3f3545ba15 fix: remove t.Parallel() from server tests to fix race conditions
The Gin framework uses a global mode setting (gin.SetMode()) which is not
thread-safe when tests run in parallel. Removing t.Parallel() from all
server tests that use gin.SetMode() prevents data races when running
tests with the race detector enabled.

All tests now pass with 'make test' which includes -race flag.
2025-11-05 21:10:06 +01:00
5fdbb729bd test: add comprehensive tests for all Epic 1 stories
Some checks failed
CI / Test (pull_request) Failing after 17s
CI / Lint (pull_request) Failing after 18s
CI / Build (pull_request) Successful in 12s
CI / Format Check (pull_request) Successful in 2s
Story 1.2: Database Layer
- Test database client creation, connection, ping, and close
- Test connection pooling configuration
- Tests skip if database is not available (short mode)

Story 1.3: Health Monitoring and Metrics
- Test health registry registration and checking
- Test database health checker
- Test liveness and readiness checks
- Test metrics creation, middleware, and handler
- Test Prometheus metrics endpoint

Story 1.4: Error Handling and Error Bus
- Test channel-based error bus creation
- Test error publishing with context
- Test nil error handling
- Test channel full scenario
- Test graceful shutdown
- Fix Close() method to handle multiple calls safely

Story 1.5: HTTP Server and Middleware
- Test server creation with all middleware
- Test request ID middleware
- Test logging middleware
- Test panic recovery middleware
- Test CORS middleware
- Test timeout middleware
- Test health and metrics endpoints
- Test server shutdown

Story 1.6: OpenTelemetry Tracing
- Test tracer initialization (enabled/disabled)
- Test development and production modes
- Test OTLP exporter configuration
- Test graceful shutdown
- Test no-op tracer provider

All tests follow Go testing best practices:
- Table-driven tests where appropriate
- Parallel execution
- Proper mocking of interfaces
- Skip tests requiring external dependencies in short mode
2025-11-05 21:05:36 +01:00
278a727b8c docs: remove all emojis from playbook document
- Remove emoji numbers from section headers (1-13)
- Remove rocket emoji from final congratulations message
- All sections now use plain numbers instead of emoji numbers
2025-11-05 20:55:49 +01:00
52d48590ae fix: resolve all linting and formatting issues
- Fix error return value checks (errcheck)
- Fix unused parameters by using underscore prefix
- Add missing package comments to all packages
- Fix context key type issue in middleware (use typed contextKey)
- Replace deprecated trace.NewNoopTracerProvider with noop.NewTracerProvider
- Fix embedded field selector in database client
- Remove trailing whitespace
- Remove revive linter (as requested) to avoid stuttering warnings for public API interfaces

All linting and formatting checks now pass.
2025-11-05 20:48:59 +01:00
926f3f927e docs: verify and update Epic 1 story statuses to Completed
- Verified all acceptance criteria for Stories 1.1-1.6
- Updated Status fields from Pending to Completed
- Marked all acceptance criteria checkboxes as completed
- All stories in Epic 1 are now fully implemented and verified
2025-11-05 20:41:51 +01:00
0e3bfb4e44 fix: ensure database and HTTP server providers execute on startup
- Add fx.Invoke in main.go to force database and HTTP server creation
- This ensures all providers execute and their lifecycle hooks are registered
- Clean up debug logging statements
- Database migrations and HTTP server now start correctly on application startup

Fixes issue where database migrations and HTTP server were not starting
because FX providers were not being executed (lazy evaluation).
2025-11-05 20:32:20 +01:00
84673c33b1 fix: add comprehensive logging to track startup hooks
- Added logging when HTTP server OnStart hook is called
- Added error logging for database migration failures
- This will help identify if hooks are being called and where failures occur
2025-11-05 19:42:32 +01:00
512d76a6fb fix: improve HTTP server startup with better error detection
- Added better error detection for HTTP server startup
- Added connectivity check to verify server is actually listening
- Increased wait time to 500ms for better error detection
- Added warning log if server connectivity check fails (may still be starting)
- Improved logging messages for server startup

This should help diagnose why the HTTP server isn't starting and provide better visibility into the startup process.
2025-11-05 19:42:13 +01:00
d1d0b170ce fix: improve logging and error visibility, move Story 1.7 to Epic 2
Fixes:
- Added database connection logging with masked DSN
- Added migration progress logging
- Added HTTP server startup logging with address
- Fixed database provider to accept logger parameter
- Improved error visibility throughout initialization

Documentation:
- Moved Story 1.7 (Service Client Interfaces) to Epic 2 as Story 2.7
- Updated Epic 1 and Epic 2 READMEs
- Updated COMPLETE_TASK_LIST.md
- Updated story metadata (ID, Epic, Dependencies)

These changes will help diagnose startup issues and provide better visibility into what the application is doing.
2025-11-05 19:39:25 +01:00
fde01bfc73 feat(epic1): complete OpenTelemetry integration and add verification documentation
Story 1.6: OpenTelemetry Distributed Tracing
- Implemented tracer initialization with stdout (dev) and OTLP (prod) exporters
- Added HTTP request instrumentation via Gin middleware
- Integrated trace ID correlation in structured logs
- Added tracing configuration to config files
- Registered tracer provider in DI container

Documentation and Setup:
- Created Docker Compose setup for PostgreSQL database
- Added comprehensive Epic 1 summary with verification instructions
- Added Epic 0 summary with verification instructions
- Linked summaries in documentation index and epic READMEs
- Included detailed database testing instructions
- Added Docker Compose commands and troubleshooting guide

All Epic 1 stories (1.1-1.6) are now complete. Story 1.7 depends on Epic 2.
2025-11-05 18:20:15 +01:00
30320304f6 feat(epic1): implement core infrastructure (stories 1.1-1.5)
Implemented Epic 1 core kernel and infrastructure stories:

Story 1.1: Enhanced DI Container
- Added providers for database, health, metrics, and error bus
- Extended CoreModule to include all core services

Story 1.2: Database Layer with Ent ORM
- Created Ent schema for User, Role, Permission, AuditLog entities
- Implemented many-to-many relationships (User-Role, Role-Permission)
- Created database client wrapper with connection pooling
- Added database provider to DI container with migration support

Story 1.3: Health Monitoring and Metrics System
- Implemented health check registry and interface
- Added database health checker
- Created Prometheus metrics system with HTTP instrumentation
- Added health and metrics providers to DI container

Story 1.4: Error Handling and Error Bus
- Implemented channel-based error bus
- Created ErrorPublisher interface
- Added error bus provider with lifecycle management

Story 1.5: HTTP Server Foundation
- Created HTTP server with Gin framework
- Implemented comprehensive middleware stack:
  - Request ID generation
  - Structured logging
  - Panic recovery with error bus integration
  - Prometheus metrics collection
  - CORS support
- Registered core routes: /healthz, /ready, /metrics
- Integrated with FX lifecycle for graceful shutdown

All components are integrated via DI container and ready for use.
2025-11-05 18:11:11 +01:00
a38a08ca17 docs: add ADR-0031 for service repository structure decision
All checks were successful
CI / Test (push) Successful in 11s
CI / Lint (push) Successful in 9s
CI / Build (push) Successful in 6s
CI / Format Check (push) Successful in 3s
Create ADR documenting the decision to use a monorepo structure with
service directories for organizing microservices (Option 1).

The ADR covers:
- Decision to use monorepo with service directories
- Rationale for monorepo vs separate repositories
- Proposed directory structure with cmd/ and services/
- Implementation strategy across epics
- Consequences and mitigations
- Migration path if needed in the future

This clarifies how services (Auth, Identity, Authz, Audit) should be
organized in the codebase while maintaining service independence and
microservices architecture principles.
2025-11-05 15:13:34 +01:00
7cadc3b3c0 Merge pull request 'feature/epic0-foundation' (#1) from feature/epic0-foundation into main
All checks were successful
CI / Test (push) Successful in 11s
CI / Lint (push) Successful in 9s
CI / Build (push) Successful in 6s
CI / Format Check (push) Successful in 2s
Reviewed-on: #1
2025-11-05 13:44:59 +01:00
610677af72 docs: mark all epic0 stories as completed
All checks were successful
CI / Test (pull_request) Successful in 11s
CI / Lint (pull_request) Successful in 10s
CI / Build (pull_request) Successful in 6s
CI / Format Check (pull_request) Successful in 2s
Update status of all epic0 stories (0.1-0.5) from Pending to Completed:
- 0.1: Project Initialization - Directory structure and Go module setup
- 0.2: Configuration Management System - Viper-based config implemented
- 0.3: Structured Logging System - Zap logger with middleware implemented
- 0.4: CI/CD Pipeline - GitHub Actions workflow with tests and linting
- 0.5: DI and Bootstrap - FX-based DI container with lifecycle management

All stories have been implemented with tests and are working.
2025-11-05 13:44:00 +01:00
93bc6e082c fix(ci): install golangci-lint v2.1.6 manually to match local environment
All checks were successful
CI / Test (pull_request) Successful in 1m48s
CI / Lint (pull_request) Successful in 29s
CI / Build (pull_request) Successful in 15s
CI / Format Check (pull_request) Successful in 3s
Install v2.1.6 manually in CI (not via action) to avoid --out-format flag
compatibility issues. This ensures both local and CI use the same version
and support version: 2 config format.
2025-11-05 13:36:48 +01:00
ef54924137 fix(ci): install golangci-lint manually and remove version field
The golangci-lint-action has compatibility issues with v2.1.6 (uses
--out-format flag which v2 doesn't support). Install golangci-lint manually
to avoid action limitations and remove version field from config to be
compatible with CI v1.64.8. Local v2.x will work but may show warnings.
2025-11-05 13:36:24 +01:00
f9f4add257 fix: remove version field to work with CI v1.64.8
All checks were successful
CI / Build (pull_request) Successful in 6s
CI / Format Check (pull_request) Successful in 2s
CI / Test (pull_request) Successful in 11s
CI / Lint (pull_request) Successful in 5s
The golangci-lint-action doesn't properly support v2.1.6 (uses --out-format
flag which v2 doesn't support). Remove version: 2 from config to make it
compatible with CI v1.64.8. Local v2.x will still work but may show a
deprecation warning about the config format.
2025-11-05 13:35:40 +01:00
25bfa410e7 fix(ci): upgrade golangci-lint-action to v4 for v2 compatibility
Some checks failed
CI / Test (pull_request) Successful in 10s
CI / Lint (pull_request) Failing after 4s
CI / Build (pull_request) Successful in 6s
CI / Format Check (pull_request) Successful in 2s
v3 action uses --out-format flag which v2.1.6 doesn't support.
Upgrade to v4 action which should properly support golangci-lint v2.
This should resolve the 'unknown flag: --out-format' error.
2025-11-05 13:34:43 +01:00
4f3d65a50d fix(ci): pin golangci-lint to v2.1.6 to match local and support v2 config
Some checks failed
CI / Test (pull_request) Successful in 11s
CI / Lint (pull_request) Failing after 4s
CI / Build (pull_request) Successful in 6s
CI / Format Check (pull_request) Successful in 2s
'latest' resolves to v1.64.8 which doesn't support version: 2 config.
Pin to v2.1.6 to match local development environment and ensure
v2 config compatibility.
2025-11-05 13:33:35 +01:00
018c9eb9c5 fix(ci): use latest golangci-lint version for v2 config support
Some checks failed
CI / Test (pull_request) Successful in 10s
CI / Lint (pull_request) Failing after 4s
CI / Build (pull_request) Successful in 6s
CI / Format Check (pull_request) Successful in 3s
v1.65.0 doesn't exist. Use 'latest' to automatically get a version
that supports v2 config format (version: 2). This should work with
both local v2.1.6 and CI.
2025-11-05 13:32:34 +01:00
6e90bdf6e3 fix(ci): upgrade golangci-lint to v1.65.0 for v2 config support
Some checks failed
CI / Test (pull_request) Successful in 10s
CI / Lint (pull_request) Failing after 33s
CI / Build (pull_request) Successful in 6s
CI / Format Check (pull_request) Successful in 2s
v1.64.8 doesn't support version: 2 in config files.
Upgrade to v1.65.0 which supports v2 config format.
This ensures compatibility with local v2.1.6 and CI.
2025-11-05 13:30:59 +01:00
d8aab7f5c4 fix: add version field for local v2 compatibility and pin CI to v1.64.8
Some checks failed
CI / Test (pull_request) Successful in 11s
CI / Lint (pull_request) Failing after 4s
CI / Build (pull_request) Successful in 6s
CI / Format Check (pull_request) Successful in 2s
- Add version: 2 to .golangci.yml for local golangci-lint v2.1.6 compatibility
- Pin CI to use golangci-lint v1.64.8 explicitly (should support v2 config)

This ensures the config works both locally (v2.1.6) and in CI (v1.64.8).
If v1.64.8 doesn't support v2 config, we may need to upgrade CI to v2.
2025-11-05 13:29:32 +01:00
5f15ebd967 fix: add missing comments to noOpLogger methods and remove deprecated output.format
- Add comments to all noOpLogger methods to satisfy revive exported rule
- Remove deprecated output.format option (use default format instead)

This fixes the linting issues:
- exported: exported method noOpLogger.* should have comment or be unexported
- warning about deprecated output.format option
2025-11-05 13:27:55 +01:00
28f72917b8 fix(config): remove version field and formatters for golangci-lint v1 compatibility
Some checks failed
CI / Test (pull_request) Successful in 11s
CI / Lint (pull_request) Failing after 10s
CI / Build (pull_request) Successful in 6s
CI / Format Check (pull_request) Successful in 3s
The CI uses golangci-lint v1.64.8 which doesn't support:
- version: 2 field (v2-only feature)
- formatters section (v2-only feature)

Removed version field and formatters section to make config compatible with v1.
Formatting will be checked by gofmt/goimports separately if needed.
2025-11-05 13:25:52 +01:00
b132a48efe fix(ci): downgrade golangci-lint-action to v3 for Gitea compatibility
Some checks failed
CI / Test (pull_request) Successful in 11s
CI / Lint (pull_request) Failing after 20s
CI / Build (pull_request) Successful in 6s
CI / Format Check (pull_request) Successful in 2s
Downgrade golangci-lint-action from v4 to v3 to match other actions
that were downgraded for Gitea compatibility (upload-artifact, codecov).
v4 actions are not fully supported on Gitea/GHES.
2025-11-05 13:23:42 +01:00
c69fbec95f fix(ci): skip cache for golangci-lint to avoid BusyBox tar compatibility issue
Some checks failed
CI / Test (pull_request) Successful in 11s
CI / Lint (pull_request) Failing after 4s
CI / Build (pull_request) Successful in 6s
CI / Format Check (pull_request) Successful in 2s
The golangci-lint-action tries to use tar with --posix option for caching,
but BusyBox tar (used in Alpine-based runners) doesn't support this option.
Skipping the cache avoids this compatibility issue.
2025-11-05 13:21:43 +01:00
784f0f601f fix: resolve all golangci-lint issues
Some checks failed
CI / Test (pull_request) Successful in 10s
CI / Lint (pull_request) Failing after 4s
CI / Build (pull_request) Successful in 6s
CI / Format Check (pull_request) Successful in 3s
- Add package comments to all packages (pkg/config, pkg/logger, internal/*, cmd/platform)

- Fix context key warnings by using custom ContextKey type
  - Export ContextKey type to avoid unexported-return warnings
  - Update all context value operations to use ContextKey instead of string
  - Update RequestIDKey() and UserIDKey() to return ContextKey

- Fix error checking issues (errcheck)
  - Properly handle os.Chdir errors in defer statements
  - Properly handle os.Setenv/os.Unsetenv errors in tests

- Fix security warnings (gosec)
  - Change directory permissions from 0755 to 0750 in tests
  - Change file permissions from 0644 to 0600 in tests

- Fix unused parameter warnings (revive)
  - Replace unused parameters with _ in:
    * RegisterLifecycleHooks lifecycle functions
    * Mock logger implementations
    * noOpLogger methods

- Fix type assertion issues (staticcheck)
  - Remove unnecessary type assertions in tests
  - Use simpler compile-time checks

- Fix exported type stuttering warning
  - Add nolint directive for ConfigProvider (standard interface pattern)

- Update golangci-lint configuration
  - Add version: 2 field (required for newer versions)
  - Remove unsupported linters (typecheck, gosimple)
  - Move formatters (gofmt, goimports) to separate formatters section
  - Simplify linter list to only well-supported linters

All linting issues resolved (0 issues reported by golangci-lint).
All tests pass and code compiles successfully.
2025-11-05 13:19:54 +01:00
82707186a0 fix: remove parallel execution from Gin tests to prevent data races
Some checks failed
CI / Test (pull_request) Successful in 11s
CI / Lint (pull_request) Failing after 5s
CI / Build (pull_request) Successful in 6s
CI / Format Check (pull_request) Successful in 2s
- Remove t.Parallel() from tests that use gin.SetMode()
  - gin.SetMode() modifies global state and is not thread-safe
  - Tests affected:
    * TestRequestIDMiddleware_GenerateNewID
    * TestRequestIDMiddleware_UseExistingID
    * TestLoggingMiddleware
    * TestLoggingMiddleware_WithRequestID
    * TestRequestIDMiddleware_MultipleRequests

- Add comments explaining why these tests cannot run in parallel
- All tests now pass with race detector enabled (-race flag)

This fixes data race warnings that were occurring when running tests
with the race detector, specifically when multiple tests tried to set
Gin's mode concurrently.
2025-11-05 13:01:27 +01:00
6b0ba2edc7 fix: improve CI workflow for Gitea compatibility and test detection
Some checks failed
CI / Test (pull_request) Failing after 46s
CI / Lint (pull_request) Failing after 5s
CI / Build (pull_request) Successful in 14s
CI / Format Check (pull_request) Successful in 3s
- Improve test file detection with more robust find command
  - Use explicit variable assignment instead of pipe to grep
  - Add debug output to show found test files
  - Handle errors gracefully with 2>/dev/null || true

- Downgrade actions for Gitea compatibility
  - upload-artifact@v4 -> v3 (Gitea doesn't support v4+)
  - codecov-action@v4 -> v3 (preventive downgrade)

- Add Alpine Linux build dependencies installation step
  - Install build-base and musl-dev when running on Alpine
  - Required for CGO-enabled builds and race detector

- Disable CGO for verify build step when no tests exist
  - Avoids requiring C build tools for simple compilation check
2025-11-05 12:57:04 +01:00
0bfdb2c2d7 Add comprehensive test suite for current implementation
Some checks failed
CI / Test (pull_request) Successful in 1m43s
CI / Lint (pull_request) Failing after 27s
CI / Build (pull_request) Failing after 13s
CI / Format Check (pull_request) Successful in 2s
- Add tests for internal/config package (90.9% coverage)
  - Test all viperConfig getter methods
  - Test LoadConfig with default and environment-specific configs
  - Test error handling for missing config files

- Add tests for internal/di package (88.1% coverage)
  - Test Container lifecycle (NewContainer, Start, Stop)
  - Test providers (ProvideConfig, ProvideLogger, CoreModule)
  - Test lifecycle hooks registration
  - Include mock implementations for testing

- Add tests for internal/logger package (96.5% coverage)
  - Test zapLogger with JSON and console formats
  - Test all logging levels and methods
  - Test middleware (RequestIDMiddleware, LoggingMiddleware)
  - Test context helper functions
  - Include benchmark tests

- Update CI workflow to skip tests when no test files exist
  - Add conditional test execution based on test file presence
  - Add timeout for test execution
  - Verify build when no tests are present

All tests follow Go best practices with table-driven patterns,
parallel execution where safe, and comprehensive coverage.
2025-11-05 12:45:37 +01:00
a1fc6e69a7 fix: enable CGO for race detector in tests
Some checks failed
CI / Test (pull_request) Waiting to run
CI / Lint (pull_request) Waiting to run
CI / Format Check (pull_request) Has been cancelled
CI / Build (pull_request) Has been cancelled
- Add CGO_ENABLED=1 to CI test step
- Add CGO_ENABLED=1 to Makefile test commands
- Fixes: go: -race requires cgo; enable cgo by setting CGO_ENABLED=1
2025-11-05 12:24:43 +01:00
930b599af9 chore: remove accidentally committed binary and update .gitignore
Some checks failed
CI / Test (pull_request) Failing after 1m37s
CI / Lint (pull_request) Failing after 2m32s
CI / Build (pull_request) Failing after 14s
CI / Format Check (pull_request) Failing after 2s
2025-11-05 12:21:22 +01:00
4724a2efb5 feat: implement Epic 0 - Project Setup & Foundation
Implemented all 5 stories from Epic 0:

Story 0.1: Project Initialization
- Initialize Go module with path git.dcentral.systems/toolz/goplt
- Create complete directory structure (cmd/, internal/, pkg/, modules/, config/, etc.)
- Add comprehensive .gitignore for Go projects
- Create README.md with project overview and setup instructions

Story 0.2: Configuration Management System
- Define ConfigProvider interface in pkg/config
- Implement Viper-based configuration in internal/config
- Create configuration loader with environment support
- Add default, development, and production YAML config files

Story 0.3: Structured Logging System
- Define Logger interface in pkg/logger
- Implement Zap-based logger in internal/logger
- Add request ID middleware for Gin
- Create global logger export with convenience functions
- Support context-aware logging with request/user ID extraction

Story 0.4: CI/CD Pipeline
- Create GitHub Actions workflow for CI (test, lint, build, fmt)
- Add comprehensive Makefile with development commands
- Configure golangci-lint with reasonable defaults

Story 0.5: Dependency Injection and Bootstrap
- Create FX-based DI container in internal/di
- Implement provider functions for Config and Logger
- Create application entry point in cmd/platform/main.go
- Add lifecycle management with graceful shutdown

All acceptance criteria met:
- go build ./cmd/platform succeeds
- go test ./... runs successfully
- go mod verify passes
- Config loads from config/default.yaml
- Logger can be injected and used
- Application starts and shuts down gracefully
2025-11-05 12:21:15 +01:00
3f90262860 feat: update AI guidlines 2025-11-05 11:19:24 +01:00
66b0c3b40d docs: update dead links 2025-11-05 11:00:36 +01:00
b4f8875a0e docs: add system specifications 2025-11-05 09:51:47 +01:00
ace9678f6c feat: reword phase to epic, update mkdocs 2025-11-05 09:28:33 +01:00
65a428534c feat: microservice architecture 2025-11-05 09:12:34 +01:00
54a047f5dc docs: add mkdocs, update links, add architecture documentation 2025-11-05 07:44:21 +01:00
526 changed files with 31165 additions and 14300 deletions

View File

@@ -1,95 +0,0 @@
You are an expert in Go, microservices architecture, and clean backend development practices. Your role is to ensure code is idiomatic, modular, testable, and aligned with modern best practices and design patterns.
### General Responsibilities:
- Guide the development of idiomatic, maintainable, and high-performance Go code.
- Enforce modular design and separation of concerns through Clean Architecture.
- Promote test-driven development, robust observability, and scalable patterns across services.
### Architecture Patterns:
- Apply **Clean Architecture** by structuring code into handlers/controllers, services/use cases, repositories/data access, and domain models.
- Use **domain-driven design** principles where applicable.
- Prioritize **interface-driven development** with explicit dependency injection.
- Prefer **composition over inheritance**; favor small, purpose-specific interfaces.
- Ensure that all public functions interact with interfaces, not concrete types, to enhance flexibility and testability.
### Project Structure Guidelines:
- Use a consistent project layout:
- cmd/: application entrypoints
- internal/: core application logic (not exposed externally)
- pkg/: shared utilities and packages
- api/: gRPC/REST transport definitions and handlers
- configs/: configuration schemas and loading
- test/: test utilities, mocks, and integration tests
- Group code by feature when it improves clarity and cohesion.
- Keep logic decoupled from framework-specific code.
### Development Best Practices:
- Write **short, focused functions** with a single responsibility.
- Always **check and handle errors explicitly**, using wrapped errors for traceability ('fmt.Errorf("context: %w", err)').
- Avoid **global state**; use constructor functions to inject dependencies.
- Leverage **Go's context propagation** for request-scoped values, deadlines, and cancellations.
- Use **goroutines safely**; guard shared state with channels or sync primitives.
- **Defer closing resources** and handle them carefully to avoid leaks.
### Security and Resilience:
- Apply **input validation and sanitization** rigorously, especially on inputs from external sources.
- Use secure defaults for **JWT, cookies**, and configuration settings.
- Isolate sensitive operations with clear **permission boundaries**.
- Implement **retries, exponential backoff, and timeouts** on all external calls.
- Use **circuit breakers and rate limiting** for service protection.
- Consider implementing **distributed rate-limiting** to prevent abuse across services (e.g., using Redis).
### Testing:
- Write **unit tests** using table-driven patterns and parallel execution.
- **Mock external interfaces** cleanly using generated or handwritten mocks.
- Separate **fast unit tests** from slower integration and E2E tests.
- Ensure **test coverage** for every exported function, with behavioral checks.
- Use tools like 'go test -cover' to ensure adequate test coverage.
### Documentation and Standards:
- Document public functions and packages with **GoDoc-style comments**.
- Provide concise **READMEs** for services and libraries.
- Maintain a 'CONTRIBUTING.md' and 'ARCHITECTURE.md' to guide team practices.
- Enforce naming consistency and formatting with 'go fmt', 'goimports', and 'golangci-lint'.
### Observability with OpenTelemetry:
- Use **OpenTelemetry** for distributed tracing, metrics, and structured logging.
- Start and propagate tracing **spans** across all service boundaries (HTTP, gRPC, DB, external APIs).
- Always attach 'context.Context' to spans, logs, and metric exports.
- Use **otel.Tracer** for creating spans and **otel.Meter** for collecting metrics.
- Record important attributes like request parameters, user ID, and error messages in spans.
- Use **log correlation** by injecting trace IDs into structured logs.
- Export data to **OpenTelemetry Collector**, **Jaeger**, or **Prometheus**.
### Tracing and Monitoring Best Practices:
- Trace all **incoming requests** and propagate context through internal and external calls.
- Use **middleware** to instrument HTTP and gRPC endpoints automatically.
- Annotate slow, critical, or error-prone paths with **custom spans**.
- Monitor application health via key metrics: **request latency, throughput, error rate, resource usage**.
- Define **SLIs** (e.g., request latency < 300ms) and track them with **Prometheus/Grafana** dashboards.
- Alert on key conditions (e.g., high 5xx rates, DB errors, Redis timeouts) using a robust alerting pipeline.
- Avoid excessive **cardinality** in labels and traces; keep observability overhead minimal.
- Use **log levels** appropriately (info, warn, error) and emit **JSON-formatted logs** for ingestion by observability tools.
- Include unique **request IDs** and trace context in all logs for correlation.
### Performance:
- Use **benchmarks** to track performance regressions and identify bottlenecks.
- Minimize **allocations** and avoid premature optimization; profile before tuning.
- Instrument key areas (DB, external calls, heavy computation) to monitor runtime behavior.
### Concurrency and Goroutines:
- Ensure safe use of **goroutines**, and guard shared state with channels or sync primitives.
- Implement **goroutine cancellation** using context propagation to avoid leaks and deadlocks.
### Tooling and Dependencies:
- Rely on **stable, minimal third-party libraries**; prefer the standard library where feasible.
- Use **Go modules** for dependency management and reproducibility.
- Version-lock dependencies for deterministic builds.
- Integrate **linting, testing, and security checks** in CI pipelines.
### Key Conventions:
1. Prioritize **readability, simplicity, and maintainability**.
2. Design for **change**: isolate business logic and minimize framework lock-in.
3. Emphasize clear **boundaries** and **dependency inversion**.
4. Ensure all behavior is **observable, testable, and documented**.
5. **Automate workflows** for testing, building, and deployment.

209
.cursor/rules/golang.mdc Normal file
View File

@@ -0,0 +1,209 @@
---
alwaysApply: true
---
# Go Development Rules for Cursor AI
You are an expert in Go, microservices architecture, and clean backend development practices. When working with Go code in this project, you MUST follow these rules strictly.
## Architecture Patterns
- **ALWAYS** apply Clean Architecture by structuring code into handlers/controllers, services/use cases, repositories/data access, and domain models
- **ALWAYS** use domain-driven design principles where applicable
- **ALWAYS** prioritize interface-driven development with explicit dependency injection
- **ALWAYS** prefer composition over inheritance; favor small, purpose-specific interfaces
- **ALWAYS** ensure that all public functions interact with interfaces, not concrete types, to enhance flexibility and testability
## Project Structure
When organizing code, you MUST follow this structure:
- `cmd/`: application entrypoints
- `internal/`: core application logic (not exposed externally)
- `pkg/`: shared utilities and packages
- `api/`: gRPC/REST transport definitions and handlers
- `configs/`: configuration schemas and loading
- `test/`: test utilities, mocks, and integration tests
Group code by feature when it improves clarity and cohesion. Keep logic decoupled from framework-specific code.
## Code Quality Requirements
- **ALWAYS** write short, focused functions with a single responsibility
- **ALWAYS** check and handle errors explicitly, using wrapped errors for traceability: `fmt.Errorf("context: %w", err)`
- **NEVER** use global state; use constructor functions to inject dependencies
- **ALWAYS** leverage Go's context propagation for request-scoped values, deadlines, and cancellations
- **ALWAYS** use goroutines safely; guard shared state with channels or sync primitives
- **ALWAYS** defer closing resources and handle them carefully to avoid leaks
## Security and Resilience
- **ALWAYS** apply input validation and sanitization rigorously, especially on inputs from external sources
- **ALWAYS** use secure defaults for JWT, cookies, and configuration settings
- **ALWAYS** isolate sensitive operations with clear permission boundaries
- **ALWAYS** implement retries, exponential backoff, and timeouts on all external calls
- **ALWAYS** use circuit breakers and rate limiting for service protection
- Consider implementing distributed rate-limiting to prevent abuse across services (e.g., using Redis)
## Testing Requirements
- **ALWAYS** write unit tests using table-driven patterns and parallel execution
- **ALWAYS** mock external interfaces cleanly using generated or handwritten mocks
- **ALWAYS** separate fast unit tests from slower integration and E2E tests
- **ALWAYS** ensure test coverage for every exported function, with behavioral checks
- Use tools like `go test -cover` to ensure adequate test coverage
## Documentation Standards
- **ALWAYS** document public functions and packages with GoDoc-style comments
- **ALWAYS** provide concise READMEs for services and libraries
- **ALWAYS** enforce naming consistency and formatting with `go fmt`, `goimports`, and `golangci-lint`
## Observability with OpenTelemetry
- **ALWAYS** use OpenTelemetry for distributed tracing, metrics, and structured logging
- **ALWAYS** start and propagate tracing spans across all service boundaries (HTTP, gRPC, DB, external APIs)
- **ALWAYS** attach `context.Context` to spans, logs, and metric exports
- **ALWAYS** use `otel.Tracer` for creating spans and `otel.Meter` for collecting metrics
- **ALWAYS** record important attributes like request parameters, user ID, and error messages in spans
- **ALWAYS** use log correlation by injecting trace IDs into structured logs
- Export data to OpenTelemetry Collector, Jaeger, or Prometheus
## Tracing and Monitoring
- **ALWAYS** trace all incoming requests and propagate context through internal and external calls
- **ALWAYS** use middleware to instrument HTTP and gRPC endpoints automatically
- **ALWAYS** annotate slow, critical, or error-prone paths with custom spans
- Monitor application health via key metrics: request latency, throughput, error rate, resource usage
- Define SLIs (e.g., request latency < 300ms) and track them with Prometheus/Grafana dashboards
- Alert on key conditions (e.g., high 5xx rates, DB errors, Redis timeouts) using a robust alerting pipeline
- **NEVER** create excessive cardinality in labels and traces; keep observability overhead minimal
- **ALWAYS** use log levels appropriately (info, warn, error) and emit JSON-formatted logs for ingestion by observability tools
- **ALWAYS** include unique request IDs and trace context in all logs for correlation
## Performance
- Use benchmarks to track performance regressions and identify bottlenecks
- Minimize allocations and avoid premature optimization; profile before tuning
- Instrument key areas (DB, external calls, heavy computation) to monitor runtime behavior
## Concurrency and Goroutines
- **ALWAYS** ensure safe use of goroutines, and guard shared state with channels or sync primitives
- **ALWAYS** implement goroutine cancellation using context propagation to avoid leaks and deadlocks
## Tooling and Dependencies
- **ALWAYS** rely on stable, minimal third-party libraries; prefer the standard library where feasible
- **ALWAYS** use Go modules for dependency management and reproducibility
- **ALWAYS** version-lock dependencies for deterministic builds
- **ALWAYS** integrate linting, testing, and security checks in CI pipelines
## Key Conventions
1. **ALWAYS** prioritize readability, simplicity, and maintainability
2. **ALWAYS** design for change: isolate business logic and minimize framework lock-in
3. **ALWAYS** emphasize clear boundaries and dependency inversion
4. **ALWAYS** ensure all behavior is observable, testable, and documented
5. **ALWAYS** automate workflows for testing, building, and deployment
# Go Development Rules for Cursor AI
You are an expert in Go, microservices architecture, and clean backend development practices. When working with Go code in this project, you MUST follow these rules strictly.
## Architecture Patterns
- **ALWAYS** apply Clean Architecture by structuring code into handlers/controllers, services/use cases, repositories/data access, and domain models
- **ALWAYS** use domain-driven design principles where applicable
- **ALWAYS** prioritize interface-driven development with explicit dependency injection
- **ALWAYS** prefer composition over inheritance; favor small, purpose-specific interfaces
- **ALWAYS** ensure that all public functions interact with interfaces, not concrete types, to enhance flexibility and testability
## Project Structure
When organizing code, you MUST follow this structure:
- `cmd/`: application entrypoints
- `internal/`: core application logic (not exposed externally)
- `pkg/`: shared utilities and packages
- `api/`: gRPC/REST transport definitions and handlers
- `configs/`: configuration schemas and loading
- `test/`: test utilities, mocks, and integration tests
Group code by feature when it improves clarity and cohesion. Keep logic decoupled from framework-specific code.
## Code Quality Requirements
- **ALWAYS** write short, focused functions with a single responsibility
- **ALWAYS** check and handle errors explicitly, using wrapped errors for traceability: `fmt.Errorf("context: %w", err)`
- **NEVER** use global state; use constructor functions to inject dependencies
- **ALWAYS** leverage Go's context propagation for request-scoped values, deadlines, and cancellations
- **ALWAYS** use goroutines safely; guard shared state with channels or sync primitives
- **ALWAYS** defer closing resources and handle them carefully to avoid leaks
## Security and Resilience
- **ALWAYS** apply input validation and sanitization rigorously, especially on inputs from external sources
- **ALWAYS** use secure defaults for JWT, cookies, and configuration settings
- **ALWAYS** isolate sensitive operations with clear permission boundaries
- **ALWAYS** implement retries, exponential backoff, and timeouts on all external calls
- **ALWAYS** use circuit breakers and rate limiting for service protection
- Consider implementing distributed rate-limiting to prevent abuse across services (e.g., using Redis)
## Testing Requirements
- **ALWAYS** write unit tests using table-driven patterns and parallel execution
- **ALWAYS** mock external interfaces cleanly using generated or handwritten mocks
- **ALWAYS** separate fast unit tests from slower integration and E2E tests
- **ALWAYS** ensure test coverage for every exported function, with behavioral checks
- Use tools like `go test -cover` to ensure adequate test coverage
## Documentation Standards
- **ALWAYS** document public functions and packages with GoDoc-style comments
- **ALWAYS** provide concise READMEs for services and libraries
- **ALWAYS** enforce naming consistency and formatting with `go fmt`, `goimports`, and `golangci-lint`
## Observability with OpenTelemetry
- **ALWAYS** use OpenTelemetry for distributed tracing, metrics, and structured logging
- **ALWAYS** start and propagate tracing spans across all service boundaries (HTTP, gRPC, DB, external APIs)
- **ALWAYS** attach `context.Context` to spans, logs, and metric exports
- **ALWAYS** use `otel.Tracer` for creating spans and `otel.Meter` for collecting metrics
- **ALWAYS** record important attributes like request parameters, user ID, and error messages in spans
- **ALWAYS** use log correlation by injecting trace IDs into structured logs
- Export data to OpenTelemetry Collector, Jaeger, or Prometheus
## Tracing and Monitoring
- **ALWAYS** trace all incoming requests and propagate context through internal and external calls
- **ALWAYS** use middleware to instrument HTTP and gRPC endpoints automatically
- **ALWAYS** annotate slow, critical, or error-prone paths with custom spans
- Monitor application health via key metrics: request latency, throughput, error rate, resource usage
- Define SLIs (e.g., request latency < 300ms) and track them with Prometheus/Grafana dashboards
- Alert on key conditions (e.g., high 5xx rates, DB errors, Redis timeouts) using a robust alerting pipeline
- **NEVER** create excessive cardinality in labels and traces; keep observability overhead minimal
- **ALWAYS** use log levels appropriately (info, warn, error) and emit JSON-formatted logs for ingestion by observability tools
- **ALWAYS** include unique request IDs and trace context in all logs for correlation
## Performance
- Use benchmarks to track performance regressions and identify bottlenecks
- Minimize allocations and avoid premature optimization; profile before tuning
- Instrument key areas (DB, external calls, heavy computation) to monitor runtime behavior
## Concurrency and Goroutines
- **ALWAYS** ensure safe use of goroutines, and guard shared state with channels or sync primitives
- **ALWAYS** implement goroutine cancellation using context propagation to avoid leaks and deadlocks
## Tooling and Dependencies
- **ALWAYS** rely on stable, minimal third-party libraries; prefer the standard library where feasible
- **ALWAYS** use Go modules for dependency management and reproducibility
- **ALWAYS** version-lock dependencies for deterministic builds
- **ALWAYS** integrate linting, testing, and security checks in CI pipelines
## Key Conventions
1. **ALWAYS** prioritize readability, simplicity, and maintainability
2. **ALWAYS** design for change: isolate business logic and minimize framework lock-in
3. **ALWAYS** emphasize clear boundaries and dependency inversion
4. **ALWAYS** ensure all behavior is observable, testable, and documented
5. **ALWAYS** automate workflows for testing, building, and deployment

64
.dockerignore Normal file
View File

@@ -0,0 +1,64 @@
# Git files
.git
.gitignore
.gitattributes
# Documentation
docs/
*.md
!README.md
# Development files
.envrc
shell.nix
.direnv/
# Build artifacts
bin/
*.exe
*.exe~
*.dll
*.so
*.dylib
*.test
*.out
auth-service
identity-service
authz-service
audit-service
platform
api-gateway
# Test files
*_test.go
test/
*.test
# IDE files
.vscode/
.idea/
*.swp
*.swo
*~
.DS_Store
# Logs
*.log
# Temporary files
tmp/
temp/
*.tmp
# Docker files (don't copy into Docker)
docker-compose*.yml
Dockerfile*
# CI/CD
.github/
.gitlab-ci.yml
.circleci/
# Coverage
coverage.out
coverage.html

5
.envrc Normal file
View File

@@ -0,0 +1,5 @@
# Automatically load nix-shell when entering this directory
# Requires direnv: https://direnv.net/
# Run: direnv allow
use nix

221
.github/workflows/ci.yml vendored Normal file
View File

@@ -0,0 +1,221 @@
name: CI
on:
push:
branches: [ main, develop ]
pull_request:
branches: [ main, develop ]
jobs:
test:
name: Test
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: '1.25.3'
- name: Cache Go modules
uses: actions/cache@v4
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-
- name: Download dependencies
run: go mod download
- name: Verify dependencies
run: go mod verify
- name: Install protoc and plugins
run: |
apk add --no-cache protobuf-dev protoc
go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest
echo "$HOME/go/bin" >> $GITHUB_PATH
- name: Generate code
run: |
make generate-proto
echo "Checking for Ent schema directory..."
if [ -d "ent/schema" ]; then
echo "Generating Ent code..."
go install entgo.io/ent/cmd/ent@latest
cd ent/schema && go run -mod=mod entgo.io/ent/cmd/ent generate .
echo "Copying Ent code to internal/ent..."
cd .. && mkdir -p ../internal/ent
cp -r *.go */ ../internal/ent/ 2>/dev/null || true
rm -f ../internal/ent/generate.go
rm -rf ../internal/ent/schema
echo "Verifying internal/ent/ent.go exists..."
ls -la ../internal/ent/ent.go || echo "ERROR: ent.go not found!"
else
echo "WARNING: ent/schema directory not found!"
fi
- name: Check for test files
id: check-tests
run: |
echo "Checking for test files..."
TEST_FILES=$(find . -name "*_test.go" -not -path "./vendor/*" -not -path "./.git/*" 2>/dev/null || true)
if [ -n "$TEST_FILES" ]; then
echo "Found test files:"
echo "$TEST_FILES"
echo "tests_exist=true" >> $GITHUB_OUTPUT
else
echo "No test files found. Skipping test execution."
echo "tests_exist=false" >> $GITHUB_OUTPUT
fi
- name: Run tests
if: steps.check-tests.outputs.tests_exist == 'true'
env:
CGO_ENABLED: 1
run: make test-coverage
- name: Upload coverage
if: steps.check-tests.outputs.tests_exist == 'true'
uses: codecov/codecov-action@v3
with:
file: ./coverage.out
fail_ci_if_error: false
- name: Verify build (no tests)
if: steps.check-tests.outputs.tests_exist == 'false'
run: make build
lint:
name: Lint
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: '1.25.3'
- name: Download dependencies
run: go mod download
- name: Install protoc and plugins
run: |
apk add --no-cache protobuf-dev protoc
go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest
echo "$HOME/go/bin" >> $GITHUB_PATH
- name: Generate code
run: |
make generate-proto
echo "Checking for Ent schema directory..."
if [ -d "ent/schema" ]; then
echo "Generating Ent code..."
go install entgo.io/ent/cmd/ent@latest
cd ent/schema && go run -mod=mod entgo.io/ent/cmd/ent generate .
echo "Copying Ent code to internal/ent..."
cd .. && mkdir -p ../internal/ent
cp -r *.go */ ../internal/ent/ 2>/dev/null || true
rm -f ../internal/ent/generate.go
rm -rf ../internal/ent/schema
echo "Verifying internal/ent/ent.go exists..."
ls -la ../internal/ent/ent.go || echo "ERROR: ent.go not found!"
else
echo "WARNING: ent/schema directory not found!"
fi
- name: Install golangci-lint
run: |
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin
echo "$(go env GOPATH)/bin" >> $GITHUB_PATH
golangci-lint --version
- name: Run linters
run: make lint
build:
name: Build
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: '1.25.3'
- name: Cache Go modules
uses: actions/cache@v4
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-
- name: Download dependencies
run: go mod download
- name: Install protoc and plugins
run: |
apk add --no-cache protobuf-dev protoc
go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest
echo "$HOME/go/bin" >> $GITHUB_PATH
- name: Generate code
run: |
make generate-proto
echo "Checking for Ent schema directory..."
if [ -d "ent/schema" ]; then
echo "Generating Ent code..."
go install entgo.io/ent/cmd/ent@latest
cd ent/schema && go run -mod=mod entgo.io/ent/cmd/ent generate .
echo "Copying Ent code to internal/ent..."
cd .. && mkdir -p ../internal/ent
cp -r *.go */ ../internal/ent/ 2>/dev/null || true
rm -f ../internal/ent/generate.go
rm -rf ../internal/ent/schema
echo "Verifying internal/ent/ent.go exists..."
ls -la ../internal/ent/ent.go || echo "ERROR: ent.go not found!"
else
echo "WARNING: ent/schema directory not found!"
fi
- name: Build
run: make build
- name: Upload build artifacts
uses: actions/upload-artifact@v3
with:
name: binaries
path: |
bin/platform
bin/api-gateway
bin/auth-service
bin/identity-service
bin/authz-service
bin/audit-service
retention-days: 7
fmt:
name: Format Check
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: '1.25.3'
- name: Check formatting
run: make fmt-check

77
.gitignore vendored Normal file
View File

@@ -0,0 +1,77 @@
# Binaries for programs and plugins
*.exe
*.exe~
*.dll
*.so
*.dylib
bin/
dist/
platform
/api-gateway
/audit-service
/identity-service
/auth-service
/authz-service
# Test binary, built with `go test -c`
*.test
# Output of the go coverage tool, specifically when used with LiteIDE
*.out
# Dependency directories
vendor/
# Go workspace file
go.work
# IDE files
.vscode/
.idea/
*.swp
*.swo
*~
.DS_Store
# Build artifacts
build/
*.a
*.o
# Test coverage files
coverage.txt
coverage.html
*.coverprofile
# Environment-specific files
.env
.env.local
.env.*.local
# Logs
*.log
logs/
# Temporary files
tmp/
temp/
# Documentation build artifacts
docs/site/
docs/.mkdocs_cache/
# OS-specific
Thumbs.db
.direnv/
# Generated protobuf files
api/proto/generated/
# Generated Ent ORM files (but keep schema source files)
internal/ent/
ent/*.go
ent/*/
!ent/schema/
git.dcentral.systems/

24
.golangci.yml Normal file
View File

@@ -0,0 +1,24 @@
# golangci-lint configuration
# See https://golangci-lint.run/usage/configuration/
version: "2"
run:
timeout: 5m
tests: true
modules-download-mode: readonly
linters:
enable:
- errcheck
- govet
- staticcheck
- gosec
disable:
- gocritic # Can be enabled later for stricter checks
issues:
max-issues-per-linter: 0
max-same-issues: 0
# Note: exclusion rules moved in v2 config; keep minimal allowed keys

318
AGENTS.md Normal file
View File

@@ -0,0 +1,318 @@
# Agent Instructions for Go Platform Project
This document provides essential guidance for AI agents and developers working on the Go Platform project. **Always consult the documentation before making changes or implementing features.**
## 🚨 Critical Requirement
**BEFORE making any code changes, architectural decisions, or implementing features, you MUST:**
1. ✅ Review relevant **documentation** files
2. ✅ Check applicable **architecture** documents
3. ✅ Consult relevant **Architecture Decision Records (ADRs)**
4. ✅ Review the **stories** (epic-based implementation tasks)
Failure to follow these guidelines may result in code that doesn't align with the project's architecture, principles, or implementation plan.
---
## 📚 Documentation Structure
All documentation is located in `/goplt/docs/content/`. The project follows a structured documentation approach:
### Core Documentation (`/goplt/docs/content/`)
- **`index.md`**: Main documentation index and navigation
- **`requirements.md`**: High-level architectural principles and requirements
- **`plan.md`**: Epic-based implementation plan with timelines and acceptance criteria
- **`playbook.md`**: Detailed implementation guide, best practices, and technical specifications
### Architecture Documentation (`/goplt/docs/content/architecture/`)
- **`architecture.md`**: System architecture overview with diagrams
- **`architecture-modules.md`**: Module system design and integration patterns
- **`module-requirements.md`**: Detailed requirements for each module
- **`component-relationships.md`**: Component interactions and dependencies
- **`system-behavior.md`**: System behavior overview end-to-end
- **`service-orchestration.md`**: How services work together
- **`module-integration-patterns.md`**: How modules integrate with the platform
- **`operational-scenarios.md`**: Common operational flows and use cases
- **`data-flow-patterns.md`**: How data flows through the system
### Architecture Decision Records (`/goplt/docs/content/adr/`)
All architectural decisions are documented in ADR files (ADR-0001 through ADR-0030+). These records explain:
- The context that led to the decision
- The decision itself
- The rationale and consequences
- Implementation notes
**Key ADRs to review:**
- ADR-0001: Go Module Path
- ADR-0003: Dependency Injection Framework
- ADR-0004: Configuration Management
- ADR-0005: Logging Framework
- ADR-0006: HTTP Framework
- ADR-0007: Project Directory Structure
- ADR-0008: Error Handling Strategy
- ADR-0013: Database ORM
- ADR-0016: OpenTelemetry Observability
- ADR-0017: JWT Token Strategy
- ADR-0021: Module Loading Strategy
- ADR-0022: Cache Implementation
- ADR-0023: Event Bus Implementation
- ADR-0025: Multitenancy Model
- ADR-0027: Rate Limiting Strategy
- ADR-0028: Testing Strategy
- ADR-0029: Microservices Architecture
- ADR-0030: Service Communication Strategy
**Always check ADRs before making architectural decisions!**
### Implementation Stories (`/goplt/docs/content/stories/`)
The project is organized into **8 epics**, each containing specific implementation stories:
- **Epic 0**: Project Setup & Foundation (`epic0/`)
- 0.1: Project Initialization
- 0.2: Configuration Management System
- 0.3: Structured Logging System
- 0.4: CI/CD Pipeline
- 0.5: Dependency Injection and Bootstrap
- **Epic 1**: Core Kernel & Infrastructure (`epic1/`)
- 1.1: Enhanced DI Container
- 1.2: Database Layer
- 1.3: Health & Metrics System
- 1.4: Error Handling
- 1.5: HTTP Server
- 1.6: OpenTelemetry
- 1.7: Service Abstraction Layer
- **Epic 2**: Authentication & Authorization (`epic2/`)
- 2.1: JWT Authentication
- 2.2: Identity Management
- 2.3: RBAC System
- 2.4: Role Management
- 2.5: Audit Logging
- 2.6: Database Seeding
- **Epic 3**: Module Framework (`epic3/`)
- 3.1: Module System Interface
- 3.2: Permission Code Generation
- 3.3: Module Loader
- 3.4: Module CLI
- 3.5: Service Registry
- **Epic 4**: Sample Feature Module (`epic4/`)
- 4.1: Blog Module
- **Epic 5**: Infrastructure Adapters (`epic5/`)
- 5.1: Cache System
- 5.2: Event Bus
- 5.3: Blob Storage
- 5.4: Email Notification
- 5.5: Scheduler Jobs
- 5.6: Secret Store
- 5.7: gRPC Services
- **Epic 6**: Observability & Production Readiness (`epic6/`)
- 6.1: Enhanced Observability
- 6.2: Error Reporting
- 6.3: Grafana Dashboards
- 6.4: Rate Limiting
- 6.5: Security Hardening
- 6.6: Performance Optimization
- **Epic 7**: Testing, Documentation & CI/CD (`epic7/`)
- 7.1: Testing Suite
- 7.2: Documentation
- 7.3: CI/CD Enhancement
- 7.4: Docker Deployment
- **Epic 8**: Advanced Features & Polish (`epic8/`)
Each story file contains:
- **Goal**: What needs to be accomplished
- **Deliverables**: Specific items to be created
- **Acceptance Criteria**: How to verify completion
- **Implementation Notes**: Technical guidance
**Always review the relevant story before implementing a feature!**
---
## 🔍 Workflow for Agents
When working on this project, follow this workflow:
### 0. Git Workflow (MANDATORY)
- **ALWAYS create a new branch** when working on a new feature, bug fix, or enhancement
- Use descriptive branch names (e.g., `feature/epic1-http-server`, `bugfix/auth-token-expiry`, `enhancement/rate-limiting`)
- Branch names should follow the pattern: `{type}/{epic}-{short-description}` or `{type}/{story-id}-{short-description}`
- **ALWAYS create a commit** after successfully implementing a feature that:
- ✅ Builds successfully (`make build` passes)
- ✅ Tests pass (`make test` passes)
- ✅ Lint pass (`make lint` passes)
- ✅ fmt-check pass (`make fmt-check` passes)
- ✅ Meets all acceptance criteria from the story
- Commit messages should be clear and descriptive, referencing the story/epic when applicable
- Never commit directly to `main` branch
### 1. Understand the Task
- Read the user's request carefully
- Identify which epic/story the task relates to
- Determine if it's a new feature, bug fix, or enhancement
### 2. Consult Documentation (MANDATORY)
- **Start with `index.md`** to get oriented
- **Read the relevant story** from `/goplt/docs/content/stories/epicX/`
- **Review architecture documents** that relate to the feature
- **Check ADRs** for any architectural decisions that apply
- **Review `playbook.md`** for implementation patterns and best practices
### 3. Understand the Architecture
- Review the **architecture overview** (`architecture/architecture.md`)
- Check **component relationships** if integrating with existing systems
- Review **module integration patterns** if working on modules
- Understand **data flow patterns** if working on data processing
### 4. Check ADRs
- Search for ADRs related to your task
- Ensure your implementation aligns with documented decisions
- If you need to make a new architectural decision, document it in a new ADR
### 5. Implement According to Stories
- Follow the deliverables specified in the story
- Meet the acceptance criteria
- Use the implementation notes as guidance
- Follow the patterns established in `playbook.md`
- Implement tests
### 6. Verify Alignment
- Ensure code follows Hexagonal Architecture principles
- Verify it aligns with microservices architecture
- Check that it follows plugin-first design
- Confirm security-by-design principles are followed
- Validate observability is properly implemented
### 7. Commit Changes
- **ALWAYS commit** after successful implementation
- Verify that everything is in order before commit:
- there is a Gitea Runner image in ci/pre-commit
- run scripts/pre-commit-check.sh
- Ensure the code builds (`make build`)
- Ensure all tests pass (`make test`)
- Ensure there are no linter issues (`make lint`)
- Ensure there are no fmt issues (`make fmt-check`)
- If there are issues, fix them before comitting
- Verify all acceptance criteria are met
- Write a clear, descriptive commit message
---
## 🏗️ Key Architectural Principles
The project follows these core principles (documented in `requirements.md` and `playbook.md`):
1. **Hexagonal Architecture**
- Clear separation between `pkg/` (interfaces) and `internal/` (implementations)
- Domain code in `internal/domain`
- Only interfaces exported from `pkg/`
2. **Microservices Architecture**
- Each module is an independent service from day one
- Services communicate via gRPC/HTTP
- Service discovery via service registry
3. **Plugin-First Design**
- Extensible architecture supporting static and dynamic modules
- Modules implement the `IModule` interface
- Module loader discovers and loads modules
4. **Security-by-Design**
- JWT authentication
- RBAC/ABAC authorization
- Audit logging
- Context-based user propagation
5. **Observability**
- OpenTelemetry integration
- Structured logging (Zap)
- Prometheus metrics
- Request correlation IDs
6. **Dependency Injection**
- Using `uber-go/fx` for lifecycle management
- Constructor injection preferred
- Service registry pattern
---
## 📋 Quick Reference Checklist
Before implementing any feature, verify:
- [ ] Created a new branch for the feature/bugfix/enhancement
- [ ] Read the relevant story from `/goplt/docs/content/stories/epicX/`
- [ ] Reviewed architecture documents in `/goplt/docs/content/architecture/`
- [ ] Checked applicable ADRs in `/goplt/docs/content/adr/`
- [ ] Consulted `playbook.md` for implementation patterns
- [ ] Understood the epic context and dependencies
- [ ] Verified acceptance criteria are clear
- [ ] Confirmed architectural alignment
After implementing a feature, verify:
- [ ] Code builds successfully (`go build`)
- [ ] All tests pass (`go test`)
- [ ] All acceptance criteria are met
- [ ] Created a commit with a clear, descriptive message
---
## 🚫 Common Mistakes to Avoid
1. **Working directly on main branch** - Always create a feature branch before making changes
2. **Committing without verification** - Never commit code that doesn't build, has failing tests, or doesn't meet acceptance criteria
3. **Implementing without checking stories** - Stories contain specific deliverables and acceptance criteria
4. **Ignoring ADRs** - ADRs document why decisions were made; don't reinvent the wheel
5. **Violating architecture principles** - Code must follow Hexagonal Architecture
6. **Missing acceptance criteria** - All stories have specific criteria that must be met
7. **Not following module patterns** - Modules must implement the `IModule` interface correctly
8. **Skipping observability** - All features must include proper logging, metrics, and tracing
9. **Breaking microservices boundaries** - Services must communicate via defined interfaces
---
## 📖 Additional Resources
- **MkDocs Configuration**: `/goplt/docs/mkdocs.yml` - Documentation site configuration
- **Docker Setup**: `/goplt/docs/Dockerfile` and `docker-compose.yml` - Development environment
- **Makefile**: `/goplt/Makefile` - Build and development commands
- **Story Generator**: `/goplt/docs/content/stories/generate_tasks.py` - Tool for generating story templates
---
## 💡 Tips for Agents
1. **Use semantic search** to find relevant documentation when you're unsure
2. **Read multiple related files** to get complete context
3. **Check the epic README files** (`epicX/README.md`) for epic-level overviews
4. **Review `COMPLETE_TASK_LIST.md`** for a comprehensive task overview
5. **When in doubt, ask for clarification** rather than making assumptions
---
## 📝 Documentation Updates
If you make architectural decisions or significant changes:
1. Update relevant ADRs or create new ones
2. Update architecture documents if structure changes
3. Update stories if implementation details change
4. Keep documentation in sync with code
5. Do not use any emojis
---
**Remember: Documentation is the source of truth. Always consult it before making changes!**

240
Makefile Normal file
View File

@@ -0,0 +1,240 @@
.PHONY: help test test-coverage lint fmt fmt-check build clean docker-build docker-run generate verify
.PHONY: docs-install docs-serve docs-build docs-deploy docs-clean docs-validate
.PHONY: docs-docker-build docs-docker-serve docs-docker-build-site docs-docker-clean docs-docker-compose-up docs-docker-compose-down
# Variables
GO := go
BINARY_NAME := platform
BINARY_PATH := bin/$(BINARY_NAME)
DOCKER_IMAGE := goplt:latest
# Default target
help:
@echo "Available targets:"
@echo ""
@echo "Development commands:"
@echo " make test - Run all tests"
@echo " make test-coverage - Run tests with coverage report"
@echo " make lint - Run linters"
@echo " make fmt - Format code"
@echo " make fmt-check - Check code formatting"
@echo " make build - Build all service binaries"
@echo " make clean - Clean build artifacts"
@echo " make docker-build - Build Docker image"
@echo " make docker-run - Run Docker container"
@echo " make generate - Run code generation"
@echo " make verify - Verify code (fmt, lint, test)"
@echo ""
@echo "Documentation commands (require Python/pip):"
@echo " make docs-install - Install MkDocs dependencies"
@echo " make docs-serve - Serve documentation locally (http://127.0.0.1:8000)"
@echo " make docs-build - Build static documentation site"
@echo " make docs-deploy - Deploy documentation to GitHub Pages"
@echo " make docs-clean - Clean build artifacts"
@echo " make docs-validate - Validate MkDocs configuration"
@echo ""
@echo "Docker commands (no Python installation required):"
@echo " make docs-docker-build - Build Docker image for MkDocs"
@echo " make docs-docker-serve - Serve documentation using Docker"
@echo " make docs-docker-build-site - Build static site in Docker container"
@echo " make docs-docker-clean - Clean Docker images and containers"
@echo " make docs-docker-compose-up - Start docs server with docker-compose"
@echo " make docs-docker-compose-down - Stop docs server with docker-compose"
@echo ""
@echo "Documentation shortcuts:"
@echo " make docs - Alias for docs-serve"
@echo " make build-docs - Alias for docs-build"
@echo " make docs-docker - Alias for docs-docker-serve"
# Development commands
test:
@echo "Running tests..."
CGO_ENABLED=1 $(GO) test -v -race ./...
test-coverage:
@echo "Running tests with coverage..."
CGO_ENABLED=1 $(GO) test -v -race -coverprofile=coverage.out ./...
$(GO) tool cover -html=coverage.out -o coverage.html
@echo "Coverage report generated: coverage.html"
lint:
@echo "Running linters..."
@if command -v golangci-lint > /dev/null; then \
golangci-lint run; \
else \
echo "golangci-lint not found. Install with: go install github.com/golangci/golangci-lint/cmd/golangci-lint@latest"; \
exit 1; \
fi
fmt:
@echo "Formatting code..."
$(GO) fmt ./...
@if command -v goimports > /dev/null; then \
goimports -w .; \
else \
echo "goimports not found. Install with: go install golang.org/x/tools/cmd/goimports@latest"; \
fi
fmt-check:
@echo "Checking code formatting..."
@if [ "$(shell gofmt -s -l . | wc -l)" -gt 0 ]; then \
echo "The following files need formatting:"; \
gofmt -s -d .; \
exit 1; \
fi
@echo "Code is properly formatted"
build:
@echo "Building all service binaries..."
$(GO) build -v -o bin/platform ./cmd/platform
$(GO) build -v -o bin/api-gateway ./cmd/api-gateway
$(GO) build -v -o bin/auth-service ./cmd/auth-service
$(GO) build -v -o bin/identity-service ./cmd/identity-service
$(GO) build -v -o bin/authz-service ./cmd/authz-service
$(GO) build -v -o bin/audit-service ./cmd/audit-service
@echo "Build complete: bin/platform, bin/api-gateway, bin/auth-service, bin/identity-service, bin/authz-service, bin/audit-service"
clean:
@echo "Cleaning build artifacts..."
rm -rf bin/
rm -f coverage.out coverage.html
@echo "Clean complete"
docker-build:
@echo "Building Docker image..."
docker build -t $(DOCKER_IMAGE) .
@echo "Docker image built: $(DOCKER_IMAGE)"
docker-run: docker-build
@echo "Running Docker container..."
docker run --rm -it \
-p 8080:8080 \
$(DOCKER_IMAGE)
generate:
@echo "Running code generation..."
$(GO) generate ./...
generate-proto:
@echo "Generating gRPC code from proto files..."
@if ! command -v protoc > /dev/null; then \
echo "protoc not found. Install Protocol Buffers compiler."; \
exit 1; \
fi
@if ! command -v protoc-gen-go > /dev/null; then \
echo "protoc-gen-go not found. Install with: go install google.golang.org/protobuf/cmd/protoc-gen-go@latest"; \
exit 1; \
fi
@if ! command -v protoc-gen-go-grpc > /dev/null; then \
echo "protoc-gen-go-grpc not found. Install with: go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest"; \
exit 1; \
fi
@mkdir -p api/proto/generated/audit/v1 api/proto/generated/auth/v1 api/proto/generated/authz/v1 api/proto/generated/identity/v1
@protoc --go_out=api/proto/generated --go_opt=paths=source_relative \
--go-grpc_out=api/proto/generated --go-grpc_opt=paths=source_relative \
--proto_path=api/proto \
api/proto/audit.proto
@if [ -f api/proto/generated/audit.pb.go ]; then mv api/proto/generated/audit.pb.go api/proto/generated/audit/v1/audit.pb.go; fi
@if [ -f api/proto/generated/audit_grpc.pb.go ]; then mv api/proto/generated/audit_grpc.pb.go api/proto/generated/audit/v1/audit_grpc.pb.go; fi
@protoc --go_out=api/proto/generated --go_opt=paths=source_relative \
--go-grpc_out=api/proto/generated --go-grpc_opt=paths=source_relative \
--proto_path=api/proto \
api/proto/auth.proto
@if [ -f api/proto/generated/auth.pb.go ]; then mv api/proto/generated/auth.pb.go api/proto/generated/auth/v1/auth.pb.go; fi
@if [ -f api/proto/generated/auth_grpc.pb.go ]; then mv api/proto/generated/auth_grpc.pb.go api/proto/generated/auth/v1/auth_grpc.pb.go; fi
@protoc --go_out=api/proto/generated --go_opt=paths=source_relative \
--go-grpc_out=api/proto/generated --go-grpc_opt=paths=source_relative \
--proto_path=api/proto \
api/proto/authz.proto
@if [ -f api/proto/generated/authz.pb.go ]; then mv api/proto/generated/authz.pb.go api/proto/generated/authz/v1/authz.pb.go; fi
@if [ -f api/proto/generated/authz_grpc.pb.go ]; then mv api/proto/generated/authz_grpc.pb.go api/proto/generated/authz/v1/authz_grpc.pb.go; fi
@protoc --go_out=api/proto/generated --go_opt=paths=source_relative \
--go-grpc_out=api/proto/generated --go-grpc_opt=paths=source_relative \
--proto_path=api/proto \
api/proto/identity.proto
@if [ -f api/proto/generated/identity.pb.go ]; then mv api/proto/generated/identity.pb.go api/proto/generated/identity/v1/identity.pb.go; fi
@if [ -f api/proto/generated/identity_grpc.pb.go ]; then mv api/proto/generated/identity_grpc.pb.go api/proto/generated/identity/v1/identity_grpc.pb.go; fi
@rm -f api/proto/generated/*.pb.go api/proto/generated/*.proto 2>/dev/null || true
@echo "gRPC code generation complete"
verify: fmt-check lint test
@echo "Verification complete"
# Install MkDocs and dependencies
docs-install:
@echo "Installing MkDocs dependencies..."
cd docs && pip install -r requirements.txt
# Serve documentation locally with auto-reload
docs-serve:
@echo "Starting MkDocs development server..."
@echo "Documentation will be available at http://127.0.0.1:8000"
cd docs && mkdocs serve
# Build static documentation site
docs-build:
@echo "Building static documentation site..."
cd docs && mkdocs build
@echo "Build complete! Output is in the 'docs/site/' directory"
# Deploy documentation to GitHub Pages
docs-deploy:
@echo "Deploying documentation to GitHub Pages..."
cd docs && mkdocs gh-deploy
# Clean build artifacts
docs-clean:
@echo "Cleaning MkDocs build artifacts..."
rm -rf docs/site/
rm -rf docs/.mkdocs_cache/
@echo "Clean complete!"
# Validate MkDocs configuration
docs-validate:
@echo "Validating MkDocs configuration..."
cd docs && mkdocs build --strict
@echo "Configuration is valid!"
# Docker commands
docs-docker-build:
@echo "Building Docker image for MkDocs..."
cd docs && docker build -f Dockerfile -t goplt-docs:latest .
docs-docker-serve: docs-docker-build
@echo "Starting MkDocs development server in Docker..."
@echo "Documentation will be available at http://127.0.0.1:8000"
cd docs && docker run --rm -it \
-p 8000:8000 \
-v "$$(pwd):/docs:ro" \
goplt-docs:latest
docs-docker-build-site: docs-docker-build
@echo "Building static documentation site in Docker..."
cd docs && docker run --rm \
-v "$$(pwd):/docs:ro" \
-v "$$(pwd)/site:/docs/site" \
goplt-docs:latest mkdocs build
@echo "Build complete! Output is in the 'docs/site/' directory"
docs-docker-clean:
@echo "Cleaning Docker images and containers..."
-docker stop goplt-docs 2>/dev/null || true
-docker rm goplt-docs 2>/dev/null || true
-docker rmi goplt-docs:latest 2>/dev/null || true
@echo "Clean complete!"
docs-docker-compose-up:
@echo "Starting MkDocs server with docker-compose..."
@echo "Documentation will be available at http://127.0.0.1:8000"
cd docs && docker-compose -f docker-compose.yml up --build
docs-docker-compose-down:
@echo "Stopping MkDocs server..."
cd docs && docker-compose -f docker-compose.yml down
# Convenience aliases
docs: docs-serve
build-docs: docs-build
clean-docs: docs-clean
docs-docker: docs-docker-serve

398
README.md Normal file
View File

@@ -0,0 +1,398 @@
# Go Platform
**SaaS/Enterprise Platform Go Edition**
A modular, extensible platform built with Go that provides a solid foundation for building scalable, secure, and observable applications. The platform supports plugin-based architecture, enabling teams to build feature modules independently while sharing core services.
## Architecture Overview
Go Platform follows **Hexagonal Architecture** principles with clear separation between:
- **Core Kernel**: Foundation services (configuration, logging, observability, dependency injection)
- **Core Services**: Independent microservices (Auth, Identity, Authz, Audit) with gRPC APIs
- **Module Framework**: Plugin system for extending functionality
- **Infrastructure Adapters**: Support for databases, caching, event buses, and job scheduling
- **Security-by-Design**: Built-in JWT authentication, RBAC/ABAC authorization, and audit logging
- **Observability**: OpenTelemetry integration for tracing, metrics, and logging
- **Microservices**: Each service is independently deployable with its own database schema
## Directory Structure
```
goplt/
├── cmd/ # Service entry points
│ ├── platform/ # Main platform entry point
│ ├── auth-service/ # Auth Service (JWT authentication)
│ ├── identity-service/ # Identity Service (user management)
│ ├── authz-service/ # Authz Service (authorization & RBAC)
│ └── audit-service/ # Audit Service (audit logging)
├── internal/ # Private implementation code
│ ├── di/ # Dependency injection container
│ ├── registry/ # Module registry
│ ├── pluginloader/ # Plugin loader (optional)
│ ├── config/ # Configuration implementation
│ ├── logger/ # Logger implementation
│ ├── infra/ # Infrastructure adapters
│ ├── ent/ # Ent ORM schemas
│ └── client/ # Service clients (gRPC)
│ └── grpc/ # gRPC client implementations
├── pkg/ # Public interfaces (exported)
│ ├── config/ # ConfigProvider interface
│ ├── logger/ # Logger interface
│ ├── module/ # IModule interface
│ ├── services/ # Service client interfaces
│ ├── auth/ # Auth interfaces
│ ├── perm/ # Permission DSL
│ └── infra/ # Infrastructure interfaces
├── services/ # Service-specific implementations
│ └── identity/ # Identity service internals
│ └── internal/
│ └── password/ # Password hashing (argon2id)
├── modules/ # Feature modules
│ └── blog/ # Sample Blog module (Epic 4)
├── api/ # API definitions
│ └── proto/ # gRPC protobuf definitions
├── config/ # Configuration files
│ ├── default.yaml
│ ├── development.yaml
│ └── production.yaml
├── docker-compose.yml # Full deployment (all services)
├── docker-compose.dev.yml # Development (infrastructure only)
├── scripts/ # Build/test scripts
├── docs/ # Documentation
├── ops/ # Operations (Grafana dashboards, etc.)
└── .github/
└── workflows/
└── ci.yml
```
## Quick Start
### Prerequisites
- **Go 1.25.3+**: [Install Go](https://golang.org/doc/install)
- **Make**: For using development commands
- **Docker & Docker Compose**: For infrastructure services
- **PostgreSQL 16+**: Database (or use Docker Compose)
- **Consul**: Service registry (or use Docker Compose)
- **NixOS** (optional): For development environment (`shell.nix` provided)
### Installation
1. **Clone the repository**
```bash
git clone git.dcentral.systems/toolz/goplt.git
cd goplt
```
2. **Set up development environment** (NixOS users)
```bash
# If using direnv, it will auto-activate
# Otherwise, activate manually:
nix-shell
```
3. **Install dependencies**
```bash
go mod download
```
4. **Generate code** (protobuf, Ent schemas)
```bash
make generate-proto
make generate-ent
```
### Running Services
#### Option 1: Development Mode (Recommended for Development)
Start infrastructure services (PostgreSQL + Consul) in Docker, run services locally:
```bash
# Start infrastructure
docker-compose -f docker-compose.dev.yml up -d
# Verify infrastructure is running
docker-compose -f docker-compose.dev.yml ps
# Start services locally (in separate terminals)
go run ./cmd/auth-service/*.go # Port 8081
go run ./cmd/identity-service/*.go # Port 8082
go run ./cmd/authz-service/*.go # Port 8083
go run ./cmd/audit-service/*.go # Port 8084
```
#### Option 2: Full Docker Deployment
Run everything in Docker:
```bash
# Build and start all services
docker-compose up -d --build
# View logs
docker-compose logs -f
# Stop all services
docker-compose down
```
### Service Endpoints
Once services are running:
- **Auth Service**: `localhost:8081` (gRPC)
- **Identity Service**: `localhost:8082` (gRPC)
- **Authz Service**: `localhost:8083` (gRPC)
- **Audit Service**: `localhost:8084` (gRPC)
- **Consul UI**: http://localhost:8500/ui
- **PostgreSQL**: `localhost:5432`
### Testing Services
Use `grpcurl` to test gRPC endpoints:
```bash
# List available services
grpcurl -plaintext localhost:8081 list
# Example: Create a user
grpcurl -plaintext -d '{
"email": "user@example.com",
"password": "securepassword123",
"username": "testuser"
}' localhost:8082 identity.v1.IdentityService/CreateUser
# Example: Login
grpcurl -plaintext -d '{
"email": "user@example.com",
"password": "securepassword123"
}' localhost:8081 auth.v1.AuthService/Login
```
See [Epic 2 Summary](docs/content/stories/epic2/SUMMARY.md) for detailed testing instructions.
### Configuration
The platform loads configuration from multiple sources with the following precedence:
1. Environment variables (highest priority)
2. Environment-specific YAML files (`config/development.yaml`, `config/production.yaml`)
3. Base configuration file (`config/default.yaml`)
Example environment variables:
```bash
export SERVER_PORT=8080
export DATABASE_DSN="postgres://user:pass@localhost/dbname"
export LOGGING_LEVEL=debug
```
## Development
### Make Commands
```bash
make help # Show all available commands
make test # Run all tests
make test-coverage # Run tests with coverage report
make lint # Run linters
make fmt # Format code
make fmt-check # Check code formatting
make build # Build platform binary
make generate-proto # Generate gRPC code from protobuf
make generate-ent # Generate Ent ORM code
make clean # Clean build artifacts
make docker-build # Build Docker image
make docker-run # Run Docker container
make verify # Verify code (fmt, lint, test)
```
### Building Services
```bash
# Build individual services
go build ./cmd/auth-service
go build ./cmd/identity-service
go build ./cmd/authz-service
go build ./cmd/audit-service
# Or use make (if targets are defined)
make build-services
```
### Running Tests
```bash
# Run all tests
make test
# Run tests with coverage
make test-coverage
# Run tests for a specific package
go test ./internal/config/...
```
### Code Quality
The project uses:
- **golangci-lint**: For comprehensive linting
- **gofmt**: For code formatting
- **go vet**: For static analysis
Run all checks:
```bash
make verify
```
## Documentation
Comprehensive documentation is available in the `docs/` directory:
- **[Architecture Documentation](docs/content/architecture/)**: System architecture and design patterns
- **[Architecture Decision Records (ADRs)](docs/content/adr/)**: Documented architectural decisions
- **[Implementation Stories](docs/content/stories/)**: Epic-based implementation tasks
- **[Playbook](docs/content/playbook.md)**: Detailed implementation guide and best practices
### View Documentation Locally
```bash
# Using MkDocs (requires Python)
make docs-install
make docs-serve
# Using Docker (no Python required)
make docs-docker
```
Documentation will be available at `http://127.0.0.1:8000`
## Architecture
### Core Kernel
The platform provides a core kernel with essential services:
- **Configuration Management**: Hierarchical configuration with YAML and environment variable support
- **Structured Logging**: JSON-formatted logs with request correlation
- **Dependency Injection**: FX-based DI container for service lifecycle management
- **Health & Metrics**: Health check endpoints and Prometheus metrics
- **Error Handling**: Centralized error handling with proper error wrapping
### Core Services (Epic 2 - ✅ Complete)
Four independent microservices provide authentication and authorization:
- **Auth Service** (`cmd/auth-service/`): JWT token generation, validation, refresh tokens
- **Identity Service** (`cmd/identity-service/`): User CRUD, password management, email verification
- **Authz Service** (`cmd/authz-service/`): RBAC/ABAC authorization, permission resolution
- **Audit Service** (`cmd/audit-service/`): Immutable audit logging with querying
Each service:
- Has its own gRPC API
- Uses its own database schema
- Registers with Consul service registry
- Can be deployed independently
- Communicates via gRPC clients
See [Epic 2 Documentation](docs/content/stories/epic2/) for detailed implementation.
### Module System
Modules extend the platform's functionality by implementing the `IModule` interface:
```go
type IModule interface {
Name() string
Version() string
Initialize(ctx context.Context, app *Application) error
Routes() []Route
Permissions() []Permission
}
```
### Security
- **Authentication**: JWT-based authentication with access and refresh tokens (Auth Service)
- **User Management**: Secure password hashing with Argon2id, email verification, password reset (Identity Service)
- **Authorization**: RBAC/ABAC authorization system with role and permission management (Authz Service)
- **Audit Logging**: Immutable audit trail for security-relevant actions (Audit Service)
- **Rate Limiting**: Configurable rate limiting per endpoint (planned)
### Observability
- **Distributed Tracing**: OpenTelemetry integration for request tracing
- **Metrics**: Prometheus metrics for monitoring
- **Structured Logging**: JSON-formatted logs with correlation IDs
- **Health Checks**: Kubernetes-ready health and readiness endpoints
## Configuration
Configuration is managed through YAML files and environment variables. See `config/default.yaml` for the base configuration structure.
Key configuration sections:
- **Server**: HTTP/gRPC server settings (port, host, timeouts)
- **Database**: Database connection settings (PostgreSQL with schema support)
- **Logging**: Log level, format, and output destination
- **Authentication**: JWT settings and token configuration
- **Services**: Service-specific ports and hosts
- **Registry**: Service registry settings (Consul)
### Environment Variables
Services can be configured via environment variables:
- `ENVIRONMENT`: `development` or `production`
- `DATABASE_DSN`: PostgreSQL connection string
- `REGISTRY_TYPE`: Service registry type (default: `consul`)
- `REGISTRY_CONSUL_ADDRESS`: Consul address (default: `localhost:8500`)
- `AUTH_JWT_SECRET`: JWT signing secret (required for Auth Service)
## Contributing
1. Create a feature branch: `git checkout -b feature/my-feature`
2. Make your changes following the project's architecture principles
3. Run tests and linting: `make verify`
4. Commit your changes with clear messages
5. Push to your branch and create a pull request
## License
[Add license information here]
## Implementation Status
### ✅ Completed
- **Epic 0**: Project Setup & Foundation
- **Epic 1**: Core Kernel & Infrastructure
- **Epic 2**: Core Services (Auth, Identity, Authz, Audit) - **Complete**
- All four services implemented as independent microservices
- gRPC APIs with service discovery
- Database schemas and persistence
- Docker support for all services
### 🚧 In Progress / Planned
- **Epic 3**: Module Framework
- **Epic 4**: Sample Feature Module (Blog)
- **Epic 5**: Infrastructure Adapters
- **Epic 6**: Observability & Production Readiness
- **Epic 7**: Testing, Documentation & CI/CD
- **Epic 8**: Advanced Features & Polish
See [Implementation Plan](docs/content/plan.md) for details.
## Links
- [Architecture Documentation](docs/content/architecture/)
- [ADRs](docs/content/adr/)
- [Implementation Plan](docs/content/plan.md)
- [Playbook](docs/content/playbook.md)
- [Epic 2 Summary](docs/content/stories/epic2/SUMMARY.md) - Core Services implementation details
## Support
For questions and support, please refer to the documentation or create an issue in the repository.

56
api/proto/audit.proto Normal file
View File

@@ -0,0 +1,56 @@
syntax = "proto3";
package audit.v1;
option go_package = "git.dcentral.systems/toolz/goplt/api/proto/generated/audit/v1;auditv1";
// AuditService provides audit logging operations.
service AuditService {
// Record records an audit log entry.
rpc Record(RecordRequest) returns (RecordResponse);
// Query queries audit logs based on filters.
rpc Query(QueryRequest) returns (QueryResponse);
}
// AuditLogEntry represents an audit log entry.
message AuditLogEntry {
string user_id = 1;
string action = 2; // e.g., "user.create", "user.update"
string resource = 3; // e.g., "user", "role"
string resource_id = 4;
string ip_address = 5;
string user_agent = 6;
map<string, string> metadata = 7;
int64 timestamp = 8;
}
// RecordRequest contains an audit log entry to record.
message RecordRequest {
AuditLogEntry entry = 1;
}
// RecordResponse indicates success.
message RecordResponse {
bool success = 1;
string id = 2; // Audit log entry ID
}
// QueryRequest contains filters for querying audit logs.
message QueryRequest {
optional string user_id = 1;
optional string action = 2;
optional string resource = 3;
optional string resource_id = 4;
optional int64 start_time = 5;
optional int64 end_time = 6;
int32 limit = 7; // Max number of results
int32 offset = 8; // Pagination offset
}
// QueryResponse contains audit log entries.
message QueryResponse {
repeated AuditLogEntry entries = 1;
int32 total = 2; // Total number of matching entries
}

71
api/proto/auth.proto Normal file
View File

@@ -0,0 +1,71 @@
syntax = "proto3";
package auth.v1;
option go_package = "git.dcentral.systems/toolz/goplt/api/proto/generated/auth/v1;authv1";
// AuthService provides authentication operations.
service AuthService {
// Login authenticates a user and returns access and refresh tokens.
rpc Login(LoginRequest) returns (LoginResponse);
// RefreshToken refreshes an access token using a refresh token.
rpc RefreshToken(RefreshTokenRequest) returns (RefreshTokenResponse);
// ValidateToken validates a JWT token and returns the token claims.
rpc ValidateToken(ValidateTokenRequest) returns (ValidateTokenResponse);
// Logout invalidates a refresh token.
rpc Logout(LogoutRequest) returns (LogoutResponse);
}
// LoginRequest contains login credentials.
message LoginRequest {
string email = 1;
string password = 2;
}
// LoginResponse contains authentication tokens.
message LoginResponse {
string access_token = 1;
string refresh_token = 2;
int64 expires_in = 3; // seconds
string token_type = 4; // "Bearer"
}
// RefreshTokenRequest contains a refresh token.
message RefreshTokenRequest {
string refresh_token = 1;
}
// RefreshTokenResponse contains new authentication tokens.
message RefreshTokenResponse {
string access_token = 1;
string refresh_token = 2;
int64 expires_in = 3; // seconds
string token_type = 4; // "Bearer"
}
// ValidateTokenRequest contains a JWT token to validate.
message ValidateTokenRequest {
string token = 1;
}
// ValidateTokenResponse contains token claims.
message ValidateTokenResponse {
string user_id = 1;
string email = 2;
repeated string roles = 3;
int64 expires_at = 4;
}
// LogoutRequest contains a refresh token to invalidate.
message LogoutRequest {
string refresh_token = 1;
}
// LogoutResponse indicates success.
message LogoutResponse {
bool success = 1;
}

80
api/proto/authz.proto Normal file
View File

@@ -0,0 +1,80 @@
syntax = "proto3";
package authz.v1;
option go_package = "git.dcentral.systems/toolz/goplt/api/proto/generated/authz/v1;authzv1";
// AuthzService provides authorization operations.
service AuthzService {
// Authorize checks if a user has a specific permission and returns an error if not.
rpc Authorize(AuthorizeRequest) returns (AuthorizeResponse);
// HasPermission checks if a user has a specific permission.
rpc HasPermission(HasPermissionRequest) returns (HasPermissionResponse);
// GetUserPermissions returns all permissions for a user.
rpc GetUserPermissions(GetUserPermissionsRequest) returns (GetUserPermissionsResponse);
// GetUserRoles returns all roles for a user.
rpc GetUserRoles(GetUserRolesRequest) returns (GetUserRolesResponse);
}
// Permission represents a permission in the system.
message Permission {
string id = 1;
string code = 2;
string name = 3;
string description = 4;
}
// Role represents a role in the system.
message Role {
string id = 1;
string name = 2;
string description = 3;
repeated string permissions = 4; // Permission codes
}
// AuthorizeRequest contains user ID and permission to check.
message AuthorizeRequest {
string user_id = 1;
string permission = 2;
}
// AuthorizeResponse indicates authorization result.
message AuthorizeResponse {
bool authorized = 1;
string message = 2;
}
// HasPermissionRequest contains user ID and permission to check.
message HasPermissionRequest {
string user_id = 1;
string permission = 2;
}
// HasPermissionResponse indicates if the user has the permission.
message HasPermissionResponse {
bool has_permission = 1;
}
// GetUserPermissionsRequest contains a user ID.
message GetUserPermissionsRequest {
string user_id = 1;
}
// GetUserPermissionsResponse contains all permissions for the user.
message GetUserPermissionsResponse {
repeated Permission permissions = 1;
}
// GetUserRolesRequest contains a user ID.
message GetUserRolesRequest {
string user_id = 1;
}
// GetUserRolesResponse contains all roles for the user.
message GetUserRolesResponse {
repeated Role roles = 1;
}

148
api/proto/identity.proto Normal file
View File

@@ -0,0 +1,148 @@
syntax = "proto3";
package identity.v1;
option go_package = "git.dcentral.systems/toolz/goplt/api/proto/generated/identity/v1;identityv1";
// IdentityService provides user management operations.
service IdentityService {
// GetUser retrieves a user by ID.
rpc GetUser(GetUserRequest) returns (GetUserResponse);
// GetUserByEmail retrieves a user by email address.
rpc GetUserByEmail(GetUserByEmailRequest) returns (GetUserByEmailResponse);
// CreateUser creates a new user.
rpc CreateUser(CreateUserRequest) returns (CreateUserResponse);
// UpdateUser updates an existing user.
rpc UpdateUser(UpdateUserRequest) returns (UpdateUserResponse);
// DeleteUser deletes a user.
rpc DeleteUser(DeleteUserRequest) returns (DeleteUserResponse);
// VerifyEmail verifies a user's email address using a verification token.
rpc VerifyEmail(VerifyEmailRequest) returns (VerifyEmailResponse);
// RequestPasswordReset requests a password reset token.
rpc RequestPasswordReset(RequestPasswordResetRequest) returns (RequestPasswordResetResponse);
// ResetPassword resets a user's password using a reset token.
rpc ResetPassword(ResetPasswordRequest) returns (ResetPasswordResponse);
// VerifyPassword verifies a user's password.
rpc VerifyPassword(VerifyPasswordRequest) returns (VerifyPasswordResponse);
}
// User represents a user in the system.
message User {
string id = 1;
string email = 2;
string username = 3;
string first_name = 4;
string last_name = 5;
bool email_verified = 6;
int64 created_at = 7;
int64 updated_at = 8;
}
// GetUserRequest contains a user ID.
message GetUserRequest {
string id = 1;
}
// GetUserResponse contains a user.
message GetUserResponse {
User user = 1;
}
// GetUserByEmailRequest contains an email address.
message GetUserByEmailRequest {
string email = 1;
}
// GetUserByEmailResponse contains a user.
message GetUserByEmailResponse {
User user = 1;
}
// CreateUserRequest contains user data for creation.
message CreateUserRequest {
string email = 1;
string username = 2;
string password = 3;
string first_name = 4;
string last_name = 5;
}
// CreateUserResponse contains the created user.
message CreateUserResponse {
User user = 1;
}
// UpdateUserRequest contains user data for update.
message UpdateUserRequest {
string id = 1;
optional string email = 2;
optional string username = 3;
optional string first_name = 4;
optional string last_name = 5;
}
// UpdateUserResponse contains the updated user.
message UpdateUserResponse {
User user = 1;
}
// DeleteUserRequest contains a user ID.
message DeleteUserRequest {
string id = 1;
}
// DeleteUserResponse indicates success.
message DeleteUserResponse {
bool success = 1;
}
// VerifyEmailRequest contains a verification token.
message VerifyEmailRequest {
string token = 1;
}
// VerifyEmailResponse indicates success.
message VerifyEmailResponse {
bool success = 1;
}
// RequestPasswordResetRequest contains an email address.
message RequestPasswordResetRequest {
string email = 1;
}
// RequestPasswordResetResponse indicates success.
message RequestPasswordResetResponse {
bool success = 1;
}
// ResetPasswordRequest contains a reset token and new password.
message ResetPasswordRequest {
string token = 1;
string new_password = 2;
}
// ResetPasswordResponse indicates success.
message ResetPasswordResponse {
bool success = 1;
}
// VerifyPasswordRequest contains email and password.
message VerifyPasswordRequest {
string email = 1;
string password = 2;
}
// VerifyPasswordResponse contains the user if password is valid.
message VerifyPasswordResponse {
User user = 1;
}

40
ci/pre-commit/Dockerfile Normal file
View File

@@ -0,0 +1,40 @@
FROM alpine:latest
# Install system dependencies
RUN apk add --no-cache \
nodejs \
npm \
gcc \
build-base \
musl-dev \
curl \
make \
wget \
tar \
bash \
git \
protobuf \
protobuf-dev
# Install Go 1.25.3
RUN cd /tmp && \
wget -q https://go.dev/dl/go1.25.3.linux-amd64.tar.gz && \
tar -C /usr/local -xzf go1.25.3.linux-amd64.tar.gz && \
rm go1.25.3.linux-amd64.tar.gz
# Set up Go environment
ENV PATH=/usr/local/go/bin:$PATH:/root/go/bin
ENV GOROOT=/usr/local/go
ENV GOPATH=/root/go
ENV CGO_ENABLED=1
ENV GOFLAGS=-buildvcs=false
# Install Go protobuf plugins
RUN go install google.golang.org/protobuf/cmd/protoc-gen-go@latest && \
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest
# Install golangci-lint
RUN curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b /root/go/bin
# Set working directory
WORKDIR /workspace

View File

@@ -0,0 +1,36 @@
# Build stage
FROM golang:1.25-alpine AS builder
WORKDIR /build
# Install build dependencies
RUN apk add --no-cache git make
# Copy go mod files
COPY go.mod go.sum ./
RUN go mod download
# Copy source code
COPY . .
# Build the service
WORKDIR /build/cmd/api-gateway
RUN CGO_ENABLED=0 GOOS=linux go build -o api-gateway -a -installsuffix cgo .
# Runtime stage
FROM alpine:latest
RUN apk --no-cache add ca-certificates tzdata
WORKDIR /app
# Copy binary from builder
COPY --from=builder /build/cmd/api-gateway/api-gateway .
# Copy config files
COPY --from=builder /build/config ./config
EXPOSE 8080
CMD ["./api-gateway"]

168
cmd/api-gateway/main.go Normal file
View File

@@ -0,0 +1,168 @@
// Package main provides the API Gateway service entry point.
package main
import (
"context"
"fmt"
"net/http"
"os"
"time"
"git.dcentral.systems/toolz/goplt/internal/client"
"git.dcentral.systems/toolz/goplt/internal/di"
"git.dcentral.systems/toolz/goplt/internal/health"
"git.dcentral.systems/toolz/goplt/internal/metrics"
"git.dcentral.systems/toolz/goplt/internal/server"
"git.dcentral.systems/toolz/goplt/pkg/config"
"git.dcentral.systems/toolz/goplt/pkg/errorbus"
"git.dcentral.systems/toolz/goplt/pkg/logger"
"git.dcentral.systems/toolz/goplt/pkg/registry"
"git.dcentral.systems/toolz/goplt/services/gateway"
"go.opentelemetry.io/otel/trace"
"go.uber.org/fx"
)
func main() {
// Create DI container with core kernel services
container := di.NewContainer(
// Invoke lifecycle hooks
fx.Invoke(di.RegisterLifecycleHooks),
// Create API Gateway
fx.Invoke(func(
cfg config.ConfigProvider,
log logger.Logger,
healthRegistry *health.Registry,
metricsRegistry *metrics.Metrics,
errorBus errorbus.ErrorPublisher,
tracer trace.TracerProvider,
serviceRegistry registry.ServiceRegistry,
clientFactory *client.ServiceClientFactory,
lc fx.Lifecycle,
) {
// Create HTTP server using server foundation
srv, err := server.NewServer(cfg, log, healthRegistry, metricsRegistry, errorBus, tracer)
if err != nil {
log.Error("Failed to create API Gateway server",
logger.Error(err),
)
os.Exit(1)
}
// Setup gateway routes
gateway, err := gateway.NewGateway(cfg, log, clientFactory, serviceRegistry)
if err != nil {
log.Error("Failed to create API Gateway",
logger.Error(err),
)
os.Exit(1)
}
gateway.SetupRoutes(srv.Router())
// Determine port and host for registration
gatewayPort := cfg.GetInt("gateway.port")
if gatewayPort == 0 {
gatewayPort = cfg.GetInt("server.port")
if gatewayPort == 0 {
gatewayPort = 8080
}
}
// In Docker, always use the Docker service name for health checks
// Consul (also in Docker) needs to reach the service via Docker DNS
gatewayHost := cfg.GetString("gateway.host")
if os.Getenv("ENVIRONMENT") == "production" || os.Getenv("DOCKER") == "true" {
gatewayHost = "api-gateway" // Docker service name - required for Consul health checks
} else if gatewayHost == "" {
gatewayHost = cfg.GetString("server.host")
if gatewayHost == "" || gatewayHost == "0.0.0.0" {
gatewayHost = "localhost" // Local development
}
}
serviceInstance := &registry.ServiceInstance{
ID: fmt.Sprintf("api-gateway-%d", os.Getpid()),
Name: "api-gateway",
Address: gatewayHost,
Port: gatewayPort,
Tags: []string{"gateway", "http"},
Metadata: map[string]string{
"version": "1.0.0",
"protocol": "http",
},
}
// Register lifecycle hooks
lc.Append(fx.Hook{
OnStart: func(ctx context.Context) error {
// Register with service registry
if err := serviceRegistry.Register(ctx, serviceInstance); err != nil {
log.Warn("Failed to register API Gateway with service registry",
logger.Error(err),
)
// Continue anyway - gateway can work without registry
} else {
log.Info("API Gateway registered with service registry",
logger.String("service_id", serviceInstance.ID),
)
}
// Start HTTP server
addr := fmt.Sprintf("%s:%d", cfg.GetString("server.host"), gatewayPort)
log.Info("API Gateway starting",
logger.String("addr", addr),
)
errChan := make(chan error, 1)
go func() {
if err := srv.Start(); err != nil && err != http.ErrServerClosed {
log.Error("API Gateway server failed",
logger.String("error", err.Error()),
)
errChan <- err
}
}()
// Wait a short time to detect immediate binding errors
select {
case err := <-errChan:
return fmt.Errorf("API Gateway failed to start: %w", err)
case <-time.After(500 * time.Millisecond):
log.Info("API Gateway started successfully",
logger.String("addr", addr),
)
return nil
}
},
OnStop: func(ctx context.Context) error {
// Deregister from service registry
if err := serviceRegistry.Deregister(ctx, serviceInstance.ID); err != nil {
log.Warn("Failed to deregister API Gateway from service registry",
logger.Error(err),
)
} else {
log.Info("API Gateway deregistered from service registry")
}
// Shutdown HTTP server
return srv.Shutdown(ctx)
},
})
}),
)
// Create root context
ctx := context.Background()
// Start the application
if err := container.Start(ctx); err != nil {
log := logger.GetGlobalLogger()
if log != nil {
log.Error("Failed to start API Gateway",
logger.Error(err),
)
} else {
fmt.Fprintf(os.Stderr, "Failed to start API Gateway: %v\n", err)
}
os.Exit(1)
}
}

View File

@@ -0,0 +1,156 @@
// Package main provides tests for the API Gateway service entry point.
// Note: Full integration tests for the API Gateway should be in integration test suite
// with testcontainers for service discovery and backend services.
package main
import (
"context"
"testing"
"time"
"git.dcentral.systems/toolz/goplt/internal/di"
"git.dcentral.systems/toolz/goplt/pkg/registry"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.uber.org/fx"
)
// TestAPIGatewaySetup verifies that the API Gateway setup structure is correct.
// Note: Full DI setup requires config files, so this test verifies the structure
// without actually starting the container.
func TestAPIGatewaySetup(t *testing.T) {
t.Parallel()
// Verify that container can be created
// Full setup requires config files which are not available in unit tests
container := di.NewContainer()
require.NotNil(t, container)
// Test that container can be stopped (without starting)
// This verifies the container structure is correct
ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)
defer cancel()
// Stop should work even if container wasn't started
err := container.Stop(ctx)
// It's okay if it errors - we're just testing structure
_ = err
}
// TestServiceInstanceCreation verifies that service instance is created correctly.
func TestServiceInstanceCreation(t *testing.T) {
t.Parallel()
tests := []struct {
name string
host string
port int
expected string
}{
{
name: "default host and port",
host: "",
port: 0,
expected: "localhost:8080",
},
{
name: "custom host and port",
host: "gateway.example.com",
port: 9090,
expected: "gateway.example.com:9090",
},
{
name: "custom host default port",
host: "gateway.example.com",
port: 0,
expected: "gateway.example.com:8080",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Simulate service instance creation logic from main.go
gatewayPort := tt.port
if gatewayPort == 0 {
gatewayPort = 8080
}
gatewayHost := tt.host
if gatewayHost == "" {
gatewayHost = "localhost"
}
serviceInstance := &registry.ServiceInstance{
ID: "api-gateway-test",
Name: "api-gateway",
Address: gatewayHost,
Port: gatewayPort,
Tags: []string{"gateway", "http"},
Metadata: map[string]string{
"version": "1.0.0",
},
}
assert.Equal(t, "api-gateway", serviceInstance.Name)
assert.Equal(t, gatewayHost, serviceInstance.Address)
assert.Equal(t, gatewayPort, serviceInstance.Port)
assert.Contains(t, serviceInstance.Tags, "gateway")
assert.Contains(t, serviceInstance.Tags, "http")
assert.Equal(t, "1.0.0", serviceInstance.Metadata["version"])
})
}
}
// TestLifecycleHooksStructure verifies that lifecycle hooks can be registered.
// Note: Full lifecycle testing requires config files and should be done in integration tests.
func TestLifecycleHooksStructure(t *testing.T) {
t.Parallel()
var onStartCalled bool
var onStopCalled bool
// Create a test container with custom lifecycle hooks (without core module)
// This tests the hook registration mechanism
container := di.NewContainer(
fx.Invoke(func(lc fx.Lifecycle) {
lc.Append(fx.Hook{
OnStart: func(ctx context.Context) error {
onStartCalled = true
return nil
},
OnStop: func(ctx context.Context) error {
onStopCalled = true
return nil
},
})
}),
)
require.NotNil(t, container)
// Start the container to trigger OnStart
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
defer cancel()
// Start in goroutine since it blocks on signal
go func() {
_ = container.Start(ctx)
}()
// Give it a moment to start
time.Sleep(50 * time.Millisecond)
// Stop to trigger OnStop
stopCtx, stopCancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
defer stopCancel()
err := container.Stop(stopCtx)
// Stop may error if container wasn't fully started, which is okay
_ = err
// Verify hooks were called
// Note: OnStart may not be called if container fails to start due to missing config
// This is expected in unit tests - full testing should be in integration tests
if onStartCalled {
assert.True(t, onStopCalled, "OnStop should be called if OnStart was called")
}
}

View File

@@ -0,0 +1,36 @@
# Build stage
FROM golang:1.25-alpine AS builder
WORKDIR /build
# Install build dependencies
RUN apk add --no-cache git make
# Copy go mod files
COPY go.mod go.sum ./
RUN go mod download
# Copy source code
COPY . .
# Build the service
WORKDIR /build/cmd/audit-service
RUN CGO_ENABLED=0 GOOS=linux go build -o audit-service -a -installsuffix cgo .
# Runtime stage
FROM alpine:latest
RUN apk --no-cache add ca-certificates tzdata
WORKDIR /app
# Copy binary from builder
COPY --from=builder /build/cmd/audit-service/audit-service .
# Copy config files
COPY --from=builder /build/config ./config
EXPOSE 8084
CMD ["./audit-service"]

View File

@@ -0,0 +1,351 @@
// Package main provides FX providers for Audit Service.
// This file creates the service inline to avoid importing internal packages.
package main
import (
"context"
"fmt"
"math"
"net"
"time"
auditv1 "git.dcentral.systems/toolz/goplt/api/proto/generated/audit/v1"
"git.dcentral.systems/toolz/goplt/internal/ent"
"git.dcentral.systems/toolz/goplt/internal/ent/auditlog"
"git.dcentral.systems/toolz/goplt/internal/infra/database"
"git.dcentral.systems/toolz/goplt/pkg/config"
"git.dcentral.systems/toolz/goplt/pkg/logger"
"go.uber.org/fx"
"go.uber.org/zap"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/health"
"google.golang.org/grpc/health/grpc_health_v1"
"google.golang.org/grpc/reflection"
"google.golang.org/grpc/status"
)
// auditLogEntry represents an audit log entry.
type auditLogEntry struct {
UserID string
Action string
Resource string
ResourceID string
IPAddress string
UserAgent string
Metadata map[string]string
Timestamp int64
}
// auditLogFilters contains filters for querying audit logs.
type auditLogFilters struct {
UserID *string
Action *string
Resource *string
ResourceID *string
StartTime *int64
EndTime *int64
Limit int
Offset int
}
// auditService provides audit logging functionality.
type auditService struct {
client *database.Client
logger logger.Logger
}
// record records an audit log entry.
func (s *auditService) record(ctx context.Context, entry *auditLogEntry) error {
// Convert metadata map to JSON
metadataJSON := make(map[string]interface{})
for k, v := range entry.Metadata {
metadataJSON[k] = v
}
// Create audit log entry
timestamp := time.Unix(entry.Timestamp, 0)
if entry.Timestamp == 0 {
timestamp = time.Now()
}
create := s.client.AuditLog.Create().
SetID(fmt.Sprintf("%d-%d", time.Now().Unix(), time.Now().UnixNano()%1000000)).
SetUserID(entry.UserID).
SetAction(entry.Action).
SetMetadata(metadataJSON).
SetTimestamp(timestamp)
if entry.Resource != "" {
create = create.SetResource(entry.Resource)
}
if entry.ResourceID != "" {
create = create.SetResourceID(entry.ResourceID)
}
if entry.IPAddress != "" {
create = create.SetIPAddress(entry.IPAddress)
}
if entry.UserAgent != "" {
create = create.SetUserAgent(entry.UserAgent)
}
_, err := create.Save(ctx)
if err != nil {
s.logger.Error("Failed to record audit log",
zap.Error(err),
zap.String("user_id", entry.UserID),
zap.String("action", entry.Action),
)
return fmt.Errorf("failed to record audit log: %w", err)
}
return nil
}
// query queries audit logs based on filters.
func (s *auditService) query(ctx context.Context, filters *auditLogFilters) ([]*auditLogEntry, error) {
query := s.client.AuditLog.Query()
// Apply filters
if filters.UserID != nil {
query = query.Where(auditlog.UserID(*filters.UserID))
}
if filters.Action != nil {
query = query.Where(auditlog.Action(*filters.Action))
}
if filters.Resource != nil {
query = query.Where(auditlog.Resource(*filters.Resource))
}
if filters.ResourceID != nil {
query = query.Where(auditlog.ResourceID(*filters.ResourceID))
}
if filters.StartTime != nil {
query = query.Where(auditlog.TimestampGTE(time.Unix(*filters.StartTime, 0)))
}
if filters.EndTime != nil {
query = query.Where(auditlog.TimestampLTE(time.Unix(*filters.EndTime, 0)))
}
// Apply pagination
if filters.Limit > 0 {
query = query.Limit(filters.Limit)
}
if filters.Offset > 0 {
query = query.Offset(filters.Offset)
}
// Order by timestamp descending
query = query.Order(ent.Desc(auditlog.FieldTimestamp))
// Execute query
auditLogs, err := query.All(ctx)
if err != nil {
s.logger.Error("Failed to query audit logs",
zap.Error(err),
)
return nil, fmt.Errorf("failed to query audit logs: %w", err)
}
// Convert to service entries
entries := make([]*auditLogEntry, 0, len(auditLogs))
for _, log := range auditLogs {
// Convert metadata from map[string]interface{} to map[string]string
metadata := make(map[string]string)
if log.Metadata != nil {
for k, v := range log.Metadata {
if str, ok := v.(string); ok {
metadata[k] = str
} else {
metadata[k] = fmt.Sprintf("%v", v)
}
}
}
entry := &auditLogEntry{
UserID: log.UserID,
Action: log.Action,
Resource: log.Resource,
ResourceID: log.ResourceID,
IPAddress: log.IPAddress,
UserAgent: log.UserAgent,
Metadata: metadata,
Timestamp: log.Timestamp.Unix(),
}
entries = append(entries, entry)
}
return entries, nil
}
// auditServerImpl implements the AuditService gRPC server.
type auditServerImpl struct {
auditv1.UnimplementedAuditServiceServer
service *auditService
logger *zap.Logger
}
// Record records an audit log entry.
func (s *auditServerImpl) Record(ctx context.Context, req *auditv1.RecordRequest) (*auditv1.RecordResponse, error) {
if req.Entry == nil {
return nil, status.Error(codes.InvalidArgument, "entry is required")
}
entry := req.Entry
// Convert proto entry to service entry
serviceEntry := &auditLogEntry{
UserID: entry.UserId,
Action: entry.Action,
Resource: entry.Resource,
ResourceID: entry.ResourceId,
IPAddress: entry.IpAddress,
UserAgent: entry.UserAgent,
Metadata: entry.Metadata,
Timestamp: entry.Timestamp,
}
// Record the audit log
if err := s.service.record(ctx, serviceEntry); err != nil {
s.logger.Error("Failed to record audit log",
zap.Error(err),
zap.String("user_id", entry.UserId),
zap.String("action", entry.Action),
)
return nil, status.Errorf(codes.Internal, "failed to record audit log: %v", err)
}
return &auditv1.RecordResponse{
Success: true,
}, nil
}
// Query queries audit logs based on filters.
func (s *auditServerImpl) Query(ctx context.Context, req *auditv1.QueryRequest) (*auditv1.QueryResponse, error) {
// Convert proto filters to service filters
filters := &auditLogFilters{
Limit: int(req.Limit),
Offset: int(req.Offset),
}
if req.UserId != nil {
userID := *req.UserId
filters.UserID = &userID
}
if req.Action != nil {
action := *req.Action
filters.Action = &action
}
if req.Resource != nil {
resource := *req.Resource
filters.Resource = &resource
}
if req.ResourceId != nil {
resourceID := *req.ResourceId
filters.ResourceID = &resourceID
}
if req.StartTime != nil {
startTime := *req.StartTime
filters.StartTime = &startTime
}
if req.EndTime != nil {
endTime := *req.EndTime
filters.EndTime = &endTime
}
// Query audit logs
entries, err := s.service.query(ctx, filters)
if err != nil {
s.logger.Error("Failed to query audit logs",
zap.Error(err),
)
return nil, status.Errorf(codes.Internal, "failed to query audit logs: %v", err)
}
// Convert service entries to proto entries
protoEntries := make([]*auditv1.AuditLogEntry, 0, len(entries))
for _, entry := range entries {
protoEntries = append(protoEntries, &auditv1.AuditLogEntry{
UserId: entry.UserID,
Action: entry.Action,
Resource: entry.Resource,
ResourceId: entry.ResourceID,
IpAddress: entry.IPAddress,
UserAgent: entry.UserAgent,
Metadata: entry.Metadata,
Timestamp: entry.Timestamp,
})
}
total := len(protoEntries)
var totalInt32 int32
if total > math.MaxInt32 {
totalInt32 = math.MaxInt32
} else {
totalInt32 = int32(total)
}
return &auditv1.QueryResponse{
Entries: protoEntries,
Total: totalInt32,
}, nil
}
// provideAuditService creates the audit service and gRPC server.
func provideAuditService() fx.Option {
return fx.Options(
// Audit service
fx.Provide(func(client *database.Client, log logger.Logger) (*auditService, error) {
return &auditService{
client: client,
logger: log,
}, nil
}),
// gRPC server implementation
fx.Provide(func(auditService *auditService, log logger.Logger) (*auditServerImpl, error) {
zapLogger, _ := zap.NewProduction()
return &auditServerImpl{
service: auditService,
logger: zapLogger,
}, nil
}),
// gRPC server wrapper
fx.Provide(func(
serverImpl *auditServerImpl,
cfg config.ConfigProvider,
log logger.Logger,
) (*grpcServerWrapper, error) {
port := cfg.GetInt("services.audit.port")
if port == 0 {
port = 8084
}
addr := fmt.Sprintf("0.0.0.0:%d", port)
listener, err := net.Listen("tcp", addr)
if err != nil {
return nil, fmt.Errorf("failed to listen on %s: %w", addr, err)
}
grpcServer := grpc.NewServer()
auditv1.RegisterAuditServiceServer(grpcServer, serverImpl)
// Register health service
healthServer := health.NewServer()
grpc_health_v1.RegisterHealthServer(grpcServer, healthServer)
// Set serving status for the default service (empty string) - this is what Consul checks
healthServer.SetServingStatus("", grpc_health_v1.HealthCheckResponse_SERVING)
// Also set for the specific service name
healthServer.SetServingStatus("audit.v1.AuditService", grpc_health_v1.HealthCheckResponse_SERVING)
// Register reflection for grpcurl
reflection.Register(grpcServer)
return &grpcServerWrapper{
server: grpcServer,
listener: listener,
port: port,
logger: log,
}, nil
}),
)
}

228
cmd/audit-service/main.go Normal file
View File

@@ -0,0 +1,228 @@
// Package main provides the entry point for the Audit Service.
package main
import (
"context"
"fmt"
"net"
"os"
"os/signal"
"syscall"
"time"
"git.dcentral.systems/toolz/goplt/internal/di"
healthpkg "git.dcentral.systems/toolz/goplt/internal/health"
"git.dcentral.systems/toolz/goplt/internal/infra/database"
"git.dcentral.systems/toolz/goplt/pkg/config"
"git.dcentral.systems/toolz/goplt/pkg/logger"
"git.dcentral.systems/toolz/goplt/pkg/registry"
"go.uber.org/fx"
"go.uber.org/zap"
"google.golang.org/grpc"
)
// grpcServerWrapper wraps the gRPC server for lifecycle management.
type grpcServerWrapper struct {
server *grpc.Server
listener net.Listener
port int
logger logger.Logger
}
func (s *grpcServerWrapper) Start() error {
s.logger.Info("Starting Audit Service gRPC server",
zap.Int("port", s.port),
zap.String("addr", s.listener.Addr().String()),
)
errChan := make(chan error, 1)
go func() {
if err := s.server.Serve(s.listener); err != nil {
errChan <- err
}
}()
select {
case err := <-errChan:
return fmt.Errorf("gRPC server failed to start: %w", err)
case <-time.After(100 * time.Millisecond):
s.logger.Info("Audit Service gRPC server started successfully",
zap.Int("port", s.port),
)
return nil
}
}
func (s *grpcServerWrapper) Stop(ctx context.Context) error {
s.logger.Info("Stopping Audit Service gRPC server")
stopped := make(chan struct{})
go func() {
s.server.GracefulStop()
close(stopped)
}()
select {
case <-stopped:
s.logger.Info("Audit Service gRPC server stopped gracefully")
return nil
case <-ctx.Done():
s.logger.Warn("Audit Service gRPC server stop timeout, forcing stop")
s.server.Stop()
return ctx.Err()
}
}
func (s *grpcServerWrapper) Port() int {
return s.port
}
func main() {
// Create DI container - services will be created via fx.Provide
// Note: CoreModule() is automatically included by NewContainer()
container := di.NewContainer(
// Database for audit service (audit schema)
fx.Provide(func(cfg config.ConfigProvider, log logger.Logger) (*database.Client, error) {
dsn := cfg.GetString("database.dsn")
if dsn == "" {
return nil, fmt.Errorf("database.dsn is required")
}
client, err := database.NewClientWithSchema(dsn, "audit")
if err != nil {
return nil, err
}
// Run migrations
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
if err := client.Migrate(ctx); err != nil {
log.Warn("Failed to run migrations",
zap.Error(err),
)
} else {
log.Info("Database migrations completed for audit service")
}
return client, nil
}),
// Register database health checker with existing health registry
fx.Invoke(func(registry *healthpkg.Registry, db *database.Client) {
registry.Register("database", healthpkg.NewDatabaseChecker(db))
}),
// Provide audit service and gRPC server (defined in audit_service_fx.go)
provideAuditService(),
// Lifecycle hooks
fx.Invoke(registerLifecycle),
)
// Create root context
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
// Handle signals
sigChan := make(chan os.Signal, 1)
signal.Notify(sigChan, os.Interrupt, syscall.SIGTERM)
// Start the application
if err := container.Start(ctx); err != nil {
log := logger.GetGlobalLogger()
if log != nil {
log.Error("Failed to start Audit Service",
logger.Error(err),
)
} else {
fmt.Fprintf(os.Stderr, "Failed to start Audit Service: %v\n", err)
}
os.Exit(1)
}
// Wait for interrupt signal
<-sigChan
fmt.Println("\nShutting down Audit Service...")
// Create shutdown context with timeout
shutdownCtx, shutdownCancel := context.WithTimeout(context.Background(), 30*time.Second)
defer shutdownCancel()
// Stop the application
if err := container.Stop(shutdownCtx); err != nil {
log := logger.GetGlobalLogger()
if log != nil {
log.Error("Error during Audit Service shutdown",
logger.Error(err),
)
} else {
fmt.Fprintf(os.Stderr, "Error during shutdown: %v\n", err)
}
os.Exit(1)
}
fmt.Println("Audit Service stopped successfully")
}
// registerLifecycle registers lifecycle hooks for the service.
func registerLifecycle(
lc fx.Lifecycle,
grpcServer *grpcServerWrapper,
serviceRegistry registry.ServiceRegistry,
cfg config.ConfigProvider,
log logger.Logger,
) {
lc.Append(fx.Hook{
OnStart: func(ctx context.Context) error {
// Start gRPC server
if err := grpcServer.Start(); err != nil {
return fmt.Errorf("failed to start gRPC server: %w", err)
}
// Register with service registry
serviceID := fmt.Sprintf("audit-service-%d", time.Now().Unix())
// In Docker, always use the Docker service name for health checks
// Consul (also in Docker) needs to reach the service via Docker DNS
host := cfg.GetString("services.audit.host")
if os.Getenv("ENVIRONMENT") == "production" || os.Getenv("DOCKER") == "true" {
host = "audit-service" // Docker service name - required for Consul health checks
} else if host == "" {
host = "localhost" // Local development
}
port := grpcServer.Port()
instance := &registry.ServiceInstance{
ID: serviceID,
Name: "audit-service",
Address: host,
Port: port,
Tags: []string{"grpc", "audit"},
Metadata: map[string]string{
"version": "1.0.0",
"protocol": "grpc",
},
}
if err := serviceRegistry.Register(ctx, instance); err != nil {
log.Warn("Failed to register with service registry",
zap.Error(err),
)
} else {
log.Info("Registered Audit Service with service registry",
zap.String("service_id", serviceID),
zap.String("name", instance.Name),
zap.Int("port", port),
)
}
return nil
},
OnStop: func(ctx context.Context) error {
// Stop gRPC server
if err := grpcServer.Stop(ctx); err != nil {
return fmt.Errorf("failed to stop gRPC server: %w", err)
}
return nil
},
})
}

View File

@@ -0,0 +1,36 @@
# Build stage
FROM golang:1.25-alpine AS builder
WORKDIR /build
# Install build dependencies
RUN apk add --no-cache git make
# Copy go mod files
COPY go.mod go.sum ./
RUN go mod download
# Copy source code
COPY . .
# Build the service
WORKDIR /build/cmd/auth-service
RUN CGO_ENABLED=0 GOOS=linux go build -o auth-service -a -installsuffix cgo .
# Runtime stage
FROM alpine:latest
RUN apk --no-cache add ca-certificates tzdata
WORKDIR /app
# Copy binary from builder
COPY --from=builder /build/cmd/auth-service/auth-service .
# Copy config files
COPY --from=builder /build/config ./config
EXPOSE 8081
CMD ["./auth-service"]

View File

@@ -0,0 +1,425 @@
// Package main provides FX providers for Auth Service.
// This file creates the service inline to avoid importing internal packages.
package main
import (
"context"
"crypto/rand"
"crypto/sha256"
"encoding/hex"
"fmt"
"net"
"time"
authv1 "git.dcentral.systems/toolz/goplt/api/proto/generated/auth/v1"
"git.dcentral.systems/toolz/goplt/internal/ent/refreshtoken"
"git.dcentral.systems/toolz/goplt/internal/infra/database"
"git.dcentral.systems/toolz/goplt/pkg/config"
"git.dcentral.systems/toolz/goplt/pkg/logger"
"git.dcentral.systems/toolz/goplt/pkg/services"
"github.com/golang-jwt/jwt/v5"
"go.uber.org/fx"
"go.uber.org/zap"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/health"
"google.golang.org/grpc/health/grpc_health_v1"
"google.golang.org/grpc/reflection"
"google.golang.org/grpc/status"
)
const (
accessTokenLifetime = 15 * time.Minute
refreshTokenLifetime = 7 * 24 * time.Hour
)
// authService provides authentication functionality.
type authService struct {
client *database.Client
logger logger.Logger
identityClient services.IdentityServiceClient
authzClient services.AuthzServiceClient
jwtSecret []byte
accessTokenExpiry time.Duration
refreshTokenExpiry time.Duration
}
// hashToken hashes a token using SHA256.
func hashToken(token string) string {
h := sha256.Sum256([]byte(token))
return hex.EncodeToString(h[:])
}
// generateRefreshToken generates a random refresh token.
func generateRefreshToken() (string, error) {
b := make([]byte, 32)
if _, err := rand.Read(b); err != nil {
return "", fmt.Errorf("failed to generate token: %w", err)
}
return hex.EncodeToString(b), nil
}
// generateAccessToken generates a JWT access token.
func (s *authService) generateAccessToken(userID, email string, roles []string) (string, int64, error) {
expiresAt := time.Now().Add(s.accessTokenExpiry)
claims := jwt.MapClaims{
"sub": userID,
"email": email,
"roles": roles,
"exp": expiresAt.Unix(),
"iat": time.Now().Unix(),
"token_type": "access",
}
token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
tokenString, err := token.SignedString(s.jwtSecret)
if err != nil {
return "", 0, fmt.Errorf("failed to sign token: %w", err)
}
return tokenString, int64(s.accessTokenExpiry.Seconds()), nil
}
// generateRefreshToken generates a refresh token and stores it in the database.
func (s *authService) generateRefreshToken(ctx context.Context, userID string) (string, error) {
token, err := generateRefreshToken()
if err != nil {
return "", err
}
tokenHash := hashToken(token)
expiresAt := time.Now().Add(s.refreshTokenExpiry)
// Store refresh token in database
_, err = s.client.RefreshToken.Create().
SetID(fmt.Sprintf("%d-%d", time.Now().Unix(), time.Now().UnixNano()%1000000)).
SetUserID(userID).
SetTokenHash(tokenHash).
SetExpiresAt(expiresAt).
Save(ctx)
if err != nil {
return "", fmt.Errorf("failed to store refresh token: %w", err)
}
return token, nil
}
// validateAccessToken validates a JWT access token.
func (s *authService) validateAccessToken(tokenString string) (*jwt.Token, jwt.MapClaims, error) {
token, err := jwt.Parse(tokenString, func(token *jwt.Token) (interface{}, error) {
if _, ok := token.Method.(*jwt.SigningMethodHMAC); !ok {
return nil, fmt.Errorf("unexpected signing method: %v", token.Header["alg"])
}
return s.jwtSecret, nil
})
if err != nil {
return nil, nil, fmt.Errorf("failed to parse token: %w", err)
}
if !token.Valid {
return nil, nil, fmt.Errorf("invalid token")
}
claims, ok := token.Claims.(jwt.MapClaims)
if !ok {
return nil, nil, fmt.Errorf("invalid token claims")
}
// Check expiration
if exp, ok := claims["exp"].(float64); ok {
if time.Now().Unix() > int64(exp) {
return nil, nil, fmt.Errorf("token expired")
}
}
return token, claims, nil
}
// validateRefreshToken validates a refresh token.
func (s *authService) validateRefreshToken(ctx context.Context, tokenString string) (string, error) {
tokenHash := hashToken(tokenString)
// Find refresh token by hash
rt, err := s.client.RefreshToken.Query().
Where(refreshtoken.TokenHash(tokenHash)).
Only(ctx)
if err != nil {
return "", fmt.Errorf("invalid refresh token")
}
// Check if token has expired
if rt.ExpiresAt.Before(time.Now()) {
// Delete expired token
_ = s.client.RefreshToken.DeleteOneID(rt.ID).Exec(ctx)
return "", fmt.Errorf("refresh token expired")
}
return rt.UserID, nil
}
// revokeRefreshToken revokes a refresh token.
func (s *authService) revokeRefreshToken(ctx context.Context, tokenString string) error {
tokenHash := hashToken(tokenString)
// Find and delete refresh token
rt, err := s.client.RefreshToken.Query().
Where(refreshtoken.TokenHash(tokenHash)).
Only(ctx)
if err != nil {
// Token not found, consider it already revoked
return nil
}
return s.client.RefreshToken.DeleteOneID(rt.ID).Exec(ctx)
}
// login authenticates a user and returns tokens.
func (s *authService) login(ctx context.Context, email, password string) (*authv1.LoginResponse, error) {
// Verify credentials with Identity Service
user, err := s.identityClient.VerifyPassword(ctx, email, password)
if err != nil {
return nil, fmt.Errorf("invalid credentials")
}
// Get user roles from Authz Service
roles := []string{}
if s.authzClient != nil {
userRoles, err := s.authzClient.GetUserRoles(ctx, user.ID)
if err != nil {
s.logger.Warn("Failed to get user roles",
zap.String("user_id", user.ID),
zap.Error(err),
)
// Continue without roles rather than failing login
} else {
for _, role := range userRoles {
roles = append(roles, role.Name)
}
}
}
// Generate tokens
accessToken, expiresIn, err := s.generateAccessToken(user.ID, user.Email, roles)
if err != nil {
return nil, fmt.Errorf("failed to generate access token: %w", err)
}
refreshToken, err := s.generateRefreshToken(ctx, user.ID)
if err != nil {
return nil, fmt.Errorf("failed to generate refresh token: %w", err)
}
return &authv1.LoginResponse{
AccessToken: accessToken,
RefreshToken: refreshToken,
ExpiresIn: expiresIn,
TokenType: "Bearer",
}, nil
}
// refreshToken refreshes an access token.
func (s *authService) refreshToken(ctx context.Context, refreshTokenString string) (*authv1.RefreshTokenResponse, error) {
// Validate refresh token
userID, err := s.validateRefreshToken(ctx, refreshTokenString)
if err != nil {
return nil, err
}
// Get user from Identity Service
user, err := s.identityClient.GetUser(ctx, userID)
if err != nil {
return nil, fmt.Errorf("user not found")
}
// Get user roles from Authz Service
roles := []string{}
if s.authzClient != nil {
userRoles, err := s.authzClient.GetUserRoles(ctx, user.ID)
if err != nil {
s.logger.Warn("Failed to get user roles",
zap.String("user_id", user.ID),
zap.Error(err),
)
// Continue without roles rather than failing refresh
} else {
for _, role := range userRoles {
roles = append(roles, role.Name)
}
}
}
// Generate new tokens
accessToken, expiresIn, err := s.generateAccessToken(user.ID, user.Email, roles)
if err != nil {
return nil, fmt.Errorf("failed to generate access token: %w", err)
}
// Generate new refresh token (rotate)
newRefreshToken, err := s.generateRefreshToken(ctx, user.ID)
if err != nil {
return nil, fmt.Errorf("failed to generate refresh token: %w", err)
}
// Revoke old refresh token
_ = s.revokeRefreshToken(ctx, refreshTokenString)
return &authv1.RefreshTokenResponse{
AccessToken: accessToken,
RefreshToken: newRefreshToken,
ExpiresIn: expiresIn,
TokenType: "Bearer",
}, nil
}
// validateToken validates a JWT token.
func (s *authService) validateToken(tokenString string) (*authv1.ValidateTokenResponse, error) {
_, claims, err := s.validateAccessToken(tokenString)
if err != nil {
return nil, err
}
userID, _ := claims["sub"].(string)
email, _ := claims["email"].(string)
exp, _ := claims["exp"].(float64)
roles := []string{}
if rolesClaim, ok := claims["roles"].([]interface{}); ok {
for _, r := range rolesClaim {
if role, ok := r.(string); ok {
roles = append(roles, role)
}
}
}
return &authv1.ValidateTokenResponse{
UserId: userID,
Email: email,
Roles: roles,
ExpiresAt: int64(exp),
}, nil
}
// logout invalidates a refresh token.
func (s *authService) logout(ctx context.Context, refreshTokenString string) error {
return s.revokeRefreshToken(ctx, refreshTokenString)
}
// authServerImpl implements the AuthService gRPC server.
type authServerImpl struct {
authv1.UnimplementedAuthServiceServer
service *authService
logger *zap.Logger
}
// Login authenticates a user and returns tokens.
func (s *authServerImpl) Login(ctx context.Context, req *authv1.LoginRequest) (*authv1.LoginResponse, error) {
resp, err := s.service.login(ctx, req.Email, req.Password)
if err != nil {
return nil, status.Errorf(codes.Unauthenticated, "login failed: %v", err)
}
return resp, nil
}
// RefreshToken refreshes an access token.
func (s *authServerImpl) RefreshToken(ctx context.Context, req *authv1.RefreshTokenRequest) (*authv1.RefreshTokenResponse, error) {
resp, err := s.service.refreshToken(ctx, req.RefreshToken)
if err != nil {
return nil, status.Errorf(codes.Unauthenticated, "refresh failed: %v", err)
}
return resp, nil
}
// ValidateToken validates a JWT token.
func (s *authServerImpl) ValidateToken(ctx context.Context, req *authv1.ValidateTokenRequest) (*authv1.ValidateTokenResponse, error) {
resp, err := s.service.validateToken(req.Token)
if err != nil {
return nil, status.Errorf(codes.Unauthenticated, "validation failed: %v", err)
}
return resp, nil
}
// Logout invalidates a refresh token.
func (s *authServerImpl) Logout(ctx context.Context, req *authv1.LogoutRequest) (*authv1.LogoutResponse, error) {
if err := s.service.logout(ctx, req.RefreshToken); err != nil {
return nil, status.Errorf(codes.Internal, "logout failed: %v", err)
}
return &authv1.LogoutResponse{Success: true}, nil
}
// provideAuthService creates the auth service and gRPC server.
func provideAuthService() fx.Option {
return fx.Options(
// Auth service
fx.Provide(func(
client *database.Client,
log logger.Logger,
identityClient services.IdentityServiceClient,
authzClient services.AuthzServiceClient,
cfg config.ConfigProvider,
) (*authService, error) {
jwtSecret := cfg.GetString("auth.jwt_secret")
if jwtSecret == "" {
return nil, fmt.Errorf("auth.jwt_secret is required in configuration")
}
return &authService{
client: client,
logger: log,
identityClient: identityClient,
authzClient: authzClient,
jwtSecret: []byte(jwtSecret),
accessTokenExpiry: accessTokenLifetime,
refreshTokenExpiry: refreshTokenLifetime,
}, nil
}),
// gRPC server implementation
fx.Provide(func(authService *authService, log logger.Logger) (*authServerImpl, error) {
zapLogger, _ := zap.NewProduction()
return &authServerImpl{
service: authService,
logger: zapLogger,
}, nil
}),
// gRPC server wrapper
fx.Provide(func(
serverImpl *authServerImpl,
cfg config.ConfigProvider,
log logger.Logger,
) (*grpcServerWrapper, error) {
port := cfg.GetInt("services.auth.port")
if port == 0 {
port = 8081
}
addr := fmt.Sprintf("0.0.0.0:%d", port)
listener, err := net.Listen("tcp", addr)
if err != nil {
return nil, fmt.Errorf("failed to listen on %s: %w", addr, err)
}
grpcServer := grpc.NewServer()
authv1.RegisterAuthServiceServer(grpcServer, serverImpl)
// Register health service
healthServer := health.NewServer()
grpc_health_v1.RegisterHealthServer(grpcServer, healthServer)
// Set serving status for the default service (empty string) - this is what Consul checks
healthServer.SetServingStatus("", grpc_health_v1.HealthCheckResponse_SERVING)
// Also set for the specific service name
healthServer.SetServingStatus("auth.v1.AuthService", grpc_health_v1.HealthCheckResponse_SERVING)
// Register reflection for grpcurl
reflection.Register(grpcServer)
return &grpcServerWrapper{
server: grpcServer,
listener: listener,
port: port,
logger: log,
}, nil
}),
)
}

240
cmd/auth-service/main.go Normal file
View File

@@ -0,0 +1,240 @@
// Package main provides the entry point for the Auth Service.
package main
import (
"context"
"fmt"
"net"
"os"
"os/signal"
"syscall"
"time"
"git.dcentral.systems/toolz/goplt/internal/client"
"git.dcentral.systems/toolz/goplt/internal/di"
healthpkg "git.dcentral.systems/toolz/goplt/internal/health"
"git.dcentral.systems/toolz/goplt/internal/infra/database"
"git.dcentral.systems/toolz/goplt/pkg/config"
"git.dcentral.systems/toolz/goplt/pkg/logger"
"git.dcentral.systems/toolz/goplt/pkg/registry"
"git.dcentral.systems/toolz/goplt/pkg/services"
"go.uber.org/fx"
"go.uber.org/zap"
"google.golang.org/grpc"
)
// grpcServerWrapper wraps the gRPC server for lifecycle management.
type grpcServerWrapper struct {
server *grpc.Server
listener net.Listener
port int
logger logger.Logger
}
func (s *grpcServerWrapper) Start() error {
s.logger.Info("Starting Auth Service gRPC server",
zap.Int("port", s.port),
zap.String("addr", s.listener.Addr().String()),
)
errChan := make(chan error, 1)
go func() {
if err := s.server.Serve(s.listener); err != nil {
errChan <- err
}
}()
select {
case err := <-errChan:
return fmt.Errorf("gRPC server failed to start: %w", err)
case <-time.After(100 * time.Millisecond):
s.logger.Info("Auth Service gRPC server started successfully",
zap.Int("port", s.port),
)
return nil
}
}
func (s *grpcServerWrapper) Stop(ctx context.Context) error {
s.logger.Info("Stopping Auth Service gRPC server")
stopped := make(chan struct{})
go func() {
s.server.GracefulStop()
close(stopped)
}()
select {
case <-stopped:
s.logger.Info("Auth Service gRPC server stopped gracefully")
return nil
case <-ctx.Done():
s.logger.Warn("Auth Service gRPC server stop timeout, forcing stop")
s.server.Stop()
return ctx.Err()
}
}
func (s *grpcServerWrapper) Port() int {
return s.port
}
func main() {
// Create DI container
// Note: CoreModule() is automatically included by NewContainer()
container := di.NewContainer(
// Database for auth service (auth schema)
fx.Provide(func(cfg config.ConfigProvider, log logger.Logger) (*database.Client, error) {
dsn := cfg.GetString("database.dsn")
if dsn == "" {
return nil, fmt.Errorf("database.dsn is required")
}
client, err := database.NewClientWithSchema(dsn, "auth")
if err != nil {
return nil, err
}
// Run migrations
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
if err := client.Migrate(ctx); err != nil {
log.Warn("Failed to run migrations",
zap.Error(err),
)
} else {
log.Info("Database migrations completed for auth service")
}
return client, nil
}),
// Register database health checker with existing health registry
fx.Invoke(func(registry *healthpkg.Registry, db *database.Client) {
registry.Register("database", healthpkg.NewDatabaseChecker(db))
}),
// Identity Service client
fx.Provide(func(factory *client.ServiceClientFactory) (services.IdentityServiceClient, error) {
return factory.GetIdentityClient()
}),
// Authz Service client
fx.Provide(func(factory *client.ServiceClientFactory) (services.AuthzServiceClient, error) {
return factory.GetAuthzClient()
}),
// Provide auth service and gRPC server (defined in auth_service_fx.go)
provideAuthService(),
// Lifecycle hooks
fx.Invoke(registerLifecycle),
)
// Create root context
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
// Handle signals
sigChan := make(chan os.Signal, 1)
signal.Notify(sigChan, os.Interrupt, syscall.SIGTERM)
// Start the application
if err := container.Start(ctx); err != nil {
log := logger.GetGlobalLogger()
if log != nil {
log.Error("Failed to start Auth Service",
logger.Error(err),
)
} else {
fmt.Fprintf(os.Stderr, "Failed to start Auth Service: %v\n", err)
}
os.Exit(1)
}
// Wait for interrupt signal
<-sigChan
fmt.Println("\nShutting down Auth Service...")
// Create shutdown context with timeout
shutdownCtx, shutdownCancel := context.WithTimeout(context.Background(), 30*time.Second)
defer shutdownCancel()
// Stop the application
if err := container.Stop(shutdownCtx); err != nil {
log := logger.GetGlobalLogger()
if log != nil {
log.Error("Error during Auth Service shutdown",
logger.Error(err),
)
} else {
fmt.Fprintf(os.Stderr, "Error during shutdown: %v\n", err)
}
os.Exit(1)
}
fmt.Println("Auth Service stopped successfully")
}
// registerLifecycle registers lifecycle hooks for the service.
func registerLifecycle(
lc fx.Lifecycle,
grpcServer *grpcServerWrapper,
serviceRegistry registry.ServiceRegistry,
cfg config.ConfigProvider,
log logger.Logger,
) {
lc.Append(fx.Hook{
OnStart: func(ctx context.Context) error {
// Start gRPC server
if err := grpcServer.Start(); err != nil {
return fmt.Errorf("failed to start gRPC server: %w", err)
}
// Register with service registry
serviceID := fmt.Sprintf("auth-service-%d", time.Now().Unix())
// In Docker, always use the Docker service name for health checks
// Consul (also in Docker) needs to reach the service via Docker DNS
host := cfg.GetString("services.auth.host")
if os.Getenv("ENVIRONMENT") == "production" || os.Getenv("DOCKER") == "true" {
host = "auth-service" // Docker service name - required for Consul health checks
} else if host == "" {
host = "localhost" // Local development
}
port := grpcServer.Port()
instance := &registry.ServiceInstance{
ID: serviceID,
Name: "auth-service",
Address: host,
Port: port,
Tags: []string{"grpc", "auth"},
Metadata: map[string]string{
"version": "1.0.0",
"protocol": "grpc",
},
}
if err := serviceRegistry.Register(ctx, instance); err != nil {
log.Warn("Failed to register with service registry",
zap.Error(err),
)
} else {
log.Info("Registered Auth Service with service registry",
zap.String("service_id", serviceID),
zap.String("name", instance.Name),
zap.Int("port", port),
)
}
return nil
},
OnStop: func(ctx context.Context) error {
// Stop gRPC server
if err := grpcServer.Stop(ctx); err != nil {
return fmt.Errorf("failed to stop gRPC server: %w", err)
}
return nil
},
})
}

View File

@@ -0,0 +1,36 @@
# Build stage
FROM golang:1.25-alpine AS builder
WORKDIR /build
# Install build dependencies
RUN apk add --no-cache git make
# Copy go mod files
COPY go.mod go.sum ./
RUN go mod download
# Copy source code
COPY . .
# Build the service
WORKDIR /build/cmd/authz-service
RUN CGO_ENABLED=0 GOOS=linux go build -o authz-service -a -installsuffix cgo .
# Runtime stage
FROM alpine:latest
RUN apk --no-cache add ca-certificates tzdata
WORKDIR /app
# Copy binary from builder
COPY --from=builder /build/cmd/authz-service/authz-service .
# Copy config files
COPY --from=builder /build/config ./config
EXPOSE 8083
CMD ["./authz-service"]

View File

@@ -0,0 +1,290 @@
// Package main provides FX providers for Authz Service.
// This file creates the service inline to avoid importing internal packages.
package main
import (
"context"
"fmt"
"net"
authzv1 "git.dcentral.systems/toolz/goplt/api/proto/generated/authz/v1"
"git.dcentral.systems/toolz/goplt/internal/ent"
"git.dcentral.systems/toolz/goplt/internal/ent/userrole"
"git.dcentral.systems/toolz/goplt/internal/infra/database"
"git.dcentral.systems/toolz/goplt/pkg/config"
"git.dcentral.systems/toolz/goplt/pkg/logger"
"go.uber.org/fx"
"go.uber.org/zap"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/health"
"google.golang.org/grpc/health/grpc_health_v1"
"google.golang.org/grpc/reflection"
"google.golang.org/grpc/status"
)
// authzService provides authorization functionality.
type authzService struct {
client *database.Client
logger logger.Logger
}
// hasPermission checks if a user has a specific permission.
func (s *authzService) hasPermission(ctx context.Context, userID, permCode string) (bool, error) {
// Get user's roles
userRoles, err := s.client.UserRole.Query().
Where(userrole.UserID(userID)).
WithRole(func(rq *ent.RoleQuery) {
rq.WithRolePermissions(func(rpq *ent.RolePermissionQuery) {
rpq.WithPermission()
})
}).
All(ctx)
if err != nil {
return false, fmt.Errorf("failed to get user roles: %w", err)
}
// Check if any role has the permission
for _, ur := range userRoles {
role := ur.Edges.Role
if role == nil {
continue
}
rolePerms := role.Edges.RolePermissions
for _, rp := range rolePerms {
perm := rp.Edges.Permission
if perm != nil && perm.Name == permCode {
return true, nil
}
}
}
return false, nil
}
// getUserPermissions returns all permissions for a user.
func (s *authzService) getUserPermissions(ctx context.Context, userID string) ([]*ent.Permission, error) {
// Get user's roles
userRoles, err := s.client.UserRole.Query().
Where(userrole.UserID(userID)).
WithRole(func(rq *ent.RoleQuery) {
rq.WithRolePermissions(func(rpq *ent.RolePermissionQuery) {
rpq.WithPermission()
})
}).
All(ctx)
if err != nil {
return nil, fmt.Errorf("failed to get user roles: %w", err)
}
// Collect unique permissions
permMap := make(map[string]*ent.Permission)
for _, ur := range userRoles {
role := ur.Edges.Role
if role == nil {
continue
}
rolePerms := role.Edges.RolePermissions
for _, rp := range rolePerms {
perm := rp.Edges.Permission
if perm != nil {
permMap[perm.ID] = perm
}
}
}
// Convert map to slice
permissions := make([]*ent.Permission, 0, len(permMap))
for _, perm := range permMap {
permissions = append(permissions, perm)
}
return permissions, nil
}
// getUserRoles returns all roles for a user.
func (s *authzService) getUserRoles(ctx context.Context, userID string) ([]*ent.Role, error) {
userRoles, err := s.client.UserRole.Query().
Where(userrole.UserID(userID)).
WithRole(func(rq *ent.RoleQuery) {
rq.WithRolePermissions(func(rpq *ent.RolePermissionQuery) {
rpq.WithPermission()
})
}).
All(ctx)
if err != nil {
return nil, fmt.Errorf("failed to get user roles: %w", err)
}
roles := make([]*ent.Role, 0, len(userRoles))
for _, ur := range userRoles {
if ur.Edges.Role != nil {
roles = append(roles, ur.Edges.Role)
}
}
return roles, nil
}
// authorize checks if a user has a specific permission.
func (s *authzService) authorize(ctx context.Context, userID, permCode string) (bool, string, error) {
hasPerm, err := s.hasPermission(ctx, userID, permCode)
if err != nil {
return false, "", err
}
if !hasPerm {
return false, fmt.Sprintf("user %s does not have permission %s", userID, permCode), nil
}
return true, "authorized", nil
}
// authzServerImpl implements the AuthzService gRPC server.
type authzServerImpl struct {
authzv1.UnimplementedAuthzServiceServer
service *authzService
logger *zap.Logger
}
// Authorize checks if a user has a specific permission.
func (s *authzServerImpl) Authorize(ctx context.Context, req *authzv1.AuthorizeRequest) (*authzv1.AuthorizeResponse, error) {
authorized, message, err := s.service.authorize(ctx, req.UserId, req.Permission)
if err != nil {
return nil, status.Errorf(codes.Internal, "authorization check failed: %v", err)
}
return &authzv1.AuthorizeResponse{
Authorized: authorized,
Message: message,
}, nil
}
// HasPermission checks if a user has a specific permission.
func (s *authzServerImpl) HasPermission(ctx context.Context, req *authzv1.HasPermissionRequest) (*authzv1.HasPermissionResponse, error) {
hasPerm, err := s.service.hasPermission(ctx, req.UserId, req.Permission)
if err != nil {
return nil, status.Errorf(codes.Internal, "permission check failed: %v", err)
}
return &authzv1.HasPermissionResponse{
HasPermission: hasPerm,
}, nil
}
// GetUserPermissions returns all permissions for a user.
func (s *authzServerImpl) GetUserPermissions(ctx context.Context, req *authzv1.GetUserPermissionsRequest) (*authzv1.GetUserPermissionsResponse, error) {
permissions, err := s.service.getUserPermissions(ctx, req.UserId)
if err != nil {
return nil, status.Errorf(codes.Internal, "failed to get user permissions: %v", err)
}
protoPerms := make([]*authzv1.Permission, 0, len(permissions))
for _, perm := range permissions {
protoPerms = append(protoPerms, &authzv1.Permission{
Id: perm.ID,
Code: perm.Name, // Permission.Name is the code (e.g., "blog.post.create")
Name: perm.Name,
Description: "", // Permission schema doesn't have description field
})
}
return &authzv1.GetUserPermissionsResponse{
Permissions: protoPerms,
}, nil
}
// GetUserRoles returns all roles for a user.
func (s *authzServerImpl) GetUserRoles(ctx context.Context, req *authzv1.GetUserRolesRequest) (*authzv1.GetUserRolesResponse, error) {
roles, err := s.service.getUserRoles(ctx, req.UserId)
if err != nil {
return nil, status.Errorf(codes.Internal, "failed to get user roles: %v", err)
}
protoRoles := make([]*authzv1.Role, 0, len(roles))
for _, role := range roles {
// Get permission codes for this role
permCodes := make([]string, 0)
if role.Edges.RolePermissions != nil {
for _, rp := range role.Edges.RolePermissions {
if rp.Edges.Permission != nil {
permCodes = append(permCodes, rp.Edges.Permission.Name)
}
}
}
protoRoles = append(protoRoles, &authzv1.Role{
Id: role.ID,
Name: role.Name,
Description: role.Description,
Permissions: permCodes,
})
}
return &authzv1.GetUserRolesResponse{
Roles: protoRoles,
}, nil
}
// provideAuthzService creates the authz service and gRPC server.
func provideAuthzService() fx.Option {
return fx.Options(
// Authz service
fx.Provide(func(client *database.Client, log logger.Logger) (*authzService, error) {
return &authzService{
client: client,
logger: log,
}, nil
}),
// gRPC server implementation
fx.Provide(func(authzService *authzService, log logger.Logger) (*authzServerImpl, error) {
zapLogger, _ := zap.NewProduction()
return &authzServerImpl{
service: authzService,
logger: zapLogger,
}, nil
}),
// gRPC server wrapper
fx.Provide(func(
serverImpl *authzServerImpl,
cfg config.ConfigProvider,
log logger.Logger,
) (*grpcServerWrapper, error) {
port := cfg.GetInt("services.authz.port")
if port == 0 {
port = 8083
}
addr := fmt.Sprintf("0.0.0.0:%d", port)
listener, err := net.Listen("tcp", addr)
if err != nil {
return nil, fmt.Errorf("failed to listen on %s: %w", addr, err)
}
grpcServer := grpc.NewServer()
authzv1.RegisterAuthzServiceServer(grpcServer, serverImpl)
// Register health service
healthServer := health.NewServer()
grpc_health_v1.RegisterHealthServer(grpcServer, healthServer)
// Set serving status for the default service (empty string) - this is what Consul checks
healthServer.SetServingStatus("", grpc_health_v1.HealthCheckResponse_SERVING)
// Also set for the specific service name
healthServer.SetServingStatus("authz.v1.AuthzService", grpc_health_v1.HealthCheckResponse_SERVING)
// Register reflection for grpcurl
reflection.Register(grpcServer)
return &grpcServerWrapper{
server: grpcServer,
listener: listener,
port: port,
logger: log,
}, nil
}),
)
}

228
cmd/authz-service/main.go Normal file
View File

@@ -0,0 +1,228 @@
// Package main provides the entry point for the Authz Service.
package main
import (
"context"
"fmt"
"net"
"os"
"os/signal"
"syscall"
"time"
"git.dcentral.systems/toolz/goplt/internal/di"
healthpkg "git.dcentral.systems/toolz/goplt/internal/health"
"git.dcentral.systems/toolz/goplt/internal/infra/database"
"git.dcentral.systems/toolz/goplt/pkg/config"
"git.dcentral.systems/toolz/goplt/pkg/logger"
"git.dcentral.systems/toolz/goplt/pkg/registry"
"go.uber.org/fx"
"go.uber.org/zap"
"google.golang.org/grpc"
)
// grpcServerWrapper wraps the gRPC server for lifecycle management.
type grpcServerWrapper struct {
server *grpc.Server
listener net.Listener
port int
logger logger.Logger
}
func (s *grpcServerWrapper) Start() error {
s.logger.Info("Starting Authz Service gRPC server",
zap.Int("port", s.port),
zap.String("addr", s.listener.Addr().String()),
)
errChan := make(chan error, 1)
go func() {
if err := s.server.Serve(s.listener); err != nil {
errChan <- err
}
}()
select {
case err := <-errChan:
return fmt.Errorf("gRPC server failed to start: %w", err)
case <-time.After(100 * time.Millisecond):
s.logger.Info("Authz Service gRPC server started successfully",
zap.Int("port", s.port),
)
return nil
}
}
func (s *grpcServerWrapper) Stop(ctx context.Context) error {
s.logger.Info("Stopping Authz Service gRPC server")
stopped := make(chan struct{})
go func() {
s.server.GracefulStop()
close(stopped)
}()
select {
case <-stopped:
s.logger.Info("Authz Service gRPC server stopped gracefully")
return nil
case <-ctx.Done():
s.logger.Warn("Authz Service gRPC server stop timeout, forcing stop")
s.server.Stop()
return ctx.Err()
}
}
func (s *grpcServerWrapper) Port() int {
return s.port
}
func main() {
// Create DI container
// Note: CoreModule() is automatically included by NewContainer()
container := di.NewContainer(
// Database for authz service (authz schema)
fx.Provide(func(cfg config.ConfigProvider, log logger.Logger) (*database.Client, error) {
dsn := cfg.GetString("database.dsn")
if dsn == "" {
return nil, fmt.Errorf("database.dsn is required")
}
client, err := database.NewClientWithSchema(dsn, "authz")
if err != nil {
return nil, err
}
// Run migrations
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
if err := client.Migrate(ctx); err != nil {
log.Warn("Failed to run migrations",
zap.Error(err),
)
} else {
log.Info("Database migrations completed for authz service")
}
return client, nil
}),
// Register database health checker with existing health registry
fx.Invoke(func(registry *healthpkg.Registry, db *database.Client) {
registry.Register("database", healthpkg.NewDatabaseChecker(db))
}),
// Provide authz service and gRPC server (defined in authz_service_fx.go)
provideAuthzService(),
// Lifecycle hooks
fx.Invoke(registerLifecycle),
)
// Create root context
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
// Handle signals
sigChan := make(chan os.Signal, 1)
signal.Notify(sigChan, os.Interrupt, syscall.SIGTERM)
// Start the application
if err := container.Start(ctx); err != nil {
log := logger.GetGlobalLogger()
if log != nil {
log.Error("Failed to start Authz Service",
logger.Error(err),
)
} else {
fmt.Fprintf(os.Stderr, "Failed to start Authz Service: %v\n", err)
}
os.Exit(1)
}
// Wait for interrupt signal
<-sigChan
fmt.Println("\nShutting down Authz Service...")
// Create shutdown context with timeout
shutdownCtx, shutdownCancel := context.WithTimeout(context.Background(), 30*time.Second)
defer shutdownCancel()
// Stop the application
if err := container.Stop(shutdownCtx); err != nil {
log := logger.GetGlobalLogger()
if log != nil {
log.Error("Error during Authz Service shutdown",
logger.Error(err),
)
} else {
fmt.Fprintf(os.Stderr, "Error during shutdown: %v\n", err)
}
os.Exit(1)
}
fmt.Println("Authz Service stopped successfully")
}
// registerLifecycle registers lifecycle hooks for the service.
func registerLifecycle(
lc fx.Lifecycle,
grpcServer *grpcServerWrapper,
serviceRegistry registry.ServiceRegistry,
cfg config.ConfigProvider,
log logger.Logger,
) {
lc.Append(fx.Hook{
OnStart: func(ctx context.Context) error {
// Start gRPC server
if err := grpcServer.Start(); err != nil {
return fmt.Errorf("failed to start gRPC server: %w", err)
}
// Register with service registry
serviceID := fmt.Sprintf("authz-service-%d", time.Now().Unix())
// In Docker, always use the Docker service name for health checks
// Consul (also in Docker) needs to reach the service via Docker DNS
host := cfg.GetString("services.authz.host")
if os.Getenv("ENVIRONMENT") == "production" || os.Getenv("DOCKER") == "true" {
host = "authz-service" // Docker service name - required for Consul health checks
} else if host == "" {
host = "localhost" // Local development
}
port := grpcServer.Port()
instance := &registry.ServiceInstance{
ID: serviceID,
Name: "authz-service",
Address: host,
Port: port,
Tags: []string{"grpc", "authz"},
Metadata: map[string]string{
"version": "1.0.0",
"protocol": "grpc",
},
}
if err := serviceRegistry.Register(ctx, instance); err != nil {
log.Warn("Failed to register with service registry",
zap.Error(err),
)
} else {
log.Info("Registered Authz Service with service registry",
zap.String("service_id", serviceID),
zap.String("name", instance.Name),
zap.Int("port", port),
)
}
return nil
},
OnStop: func(ctx context.Context) error {
// Stop gRPC server
if err := grpcServer.Stop(ctx); err != nil {
return fmt.Errorf("failed to stop gRPC server: %w", err)
}
return nil
},
})
}

View File

@@ -0,0 +1,36 @@
# Build stage
FROM golang:1.25-alpine AS builder
WORKDIR /build
# Install build dependencies
RUN apk add --no-cache git make
# Copy go mod files
COPY go.mod go.sum ./
RUN go mod download
# Copy source code
COPY . .
# Build the service
WORKDIR /build/cmd/identity-service
RUN CGO_ENABLED=0 GOOS=linux go build -o identity-service -a -installsuffix cgo .
# Runtime stage
FROM alpine:latest
RUN apk --no-cache add ca-certificates tzdata
WORKDIR /app
# Copy binary from builder
COPY --from=builder /build/cmd/identity-service/identity-service .
# Copy config files
COPY --from=builder /build/config ./config
EXPOSE 8082
CMD ["./identity-service"]

View File

@@ -0,0 +1,448 @@
// Package main provides FX providers for Identity Service.
// This file creates the service inline to avoid importing internal packages.
package main
import (
"context"
"crypto/rand"
"crypto/subtle"
"encoding/base64"
"fmt"
"math"
"net"
"strings"
"time"
identityv1 "git.dcentral.systems/toolz/goplt/api/proto/generated/identity/v1"
"git.dcentral.systems/toolz/goplt/internal/ent"
"git.dcentral.systems/toolz/goplt/internal/ent/user"
"git.dcentral.systems/toolz/goplt/internal/infra/database"
"git.dcentral.systems/toolz/goplt/pkg/config"
"git.dcentral.systems/toolz/goplt/pkg/logger"
"go.uber.org/fx"
"go.uber.org/zap"
"golang.org/x/crypto/argon2"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/health"
"google.golang.org/grpc/health/grpc_health_v1"
"google.golang.org/grpc/reflection"
"google.golang.org/grpc/status"
)
// userService provides user management functionality.
type userService struct {
client *database.Client
logger logger.Logger
}
// generateToken generates a random token.
func generateToken() (string, error) {
b := make([]byte, 32)
if _, err := rand.Read(b); err != nil {
return "", fmt.Errorf("failed to generate token: %w", err)
}
return fmt.Sprintf("%x", b), nil
}
// hashPassword hashes a password using argon2id.
func hashPassword(password string) (string, error) {
salt := make([]byte, 16)
if _, err := rand.Read(salt); err != nil {
return "", fmt.Errorf("failed to generate salt: %w", err)
}
hash := argon2.IDKey([]byte(password), salt, 3, 64*1024, 4, 32)
b64Salt := base64.RawStdEncoding.EncodeToString(salt)
b64Hash := base64.RawStdEncoding.EncodeToString(hash)
return fmt.Sprintf("$argon2id$v=%d$m=%d,t=%d,p=%d$%s$%s",
argon2.Version, 64*1024, 3, 4, b64Salt, b64Hash), nil
}
// verifyPassword verifies a password against a hash.
func verifyPassword(password, hash string) (bool, error) {
// Simplified verification - in production use proper parsing
parts := strings.Split(hash, "$")
if len(parts) != 6 {
return false, fmt.Errorf("invalid hash format")
}
salt, err := base64.RawStdEncoding.DecodeString(parts[4])
if err != nil {
return false, err
}
expectedHash, err := base64.RawStdEncoding.DecodeString(parts[5])
if err != nil {
return false, err
}
hashLen := len(expectedHash)
if hashLen < 0 || hashLen > math.MaxUint32 {
return false, fmt.Errorf("invalid hash length: %d", hashLen)
}
var hashLenUint32 uint32
if hashLen > math.MaxUint32 {
hashLenUint32 = math.MaxUint32
} else {
hashLenUint32 = uint32(hashLen)
}
actualHash := argon2.IDKey([]byte(password), salt, 3, 64*1024, 4, hashLenUint32)
return subtle.ConstantTimeCompare(expectedHash, actualHash) == 1, nil
}
// createUser creates a new user.
func (s *userService) createUser(ctx context.Context, email, username, pwd, firstName, lastName string) (*ent.User, error) {
exists, err := s.client.User.Query().Where(user.Email(email)).Exist(ctx)
if err != nil {
return nil, fmt.Errorf("failed to check email: %w", err)
}
if exists {
return nil, fmt.Errorf("email already exists")
}
passwordHash, err := hashPassword(pwd)
if err != nil {
return nil, fmt.Errorf("failed to hash password: %w", err)
}
verificationToken, err := generateToken()
if err != nil {
return nil, fmt.Errorf("failed to generate token: %w", err)
}
create := s.client.User.Create().
SetID(fmt.Sprintf("%d", time.Now().UnixNano())).
SetEmail(email).
SetPasswordHash(passwordHash).
SetVerified(false).
SetEmailVerificationToken(verificationToken)
if username != "" {
create = create.SetUsername(username)
}
if firstName != "" {
create = create.SetFirstName(firstName)
}
if lastName != "" {
create = create.SetLastName(lastName)
}
u, err := create.Save(ctx)
if err != nil {
return nil, fmt.Errorf("failed to create user: %w", err)
}
return u, nil
}
// getUser retrieves a user by ID.
func (s *userService) getUser(ctx context.Context, id string) (*ent.User, error) {
u, err := s.client.User.Get(ctx, id)
if err != nil {
return nil, fmt.Errorf("failed to get user: %w", err)
}
return u, nil
}
// getUserByEmail retrieves a user by email.
func (s *userService) getUserByEmail(ctx context.Context, email string) (*ent.User, error) {
u, err := s.client.User.Query().Where(user.Email(email)).Only(ctx)
if err != nil {
return nil, fmt.Errorf("failed to get user: %w", err)
}
return u, nil
}
// updateUser updates a user.
func (s *userService) updateUser(ctx context.Context, id string, email, username, firstName, lastName *string) (*ent.User, error) {
update := s.client.User.UpdateOneID(id)
if email != nil {
exists, err := s.client.User.Query().
Where(user.Email(*email), user.IDNEQ(id)).
Exist(ctx)
if err != nil {
return nil, fmt.Errorf("failed to check email: %w", err)
}
if exists {
return nil, fmt.Errorf("email already taken")
}
update = update.SetEmail(*email)
}
if username != nil {
update = update.SetUsername(*username)
}
if firstName != nil {
update = update.SetFirstName(*firstName)
}
if lastName != nil {
update = update.SetLastName(*lastName)
}
return update.Save(ctx)
}
// deleteUser deletes a user.
func (s *userService) deleteUser(ctx context.Context, id string) error {
return s.client.User.DeleteOneID(id).Exec(ctx)
}
// verifyEmail verifies a user's email.
func (s *userService) verifyEmail(ctx context.Context, token string) error {
u, err := s.client.User.Query().Where(user.EmailVerificationToken(token)).Only(ctx)
if err != nil {
return fmt.Errorf("invalid token")
}
_, err = s.client.User.UpdateOneID(u.ID).
SetVerified(true).
ClearEmailVerificationToken().
Save(ctx)
return err
}
// requestPasswordReset requests a password reset.
func (s *userService) requestPasswordReset(ctx context.Context, email string) (string, error) {
u, err := s.getUserByEmail(ctx, email)
if err != nil {
return "", nil // Don't reveal if user exists
}
token, err := generateToken()
if err != nil {
return "", err
}
expiresAt := time.Now().Add(24 * time.Hour)
_, err = s.client.User.UpdateOneID(u.ID).
SetPasswordResetToken(token).
SetPasswordResetExpiresAt(expiresAt).
Save(ctx)
return token, err
}
// resetPassword resets a password.
func (s *userService) resetPassword(ctx context.Context, token, newPassword string) error {
u, err := s.client.User.Query().Where(user.PasswordResetToken(token)).Only(ctx)
if err != nil {
return fmt.Errorf("invalid token")
}
if !u.PasswordResetExpiresAt.IsZero() && u.PasswordResetExpiresAt.Before(time.Now()) {
return fmt.Errorf("token expired")
}
passwordHash, err := hashPassword(newPassword)
if err != nil {
return err
}
_, err = s.client.User.UpdateOneID(u.ID).
SetPasswordHash(passwordHash).
ClearPasswordResetToken().
ClearPasswordResetExpiresAt().
Save(ctx)
return err
}
// verifyPassword verifies a password.
func (s *userService) verifyPassword(ctx context.Context, email, pwd string) (*ent.User, error) {
u, err := s.getUserByEmail(ctx, email)
if err != nil {
return nil, err
}
valid, err := verifyPassword(pwd, u.PasswordHash)
if err != nil || !valid {
return nil, fmt.Errorf("invalid password")
}
return u, nil
}
// identityServerImpl implements the IdentityService gRPC server.
type identityServerImpl struct {
identityv1.UnimplementedIdentityServiceServer
service *userService
logger *zap.Logger
}
// GetUser retrieves a user by ID.
func (s *identityServerImpl) GetUser(ctx context.Context, req *identityv1.GetUserRequest) (*identityv1.GetUserResponse, error) {
u, err := s.service.getUser(ctx, req.Id)
if err != nil {
return nil, status.Errorf(codes.NotFound, "user not found: %v", err)
}
return &identityv1.GetUserResponse{
User: &identityv1.User{
Id: u.ID,
Email: u.Email,
Username: u.Username,
FirstName: u.FirstName,
LastName: u.LastName,
EmailVerified: u.Verified,
CreatedAt: u.CreatedAt.Unix(),
UpdatedAt: u.UpdatedAt.Unix(),
},
}, nil
}
// GetUserByEmail retrieves a user by email.
func (s *identityServerImpl) GetUserByEmail(ctx context.Context, req *identityv1.GetUserByEmailRequest) (*identityv1.GetUserByEmailResponse, error) {
u, err := s.service.getUserByEmail(ctx, req.Email)
if err != nil {
return nil, status.Errorf(codes.NotFound, "user not found: %v", err)
}
return &identityv1.GetUserByEmailResponse{
User: &identityv1.User{
Id: u.ID,
Email: u.Email,
Username: u.Username,
FirstName: u.FirstName,
LastName: u.LastName,
EmailVerified: u.Verified,
CreatedAt: u.CreatedAt.Unix(),
UpdatedAt: u.UpdatedAt.Unix(),
},
}, nil
}
// CreateUser creates a new user.
func (s *identityServerImpl) CreateUser(ctx context.Context, req *identityv1.CreateUserRequest) (*identityv1.CreateUserResponse, error) {
u, err := s.service.createUser(ctx, req.Email, req.Username, req.Password, req.FirstName, req.LastName)
if err != nil {
return nil, status.Errorf(codes.Internal, "failed to create user: %v", err)
}
return &identityv1.CreateUserResponse{
User: &identityv1.User{
Id: u.ID,
Email: u.Email,
Username: u.Username,
FirstName: u.FirstName,
LastName: u.LastName,
EmailVerified: u.Verified,
CreatedAt: u.CreatedAt.Unix(),
UpdatedAt: u.UpdatedAt.Unix(),
},
}, nil
}
// UpdateUser updates a user.
func (s *identityServerImpl) UpdateUser(ctx context.Context, req *identityv1.UpdateUserRequest) (*identityv1.UpdateUserResponse, error) {
u, err := s.service.updateUser(ctx, req.Id, req.Email, req.Username, req.FirstName, req.LastName)
if err != nil {
return nil, status.Errorf(codes.Internal, "failed to update user: %v", err)
}
return &identityv1.UpdateUserResponse{
User: &identityv1.User{
Id: u.ID,
Email: u.Email,
Username: u.Username,
FirstName: u.FirstName,
LastName: u.LastName,
EmailVerified: u.Verified,
CreatedAt: u.CreatedAt.Unix(),
UpdatedAt: u.UpdatedAt.Unix(),
},
}, nil
}
// DeleteUser deletes a user.
func (s *identityServerImpl) DeleteUser(ctx context.Context, req *identityv1.DeleteUserRequest) (*identityv1.DeleteUserResponse, error) {
if err := s.service.deleteUser(ctx, req.Id); err != nil {
return nil, status.Errorf(codes.Internal, "failed to delete user: %v", err)
}
return &identityv1.DeleteUserResponse{Success: true}, nil
}
// VerifyEmail verifies a user's email.
func (s *identityServerImpl) VerifyEmail(ctx context.Context, req *identityv1.VerifyEmailRequest) (*identityv1.VerifyEmailResponse, error) {
if err := s.service.verifyEmail(ctx, req.Token); err != nil {
return nil, status.Errorf(codes.InvalidArgument, "failed to verify email: %v", err)
}
return &identityv1.VerifyEmailResponse{Success: true}, nil
}
// RequestPasswordReset requests a password reset.
func (s *identityServerImpl) RequestPasswordReset(ctx context.Context, req *identityv1.RequestPasswordResetRequest) (*identityv1.RequestPasswordResetResponse, error) {
_, err := s.service.requestPasswordReset(ctx, req.Email)
if err != nil {
return nil, status.Errorf(codes.Internal, "failed to request password reset: %v", err)
}
return &identityv1.RequestPasswordResetResponse{Success: true}, nil
}
// ResetPassword resets a password.
func (s *identityServerImpl) ResetPassword(ctx context.Context, req *identityv1.ResetPasswordRequest) (*identityv1.ResetPasswordResponse, error) {
if err := s.service.resetPassword(ctx, req.Token, req.NewPassword); err != nil {
return nil, status.Errorf(codes.InvalidArgument, "failed to reset password: %v", err)
}
return &identityv1.ResetPasswordResponse{Success: true}, nil
}
// VerifyPassword verifies a user's password.
func (s *identityServerImpl) VerifyPassword(ctx context.Context, req *identityv1.VerifyPasswordRequest) (*identityv1.VerifyPasswordResponse, error) {
u, err := s.service.verifyPassword(ctx, req.Email, req.Password)
if err != nil {
return nil, status.Errorf(codes.Unauthenticated, "invalid credentials: %v", err)
}
return &identityv1.VerifyPasswordResponse{
User: &identityv1.User{
Id: u.ID,
Email: u.Email,
Username: u.Username,
FirstName: u.FirstName,
LastName: u.LastName,
EmailVerified: u.Verified,
CreatedAt: u.CreatedAt.Unix(),
UpdatedAt: u.UpdatedAt.Unix(),
},
}, nil
}
// provideIdentityService creates the identity service and gRPC server.
func provideIdentityService() fx.Option {
return fx.Options(
// User service
fx.Provide(func(client *database.Client, log logger.Logger) (*userService, error) {
return &userService{
client: client,
logger: log,
}, nil
}),
// gRPC server implementation
fx.Provide(func(userService *userService, log logger.Logger) (*identityServerImpl, error) {
zapLogger, _ := zap.NewProduction()
return &identityServerImpl{
service: userService,
logger: zapLogger,
}, nil
}),
// gRPC server wrapper
fx.Provide(func(
serverImpl *identityServerImpl,
cfg config.ConfigProvider,
log logger.Logger,
) (*grpcServerWrapper, error) {
port := cfg.GetInt("services.identity.port")
if port == 0 {
port = 8082
}
addr := fmt.Sprintf("0.0.0.0:%d", port)
listener, err := net.Listen("tcp", addr)
if err != nil {
return nil, fmt.Errorf("failed to listen on %s: %w", addr, err)
}
grpcServer := grpc.NewServer()
identityv1.RegisterIdentityServiceServer(grpcServer, serverImpl)
// Register health service
healthServer := health.NewServer()
grpc_health_v1.RegisterHealthServer(grpcServer, healthServer)
// Set serving status for the default service (empty string) - this is what Consul checks
healthServer.SetServingStatus("", grpc_health_v1.HealthCheckResponse_SERVING)
// Also set for the specific service name
healthServer.SetServingStatus("identity.v1.IdentityService", grpc_health_v1.HealthCheckResponse_SERVING)
// Register reflection for grpcurl
reflection.Register(grpcServer)
return &grpcServerWrapper{
server: grpcServer,
listener: listener,
port: port,
logger: log,
}, nil
}),
)
}

View File

@@ -0,0 +1,228 @@
// Package main provides the entry point for the Identity Service.
package main
import (
"context"
"fmt"
"net"
"os"
"os/signal"
"syscall"
"time"
"git.dcentral.systems/toolz/goplt/internal/di"
healthpkg "git.dcentral.systems/toolz/goplt/internal/health"
"git.dcentral.systems/toolz/goplt/internal/infra/database"
"git.dcentral.systems/toolz/goplt/pkg/config"
"git.dcentral.systems/toolz/goplt/pkg/logger"
"git.dcentral.systems/toolz/goplt/pkg/registry"
"go.uber.org/fx"
"go.uber.org/zap"
"google.golang.org/grpc"
)
// grpcServerWrapper wraps the gRPC server for lifecycle management.
type grpcServerWrapper struct {
server *grpc.Server
listener net.Listener
port int
logger logger.Logger
}
func (s *grpcServerWrapper) Start() error {
s.logger.Info("Starting Identity Service gRPC server",
zap.Int("port", s.port),
zap.String("addr", s.listener.Addr().String()),
)
errChan := make(chan error, 1)
go func() {
if err := s.server.Serve(s.listener); err != nil {
errChan <- err
}
}()
select {
case err := <-errChan:
return fmt.Errorf("gRPC server failed to start: %w", err)
case <-time.After(100 * time.Millisecond):
s.logger.Info("Identity Service gRPC server started successfully",
zap.Int("port", s.port),
)
return nil
}
}
func (s *grpcServerWrapper) Stop(ctx context.Context) error {
s.logger.Info("Stopping Identity Service gRPC server")
stopped := make(chan struct{})
go func() {
s.server.GracefulStop()
close(stopped)
}()
select {
case <-stopped:
s.logger.Info("Identity Service gRPC server stopped gracefully")
return nil
case <-ctx.Done():
s.logger.Warn("Identity Service gRPC server stop timeout, forcing stop")
s.server.Stop()
return ctx.Err()
}
}
func (s *grpcServerWrapper) Port() int {
return s.port
}
func main() {
// Create DI container
// Note: CoreModule() is automatically included by NewContainer()
container := di.NewContainer(
// Database for identity service (identity schema)
fx.Provide(func(cfg config.ConfigProvider, log logger.Logger) (*database.Client, error) {
dsn := cfg.GetString("database.dsn")
if dsn == "" {
return nil, fmt.Errorf("database.dsn is required")
}
client, err := database.NewClientWithSchema(dsn, "identity")
if err != nil {
return nil, err
}
// Run migrations
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
if err := client.Migrate(ctx); err != nil {
log.Warn("Failed to run migrations",
zap.Error(err),
)
} else {
log.Info("Database migrations completed for identity service")
}
return client, nil
}),
// Register database health checker with existing health registry
fx.Invoke(func(registry *healthpkg.Registry, db *database.Client) {
registry.Register("database", healthpkg.NewDatabaseChecker(db))
}),
// Provide identity service and gRPC server (defined in identity_service_fx.go)
provideIdentityService(),
// Lifecycle hooks
fx.Invoke(registerLifecycle),
)
// Create root context
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
// Handle signals
sigChan := make(chan os.Signal, 1)
signal.Notify(sigChan, os.Interrupt, syscall.SIGTERM)
// Start the application
if err := container.Start(ctx); err != nil {
log := logger.GetGlobalLogger()
if log != nil {
log.Error("Failed to start Identity Service",
logger.Error(err),
)
} else {
fmt.Fprintf(os.Stderr, "Failed to start Identity Service: %v\n", err)
}
os.Exit(1)
}
// Wait for interrupt signal
<-sigChan
fmt.Println("\nShutting down Identity Service...")
// Create shutdown context with timeout
shutdownCtx, shutdownCancel := context.WithTimeout(context.Background(), 30*time.Second)
defer shutdownCancel()
// Stop the application
if err := container.Stop(shutdownCtx); err != nil {
log := logger.GetGlobalLogger()
if log != nil {
log.Error("Error during Identity Service shutdown",
logger.Error(err),
)
} else {
fmt.Fprintf(os.Stderr, "Error during shutdown: %v\n", err)
}
os.Exit(1)
}
fmt.Println("Identity Service stopped successfully")
}
// registerLifecycle registers lifecycle hooks for the service.
func registerLifecycle(
lc fx.Lifecycle,
grpcServer *grpcServerWrapper,
serviceRegistry registry.ServiceRegistry,
cfg config.ConfigProvider,
log logger.Logger,
) {
lc.Append(fx.Hook{
OnStart: func(ctx context.Context) error {
// Start gRPC server
if err := grpcServer.Start(); err != nil {
return fmt.Errorf("failed to start gRPC server: %w", err)
}
// Register with service registry
serviceID := fmt.Sprintf("identity-service-%d", time.Now().Unix())
// In Docker, always use the Docker service name for health checks
// Consul (also in Docker) needs to reach the service via Docker DNS
host := cfg.GetString("services.identity.host")
if os.Getenv("ENVIRONMENT") == "production" || os.Getenv("DOCKER") == "true" {
host = "identity-service" // Docker service name - required for Consul health checks
} else if host == "" {
host = "localhost" // Local development
}
port := grpcServer.Port()
instance := &registry.ServiceInstance{
ID: serviceID,
Name: "identity-service",
Address: host,
Port: port,
Tags: []string{"grpc", "identity"},
Metadata: map[string]string{
"version": "1.0.0",
"protocol": "grpc",
},
}
if err := serviceRegistry.Register(ctx, instance); err != nil {
log.Warn("Failed to register with service registry",
zap.Error(err),
)
} else {
log.Info("Registered Identity Service with service registry",
zap.String("service_id", serviceID),
zap.String("name", instance.Name),
zap.Int("port", port),
)
}
return nil
},
OnStop: func(ctx context.Context) error {
// Stop gRPC server
if err := grpcServer.Stop(ctx); err != nil {
return fmt.Errorf("failed to stop gRPC server: %w", err)
}
return nil
},
})
}

50
cmd/platform/main.go Normal file
View File

@@ -0,0 +1,50 @@
// Package main provides the application entry point for the Go Platform.
package main
import (
"context"
"fmt"
"os"
"git.dcentral.systems/toolz/goplt/internal/di"
"git.dcentral.systems/toolz/goplt/internal/health"
"git.dcentral.systems/toolz/goplt/internal/metrics"
"git.dcentral.systems/toolz/goplt/pkg/config"
"git.dcentral.systems/toolz/goplt/pkg/logger"
"go.uber.org/fx"
)
func main() {
// Create DI container with lifecycle hooks
// This is a minimal entry point for testing core kernel infrastructure
// Services will have their own entry points (cmd/{service}/main.go)
container := di.NewContainer(
// Invoke lifecycle hooks
fx.Invoke(di.RegisterLifecycleHooks),
// Verify core kernel services are available
fx.Invoke(func(
_ config.ConfigProvider,
_ logger.Logger,
_ *health.Registry,
_ *metrics.Metrics,
) {
// Core kernel services are available
}),
)
// Create root context
ctx := context.Background()
// Start the application
if err := container.Start(ctx); err != nil {
log := logger.GetGlobalLogger()
if log != nil {
log.Error("Failed to start application",
logger.Error(err),
)
} else {
fmt.Fprintf(os.Stderr, "Failed to start application: %v\n", err)
}
os.Exit(1)
}
}

72
config/default.yaml Normal file
View File

@@ -0,0 +1,72 @@
environment: development
server:
port: 8080
host: "0.0.0.0"
read_timeout: 30s
write_timeout: 30s
database:
driver: "postgres"
dsn: "postgres://goplt:goplt_password@localhost:5432/goplt?sslmode=disable"
max_connections: 25
max_idle_connections: 5
conn_max_lifetime: 5m
conn_max_idle_time: 10m
logging:
level: "info"
format: "json"
output: "stdout"
tracing:
enabled: true
service_name: "platform"
service_version: "1.0.0"
otlp_endpoint: ""
registry:
type: consul
consul:
address: "localhost:8500"
datacenter: "dc1"
scheme: "http"
health_check:
interval: "10s"
timeout: "3s"
deregister_after: "30s"
http: "/healthz"
grpc: "grpc.health.v1.Health"
use_grpc: true
services:
audit:
port: 8084
host: "localhost"
auth:
port: 8081
host: "localhost"
identity:
port: 8082
host: "localhost"
authz:
port: 8083
host: "localhost"
auth:
jwt_secret: "change-this-secret-in-production"
gateway:
port: 8080
host: "0.0.0.0"
routes:
- path: "/api/v1/auth/**"
service: "auth-service"
auth_required: false
- path: "/api/v1/users/**"
service: "identity-service"
auth_required: true
cors:
allowed_origins: ["*"]
allowed_methods: ["GET", "POST", "PUT", "DELETE", "PATCH"]
allowed_headers: ["Authorization", "Content-Type"]

5
config/development.yaml Normal file
View File

@@ -0,0 +1,5 @@
environment: development
logging:
level: "debug"
format: "console"

5
config/production.yaml Normal file
View File

@@ -0,0 +1,5 @@
environment: production
logging:
level: "warn"
format: "json"

49
docker-compose.dev.yml Normal file
View File

@@ -0,0 +1,49 @@
# Development docker-compose: Only infrastructure services (PostgreSQL and Consul)
# Use this for local development when running services directly with `go run`
services:
postgres:
image: postgres:16-alpine
container_name: goplt-postgres
environment:
POSTGRES_USER: goplt
POSTGRES_PASSWORD: goplt_password
POSTGRES_DB: goplt
ports:
- "5432:5432"
volumes:
- postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U goplt"]
interval: 5s
timeout: 5s
retries: 5
networks:
- goplt-network
consul:
image: consul:latest
container_name: goplt-consul
command: consul agent -dev -client=0.0.0.0
ports:
- "8500:8500"
volumes:
- consul_data:/consul/data
healthcheck:
test: ["CMD-SHELL", "consul members"]
interval: 10s
timeout: 3s
retries: 5
networks:
- goplt-network
volumes:
postgres_data:
driver: local
consul_data:
driver: local
networks:
goplt-network:
driver: bridge

160
docker-compose.yml Normal file
View File

@@ -0,0 +1,160 @@
# Full docker-compose: All services + infrastructure
# Use this to run the complete platform with all services in Docker
services:
postgres:
image: postgres:16-alpine
container_name: goplt-postgres
environment:
POSTGRES_USER: goplt
POSTGRES_PASSWORD: goplt_password
POSTGRES_DB: goplt
ports:
- "5432:5432"
volumes:
- postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U goplt"]
interval: 5s
timeout: 5s
retries: 5
networks:
- goplt-network
consul:
image: consul:1.15.4
container_name: goplt-consul
command: consul agent -dev -client=0.0.0.0
ports:
- "8500:8500"
volumes:
- consul_data:/consul/data
healthcheck:
test: ["CMD-SHELL", "consul members"]
interval: 10s
timeout: 3s
retries: 5
networks:
- goplt-network
auth-service:
build:
context: .
dockerfile: cmd/auth-service/Dockerfile
container_name: goplt-auth-service
environment:
ENVIRONMENT: production
DATABASE_DSN: "postgres://goplt:goplt_password@postgres:5432/goplt?sslmode=disable"
REGISTRY_TYPE: consul
REGISTRY_CONSUL_ADDRESS: "consul:8500"
ports:
- "8081:8081"
depends_on:
postgres:
condition: service_healthy
consul:
condition: service_healthy
networks:
- goplt-network
restart: unless-stopped
identity-service:
build:
context: .
dockerfile: cmd/identity-service/Dockerfile
container_name: goplt-identity-service
environment:
ENVIRONMENT: production
DATABASE_DSN: "postgres://goplt:goplt_password@postgres:5432/goplt?sslmode=disable"
REGISTRY_TYPE: consul
REGISTRY_CONSUL_ADDRESS: "consul:8500"
ports:
- "8082:8082"
depends_on:
postgres:
condition: service_healthy
consul:
condition: service_healthy
networks:
- goplt-network
restart: unless-stopped
authz-service:
build:
context: .
dockerfile: cmd/authz-service/Dockerfile
container_name: goplt-authz-service
environment:
ENVIRONMENT: production
DATABASE_DSN: "postgres://goplt:goplt_password@postgres:5432/goplt?sslmode=disable"
REGISTRY_TYPE: consul
REGISTRY_CONSUL_ADDRESS: "consul:8500"
ports:
- "8083:8083"
depends_on:
postgres:
condition: service_healthy
consul:
condition: service_healthy
networks:
- goplt-network
restart: unless-stopped
audit-service:
build:
context: .
dockerfile: cmd/audit-service/Dockerfile
container_name: goplt-audit-service
environment:
ENVIRONMENT: production
DATABASE_DSN: "postgres://goplt:goplt_password@postgres:5432/goplt?sslmode=disable"
REGISTRY_TYPE: consul
REGISTRY_CONSUL_ADDRESS: "consul:8500"
ports:
- "8084:8084"
depends_on:
postgres:
condition: service_healthy
consul:
condition: service_healthy
networks:
- goplt-network
restart: unless-stopped
api-gateway:
build:
context: .
dockerfile: cmd/api-gateway/Dockerfile
container_name: goplt-api-gateway
environment:
ENVIRONMENT: production
REGISTRY_TYPE: consul
REGISTRY_CONSUL_ADDRESS: "consul:8500"
GATEWAY_PORT: "8080"
GATEWAY_HOST: "0.0.0.0"
ports:
- "8080:8080"
depends_on:
consul:
condition: service_healthy
auth-service:
condition: service_started
identity-service:
condition: service_started
authz-service:
condition: service_started
audit-service:
condition: service_started
networks:
- goplt-network
restart: unless-stopped
volumes:
postgres_data:
driver: local
consul_data:
driver: local
networks:
goplt-network:
driver: bridge

14
docs/.dockerignore Normal file
View File

@@ -0,0 +1,14 @@
# Docker ignore file for MkDocs build
site/
.mkdocs_cache/
__pycache__/
*.pyc
*.pyo
*.pyd
.Python
venv/
env/
ENV/
.git/
.gitignore

26
docs/Dockerfile Normal file
View File

@@ -0,0 +1,26 @@
# Dockerfile for MkDocs documentation
FROM python:3.11-slim
WORKDIR /docs
# Install system dependencies
RUN apt-get update && \
apt-get install -y --no-install-recommends \
git \
&& rm -rf /var/lib/apt/lists/*
# Copy requirements first for better caching
COPY requirements.txt .
# Install Python dependencies
RUN pip install --no-cache-dir -r requirements.txt
# Copy all documentation files
COPY . .
# Expose MkDocs default port
EXPOSE 8000
# Default command: serve documentation
CMD ["mkdocs", "serve", "--dev-addr=0.0.0.0:8000"]

163
docs/MKDOCS_README.md Normal file
View File

@@ -0,0 +1,163 @@
# MkDocs Setup
This project uses [MkDocs](https://www.mkdocs.org/) with the [Material theme](https://squidfunk.github.io/mkdocs-material/) for documentation.
All documentation tooling files are self-contained in the `docs/` directory.
## Prerequisites
You can run MkDocs in two ways:
**Option 1: Local Python Installation**
- Python 3.8 or higher
- pip (Python package manager)
**Option 2: Docker (Recommended - No Python installation needed)**
- Docker and Docker Compose
## Installation
1. Install the required Python packages:
```bash
cd docs
pip install -r requirements.txt
```
Or if you prefer using a virtual environment:
```bash
cd docs
python3 -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
pip install -r requirements.txt
```
## Usage
You can use either the Makefile commands from the project root (recommended) or run MkDocs directly from the `docs/` directory.
### Using Makefile (Recommended)
From the project root, use the Makefile commands:
```bash
# Using Docker (easiest, no Python needed)
make docs-docker-compose-up
# or
make docs-docker
# Using local Python
make docs-install # Install dependencies
make docs-serve # Serve documentation
make docs-build # Build static site
make docs-deploy # Deploy to GitHub Pages
# Show all available commands
make help
```
The documentation will be available at `http://127.0.0.1:8000` with live reload enabled.
### Using Docker Directly
From the `docs/` directory:
```bash
cd docs
# Using docker-compose (easiest)
docker-compose up --build
# Using Docker directly
docker build -t goplt-docs:latest .
docker run --rm -it -p 8000:8000 -v "$(pwd):/docs:ro" goplt-docs:latest
# Build static site
docker run --rm -v "$(pwd):/docs:ro" -v "$(pwd)/site:/docs/site" goplt-docs:latest mkdocs build
```
### Using MkDocs Directly
From the `docs/` directory:
```bash
cd docs
# Serve documentation locally with auto-reload
mkdocs serve
# Build static site
mkdocs build
# Deploy to GitHub Pages
mkdocs gh-deploy
```
## Configuration
The main configuration file is `docs/mkdocs.yml`. Key settings:
- **Site name and metadata**: Configured at the top
- **Theme**: Material theme with light/dark mode support
- **Navigation**: Defined in the `nav` section
- **Plugins**: Search and git revision date plugins enabled
- **Markdown extensions**: Various extensions for code highlighting, tables, etc.
- **docs_dir**: Set to "content" - markdown files are in the content/ subdirectory
## Project Structure
```
goplt/
├── docs/ # Documentation directory (self-contained)
│ ├── mkdocs.yml # MkDocs configuration
│ ├── requirements.txt # Python dependencies
│ ├── Dockerfile # Docker image for MkDocs
│ ├── docker-compose.yml # Docker Compose configuration
│ ├── .dockerignore # Docker ignore patterns
│ ├── MKDOCS_README.md # This file
│ ├── content/ # Documentation content (markdown files)
│ │ ├── index.md # Home page
│ │ ├── requirements.md # Requirements document
│ │ ├── plan.md # Implementation plan
│ │ ├── playbook.md # Implementation playbook
│ │ ├── adr/ # Architecture Decision Records
│ │ └── stories/ # Implementation tasks
│ └── site/ # Generated site (gitignored)
└── Makefile # Makefile with docs commands (runs from root)
```
## Adding New Documentation
1. Add markdown files to the `docs/content/` directory
2. Update `docs/mkdocs.yml` to add entries to the `nav` section
3. Run `make docs-serve` or `make docs-docker` from project root to preview changes
## Customization
### Theme Colors
Edit the `theme.palette` section in `docs/mkdocs.yml` to change colors.
### Navigation
Update the `nav` section in `docs/mkdocs.yml` to reorganize or add new pages.
### Plugins
Add or remove plugins in the `plugins` section of `docs/mkdocs.yml`.
## Troubleshooting
- **Import errors**: Make sure all dependencies are installed: `cd docs && pip install -r requirements.txt`
- **Navigation issues**: Check that file paths in `docs/mkdocs.yml` match actual file locations
- **Build errors**: Run `mkdocs build --verbose` from the `docs/` directory for detailed error messages
- **Docker issues**: Make sure Docker is running and you have permissions to run containers
## Benefits of Self-Contained Docs
- All documentation tooling is in one place
- Easy to share or move the docs directory
- Doesn't pollute the project root
- Can be versioned or deployed independently

View File

@@ -1,54 +0,0 @@
# ADR-0013: Database ORM Selection
## Status
Accepted
## Context
The platform needs a database ORM/library that:
- Supports PostgreSQL (primary database)
- Provides type-safe query building
- Supports code generation (reduces boilerplate)
- Handles migrations
- Supports relationships (many-to-many, etc.)
- Integrates with Ent (code generation)
Options considered:
1. **entgo.io/ent** - Code-generated, type-safe ORM
2. **gorm.io/gorm** - Feature-rich ORM with reflection
3. **sqlx** - Lightweight wrapper around database/sql
4. **Standard library database/sql** - No ORM, raw SQL
## Decision
Use **entgo.io/ent** as the primary ORM for the platform.
**Rationale:**
- Code generation provides compile-time type safety
- Excellent schema definition and migration support
- Strong relationship modeling
- Good performance (no reflection at runtime)
- Active development and good documentation
- Recommended in playbook-golang.md
- Easy to integrate with OpenTelemetry
## Consequences
### Positive
- Type-safe queries eliminate runtime errors
- Schema changes are explicit and versioned
- Code generation reduces boilerplate
- Good migration support
- Strong relationship support
### Negative
- Requires code generation step (`go generate`)
- Learning curve for developers unfamiliar with Ent
- Less flexible than raw SQL for complex queries
- Generated code must be committed or verified in CI
### Implementation Notes
- Install: `go get entgo.io/ent/cmd/ent`
- Initialize schema: `go run entgo.io/ent/cmd/ent init User Role Permission`
- Use `//go:generate` directives for code generation
- Run migrations on startup via `client.Schema.Create()`
- Create wrapper in `internal/infra/database/client.go` for DI injection

View File

@@ -1,10 +1,11 @@
# ADR-0002: Go Version # ADR-0002: Go Version
## Status ## Status
Accepted Superseded by [ADR-0034: Go Version Upgrade to 1.25.3](./0034-go-version-upgrade.md)
## Context ## Context
Go releases new versions regularly with new features, performance improvements, and security fixes. We need to choose a Go version that: Go releases new versions regularly with new features, performance improvements, and security fixes. We need to choose a Go version that:
- Provides necessary features for the platform - Provides necessary features for the platform
- Has good ecosystem support - Has good ecosystem support
- Is stable and production-ready - Is stable and production-ready

View File

@@ -27,7 +27,7 @@ Use **gin-gonic/gin** (v1.9.1+) as the HTTP framework.
- Easy route grouping (useful for modules) - Easy route grouping (useful for modules)
- Good OpenTelemetry integration support - Good OpenTelemetry integration support
- Widely used and well-documented - Widely used and well-documented
- Recommended in playbook-golang.md - Recommended in playbook.md
## Consequences ## Consequences

View File

@@ -29,11 +29,11 @@ goplt/
│ ├── config/ # ConfigProvider interface │ ├── config/ # ConfigProvider interface
│ ├── logger/ # Logger interface │ ├── logger/ # Logger interface
│ ├── module/ # IModule interface │ ├── module/ # IModule interface
│ ├── auth/ # Auth interfaces (Phase 2) │ ├── auth/ # Auth interfaces (Epic 2)
│ ├── perm/ # Permission DSL (Phase 2) │ ├── perm/ # Permission DSL (Epic 2)
│ └── infra/ # Infrastructure interfaces │ └── infra/ # Infrastructure interfaces
├── modules/ # Feature modules ├── modules/ # Feature modules
│ └── blog/ # Sample module (Phase 4) │ └── blog/ # Sample module (Epic 4)
├── config/ # Configuration files ├── config/ # Configuration files
│ ├── default.yaml │ ├── default.yaml
│ ├── development.yaml │ ├── development.yaml
@@ -75,7 +75,7 @@ goplt/
### Implementation Notes ### Implementation Notes
- Initialize with `go mod init git.dcentral.systems/toolz/goplt` - Initialize with `go mod init git.dcentral.systems/toolz/goplt`
- Create all directories upfront in Phase 0 - Create all directories upfront in Epic 0
- Document structure in `README.md` - Document structure in `README.md`
- Enforce boundaries via `internal/` package visibility - Enforce boundaries via `internal/` package visibility
- Use `go build ./...` to verify structure - Use `go build ./...` to verify structure

View File

@@ -26,7 +26,7 @@ Use **GitHub Actions** for CI/CD pipeline.
- Rich ecosystem of actions - Rich ecosystem of actions
- Easy to configure with YAML - Easy to configure with YAML
- Good documentation and community support - Good documentation and community support
- Recommended in playbook-golang.md - Recommended in playbook.md
## Consequences ## Consequences

View File

@@ -0,0 +1,64 @@
# ADR-0013: Database ORM Selection
## Status
Accepted
## Context
The platform follows a microservices architecture where each service has its own database connection. The ORM/library must:
- Support PostgreSQL (primary database)
- Provide type-safe query building
- Support code generation (reduces boilerplate)
- Handle migrations per service
- Support relationships (many-to-many, etc.)
- Integrate with Ent (code generation)
- Support schema isolation (each service owns its schema)
Options considered:
1. **entgo.io/ent** - Code-generated, type-safe ORM
2. **gorm.io/gorm** - Feature-rich ORM with reflection
3. **sqlx** - Lightweight wrapper around database/sql
4. **Standard library database/sql** - No ORM, raw SQL
## Decision
Use **entgo.io/ent** as the primary ORM for the platform.
**Rationale:**
- Code generation provides compile-time type safety
- Excellent schema definition and migration support
- Strong relationship modeling
- Good performance (no reflection at runtime)
- Active development and good documentation
- Recommended in playbook.md
- Easy to integrate with OpenTelemetry
## Consequences
### Positive
- Type-safe queries eliminate runtime errors
- Schema changes are explicit and versioned
- Code generation reduces boilerplate
- Good migration support
- Strong relationship support
### Negative
- Requires code generation step (`go generate`)
- Learning curve for developers unfamiliar with Ent
- Less flexible than raw SQL for complex queries
- Generated code must be committed or verified in CI
### Database Access Pattern
- **Each service has its own database connection pool**: Services do not share database connections
- **Schema isolation**: Each service owns its database schema (e.g., `auth_schema`, `identity_schema`, `blog_schema`)
- **No cross-service database access**: Services communicate via APIs, not direct database queries
- **Shared database instance**: Services share the same PostgreSQL instance but use different schemas
- **Alternative**: Database-per-service pattern (each service has its own database) for maximum isolation
### Implementation Notes
- Install: `go get entgo.io/ent/cmd/ent`
- Each service initializes its own schema: `go run entgo.io/ent/cmd/ent init User Role Permission` (Identity Service)
- Use `//go:generate` directives for code generation per service
- Run migrations on startup via `client.Schema.Create()` for each service
- Create database client wrapper per service in `services/{service}/internal/database/client.go`
- Each service manages its own connection pool configuration

View File

@@ -21,7 +21,7 @@ Implement a **channel-based error bus** with pluggable sinks:
1. **Error bus interface**: `type ErrorPublisher interface { Publish(err error) }` 1. **Error bus interface**: `type ErrorPublisher interface { Publish(err error) }`
2. **Channel-based implementation**: Background goroutine consumes errors from channel 2. **Channel-based implementation**: Background goroutine consumes errors from channel
3. **Pluggable sinks**: Logger (always), Sentry (optional, Phase 6) 3. **Pluggable sinks**: Logger (always), Sentry (optional, Epic 6)
4. **Panic recovery middleware**: Automatically publishes panics to error bus 4. **Panic recovery middleware**: Automatically publishes panics to error bus
**Rationale:** **Rationale:**

View File

@@ -29,7 +29,7 @@ Use **OpenTelemetry (OTEL)** for all observability:
- Excellent Go SDK support - Excellent Go SDK support
- Integrates with major observability tools - Integrates with major observability tools
- Supports metrics, traces, and logs - Supports metrics, traces, and logs
- Recommended in playbook-golang.md - Recommended in playbook.md
- Future-proof (not locked to specific vendor) - Future-proof (not locked to specific vendor)
## Consequences ## Consequences

View File

@@ -0,0 +1,113 @@
# ADR-0029: Microservices Architecture
## Status
Accepted
## Context
The platform needs to scale independently, support team autonomy, and enable flexible deployment. A microservices architecture provides these benefits from day one, and the complexity of supporting both monolith and microservices modes is unnecessary.
## Decision
Design the platform as **microservices architecture from day one**:
1. **Core Services**: Core business services are separate microservices:
- **Auth Service** (`cmd/auth-service/`): JWT token generation/validation
- **Identity Service** (`cmd/identity-service/`): User CRUD, password management
- **Authz Service** (`cmd/authz-service/`): Permission resolution, authorization
- **Audit Service** (`cmd/audit-service/`): Audit logging
- Each service has its own process, database connection, and deployment
2. **API Gateway**: Core infrastructure component (implemented in Epic 1):
- Single entry point for all external traffic
- Routes requests to backend services via service discovery
- Handles authentication, rate limiting, CORS at the edge
- Not optional - required for microservices architecture
3. **Service-Based Architecture**: All modules are independent services:
- Each module/service is a separate service with its own process
- Services communicate via gRPC (primary) or HTTP (fallback)
- Service client interfaces for all inter-service communication
- No direct in-process calls between services
4. **Service Registry**: Central registry for service discovery:
- All services register on startup
- Service discovery via registry
- Health checking and automatic deregistration
- Support for Consul, etcd, or Kubernetes service discovery
5. **Communication Patterns**:
- **Synchronous**: gRPC service calls (primary), HTTP/REST (fallback)
- **Asynchronous**: Event bus via Kafka
- **Shared Infrastructure**: Cache (Redis) and Database (PostgreSQL instance)
- **Database Access**: Each service has its own connection pool and schema
6. **Service Boundaries**: Each service is independent:
- Independent Go modules (`go.mod`)
- Own database schema (via Ent) - schema isolation
- Own API routes (gRPC/HTTP)
- Own process and deployment
- Can be scaled independently
7. **Development Mode**: For local development, services run in the same repository:
- Each service has its own entry point and process
- Services still communicate via service clients (gRPC/HTTP)
- No direct in-process calls
- Docker Compose for easy local setup
## Consequences
### Positive
- **Simplified Architecture**: Single architecture pattern, no dual-mode complexity
- **Independent Scaling**: Scale individual services based on load
- **Team Autonomy**: Teams can own and deploy their services independently
- **Technology Diversity**: Different services can use different tech stacks (future)
- **Fault Isolation**: Failure in one service doesn't bring down entire platform
- **Deployment Flexibility**: Deploy services independently
- **Clear Boundaries**: Service boundaries are explicit from the start
### Negative
- **Network Latency**: Inter-service calls have network overhead
- **Distributed System Challenges**: Need to handle network failures, retries, timeouts
- **Service Discovery Overhead**: Additional infrastructure needed
- **Debugging Complexity**: Distributed tracing becomes essential
- **Data Consistency**: Cross-service transactions become challenging
- **Development Setup**: More complex local development (multiple services)
### Mitigations
- **API Gateway**: Implemented in Epic 1 as core infrastructure - handles routing, authentication, rate limiting
- **Service Mesh**: Use service mesh (Istio, Linkerd) for advanced microservices features (optional)
- **Event Sourcing**: Use events for eventual consistency
- **Circuit Breakers**: Implement circuit breakers for resilience
- **Comprehensive Observability**: OpenTelemetry, metrics, logging essential
- **Docker Compose**: Simplify local development with docker-compose
- **Service Clients**: All inter-service communication via service clients (gRPC/HTTP)
## Implementation Strategy
### Epic 1: Core Kernel & Infrastructure
- Core kernel (infrastructure only): config, logger, DI, health, metrics, observability
- API Gateway implementation (core infrastructure component)
- Service client interfaces for all core services
- Service registry interface and basic implementation
### Epic 2: Core Services Separation
- Separate Auth, Identity, Authz, Audit into independent services
- Each service: own entry point (`cmd/{service}/`), gRPC server, database connection
- Service client implementations (gRPC/HTTP)
- Service registration with registry
### Epic 3: Service Registry & Discovery (Epic 3)
- Complete service registry implementation
- Service discovery (Consul, Kubernetes)
- Service health checking and deregistration
### Epic 5: gRPC Services (Epic 5)
- Complete gRPC service definitions for all services
- gRPC clients for service communication
- HTTP clients as fallback option
## References
- [Service Abstraction Pattern](https://microservices.io/patterns/data/service-per-database.html)
- [Service Discovery Patterns](https://microservices.io/patterns/service-registry.html)
- [gRPC Documentation](https://grpc.io/docs/)

View File

@@ -0,0 +1,95 @@
# ADR-0030: Service Communication Strategy
## Status
Accepted
## Context
Services need to communicate with each other in a microservices architecture. All communication must go through well-defined interfaces that support network calls.
## Decision
Use a **service client-based communication strategy** with API Gateway as the entry point:
1. **API Gateway** (Entry Point):
- All external traffic enters through API Gateway
- Gateway routes requests to backend services via service discovery
- Gateway handles authentication (JWT validation via Auth Service)
- Gateway handles rate limiting, CORS, request transformation
2. **Service Client Interfaces** (Primary for synchronous calls):
- Define interfaces in `pkg/services/` for all services
- All implementations are network-based:
- `internal/services/grpc/client/` - gRPC clients (primary)
- `internal/services/http/client/` - HTTP clients (fallback)
- Gateway uses service clients to communicate with backend services
- Services use service clients for inter-service communication
3. **Event Bus** (Primary for asynchronous communication):
- Distributed via Kafka
- Preferred for cross-service communication
- Event-driven architecture for loose coupling
4. **Shared Infrastructure** (For state):
- Redis for cache and distributed state
- PostgreSQL instance for persistent data (each service has its own schema)
- Kafka for events
## Service Client Pattern
```go
// Interface in pkg/services/
type IdentityServiceClient interface {
GetUser(ctx context.Context, id string) (*User, error)
CreateUser(ctx context.Context, user *User) (*User, error)
}
// gRPC implementation (primary)
type grpcIdentityClient struct {
conn *grpc.ClientConn
client pb.IdentityServiceClient
}
// HTTP implementation (fallback)
type httpIdentityClient struct {
baseURL string
httpClient *http.Client
}
```
## Communication Flow
```
Client → API Gateway → Backend Service (via service client)
Backend Service → Other Service (via service client)
```
All communication goes through service clients - no direct in-process calls even in development mode.
## Development Mode
For local development, services run in the same repository but as separate processes:
- Each service has its own entry point (`cmd/{service}/`)
- Services communicate via service clients (gRPC or HTTP) - no direct in-process calls
- Docker Compose orchestrates all services
- This ensures the architecture is consistent with production
## Consequences
### Positive
- **Unified Interface**: Consistent interface across all services
- **Easy Testing**: Can mock service clients
- **Type Safety**: gRPC provides type-safe contracts
- **Clear Boundaries**: Service boundaries are explicit
- **Scalability**: Services can be scaled independently
### Negative
- **Network Overhead**: All calls go over network
- **Interface Evolution**: Changes require coordination
- **Versioning**: Need service versioning strategy
- **Development Complexity**: More setup required for local development
## Implementation
- All services use gRPC clients (primary)
- HTTP clients as fallback option
- Service registry for service discovery
- Circuit breakers and retries for resilience

View File

@@ -0,0 +1,244 @@
# ADR-0031: Service Repository Structure
## Status
Accepted
## Context
The platform follows a microservices architecture where each service (Auth, Identity, Authz, Audit, and feature modules) is an independent service. We need to decide how to organize these services in the codebase:
- **Option 1**: Monorepo with service directories (all services in one repository)
- **Option 2**: Separate repositories (each service in its own repository)
The decision affects:
- Code sharing and dependencies
- Development workflow
- CI/CD complexity
- Team autonomy and ownership
- Deployment strategies
- Code discoverability and navigation
## Decision
Use a **monorepo structure with service directories** for all services:
1. **Repository Structure**: Single repository (`goplt/`) containing:
- Core Kernel (shared infrastructure)
- All core services (Auth, Identity, Authz, Audit)
- All feature modules (Blog, Billing, etc.)
- Shared interfaces and utilities
2. **Service Organization**: Each service lives in `cmd/` directory:
```
goplt/
├── cmd/
│ ├── platform/ # Core kernel entry point (minimal, infrastructure only)
│ ├── api-gateway/ # API Gateway service entry point
│ ├── auth-service/ # Auth Service entry point
│ ├── identity-service/ # Identity Service entry point
│ ├── authz-service/ # Authz Service entry point
│ ├── audit-service/ # Audit Service entry point
│ └── blog-service/ # Blog feature service entry point
├── services/ # Service implementations (optional alternative)
│ ├── auth/
│ │ ├── internal/ # Service implementation
│ │ └── api/ # gRPC/HTTP definitions
│ ├── identity/
│ ├── authz/
│ └── audit/
├── internal/ # Core kernel implementation (shared)
│ ├── config/
│ ├── logger/
│ └── di/
├── pkg/ # Public interfaces
│ ├── config/ # ConfigProvider interface
│ ├── logger/ # Logger interface
│ ├── services/ # Service client interfaces
│ └── module/ # IModule interface
└── modules/ # Feature modules (optional structure)
└── blog/
```
3. **Service Independence**: Each service maintains:
- Own `go.mod` module (or shared workspace)
- Own internal implementation (`internal/` or `services/{service}/internal/`)
- Own gRPC/HTTP API definitions
- Own database schema and migrations
- Own entry point in `cmd/{service-name}/`
4. **Shared Code**: Core kernel infrastructure shared across all services:
- Configuration management (`internal/config/`, `pkg/config/`)
- Logging system (`internal/logger/`, `pkg/logger/`)
- Dependency injection (`internal/di/`)
- Service client implementations (`internal/services/`)
- Common utilities and interfaces
5. **Development Mode**: All services can run from same repository:
- Single `go.work` workspace file (Go 1.18+)
- Shared dependencies and versioning
- Easy local development with `docker-compose`
- Atomic changes across services when needed
## Rationale
### Why Monorepo?
1. **Code Sharing**: Core kernel (config, logger, DI) is shared across all services. Monorepo makes this easy.
2. **Atomic Changes**: Changes to shared interfaces can be updated across all services in a single commit.
3. **Simplified CI/CD**: Single CI pipeline can test all services and their interactions.
4. **Easier Development**: Developers can run all services locally from one repository.
5. **Version Consistency**: Shared dependencies (like core kernel) are always in sync.
6. **Cross-Service Refactoring**: Easier to refactor shared code across services.
7. **Unified Documentation**: All documentation in one place.
### Why Not Separate Repositories?
1. **Code Duplication**: Shared core kernel would need to be published as a library or duplicated.
2. **Version Management**: Core kernel changes would require coordinated releases across repositories.
3. **Development Complexity**: Developers would need to clone multiple repositories.
4. **CI/CD Complexity**: Multiple pipelines to manage and coordinate.
5. **Breaking Changes**: Harder to coordinate breaking changes to shared interfaces.
## Consequences
### Positive
- **Simplified Development**: Single repository to clone and work with
- **Code Reuse**: Easy sharing of core kernel and utilities
- **Atomic Changes**: Update shared interfaces and all services together
- **Consistent Dependencies**: All services use same versions of core kernel
- **Unified CI/CD**: Single pipeline for all services
- **Easier Onboarding**: New developers work with one repository
- **Cross-Service Testing**: Can test service interactions easily
- **Shared Documentation**: All docs in one place
- **Go Workspace Support**: Can use `go.work` for multi-module development
### Negative
- **Repository Size**: Repository grows as more services are added
- **Permission Granularity**: Harder to restrict access to specific services (Git permissions)
- **Build Time**: CI/CD may take longer as it builds all services
- **Deployment Coupling**: Services are in same repo (but can still deploy independently)
- **Merge Conflicts**: More likely with multiple teams working on same repo
- **Git History**: History can become cluttered with changes from many services
### Mitigations
1. **Repository Size**: Use Git LFS for large files, sparse checkout for large repos
2. **Permissions**: Use CODEOWNERS file and branch protection rules
3. **Build Time**: Use build caching and parallel builds in CI/CD
4. **Deployment**: Services are independently deployable despite shared repo
5. **Merge Conflicts**: Use feature branches, code review, and clear ownership
6. **Git History**: Use conventional commits and clear service prefixes
## Implementation Strategy
### Phase 1: Core Kernel (Epic 0-1) - Current
- Core kernel in `internal/` and `pkg/`
- Single entry point `cmd/platform/`
- Shared infrastructure established
### Phase 2: Service Structure (Epic 1-2)
- **Epic 1**: Create API Gateway service:
- `cmd/api-gateway/` - API Gateway entry point
- Service discovery integration
- Request routing to backend services
- **Epic 2**: Create core service directories:
- `cmd/auth-service/` - Auth Service entry point
- `cmd/identity-service/` - Identity Service entry point
- `cmd/authz-service/` - Authz Service entry point
- `cmd/audit-service/` - Audit Service entry point
- Create service implementations:
- Option A: `services/{service}/internal/` for each service (recommended)
- Option B: `internal/{service}/` for each service
- Each service has its own database connection pool
- Define service client interfaces in `pkg/services/`:
- `AuthServiceClient`, `IdentityServiceClient`, `AuthzServiceClient`, `AuditServiceClient`
- Implement gRPC/HTTP clients in `internal/services/`
### Phase 3: Module Services (Epic 4+)
- Feature modules follow same pattern:
- `cmd/blog-service/` for blog module
- Module implementation in `modules/blog/` or `services/blog/`
### Directory Structure Decision
**Recommended**: Use `cmd/{service}/` for entry points and `services/{service}/` for implementations:
```
goplt/
├── cmd/
│ ├── platform/ # Core kernel (minimal, infrastructure only)
│ ├── api-gateway/ # API Gateway entry point
│ ├── auth-service/ # Auth entry point
│ ├── identity-service/ # Identity entry point
│ ├── authz-service/ # Authz entry point
│ ├── audit-service/ # Audit entry point
│ └── blog-service/ # Blog feature service entry point
├── services/ # Service implementations
│ ├── gateway/
│ │ ├── internal/ # Gateway implementation
│ │ └── api/ # Routing logic
│ ├── auth/
│ │ ├── internal/ # Service implementation
│ │ └── api/ # gRPC/HTTP definitions
│ ├── identity/
│ ├── authz/
│ ├── audit/
│ └── blog/
├── internal/ # Core kernel (shared infrastructure)
├── pkg/ # Public interfaces
└── modules/ # Feature modules (optional structure)
```
This provides:
- Clear separation between entry points and implementations
- Easy to find service entry points
- Service code is organized together
- Consistent with Go best practices
## Alternative: Go Workspace (Go 1.18+)
For maximum service independence while maintaining monorepo benefits:
```go
// go.work
go 1.24
use (
. # Core kernel
./services/auth # Auth service module
./services/identity # Identity service module
./cmd/auth-service # Auth service entry
./cmd/identity-service # Identity service entry
)
```
Each service can have its own `go.mod` while sharing the workspace.
## Migration Path
If we later need to split into separate repositories:
1. Core kernel can be extracted to a shared library
2. Services can be moved to separate repos with git history
3. Service client interfaces remain in shared package
4. CI/CD can be adapted to multi-repo workflows
## References
- [ADR-0007: Project Directory Structure](./0007-project-directory-structure.md)
- [ADR-0029: Microservices Architecture](./0029-microservices-architecture.md)
- [ADR-0030: Service Communication Strategy](./0030-service-communication-strategy.md)
- [Go Workspaces](https://go.dev/doc/tutorial/workspaces)
- [Monorepo vs. Multi-repo](https://www.atlassian.com/git/tutorials/monorepos)
- [Google's Monorepo Strategy](https://cacm.acm.org/magazines/2016/7/204032-why-google-stores-billions-of-lines-of-code-in-a-single-repository/fulltext)

View File

@@ -0,0 +1,167 @@
# ADR-0032: API Gateway Strategy
## Status
Accepted
## Context
The platform follows a microservices architecture where each service is independently deployable. We need a central entry point that handles:
- Request routing to backend services
- Authentication and authorization at the edge
- Rate limiting and throttling
- CORS and request/response transformation
- Service discovery integration
Options considered:
1. **Custom API Gateway** - Build our own gateway service
2. **Kong** - Open-source API Gateway
3. **Envoy** - High-performance proxy
4. **Traefik** - Modern reverse proxy
## Decision
Implement a **custom API Gateway service** as a core infrastructure component in Epic 1:
1. **API Gateway as Core Component**:
- Entry point: `cmd/api-gateway/`
- Implementation: `services/gateway/internal/`
- Implemented in Epic 1 (not deferred to Epic 8)
- Required for microservices architecture, not optional
2. **Responsibilities**:
- **Request Routing**: Route requests to backend services via service discovery
- **Authentication**: Validate JWT tokens via Auth Service
- **Authorization**: Check permissions via Authz Service (for route-level auth)
- **Rate Limiting**: Per-user and per-IP rate limiting
- **CORS**: Handle cross-origin requests
- **Request Transformation**: Modify requests before forwarding
- **Response Transformation**: Modify responses before returning
- **Load Balancing**: Distribute requests across service instances
3. **Integration Points**:
- Service registry for service discovery
- Auth Service client for token validation
- Authz Service client for permission checks
- Cache (Redis) for rate limiting state
4. **Implementation Approach**:
- Built with Go (Gin/Echo framework)
- Uses service clients for backend communication
- Configurable routing rules
- Middleware-based architecture
## Architecture
```mermaid
graph TB
Client[Client] --> Gateway[API Gateway<br/>:8080]
Gateway --> AuthClient[Auth Service Client]
Gateway --> AuthzClient[Authz Service Client]
Gateway --> ServiceRegistry[Service Registry]
Gateway --> Cache[Cache<br/>Rate Limiting]
AuthClient --> AuthSvc[Auth Service<br/>:8081]
AuthzClient --> AuthzSvc[Authz Service<br/>:8083]
ServiceRegistry --> BackendSvc[Backend Services]
Gateway --> BackendSvc
style Gateway fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style AuthSvc fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
style BackendSvc fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
```
## Request Flow
```mermaid
sequenceDiagram
participant Client
participant Gateway
participant AuthSvc
participant AuthzSvc
participant Registry
participant BackendSvc
Client->>Gateway: HTTP Request
Gateway->>Gateway: Rate limiting check
Gateway->>AuthSvc: Validate JWT (gRPC)
AuthSvc-->>Gateway: Token valid + user info
Gateway->>AuthzSvc: Check route permission (gRPC, optional)
AuthzSvc-->>Gateway: Authorized
Gateway->>Registry: Discover backend service
Registry-->>Gateway: Service endpoint
Gateway->>BackendSvc: Forward request (gRPC/HTTP)
BackendSvc-->>Gateway: Response
Gateway-->>Client: HTTP Response
```
## Consequences
### Positive
- **Single Entry Point**: All external traffic goes through one gateway
- **Centralized Security**: Authentication and authorization at the edge
- **Performance**: Rate limiting and caching at gateway level
- **Flexibility**: Easy to add new routes and services
- **Consistency**: Uniform API interface for clients
- **Observability**: Central point for metrics and logging
### Negative
- **Single Point of Failure**: Gateway failure affects all traffic
- **Additional Latency**: Extra hop in request path
- **Complexity**: Additional service to maintain and deploy
- **Scaling**: Gateway must scale to handle all traffic
### Mitigations
1. **High Availability**: Deploy multiple gateway instances behind load balancer
2. **Circuit Breakers**: Implement circuit breakers for backend service failures
3. **Caching**: Cache authentication results and service endpoints
4. **Monitoring**: Comprehensive monitoring and alerting
5. **Graceful Degradation**: Fallback mechanisms for service failures
## Implementation Strategy
### Epic 1: Core Infrastructure
- Create `cmd/api-gateway/` entry point
- Implement basic routing with service discovery
- JWT validation via Auth Service client
- Rate limiting middleware
- CORS support
### Epic 2-3: Enhanced Features
- Permission-based routing (via Authz Service)
- Request/response transformation
- Advanced load balancing
- Health check integration
## Configuration
```yaml
gateway:
port: 8080
routes:
- path: /api/v1/auth/**
service: auth-service
auth_required: false
- path: /api/v1/users/**
service: identity-service
auth_required: true
permission: user.read
- path: /api/v1/blog/**
service: blog-service
auth_required: true
permission: blog.post.read
rate_limiting:
enabled: true
per_user: 100/minute
per_ip: 1000/minute
cors:
allowed_origins: ["*"]
allowed_methods: ["GET", "POST", "PUT", "DELETE"]
```
## References
- [ADR-0029: Microservices Architecture](./0029-microservices-architecture.md)
- [ADR-0030: Service Communication Strategy](./0030-service-communication-strategy.md)
- [ADR-0031: Service Repository Structure](./0031-service-repository-structure.md)
- [API Gateway Pattern](https://microservices.io/patterns/apigateway.html)

View File

@@ -0,0 +1,310 @@
# ADR-0033: Service Discovery Implementation
## Status
Accepted
## Context
The platform follows a microservices architecture where services need to discover and communicate with each other. We need a service discovery mechanism that:
- Enables services to find each other dynamically
- Supports health checking and automatic deregistration
- Works in both development (Docker Compose) and production (Kubernetes) environments
- Provides service registration and discovery APIs
- Supports multiple service instances (load balancing)
Options considered:
1. **Consul** - HashiCorp's service discovery and configuration tool
2. **etcd** - Distributed key-value store with service discovery
3. **Kubernetes Service Discovery** - Native K8s service discovery
4. **Eureka** - Netflix service discovery (Java-focused)
5. **Custom Registry** - Build our own service registry
## Decision
Use **Consul** as the primary service discovery implementation with support for Kubernetes service discovery as an alternative.
### Rationale
1. **Mature and Production-Ready**:
- Battle-tested in production environments
- Active development and strong community
- Comprehensive documentation
2. **Feature-Rich**:
- Service registration and health checking
- Key-value store for configuration
- Service mesh capabilities (Consul Connect)
- Multi-datacenter support
- DNS-based service discovery
3. **Development-Friendly**:
- Easy to run locally (single binary or Docker)
- Docker Compose integration
- Good for local development setup
4. **Production-Ready**:
- Works well in Kubernetes (Consul K8s)
- Can be used alongside Kubernetes service discovery
- Supports high availability and clustering
5. **Language Agnostic**:
- HTTP API for service registration
- gRPC support
- Go client library available
6. **Health Checking**:
- Built-in health checking with automatic deregistration
- Multiple health check types (HTTP, TCP, gRPC, script)
- Health status propagation
## Architecture
### Service Registry Interface
```go
// pkg/registry/registry.go
type ServiceRegistry interface {
// Register a service instance
Register(ctx context.Context, service *ServiceInstance) error
// Deregister a service instance
Deregister(ctx context.Context, serviceID string) error
// Discover service instances
Discover(ctx context.Context, serviceName string) ([]*ServiceInstance, error)
// Watch for service changes
Watch(ctx context.Context, serviceName string) (<-chan []*ServiceInstance, error)
// Get service health
Health(ctx context.Context, serviceID string) (*HealthStatus, error)
}
type ServiceInstance struct {
ID string
Name string
Address string
Port int
Tags []string
Metadata map[string]string
}
```
### Consul Implementation
```go
// internal/registry/consul/consul.go
type ConsulRegistry struct {
client *consul.Client
config *ConsulConfig
}
// Register service with Consul
func (r *ConsulRegistry) Register(ctx context.Context, service *ServiceInstance) error {
registration := &consul.AgentServiceRegistration{
ID: service.ID,
Name: service.Name,
Address: service.Address,
Port: service.Port,
Tags: service.Tags,
Meta: service.Metadata,
Check: &consul.AgentServiceCheck{
HTTP: fmt.Sprintf("http://%s:%d/healthz", service.Address, service.Port),
Interval: "10s",
Timeout: "3s",
DeregisterCriticalServiceAfter: "30s",
},
}
return r.client.Agent().ServiceRegister(registration)
}
```
## Implementation Strategy
### Phase 1: Consul Implementation (Epic 1)
- Create service registry interface in `pkg/registry/`
- Implement Consul registry in `internal/registry/consul/`
- Basic service registration and discovery
- Health check integration
### Phase 2: Kubernetes Support (Epic 6)
- Implement Kubernetes service discovery as alternative
- Service registry factory that selects implementation based on environment
- Support for both Consul and K8s in same codebase
### Phase 3: Advanced Features (Epic 6)
- Service mesh integration (Consul Connect)
- Multi-datacenter support
- Service tags and filtering
- Service metadata and configuration
## Configuration
```yaml
registry:
type: consul # or "kubernetes"
consul:
address: "localhost:8500"
datacenter: "dc1"
scheme: "http"
health_check:
interval: "10s"
timeout: "3s"
deregister_after: "30s"
kubernetes:
namespace: "default"
in_cluster: true
```
## Service Registration Flow
```mermaid
sequenceDiagram
participant Service
participant Registry[Service Registry Interface]
participant Consul
participant Health[Health Check]
Service->>Registry: Register(serviceInstance)
Registry->>Consul: Register service
Consul->>Consul: Store service info
Consul->>Health: Start health checks
loop Health Check
Health->>Service: GET /healthz
Service-->>Health: 200 OK
Health->>Consul: Update health status
end
Service->>Registry: Deregister(serviceID)
Registry->>Consul: Deregister service
Consul->>Consul: Remove service
```
## Service Discovery Flow
```mermaid
sequenceDiagram
participant Client
participant Registry[Service Registry]
participant Consul
participant Service1[Service Instance 1]
participant Service2[Service Instance 2]
Client->>Registry: Discover("auth-service")
Registry->>Consul: Query service instances
Consul-->>Registry: [instance1, instance2]
Registry->>Registry: Filter healthy instances
Registry-->>Client: [healthy instances]
Client->>Service1: gRPC call
Service1-->>Client: Response
```
## Development Setup
### Docker Compose
```yaml
services:
consul:
image: consul:latest
ports:
- "8500:8500"
command: consul agent -dev -client=0.0.0.0
volumes:
- consul-data:/consul/data
volumes:
consul-data:
```
### Local Development
```bash
# Run Consul in dev mode
consul agent -dev
# Or use Docker
docker run -d --name consul -p 8500:8500 consul:latest
```
## Production Deployment
### Kubernetes
```yaml
# Consul Helm Chart
helm repo add hashicorp https://helm.releases.hashicorp.com
helm install consul hashicorp/consul --set global.datacenter=dc1
```
### Standalone Cluster
- Deploy Consul cluster (3-5 nodes)
- Configure service discovery endpoints
- Set up Consul Connect for service mesh (optional)
## Consequences
### Positive
- **Dynamic Service Discovery**: Services can be added/removed without configuration changes
- **Health Checking**: Automatic removal of unhealthy services
- **Load Balancing**: Multiple service instances automatically discovered
- **Configuration Management**: Consul KV store for service configuration
- **Service Mesh Ready**: Can use Consul Connect for advanced features
- **Development Friendly**: Easy local setup with Docker
### Negative
- **Additional Infrastructure**: Requires Consul cluster in production
- **Network Dependency**: Services depend on Consul availability
- **Configuration Complexity**: Need to configure Consul cluster
- **Learning Curve**: Team needs to understand Consul concepts
### Mitigations
1. **High Availability**: Deploy Consul cluster (3+ nodes)
2. **Caching**: Cache service instances to reduce Consul queries
3. **Fallback**: Support Kubernetes service discovery as fallback
4. **Documentation**: Comprehensive setup and usage documentation
5. **Monitoring**: Monitor Consul health and service registration
## Alternative: Kubernetes Service Discovery
For Kubernetes deployments, we also support native Kubernetes service discovery:
```go
// internal/registry/kubernetes/k8s.go
type KubernetesRegistry struct {
clientset kubernetes.Interface
namespace string
}
func (r *KubernetesRegistry) Discover(ctx context.Context, serviceName string) ([]*ServiceInstance, error) {
endpoints, err := r.clientset.CoreV1().Endpoints(r.namespace).Get(ctx, serviceName, metav1.GetOptions{})
// Convert K8s endpoints to ServiceInstance
}
```
## Service Registry Factory
```go
// internal/registry/factory.go
func NewServiceRegistry(cfg *config.Config) (registry.ServiceRegistry, error) {
switch cfg.Registry.Type {
case "consul":
return consul.NewRegistry(cfg.Registry.Consul)
case "kubernetes":
return kubernetes.NewRegistry(cfg.Registry.Kubernetes)
default:
return nil, fmt.Errorf("unknown registry type: %s", cfg.Registry.Type)
}
}
```
## References
- [ADR-0029: Microservices Architecture](./0029-microservices-architecture.md)
- [ADR-0030: Service Communication Strategy](./0030-service-communication-strategy.md)
- [ADR-0031: Service Repository Structure](./0031-service-repository-structure.md)
- [Consul Documentation](https://www.consul.io/docs)
- [Consul Go Client](https://github.com/hashicorp/consul/api)
- [Consul Kubernetes](https://www.consul.io/docs/k8s)

View File

@@ -0,0 +1,57 @@
# ADR-0034: Go Version Upgrade to 1.25.3
## Status
Accepted
## Context
ADR-0002 established Go 1.24.3 as the minimum required version. Since then:
- Go 1.25.3 has been released with new features, performance improvements, and security fixes
- The project requires access to newer language features and tooling improvements
- Dependencies may benefit from Go 1.25.3 optimizations
- The development team has Go 1.25.3 available and working
## Decision
Upgrade from **Go 1.24.3** to **Go 1.25.3** as the minimum required version for the platform.
This decision supersedes [ADR-0002: Go Version](./0002-go-version.md).
**Rationale:**
- Access to latest Go features and performance improvements
- Better security with latest patches
- Improved tooling support and compiler optimizations
- Ensures compatibility with latest ecosystem dependencies
- Supports all planned features (modules, plugins, generics) with enhanced capabilities
## Consequences
### Positive
- Access to latest Go features and performance improvements
- Better security with latest patches
- Modern tooling support with improved compiler optimizations
- Better compatibility with latest dependency versions
- Enhanced performance characteristics
### Negative
- Requires developers to upgrade to Go 1.25.3+
- CI/CD must be updated to use Go 1.25.3
- May require dependency updates for compatibility
- Slightly higher barrier for developers still on older versions
### Mitigations
1. **Clear Documentation**: Update all documentation to specify Go 1.25.3 requirement
2. **CI/CD Updates**: Ensure CI/CD pipelines use Go 1.25.3
3. **Version Check**: Add version validation in build scripts if needed
4. **Developer Communication**: Notify team of version requirement
## Implementation Notes
- Update `go.mod`: `go 1.25.3`
- Update `.github/workflows/ci.yml` to use `actions/setup-go@v5` with version `1.25.3`
- Update ADR-0002 to mark as Superseded
- Update README.md and other documentation to reflect Go 1.25.3 requirement
- Run `go mod tidy` to ensure dependency compatibility
## References
- [ADR-0002: Go Version](./0002-go-version.md) - Superseded by this ADR
- [Go 1.25 Release Notes](https://go.dev/doc/go1.25)

View File

@@ -5,6 +5,7 @@ This directory contains Architecture Decision Records (ADRs) for the Go Platform
## What are ADRs? ## What are ADRs?
ADRs document important architectural decisions made during the project. They help: ADRs document important architectural decisions made during the project. They help:
- Track why decisions were made - Track why decisions were made
- Understand the context and constraints - Understand the context and constraints
- Review decisions when requirements change - Review decisions when requirements change
@@ -13,6 +14,7 @@ ADRs document important architectural decisions made during the project. They he
## ADR Format ## ADR Format
Each ADR follows this structure: Each ADR follows this structure:
- **Status**: Proposed | Accepted | Rejected | Superseded - **Status**: Proposed | Accepted | Rejected | Superseded
- **Context**: The situation that led to the decision - **Context**: The situation that led to the decision
- **Decision**: What was decided - **Decision**: What was decided
@@ -20,10 +22,11 @@ Each ADR follows this structure:
## ADR Index ## ADR Index
### Phase 0: Project Setup & Foundation ### Epic 0: Project Setup & Foundation
- [ADR-0001: Go Module Path](./0001-go-module-path.md) - Module path: `git.dcentral.systems/toolz/goplt` - [ADR-0001: Go Module Path](./0001-go-module-path.md) - Module path: `git.dcentral.systems/toolz/goplt`
- [ADR-0002: Go Version](./0002-go-version.md) - Go 1.24.3 - [ADR-0002: Go Version](./0002-go-version.md) - Go 1.24.3 (Superseded by ADR-0034)
- [ADR-0034: Go Version Upgrade to 1.25.3](./0034-go-version-upgrade.md) - Go 1.25.3
- [ADR-0003: Dependency Injection Framework](./0003-dependency-injection-framework.md) - uber-go/fx - [ADR-0003: Dependency Injection Framework](./0003-dependency-injection-framework.md) - uber-go/fx
- [ADR-0004: Configuration Management](./0004-configuration-management.md) - spf13/viper + cobra - [ADR-0004: Configuration Management](./0004-configuration-management.md) - spf13/viper + cobra
- [ADR-0005: Logging Framework](./0005-logging-framework.md) - go.uber.org/zap - [ADR-0005: Logging Framework](./0005-logging-framework.md) - go.uber.org/zap
@@ -35,40 +38,48 @@ Each ADR follows this structure:
- [ADR-0011: Code Generation Tools](./0011-code-generation-tools.md) - go generate workflow - [ADR-0011: Code Generation Tools](./0011-code-generation-tools.md) - go generate workflow
- [ADR-0012: Logger Interface Design](./0012-logger-interface-design.md) - Logger interface abstraction - [ADR-0012: Logger Interface Design](./0012-logger-interface-design.md) - Logger interface abstraction
### Phase 1: Core Kernel & Infrastructure ### Epic 1: Core Kernel & Infrastructure
- [ADR-0013: Database ORM Selection](./0013-database-orm.md) - entgo.io/ent - [ADR-0013: Database ORM Selection](./0013-database-orm.md) - entgo.io/ent
- [ADR-0014: Health Check Implementation](./0014-health-check-implementation.md) - Custom health check registry - [ADR-0014: Health Check Implementation](./0014-health-check-implementation.md) - Custom health check registry
- [ADR-0015: Error Bus Implementation](./0015-error-bus-implementation.md) - Channel-based error bus with pluggable sinks - [ADR-0015: Error Bus Implementation](./0015-error-bus-implementation.md) - Channel-based error bus with pluggable sinks
- [ADR-0016: OpenTelemetry Observability Strategy](./0016-opentelemetry-observability.md) - OpenTelemetry for tracing, metrics, logs - [ADR-0016: OpenTelemetry Observability Strategy](./0016-opentelemetry-observability.md) - OpenTelemetry for tracing, metrics, logs
### Phase 2: Authentication & Authorization ### Epic 2: Authentication & Authorization
- [ADR-0017: JWT Token Strategy](./0017-jwt-token-strategy.md) - Short-lived access tokens + long-lived refresh tokens - [ADR-0017: JWT Token Strategy](./0017-jwt-token-strategy.md) - Short-lived access tokens + long-lived refresh tokens
- [ADR-0018: Password Hashing Algorithm](./0018-password-hashing.md) - argon2id - [ADR-0018: Password Hashing Algorithm](./0018-password-hashing.md) - argon2id
- [ADR-0019: Permission DSL Format](./0019-permission-dsl-format.md) - String-based format with code generation - [ADR-0019: Permission DSL Format](./0019-permission-dsl-format.md) - String-based format with code generation
- [ADR-0020: Audit Logging Storage](./0020-audit-logging-storage.md) - PostgreSQL append-only table with JSONB metadata - [ADR-0020: Audit Logging Storage](./0020-audit-logging-storage.md) - PostgreSQL append-only table with JSONB metadata
### Phase 3: Module Framework ### Epic 3: Module Framework
- [ADR-0021: Module Loading Strategy](./0021-module-loading-strategy.md) - Static registration (primary) + dynamic plugin loading (optional) - [ADR-0021: Module Loading Strategy](./0021-module-loading-strategy.md) - Static registration (primary) + dynamic plugin loading (optional)
### Phase 5: Infrastructure Adapters ### Epic 5: Infrastructure Adapters
- [ADR-0022: Cache Implementation](./0022-cache-implementation.md) - Redis with in-memory fallback - [ADR-0022: Cache Implementation](./0022-cache-implementation.md) - Redis with in-memory fallback
- [ADR-0023: Event Bus Implementation](./0023-event-bus-implementation.md) - In-process bus (default) + Kafka (production) - [ADR-0023: Event Bus Implementation](./0023-event-bus-implementation.md) - In-process bus (default) + Kafka (production)
- [ADR-0024: Background Job Scheduler](./0024-job-scheduler.md) - asynq (Redis-backed) + cron - [ADR-0024: Background Job Scheduler](./0024-job-scheduler.md) - asynq (Redis-backed) + cron
- [ADR-0025: Multi-tenancy Model](./0025-multitenancy-model.md) - Shared database with tenant_id column (optional) - [ADR-0025: Multi-tenancy Model](./0025-multitenancy-model.md) - Shared database with tenant_id column (optional)
### Phase 6: Observability & Production Readiness ### Epic 6: Observability & Production Readiness
- [ADR-0026: Error Reporting Service](./0026-error-reporting-service.md) - Sentry (optional, configurable) - [ADR-0026: Error Reporting Service](./0026-error-reporting-service.md) - Sentry (optional, configurable)
- [ADR-0027: Rate Limiting Strategy](./0027-rate-limiting-strategy.md) - Multi-level (per-user + per-IP) with Redis - [ADR-0027: Rate Limiting Strategy](./0027-rate-limiting-strategy.md) - Multi-level (per-user + per-IP) with Redis
### Phase 7: Testing, Documentation & CI/CD ### Epic 7: Testing, Documentation & CI/CD
- [ADR-0028: Testing Strategy](./0028-testing-strategy.md) - Multi-layered (unit, integration, contract, load) - [ADR-0028: Testing Strategy](./0028-testing-strategy.md) - Multi-layered (unit, integration, contract, load)
### Architecture & Scaling
- [ADR-0029: Microservices Architecture](./0029-microservices-architecture.md) - micromicroservices architecture from day one
- [ADR-0030: Service Communication Strategy](./0030-service-communication-strategy.md) - Service client abstraction and communication patterns with API Gateway
- [ADR-0031: Service Repository Structure](./0031-service-repository-structure.md) - Monorepo with service directories
- [ADR-0032: API Gateway Strategy](./0032-api-gateway-strategy.md) - API Gateway as core infrastructure component
- [ADR-0033: Service Discovery Implementation](./0033-service-discovery-implementation.md) - Consul-based service discovery (primary), Kubernetes as alternative
## Adding New ADRs ## Adding New ADRs
When making a new architectural decision: When making a new architectural decision:

View File

@@ -0,0 +1,603 @@
# Module Architecture
This document details the architecture of modules (feature services), how they are structured as independent services, how they interact with core services, and how multiple services work together in the microservices architecture.
## Table of Contents
- [Module Structure](#module-structure)
- [Module Interface](#module-interface)
- [Module Lifecycle](#module-lifecycle)
- [Module Dependencies](#module-dependencies)
- [Module Communication](#module-communication)
- [Module Data Isolation](#module-data-isolation)
- [Module Examples](#module-examples)
## Module Structure
Every module follows a consistent structure that separates concerns and enables clean integration with the platform.
```mermaid
graph TD
subgraph "Module Structure"
Manifest[module.yaml<br/>Manifest]
subgraph "Public API (pkg/)"
ModuleInterface[IModule Interface]
ModuleTypes[Public Types]
end
subgraph "Internal Implementation (internal/)"
API[API Handlers]
Service[Domain Services]
Repo[Repositories]
Domain[Domain Models]
end
subgraph "Database Schema"
EntSchema[Ent Schemas]
Migrations[Migrations]
end
end
Manifest --> ModuleInterface
ModuleInterface --> API
API --> Service
Service --> Repo
Repo --> Domain
Repo --> EntSchema
EntSchema --> Migrations
style Manifest fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style ModuleInterface fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style Service fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
```
### Service Directory Structure
Each module is an independent service with its own entry point:
```
cmd/blog-service/
└── main.go # Service entry point
services/blog/
├── go.mod # Service dependencies
├── module.yaml # Service manifest
├── api/
│ └── blog.proto # gRPC service definition
├── internal/
│ ├── api/
│ │ └── handler.go # gRPC handlers
│ ├── domain/
│ │ ├── post.go # Domain entities
│ │ └── post_repo.go # Repository interface
│ ├── service/
│ │ └── post_service.go # Business logic
│ └── database/
│ └── client.go # Database connection
└── ent/
├── schema/
│ └── post.go # Ent schema
└── migrate/ # Migrations
```
## Module Interface
All modules must implement the `IModule` interface to integrate with the platform.
```mermaid
classDiagram
class IModule {
<<interface>>
+Name() string
+Version() string
+Dependencies() []string
+Init() fx.Option
+Migrations() []MigrationFunc
+OnStart(ctx) error
+OnStop(ctx) error
}
class BlogModule {
+Name() string
+Version() string
+Dependencies() []string
+Init() fx.Option
+Migrations() []MigrationFunc
}
class BillingModule {
+Name() string
+Version() string
+Dependencies() []string
+Init() fx.Option
+Migrations() []MigrationFunc
}
IModule <|.. BlogModule
IModule <|.. BillingModule
```
### IModule Interface
```go
type IModule interface {
// Name returns a unique, human-readable identifier
Name() string
// Version returns the module version (semantic versioning)
Version() string
// Dependencies returns list of required modules (e.g., ["core >= 1.0.0"])
Dependencies() []string
// Init returns fx.Option that registers all module services
Init() fx.Option
// Migrations returns database migration functions
Migrations() []func(*ent.Client) error
// OnStart is called during application startup (optional)
OnStart(ctx context.Context) error
// OnStop is called during graceful shutdown (optional)
OnStop(ctx context.Context) error
}
```
## Module Lifecycle
Modules go through a well-defined lifecycle from discovery to shutdown.
```mermaid
stateDiagram-v2
[*] --> Discovered: Module found
Discovered --> Validated: Check dependencies
Validated --> Loaded: Load module
Loaded --> Initialized: Call Init()
Initialized --> Migrated: Run migrations
Migrated --> Started: Call OnStart()
Started --> Running: Module active
Running --> Stopping: Shutdown signal
Stopping --> Stopped: Call OnStop()
Stopped --> [*]
Validated --> Rejected: Dependency check fails
Rejected --> [*]
```
### Module Initialization Sequence
```mermaid
sequenceDiagram
participant Main
participant Loader
participant Registry
participant Module
participant DI
participant Router
participant DB
participant Scheduler
Main->>Loader: DiscoverModules()
Loader->>Registry: Scan for modules
Registry-->>Loader: Module list
loop For each module
Loader->>Module: Load module
Module->>Registry: Register module
Registry->>Registry: Validate dependencies
end
Main->>Registry: GetAllModules()
Registry->>Registry: Resolve dependencies (topological sort)
Registry-->>Main: Ordered module list
Main->>DI: Create fx container
loop For each module (in dependency order)
Main->>Module: Init()
Module->>DI: fx.Provide(services)
Module->>Router: Register routes
Module->>Scheduler: Register jobs
Module->>DB: Register migrations
end
Main->>DB: Run migrations (core first)
Main->>DI: Start container
Main->>Module: OnStart() (optional)
Main->>Router: Start HTTP server
```
## Module Dependencies
Modules can depend on other modules, creating a dependency graph that must be resolved.
```mermaid
graph TD
Core[Core Kernel]
Blog[Blog Module]
Billing[Billing Module]
Analytics[Analytics Module]
Notifications[Notification Module]
Blog --> Core
Billing --> Core
Analytics --> Core
Notifications --> Core
Analytics --> Blog
Analytics --> Billing
Billing --> Blog
Notifications --> Blog
Notifications --> Billing
style Core fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style Blog fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
style Billing fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
```
### Dependency Resolution
```mermaid
graph LR
subgraph "Module Dependency Graph"
M1[Module A<br/>depends on: Core]
M2[Module B<br/>depends on: Core, Module A]
M3[Module C<br/>depends on: Core, Module B]
Core[Core Kernel]
end
subgraph "Resolved Load Order"
Step1[1. Core Kernel]
Step2[2. Module A]
Step3[3. Module B]
Step4[4. Module C]
end
Core --> M1
M1 --> M2
M2 --> M3
Step1 --> Step2
Step2 --> Step3
Step3 --> Step4
style Core fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style Step1 fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
```
## Service Communication
Modules are implemented as independent services that communicate through service client interfaces. All inter-service communication uses gRPC (primary) or HTTP (fallback) via service clients. Services discover each other through the service registry (Consul).
### Communication Patterns
```mermaid
graph TB
subgraph "Communication Patterns"
ServiceClients[Service Clients<br/>gRPC/HTTP]
Events[Event Bus<br/>Kafka]
Shared[Shared Infrastructure<br/>Redis, PostgreSQL]
end
subgraph "Blog Service"
BlogService[Blog Service]
BlogHandler[Blog Handler]
end
subgraph "Service Clients"
AuthClient[Auth Service Client]
IdentityClient[Identity Service Client]
AuthzClient[Authz Service Client]
end
subgraph "Core Services"
EventBus[Event Bus]
AuthService[Auth Service<br/>:8081]
IdentityService[Identity Service<br/>:8082]
end
subgraph "Analytics Service"
AnalyticsService[Analytics Service]
end
BlogHandler --> BlogService
BlogService -->|gRPC| AuthClient
BlogService -->|gRPC| IdentityClient
BlogService -->|gRPC| AuthzClient
BlogService -->|gRPC| AuditClient
BlogService -->|Publish| EventBus
EventBus -->|Subscribe| AnalyticsService
AuthClient -->|Discover| Registry
IdentityClient -->|Discover| Registry
AuthzClient -->|Discover| Registry
AuditClient -->|Discover| Registry
Registry --> AuthService
Registry --> IdentityService
Registry --> AuthzService
Registry --> AuditService
AuthClient --> AuthService
IdentityClient --> IdentityService
AuthzClient --> AuthzService
AuditClient --> AuditService
style EventBus fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style Registry fill:#50c878,stroke:#2e7d4e,stroke-width:3px,color:#fff
style BlogService fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
style AnalyticsService fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
style ServiceClients fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
```
### Event-Driven Communication
```mermaid
sequenceDiagram
participant BlogModule
participant EventBus
participant AnalyticsModule
participant NotificationModule
participant AuditModule
BlogModule->>EventBus: Publish("blog.post.created", event)
EventBus->>AnalyticsModule: Deliver event
EventBus->>NotificationModule: Deliver event
EventBus->>AuditModule: Deliver event
AnalyticsModule->>AnalyticsModule: Track post creation
NotificationModule->>NotificationModule: Send notification
AuditModule->>AuditModule: Log audit entry
```
## Module Data Isolation
Modules can have their own database tables while sharing core tables.
```mermaid
erDiagram
USERS ||--o{ USER_ROLES : has
ROLES ||--o{ USER_ROLES : assigned_to
ROLES ||--o{ ROLE_PERMISSIONS : has
PERMISSIONS ||--o{ ROLE_PERMISSIONS : assigned_to
BLOG_POSTS {
string id PK
string author_id FK
string title
string content
timestamp created_at
}
BILLING_SUBSCRIPTIONS {
string id PK
string user_id FK
string plan
timestamp expires_at
}
USERS ||--o{ BLOG_POSTS : creates
USERS ||--o{ BILLING_SUBSCRIPTIONS : subscribes
AUDIT_LOGS {
string id PK
string actor_id
string action
string target_id
jsonb metadata
}
USERS ||--o{ AUDIT_LOGS : performs
```
### Multi-Tenancy Data Isolation
```mermaid
graph TB
subgraph "Single Database"
subgraph "Core Tables"
Users[users<br/>tenant_id]
Roles[roles<br/>tenant_id]
end
subgraph "Blog Module Tables"
Posts[blog_posts<br/>tenant_id]
Comments[blog_comments<br/>tenant_id]
end
subgraph "Billing Module Tables"
Subscriptions[billing_subscriptions<br/>tenant_id]
Invoices[billing_invoices<br/>tenant_id]
end
end
subgraph "Query Filtering"
EntInterceptor[Ent Interceptor]
TenantFilter[WHERE tenant_id = ?]
end
Users --> EntInterceptor
Posts --> EntInterceptor
Subscriptions --> EntInterceptor
EntInterceptor --> TenantFilter
style EntInterceptor fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
```
## Module Examples
### Example: Blog Module
```mermaid
graph TB
subgraph "Blog Module"
BlogHandler[Blog Handler<br/>/api/v1/blog/posts]
BlogService[Post Service]
PostRepo[Post Repository]
PostEntity[Post Entity]
end
subgraph "Service Clients"
AuthClient[Auth Service Client<br/>gRPC]
AuthzClient[Authz Service Client<br/>gRPC]
IdentityClient[Identity Service Client<br/>gRPC]
AuditClient[Audit Service Client<br/>gRPC]
end
subgraph "Core Services"
EventBus[Event Bus<br/>Kafka]
CacheService[Cache Service<br/>Redis]
end
subgraph "Database"
PostsTable[(blog_posts)]
end
BlogHandler --> BlogService
BlogService -->|gRPC| AuthClient
BlogService -->|gRPC| AuthzClient
BlogService -->|gRPC| IdentityClient
BlogService -->|gRPC| AuditClient
BlogService --> PostRepo
BlogService --> EventBus
BlogService --> CacheService
PostRepo --> PostsTable
PostRepo --> PostEntity
style BlogModule fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
style AuthService fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
```
### Module Integration Example
```mermaid
graph LR
subgraph "Request Flow"
Request[HTTP Request<br/>POST /api/v1/blog/posts]
Auth[Auth Middleware]
Authz[Authz Middleware]
Handler[Blog Handler]
Service[Blog Service]
Repo[Blog Repository]
DB[(Database)]
end
subgraph "Service Clients"
AuthClient[Auth Service Client]
IdentityClient[Identity Service Client]
AuthzClient[Authz Service Client]
end
subgraph "Side Effects"
EventBus[Event Bus]
AuditClient[Audit Service Client]
Cache[Cache]
end
Request --> Auth
Auth --> Authz
Authz --> Handler
Handler --> Service
Service --> Repo
Repo --> DB
Service -->|gRPC| AuthClient
Service -->|gRPC| IdentityClient
Service -->|gRPC| AuthzClient
Service -->|gRPC| AuditClient
Service --> EventBus
Service --> Cache
style Request fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
style Service fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
style ServiceClients fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
```
## Module Registration Flow
```mermaid
flowchart TD
Start([Application Start]) --> LoadManifests["Load module.yaml files"]
LoadManifests --> ValidateDeps["Validate dependencies"]
ValidateDeps -->|Valid| SortModules["Topological sort modules"]
ValidateDeps -->|Invalid| Error([Error: Missing dependencies])
SortModules --> CreateDI["Create DI container"]
CreateDI --> RegisterCore["Register core services"]
RegisterCore --> LoopModules{"More modules?"}
LoopModules -->|Yes| LoadModule["Load module"]
LoadModule --> CallInit["Call module.Init()"]
CallInit --> RegisterServices["Register module services"]
RegisterServices --> RegisterRoutes["Register module routes"]
RegisterRoutes --> RegisterJobs["Register module jobs"]
RegisterJobs --> RegisterMigrations["Register module migrations"]
RegisterMigrations --> LoopModules
LoopModules -->|No| RunMigrations["Run all migrations"]
RunMigrations --> StartModules["Call OnStart() for each module"]
StartModules --> StartServer["Start HTTP server"]
StartServer --> Running([Application Running])
Running --> Shutdown([Shutdown Signal])
Shutdown --> StopServer["Stop HTTP server"]
StopServer --> StopModules["Call OnStop() for each module"]
StopModules --> Cleanup["Cleanup resources"]
Cleanup --> End([Application Stopped])
style Start fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style Running fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
style Error fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
```
## Module Permissions Integration
Modules declare permissions that are automatically integrated into the permission system.
```mermaid
graph TB
subgraph "Permission Generation"
Manifest["module.yaml<br/>permissions: array"]
Generator["Permission Generator"]
GeneratedCode["pkg/perm/generated.go"]
end
subgraph "Permission Resolution"
Request["HTTP Request"]
AuthzMiddleware["Authz Middleware"]
PermissionResolver["Permission Resolver"]
UserRoles["User Roles"]
RolePermissions["Role Permissions"]
Response["HTTP Response"]
end
Manifest --> Generator
Generator --> GeneratedCode
GeneratedCode --> PermissionResolver
Request --> AuthzMiddleware
AuthzMiddleware --> PermissionResolver
PermissionResolver --> UserRoles
PermissionResolver --> RolePermissions
UserRoles --> PermissionResolver
RolePermissions --> PermissionResolver
PermissionResolver --> AuthzMiddleware
AuthzMiddleware --> Response
classDef generation fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
classDef resolution fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
class Manifest,Generator,GeneratedCode generation
class PermissionResolver resolution
```
## Next Steps
- [Module Requirements](./module-requirements.md) - Detailed requirements for each module
- [Component Relationships](./component-relationships.md) - How components interact
- [System Architecture](./architecture.md) - Overall system architecture

View File

@@ -0,0 +1,894 @@
# System Architecture
This document provides a comprehensive overview of the Go Platform architecture, including system components, their relationships, and how modules integrate with the core platform.
## Table of Contents
- [High-Level Architecture](#high-level-architecture)
- [Layered Architecture](#layered-architecture)
- [Module System Architecture](#module-system-architecture)
- [Component Relationships](#component-relationships)
- [Data Flow](#data-flow)
- [Deployment Architecture](#deployment-architecture)
## High-Level Architecture
The Go Platform follows a **microservices architecture** where each service is independently deployable from day one:
- **Core Kernel**: Infrastructure only (config, logger, DI, health, metrics, observability) - no business logic
- **Core Services**: Auth Service, Identity Service, Authz Service, Audit Service - separate microservices
- **API Gateway**: Single entry point for all external traffic, handles routing and authentication
- **Feature Services**: Blog, Billing, Analytics, etc. - independent services
- **Infrastructure Adapters**: Cache, Event Bus, Scheduler, etc. - shared infrastructure
All services communicate via gRPC (primary) or HTTP (fallback) through service client interfaces, with service discovery via a service registry. Each service has its own database connection pool and schema. Services share infrastructure (PostgreSQL instance, Redis, Kafka) but are independently deployable and scalable.
```mermaid
graph TB
subgraph "API Gateway"
Gateway[API Gateway<br/>:8080]
end
subgraph "Core Services"
AuthSvc[Auth Service<br/>:8081]
IdentitySvc[Identity Service<br/>:8082]
AuthzSvc[Authz Service<br/>:8083]
AuditSvc[Audit Service<br/>:8084]
end
subgraph "Feature Services"
BlogSvc[Blog Service<br/>:8091]
BillingSvc[Billing Service<br/>:8092]
AnalyticsSvc[Analytics Service<br/>:8093]
end
subgraph "Core Kernel"
Kernel[Core Kernel<br/>Infrastructure Only]
end
subgraph "Infrastructure"
DB[(PostgreSQL)]
Cache[(Redis)]
Queue[Kafka/Event Bus]
Storage[S3/Blob Storage]
Registry[Service Registry]
end
subgraph "External Services"
OIDC[OIDC Provider]
Email[Email Service]
Sentry[Sentry]
end
Gateway --> AuthSvc
Gateway --> IdentitySvc
Gateway --> AuthzSvc
Gateway --> BlogSvc
Gateway --> BillingSvc
AuthSvc --> IdentitySvc
AuthSvc --> Registry
AuthzSvc --> IdentitySvc
AuthzSvc --> Cache
AuthzSvc --> AuditSvc
BlogSvc --> AuthzSvc
BlogSvc --> IdentitySvc
BlogSvc --> Registry
AuthSvc --> DB
IdentitySvc --> DB
AuthzSvc --> DB
AuditSvc --> DB
BlogSvc --> DB
BillingSvc --> DB
AuthSvc --> Cache
AuthzSvc --> Cache
BlogSvc --> Cache
BillingSvc --> Cache
BlogSvc --> Queue
BillingSvc --> Queue
AnalyticsSvc --> Queue
Kernel --> DB
Kernel --> Cache
Kernel --> Queue
Kernel --> Registry
AuthSvc --> OIDC
IdentitySvc --> Email
AuditSvc --> Sentry
style Gateway fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style Kernel fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style AuthSvc fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
style IdentitySvc fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
style BlogSvc fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
style BillingSvc fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
```
## Layered Architecture
The platform follows a **hexagonal architecture** with clear separation of concerns across layers.
```mermaid
graph TD
subgraph "Presentation Layer"
HTTP[HTTP/REST API]
GraphQL[GraphQL API]
CLI[CLI Interface]
end
subgraph "Application Layer"
AuthMiddleware[Auth Middleware]
AuthzMiddleware[Authorization Middleware]
RateLimit[Rate Limiting]
Handlers[Request Handlers]
end
subgraph "Domain Layer"
Services[Domain Services]
Entities[Domain Entities]
Policies[Business Policies]
end
subgraph "Infrastructure Layer"
Repos[Repositories]
CacheAdapter[Cache Adapter]
EventBus[Event Bus]
Jobs[Scheduler/Jobs]
end
subgraph "Core Kernel (Infrastructure Only)"
DI[DI Container]
Config[Config Manager]
Logger[Logger]
Metrics[Metrics]
Health[Health Checks]
Tracer[OpenTelemetry Tracer]
end
HTTP --> AuthMiddleware
GraphQL --> AuthMiddleware
CLI --> AuthMiddleware
AuthMiddleware --> AuthzMiddleware
AuthzMiddleware --> RateLimit
RateLimit --> Handlers
Handlers --> Services
Services --> Entities
Services --> Policies
Services --> Repos
Services --> CacheAdapter
Services --> EventBus
Services --> Jobs
Repos --> DB[(Database)]
CacheAdapter --> Cache[(Redis)]
EventBus --> Queue[(Kafka)]
Services --> DI
Repos --> DI
Handlers --> DI
DI --> Config
DI --> Logger
DI --> Metrics
DI --> Health
style Core fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style Services fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style Repos fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
```
## Module System Architecture
Modules are the building blocks of the platform. Each module can register services, routes, permissions, and background jobs.
```mermaid
graph TB
subgraph Lifecycle["Module Lifecycle"]
Discover["1. Discover Modules"]
Load["2. Load Module"]
Validate["3. Validate Dependencies"]
Init["4. Initialize Module"]
Start["5. Start Module"]
end
subgraph Registration["Module Registration"]
Static["Static Registration<br/>via init()"]
Dynamic["Dynamic Loading<br/>via .so files"]
end
subgraph Components["Module Components"]
Routes["HTTP Routes"]
Services["Services"]
Repos["Repositories"]
Perms["Permissions"]
Jobs["Background Jobs"]
Migrations["Database Migrations"]
end
Discover --> Load
Load --> Static
Load --> Dynamic
Static --> Validate
Dynamic --> Validate
Validate --> Init
Init --> Routes
Init --> Services
Init --> Repos
Init --> Perms
Init --> Jobs
Init --> Migrations
Routes --> Start
Services --> Start
Repos --> Start
Perms --> Start
Jobs --> Start
Migrations --> Start
classDef lifecycle fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
classDef registration fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
classDef components fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
classDef start fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
class Discover,Load,Validate,Init lifecycle
class Static,Dynamic registration
class Routes,Services,Repos,Perms,Jobs,Migrations components
class Start start
```
### Module Initialization Sequence
```mermaid
sequenceDiagram
participant Main
participant Loader
participant Registry
participant Module
participant DI
participant Router
participant DB
Main->>Loader: LoadModules()
Loader->>Registry: Discover modules
Registry-->>Loader: List of modules
loop For each module
Loader->>Module: Load module
Module->>Registry: Register(module)
Registry->>Registry: Validate dependencies
end
Main->>Registry: GetAllModules()
Registry-->>Main: Ordered module list
Main->>DI: Create container
loop For each module
Main->>Module: Init()
Module->>DI: Provide services
Module->>Router: Register routes
Module->>DB: Register migrations
end
Main->>DB: Run migrations
Main->>Router: Start HTTP server
```
## Component Relationships
This diagram shows how core components interact with each other and with modules.
```mermaid
graph TB
subgraph "Core Kernel Components (Infrastructure)"
ConfigMgr[Config Manager]
LoggerService[Logger Service]
DI[DI Container]
ModuleLoader[Module Loader]
HealthRegistry[Health Registry]
MetricsRegistry[Metrics Registry]
ErrorBus[Error Bus]
Tracer[OpenTelemetry Tracer]
ServiceRegistry[Service Registry]
end
subgraph "Core Services (Separate Microservices)"
AuthService[Auth Service<br/>:8081]
IdentityService[Identity Service<br/>:8082]
AuthzService[Authz Service<br/>:8083]
AuditService[Audit Service<br/>:8084]
end
subgraph "Infrastructure Adapters"
EventBus[Event Bus<br/>Kafka]
CacheService[Cache Service<br/>Redis]
end
subgraph "Infrastructure Components"
DBClient[Database Client]
CacheClient[Cache Client]
Scheduler[Scheduler]
Notifier[Notifier]
end
subgraph "External Services"
Sentry[Sentry<br/>Error Reporting]
end
subgraph "Module Components"
ModuleRoutes[Module Routes]
ModuleServices[Module Services]
ModuleRepos[Module Repositories]
end
DI --> ConfigMgr
DI --> LoggerService
DI --> ModuleLoader
DI --> HealthRegistry
DI --> MetricsRegistry
DI --> ErrorBus
DI --> Tracer
DI --> ServiceRegistry
ModuleServices -->|gRPC| AuthService
ModuleServices -->|gRPC| IdentityService
ModuleServices -->|gRPC| AuthzService
ModuleServices -->|gRPC| AuditService
ModuleServices --> EventBus
ModuleServices --> CacheService
ModuleServices --> DBClient
ModuleRepos --> DBClient
AuthService --> DBClient
IdentityService --> DBClient
AuthzService --> DBClient
AuditService --> DBClient
AuthService --> ServiceRegistry
IdentityService --> ServiceRegistry
AuthzService --> ServiceRegistry
AuditService --> ServiceRegistry
ErrorBus --> LoggerService
ErrorBus --> Sentry
style DI fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style AuthService fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
style IdentityService fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
style ModuleServices fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
```
## Data Flow
### Request Processing Flow
```mermaid
sequenceDiagram
participant Client
participant Gateway[API Gateway]
participant AuthSvc[Auth Service]
participant AuthzSvc[Authz Service]
participant Service[Feature Service]
participant IdentitySvc[Identity Service]
participant AuditSvc[Audit Service]
participant Repo
participant DB
participant Cache
participant EventBus
Client->>Gateway: HTTP Request
Gateway->>Gateway: Rate limiting
Gateway->>AuthSvc: Validate JWT token (gRPC)
AuthSvc->>AuthSvc: Verify token
AuthSvc-->>Gateway: Token valid + user info
Gateway->>AuthzSvc: Check permissions (gRPC)
AuthzSvc->>AuthzSvc: Resolve permissions
AuthzSvc-->>Gateway: Authorized
Gateway->>Service: Route to service (gRPC/HTTP)
Service->>Cache: Check cache
Cache-->>Service: Cache miss
Service->>Repo: Query data
Repo->>DB: Execute query
DB-->>Repo: Return data
Repo-->>Service: Domain entity
Service->>Cache: Store in cache
Service->>IdentitySvc: Get user info (gRPC, if needed)
IdentitySvc-->>Service: User data
Service->>EventBus: Publish event
Service->>AuditSvc: Record action (gRPC)
Service-->>Gateway: Response data
Gateway-->>Client: JSON response
```
### Module Event Flow
```mermaid
graph LR
subgraph "Module A"
AService[Service A]
AHandler[Handler A]
end
subgraph "Event Bus"
Bus[Event Bus]
end
subgraph "Module B"
BService[Service B]
BHandler[Handler B]
end
subgraph "Module C"
CService[Service C]
end
AHandler --> AService
AService -->|Publish Event| Bus
Bus -->|Subscribe| BService
Bus -->|Subscribe| CService
BService --> BHandler
CService --> CService
```
## Deployment Architecture
### Development Deployment
```mermaid
graph TB
subgraph "Developer Machine"
IDE[IDE/Editor]
Go[Go Runtime]
Docker[Docker]
end
subgraph "Local Services"
Gateway[API Gateway<br/>:8080]
AuthSvc[Auth Service<br/>:8081]
IdentitySvc[Identity Service<br/>:8082]
AuthzSvc[Authz Service<br/>:8083]
AuditSvc[Audit Service<br/>:8084]
BlogSvc[Blog Service<br/>:8091]
DB[(PostgreSQL<br/>:5432)]
Redis[(Redis<br/>:6379)]
Kafka[Kafka<br/>:9092]
Consul[Consul<br/>:8500]
end
IDE --> Go
Go --> Gateway
Go --> AuthSvc
Go --> IdentitySvc
Go --> AuthzSvc
Go --> AuditSvc
Go --> BlogSvc
Gateway --> AuthSvc
Gateway --> IdentitySvc
Gateway --> BlogSvc
AuthSvc --> DB
IdentitySvc --> DB
AuthzSvc --> DB
AuditSvc --> DB
BlogSvc --> DB
AuthSvc --> Redis
AuthzSvc --> Redis
BlogSvc --> Redis
BlogSvc --> Kafka
AuthSvc --> Consul
IdentitySvc --> Consul
AuthzSvc --> Consul
AuditSvc --> Consul
BlogSvc --> Consul
Docker --> DB
Docker --> Redis
Docker --> Kafka
Docker --> Consul
style Gateway fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
style AuthSvc fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
style IdentitySvc fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
```
### Production Deployment
```mermaid
graph TB
subgraph "Load Balancer"
LB[Load Balancer<br/>HTTPS]
end
subgraph "Service Instances"
Gateway1[API Gateway 1]
Gateway2[API Gateway 2]
AuthSvc1[Auth Service 1]
AuthSvc2[Auth Service 2]
IdentitySvc1[Identity Service 1]
IdentitySvc2[Identity Service 2]
BlogSvc1[Blog Service 1]
BlogSvc2[Blog Service 2]
end
subgraph "Database Cluster"
Primary[(PostgreSQL<br/>Primary)]
Replica[(PostgreSQL<br/>Replica)]
end
subgraph "Cache Cluster"
Redis1[(Redis<br/>Master)]
Redis2[(Redis<br/>Replica)]
end
subgraph "Message Queue"
Kafka1[Kafka Broker 1]
Kafka2[Kafka Broker 2]
Kafka3[Kafka Broker 3]
end
subgraph "Observability"
Prometheus[Prometheus]
Grafana[Grafana]
Jaeger[Jaeger]
Loki[Loki]
end
subgraph "External Services"
Sentry[Sentry]
S3[S3 Storage]
end
LB --> Gateway1
LB --> Gateway2
Gateway1 --> AuthSvc1
Gateway1 --> AuthSvc2
Gateway1 --> IdentitySvc1
Gateway1 --> IdentitySvc2
Gateway1 --> BlogSvc1
Gateway1 --> BlogSvc2
Gateway2 --> AuthSvc1
Gateway2 --> AuthSvc2
Gateway2 --> IdentitySvc1
Gateway2 --> IdentitySvc2
Gateway2 --> BlogSvc1
Gateway2 --> BlogSvc2
AuthSvc1 --> Primary
AuthSvc2 --> Primary
IdentitySvc1 --> Primary
IdentitySvc2 --> Primary
BlogSvc1 --> Primary
BlogSvc2 --> Primary
AuthSvc1 --> Replica
AuthSvc2 --> Replica
IdentitySvc1 --> Replica
IdentitySvc2 --> Replica
BlogSvc1 --> Replica
BlogSvc2 --> Replica
AuthSvc1 --> Redis1
AuthSvc2 --> Redis1
IdentitySvc1 --> Redis1
IdentitySvc2 --> Redis1
BlogSvc1 --> Redis1
BlogSvc2 --> Redis1
BlogSvc1 --> Kafka1
BlogSvc2 --> Kafka2
AuthSvc1 --> Prometheus
AuthSvc2 --> Prometheus
IdentitySvc1 --> Prometheus
IdentitySvc2 --> Prometheus
BlogSvc1 --> Prometheus
BlogSvc2 --> Prometheus
Prometheus --> Grafana
AuthSvc1 --> Jaeger
AuthSvc2 --> Jaeger
IdentitySvc1 --> Jaeger
IdentitySvc2 --> Jaeger
BlogSvc1 --> Jaeger
BlogSvc2 --> Jaeger
AuthSvc1 --> Loki
AuthSvc2 --> Loki
IdentitySvc1 --> Loki
IdentitySvc2 --> Loki
BlogSvc1 --> Loki
BlogSvc2 --> Loki
AuthSvc1 --> Sentry
AuthSvc2 --> Sentry
IdentitySvc1 --> Sentry
IdentitySvc2 --> Sentry
BlogSvc1 --> Sentry
BlogSvc2 --> Sentry
BlogSvc1 --> S3
BlogSvc2 --> S3
style LB fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style Gateway1 fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
style Gateway2 fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
style Primary fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style Redis1 fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
```
## Core Kernel Components
The core kernel provides **infrastructure only** - no business logic. It is the foundation that all services depend on. Business logic resides in separate services (Auth, Identity, Authz, Audit).
### Component Responsibilities
```mermaid
mindmap
root((Core Kernel))
Configuration
Load configs
Environment vars
Secret management
Dependency Injection
Service registration
Lifecycle management
Module wiring
Logging
Structured logs
Request correlation
Log levels
Observability
Metrics
Tracing
Health checks
Service Discovery
Service registry
Service registration
Health checking
Module System
Module discovery
Module loading
Dependency resolution
```
## Module Integration Points
Modules integrate with the core through well-defined interfaces:
```mermaid
graph TB
subgraph "Core Kernel Interfaces"
IConfig[ConfigProvider]
ILogger[Logger]
ITracer[Tracer]
IMetrics[Metrics]
IHealth[Health]
end
subgraph "Service Client Interfaces"
IAuthClient[AuthServiceClient]
IIdentityClient[IdentityServiceClient]
IAuthzClient[AuthzServiceClient]
IAuditClient[AuditServiceClient]
end
subgraph "Infrastructure Interfaces"
IEventBus[EventBus]
ICache[Cache]
IBlobStore[BlobStore]
IScheduler[Scheduler]
INotifier[Notifier]
end
subgraph "Feature Service Implementation"
Module[Feature Service]
ModuleServices[Service Layer]
ModuleRoutes[HTTP/gRPC Routes]
end
Module --> IConfig
Module --> ILogger
Module --> ITracer
Module --> IMetrics
Module --> IHealth
ModuleServices -->|gRPC| IAuthClient
ModuleServices -->|gRPC| IIdentityClient
ModuleServices -->|gRPC| IAuthzClient
ModuleServices -->|gRPC| IAuditClient
ModuleServices --> IEventBus
ModuleServices --> ICache
ModuleServices --> IBlobStore
ModuleServices --> IScheduler
ModuleServices --> INotifier
ModuleRoutes -->|gRPC| IAuthzClient
style IConfig fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
style Module fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
```
## Microservices Architecture
The platform is designed as **microservices from day one**, with each module being an independent service.
### Service Architecture
```mermaid
graph TB
subgraph "API Gateway"
Gateway[API Gateway<br/>Routing & Auth]
end
subgraph "Core Services"
AuthSvc[Auth Service<br/>:8081]
IdentitySvc[Identity Service<br/>:8082]
AuthzSvc[Authz Service<br/>:8083]
AuditSvc[Audit Service<br/>:8084]
end
subgraph "Feature Services"
BlogSvc[Blog Service<br/>:8091]
BillingSvc[Billing Service<br/>:8092]
AnalyticsSvc[Analytics Service<br/>:8093]
end
subgraph "Infrastructure"
DB[(PostgreSQL)]
Cache[(Redis)]
Queue[Kafka]
Registry[Service Registry]
end
Gateway --> AuthSvc
Gateway --> IdentitySvc
Gateway --> BlogSvc
Gateway --> BillingSvc
AuthSvc --> IdentitySvc
AuthSvc --> Registry
BlogSvc --> AuthzSvc
BlogSvc --> IdentitySvc
BlogSvc --> Registry
BillingSvc --> IdentitySvc
BillingSvc --> Registry
AuthSvc --> DB
IdentitySvc --> DB
BlogSvc --> DB
BillingSvc --> DB
AuthSvc --> Cache
BlogSvc --> Cache
BillingSvc --> Cache
BlogSvc --> Queue
BillingSvc --> Queue
AnalyticsSvc --> Queue
style Gateway fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style Registry fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
```
### Service Communication
All inter-service communication uses service client interfaces:
```mermaid
graph TB
subgraph "Service Client Interface"
Interface[Service Interface<br/>pkg/services/]
end
subgraph "Implementations"
GRPC[gRPC Client<br/>Primary]
HTTP[HTTP Client<br/>Fallback]
end
subgraph "Service Registry"
Registry[Service Registry<br/>Discovery & Resolution]
end
Interface --> GRPC
Interface --> HTTP
Registry --> GRPC
Registry --> HTTP
style Interface fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style Registry fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
```
### Service Communication Patterns
The platform uses three communication patterns:
1. **Synchronous Service Calls** (via Service Clients):
- gRPC calls (primary) - type-safe, efficient
- HTTP/REST calls (fallback) - for external integration
- All calls go through service client interfaces
- Service discovery via registry
2. **Asynchronous Events** (via Event Bus):
- Distributed via Kafka
- Preferred for cross-service communication
- Event-driven architecture for loose coupling
3. **Shared Infrastructure** (For state):
- Redis for cache and distributed state
- PostgreSQL for persistent data
- Kafka for events
### Service Registry
The service registry enables service discovery and resolution:
```mermaid
graph TB
subgraph "Service Registry"
Registry[Service Registry Interface]
Consul[Consul Registry]
K8s[K8s Service Discovery]
Etcd[etcd Registry]
end
subgraph "Services"
AuthSvc[Auth Service]
IdentitySvc[Identity Service]
BlogSvc[Blog Service]
end
Registry --> Consul
Registry --> K8s
Registry --> Etcd
Consul --> AuthSvc
K8s --> IdentitySvc
Etcd --> BlogSvc
style Registry fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
```
### Scaling Strategy
#### Independent Service Scaling
- Scale individual services based on load
- Independent resource allocation
- Independent deployment
- Better resource utilization
- Team autonomy
#### Development Mode
- For local development, services can run in the same repository/monorepo
- Services still communicate via gRPC/HTTP through service clients (no direct in-process calls)
- Each service has its own process and entry point
- Docker Compose for easy local setup with all services
- Maintains microservices architecture even in development
- Services can be started individually for debugging
## Next Steps
- [Module Architecture](./architecture-modules.md) - Detailed module architecture and design
- [Module Requirements](./module-requirements.md) - Requirements for each module
- [Component Relationships](./component-relationships.md) - Detailed component interactions
- [ADRs](../adr/README.md) - Architecture Decision Records
- [ADR-0029: Microservices Architecture](../adr/0029-microservices-architecture.md) - Microservices strategy
- [ADR-0030: Service Communication](../adr/0030-service-communication-strategy.md) - Communication patterns

View File

@@ -0,0 +1,489 @@
# Component Relationships
This document details how different components of the Go Platform interact with each other, including dependency relationships, data flow, and integration patterns.
## Table of Contents
- [Core Component Dependencies](#core-component-dependencies)
- [Module to Core Integration](#module-to-core-integration)
- [Service Interaction Patterns](#service-interaction-patterns)
- [Data Flow Patterns](#data-flow-patterns)
- [Dependency Graph](#dependency-graph)
## Core Component Dependencies
The core kernel components have well-defined dependencies that form the foundation of the platform.
```mermaid
graph TD
subgraph "Foundation Layer"
Config[Config Manager]
Logger[Logger Service]
end
subgraph "DI Layer"
DI[DI Container]
end
subgraph "Infrastructure Layer"
DB[Database Client]
Cache[Cache Client]
EventBus[Event Bus]
Scheduler[Scheduler]
end
subgraph "Service Registry"
Registry[Service Registry<br/>Consul]
end
subgraph "Observability Layer"
Metrics[Metrics Registry]
Health[Health Registry]
Tracer[OpenTelemetry Tracer]
end
Config --> Logger
Config --> DI
Logger --> DI
DI --> DB
DI --> Cache
DI --> EventBus
DI --> Scheduler
DI --> Metrics
DI --> Health
DI --> Tracer
DI --> Registry
DB --> Tracer
Cache --> Tracer
EventBus --> Tracer
style Config fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style DI fill:#50c878,stroke:#2e7d4e,stroke-width:3px,color:#fff
```
## Service to Service Integration
Feature services integrate with core services through service client interfaces. All communication uses gRPC (primary) or HTTP (fallback). Services discover each other via the service registry (Consul).
```mermaid
graph LR
subgraph "Feature Service (e.g., Blog)"
ModuleHandler[Module Handler]
ModuleService[Module Service]
ModuleRepo[Module Repository]
end
subgraph "Service Clients"
AuthClient[Auth Service Client<br/>gRPC]
AuthzClient[Authz Service Client<br/>gRPC]
IdentityClient[Identity Service Client<br/>gRPC]
AuditClient[Audit Service Client<br/>gRPC]
end
subgraph "Service Registry"
Registry[Consul<br/>Service Discovery]
end
subgraph "Core Services"
AuthService[Auth Service<br/>:8081]
AuthzService[Authz Service<br/>:8083]
IdentityService[Identity Service<br/>:8082]
AuditService[Audit Service<br/>:8084]
EventBusService[Event Bus<br/>Kafka]
CacheService[Cache Service<br/>Redis]
end
subgraph "Infrastructure"
DBClient[Database Client]
CacheClient[Cache Client]
QueueClient[Message Queue]
end
ModuleHandler --> ModuleService
ModuleService -->|gRPC| AuthClient
ModuleService -->|gRPC| AuthzClient
ModuleService -->|gRPC| IdentityClient
ModuleService -->|gRPC| AuditClient
ModuleService --> ModuleRepo
ModuleService --> EventBusService
ModuleService --> CacheService
AuthClient -->|Discover| Registry
AuthzClient -->|Discover| Registry
IdentityClient -->|Discover| Registry
AuditClient -->|Discover| Registry
Registry --> AuthService
Registry --> AuthzService
Registry --> IdentityService
Registry --> AuditService
AuthClient --> AuthService
AuthzClient --> AuthzService
IdentityClient --> IdentityService
AuditClient --> AuditService
ModuleRepo --> DBClient
CacheService --> CacheClient
EventBusService --> QueueClient
style ModuleService fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
style Registry fill:#50c878,stroke:#2e7d4e,stroke-width:3px,color:#fff
style AuthService fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
style DBClient fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style AuthClient fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style AuthzClient fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style IdentityClient fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style AuditClient fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
```
## Service Interaction Patterns
### Authentication Flow
```mermaid
sequenceDiagram
participant Client
participant Router
participant AuthMiddleware
participant AuthService
participant TokenProvider
participant UserRepo
participant DB
Client->>Router: POST /api/v1/auth/login
Router->>AuthMiddleware: Extract credentials
AuthMiddleware->>AuthService: Authenticate(email, password)
AuthService->>UserRepo: FindByEmail(email)
UserRepo->>DB: Query user
DB-->>UserRepo: User data
UserRepo-->>AuthService: User entity
AuthService->>AuthService: Verify password
AuthService->>TokenProvider: GenerateAccessToken(user)
AuthService->>TokenProvider: GenerateRefreshToken(user)
TokenProvider-->>AuthService: Tokens
AuthService->>DB: Store refresh token
AuthService-->>AuthMiddleware: Auth response
AuthMiddleware-->>Router: Tokens
Router-->>Client: JSON response with tokens
```
### Authorization Flow
```mermaid
sequenceDiagram
participant Request
participant AuthzMiddleware
participant Authorizer
participant PermissionResolver
participant Cache
participant UserRepo
participant RoleRepo
participant DB
Request->>AuthzMiddleware: HTTP request + permission
AuthzMiddleware->>Authorizer: Authorize(ctx, permission)
Authorizer->>Authorizer: Extract user from context
Authorizer->>PermissionResolver: HasPermission(user, permission)
PermissionResolver->>Cache: Check cache
Cache-->>PermissionResolver: Cache miss
PermissionResolver->>UserRepo: GetUserRoles(userID)
UserRepo->>DB: Query user_roles
DB-->>UserRepo: Role IDs
UserRepo-->>PermissionResolver: Roles
PermissionResolver->>RoleRepo: GetRolePermissions(roleIDs)
RoleRepo->>DB: Query role_permissions
DB-->>RoleRepo: Permissions
RoleRepo-->>PermissionResolver: Permission list
PermissionResolver->>PermissionResolver: Check if permission in list
PermissionResolver->>Cache: Store in cache
PermissionResolver-->>Authorizer: Has permission: true/false
Authorizer-->>AuthzMiddleware: Authorized or error
AuthzMiddleware-->>Request: Continue or 403
```
### Event Publishing Flow
```mermaid
sequenceDiagram
participant ModuleService
participant EventBus
participant Kafka
participant Subscriber1
participant Subscriber2
ModuleService->>EventBus: Publish(topic, event)
EventBus->>EventBus: Serialize event
EventBus->>Kafka: Send to topic
Kafka-->>EventBus: Acknowledged
Kafka->>Subscriber1: Deliver event
Kafka->>Subscriber2: Deliver event
Subscriber1->>Subscriber1: Process event
Subscriber2->>Subscriber2: Process event
```
## Data Flow Patterns
### Request to Response Flow
```mermaid
graph LR
Client[Client] -->|HTTP Request| LB[Load Balancer]
LB -->|Route| Server1[Instance 1]
LB -->|Route| Server2[Instance 2]
Server1 --> AuthMW[Auth Middleware]
Server1 --> AuthzMW[Authz Middleware]
Server1 --> RateLimit[Rate Limiter]
Server1 --> Handler[Request Handler]
Server1 --> Service[Domain Service]
Server1 --> Cache[Cache Check]
Server1 --> Repo[Repository]
Server1 --> DB[(Database)]
Service --> EventBus[Event Bus]
Service --> Audit[Audit Log]
Handler -->|Response| Server1
Server1 -->|HTTP Response| LB
LB -->|Response| Client
style Server1 fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
style Service fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
```
### Caching Flow
```mermaid
graph TD
Request[Service Request] --> CacheCheck{Cache Hit?}
CacheCheck -->|Yes| CacheGet[Get from Cache]
CacheCheck -->|No| DBQuery[Query Database]
DBQuery --> DBResponse[Database Response]
DBResponse --> CacheStore[Store in Cache]
CacheStore --> Return[Return Data]
CacheGet --> Return
style CacheCheck fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
style CacheGet fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style DBQuery fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
```
## Dependency Graph
Complete dependency graph showing all components and their relationships.
```mermaid
graph TB
subgraph "Application Entry"
Main[Main Application]
end
subgraph "Core Kernel"
Config[Config]
Logger[Logger]
DI[DI Container]
ModuleLoader[Module Loader]
end
subgraph "Security"
Auth[Auth]
Authz[Authz]
Identity[Identity]
Audit[Audit]
end
subgraph "Infrastructure"
DB[Database]
Cache[Cache]
EventBus[Event Bus]
Scheduler[Scheduler]
BlobStore[Blob Store]
Notifier[Notifier]
end
subgraph "Observability"
Metrics[Metrics]
Health[Health]
Tracer[Tracer]
ErrorBus[Error Bus]
end
subgraph "Module"
ModuleHandler[Module Handler]
ModuleService[Module Service]
ModuleRepo[Module Repo]
end
Main --> Config
Main --> Logger
Main --> DI
Main --> ModuleLoader
Config --> Logger
Config --> DI
DI --> Auth
DI --> Authz
DI --> Identity
DI --> Audit
DI --> DB
DI --> Cache
DI --> EventBus
DI --> Scheduler
DI --> BlobStore
DI --> Notifier
DI --> Metrics
DI --> Health
DI --> Tracer
DI --> ErrorBus
Auth --> Identity
Auth --> DB
Authz --> Identity
Authz --> Cache
Authz --> Audit
Audit --> DB
Audit --> Logger
ModuleLoader --> DI
ModuleHandler --> ModuleService
ModuleService --> ModuleRepo
ModuleService -->|gRPC| Auth
ModuleService -->|gRPC| Authz
ModuleService -->|gRPC| Identity
ModuleService -->|gRPC| Audit
ModuleService --> EventBus
ModuleService --> Cache
ModuleRepo --> DB
Scheduler --> Cache
Notifier --> EventBus
ErrorBus --> Logger
DB --> Tracer
Cache --> Tracer
EventBus --> Tracer
style Main fill:#4a90e2,stroke:#2e5c8a,stroke-width:4px,color:#fff
style DI fill:#50c878,stroke:#2e7d4e,stroke-width:3px,color:#fff
style ModuleService fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
```
## Component Interaction Matrix
| Component | Depends On | Used By |
|-----------|-----------|---------|
| Config | None | All components |
| Logger | Config | All components |
| DI Container | Config, Logger | All components |
| Auth Service | Identity, DB | Auth Middleware, Modules |
| Authz Service | Identity, Cache, Audit | Authz Middleware, Modules |
| Identity Service | DB, Cache, Notifier | Auth, Authz, Modules |
| Database Client | Config, Logger, Tracer | All repositories |
| Cache Client | Config, Logger | Authz, Scheduler, Modules |
| Event Bus | Config, Logger, Tracer | Modules, Notifier |
| Scheduler | Cache, Logger | Modules |
| Error Bus | Logger | All components (via panic recovery) |
## Integration Patterns
### Module Service Integration
```mermaid
graph TB
subgraph "Module Layer"
Handler[HTTP Handler]
Service[Domain Service]
Repo[Repository]
end
subgraph "Core Services"
Auth[Auth Service]
Authz[Authz Service]
EventBus[Event Bus]
Cache[Cache]
Audit[Audit]
end
subgraph "Infrastructure"
DB[(Database)]
Redis[(Redis)]
Kafka[Kafka]
end
Handler --> Auth
Handler --> Authz
Handler --> Service
Service --> Repo
Service --> EventBus
Service --> Cache
Service --> Audit
Repo --> DB
Cache --> Redis
EventBus --> Kafka
Audit --> DB
style Service fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
style Auth fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
style DB fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
```
### Cross-Service Communication
```mermaid
graph LR
subgraph "Blog Service"
BlogService[Blog Service]
end
subgraph "Analytics Service"
AnalyticsService[Analytics Service]
end
subgraph "Service Clients"
AuthzClient[Authz Service Client]
IdentityClient[Identity Service Client]
end
subgraph "Core Services"
EventBus[Event Bus<br/>Kafka]
AuthzService[Authz Service<br/>:8083]
IdentityService[Identity Service<br/>:8082]
Cache[Cache<br/>Redis]
end
BlogService -->|gRPC| AuthzClient
BlogService -->|gRPC| IdentityClient
BlogService -->|Publish Event| EventBus
EventBus -->|Subscribe| AnalyticsService
BlogService -->|Cache Access| Cache
AnalyticsService -->|Cache Access| Cache
AuthzClient --> AuthzService
IdentityClient --> IdentityService
style BlogService fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
style AnalyticsService fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
style EventBus fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style AuthzClient fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style IdentityClient fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
```
## Next Steps
- [System Architecture](./architecture.md) - Overall system architecture
- [Module Architecture](./architecture-modules.md) - Module design and integration
- [Module Requirements](./module-requirements.md) - Detailed module requirements

View File

@@ -0,0 +1,440 @@
# Data Flow Patterns
## Purpose
This document describes how data flows through the Go Platform system, covering request/response flows, event flows, cache patterns, and observability data collection.
## Overview
Data flows through the platform in multiple patterns depending on the type of operation. Understanding these patterns helps in debugging, performance optimization, and system design decisions.
## Key Concepts
- **Request Flow**: Data flow from HTTP request to response
- **Event Flow**: Asynchronous data flow through event bus
- **Cache Flow**: Data flow through caching layers
- **Observability Flow**: Telemetry data collection and export
## Request/Response Data Flow
### Standard HTTP Request Flow
Complete data flow from HTTP request through API Gateway to backend service and response.
```mermaid
graph TD
Start[HTTP Request] --> Gateway[API Gateway]
Gateway --> RateLimit{Rate Limit Check}
RateLimit -->|Allowed| Auth[Validate JWT via Auth Service]
RateLimit -->|Exceeded| Error0[429 Too Many Requests]
Auth -->|Valid| Authz[Check Permission via Authz Service]
Auth -->|Invalid| Error1[401 Unauthorized]
Authz -->|Authorized| Route[Route to Backend Service]
Authz -->|Unauthorized| Error2[403 Forbidden]
Route --> Service[Backend Service]
Service --> Cache{Cache Check}
Cache -->|Hit| CacheData[Return Cached Data]
Cache -->|Miss| Repo[Repository]
Repo --> DB[(Database)]
DB --> Repo
Repo --> Service
Service --> CacheStore[Update Cache]
Service --> EventBus[Publish Events]
Service --> AuditSvc[Audit Service<br/>gRPC]
Service --> Metrics[Update Metrics]
Service --> Gateway
Gateway --> Response[HTTP Response]
CacheData --> Gateway
Error0 --> Response
Error1 --> Response
Error2 --> Response
Response --> Client[Client]
style Gateway fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style Auth fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
style Service fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style Cache fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
```
### Request Data Transformation
How request data is transformed as it flows through API Gateway to backend service.
```mermaid
sequenceDiagram
participant Client
participant Gateway
participant BackendService
participant Service
participant Repo
participant DB
Client->>Gateway: HTTP Request (JSON)
Gateway->>Gateway: Rate limiting
Gateway->>Gateway: Validate JWT (via Auth Service)
Gateway->>Gateway: Check permission (via Authz Service)
Gateway->>Gateway: Route to service (via service discovery)
Gateway->>Gateway: Forward request (gRPC/HTTP)
Gateway->>BackendService: Request (gRPC/HTTP)
BackendService->>BackendService: Parse request
BackendService->>BackendService: Validate request
BackendService->>BackendService: Convert to DTO
BackendService->>Service: Business DTO
Service->>Service: Business logic
Service->>Service: Domain entity
Service->>Repo: Domain entity
Repo->>Repo: Convert to DB model
Repo->>DB: SQL query
DB-->>Repo: DB result
Repo->>Repo: Convert to domain entity
Repo-->>Service: Domain entity
Service->>Service: Business logic
Service->>Service: Response DTO
Service-->>BackendService: Response DTO
BackendService->>BackendService: Convert to response format
BackendService-->>Gateway: Response (gRPC/HTTP)
Gateway->>Gateway: Transform response (if needed)
Gateway-->>Client: HTTP Response (JSON)
```
## Event Data Flow
### Event Publishing Flow
How events are published and flow through the event bus.
```mermaid
graph LR
Publisher[Event Publisher] --> Serialize[Serialize Event]
Serialize --> Metadata[Add Metadata]
Metadata --> EventBus[Event Bus]
EventBus --> Topic[Kafka Topic]
Topic --> Subscriber1[Subscriber 1]
Topic --> Subscriber2[Subscriber 2]
Topic --> SubscriberN[Subscriber N]
Subscriber1 --> Process1[Process Event]
Subscriber2 --> Process2[Process Event]
SubscriberN --> ProcessN[Process Event]
style EventBus fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
style Topic fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
```
### Event Data Transformation
How event data is transformed during publishing and consumption.
```mermaid
sequenceDiagram
participant Publisher
participant EventBus
participant Kafka
participant Subscriber
Publisher->>Publisher: Domain event
Publisher->>EventBus: Publish(event)
EventBus->>EventBus: Serialize to JSON
EventBus->>EventBus: Add metadata:
- trace_id
- user_id
- timestamp
- source
EventBus->>Kafka: Send to topic
Kafka-->>EventBus: Acknowledged
Kafka->>Subscriber: Deliver event
Subscriber->>Subscriber: Deserialize JSON
Subscriber->>Subscriber: Extract metadata
Subscriber->>Subscriber: Domain event
Subscriber->>Subscriber: Process event
```
## Cache Data Flow
### Cache-Aside Pattern Flow
How data flows through cache using the cache-aside pattern.
```mermaid
graph TD
Start[Service Request] --> Check{Cache Hit?}
Check -->|Yes| GetCache[Get from Cache]
Check -->|No| GetDB[Query Database]
GetCache --> Deserialize[Deserialize Data]
Deserialize --> Return[Return Data]
GetDB --> DB[(Database)]
DB --> DBData[Database Result]
DBData --> Serialize[Serialize Data]
Serialize --> StoreCache[Store in Cache]
StoreCache --> Return
style Check fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
style StoreCache fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
```
### Cache Invalidation Flow
How cache is invalidated when data changes.
```mermaid
sequenceDiagram
participant Service
participant Repository
participant DB
participant Cache
Service->>Repository: Update entity
Repository->>DB: Update database
DB-->>Repository: Update complete
Repository->>Cache: Invalidate(key)
Cache->>Cache: Remove from cache
Cache-->>Repository: Invalidated
Repository-->>Service: Update complete
Note over Service,Cache: Next read will fetch from DB and cache
```
### Cache Write-Through Pattern
How data is written through cache to database.
```mermaid
sequenceDiagram
participant Service
participant Cache
participant Repository
participant DB
Service->>Cache: Write data
Cache->>Cache: Store in cache
Cache->>Repository: Write to database
Repository->>DB: Insert/Update
DB-->>Repository: Success
Repository-->>Cache: Write complete
Cache-->>Service: Data written
```
## Observability Data Flow
### Tracing Data Flow
How distributed tracing data flows through the system.
```mermaid
graph TD
Request[HTTP Request] --> Trace[Start Trace]
Trace --> Span1[HTTP Span]
Span1 --> Service[Service Call]
Service --> Span2[Service Span]
Span2 --> DB[Database Query]
DB --> Span3[DB Span]
Span2 --> gRPC[gRPC Call]
gRPC --> Span4[gRPC Span]
Span3 --> Aggregate[Collect Spans]
Span4 --> Aggregate
Aggregate --> Export[Export to Collector]
Export --> Collector[OpenTelemetry Collector]
Collector --> Backend[Backend Storage]
style Trace fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
style Aggregate fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
```
### Metrics Data Flow
How metrics are collected and exported.
```mermaid
sequenceDiagram
participant Service
participant MetricsRegistry
participant Exporter
participant Prometheus
participant Grafana
Service->>Service: Business operation
Service->>MetricsRegistry: Increment counter
Service->>MetricsRegistry: Record duration
Service->>MetricsRegistry: Set gauge
MetricsRegistry->>MetricsRegistry: Aggregate metrics
Prometheus->>Exporter: Scrape metrics
Exporter->>MetricsRegistry: Get metrics
MetricsRegistry-->>Exporter: Metrics data
Exporter-->>Prometheus: Prometheus format
Prometheus->>Prometheus: Store metrics
Grafana->>Prometheus: Query metrics
Prometheus-->>Grafana: Metrics data
Grafana->>Grafana: Render dashboard
```
### Log Data Flow
How logs flow through the system to various sinks.
```mermaid
graph TD
Service[Service] --> Logger[Logger]
Logger --> Format[Format Log]
Format --> Output[Output Log]
Output --> Stdout[stdout]
Output --> File[File]
Output --> LogCollector[Log Collector]
LogCollector --> Elasticsearch[Elasticsearch]
LogCollector --> CloudLogging[Cloud Logging]
Stdout --> Container[Container Logs]
style Logger fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
style LogCollector fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
```
## Audit Data Flow
How audit logs flow through the system.
```mermaid
sequenceDiagram
participant Service
participant AuditClient
participant AuditService
participant DB
participant Archive
Service->>Service: Security-sensitive action
Service->>AuditClient: Record audit log
AuditClient->>AuditClient: Build audit entry:
- actor
- action
- target
- metadata
- timestamp
AuditClient->>AuditService: Store audit log
AuditService->>AuditService: Validate entry
AuditService->>AuditService: Ensure immutability
AuditService->>DB: Insert audit log
DB-->>AuditService: Log stored
AuditService->>Archive: Archive old logs
Archive->>Archive: Long-term storage
Note over Service,Archive: Audit logs are immutable
```
## Cross-Service Data Flow
### Inter-Service Request Flow
How data flows when services communicate via service clients.
```mermaid
sequenceDiagram
participant ServiceA
participant ServiceClient
participant ServiceRegistry
participant ServiceB
participant DB
ServiceA->>ServiceClient: Call service method
ServiceClient->>ServiceRegistry: Discover service
ServiceRegistry-->>ServiceClient: Service endpoint
ServiceClient->>ServiceB: gRPC request
ServiceB->>ServiceB: Process request
ServiceB->>DB: Query data
DB-->>ServiceB: Data
ServiceB->>ServiceB: Business logic
ServiceB-->>ServiceClient: gRPC response
ServiceClient-->>ServiceA: Return data
```
### Service-to-Service Event Flow
How events flow between services.
```mermaid
graph LR
ServiceA[Service A] -->|Publish| EventBus[Event Bus]
EventBus -->|Route| ServiceB[Service B]
EventBus -->|Route| ServiceC[Service C]
ServiceB -->|Publish| EventBus
EventBus -->|Route| ServiceD[Service D]
ServiceC -->|Publish| EventBus
EventBus -->|Route| ServiceE[Service E]
style EventBus fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
```
## Data Flow Patterns Summary
### Request Flow Pattern
- **Path**: Client → HTTP → Handler → Service → Repository → Database
- **Response**: Database → Repository → Service → Handler → HTTP → Client
- **Side Effects**: Cache updates, event publishing, audit logging, metrics
### Event Flow Pattern
- **Path**: Publisher → Event Bus → Kafka → Subscribers
- **Characteristics**: Asynchronous, eventual consistency, decoupled
### Cache Flow Pattern
- **Read**: Cache → (miss) → Database → Cache
- **Write**: Service → Database → Cache invalidation
- **Characteristics**: Performance optimization, cache-aside pattern
### Observability Flow Pattern
- **Tracing**: Service → OpenTelemetry → Collector → Backend
- **Metrics**: Service → Metrics Registry → Prometheus → Grafana
- **Logs**: Service → Logger → Collector → Storage
## Integration Points
This data flow patterns document integrates with:
- **[System Behavior Overview](system-behavior.md)**: How data flows fit into system behavior
- **[Service Orchestration](service-orchestration.md)**: How data flows between services
- **[Module Integration Patterns](module-integration-patterns.md)**: How data flows through modules
- **[Operational Scenarios](operational-scenarios.md)**: Data flow in specific scenarios
- **[Component Relationships](component-relationships.md)**: Component-level data flow
## Related Documentation
- [System Behavior Overview](system-behavior.md) - System-level behavior
- [Service Orchestration](service-orchestration.md) - Service coordination
- [Module Integration Patterns](module-integration-patterns.md) - Module integration
- [Operational Scenarios](operational-scenarios.md) - Operational flows
- [Component Relationships](component-relationships.md) - Component dependencies
- [Architecture Overview](architecture.md) - System architecture

View File

@@ -0,0 +1,406 @@
# Module Integration Patterns
## Purpose
This document explains how modules integrate with the core platform, focusing on module discovery, initialization, service integration, and communication patterns rather than detailed implementation.
## Overview
Modules are implemented as independent services that extend the platform's functionality. They integrate with core services through service client interfaces, service discovery (Consul), and a standardized initialization process. Each module operates as an independent service with its own entry point, database connection, and deployment while leveraging core platform capabilities.
## Key Concepts
- **Module**: Independent service providing specific functionality
- **Module Manifest**: YAML file defining module metadata and configuration
- **Module Interface**: Standard interface all modules implement
- **Service Clients**: Abstraction for inter-service communication
- **Module Registry**: Registry tracking all loaded modules
## Module Discovery Process
Modules are discovered automatically during application startup by scanning module directories.
```mermaid
sequenceDiagram
participant Main
participant ModuleLoader
participant FileSystem
participant ModuleManifest
participant ModuleRegistry
Main->>ModuleLoader: DiscoverModules()
ModuleLoader->>FileSystem: Scan modules/ directory
FileSystem-->>ModuleLoader: Module directories
loop For each module directory
ModuleLoader->>FileSystem: Read module.yaml
FileSystem-->>ModuleLoader: Module manifest
ModuleLoader->>ModuleManifest: Parse manifest
ModuleManifest-->>ModuleLoader: Module metadata
ModuleLoader->>ModuleRegistry: Register module
ModuleRegistry->>ModuleRegistry: Validate manifest
ModuleRegistry->>ModuleRegistry: Check dependencies
ModuleRegistry-->>ModuleLoader: Module registered
end
ModuleLoader->>ModuleRegistry: Resolve dependencies
ModuleRegistry->>ModuleRegistry: Build dependency graph
ModuleRegistry->>ModuleRegistry: Order modules
ModuleRegistry-->>ModuleLoader: Ordered module list
ModuleLoader-->>Main: Module list ready
```
### Discovery Steps
1. **Directory Scanning**: Scan `modules/` directory for module subdirectories
2. **Manifest Loading**: Load `module.yaml` from each module directory
3. **Manifest Parsing**: Parse manifest to extract metadata
4. **Dependency Extraction**: Extract module dependencies from manifest
5. **Module Registration**: Register module in module registry
6. **Dependency Resolution**: Build dependency graph and order modules
7. **Validation**: Validate all dependencies are available
## Module Initialization Flow
Modules are initialized in dependency order, ensuring all dependencies are available before module initialization.
```mermaid
sequenceDiagram
participant Main
participant ModuleRegistry
participant Module
participant DI
participant Router
participant ServiceRegistry
participant DB
Main->>ModuleRegistry: GetOrderedModules()
ModuleRegistry-->>Main: Ordered module list
loop For each module (dependency order)
Main->>Module: Init()
Module->>DI: Provide services
DI->>DI: Register module services
DI-->>Module: Services registered
Module->>Router: Register routes
Router->>Router: Add route handlers
Router-->>Module: Routes registered
Module->>DB: Register migrations
DB->>DB: Store migration info
DB-->>Module: Migrations registered
Module->>ServiceRegistry: Register service
ServiceRegistry->>ServiceRegistry: Register with Consul
ServiceRegistry-->>Module: Service registered
Module->>Module: OnStart hook (optional)
Module-->>Main: Module initialized
end
Main->>DB: Run migrations
DB->>DB: Execute in dependency order
DB-->>Main: Migrations complete
Main->>Router: Start HTTP server
Main->>ServiceRegistry: Start service discovery
```
### Initialization Phases
1. **Dependency Resolution**: Determine module initialization order
2. **Service Registration**: Register module services in DI container
3. **Route Registration**: Register HTTP routes
4. **Migration Registration**: Register database migrations
5. **Service Registration**: Register module as service in service registry
6. **Lifecycle Hooks**: Execute OnStart hooks if defined
7. **Migration Execution**: Run migrations in dependency order
8. **Server Startup**: Start HTTP and gRPC servers
## Module Service Integration
Modules integrate with core services through service client interfaces, ensuring all communication goes through well-defined abstractions.
```mermaid
graph TB
subgraph "Module Service"
ModuleHandler[Module Handler]
ModuleService[Module Service]
ModuleRepo[Module Repository]
end
subgraph "Service Clients"
AuthClient[Auth Service Client]
IdentityClient[Identity Service Client]
AuthzClient[Authz Service Client]
AuditClient[Audit Service Client]
end
subgraph "Core Services"
AuthService[Auth Service<br/>:8081]
IdentityService[Identity Service<br/>:8082]
AuthzService[Authz Service<br/>:8083]
AuditService[Audit Service<br/>:8084]
end
subgraph "Infrastructure"
EventBus[Event Bus]
Cache[Cache]
DB[(Database)]
end
ModuleHandler --> ModuleService
ModuleService --> ModuleRepo
ModuleRepo --> DB
ModuleService -->|gRPC| AuthClient
ModuleService -->|gRPC| IdentityClient
ModuleService -->|gRPC| AuthzClient
ModuleService -->|gRPC| AuditClient
AuthClient --> AuthService
IdentityClient --> IdentityService
AuthzClient --> AuthzService
AuditClient --> AuditService
ModuleService --> EventBus
ModuleService --> Cache
style ModuleService fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
style AuthClient fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
```
### Service Integration Points
1. **Authentication**: Use Auth Service Client for token validation
2. **Identity**: Use Identity Service Client for user operations
3. **Authorization**: Use Authz Service Client for permission checks
4. **Audit**: Use Audit Service Client for audit logging
5. **Event Bus**: Publish and subscribe to events
6. **Cache**: Use cache for performance optimization
7. **Database**: Direct database access via repositories
## Module Data Management
Modules manage their own data while sharing database infrastructure.
```mermaid
graph TD
subgraph "Module A"
ModuleA[Module A Service]
RepoA[Module A Repository]
SchemaA[Module A Schema<br/>blog_posts]
end
subgraph "Module B"
ModuleB[Module B Service]
RepoB[Module B Repository]
SchemaB[Module B Schema<br/>billing_subscriptions]
end
subgraph "Shared Database"
DB[(PostgreSQL)]
end
subgraph "Migrations"
MigrationA[Module A Migrations]
MigrationB[Module B Migrations]
end
ModuleA --> RepoA
RepoA --> SchemaA
SchemaA --> DB
ModuleB --> RepoB
RepoB --> SchemaB
SchemaB --> DB
MigrationA --> DB
MigrationB --> DB
style ModuleA fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
style ModuleB fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
style DB fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
```
### Data Isolation Patterns
1. **Schema Isolation**: Each module has its own database schema
2. **Table Prefixing**: Module tables prefixed with module name
3. **Migration Isolation**: Each module manages its own migrations
4. **Shared Database**: Modules share database instance but not schemas
5. **Cross-Module Queries**: Use service clients, not direct SQL joins
## Module Permission System
Modules register permissions that are automatically integrated into the platform's permission system.
```mermaid
sequenceDiagram
participant Module
participant ModuleManifest
participant PermissionGenerator
participant PermissionRegistry
participant AuthzService
Module->>ModuleManifest: Define permissions
ModuleManifest->>ModuleManifest: permissions: blog.post.create, blog.post.read, blog.post.update, blog.post.delete
Module->>PermissionGenerator: Generate permission code
PermissionGenerator->>PermissionGenerator: Parse manifest
PermissionGenerator->>PermissionGenerator: Generate constants
PermissionGenerator-->>Module: Permission code generated
Module->>PermissionRegistry: Register permissions
PermissionRegistry->>PermissionRegistry: Validate format
PermissionRegistry->>PermissionRegistry: Store permissions
PermissionRegistry-->>Module: Permissions registered
AuthzService->>PermissionRegistry: Resolve permissions
PermissionRegistry-->>AuthzService: Permission list
AuthzService->>AuthzService: Check permissions
```
### Permission Registration Flow
1. **Permission Definition**: Define permissions in `module.yaml`
2. **Code Generation**: Generate permission constants from manifest
3. **Permission Registration**: Register permissions during module initialization
4. **Permission Validation**: Validate permission format and uniqueness
5. **Permission Resolution**: Permissions available for authorization checks
## Module Communication Patterns
Modules communicate with each other through event bus and service clients.
```mermaid
graph TB
subgraph "Module A"
ServiceA[Module A Service]
end
subgraph "Module B"
ServiceB[Module B Service]
end
subgraph "Event Bus"
EventBus[Event Bus<br/>Kafka]
end
subgraph "Service Clients"
ClientA[Service Client A]
ClientB[Service Client B]
end
subgraph "Module C"
ServiceC[Module C Service]
end
ServiceA -->|Publish Event| EventBus
EventBus -->|Subscribe| ServiceB
EventBus -->|Subscribe| ServiceC
ServiceA -->|gRPC Call| ClientA
ClientA --> ServiceB
ServiceB -->|gRPC Call| ClientB
ClientB --> ServiceC
style ServiceA fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
style ServiceB fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
style EventBus fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
```
### Communication Patterns
#### Event-Based Communication
```mermaid
sequenceDiagram
participant ModuleA
participant EventBus
participant ModuleB
participant ModuleC
ModuleA->>EventBus: Publish event
EventBus->>EventBus: Route to subscribers
EventBus->>ModuleB: Deliver event
EventBus->>ModuleC: Deliver event
ModuleB->>ModuleB: Process event
ModuleC->>ModuleC: Process event
Note over ModuleB,ModuleC: Events processed independently
```
#### Service Client Communication
```mermaid
sequenceDiagram
participant ModuleA
participant Client
participant ServiceRegistry
participant ModuleB
ModuleA->>Client: Call service method
Client->>ServiceRegistry: Discover Module B
ServiceRegistry-->>Client: Module B endpoint
Client->>ModuleB: gRPC call
ModuleB->>ModuleB: Process request
ModuleB-->>Client: Response
Client-->>ModuleA: Return result
```
## Module Route Registration
Modules register their HTTP routes with the platform's router.
```mermaid
sequenceDiagram
participant Module
participant Router
participant AuthzMiddleware
participant ModuleHandler
Module->>Router: Register routes
Module->>Router: Define route: /api/v1/blog/posts
Module->>Router: Define permission: blog.post.create
Module->>Router: Define handler: CreatePostHandler
Router->>Router: Create route
Router->>AuthzMiddleware: Register permission check
Router->>Router: Attach handler
Router->>Router: Route registered
Note over Router: Routes are registered with<br/>permission requirements
```
### Route Registration Process
1. **Route Definition**: Module defines routes in `Init()` method
2. **Permission Association**: Routes associated with required permissions
3. **Handler Registration**: Handlers registered with router
4. **Middleware Attachment**: Authorization middleware automatically attached
5. **Route Activation**: Routes available when HTTP server starts
## Integration Points
This module integration integrates with:
- **[System Behavior Overview](system-behavior.md)**: How modules participate in system bootstrap
- **[Service Orchestration](service-orchestration.md)**: How modules operate as services
- **[Operational Scenarios](operational-scenarios.md)**: Module behavior in specific scenarios
- **[Architecture Modules](architecture-modules.md)**: Detailed module architecture
## Related Documentation
- [System Behavior Overview](system-behavior.md) - System-level behavior
- [Service Orchestration](service-orchestration.md) - Service coordination
- [Operational Scenarios](operational-scenarios.md) - Module usage scenarios
- [Architecture Modules](architecture-modules.md) - Module architecture details
- [Module Requirements](module-requirements.md) - Module requirements and interfaces

View File

@@ -0,0 +1,816 @@
# Module Requirements
This document provides detailed requirements for each module in the Go Platform, including interfaces, responsibilities, and integration points.
## Table of Contents
- [Core Kernel Modules](#core-kernel-modules)
- [Security Modules](#security-modules)
- [Infrastructure Modules](#infrastructure-modules)
- [Feature Modules](#feature-modules)
## Core Kernel Modules
### Configuration Module
**Purpose**: Hierarchical configuration management with support for multiple sources.
**Requirements**:
- Load configuration from YAML files (default, environment-specific)
- Support environment variable overrides
- Support secret manager integration (AWS Secrets Manager, Vault)
- Type-safe configuration access
- Configuration validation
**Interface**:
```go
type ConfigProvider interface {
Get(key string) any
Unmarshal(v any) error
GetString(key string) string
GetInt(key string) int
GetBool(key string) bool
GetStringSlice(key string) []string
GetDuration(key string) time.Duration
IsSet(key string) bool
}
```
**Implementation**:
- Uses `github.com/spf13/viper` for configuration loading
- Load order: `default.yaml``{env}.yaml` → environment variables → secrets
- Supports nested configuration keys (e.g., `server.port`)
**Configuration Schema**:
```yaml
environment: development
server:
port: 8080
host: "0.0.0.0"
timeout: 30s
database:
driver: "postgres"
dsn: ""
max_connections: 25
max_idle_connections: 5
logging:
level: "info"
format: "json"
output: "stdout"
cache:
enabled: true
ttl: 5m
```
**Dependencies**: None (foundation module)
---
### Logging Module
**Purpose**: Structured logging with support for multiple outputs and log levels.
**Requirements**:
- Structured JSON logging for production
- Human-readable logging for development
- Support for log levels (debug, info, warn, error)
- Request-scoped fields (request_id, user_id, trace_id)
- Contextual logging (with fields)
- Performance: minimal overhead
**Interface**:
```go
type Field interface{}
type Logger interface {
Debug(msg string, fields ...Field)
Info(msg string, fields ...Field)
Warn(msg string, fields ...Field)
Error(msg string, fields ...Field)
Fatal(msg string, fields ...Field)
With(fields ...Field) Logger
}
// Helper functions
func String(key, value string) Field
func Int(key string, value int) Field
func Error(err error) Field
func Duration(key string, value time.Duration) Field
```
**Implementation**:
- Uses `go.uber.org/zap` for high-performance logging
- JSON encoder for production, console encoder for development
- Global logger instance accessible via `pkg/logger`
- Request-scoped logger via context
**Example Usage**:
```go
logger.Info("User logged in",
logger.String("user_id", userID),
logger.String("ip", ipAddress),
logger.Duration("duration", duration),
)
```
**Dependencies**: Configuration Module
---
### Dependency Injection Module
**Purpose**: Service registration and lifecycle management.
**Requirements**:
- Service registration via constructors
- Lifecycle management (OnStart/OnStop hooks)
- Dependency resolution
- Service overrides for testing
- Module-based service composition
**Implementation**:
- Uses `go.uber.org/fx` for dependency injection
- Core services registered in `internal/di/core_module.go`
- Modules register services via `fx.Provide()` in `Init()`
- Lifecycle hooks via `fx.Lifecycle`
**Core Module Structure**:
```go
var CoreModule = fx.Options(
fx.Provide(ProvideConfig),
fx.Provide(ProvideLogger),
fx.Provide(ProvideDatabase),
fx.Provide(ProvideHealthCheckers),
fx.Provide(ProvideMetrics),
fx.Provide(ProvideErrorBus),
fx.Provide(ProvideEventBus),
// ... other core services
)
```
**Dependencies**: Configuration Module, Logging Module
---
### Health & Metrics Module
**Purpose**: Health checks and metrics collection.
**Requirements**:
- Liveness endpoint (`/healthz`)
- Readiness endpoint (`/ready`)
- Metrics endpoint (`/metrics`) in Prometheus format
- Composable health checkers
- Custom metrics support
**Interface**:
```go
type HealthChecker interface {
Check(ctx context.Context) error
}
type HealthRegistry interface {
Register(name string, checker HealthChecker)
Check(ctx context.Context) map[string]error
}
```
**Core Health Checkers**:
- Database connectivity
- Redis connectivity
- Kafka connectivity (if enabled)
- Disk space
- Memory usage
**Metrics**:
- HTTP request duration (histogram)
- HTTP request count (counter)
- Database query duration (histogram)
- Cache hit/miss ratio (gauge)
- Error count (counter)
**Dependencies**: Configuration Module, Logging Module
---
### Error Bus Module
**Purpose**: Centralized error handling and reporting.
**Requirements**:
- Non-blocking error publishing
- Multiple error sinks (logger, Sentry)
- Error context preservation
- Panic recovery integration
**Interface**:
```go
type ErrorPublisher interface {
Publish(err error)
PublishWithContext(ctx context.Context, err error)
}
```
**Implementation**:
- Channel-based error bus
- Background goroutine consumes errors
- Pluggable sinks (logger, Sentry)
- Context extraction (user_id, trace_id, module)
**Dependencies**: Logging Module
---
## Security Modules
### Authentication Module
**Purpose**: User authentication via JWT tokens.
**Requirements**:
- JWT access token generation (short-lived, 15 minutes)
- JWT refresh token generation (long-lived, 7 days)
- Token validation and verification
- Token claims extraction
- Refresh token storage and revocation
**Interface**:
```go
type Authenticator interface {
GenerateAccessToken(userID string, roles []string, tenantID string) (string, error)
GenerateRefreshToken(userID string) (string, error)
VerifyToken(token string) (*TokenClaims, error)
RevokeRefreshToken(tokenHash string) error
}
type TokenClaims struct {
UserID string
Roles []string
TenantID string
ExpiresAt time.Time
IssuedAt time.Time
}
```
**Token Format**:
- Algorithm: HS256 or RS256
- Claims: `sub` (user ID), `roles`, `tenant_id`, `exp`, `iat`
- Refresh tokens stored in database with hash
**Endpoints**:
- `POST /api/v1/auth/login` - Authenticate and get tokens
- `POST /api/v1/auth/refresh` - Refresh access token
- `POST /api/v1/auth/logout` - Revoke refresh token
**Dependencies**: Identity Module, Configuration Module
---
### Authorization Module
**Purpose**: Role-based and attribute-based access control.
**Requirements**:
- Permission-based authorization
- Role-to-permission mapping
- User-to-role assignment
- Permission caching
- Context-aware authorization
**Interface**:
```go
type PermissionResolver interface {
HasPermission(ctx context.Context, userID string, perm Permission) (bool, error)
GetUserPermissions(ctx context.Context, userID string) ([]Permission, error)
}
type Authorizer interface {
Authorize(ctx context.Context, perm Permission) error
}
```
**Permission Format**:
- String format: `"{module}.{resource}.{action}"`
- Examples: `blog.post.create`, `user.read`, `system.health.check`
- Code-generated constants for type safety
**Authorization Flow**:
```mermaid
sequenceDiagram
participant Request
participant AuthzMiddleware
participant Authorizer
participant PermissionResolver
participant Cache
participant DB
Request->>AuthzMiddleware: HTTP request with permission
AuthzMiddleware->>Authorizer: Authorize(ctx, permission)
Authorizer->>Authorizer: Extract user from context
Authorizer->>PermissionResolver: HasPermission(user, permission)
PermissionResolver->>Cache: Check cache
Cache-->>PermissionResolver: Cache miss
PermissionResolver->>DB: Load user roles
PermissionResolver->>DB: Load role permissions
DB-->>PermissionResolver: Permissions
PermissionResolver->>Cache: Store in cache
PermissionResolver-->>Authorizer: Has permission: true/false
Authorizer-->>AuthzMiddleware: Authorized or error
AuthzMiddleware-->>Request: Continue or 403
```
**Dependencies**: Identity Module, Cache Module
---
### Identity Module
**Purpose**: User and role management.
**Requirements**:
- User CRUD operations
- Password hashing (argon2id)
- Email verification
- Password reset flow
- Role management
- Permission management
- User-role assignment
**Interfaces**:
```go
type UserRepository interface {
FindByID(ctx context.Context, id string) (*User, error)
FindByEmail(ctx context.Context, email string) (*User, error)
Create(ctx context.Context, u *User) error
Update(ctx context.Context, u *User) error
Delete(ctx context.Context, id string) error
List(ctx context.Context, filters UserFilters) ([]*User, error)
}
type UserService interface {
Register(ctx context.Context, email, password string) (*User, error)
VerifyEmail(ctx context.Context, token string) error
ResetPassword(ctx context.Context, email string) error
ChangePassword(ctx context.Context, userID, oldPassword, newPassword string) error
UpdateProfile(ctx context.Context, userID string, updates UserUpdates) error
}
type RoleRepository interface {
FindByID(ctx context.Context, id string) (*Role, error)
Create(ctx context.Context, r *Role) error
Update(ctx context.Context, r *Role) error
Delete(ctx context.Context, id string) error
AssignPermissions(ctx context.Context, roleID string, permissions []Permission) error
AssignToUser(ctx context.Context, userID string, roleIDs []string) error
}
```
**User Entity**:
- ID (UUID)
- Email (unique, verified)
- Password hash (argon2id)
- Email verified (boolean)
- Created at, updated at
- Tenant ID (optional, for multi-tenancy)
**Role Entity**:
- ID (UUID)
- Name (unique)
- Description
- Created at
- Permissions (many-to-many)
**Dependencies**: Database Module, Notification Module, Cache Module
---
### Audit Module
**Purpose**: Immutable audit logging of security-relevant actions.
**Requirements**:
- Append-only audit log
- Actor tracking (user ID)
- Action tracking (what was done)
- Target tracking (what was affected)
- Metadata storage (JSON)
- Correlation IDs
- High-performance writes
**Interface**:
```go
type Auditor interface {
Record(ctx context.Context, action AuditAction) error
Query(ctx context.Context, filters AuditFilters) ([]AuditEntry, error)
}
type AuditAction struct {
ActorID string
Action string // e.g., "user.created", "role.assigned"
TargetID string
Metadata map[string]any
IPAddress string
UserAgent string
}
```
**Audit Log Schema**:
- ID (UUID)
- Actor ID (user ID)
- Action (string)
- Target ID (resource ID)
- Metadata (JSONB)
- Timestamp
- Request ID
- IP Address
- User Agent
**Automatic Audit Events**:
- User login/logout
- Password changes
- Role assignments
- Permission grants
- Data modifications (configurable)
**Dependencies**: Database Module, Logging Module
---
## Infrastructure Modules
### Database Module
**Purpose**: Database access and ORM functionality.
**Requirements**:
- PostgreSQL support (primary)
- Connection pooling
- Transaction support
- Migration management
- Query instrumentation (OpenTelemetry)
- Multi-tenancy support (tenant_id filtering)
**Implementation**:
- Uses `entgo.io/ent` for code generation
- Ent schemas for all entities
- Migration runner on startup
- Connection pool configuration
**Database Client Interface**:
```go
type DatabaseClient interface {
Client() *ent.Client
Migrate(ctx context.Context) error
Close() error
HealthCheck(ctx context.Context) error
}
```
**Connection Pooling**:
- Max connections: 25
- Max idle connections: 5
- Connection lifetime: 5 minutes
- Idle timeout: 10 minutes
**Multi-Tenancy**:
- Automatic tenant_id filtering via Ent interceptors
- Tenant-aware queries
- Tenant isolation at application level
**Dependencies**: Configuration Module, Logging Module
---
### Cache Module
**Purpose**: Distributed caching with Redis.
**Requirements**:
- Key-value storage
- TTL support
- Distributed caching (shared across instances)
- Cache invalidation
- Fallback to in-memory cache
**Interface**:
```go
type Cache interface {
Get(ctx context.Context, key string) ([]byte, error)
Set(ctx context.Context, key string, value []byte, ttl time.Duration) error
Delete(ctx context.Context, key string) error
Exists(ctx context.Context, key string) (bool, error)
Increment(ctx context.Context, key string) (int64, error)
}
```
**Use Cases**:
- User permissions caching
- Role assignments caching
- Session data
- Rate limiting state
- Query result caching (optional)
**Cache Key Format**:
- `user:{user_id}:permissions`
- `role:{role_id}:permissions`
- `session:{session_id}`
- `ratelimit:{user_id}:{endpoint}`
**Dependencies**: Configuration Module, Logging Module
---
### Event Bus Module
**Purpose**: Event-driven communication between modules.
**Requirements**:
- Publish/subscribe pattern
- Topic-based routing
- In-process bus (development)
- Kafka bus (production)
- Error handling and retries
- Event ordering (per partition)
**Interface**:
```go
type EventBus interface {
Publish(ctx context.Context, topic string, event Event) error
Subscribe(topic string, handler EventHandler) error
Unsubscribe(topic string) error
}
type Event struct {
ID string
Type string
Source string
Timestamp time.Time
Data map[string]any
}
type EventHandler func(ctx context.Context, event Event) error
```
**Core Events**:
- `platform.user.created`
- `platform.user.updated`
- `platform.user.deleted`
- `platform.role.assigned`
- `platform.role.revoked`
- `platform.permission.granted`
**Event Flow**:
```mermaid
graph LR
Publisher[Module Publisher]
Bus[Event Bus]
Subscriber1[Module Subscriber 1]
Subscriber2[Module Subscriber 2]
Subscriber3[Module Subscriber 3]
Publisher -->|Publish| Bus
Bus -->|Deliver| Subscriber1
Bus -->|Deliver| Subscriber2
Bus -->|Deliver| Subscriber3
```
**Dependencies**: Configuration Module, Logging Module
---
### Scheduler Module
**Purpose**: Background job processing and cron scheduling.
**Requirements**:
- Cron job scheduling
- Async job queuing
- Job retries with backoff
- Job status tracking
- Concurrency control
- Job persistence
**Interface**:
```go
type Scheduler interface {
Cron(spec string, job JobFunc) error
Enqueue(queue string, payload any) error
EnqueueWithRetry(queue string, payload any, retries int) error
}
type JobFunc func(ctx context.Context) error
```
**Implementation**:
- Uses `github.com/robfig/cron/v3` for cron jobs
- Uses `github.com/hibiken/asynq` for job queuing
- Redis-backed job queue
- Job processor with worker pool
**Example Jobs**:
- Cleanup expired tokens (daily)
- Send digest emails (weekly)
- Generate reports (monthly)
- Data archival (custom schedule)
**Dependencies**: Cache Module (Redis), Logging Module
---
### Blob Storage Module
**Purpose**: File and blob storage abstraction.
**Requirements**:
- File upload
- File download
- File deletion
- Signed URL generation
- Versioning support (optional)
**Interface**:
```go
type BlobStore interface {
Upload(ctx context.Context, key string, data []byte, contentType string) error
Download(ctx context.Context, key string) ([]byte, error)
Delete(ctx context.Context, key string) error
GetSignedURL(ctx context.Context, key string, ttl time.Duration) (string, error)
Exists(ctx context.Context, key string) (bool, error)
}
```
**Implementation**:
- AWS S3 adapter (primary)
- Local file system adapter (development)
- GCS adapter (optional)
**Key Format**:
- `{module}/{resource_type}/{resource_id}/{filename}`
- Example: `blog/posts/abc123/image.jpg`
**Dependencies**: Configuration Module, Logging Module
---
### Notification Module
**Purpose**: Multi-channel notifications (email, SMS, push).
**Requirements**:
- Email sending (SMTP, AWS SES)
- SMS sending (Twilio, optional)
- Push notifications (FCM, APNs, optional)
- Webhook notifications
- Template support
- Retry logic
**Interface**:
```go
type Notifier interface {
SendEmail(ctx context.Context, to, subject, body string) error
SendEmailWithTemplate(ctx context.Context, to, template string, data map[string]any) error
SendSMS(ctx context.Context, to, message string) error
SendPush(ctx context.Context, deviceToken string, payload PushPayload) error
SendWebhook(ctx context.Context, url string, payload map[string]any) error
}
```
**Email Templates**:
- Email verification
- Password reset
- Welcome email
- Notification digest
**Dependencies**: Configuration Module, Logging Module, Event Bus Module
---
## Feature Modules
### Blog Module (Example)
**Purpose**: Blog post management functionality.
**Requirements**:
- Post CRUD operations
- Comment system (optional)
- Author-based access control
- Post publishing workflow
- Tag/category support
**Permissions**:
- `blog.post.create`
- `blog.post.read`
- `blog.post.update`
- `blog.post.delete`
- `blog.post.publish`
**Routes**:
- `POST /api/v1/blog/posts` - Create post
- `GET /api/v1/blog/posts` - List posts
- `GET /api/v1/blog/posts/:id` - Get post
- `PUT /api/v1/blog/posts/:id` - Update post
- `DELETE /api/v1/blog/posts/:id` - Delete post
**Domain Model**:
```go
type Post struct {
ID string
Title string
Content string
AuthorID string
Status PostStatus // draft, published, archived
CreatedAt time.Time
UpdatedAt time.Time
PublishedAt *time.Time
}
```
**Events Published**:
- `blog.post.created`
- `blog.post.updated`
- `blog.post.published`
- `blog.post.deleted`
**Dependencies**: Core Kernel, Identity Module, Event Bus Module
---
## Module Integration Matrix
```mermaid
graph TB
subgraph "Core Kernel (Required)"
Config[Config]
Logger[Logger]
DI[DI Container]
Health[Health]
end
subgraph "Security (Required)"
Auth[Auth]
Authz[Authz]
Identity[Identity]
Audit[Audit]
end
subgraph "Infrastructure (Optional)"
DB[Database]
Cache[Cache]
EventBus[Event Bus]
Scheduler[Scheduler]
BlobStore[Blob Store]
Notifier[Notifier]
end
subgraph "Feature Modules"
Blog[Blog]
Billing[Billing]
Custom[Custom Modules]
end
Config --> Logger
Config --> DI
DI --> Health
DI --> Auth
DI --> Authz
DI --> Identity
DI --> Audit
DI --> DB
DI --> Cache
DI --> EventBus
DI --> Scheduler
DI --> BlobStore
DI --> Notifier
Auth --> Identity
Authz --> Identity
Authz --> Audit
Blog --> Auth
Blog --> Authz
Blog --> DB
Blog --> EventBus
Blog --> Cache
Billing --> Auth
Billing --> Authz
Billing --> DB
Billing --> EventBus
Billing --> Cache
Custom --> Auth
Custom --> Authz
Custom --> DB
style Config fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style Auth fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style Blog fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
```
## Next Steps
- [Component Relationships](./component-relationships.md) - Detailed component interactions
- [System Architecture](./architecture.md) - Overall system architecture
- [Module Architecture](./architecture-modules.md) - Module design and integration

View File

@@ -0,0 +1,417 @@
# Operational Scenarios
## Purpose
This document describes common operational scenarios in the Go Platform, focusing on how different components interact to accomplish specific tasks.
## Overview
Operational scenarios illustrate how the platform handles common use cases such as user authentication, authorization checks, event processing, and background job execution. Each scenario shows the complete flow from initiation to completion.
## Key Concepts
- **Scenario**: A specific operational use case
- **Flow**: Sequence of interactions to accomplish the scenario
- **Components**: Services and modules involved in the scenario
- **State Changes**: How system state changes during the scenario
## Authentication and Authorization Flows
### User Authentication Flow
Complete flow of user logging in and receiving authentication tokens.
```mermaid
sequenceDiagram
participant User
participant Client
participant Gateway[API Gateway]
participant AuthService
participant IdentityService
participant DB
participant TokenProvider
participant AuditService
participant Registry[Consul]
User->>Client: Enter credentials
Client->>Gateway: POST /api/v1/auth/login
Gateway->>Gateway: Rate limiting check
Gateway->>AuthService: Login request (gRPC)
AuthService->>AuthService: Validate request format
AuthService->>Registry: Discover Identity Service
Registry-->>AuthService: Identity Service endpoint
AuthService->>IdentityService: Verify credentials (gRPC)
IdentityService->>DB: Query user by email
DB-->>IdentityService: User data
IdentityService->>IdentityService: Verify password hash
IdentityService-->>AuthService: Credentials valid
AuthService->>TokenProvider: Generate access token
TokenProvider->>TokenProvider: Create JWT claims
TokenProvider-->>AuthService: Access token
AuthService->>TokenProvider: Generate refresh token
TokenProvider->>DB: Store refresh token hash
DB-->>TokenProvider: Token stored
TokenProvider-->>AuthService: Refresh token
AuthService->>Registry: Discover Audit Service
Registry-->>AuthService: Audit Service endpoint
AuthService->>AuditService: Log login (gRPC)
AuditService->>DB: Store audit log
AuditService-->>AuthService: Logged
AuthService-->>Gateway: Access + Refresh tokens
Gateway-->>Client: Access + Refresh tokens
Client-->>User: Authentication successful
```
### Authorization Check Flow
How the system checks if a user has permission to perform an action.
```mermaid
sequenceDiagram
participant Gateway[API Gateway]
participant Handler
participant AuthzService
participant PermissionResolver
participant Cache
participant DB
participant IdentityService
participant Registry[Consul]
Gateway->>Handler: Request with user context
Handler->>AuthzService: Authorize(user, permission) (gRPC)
AuthzService->>Registry: Discover Identity Service
Registry-->>AuthzService: Identity Service endpoint
AuthzService->>Cache: Check permission cache
Cache-->>AuthzService: Cache miss
AuthzService->>PermissionResolver: Resolve permissions
PermissionResolver->>IdentityService: Get user roles (gRPC)
IdentityService->>DB: Query user roles
DB-->>IdentityService: User roles
IdentityService-->>PermissionResolver: Roles list
PermissionResolver->>DB: Query role permissions
DB-->>PermissionResolver: Permissions
PermissionResolver->>PermissionResolver: Aggregate permissions
PermissionResolver-->>AuthzService: User permissions
AuthzService->>AuthzService: Check permission in list
AuthzService->>Cache: Store in cache
AuthzService-->>Handler: Authorized/Unauthorized
alt Authorized
Handler-->>Gateway: Continue request
else Unauthorized
Handler-->>Gateway: 403 Forbidden
end
```
### Permission Resolution Flow
How user permissions are resolved from roles and cached for performance.
```mermaid
graph TD
Start[Permission Check] --> Cache{Cache Hit?}
Cache -->|Yes| Return[Return Cached Permissions]
Cache -->|No| GetRoles[Get User Roles]
GetRoles --> DB1[(Database)]
DB1 --> Roles[User Roles]
Roles --> GetPermissions[Get Role Permissions]
GetPermissions --> DB2[(Database)]
DB2 --> Permissions[Role Permissions]
Permissions --> Aggregate[Aggregate Permissions]
Aggregate --> StoreCache[Store in Cache]
StoreCache --> Return
Return --> Check[Check Permission]
Check -->|Has Permission| Allow[Allow Access]
Check -->|No Permission| Deny[Deny Access]
style Cache fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
style Aggregate fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
```
## Data Access Patterns
### Cache-Aside Pattern
How data is accessed with caching to improve performance.
```mermaid
sequenceDiagram
participant Service
participant Cache
participant Repository
participant DB
Service->>Cache: Get(key)
Cache-->>Service: Cache miss
Service->>Repository: Find by ID
Repository->>DB: Query database
DB-->>Repository: Data
Repository-->>Service: Domain entity
Service->>Cache: Set(key, entity)
Cache-->>Service: Cached
Service-->>Service: Return entity
Note over Service,Cache: Next request will hit cache
```
### Write-Through Cache Pattern
How data writes are synchronized with cache.
```mermaid
sequenceDiagram
participant Service
participant Cache
participant Repository
participant DB
Service->>Repository: Save entity
Repository->>DB: Insert/Update
DB-->>Repository: Success
Repository->>Cache: Invalidate(key)
Cache->>Cache: Remove from cache
Cache-->>Repository: Invalidated
Repository-->>Service: Entity saved
Note over Service,Cache: Cache invalidated on write
```
## Event Processing Scenarios
### Event Publishing and Consumption
How events are published and consumed across services.
```mermaid
sequenceDiagram
participant Publisher
participant EventBus
participant Kafka
participant Subscriber1
participant Subscriber2
Publisher->>Publisher: Business event occurs
Publisher->>EventBus: Publish(event)
EventBus->>EventBus: Serialize event
EventBus->>EventBus: Add metadata
EventBus->>Kafka: Send to topic
Kafka-->>EventBus: Acknowledged
EventBus-->>Publisher: Event published
Kafka->>Subscriber1: Deliver event
Subscriber1->>Subscriber1: Deserialize event
Subscriber1->>Subscriber1: Process event
Subscriber1->>Subscriber1: Update state
Subscriber1-->>Kafka: Acknowledge
Kafka->>Subscriber2: Deliver event
Subscriber2->>Subscriber2: Deserialize event
Subscriber2->>Subscriber2: Process event
Subscriber2->>Subscriber2: Update state
Subscriber2-->>Kafka: Acknowledge
```
### Event-Driven Workflow
How multiple services coordinate through events.
```mermaid
graph LR
A[Service A] -->|Publish Event A| EventBus[Event Bus]
EventBus -->|Subscribe| B[Service B]
EventBus -->|Subscribe| C[Service C]
B -->|Publish Event B| EventBus
C -->|Publish Event C| EventBus
EventBus -->|Subscribe| D[Service D]
style EventBus fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
```
## Background Processing Scenarios
### Background Job Scheduling
How background jobs are scheduled and executed.
```mermaid
sequenceDiagram
participant Scheduler
participant JobQueue
participant Worker
participant Service
participant DB
participant EventBus
Scheduler->>Scheduler: Cron trigger
Scheduler->>JobQueue: Enqueue job
JobQueue->>JobQueue: Store job definition
JobQueue-->>Scheduler: Job enqueued
Worker->>JobQueue: Poll for jobs
JobQueue-->>Worker: Job definition
Worker->>Worker: Lock job
Worker->>Service: Execute job
Service->>DB: Update data
Service->>EventBus: Publish events
Service-->>Worker: Job complete
Worker->>JobQueue: Mark complete
JobQueue->>JobQueue: Remove job
alt Job fails
Worker->>JobQueue: Mark failed
JobQueue->>JobQueue: Schedule retry
end
```
### Job Retry Flow
How failed jobs are retried with exponential backoff.
```mermaid
stateDiagram-v2
[*] --> Pending: Job created
Pending --> Running: Worker picks up
Running --> Success: Job completes
Running --> Failed: Job fails
Failed --> RetryScheduled: Schedule retry
RetryScheduled --> Waiting: Wait (exponential backoff)
Waiting --> Pending: Retry time reached
Failed --> MaxRetries: Max retries reached
MaxRetries --> DeadLetter: Move to dead letter
Success --> [*]
DeadLetter --> [*]
```
## Configuration Management Scenarios
### Configuration Reload Flow
How configuration is reloaded without service restart.
```mermaid
sequenceDiagram
participant Admin
participant ConfigService
participant ConfigManager
participant Services
participant SecretStore
Admin->>ConfigService: Update configuration
ConfigService->>SecretStore: Fetch secrets (if needed)
SecretStore-->>ConfigService: Secrets
ConfigService->>ConfigManager: Reload configuration
ConfigManager->>ConfigManager: Validate configuration
ConfigManager->>ConfigManager: Merge with defaults
ConfigManager->>Services: Notify config change
Services->>Services: Update configuration
Services-->>ConfigManager: Config updated
ConfigManager-->>ConfigService: Reload complete
ConfigService-->>Admin: Configuration reloaded
```
## Audit Logging Flow
How all security-sensitive actions are logged.
```mermaid
sequenceDiagram
participant Service
participant AuditClient
participant AuditService
participant DB
Service->>Service: Security-sensitive action
Service->>AuditClient: Record audit log
AuditClient->>AuditClient: Extract context
AuditClient->>AuditClient: Build audit entry
AuditClient->>AuditService: Store audit log
AuditService->>AuditService: Validate audit entry
AuditService->>DB: Insert audit log
DB-->>AuditService: Log stored
AuditService-->>AuditClient: Audit logged
AuditClient-->>Service: Continue
Note over Service,DB: Audit logs are immutable
```
## Database Migration Flow
How database migrations are executed during module initialization.
```mermaid
sequenceDiagram
participant Main
participant ModuleRegistry
participant Module
participant MigrationRunner
participant DB
Main->>ModuleRegistry: Get modules (dependency order)
ModuleRegistry-->>Main: Ordered modules
loop For each module
Main->>Module: Get migrations
Module-->>Main: Migration list
end
Main->>MigrationRunner: Run migrations
MigrationRunner->>DB: Check migration table
DB-->>MigrationRunner: Existing migrations
loop For each pending migration
MigrationRunner->>DB: Start transaction
MigrationRunner->>DB: Execute migration
DB-->>MigrationRunner: Migration complete
MigrationRunner->>DB: Record migration
MigrationRunner->>DB: Commit transaction
end
MigrationRunner-->>Main: Migrations complete
```
## Integration Points
This operational scenarios document integrates with:
- **[System Behavior Overview](system-behavior.md)**: How these scenarios fit into overall system behavior
- **[Service Orchestration](service-orchestration.md)**: How services coordinate in these scenarios
- **[Module Integration Patterns](module-integration-patterns.md)**: How modules participate in these scenarios
- **[Data Flow Patterns](data-flow-patterns.md)**: Detailed data flow in these scenarios
## Related Documentation
- [System Behavior Overview](system-behavior.md) - System-level behavior
- [Service Orchestration](service-orchestration.md) - Service coordination
- [Module Integration Patterns](module-integration-patterns.md) - Module integration
- [Data Flow Patterns](data-flow-patterns.md) - Data flow details
- [Architecture Overview](architecture.md) - System architecture

View File

@@ -0,0 +1,403 @@
# Service Orchestration
## Purpose
This document explains how services work together in the Go Platform's microservices architecture, focusing on service lifecycle management, discovery, communication patterns, and failure handling.
## Overview
The Go Platform consists of multiple independent services that communicate via service clients (gRPC/HTTP) and share infrastructure components. Services are discovered and registered through a service registry (Consul), enabling dynamic service location and health monitoring.
## Key Concepts
- **Service**: Independent process providing specific functionality
- **Service Registry**: Central registry for service discovery (Consul - primary, Kubernetes as alternative)
- **Service Client**: Abstraction for inter-service communication
- **Service Discovery**: Process of locating services by name
- **Service Health**: Health status of a service (healthy, unhealthy, degraded)
## Service Lifecycle Management
Services follow a well-defined lifecycle from startup to shutdown.
```mermaid
stateDiagram-v2
[*] --> Starting: Service starts
Starting --> Registering: Initialize services
Registering --> StartingServer: Register with service registry
StartingServer --> Running: Start HTTP/gRPC servers
Running --> Healthy: Health checks pass
Running --> Unhealthy: Health checks fail
Unhealthy --> Running: Health checks recover
Healthy --> Degrading: Dependency issues
Degrading --> Healthy: Dependencies recover
Degrading --> Unhealthy: Critical failure
Running --> ShuttingDown: Receive shutdown signal
ShuttingDown --> Deregistering: Stop accepting requests
Deregistering --> Stopped: Deregister from registry
Stopped --> [*]
```
### Lifecycle States
1. **Starting**: Service is initializing, loading configuration
2. **Registering**: Service registers with service registry
3. **Starting Server**: HTTP and gRPC servers starting
4. **Running**: Service is running and processing requests
5. **Healthy**: All health checks passing
6. **Unhealthy**: Health checks failing
7. **Degrading**: Service operational but with degraded functionality
8. **Shutting Down**: Service received shutdown signal
9. **Deregistering**: Service removing itself from registry
10. **Stopped**: Service has stopped
## Service Discovery and Registration
Services automatically register themselves with the service registry on startup and deregister on shutdown.
```mermaid
sequenceDiagram
participant Service
participant ServiceRegistry
participant Registry[Consul<br/>Service Registry]
participant Client
Service->>ServiceRegistry: Register(serviceInfo)
ServiceRegistry->>Registry: Register service
Registry->>Registry: Store service info
Registry-->>ServiceRegistry: Registration confirmed
ServiceRegistry-->>Service: Service registered
Note over Service: Service starts health checks
loop Every health check interval
Service->>ServiceRegistry: Update health status
ServiceRegistry->>Registry: Update health
end
Client->>ServiceRegistry: Discover(serviceName)
ServiceRegistry->>Registry: Query services
Registry-->>ServiceRegistry: Service list
ServiceRegistry->>ServiceRegistry: Filter healthy services
ServiceRegistry->>ServiceRegistry: Load balance
ServiceRegistry-->>Client: Service endpoint
Client->>Service: Connect via gRPC/HTTP
Service->>ServiceRegistry: Deregister()
ServiceRegistry->>Registry: Remove service
Registry-->>ServiceRegistry: Service removed
```
### Service Registration Process
1. **Service Startup**: Service initializes and loads configuration
2. **Service Info Creation**: Create service info with name, version, address, protocol
3. **Registry Registration**: Register service with Consul (primary) or Kubernetes service discovery (alternative)
4. **Health Check Setup**: Start health check endpoint
5. **Health Status Updates**: Periodically update health status in registry
6. **Service Discovery**: Clients query registry for service endpoints
7. **Load Balancing**: Registry returns healthy service instances
8. **Service Deregistration**: On shutdown, remove service from registry
## Service Communication Patterns
Services communicate through well-defined patterns using service clients.
```mermaid
graph TB
subgraph "Service A"
ServiceA[Service A Handler]
ClientA[Service Client]
end
subgraph "Service Registry"
Registry[Service Registry]
end
subgraph "Service B"
ServiceB[Service B Handler]
ServerB[gRPC Server]
end
subgraph "Service C"
ServiceC[Service C Handler]
end
subgraph "Event Bus"
EventBus[Event Bus<br/>Kafka]
end
ServiceA -->|Discover| Registry
Registry -->|Service B endpoint| ClientA
ClientA -->|gRPC Call| ServerB
ServerB --> ServiceB
ServiceB -->|Response| ClientA
ServiceA -->|Publish Event| EventBus
EventBus -->|Subscribe| ServiceC
ServiceC -->|Process Event| ServiceC
style ClientA fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style ServerB fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
style EventBus fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
```
### Communication Patterns
#### Synchronous Communication (gRPC/HTTP)
```mermaid
sequenceDiagram
participant Client
participant ServiceClient
participant Registry
participant Service
Client->>ServiceClient: Call service method
ServiceClient->>Registry: Discover service
Registry-->>ServiceClient: Service endpoint
ServiceClient->>Service: gRPC/HTTP call
Service->>Service: Process request
Service-->>ServiceClient: Response
ServiceClient-->>Client: Return result
alt Service unavailable
ServiceClient->>Registry: Retry discovery
Registry-->>ServiceClient: Alternative endpoint
ServiceClient->>Service: Retry call
end
```
#### Asynchronous Communication (Event Bus)
```mermaid
sequenceDiagram
participant Publisher
participant EventBus
participant Kafka
participant Subscriber1
participant Subscriber2
Publisher->>EventBus: Publish event
EventBus->>Kafka: Send to topic
Kafka-->>EventBus: Acknowledged
Kafka->>Subscriber1: Deliver event
Kafka->>Subscriber2: Deliver event
Subscriber1->>Subscriber1: Process event
Subscriber2->>Subscriber2: Process event
Note over Subscriber1,Subscriber2: Events processed independently
```
## Service Dependency Graph
Services have dependencies that determine startup ordering and communication patterns.
```mermaid
graph TD
subgraph "Core Services"
Identity[Identity Service]
Auth[Auth Service]
Authz[Authz Service]
Audit[Audit Service]
end
subgraph "Feature Services"
Blog[Blog Service]
Billing[Billing Service]
Analytics[Analytics Service]
end
subgraph "Infrastructure Services"
Registry[Service Registry]
EventBus[Event Bus]
Cache[Cache Service]
end
Auth --> Identity
Auth --> Registry
Authz --> Identity
Authz --> Cache
Authz --> Audit
Audit --> Registry
Blog --> Authz
Blog --> Identity
Blog --> Audit
Blog --> Registry
Blog --> EventBus
Blog --> Cache
Billing --> Authz
Billing --> Identity
Billing --> Registry
Billing --> EventBus
Analytics --> EventBus
Analytics --> Registry
style Identity fill:#4a90e2,stroke:#2e5c8a,stroke-width:3px,color:#fff
style Auth fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style Blog fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
```
### Dependency Types
1. **Hard Dependencies**: Service cannot start without dependency (e.g., Auth depends on Identity)
2. **Soft Dependencies**: Service can start but with degraded functionality
3. **Runtime Dependencies**: Dependencies discovered at runtime via service registry
## Service Health and Failure Handling
Services continuously report their health status, enabling automatic failure detection and recovery.
```mermaid
graph TD
Service[Service] --> HealthCheck[Health Check Endpoint]
HealthCheck --> CheckDB[Check Database]
HealthCheck --> CheckCache[Check Cache]
HealthCheck --> CheckDeps[Check Dependencies]
CheckDB -->|Healthy| Aggregate[Aggregate Health]
CheckCache -->|Healthy| Aggregate
CheckDeps -->|Healthy| Aggregate
Aggregate -->|All Healthy| Healthy[Healthy Status]
Aggregate -->|Degraded| Degraded[Degraded Status]
Aggregate -->|Unhealthy| Unhealthy[Unhealthy Status]
Healthy --> Registry[Update Registry]
Degraded --> Registry
Unhealthy --> Registry
Registry --> LoadBalancer[Load Balancer]
LoadBalancer -->|Healthy Only| RouteTraffic[Route Traffic]
LoadBalancer -->|Unhealthy| NoTraffic[No Traffic]
style Healthy fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style Degraded fill:#ffa500,stroke:#ff8c00,stroke-width:2px,color:#fff
style Unhealthy fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
```
### Health Check Types
1. **Liveness Check**: Service process is running
2. **Readiness Check**: Service is ready to accept requests
3. **Dependency Checks**: Database, cache, and other dependencies are accessible
4. **Business Health**: Service-specific health indicators
### Failure Handling Strategies
#### Circuit Breaker Pattern
```mermaid
stateDiagram-v2
[*] --> Closed: Service healthy
Closed --> Open: Failure threshold exceeded
Open --> HalfOpen: Timeout period
HalfOpen --> Closed: Success
HalfOpen --> Open: Failure
```
#### Retry Strategy
```mermaid
sequenceDiagram
participant Client
participant Service
Client->>Service: Request
Service-->>Client: Failure
Client->>Client: Wait (exponential backoff)
Client->>Service: Retry 1
Service-->>Client: Failure
Client->>Client: Wait (exponential backoff)
Client->>Service: Retry 2
Service-->>Client: Success
```
#### Service Degradation
When a service dependency fails, the service may continue operating with degraded functionality:
- **Cache Unavailable**: Service continues but without caching
- **Event Bus Unavailable**: Service continues but events are queued
- **Non-Critical Dependency Fails**: Service continues with reduced features
## Service Scaling Scenarios
Services can be scaled independently based on load and requirements.
```mermaid
graph TB
subgraph "Load Balancer"
LB[Load Balancer]
end
subgraph "Service Instances"
Instance1[Service Instance 1<br/>Healthy]
Instance2[Service Instance 2<br/>Healthy]
Instance3[Service Instance 3<br/>Starting]
Instance4[Service Instance 4<br/>Unhealthy]
end
subgraph "Service Registry"
Registry[Service Registry]
end
subgraph "Infrastructure"
DB[(Database)]
Cache[(Cache)]
end
LB -->|Discover| Registry
Registry -->|Healthy Instances| LB
LB --> Instance1
LB --> Instance2
LB -.->|No Traffic| Instance3
LB -.->|No Traffic| Instance4
Instance1 --> DB
Instance2 --> DB
Instance3 --> DB
Instance4 --> DB
Instance1 --> Cache
Instance2 --> Cache
style Instance1 fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style Instance2 fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style Instance3 fill:#ffa500,stroke:#ff8c00,stroke-width:2px,color:#fff
style Instance4 fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
```
### Scaling Patterns
1. **Horizontal Scaling**: Add more service instances
2. **Vertical Scaling**: Increase resources for existing instances
3. **Auto-Scaling**: Automatically scale based on metrics
4. **Load-Based Routing**: Route traffic to healthy instances only
## Integration Points
This service orchestration integrates with:
- **[System Behavior Overview](system-behavior.md)**: How services behave during startup and operation
- **[Module Integration Patterns](module-integration-patterns.md)**: How modules are loaded as services
- **[Operational Scenarios](operational-scenarios.md)**: Service interaction in specific scenarios
- **[Architecture Overview](architecture.md)**: Overall system architecture
## Related Documentation
- [System Behavior Overview](system-behavior.md) - System-level behavior
- [Module Integration Patterns](module-integration-patterns.md) - Module service integration
- [Operational Scenarios](operational-scenarios.md) - Service interaction scenarios
- [Architecture Overview](architecture.md) - System architecture
- [ADR-0029: Microservices Architecture](../adr/0029-microservices-architecture.md) - Architecture decision
- [ADR-0030: Service Communication Strategy](../adr/0030-service-communication-strategy.md) - Communication patterns

View File

@@ -0,0 +1,435 @@
# System Behavior Overview
## Purpose
This document provides a high-level explanation of how the Go Platform behaves end-to-end, focusing on system-level operations, flows, and interactions rather than implementation details.
## Overview
The Go Platform is a microservices-based system where each service is independently deployable from day one. Services communicate via gRPC (primary) or HTTP (fallback) through service clients, share infrastructure components (PostgreSQL instance, Redis, Kafka), and are orchestrated through service discovery and dependency injection. All external traffic enters through the API Gateway.
## Key Concepts
- **Services**: Independent processes that can be deployed and scaled separately
- **Service Clients**: Abstraction layer for inter-service communication
- **Service Registry**: Central registry for service discovery
- **Event Bus**: Asynchronous communication channel for events
- **DI Container**: Dependency injection container managing service lifecycle
## Service Bootstrap Sequence
Each service (API Gateway, Auth, Identity, Authz, Audit, and feature services) follows a well-defined startup sequence. Services bootstrap independently.
### Individual Service Startup
```mermaid
sequenceDiagram
participant Main
participant Config
participant Logger
participant DI
participant ServiceImpl
participant ServiceRegistry
participant DB
participant HTTP
participant gRPC
Main->>Config: Load configuration
Config-->>Main: Config ready
Main->>Logger: Initialize logger
Logger-->>Main: Logger ready
Main->>DI: Create DI container
DI->>DI: Register core kernel services
DI-->>Main: DI container ready
Main->>ServiceImpl: Register service implementation
ServiceImpl->>DI: Register service dependencies
ServiceImpl->>DB: Connect to database
DB-->>ServiceImpl: Connection ready
Main->>DB: Run migrations
DB-->>Main: Migrations complete
Main->>ServiceRegistry: Register service
ServiceRegistry->>ServiceRegistry: Register with Consul/K8s
ServiceRegistry-->>Main: Service registered
Main->>gRPC: Start gRPC server
Main->>HTTP: Start HTTP server (if needed)
HTTP-->>Main: HTTP server ready
gRPC-->>Main: gRPC server ready
Main->>DI: Start lifecycle
DI->>DI: Execute OnStart hooks
DI-->>Main: Service started
```
### Platform Startup (All Services)
```mermaid
sequenceDiagram
participant Docker
participant Gateway
participant AuthSvc
participant IdentitySvc
participant AuthzSvc
participant AuditSvc
participant BlogSvc
participant Registry
participant DB
Docker->>DB: Start PostgreSQL
Docker->>Registry: Start Consul
DB-->>Docker: Database ready
Registry-->>Docker: Registry ready
par Service Startup (in parallel)
Docker->>Gateway: Start API Gateway
Gateway->>Registry: Register
Gateway->>Gateway: Start HTTP server
Gateway-->>Docker: Gateway ready
and
Docker->>AuthSvc: Start Auth Service
AuthSvc->>DB: Connect
AuthSvc->>Registry: Register
AuthSvc->>AuthSvc: Start gRPC server
AuthSvc-->>Docker: Auth Service ready
and
Docker->>IdentitySvc: Start Identity Service
IdentitySvc->>DB: Connect
IdentitySvc->>Registry: Register
IdentitySvc->>IdentitySvc: Start gRPC server
IdentitySvc-->>Docker: Identity Service ready
and
Docker->>AuthzSvc: Start Authz Service
AuthzSvc->>DB: Connect
AuthzSvc->>Registry: Register
AuthzSvc->>AuthzSvc: Start gRPC server
AuthzSvc-->>Docker: Authz Service ready
and
Docker->>AuditSvc: Start Audit Service
AuditSvc->>DB: Connect
AuditSvc->>Registry: Register
AuditSvc->>AuditSvc: Start gRPC server
AuditSvc-->>Docker: Audit Service ready
and
Docker->>BlogSvc: Start Blog Service
BlogSvc->>DB: Connect
BlogSvc->>Registry: Register
BlogSvc->>BlogSvc: Start gRPC server
BlogSvc-->>Docker: Blog Service ready
end
Docker->>Docker: All services ready
```
### Service Bootstrap Phases (Per Service)
1. **Configuration Loading**: Load YAML files, environment variables, and secrets
2. **Foundation Services**: Initialize core kernel (logger, config, DI container)
3. **Database Connection**: Connect to database with own connection pool
4. **Service Implementation**: Register service-specific implementations
5. **Database Migrations**: Run service-specific migrations
6. **Service Registration**: Register service with service registry
7. **Server Startup**: Start gRPC server (and HTTP if needed)
8. **Lifecycle Hooks**: Execute OnStart hooks
### Platform Startup Order
1. **Infrastructure**: Start PostgreSQL, Redis, Kafka, Consul
2. **Core Services**: Start Auth, Identity, Authz, Audit services (can start in parallel)
3. **API Gateway**: Start API Gateway (depends on service registry)
4. **Feature Services**: Start Blog, Billing, etc. (can start in parallel)
5. **Health Checks**: All services report healthy to registry
## Request Processing Pipeline
Every HTTP request flows through API Gateway first, then to backend services. The pipeline ensures security, observability, and proper error handling.
```mermaid
graph TD
Start([HTTP Request]) --> Gateway[API Gateway]
Gateway --> RateLimit[Rate Limiting]
RateLimit -->|Allowed| Auth[Validate JWT via Auth Service]
RateLimit -->|Exceeded| Error0[429 Too Many Requests]
Auth -->|Valid Token| Authz[Check Permission via Authz Service]
Auth -->|Invalid Token| Error1[401 Unauthorized]
Authz -->|Authorized| RateLimit[Rate Limiting]
Authz -->|Unauthorized| Error2[403 Forbidden]
RateLimit -->|Within Limits| Tracing[OpenTelemetry Tracing]
RateLimit -->|Rate Limited| Error3[429 Too Many Requests]
Tracing --> Handler[Request Handler]
Handler --> Service[Domain Service]
Service --> Cache{Cache Check}
Cache -->|Hit| Return[Return Cached Data]
Cache -->|Miss| Repo[Repository]
Repo --> DB[(Database)]
DB --> Repo
Repo --> Service
Service --> CacheStore[Update Cache]
Service --> EventBus[Publish Events]
Service --> Audit[Audit Logging]
Service --> Metrics[Update Metrics]
Service --> Handler
Handler --> Tracing
Tracing --> Response[HTTP Response]
Error1 --> Response
Error2 --> Response
Error3 --> Response
Return --> Response
style Auth fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
style Authz fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
style Service fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
```
### Request Processing Stages
1. **Authentication**: Extract and validate JWT token, add user to context
2. **Authorization**: Check user permissions for requested resource
3. **Rate Limiting**: Enforce per-user and per-IP rate limits
4. **Tracing**: Start/continue distributed trace
5. **Handler Processing**: Execute request handler
6. **Service Logic**: Execute business logic
7. **Data Access**: Query database or cache
8. **Side Effects**: Publish events, audit logs, update metrics
9. **Response**: Return HTTP response with tracing context
## Event-Driven Interactions
The platform uses an event bus for asynchronous communication between services, enabling loose coupling and scalability.
```mermaid
sequenceDiagram
participant Publisher
participant EventBus
participant Kafka
participant Subscriber1
participant Subscriber2
Publisher->>EventBus: Publish(event)
EventBus->>EventBus: Serialize event
EventBus->>EventBus: Add metadata (trace_id, user_id)
EventBus->>Kafka: Send to topic
Kafka-->>EventBus: Acknowledged
Kafka->>Subscriber1: Deliver event
Kafka->>Subscriber2: Deliver event
Subscriber1->>Subscriber1: Process event
Subscriber1->>Subscriber1: Update state
Subscriber1->>Subscriber1: Emit new events (optional)
Subscriber2->>Subscriber2: Process event
Subscriber2->>Subscriber2: Update state
Note over Subscriber1,Subscriber2: Events processed asynchronously
```
### Event Processing Flow
1. **Event Publishing**: Service publishes event to event bus
2. **Event Serialization**: Event is serialized with metadata
3. **Event Distribution**: Event bus distributes to Kafka topic
4. **Event Consumption**: Subscribers consume events from Kafka
5. **Event Processing**: Each subscriber processes event independently
6. **State Updates**: Subscribers update their own state
7. **Cascade Events**: Subscribers may publish new events
## Background Job Processing
Background jobs are scheduled and processed asynchronously, enabling long-running tasks and scheduled operations.
```mermaid
sequenceDiagram
participant Scheduler
participant JobQueue
participant Worker
participant Service
participant DB
participant EventBus
Scheduler->>JobQueue: Enqueue job
JobQueue->>JobQueue: Store job definition
Worker->>JobQueue: Poll for jobs
JobQueue-->>Worker: Job definition
Worker->>Worker: Start job execution
Worker->>Service: Execute job logic
Service->>DB: Update data
Service->>EventBus: Publish events
Service-->>Worker: Job complete
Worker->>JobQueue: Mark job complete
alt Job fails
Worker->>JobQueue: Mark job failed
JobQueue->>JobQueue: Schedule retry
end
```
### Background Job Flow
1. **Job Scheduling**: Jobs scheduled via cron or programmatically
2. **Job Enqueueing**: Job definition stored in job queue
3. **Job Polling**: Workers poll queue for available jobs
4. **Job Execution**: Worker executes job logic
5. **Job Completion**: Job marked as complete or failed
6. **Job Retry**: Failed jobs retried with exponential backoff
## Error Recovery and Resilience
The platform implements multiple layers of error handling to ensure system resilience.
```mermaid
graph TD
Error[Error Occurs] --> Handler{Error Handler}
Handler -->|Business Error| BusinessError[Business Error Handler]
Handler -->|System Error| SystemError[System Error Handler]
Handler -->|Panic| PanicHandler[Panic Recovery]
BusinessError --> ErrorBus[Error Bus]
SystemError --> ErrorBus
PanicHandler --> ErrorBus
ErrorBus --> Logger[Logger]
ErrorBus --> Sentry[Sentry]
ErrorBus --> Metrics[Metrics]
BusinessError --> Response[HTTP Response]
SystemError --> Response
PanicHandler --> Response
Response --> Client[Client]
style Error fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
style ErrorBus fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
```
### Error Handling Layers
1. **Panic Recovery**: Middleware catches panics and prevents crashes
2. **Error Classification**: Errors classified as business or system errors
3. **Error Bus**: Central error bus collects all errors
4. **Error Logging**: Errors logged with full context
5. **Error Reporting**: Critical errors reported to Sentry
6. **Error Metrics**: Errors tracked in metrics
7. **Error Response**: Appropriate HTTP response returned
## System Shutdown Sequence
The platform implements graceful shutdown to ensure data consistency and proper resource cleanup.
```mermaid
sequenceDiagram
participant Signal
participant Main
participant HTTP
participant gRPC
participant ServiceRegistry
participant DI
participant Workers
participant DB
Signal->>Main: SIGTERM/SIGINT
Main->>HTTP: Stop accepting requests
HTTP->>HTTP: Wait for active requests
HTTP-->>Main: HTTP server stopped
Main->>gRPC: Stop accepting connections
gRPC->>gRPC: Wait for active calls
gRPC-->>Main: gRPC server stopped
Main->>ServiceRegistry: Deregister service
ServiceRegistry->>ServiceRegistry: Remove from registry
ServiceRegistry-->>Main: Service deregistered
Main->>Workers: Stop workers
Workers->>Workers: Finish current jobs
Workers-->>Main: Workers stopped
Main->>DI: Stop lifecycle
DI->>DI: Execute OnStop hooks
DI->>DI: Close connections
DI->>DB: Close DB connections
DI-->>Main: Services stopped
Main->>Main: Exit
```
### Shutdown Phases
1. **Signal Reception**: Receive SIGTERM or SIGINT
2. **Stop Accepting Requests**: HTTP and gRPC servers stop accepting new requests
3. **Wait for Active Requests**: Wait for in-flight requests to complete
4. **Service Deregistration**: Remove service from service registry
5. **Worker Shutdown**: Stop background workers gracefully
6. **Lifecycle Hooks**: Execute OnStop hooks for all services
7. **Resource Cleanup**: Close database connections, release resources
8. **Application Exit**: Exit application cleanly
## Health Check and Monitoring Flow
Health checks and metrics provide visibility into system health and performance.
```mermaid
graph TD
HealthEndpoint["/healthz"] --> HealthRegistry[Health Registry]
HealthRegistry --> CheckDB[Check Database]
HealthRegistry --> CheckCache[Check Cache]
HealthRegistry --> CheckEventBus[Check Event Bus]
CheckDB -->|Healthy| Aggregate[Aggregate Results]
CheckCache -->|Healthy| Aggregate
CheckEventBus -->|Healthy| Aggregate
Aggregate -->|All Healthy| Response200[200 OK]
Aggregate -->|Unhealthy| Response503[503 Service Unavailable]
MetricsEndpoint["/metrics"] --> MetricsRegistry[Metrics Registry]
MetricsRegistry --> Prometheus[Prometheus Format]
Prometheus --> ResponseMetrics[Metrics Response]
style HealthRegistry fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
style MetricsRegistry fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
```
### Health Check Components
- **Liveness Check**: Service is running (process health)
- **Readiness Check**: Service is ready to accept requests (dependency health)
- **Dependency Checks**: Database, cache, event bus connectivity
- **Metrics Collection**: Request counts, durations, error rates
- **Metrics Export**: Prometheus-formatted metrics
## Integration Points
This system behavior integrates with:
- **[Service Orchestration](service-orchestration.md)**: How services coordinate during startup and operation
- **[Module Integration Patterns](module-integration-patterns.md)**: How modules integrate during bootstrap
- **[Operational Scenarios](operational-scenarios.md)**: Specific operational flows and use cases
- **[Data Flow Patterns](data-flow-patterns.md)**: Detailed data flow through the system
- **[Architecture Overview](architecture.md)**: System architecture and component relationships
## Related Documentation
- [Architecture Overview](architecture.md) - System architecture
- [Service Orchestration](service-orchestration.md) - Service coordination
- [Module Integration Patterns](module-integration-patterns.md) - Module integration
- [Operational Scenarios](operational-scenarios.md) - Common operational flows
- [Component Relationships](component-relationships.md) - Component dependencies

View File

@@ -0,0 +1,23 @@
/* Full width content */
.md-content {
max-width: 100% !important;
}
.md-main__inner {
max-width: 100% !important;
}
.md-grid {
max-width: 100% !important;
}
/* Ensure content area uses full width while keeping readable line length */
.md-content__inner {
max-width: 100%;
}
/* Adjust container padding for better full-width experience */
.md-container {
max-width: 100%;
}

91
docs/content/index.md Normal file
View File

@@ -0,0 +1,91 @@
# Go Platform Documentation
Welcome to the Go Platform documentation! This is a plugin-friendly SaaS/Enterprise platform built with Go.
## What is Go Platform?
Go Platform is a microservices platform designed to support multiple business domains through independent, deployable services. It provides:
- **Core Kernel**: Infrastructure only (configuration, logging, DI, health, metrics, observability) - no business logic
- **Core Services**: Independent microservices (Auth, Identity, Authz, Audit) with their own entry points and databases
- **API Gateway**: Single entry point for all external traffic, handles routing, authentication, and rate limiting
- **Service Discovery**: Consul-based service registry for dynamic service discovery
- **Module Framework**: Feature services (Blog, Billing, etc.) as independent services
- **Infrastructure Adapters**: Support for databases, caching, event buses, and job scheduling
- **Security-by-Design**: Built-in JWT authentication, RBAC/ABAC authorization, and audit logging
- **Observability**: OpenTelemetry integration for distributed tracing, metrics, and logging across services
## Documentation Structure
### Overview
- **[Requirements](requirements.md)**: High-level architectural principles and requirements
- **[Implementation Plan](plan.md)**: Epic-based implementation plan with timelines
- **[Playbook](playbook.md)**: Detailed implementation guide and best practices
### Architecture
- **[Architecture Overview](architecture/architecture.md)**: System architecture with diagrams
- **[Module Architecture](architecture/architecture-modules.md)**: Module system design and integration
- **[Module Requirements](architecture/module-requirements.md)**: Detailed requirements for each module
- **[Component Relationships](architecture/component-relationships.md)**: Component interactions and dependencies
- **System Specifications**
- **[System Behavior Overview](architecture/system-behavior.md)**: How the system behaves end-to-end
- **[Service Orchestration](architecture/service-orchestration.md)**: How services work together
- **[Module Integration Patterns](architecture/module-integration-patterns.md)**: How modules integrate with the platform
- **[Operational Scenarios](architecture/operational-scenarios.md)**: Common operational flows and use cases
- **[Data Flow Patterns](architecture/data-flow-patterns.md)**: How data flows through the system
### Architecture Decision Records (ADRs)
All architectural decisions are documented in [ADR records](adr/README.md), organized by implementation epic:
- **Epic 0**: Project Setup & Foundation
- **Epic 1**: Core Kernel & Infrastructure
- **Epic 2**: Authentication & Authorization
- **Epic 3**: Module Framework
- **Epic 5**: Infrastructure Adapters
- **Epic 6**: Observability & Production Readiness
- **Epic 7**: Testing, Documentation & CI/CD
### Implementation Tasks
Detailed task definitions for each epic are available in the [Stories section](stories/README.md):
- **[Epic 0: Project Setup & Foundation](stories/epic0/README.md)** - [Implementation Summary](stories/epic0/SUMMARY.md)
- **[Epic 1: Core Kernel & Infrastructure](stories/epic1/README.md)** - [Implementation Summary](stories/epic1/SUMMARY.md)
- Epic 2: Authentication & Authorization
- Epic 3: Module Framework
- Epic 4: Sample Feature Module (Blog)
- Epic 5: Infrastructure Adapters
- Epic 6: Observability & Production Readiness
- Epic 7: Testing, Documentation & CI/CD
- Epic 8: Advanced Features & Polish (Optional)
## Quick Start
1. Review the [Requirements](requirements.md) to understand the platform goals
2. Check the [Implementation Plan](plan.md) for the epic-based approach
3. Follow the [Playbook](playbook.md) for implementation details
4. Refer to [ADRs](adr/README.md) when making architectural decisions
5. Use the [Task Stories](stories/README.md) as a checklist for implementation
## Key Principles
- **microMicroservices Architecture**: Each service is independently deployable from day one
- Core Kernel: Infrastructure only (config, logger, DI, health, metrics)
- Core Services: Auth, Identity, Authz, Audit as separate services
- Feature Services: Blog, Billing, etc. as independent services
- **API Gateway**: Single entry point for all external traffic
- **Service Discovery**: Consul-based service registry for dynamic service location
- **Service Clients**: All inter-service communication via gRPC/HTTP through service clients
- **Database Isolation**: Each service has its own database connection pool and schema
- **Hexagonal Architecture**: Clear separation between interfaces and implementations
- **Security-by-Design**: Built-in authentication, authorization, and audit capabilities
- **Observability**: Comprehensive distributed tracing, metrics, and logging across services
- **API-First**: OpenAPI/GraphQL schema generation
## Contributing
When contributing to the platform:
1. Review relevant ADRs before making architectural decisions
2. Follow the task structure defined in the Stories
3. Update documentation as you implement features
4. Ensure all tests pass before submitting changes

1732
docs/content/plan.md Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -1,13 +1,13 @@
# GoPlatform Boilerplate Playbook # GoPlatform Boilerplate Playbook
**“Pluginfriendly SaaS/Enterprise Platform Go Edition”** **“Pluginfriendly SaaS/Enterprise Platform Go Edition”**
## 1️⃣ ARCHITECTURAL IMPERATIVES (Goflavoured) ## 1 ARCHITECTURAL IMPERATIVES (Goflavoured)
| Principle | Gospecific rationale | Enforcement Technique | | Principle | Gospecific rationale | Enforcement Technique |
|-----------|-----------------------|------------------------| |-----------|-----------------------|------------------------|
| **Clean / Hexagonal Architecture** | Gos packagelevel visibility (`internal/`) naturally creates a *boundary* between core and plugins. | Keep all **domain** code in `internal/domain`, expose only **interfaces** in `pkg/`. | | **Hexagonal Architecture** | Gos packagelevel visibility (`internal/`) naturally creates a *boundary* between core and plugins. | Keep all **domain** code in `internal/domain`, expose only **interfaces** in `pkg/`. |
| **Dependency Injection (DI) via Constructors** | Go avoids reflectionheavy containers; compiletime wiring is preferred. | Use **ubergo/fx** (runtime graph) *or* **ubergo/dig** for optional runtime DI. For a lighter weight solution, use plain **constructor injection** with a small **registry**. | | **Dependency Injection (DI) via Constructors** | Go avoids reflectionheavy containers; compiletime wiring is preferred. | Use **ubergo/fx** (runtime graph) *or* **ubergo/dig** for optional runtime DI. For a lighter weight solution, use plain **constructor injection** with a small **registry**. |
| **Modular Monolith → Microserviceready** | A single binary is cheap in Go; later you can extract modules into separate services without breaking APIs. | Each module lives in its own Go **module** (`go.mod`) under `./modules/*`. The core loads them via the **Go plugin** system *or* static registration (preferred for CI stability). | | **microMicroservices Architecture** | Each service is independently deployable from day one. Services communicate via gRPC/HTTP through service clients. | Each service has its own entry point (`cmd/{service}/`), Go module (`go.mod`), database connection, and deployment. Services discover each other via Consul service registry. |
| **Pluginfirst design** | Gos `plugin` package allows runtime loading of compiled `.so` files (Linux/macOS). | Provide an **IModule** interface and a **loader** that discovers `*.so` files (or compiledin modules for CI). | | **Pluginfirst design** | Gos `plugin` package allows runtime loading of compiled `.so` files (Linux/macOS). | Provide an **IModule** interface and a **loader** that discovers `*.so` files (or compiledin modules for CI). |
| **APIFirst (OpenAPI + gin/gorilla)** | Guarantees languageagnostic contracts. | Generate server stubs from an `openapi.yaml` stored in `api/`. | | **APIFirst (OpenAPI + gin/gorilla)** | Guarantees languageagnostic contracts. | Generate server stubs from an `openapi.yaml` stored in `api/`. |
| **SecuritybyDesign** | Gos static typing makes it easy to keep auth data out of the request flow. | Central middleware for JWT verification + contextbased user propagation. | | **SecuritybyDesign** | Gos static typing makes it easy to keep auth data out of the request flow. | Central middleware for JWT verification + contextbased user propagation. |
@@ -18,29 +18,51 @@
--- ---
## 2️⃣ CORE KERNEL (What every Goplatform must ship) ## 2 CORE KERNEL (Infrastructure Only)
| Module | Public Interfaces (exported from `pkg/`) | Recommended Packages | Brief Implementation Sketch | The core kernel provides foundational infrastructure that all services depend on. It contains **no business logic**.
| Component | Public Interfaces (exported from `pkg/`) | Recommended Packages | Brief Implementation Sketch |
|--------|-------------------------------------------|----------------------|------------------------------| |--------|-------------------------------------------|----------------------|------------------------------|
| **Config** | `type ConfigProvider interface { Get(key string) any; Unmarshal(v any) error }` | `github.com/spf13/viper` | Load defaults (`config/default.yaml`), then env overrides, then optional secretstore. | | **Config** | `type ConfigProvider interface { Get(key string) any; Unmarshal(v any) error }` | `github.com/spf13/viper` | Load defaults (`config/default.yaml`), then env overrides, then optional secretstore. |
| **Logger** | `type Logger interface { Debug(msg string, fields ...Field); Info(...); Error(...); With(fields ...Field) Logger }` | `go.uber.org/zap` (or `zerolog`) | Global logger is created in `cmd/main.go`; exported via `pkg/logger`. | | **Logger** | `type Logger interface { Debug(msg string, fields ...Field); Info(...); Error(...); With(fields ...Field) Logger }` | `go.uber.org/zap` (or `zerolog`) | Global logger is created in each service's `cmd/{service}/main.go`; exported via `pkg/logger`. |
| **DI / Service Registry** | `type Container interface { Provide(constructor any) error; Invoke(fn any) error }` | `go.uber.org/dig` (or `fx` for lifecycle) | Core creates a `dig.New()` container, registers core services, then calls `container.Invoke(app.Start)`. | | **DI / Service Registry** | `type Container interface { Provide(constructor any) error; Invoke(fn any) error }` | `go.uber.org/fx` (for lifecycle) | Each service creates its own `fx.New()` container, registers service-specific services. |
| **Health & Metrics** | `type HealthChecker interface { Check(ctx context.Context) error }` | `github.com/prometheus/client_golang/prometheus`, `github.com/heptiolabs/healthcheck` | Expose `/healthz`, `/ready`, `/metrics`. | | **Health & Metrics** | `type HealthChecker interface { Check(ctx context.Context) error }` | `github.com/prometheus/client_golang/prometheus`, `github.com/heptiolabs/healthcheck` | Each service exposes `/healthz`, `/ready`, `/metrics`. |
| **Error Bus** | `type ErrorPublisher interface { Publish(err error) }` | Simple channelbased implementation + optional Sentry (`github.com/getsentry/sentry-go`) | Core registers a singleton `ErrorBus`. | | **Error Bus** | `type ErrorPublisher interface { Publish(err error) }` | Simple channelbased implementation + optional Sentry (`github.com/getsentry/sentry-go`) | Each service registers its own `ErrorBus`. |
| **Auth (JWT + OIDC)** | `type Authenticator interface { GenerateToken(userID string, roles []string) (string, error); VerifyToken(token string) (*TokenClaims, error) }` | `github.com/golang-jwt/jwt/v5`, `github.com/coreos/go-oidc` | Token claims embed `sub`, `roles`, `tenant_id`. Middleware adds `User` to `context.Context`. | | **Service Registry** | `type ServiceRegistry interface { Register(ctx, service) error; Discover(ctx, name) ([]Service, error) }` | `github.com/hashicorp/consul/api` | Consul-based service discovery. Services register on startup, clients discover via registry. |
| **Authorization (RBAC/ABAC)** | `type Authorizer interface { Authorize(ctx context.Context, perm Permission) error }` | Custom DSL, `github.com/casbin/casbin/v2` (optional) | Permission format: `"module.resource.action"`; core ships a simple inmemory resolver and a `casbin` adapter. | | **Observability** | `type Tracer interface { StartSpan(ctx, name) (Span, context.Context) }` | `go.opentelemetry.io/otel` | OpenTelemetry integration for distributed tracing across services. |
| **Audit** | `type Auditor interface { Record(ctx context.Context, act AuditAction) error }` | Write to appendonly table (Postgres) or Elastic via `olivere/elastic` | Audits include `actorID`, `action`, `targetID`, `metadata`. | | **Event Bus** | `type EventBus interface { Publish(ctx context.Context, ev Event) error; Subscribe(topic string, handler EventHandler) }` | `github.com/segmentio/kafka-go` | Kafka-based event bus for asynchronous cross-service communication. |
| **Event Bus** | `type EventBus interface { Publish(ctx context.Context, ev Event) error; Subscribe(topic string, handler EventHandler) }` | `github.com/segmentio/kafka-go` (for production) + inprocess fallback | Core ships an **inprocess bus** used by tests and a **Kafka bus** for real deployments. | | **Scheduler / Background Jobs** | `type Scheduler interface { Cron(spec string, job JobFunc) error; Enqueue(q string, payload any) error }` | `github.com/robfig/cron/v3`, `github.com/hibiken/asynq` (Redisbacked) | Shared infrastructure for background jobs. |
| **Persistence (Repository)** | `type UserRepo interface { FindByID(id string) (*User, error); Create(u *User) error; … }` | `entgo.io/ent` (codegen ORM) **or** `gorm.io/gorm` | Core provides an `EntClient` wrapper that implements all core repos. | | **Notification** | `type Notifier interface { Send(ctx context.Context, n Notification) error }` | `github.com/go-mail/mail` (SMTP), `github.com/aws/aws-sdk-go-v2/service/ses` | Shared infrastructure for notifications. |
| **Scheduler / Background Jobs** | `type Scheduler interface { Cron(spec string, job JobFunc) error; Enqueue(q string, payload any) error }` | `github.com/robfig/cron/v3`, `github.com/hibiken/asynq` (Redisbacked) | Expose a `JobRegistry` where modules can register periodic jobs. | | **Multitenancy (optional)** | `type TenantResolver interface { Resolve(ctx context.Context) (tenantID string, err error) }` | Header/ subdomain parser + JWT claim scanner | Tenant ID is stored in request context and automatically added to SQL queries via Ent's `Client` interceptor. |
| **Notification** | `type Notifier interface { Send(ctx context.Context, n Notification) error }` | `github.com/go-mail/mail` (SMTP), `github.com/aws/aws-sdk-go-v2/service/ses`, `github.com/IBM/sarama` (for push) | Core supplies an `EmailNotifier` and a `WebhookNotifier`. |
| **Multitenancy (optional)** | `type TenantResolver interface { Resolve(ctx context.Context) (tenantID string, err error) }` | Header/ subdomain parser + JWT claim scanner | Tenant ID is stored in request context and automatically added to SQL queries via Ents `Client` interceptor. |
All *public* interfaces live under `pkg/` so that plugins can import them without pulling in implementation details. The concrete implementations stay in `internal/` (or separate go.mod modules) and are **registered with the container** during bootstrap. ## 2.1 CORE SERVICES (Independent Microservices)
Core business services are implemented as separate, independently deployable services:
| Service | Entry Point | Responsibilities | Service Client Interface |
|--------|-------------|------------------|-------------------------|
| **Auth Service** | `cmd/auth-service/` | JWT token generation/validation, authentication | `AuthServiceClient` in `pkg/services/auth.go` |
| **Identity Service** | `cmd/identity-service/` | User CRUD, password management, email verification | `IdentityServiceClient` in `pkg/services/identity.go` |
| **Authz Service** | `cmd/authz-service/` | Permission resolution, RBAC/ABAC authorization | `AuthzServiceClient` in `pkg/services/authz.go` |
| **Audit Service** | `cmd/audit-service/` | Audit logging, immutable audit records | `AuditServiceClient` in `pkg/services/audit.go` |
| **API Gateway** | `cmd/api-gateway/` | Request routing, authentication, rate limiting, CORS | N/A (entry point) |
Each service:
- Has its own `go.mod` (or shared workspace)
- Manages its own database connection pool and schema
- Exposes gRPC server (and optional HTTP)
- Registers with Consul service registry
- Uses service clients for inter-service communication
All *public* interfaces live under `pkg/` so that services can import them without pulling in implementation details. The concrete implementations stay in `internal/` (for core kernel) or `services/{service}/internal/` (for service implementations) and are **registered with the container** during service bootstrap.
**Note:** Business logic services (Auth, Identity, Authz, Audit) are NOT in the core kernel. They are separate services implemented in Epic 2.
--- ---
## 3️⃣ MODULE (PLUGIN) FRAMEWORK ## 3 MODULE (PLUGIN) FRAMEWORK
### 3.1 Interface that every module must implement ### 3.1 Interface that every module must implement
@@ -167,13 +189,21 @@ A **codegen** tool (`go generate ./...`) can scan each modules `module.yam
--- ---
## 4️⃣ SAMPLE FEATURE MODULE **Blog** ## 4 SAMPLE FEATURE SERVICE **Blog Service**
Each feature module is implemented as an independent service:
``` ```
modules/ cmd/
└─ blog-service/
└─ main.go # Service entry point
services/
└─ blog/ └─ blog/
├─ go.mod # (module github.com/yourorg/blog) ├─ go.mod # Service dependencies
├─ module.yaml ├─ module.yaml # Service manifest
├─ api/
│ └─ blog.proto # gRPC service definition
├─ internal/ ├─ internal/
│ ├─ api/ │ ├─ api/
│ │ └─ handler.go │ │ └─ handler.go
@@ -207,74 +237,165 @@ routes:
permission: blog.post.read permission: blog.post.read
``` ```
### 4.2 Go implementation ### 4.2 Service Entry Point
```go ```go
// pkg/module.go // cmd/blog-service/main.go
package blog package main
import ( import (
"github.com/yourorg/platform/pkg/module" "context"
"github.com/yourorg/platform/internal/config"
"github.com/yourorg/platform/internal/di"
"github.com/yourorg/platform/services/blog/internal/api"
"github.com/yourorg/platform/services/blog/internal/service"
"go.uber.org/fx" "go.uber.org/fx"
) )
type BlogModule struct{} func main() {
cfg := config.Load()
func (b BlogModule) Name() string { return "blog" } fx.New(
// Core kernel services
di.CoreModule(cfg),
func (b BlogModule) Init() fx.Option { // Blog service implementation
return fx.Options( fx.Provide(service.NewPostService),
// Register repository implementation fx.Provide(service.NewPostRepo),
fx.Provide(NewPostRepo),
// Register service layer // gRPC server
fx.Provide(NewPostService), fx.Provide(api.NewGRPCServer),
// Register HTTP handlers (using Gin) // Service registry
fx.Invoke(RegisterHandlers), fx.Provide(di.ProvideServiceRegistry),
// Register permissions (optional just for documentation) // Start service
fx.Invoke(RegisterPermissions), fx.Invoke(startService),
) ).Run()
} }
func (b BlogModule) Migrations() []func(*ent.Client) error { func startService(lc fx.Lifecycle, server *api.GRPCServer, registry registry.ServiceRegistry) {
// Ent migration generated in internal/ent/migrate lc.Append(fx.Hook{
return []func(*ent.Client) error{ OnStart: func(ctx context.Context) error {
func(c *ent.Client) error { return c.Schema.Create(context.Background()) }, // Register with Consul
registry.Register(ctx, &registry.ServiceInstance{
ID: "blog-service-1",
Name: "blog-service",
Address: "localhost",
Port: 8091,
})
// Start gRPC server
return server.Start()
},
OnStop: func(ctx context.Context) error {
registry.Deregister(ctx, "blog-service-1")
return server.Stop()
},
})
}
```
### 4.3 Service Implementation
```go
// services/blog/internal/service/post_service.go
package service
import (
"context"
"github.com/yourorg/platform/pkg/services"
"github.com/yourorg/platform/services/blog/internal/domain"
)
type PostService struct {
repo *domain.PostRepo
authzClient services.AuthzServiceClient
identityClient services.IdentityServiceClient
auditClient services.AuditServiceClient
}
func NewPostService(
repo *domain.PostRepo,
authzClient services.AuthzServiceClient,
identityClient services.IdentityServiceClient,
auditClient services.AuditServiceClient,
) *PostService {
return &PostService{
repo: repo,
authzClient: authzClient,
identityClient: identityClient,
auditClient: auditClient,
} }
} }
// Export a variable for the plugin loader func (s *PostService) CreatePost(ctx context.Context, req *CreatePostRequest) (*Post, error) {
var Module BlogModule // Check permission via Authz Service
if err := s.authzClient.Authorize(ctx, "blog.post.create"); err != nil {
return nil, err
}
// Get user info via Identity Service
user, err := s.identityClient.GetUser(ctx, req.AuthorID)
if err != nil {
return nil, err
}
// Create post
post, err := s.repo.Create(ctx, &domain.Post{
Title: req.Title,
Content: req.Content,
AuthorID: user.ID,
})
if err != nil {
return nil, err
}
// Audit log via Audit Service
s.auditClient.Record(ctx, &services.AuditAction{
ActorID: user.ID,
Action: "blog.post.create",
TargetID: post.ID,
})
return post, nil
}
``` ```
**Handler registration (Gin example)** ### 4.4 gRPC Handler
```go ```go
// internal/api/handler.go // services/blog/internal/api/handler.go
package api package api
import ( import (
"github.com/gin-gonic/gin" "context"
"github.com/yourorg/blog/internal/service" "github.com/yourorg/platform/services/blog/api/pb"
"github.com/yourorg/platform/pkg/perm" "github.com/yourorg/platform/services/blog/internal/service"
"github.com/yourorg/platform/pkg/auth"
) )
func RegisterHandlers(r *gin.Engine, svc *service.PostService, authz auth.Authorizer) { type BlogServer struct {
grp := r.Group("/api/v1/blog") pb.UnimplementedBlogServiceServer
grp.Use(auth.AuthMiddleware()) // verifies JWT, injects user in context service *service.PostService
}
// POST /posts func (s *BlogServer) CreatePost(ctx context.Context, req *pb.CreatePostRequest) (*pb.CreatePostResponse, error) {
grp.POST("/posts", func(c *gin.Context) { post, err := s.service.CreatePost(ctx, &service.CreatePostRequest{
if err := authz.Authorize(c.Request.Context(), perm.BlogPostCreate); err != nil { Title: req.Title,
c.JSON(403, gin.H{"error": "forbidden"}) Content: req.Content,
return AuthorID: req.AuthorId,
}
// decode request, call svc.Create, return 201…
}) })
// GET /posts/:id (similar) if err != nil {
return nil, err
}
return &pb.CreatePostResponse{
Post: &pb.Post{
Id: post.ID,
Title: post.Title,
Content: post.Content,
},
}, nil
} }
``` ```
@@ -312,7 +433,7 @@ func (r *PostRepo) Create(ctx context.Context, p *Post) (*Post, error) {
--- ---
## 5️⃣ INFRASTRUCTURE ADAPTERS (swapable, perenvironment) ## 5 INFRASTRUCTURE ADAPTERS (swapable, perenvironment)
| Concern | Implementation (Go) | Where it lives | | Concern | Implementation (Go) | Where it lives |
|---------|---------------------|----------------| |---------|---------------------|----------------|
@@ -328,7 +449,7 @@ All adapters expose an **interface** in `pkg/infra/…` and are registered in th
--- ---
## 6️⃣ OBSERVABILITY STACK ## 6 OBSERVABILITY STACK
| Layer | Library | What it does | | Layer | Library | What it does |
|-------|---------|--------------| |-------|---------|--------------|
@@ -372,7 +493,7 @@ func PromMetrics() gin.HandlerFunc {
--- ---
## 7️⃣ CONFIGURATION & ENVIRONMENT ## 7 CONFIGURATION & ENVIRONMENT
``` ```
config/ config/
@@ -405,7 +526,7 @@ All services receive a `*Config` via DI.
--- ---
## 8️⃣ CI / CD PIPELINE (GitHub Actions) ## 8 CI / CD PIPELINE (GitHub Actions)
```yaml ```yaml
name: CI name: CI
@@ -469,7 +590,7 @@ jobs:
--- ---
## 9️⃣ TESTING STRATEGY ## 9 TESTING STRATEGY
| Test type | Tools | Typical coverage | | Test type | Tools | Typical coverage |
|-----------|-------|------------------| |-----------|-------|------------------|
@@ -523,7 +644,7 @@ func TestCreatePost_Integration(t *testing.T) {
--- ---
## 10️⃣ COMMON PITFALLS & SOLUTIONS (Gocentric) ## 10 COMMON PITFALLS & SOLUTIONS (Gocentric)
| Pitfall | Symptom | Remedy | | Pitfall | Symptom | Remedy |
|---------|----------|--------| |---------|----------|--------|
@@ -540,7 +661,7 @@ func TestCreatePost_Integration(t *testing.T) {
--- ---
## 11️⃣ QUICKSTART STEPS (What to code first) ## 11 QUICKSTART STEPS (What to code first)
1. **Bootstrap repo** 1. **Bootstrap repo**
```bash ```bash
@@ -586,7 +707,7 @@ After step10 you have a **complete, productiongrade scaffolding** that:
--- ---
## 12️⃣ REFERENCE IMPLEMENTATION (public) ## 12 REFERENCE IMPLEMENTATION (public)
If you prefer to start from a **real opensource baseline**, check out the following community projects that already adopt most of the ideas above: If you prefer to start from a **real opensource baseline**, check out the following community projects that already adopt most of the ideas above:
@@ -602,7 +723,7 @@ Fork one, strip the business logic, and rename the packages to match *your* `git
--- ---
## 13️⃣ FINAL CHECKLIST (before you ship) ## 13 FINAL CHECKLIST (before you ship)
- [ ] Core modules compiled & registered in `internal/di`. - [ ] Core modules compiled & registered in `internal/di`.
- [ ] `module.IModule` interface and static registry in place. - [ ] `module.IModule` interface and static registry in place.
@@ -617,4 +738,4 @@ Fork one, strip the business logic, and rename the packages to match *your* `git
- [ ] Sample plugin (Blog) builds, loads, registers routes, and passes integration test. - [ ] Sample plugin (Blog) builds, loads, registers routes, and passes integration test.
- [ ] Documentation: `README.md`, `docs/architecture.md`, `docs/extension-points.md`. - [ ] Documentation: `README.md`, `docs/architecture.md`, `docs/extension-points.md`.
> **Congratulations!** You now have a **robust, extensible Go platform boilerplate** that can be the foundation for any SaaS, internal toolset, or microservice ecosystem you wish to build. Happy coding! 🚀 > **Congratulations!** You now have a **robust, extensible Go platform boilerplate** that can be the foundation for any SaaS, internal toolset, or microservice ecosystem you wish to build. Happy coding!

View File

@@ -1,12 +1,12 @@
# Requirements # Requirements
## 1 HIGHLEVEL ARCHITECTURAL PRINCIPLES ## HIGHLEVEL ARCHITECTURAL PRINCIPLES
| Principle | Why it matters for a modular platform | How to enforce it | | Principle | Why it matters for a modular platform | How to enforce it |
|-----------|----------------------------------------|-------------------| |-----------|----------------------------------------|-------------------|
| **Separation of Concerns (SoC)** | Keeps core services (auth, audit, config) independent from business modules. | Use **layered** or **hexagonal/cleanarchitecture** boundaries. | | **Separation of Concerns (SoC)** | Keeps core services (auth, audit, config) independent from business modules. | Use **layered** or **hexagonal/cleanarchitecture** boundaries. |
| **DomainDriven Design (DDD) Bounded Contexts** | Allows each module to own its own model & rules while sharing a common identity kernel. | Define a **Core Context** (Identity, Security, Infrastructure) and **Feature Contexts** (Billing, CMS, Chat, …). | | **DomainDriven Design (DDD) Bounded Contexts** | Allows each module to own its own model & rules while sharing a common identity kernel. | Define a **Core Context** (Identity, Security, Infrastructure) and **Feature Contexts** (Billing, CMS, Chat, …). |
| **Modular Monolith → Microserviceready** | Start simple (single process) but keep each module in its own package so you can later split to services if needed. | Package each module as an **independent library** with its own **DI containermodule** and **routing**. | | **microMicroservices Architecture** | Each service is independently deployable from day one. Services communicate via gRPC/HTTP through service clients. | Each service has its own entry point (`cmd/{service}/`), database connection, and deployment configuration. |
| **Plugin / Extensionpoint model** | Enables customers or internal teams to drop new features without touching core code. | Export **welldefined interfaces** (e.g., `IUserProvider`, `IPermissionResolver`, `IModuleInitializer`). | | **Plugin / Extensionpoint model** | Enables customers or internal teams to drop new features without touching core code. | Export **welldefined interfaces** (e.g., `IUserProvider`, `IPermissionResolver`, `IModuleInitializer`). |
| **APIFirst** | Guarantees that any UI (web, mobile, CLI) can be built on top of the same contract. | Publish **OpenAPI/GraphQL schema** as part of the build artefact. | | **APIFirst** | Guarantees that any UI (web, mobile, CLI) can be built on top of the same contract. | Publish **OpenAPI/GraphQL schema** as part of the build artefact. |
| **SecuritybyDesign** | The platform will hold user credentials, roles and possibly PII. | Centralize **authentication**, **authorization**, **audit**, **ratelimiting**, **CORS**, **CSP**, **secure defaults**. | | **SecuritybyDesign** | The platform will hold user credentials, roles and possibly PII. | Centralize **authentication**, **authorization**, **audit**, **ratelimiting**, **CORS**, **CSP**, **secure defaults**. |
@@ -18,7 +18,7 @@
--- ---
## 2 LAYERED / HEXAGONAL BLUEPRINT ## LAYERED / HEXAGONAL BLUEPRINT
``` ```
+---------------------------------------------------+ +---------------------------------------------------+
@@ -50,7 +50,17 @@
--- ---
## 3⃣ REQUIRED BASE MODULES (THE “CORE KERNEL”) ## CORE KERNEL (INFRASTRUCTURE ONLY)
The core kernel provides foundational infrastructure services that all other services depend on. It contains **no business logic** and is purely infrastructure.
## CORE SERVICES (INDEPENDENT MICROSERVICES)
Core services provide business logic and are deployed as separate, independently scalable services. Each service has its own database connection, API endpoints, and deployment configuration.
## INFRASTRUCTURE ADAPTERS (SHARED SERVICES)
These adapters provide infrastructure capabilities that services can use.
| Module | Core responsibilities | Public API / Extension points | | Module | Core responsibilities | Public API / Extension points |
|--------|-----------------------|--------------------------------| |--------|-----------------------|--------------------------------|
@@ -74,7 +84,7 @@
--- ---
## 4 EXTENSIONPOINT DESIGN (HOW PLUGINS HOOK IN) ## EXTENSIONPOINT DESIGN (HOW PLUGINS HOOK IN)
1. **Module Manifest** a tiny JSON/YAML file (`module.yaml`) that declares: 1. **Module Manifest** a tiny JSON/YAML file (`module.yaml`) that declares:
- Module name, version, dependencies (core ≥ 1.2.0, other modules) - Module name, version, dependencies (core ≥ 1.2.0, other modules)
@@ -123,43 +133,70 @@
--- ---
## 5 SAMPLE REPOSITORY LAYOUT (languageagnostic) ## SAMPLE REPOSITORY LAYOUT (languageagnostic)
``` ```
/platform-root /platform-root
├─ /core # ---- Kernel / Base modules ---- ├─ /cmd # ---- Service Entry Points ----
│ ├─ /auth │ ├─ /api-gateway # API Gateway service
│ │ src/ │ │ main.go
│ └─ package.json ├─ /auth-service # Auth service
├─ /identity │ └─ main.go
│ ├─ /authorization │ ├─ /identity-service # Identity service
├─ /audit │ └─ main.go
│ ├─ /config │ ├─ /authz-service # Authorization service
├─ /logging │ └─ main.go
│ ├─ /metrics │ ├─ /audit-service # Audit service
│ └─ index.ts (exports all core APIs) │ └─ main.go
│ └─ /blog-service # Blog feature service
│ └─ main.go
├─ /modules # ---- Feature plugins ---- ├─ /services # ---- Service Implementations ----
│ ├─ /blog │ ├─ /auth/
│ │ ├─ module.yaml # manifest │ │ ├─ internal/ # Service implementation
│ │ src/ │ │ api/ # gRPC/HTTP definitions
│ │ ├─ BlogController.ts ├─ /identity/
│ │ ├─ BlogService.ts ├─ /authz/
│ │ └─ BlogModule.ts (implements IModuleInitializer) ├─ /audit/
│ └─ package.json │ └─ /blog/
├─ /internal # ---- Core Kernel (Infrastructure) ----
│ ├─ /config # Configuration management
│ ├─ /logger # Logging system
│ ├─ /di # Dependency injection
│ ├─ /health # Health checks
│ ├─ /metrics # Metrics collection
│ ├─ /observability # OpenTelemetry integration
│ ├─ /registry # Service registry
│ └─ /pluginloader # Module loader
├─ /pkg # ---- Public Interfaces ----
│ ├─ /config # ConfigProvider interface
│ ├─ /logger # Logger interface
│ ├─ /services # Service client interfaces
│ │ ├─ auth.go # AuthServiceClient
│ │ ├─ identity.go # IdentityServiceClient
│ │ ├─ authz.go # AuthzServiceClient
│ │ └─ audit.go # AuditServiceClient
│ └─ /module # IModule interface
├─ /modules # ---- Feature Services ----
│ ├─ /blog/
│ │ ├─ go.mod # Service module
│ │ ├─ module.yaml # Service manifest
│ │ ├─ internal/ # Service implementation
│ │ └─ pkg/
│ │ └─ module.go # IModule implementation
│ │ │ │
│ ├─ /billing │ ├─ /billing/
│ └─ /chat │ └─ /chat/
├─ /infra # ---- Infrastructure adapters ---- ├─ /infra # ---- Infrastructure Adapters ----
│ ├─ /orm (typeorm/hibernate/EFCore etc.)
│ ├─ /cache (redis) │ ├─ /cache (redis)
│ ├─ /queue (rabbit/kafka) │ ├─ /queue (kafka)
│ └─ /storage (s3/azureblob) │ └─ /storage (s3/azureblob)
├─ /gateway (optional APIgateway layer)
├─ /scripts # build / lint / test helpers ├─ /scripts # build / lint / test helpers
├─ /ci ├─ /ci
@@ -168,45 +205,54 @@
├─ /docs ├─ /docs
│ └─ architecture.md │ └─ architecture.md
├─ package.json (or pom.xml / go.mod) ├─ go.mod # Workspace root
└─ README.md └─ README.md
``` ```
### How it boots ### How Services Boot
```ts Each service has its own entry point and bootstraps independently:
// platform-root/src/main.ts
import { createApp } from '@core/app';
import { loadModules } from '@core/module-loader';
import { CoreModule } from '@core';
async function bootstrap() { ```go
const app = await createApp(); // cmd/auth-service/main.go
func main() {
// 1⃣ Load configuration
cfg := config.Load()
// 1️⃣ Load core kernel (DI, config, logger) // 2️⃣ Initialize core kernel (DI, logger, metrics)
await app.register(CoreModule); container := di.NewContainer(cfg)
// 2️⃣ Dynamically discover all `module.yaml` under /modules // 3️⃣ Register service implementations
const modules = await loadModules(__dirname + '/modules'); container.Provide(NewAuthService)
container.Provide(NewTokenProvider)
// 3️⃣ Initialise each module (order can be defined in manifest) // 4️⃣ Register gRPC server
for (const mod of modules) { container.Provide(NewGRPCServer)
await mod.instance.init(app.builder, app.container);
}
// 4️⃣ Start HTTP / gRPC server // 5️⃣ Register with service registry
await app.listen(process.env.PORT || 3000); container.Provide(NewServiceRegistry)
// 6⃣ Start service
container.Start()
} }
bootstrap().catch(err => { // cmd/api-gateway/main.go
console.error('❌ Platform failed to start', err); func main() {
process.exit(1); // API Gateway bootstraps similarly
}); // Routes requests to backend services via service discovery
}
``` ```
Services communicate via service clients:
- All inter-service communication uses gRPC (primary) or HTTP (fallback)
- Service discovery via service registry
- Each service manages its own database connection
- Services can be deployed independently
--- ---
## 6 KEY DECISIONS YOU MUST TAKE EARLY ## KEY DECISIONS YOU MUST TAKE EARLY
| Decision | Options | Implications | | Decision | Options | Implications |
|----------|---------|--------------| |----------|---------|--------------|
@@ -222,7 +268,7 @@ bootstrap().catch(err => {
--- ---
## 7 COMMON PITFALLS & HOW TO AVOID THEM ## COMMON PITFALLS & HOW TO AVOID THEM
| Pitfall | Symptoms | Fix / Guardrail | | Pitfall | Symptoms | Fix / Guardrail |
|---------|----------|-----------------| |---------|----------|-----------------|
@@ -238,49 +284,62 @@ bootstrap().catch(err => {
--- ---
## 8 QUICK START GUIDE (What to Build First) ## QUICK START GUIDE (What to Build First)
1. **Create the Core Kernel** 1. **Create the Core Kernel**
- Set up DI container, config loader, logger, health/metrics endpoint. - Set up DI container, config loader, logger, health/metrics endpoint.
- Scaffold `IUserRepository`, `IPermissionResolver`, `ITokenProvider`. - Infrastructure only - no business logic.
2. **Implement Identity & Auth** 2. **Implement API Gateway**
- Choose JWT + Refresh + optional OpenID Connect. - Request routing to backend services.
- Add password hashing (bcrypt/argon2) and email verification flow. - Authentication at edge, rate limiting, CORS.
- Integration with service discovery.
3. **Add Role/Permission Engine** 3. **Implement Core Services**
- Simple RBAC matrix with an extensible `Permission` type. - **Identity Service**: User CRUD, password hashing, email verification.
- Provide a UI admin UI (or API only) to manage roles. - **Auth Service**: JWT token generation/validation, refresh tokens.
- **Authz Service**: Permission resolution, RBAC/ABAC.
- **Audit Service**: Immutable audit logging.
4. **Set Up Event Bus & Audit** 4. **Set Up Service Communication**
- Publish `user.created`, `role.granted` events. - Define service client interfaces.
- Store audit entries in an appendonly table (or log to Elastic). - Implement gRPC clients (primary) and HTTP clients (fallback).
- Service registry for discovery.
5. **Build the Module Loader** 5. **Set Up Event Bus & Infrastructure**
- Scan `modules/*/module.yaml`, load via `require()`/classpath. - Kafka-based event bus.
- Register each `IModuleInitializer`. - Redis cache.
- Shared infrastructure adapters.
6. **Create a Sample Feature Module** e.g., **Blog** 6. **Build the Module Loader**
- Scan `modules/*/module.yaml` for service modules.
- Register services with service registry.
- Manage service lifecycle.
7. **Create a Sample Feature Service** e.g., **Blog Service**
- Own entry point (`cmd/blog-service/`).
- Own database connection and schema.
- Use service clients for Auth, Identity, Authz.
- Define its own entities (`Post`, `Comment`). - Define its own entities (`Post`, `Comment`).
- Register routes (`/api/v1/blog/posts`).
- Declare required permissions (`blog.post.create`).
7. **Write Integration Tests** 8. **Write Integration Tests**
- Spin up an inmemory DB (SQLite or H2). - Test service interactions via service clients.
- Load core + blog module, assert that a user without `blog.post.create` receives 403. - Spin up services in Docker Compose.
- Test cross-service communication.
8. **Add CI Pipeline** 9. **Add CI Pipeline**
- Lint → Unit → Integration (Docker Compose with DB + Redis). - Build and test each service independently.
- On tag, publish `core` and `blog` packages to your private registry. - Docker images for each service.
- Service deployment automation.
9. **Document Extension Points** 10. **Document Service Architecture**
- Provide a **Developer Handbook** (README + `docs/extension-points.md`). - Service boundaries and responsibilities.
- Service client interfaces.
10. **Iterate** add Notification, Scheduler, Multitenancy, APIGateway as needed. - Deployment and scaling guides.
--- ---
## 9 TOOLS & LIBRARIES (starter suggestions per stack) ## TOOLS & LIBRARIES (starter suggestions per stack)
| Stack | Core | Auth | DI / Module | Event Bus | ORM | Validation | Testing | | Stack | Core | Auth | DI / Module | Event Bus | ORM | Validation | Testing |
|-------|------|------|-------------|-----------|-----|------------|---------| |-------|------|------|-------------|-----------|-----|------------|---------|
@@ -294,18 +353,18 @@ Pick the stack youre most comfortable with; the concepts stay identical.
--- ---
## 🎉 TL;DR What You Must Deliver ## TL;DR What You Must Deliver
| Layer | Musthave components | Why | | Layer | Musthave components | Why |
|-------|----------------------|-----| |-------|----------------------|-----|
| **Core Kernel** | Config, Logger, DI, Health, Metrics, Error Bus | Foundation for any module. | | **Core Kernel** | Config, Logger, DI, Health, Metrics, Error Bus, Observability | Foundation infrastructure for all services. |
| **Security** | Auth (JWT/OIDC), Authorization (RBAC + ABAC), Audit | Guarantees secure, traceable access. | | **API Gateway** | Request routing, authentication, rate limiting, CORS | Single entry point for all external traffic. |
| **User & Role Management** | User CRUD, Password reset, Role ↔ Permission matrix | The “identity” piece everyone will reuse. | | **Core Services** | Identity, Auth, Authz, Audit services (separate services) | Independent, scalable security services. |
| **Extension System** | `IModuleInitializer`, `module.yaml`, EventBus, Permission DSL | Enables plugins without touching core. | | **Service Clients** | gRPC/HTTP clients, service discovery, service registry | Enables service-to-service communication. |
| **Infrastructure Adapters** | DB repo, Cache, Queue, Blob storage, Email/SMS | Keeps core agnostic to any concrete tech. | | **Infrastructure Adapters** | Cache, Event Bus, Blob storage, Email/SMS, Scheduler | Shared infrastructure capabilities. |
| **Observability** | Structured logs, Prometheus metrics, OpenTelemetry traces | You can monitor each module individually. | | **Observability** | Structured logs, Prometheus metrics, OpenTelemetry traces | Cross-service observability and monitoring. |
| **DevOps Boilerplate** | CI pipelines, Dockerfiles, Semanticrelease, Docs | Makes the framework productionready outofthebox. | | **DevOps Boilerplate** | CI pipelines, Dockerfiles, service deployment, Docs | Makes each service productionready. |
| **Sample Feature Module** | (e.g., Blog) to show how to add routes, permissions, DB entities | Provides a reference implementation for future developers. | | **Sample Feature Service** | (e.g., Blog Service) with own entry point, DB, service clients | Provides a reference implementation for future services. |
When you scaffold those pieces **once**, any downstream team can drop a new folder that follows the `module.yaml` contract, implement the initializer, add its own tables & APIs, and instantly get: When you scaffold those pieces **once**, any downstream team can drop a new folder that follows the `module.yaml` contract, implement the initializer, add its own tables & APIs, and instantly get:

View File

@@ -0,0 +1,103 @@
# Complete Task List
This document provides a comprehensive list of all tasks across all epics. Each task has a corresponding detailed file in the epic-specific directories.
## Task Organization
Tasks are organized by epic and section. Each task file follows the naming convention: `{section}.{subtask}-{description}.md`
## Epic 0: Project Setup & Foundation
### Epic 0 Stories
- [0.1 Project Initialization and Repository Structure](./epic0/0.1-project-initialization.md)
- [0.2 Configuration Management System](./epic0/0.2-configuration-management-system.md)
- [0.3 Structured Logging System](./epic0/0.3-structured-logging-system.md)
- [0.4 CI/CD Pipeline and Development Tooling](./epic0/0.4-cicd-pipeline.md)
- [0.5 Dependency Injection and Application Bootstrap](./epic0/0.5-di-and-bootstrap.md)
## Epic 1: Core Kernel & Infrastructure
- [1.1 Enhanced DI Container](./epic1/1.1-enhanced-di-container.md) - Core kernel services only
- [1.2 Database Client Foundation](./epic1/1.2-database-layer.md) - Per-service database connections
- [1.3 Health & Metrics System](./epic1/1.3-health-metrics-system.md)
- [1.4 Error Handling](./epic1/1.4-error-handling.md)
- [1.5 HTTP/gRPC Server Foundation](./epic1/1.5-http-server.md) - Server foundations for services
- [1.6 OpenTelemetry](./epic1/1.6-opentelemetry.md) - Distributed tracing across services
- [1.7 Service Client Interfaces](./epic1/1.7-service-client-interfaces.md) - Service client interfaces
- [1.8 API Gateway Implementation](./epic1/1.8-api-gateway.md) - API Gateway as core infrastructure
- [Epic 1 Overview](./epic1/README.md)
## Epic 2: Core Services (Authentication & Authorization)
- [2.1 Auth Service - JWT Authentication](./epic2/2.1-jwt-authentication.md) - Independent Auth Service
- [2.2 Identity Service - User Management](./epic2/2.2-identity-management.md) - Independent Identity Service
- [2.3 Authz Service - Authorization & RBAC](./epic2/2.3-rbac-system.md) - Independent Authz Service
- [2.4 Role Management (Part of Authz Service)](./epic2/2.4-role-management.md) - Role management gRPC endpoints
- [2.5 Audit Service - Audit Logging](./epic2/2.5-audit-logging.md) - Independent Audit Service
- [2.6 Database Seeding](./epic2/2.6-database-seeding.md) - Per-service seeding
- [Epic 2 Overview](./epic2/README.md)
## Epic 3: Module Framework (Feature Services)
- [3.1 Module System Interface](./epic3/3.1-module-system-interface.md) - Module interface for feature services
- [3.2 Permission Code Generation](./epic3/3.2-permission-code-generation.md)
- [3.3 Service Loader](./epic3/3.3-module-loader.md) - Service initialization helpers
- [3.4 Service Management CLI](./epic3/3.4-module-cli.md) - Service management CLI
- [3.5 Service Registry Verification](./epic3/3.5-service-registry.md) - Verify Consul integration
- [Epic 3 Overview](./epic3/README.md)
## Epic 4: Sample Feature Service (Blog Service)
- [4.1 Complete Blog Service](./epic4/4.1-blog-module.md) - Blog Service as reference implementation
- [Epic 4 Overview](./epic4/README.md)
## Epic 5: Infrastructure Adapters
- [5.1 Cache System](./epic5/5.1-cache-system.md)
- [5.2 Event Bus](./epic5/5.2-event-bus.md)
- [5.3 Blob Storage](./epic5/5.3-blob-storage.md)
- [5.4 Email Notification](./epic5/5.4-email-notification.md)
- [5.5 Scheduler & Jobs](./epic5/5.5-scheduler-jobs.md)
- [5.6 Secret Store](./epic5/5.6-secret-store.md)
- [5.7 Advanced gRPC Features](./epic5/5.7-grpc-services.md) - Streaming, gRPC-Gateway (basic gRPC in Epic 1-2)
- [Epic 5 Overview](./epic5/README.md)
## Epic 6: Observability & Production Readiness
- [6.1 Enhanced Observability](./epic6/6.1-enhanced-observability.md)
- [6.2 Error Reporting](./epic6/6.2-error-reporting.md)
- [6.3 Grafana Dashboards](./epic6/6.3-grafana-dashboards.md)
- [6.4 Rate Limiting](./epic6/6.4-rate-limiting.md)
- [6.5 Security Hardening](./epic6/6.5-security-hardening.md)
- [6.6 Performance Optimization](./epic6/6.6-performance-optimization.md)
- [Epic 6 Overview](./epic6/README.md)
## Epic 7: Testing, Documentation & CI/CD
- [7.1 Testing Suite](./epic7/7.1-testing-suite.md)
- [7.2 Documentation](./epic7/7.2-documentation.md)
- [7.3 CI/CD Enhancement](./epic7/7.3-cicd-enhancement.md)
- [7.4 Docker Deployment](./epic7/7.4-docker-deployment.md)
- [Epic 7 Overview](./epic7/README.md)
## Epic 8: Advanced Features & Polish
- [8.1 OIDC Support](./epic8/8.1-oidc-support.md)
- [8.2 GraphQL API](./epic8/8.2-graphql-api.md)
- [8.3 Additional Sample Feature Services](./epic8/8.3-additional-modules.md) - Notification & Analytics Services
- [8.4 Final Polish](./epic8/8.4-final-polish.md)
- [Epic 8 Overview](./epic8/README.md)
**Note:** API Gateway is now in Epic 1 (Story 1.8) as core infrastructure, not an advanced feature.
## Task Status Tracking
To track task completion:
1. Update the Status field in each task file
2. Update checkboxes in the main plan.md
3. Reference task IDs in commit messages: `[0.1.1] Initialize Go module`
4. Link GitHub issues to tasks if using issue tracking
## Generating Missing Task Files
A script is available to generate task files from plan.md:
```bash
cd docs/tasks
python3 generate_tasks.py
```
Note: Manually review and refine generated task files as needed.

View File

@@ -0,0 +1,65 @@
# Implementation Tasks
This directory contains detailed task definitions for each epic of the Go Platform implementation.
## Task Organization
Tasks are organized by epic, with each major task section having its own detailed file:
### Epic 0: Project Setup & Foundation
- [Epic 0 Tasks](./epic0/README.md) - All Epic 0 tasks
### Epic 1: Core Kernel & Infrastructure
- [Epic 1 Tasks](./epic1/README.md) - All Epic 1 tasks
### Epic 2: Core Services (Authentication & Authorization)
- [Epic 2 Tasks](./epic2/README.md) - Auth, Identity, Authz, Audit as independent services
### Epic 3: Module Framework (Feature Services)
- [Epic 3 Tasks](./epic3/README.md) - Module framework for feature services
### Epic 4: Sample Feature Service (Blog Service)
- [Epic 4 Tasks](./epic4/README.md) - Blog Service as reference implementation
### Epic 5: Infrastructure Adapters
- [Epic 5 Tasks](./epic5/README.md) - All Epic 5 tasks
### Epic 6: Observability & Production Readiness
- [Epic 6 Tasks](./epic6/README.md) - All Epic 6 tasks
### Epic 7: Testing, Documentation & CI/CD
- [Epic 7 Tasks](./epic7/README.md) - All Epic 7 tasks
### Epic 8: Advanced Features & Polish (Optional)
- [Epic 8 Tasks](./epic8/README.md) - All Epic 8 tasks
## Task Status
Each task file includes:
- **Task ID**: Unique identifier (e.g., `0.1.1`)
- **Title**: Descriptive task name
- **Epic**: Implementation epic
- **Status**: Pending | In Progress | Completed | Blocked
- **Priority**: High | Medium | Low
- **Dependencies**: Tasks that must complete first
- **Description**: Detailed requirements
- **Acceptance Criteria**: How to verify completion
- **Implementation Notes**: Technical details and references
- **Related ADRs**: Links to relevant architecture decisions
## Task Tracking
Tasks can be tracked using:
- GitHub Issues (linked from tasks)
- Project boards
- Task management tools
- Direct commit messages referencing task IDs
## Task Naming Convention
Tasks follow the format: `{epic}.{section}.{subtask}`
Example: `0.1.1` = Epic 0, Section 1 (Repository Bootstrap), Subtask 1

View File

@@ -0,0 +1,181 @@
# Story Consolidation Guide
## Overview
The stories have been reworked from granular, task-based items into meaningful, cohesive stories that solve complete problems. Each story now represents a complete feature or capability that can be tested end-to-end.
## Transformation Pattern
### Before (Granular Tasks)
- ❌ "Install dependency X"
- ❌ "Create file Y"
- ❌ "Add function Z"
- ❌ Multiple tiny tasks that don't deliver value alone
### After (Meaningful Stories)
- ✅ "Configuration Management System" - Complete config system with interface, implementation, and files
- ✅ "JWT Authentication System" - Complete auth with tokens, middleware, and endpoints
- ✅ "Database Layer with Ent ORM" - Complete database setup with entities, migrations, and client
## Story Structure
Each consolidated story follows this structure:
1. **Goal** - High-level objective
2. **Description** - What problem it solves
3. **Deliverables** - Complete list of what will be delivered
4. **Acceptance Criteria** - How we know it's done
5. **Implementation Steps** - High-level steps (not micro-tasks)
## Epic 0 - Completed Examples
### 0.1 Project Initialization and Repository Structure
**Consolidates:** Go module init, directory structure, .gitignore, README
### 0.2 Configuration Management System
**Consolidates:** Install viper, create interface, implement loader, create config files
### 0.3 Structured Logging System
**Consolidates:** Install zap, create interface, implement logger, request ID middleware
### 0.4 CI/CD Pipeline
**Consolidates:** GitHub Actions workflow, Makefile creation
### 0.5 DI and Application Bootstrap
**Consolidates:** Install FX, create DI container, create main.go
## Remaining Epics - Consolidation Pattern
### Epic 1: Core Kernel
- **1.1 Enhanced DI Container** - All DI providers and extensions
- **1.2 Database Layer** - Complete Ent setup with entities and migrations
- **1.3 Health & Metrics** - Complete monitoring system
- **1.4 Error Handling** - Complete error bus system
- **1.5 HTTP Server** - Complete server with all middleware
- **1.6 OpenTelemetry** - Complete tracing setup
- **1.7 Service Client Interfaces** - Complete service abstraction layer
### Epic 2: Authentication & Authorization
- **2.1 JWT Authentication System** - Complete auth with tokens
- **2.2 Identity Management** - Complete user lifecycle
- **2.3 RBAC System** - Complete permission system
- **2.4 Role Management API** - Complete role management
- **2.5 Audit Logging** - Complete audit system
- **2.6 Database Seeding** - Complete seeding system
### Epic 3: Module Framework
- **3.1 Module Interface & Registry** - Complete module system
- **3.2 Permission Code Generation** - Complete code gen system
- **3.3 Module Loader** - Complete loading and initialization
- **3.4 Module CLI** - Complete CLI tool
- **3.5 Service Registry** - Complete service discovery
### Epic 4: Sample Blog Module
- **4.1 Complete Blog Module** - Full module with CRUD, permissions, API
### Epic 5: Infrastructure Adapters
- **5.1 Cache System** - Complete Redis cache
- **5.2 Event Bus** - Complete event system
- **5.3 Blob Storage** - Complete S3 storage
- **5.4 Email Notification** - Complete email system
- **5.5 Scheduler & Jobs** - Complete job system
- **5.6 Secret Store** - Complete secret management
- **5.7 gRPC Services** - Complete gRPC service definitions and clients
### Epic 6: Observability
- **6.1 Enhanced OpenTelemetry** - Complete tracing
- **6.2 Error Reporting** - Complete Sentry integration
- **6.3 Enhanced Logging** - Complete log correlation
- **6.4 Metrics Expansion** - Complete metrics
- **6.5 Grafana Dashboards** - Complete dashboards
- **6.6 Rate Limiting** - Complete rate limiting
- **6.7 Security Hardening** - Complete security
- **6.8 Performance Optimization** - Complete optimizations
### Epic 7: Testing & Documentation
- **7.1 Unit Testing** - Complete test suite
- **7.2 Integration Testing** - Complete integration tests
- **7.3 Documentation** - Complete docs
- **7.4 CI/CD Enhancement** - Complete pipeline
- **7.5 Docker & Deployment** - Complete deployment setup
### Epic 8: Advanced Features
- **8.1 OIDC Support** - Complete OIDC
- **8.2 GraphQL API** - Complete GraphQL
- **8.3 Additional Modules** - Complete sample modules
- **8.4 Performance** - Complete optimizations
## Creating New Story Files
When creating story files for remaining epics, follow this template:
```markdown
# Story X.Y: [Meaningful Title]
## Metadata
- **Story ID**: X.Y
- **Title**: [Complete Feature Name]
- **Epic**: X - [Epic Name]
- **Status**: Pending
- **Priority**: High
- **Estimated Time**: [hours]
- **Dependencies**: [story IDs]
## Goal
[High-level objective - what problem does this solve?]
## Description
[What this story delivers as a complete capability]
## Deliverables
- [Complete list of deliverables - not just files, but complete features]
- [Interface definitions]
- [Implementations]
- [API endpoints]
- [Integration points]
## Implementation Steps
1. [High-level step 1]
2. [High-level step 2]
3. [High-level step 3]
## Acceptance Criteria
- [ ] [End-to-end testable criteria]
- [ ] [Feature works completely]
- [ ] [Integration works]
## Related ADRs
- [ADR links]
## Implementation Notes
- [Important considerations]
## Testing
[How to test the complete feature]
## Files to Create/Modify
- [List of files]
```
## Key Principles
1. **Each story solves a complete problem** - Not just "install X" or "create file Y"
2. **Stories are testable end-to-end** - You can verify the complete feature works
3. **Stories deliver business value** - Even infrastructure stories solve complete problems
4. **Stories are independent where possible** - Can be worked on separately
5. **Stories have clear acceptance criteria** - You know when they're done
## Next Steps
1. Update remaining epic README files to reference consolidated stories
2. Create story files for remaining epics following the pattern
3. Update plan.md to complete all epics with story-based structure
4. Remove old granular task files (or archive them)
## Benefits of This Approach
- **Better Planning**: Stories represent complete features
- **Clearer Progress**: You can see complete features being delivered
- **Better Testing**: Each story can be tested end-to-end
- **Reduced Overhead**: Fewer story files to manage
- **More Meaningful**: Stories solve real problems, not just tasks

View File

@@ -31,7 +31,7 @@ Use this template for creating new task files.
- [ ] {Criterion 3} - [ ] {Criterion 3}
## Related ADRs ## Related ADRs
- [ADR-XXXX: {ADR Title}](../../adr/XXXX-adr-title.md) - [ADR-XXXX: {ADR Title}](../adr/XXXX-adr-title.md)
## Implementation Notes ## Implementation Notes
- {Note 1} - {Note 1}

View File

@@ -0,0 +1,162 @@
# Story 0.1: Project Initialization and Repository Structure
## Metadata
- **Story ID**: 0.1
- **Title**: Project Initialization and Repository Structure
- **Epic**: 0 - Project Setup & Foundation
- **Status**: Completed
- **Priority**: High
- **Estimated Time**: 2-3 hours
- **Dependencies**: None
## Goal
Establish a properly structured Go project with all necessary directories, configuration files, and documentation that follows Go best practices and supports the platform's modular architecture.
## Description
This story covers the complete project initialization, including Go module setup, directory structure creation, and initial documentation. The project structure must support the microservices architecture with clear separation between core services, feature services (modules), and infrastructure.
## Deliverables
### 1. Go Module Initialization
- Initialize Go module with correct module path: `git.dcentral.systems/toolz/goplt`
- Set Go version to 1.24 in `go.mod`
- Verify module initialization with `go mod verify`
### 2. Complete Directory Structure
Create the following directory structure:
```
platform/
├── cmd/
│ └── platform/ # Main entry point
├── internal/ # Private implementation code
│ ├── di/ # Dependency injection container
│ ├── registry/ # Module registry
│ ├── pluginloader/ # Plugin loader (optional)
│ ├── config/ # Config implementation
│ ├── logger/ # Logger implementation
│ ├── infra/ # Infrastructure adapters
│ └── ent/ # Ent ORM schemas
├── pkg/ # Public interfaces (exported)
│ ├── config/ # ConfigProvider interface
│ ├── logger/ # Logger interface
│ ├── module/ # IModule interface
│ ├── auth/ # Auth interfaces
│ ├── perm/ # Permission DSL
│ └── infra/ # Infrastructure interfaces
├── modules/ # Feature modules
│ └── blog/ # Sample Blog module (Epic 4)
├── config/ # Configuration files
│ ├── default.yaml
│ ├── development.yaml
│ └── production.yaml
├── api/ # OpenAPI specs
├── scripts/ # Build/test scripts
├── docs/ # Documentation
├── ops/ # Operations (Grafana dashboards, etc.)
├── .github/
│ └── workflows/
│ └── ci.yml
├── Dockerfile
├── docker-compose.yml
├── docker-compose.test.yml
├── .gitignore
├── README.md
└── go.mod
```
### 3. .gitignore Configuration
- Exclude build artifacts (`bin/`, `dist/`)
- Exclude Go build cache
- Exclude IDE files (`.vscode/`, `.idea/`, etc.)
- Exclude test coverage files
- Exclude dependency directories
- Exclude environment-specific files
### 4. Initial README.md
Create comprehensive README with:
- Project overview and purpose
- Architecture overview
- Quick start guide
- Development setup instructions
- Directory structure explanation
- Links to documentation
- Contributing guidelines
### 5. Basic Documentation Structure
- Set up `docs/` directory
- Create architecture documentation placeholder
- Create API documentation structure
## Implementation Steps
1. **Initialize Go Module**
```bash
go mod init git.dcentral.systems/toolz/goplt
```
- Verify `go.mod` is created
- Set Go version constraint
2. **Create Directory Structure**
- Create all directories listed above
- Ensure proper nesting and organization
- Add placeholder `.gitkeep` files in empty directories if needed
3. **Configure .gitignore**
- Add Go-specific ignore patterns
- Add IDE-specific patterns
- Add OS-specific patterns
- Add build artifact patterns
4. **Create README.md**
- Write comprehensive project overview
- Document architecture principles
- Provide setup instructions
- Include example commands
5. **Verify Structure**
- Run `go mod verify`
- Check all directories exist
- Verify .gitignore works
- Test README formatting
## Acceptance Criteria
- [x] `go mod init` creates module with correct path `git.dcentral.systems/toolz/goplt`
- [x] Go version is set to `1.24` in `go.mod`
- [x] All directories from the structure are in place
- [x] `.gitignore` excludes build artifacts, dependencies, and IDE files
- [x] `README.md` provides clear project overview and setup instructions
- [x] Project structure matches architecture documentation
- [x] `go mod verify` passes
- [x] Directory structure follows Go best practices
## Related ADRs
- [ADR-0001: Go Module Path](../../adr/0001-go-module-path.md)
- [ADR-0002: Go Version](../../adr/0002-go-version.md)
- [ADR-0007: Project Directory Structure](../../adr/0007-project-directory-structure.md)
## Implementation Notes
- The module path should match the organization's Git hosting structure
- All internal packages must use `internal/` prefix to ensure they are not importable by external modules
- Public interfaces in `pkg/` should be minimal and well-documented
- Empty directories can have `.gitkeep` files to ensure they are tracked in Git
- The directory structure should be documented in the README
## Testing
```bash
# Verify module initialization
go mod verify
go mod tidy
# Check directory structure
tree -L 3 -a
# Verify .gitignore
git status
```
## Files to Create/Modify
- `go.mod` - Go module definition
- `README.md` - Project documentation
- `.gitignore` - Git ignore patterns
- All directory structure as listed above

View File

@@ -0,0 +1,171 @@
# Story 0.2: Configuration Management System
## Metadata
- **Story ID**: 0.2
- **Title**: Configuration Management System
- **Epic**: 0 - Project Setup & Foundation
- **Status**: Completed
- **Priority**: High
- **Estimated Time**: 4-6 hours
- **Dependencies**: 0.1
## Goal
Implement a flexible configuration system that loads settings from YAML files, environment variables, and supports type-safe access. The system must be injectable via DI and usable by all modules.
## Description
This story implements a complete configuration management system using Viper that provides a clean interface for accessing configuration values. The system supports multiple configuration sources (YAML files, environment variables) with proper precedence and type-safe accessors.
## Deliverables
### 1. Configuration Interface (`pkg/config/config.go`)
Define `ConfigProvider` interface with:
- `Get(key string) any` - Get any value
- `Unmarshal(v any) error` - Unmarshal into struct
- `GetString(key string) string` - Type-safe string getter
- `GetInt(key string) int` - Type-safe int getter
- `GetBool(key string) bool` - Type-safe bool getter
- `GetStringSlice(key string) []string` - Type-safe slice getter
- `GetDuration(key string) time.Duration` - Type-safe duration getter
- `IsSet(key string) bool` - Check if key exists
### 2. Viper Implementation (`internal/config/config.go`)
Implement `ConfigProvider` using Viper:
- Load `config/default.yaml` as baseline
- Merge environment-specific YAML files (development/production)
- Apply environment variable overrides (uppercase with underscores)
- Support nested configuration keys (dot notation)
- Provide all type-safe getters
- Support unmarshaling into structs
- Handle configuration validation errors
### 3. Configuration Loader (`internal/config/loader.go`)
- `LoadConfig(env string) (ConfigProvider, error)` function
- Environment detection (development/production)
- Configuration file discovery
- Validation of required configuration keys
- Error handling and reporting
### 4. Configuration Files
Create configuration files:
**`config/default.yaml`** - Base configuration:
```yaml
environment: development
server:
port: 8080
host: "0.0.0.0"
read_timeout: 30s
write_timeout: 30s
database:
driver: "postgres"
dsn: ""
max_connections: 25
max_idle_connections: 5
logging:
level: "info"
format: "json"
output: "stdout"
```
**`config/development.yaml`** - Development overrides:
```yaml
environment: development
logging:
level: "debug"
format: "console"
```
**`config/production.yaml`** - Production overrides:
```yaml
environment: production
logging:
level: "warn"
format: "json"
```
### 5. DI Integration
- Provider function for ConfigProvider
- Register in DI container
- Make configurable via FX
## Implementation Steps
1. **Install Dependencies**
```bash
go get github.com/spf13/viper@v1.18.0
go get github.com/spf13/cobra@v1.8.0
```
2. **Create Configuration Interface**
- Define `ConfigProvider` interface in `pkg/config/config.go`
- Add package documentation
- Export interface for use by modules
3. **Implement Viper Configuration**
- Create `internal/config/config.go`
- Implement all interface methods
- Handle configuration loading and merging
- Support nested keys and environment variables
4. **Create Configuration Loader**
- Implement `LoadConfig()` function
- Add environment detection logic
- Add validation logic
- Handle errors gracefully
5. **Create Configuration Files**
- Create `config/default.yaml` with base configuration
- Create `config/development.yaml` with dev overrides
- Create `config/production.yaml` with prod overrides
- Ensure proper YAML structure
6. **Integrate with DI**
- Create provider function
- Register in DI container
- Test injection
## Acceptance Criteria
- [x] `ConfigProvider` interface is defined and documented
- [x] Viper implementation loads YAML files successfully
- [x] Environment variables override YAML values
- [x] Type-safe getters work correctly (string, int, bool, etc.)
- [x] Configuration can be unmarshaled into structs
- [x] Nested keys work with dot notation
- [x] Configuration system is injectable via DI container
- [x] All modules can access configuration through interface
- [x] Configuration validation works
- [x] Error handling is comprehensive
## Related ADRs
- [ADR-0004: Configuration Management](../../adr/0004-configuration-management.md)
## Implementation Notes
- Use Viper's automatic environment variable support (uppercase with underscores)
- Support both single-level and nested configuration keys
- Consider adding configuration schema validation in future
- Environment variable format: `SERVER_PORT`, `DATABASE_DSN`, etc.
- Configuration files should be in YAML for readability
- Support secret manager integration placeholder (Epic 6)
## Testing
```bash
# Test configuration loading
go test ./internal/config/...
# Test type-safe getters
go run -c "test config getters"
# Test environment variable overrides
export SERVER_PORT=9090
go run cmd/platform/main.go
```
## Files to Create/Modify
- `pkg/config/config.go` - Configuration interface
- `internal/config/config.go` - Viper implementation
- `internal/config/loader.go` - Configuration loader
- `config/default.yaml` - Base configuration
- `config/development.yaml` - Development configuration
- `config/production.yaml` - Production configuration
- `internal/di/providers.go` - Add config provider

View File

@@ -0,0 +1,139 @@
# Story 0.3: Structured Logging System
## Metadata
- **Story ID**: 0.3
- **Title**: Structured Logging System
- **Epic**: 0 - Project Setup & Foundation
- **Status**: Completed
- **Priority**: High
- **Estimated Time**: 4-6 hours
- **Dependencies**: 0.1, 0.2
## Goal
Implement a production-ready logging system with structured JSON output, request correlation, and configurable log levels that can be used by all modules.
## Description
This story implements a complete logging system using Zap that provides structured logging, request correlation via request IDs, and context-aware logging. The system must support both development (human-readable) and production (JSON) formats.
## Deliverables
### 1. Logger Interface (`pkg/logger/logger.go`)
Define `Logger` interface with:
- `Debug(msg string, fields ...Field)` - Debug level logging
- `Info(msg string, fields ...Field)` - Info level logging
- `Warn(msg string, fields ...Field)` - Warning level logging
- `Error(msg string, fields ...Field)` - Error level logging
- `With(fields ...Field) Logger` - Create child logger with fields
- `WithContext(ctx context.Context) Logger` - Create logger with context fields
- `Field` type for structured fields
- Package-level convenience functions
### 2. Zap Implementation (`internal/logger/zap_logger.go`)
Implement `Logger` interface using Zap:
- Structured JSON logging for production mode
- Human-readable console logging for development mode
- Configurable log levels (debug, info, warn, error)
- Request-scoped fields support
- Context-aware logging (extract request ID, user ID from context)
- Field mapping to Zap fields
- Error stack trace support
### 3. Request ID Middleware (`internal/logger/middleware.go`)
Gin middleware for request correlation:
- Generate unique request ID per HTTP request
- Add request ID to request context
- Add request ID to all logs within request context
- Return request ID in response headers (`X-Request-ID`)
- Support for existing request IDs in headers
### 4. Global Logger Export (`pkg/logger/global.go`)
- Export default logger instance
- Package-level convenience functions
- Thread-safe logger access
### 5. DI Integration
- Provider function for Logger
- Register in DI container
- Make configurable via FX
## Implementation Steps
1. **Install Dependencies**
```bash
go get go.uber.org/zap@v1.26.0
```
2. **Create Logger Interface**
- Define `Logger` interface in `pkg/logger/logger.go`
- Define `Field` type for structured fields
- Add package documentation
3. **Implement Zap Logger**
- Create `internal/logger/zap_logger.go`
- Implement all interface methods
- Support both JSON and console encoders
- Handle log levels and field mapping
4. **Create Request ID Middleware**
- Create `internal/logger/middleware.go`
- Implement Gin middleware
- Generate and propagate request IDs
- Add to response headers
5. **Add Global Logger**
- Create `pkg/logger/global.go`
- Export default logger
- Add convenience functions
6. **Integrate with DI**
- Create provider function
- Register in DI container
- Test injection
## Acceptance Criteria
- [x] `Logger` interface is defined and documented
- [x] Zap implementation supports JSON and console formats
- [x] Log levels are configurable and respected
- [x] Request IDs are generated and included in all logs
- [x] Request ID middleware works with Gin
- [x] Context-aware logging extracts request ID and user ID
- [x] Logger can be injected via DI container
- [x] All modules can use logger through interface
- [x] Request correlation works across service boundaries
- [x] Structured fields work correctly
## Related ADRs
- [ADR-0005: Logging Framework](../../adr/0005-logging-framework.md)
- [ADR-0012: Logger Interface Design](../../adr/0012-logger-interface-design.md)
## Implementation Notes
- Use Zap's production and development presets
- Request IDs should be UUIDs or similar unique identifiers
- Context should be propagated through all service calls
- Log levels should be configurable via configuration system
- Consider adding log sampling for high-volume production
- Support for log rotation and file output (future enhancement)
## Testing
```bash
# Test logger interface
go test ./pkg/logger/...
# Test Zap implementation
go test ./internal/logger/...
# Test request ID middleware
go test ./internal/logger/... -run TestRequestIDMiddleware
```
## Files to Create/Modify
- `pkg/logger/logger.go` - Logger interface
- `pkg/logger/global.go` - Global logger export
- `internal/logger/zap_logger.go` - Zap implementation
- `internal/logger/middleware.go` - Request ID middleware
- `internal/di/providers.go` - Add logger provider
- `config/default.yaml` - Add logging configuration

View File

@@ -0,0 +1,128 @@
# Story 0.4: CI/CD Pipeline and Development Tooling
## Metadata
- **Story ID**: 0.4
- **Title**: CI/CD Pipeline and Development Tooling
- **Epic**: 0 - Project Setup & Foundation
- **Status**: Completed
- **Priority**: High
- **Estimated Time**: 3-4 hours
- **Dependencies**: 0.1
## Goal
Establish automated testing, linting, and build processes with a developer-friendly Makefile that enables efficient development workflow.
## Description
This story sets up the complete CI/CD pipeline using GitHub Actions and provides a comprehensive Makefile with common development commands. The pipeline should run on every push and pull request, ensuring code quality and buildability.
## Deliverables
### 1. GitHub Actions Workflow (`.github/workflows/ci.yml`)
Complete CI pipeline with:
- Go 1.24 setup
- Go module caching for faster builds
- Linting with golangci-lint or staticcheck
- Unit tests execution
- Test coverage reporting
- Binary build validation
- Code formatting validation (gofmt)
- Artifact uploads for build outputs
### 2. Comprehensive Makefile
Developer-friendly Makefile with commands:
- `make test` - Run all tests
- `make test-coverage` - Run tests with coverage report
- `make lint` - Run linters
- `make fmt` - Format code
- `make fmt-check` - Check code formatting
- `make build` - Build platform binary
- `make clean` - Clean build artifacts
- `make docker-build` - Build Docker image
- `make docker-run` - Run Docker container
- `make generate` - Run code generation
- `make verify` - Verify code (fmt, lint, test)
- `make help` - Show available commands
### 3. Linter Configuration
- `.golangci.yml` or similar linter config
- Configured rules and exclusions
- Reasonable defaults for Go projects
### 4. Pre-commit Hooks (Optional)
- Git hooks for formatting and linting
- Prevent committing unformatted code
## Implementation Steps
1. **Create GitHub Actions Workflow**
- Create `.github/workflows/ci.yml`
- Set up Go environment
- Configure module caching
- Add linting step
- Add testing step
- Add build step
- Add artifact uploads
2. **Create Makefile**
- Define common variables (GO, BINARY_NAME, etc.)
- Add test target
- Add lint target
- Add build target
- Add format targets
- Add Docker targets
- Add help target
3. **Configure Linter**
- Install golangci-lint or configure staticcheck
- Create linter configuration file
- Set up reasonable rules
4. **Test CI Pipeline**
- Push changes to trigger CI
- Verify all steps pass
- Check artifact uploads
## Acceptance Criteria
- [x] CI pipeline runs on every push and PR
- [x] All linting checks pass
- [x] Tests run successfully (even if empty initially)
- [x] Binary builds successfully
- [x] Docker image builds successfully
- [x] Makefile commands work as expected
- [x] CI pipeline fails fast on errors
- [x] Code formatting is validated
- [x] Test coverage is reported
- [x] Artifacts are uploaded correctly
## Related ADRs
- [ADR-0010: CI/CD Platform](../../adr/0010-ci-cd-platform.md)
## Implementation Notes
- Use Go 1.24 in CI to match project requirements
- Cache Go modules to speed up CI runs
- Use golangci-lint for comprehensive linting
- Set up test coverage threshold (e.g., 80%)
- Make sure CI fails on any error
- Consider adding security scanning (gosec) in future
- Docker builds should use multi-stage builds
## Testing
```bash
# Test Makefile commands
make test
make lint
make build
make clean
# Test CI locally (using act or similar)
act push
```
## Files to Create/Modify
- `.github/workflows/ci.yml` - GitHub Actions workflow
- `Makefile` - Development commands
- `.golangci.yml` - Linter configuration (optional)
- `.git/hooks/pre-commit` - Pre-commit hooks (optional)

View File

@@ -0,0 +1,125 @@
# Story 0.5: Dependency Injection and Application Bootstrap
## Metadata
- **Story ID**: 0.5
- **Title**: Dependency Injection and Application Bootstrap
- **Epic**: 0 - Project Setup & Foundation
- **Status**: Completed
- **Priority**: High
- **Estimated Time**: 4-5 hours
- **Dependencies**: 0.1, 0.2, 0.3
## Goal
Set up dependency injection container using Uber FX and create the application entry point that initializes the platform with proper lifecycle management.
## Description
This story implements the dependency injection system using Uber FX and creates the main application entry point. The DI container will manage service lifecycle, dependencies, and provide a clean way to wire services together.
## Deliverables
### 1. DI Container (`internal/di/container.go`)
FX-based dependency injection container:
- Initialize FX container
- Register Config and Logger providers
- Basic lifecycle hooks (OnStart, OnStop)
- Support service overrides for testing
- Graceful shutdown handling
### 2. DI Providers (`internal/di/providers.go`)
Provider functions for core services:
- `ProvideConfig() fx.Option` - Configuration provider
- `ProvideLogger() fx.Option` - Logger provider
- Provider functions return FX options for easy composition
### 3. Application Entry Point (`cmd/platform/main.go`)
Main application bootstrap:
- Load configuration
- Initialize DI container with core services
- Set up basic application lifecycle
- Start minimal HTTP server (placeholder for Epic 1)
- Handle graceful shutdown (SIGINT, SIGTERM)
- Proper error handling and logging
### 4. Core Module (`internal/di/core_module.go`)
Optional: Export core module as FX option:
- `CoreModule() fx.Option` - Provides all core services
- Easy to compose with future modules
## Implementation Steps
1. **Install Dependencies**
```bash
go get go.uber.org/fx@latest
```
2. **Create DI Container**
- Create `internal/di/container.go`
- Initialize FX app
- Set up lifecycle hooks
- Add graceful shutdown
3. **Create Provider Functions**
- Create `internal/di/providers.go`
- Implement `ProvideConfig()` function
- Implement `ProvideLogger()` function
- Return FX options
4. **Create Application Entry Point**
- Create `cmd/platform/main.go`
- Load configuration
- Initialize FX app with providers
- Set up signal handling
- Start minimal server (placeholder)
- Handle shutdown gracefully
5. **Test Application**
- Verify application starts
- Verify graceful shutdown works
- Test service injection
## Acceptance Criteria
- [x] DI container initializes successfully
- [x] Config and Logger are provided via DI
- [x] Application starts and runs
- [x] Application shuts down gracefully on signals
- [x] Lifecycle hooks work correctly
- [x] Services can be overridden for testing
- [x] Application compiles and runs successfully
- [x] Error handling is comprehensive
- [x] Logging works during startup/shutdown
## Related ADRs
- [ADR-0003: Dependency Injection Framework](../../adr/0003-dependency-injection-framework.md)
## Implementation Notes
- Use FX for dependency injection and lifecycle management
- Support graceful shutdown with context cancellation
- Handle SIGINT and SIGTERM signals
- Log startup and shutdown events
- Make services easily testable via interfaces
- Consider adding health check endpoint in future
- Support for service overrides is important for testing
## Testing
```bash
# Test application startup
go run cmd/platform/main.go
# Test graceful shutdown
# Start app, then send SIGTERM
kill -TERM <pid>
# Test DI container
go test ./internal/di/...
```
## Files to Create/Modify
- `internal/di/container.go` - DI container
- `internal/di/providers.go` - Provider functions
- `internal/di/core_module.go` - Core module (optional)
- `cmd/platform/main.go` - Application entry point
- `go.mod` - Add FX dependency

View File

@@ -0,0 +1,50 @@
# Epic 0: Project Setup & Foundation
## Overview
Initialize repository structure with proper Go project layout, implement configuration management system, establish structured logging system, set up CI/CD pipeline and development tooling, and bootstrap dependency injection and application entry point.
## Stories
### 0.1 Project Initialization and Repository Structure
- [Story: 0.1 - Project Initialization](./0.1-project-initialization.md)
- **Goal:** Establish a properly structured Go project with all necessary directories, configuration files, and documentation.
- **Deliverables:** Go module initialization, complete directory structure, .gitignore, comprehensive README.md
### 0.2 Configuration Management System
- [Story: 0.2 - Configuration Management System](./0.2-configuration-management-system.md)
- **Goal:** Implement a flexible configuration system that loads settings from YAML files, environment variables, and supports type-safe access.
- **Deliverables:** ConfigProvider interface, Viper implementation, configuration files, DI integration
### 0.3 Structured Logging System
- [Story: 0.3 - Structured Logging System](./0.3-structured-logging-system.md)
- **Goal:** Implement a production-ready logging system with structured JSON output, request correlation, and configurable log levels.
- **Deliverables:** Logger interface, Zap implementation, request ID middleware, context-aware logging
### 0.4 CI/CD Pipeline and Development Tooling
- [Story: 0.4 - CI/CD Pipeline and Development Tooling](./0.4-cicd-pipeline.md)
- **Goal:** Establish automated testing, linting, and build processes with a developer-friendly Makefile.
- **Deliverables:** GitHub Actions workflow, comprehensive Makefile, build automation
### 0.5 Dependency Injection and Application Bootstrap
- [Story: 0.5 - Dependency Injection and Application Bootstrap](./0.5-di-and-bootstrap.md)
- **Goal:** Set up dependency injection container using Uber FX and create the application entry point that initializes the platform.
- **Deliverables:** DI container, FX providers, application entry point, lifecycle management
## Deliverables Checklist
- [x] Repository structure in place
- [x] Configuration system loads YAML files and env vars
- [x] Structured logging works
- [x] CI pipeline runs linting and builds binary
- [x] Basic DI container initialized
## Acceptance Criteria
- `go build ./cmd/platform` succeeds
- `go test ./...` runs (even if tests are empty)
- CI pipeline passes on empty commit
- Config loads from `config/default.yaml`
- Logger can be injected and used
- Application starts and shuts down gracefully
## Implementation Summary
- [Implementation Summary and Verification Instructions](./SUMMARY.md) - Complete guide on how to verify all Epic 0 functionality

View File

@@ -0,0 +1,152 @@
# Epic 0: Implementation Summary
## Overview
Epic 0 establishes the foundation of the Go Platform project with core infrastructure components that enable all future development. This epic includes project initialization, configuration management, structured logging, CI/CD pipeline, and dependency injection setup.
## Completed Stories
### ✅ 0.1 Project Initialization
- Go module initialized with proper module path
- Complete directory structure following Clean Architecture
- `.gitignore` configured for Go projects
- Comprehensive README with project overview
### ✅ 0.2 Configuration Management System
- `ConfigProvider` interface in `pkg/config/`
- Viper-based implementation in `internal/config/`
- YAML configuration files in `config/` directory
- Environment variable support with automatic mapping
- Type-safe configuration access methods
### ✅ 0.3 Structured Logging System
- `Logger` interface in `pkg/logger/`
- Zap-based implementation in `internal/logger/`
- JSON and console output formats
- Configurable log levels
- Request ID and context-aware logging support
### ✅ 0.4 CI/CD Pipeline
- GitHub Actions workflow for automated testing and linting
- Comprehensive Makefile with common development tasks
- Automated build and test execution
### ✅ 0.5 Dependency Injection and Bootstrap
- DI container using Uber FX in `internal/di/`
- Provider functions for core services
- Application entry point in `cmd/platform/main.go`
- Lifecycle management with graceful shutdown
## Verification Instructions
### Prerequisites
- Go 1.24 or later installed
- Make installed (optional, for using Makefile commands)
### 1. Verify Project Structure
```bash
# Check Go module
go mod verify
# Check directory structure
ls -la
# Should see: cmd/, internal/, pkg/, config/, docs/, etc.
```
### 2. Verify Configuration System
```bash
# Build the application
go build ./cmd/platform
# Check if config files exist
ls -la config/
# Should see: default.yaml, development.yaml, production.yaml
# Test config loading (will fail without database, but config should load)
# This will be tested in Epic 1 when database is available
```
### 3. Verify Logging System
```bash
# Run tests for logging
go test ./internal/logger/...
# Expected output: Tests should pass
```
### 4. Verify CI/CD Pipeline
```bash
# Run linting (if golangci-lint is installed)
make lint
# Run tests
make test
# Build the application
make build
# Binary should be created in bin/platform
# Run all checks
make check
```
### 5. Verify Dependency Injection
```bash
# Build the application
go build ./cmd/platform
# Check if DI container compiles
go build ./internal/di/...
# Run the application (will fail without database in Epic 1)
# go run ./cmd/platform/main.go
```
### 6. Verify Application Bootstrap
```bash
# Build the application
make build
# Check if binary exists
ls -la bin/platform
# The application should be ready to run (database connection will be tested in Epic 1)
```
## Testing Configuration
The configuration system can be tested by:
1. **Modifying config files**: Edit `config/default.yaml` and verify changes are loaded
2. **Environment variables**: Set `ENVIRONMENT=production` and verify production config is loaded
3. **Type safety**: Configuration access methods (`GetString`, `GetInt`, etc.) provide compile-time safety
## Testing Logging
The logging system can be tested by:
1. **Unit tests**: Run `go test ./internal/logger/...`
2. **Integration**: Logging will be tested in Epic 1 when HTTP server is available
3. **Format switching**: Change `logging.format` in config to switch between JSON and console output
## Common Issues and Solutions
### Issue: `go mod verify` fails
**Solution**: Run `go mod tidy` to update dependencies
### Issue: Build fails
**Solution**: Ensure Go 1.24+ is installed and all dependencies are downloaded (`go mod download`)
### Issue: Config not loading
**Solution**: Ensure `config/default.yaml` exists and is in the correct location relative to the binary
## Next Steps
After verifying Epic 0, proceed to [Epic 1](../epic1/SUMMARY.md) to set up the database and HTTP server, which will enable full end-to-end testing of the configuration and logging systems.

Some files were not shown because too many files have changed in this diff Show More