toolz/goplt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: reword phase to epic, update mkdocs

Browse Source
This commit is contained in:
0x1d
2025-11-05 09:28:33 +01:00
parent 65a428534c
commit ace9678f6c
64 changed files with 214 additions and 208 deletions
Download Patch File Download Diff File Expand all files Collapse all files

64
docs/content/stories/epic5/5.1-cache-system.md Normal file
View File

@@ -0,0 +1,64 @@
# Story 5.1: Cache System (Redis)
## Metadata
- **Story ID**: 5.1
- **Title**: Cache System (Redis)
- **Epic**: 5 - Infrastructure Adapters
- **Status**: Pending
- **Priority**: High
- **Estimated Time**: 4-5 hours
- **Dependencies**: 1.1
## Goal
Implement a complete Redis-based caching system with a clean interface that can be swapped for other cache implementations.
## Description
This story implements a Redis cache adapter with a clean interface, allowing modules to cache data efficiently. The cache system supports TTL, key-based operations, and optional cache middleware for HTTP responses.
## Deliverables
### 1. Cache Interface (`pkg/infra/cache/cache.go`)
- `Cache` interface with:
- `Get(ctx context.Context, key string) ([]byte, error)`
- `Set(ctx context.Context, key string, value []byte, ttl time.Duration) error`
- `Delete(ctx context.Context, key string) error`
- `Exists(ctx context.Context, key string) (bool, error)`
- `Clear(ctx context.Context) error`
### 2. Redis Implementation (`internal/infra/cache/redis_cache.go`)
- Redis client setup
- Connection pooling
- All interface methods implemented
- Error handling
- Connection health checks
### 3. Configuration
- Redis config in `config/default.yaml`:
- Connection URL
- Pool settings
- Default TTL
### 4. DI Integration
- Provider function for Cache
- Register in DI container
### 5. Optional Cache Middleware
- HTTP response caching middleware
- Configurable cache keys
- TTL per route
## Acceptance Criteria
- [ ] Cache interface is defined
- [ ] Redis implementation works correctly
- [ ] Cache operations (get, set, delete) work
- [ ] TTL is respected
- [ ] Cache is injectable via DI
- [ ] Configuration is loaded from config
- [ ] Optional middleware works
## Files to Create/Modify
- `pkg/infra/cache/cache.go` - Cache interface
- `internal/infra/cache/redis_cache.go` - Redis implementation
- `internal/di/providers.go` - Add cache provider
- `config/default.yaml` - Add Redis config

69
docs/content/stories/epic5/5.2-event-bus.md Normal file
View File

@@ -0,0 +1,69 @@
# Story 5.2: Event Bus System
## Metadata
- **Story ID**: 5.2
- **Title**: Event Bus System
- **Epic**: 5 - Infrastructure Adapters
- **Status**: Pending
- **Priority**: High
- **Estimated Time**: 6-8 hours
- **Dependencies**: 1.1
## Goal
Implement a complete event bus system supporting both in-process (for development/testing) and Kafka (for production) with publish/subscribe capabilities.
## Description
This story implements an event bus that allows modules to publish and subscribe to events. It supports both in-process channels for development and Kafka for production, with a clean interface that makes the implementation swappable.
## Deliverables
### 1. Event Bus Interface (`pkg/eventbus/eventbus.go`)
- `EventBus` interface with:
- `Publish(ctx context.Context, topic string, event Event) error`
- `Subscribe(topic string, handler EventHandler) error`
- `Unsubscribe(topic string) error`
- `Event` and `EventHandler` types
### 2. In-Process Bus (`internal/infra/bus/inprocess_bus.go`)
- Channel-based in-process bus
- Used for testing and development
- Thread-safe implementation
### 3. Kafka Bus (`internal/infra/bus/kafka_bus.go`)
- Kafka producer for publishing
- Consumer groups for subscribing
- Error handling and retries
- Connection management
### 4. Core Events
- Define core platform events:
- `platform.user.created`
- `platform.user.updated`
- `platform.role.assigned`
- `platform.permission.granted`
### 5. Configuration
- Kafka config in `config/default.yaml`
- Bus selection (in-process vs Kafka)
### 6. DI Integration
- Provider function for EventBus
- Register in DI container
- Switchable via config
## Acceptance Criteria
- [ ] Event bus interface is defined
- [ ] In-process bus works for development
- [ ] Kafka bus works for production
- [ ] Events can be published and subscribed
- [ ] Bus is swappable via config
- [ ] Error handling works correctly
- [ ] Core events are defined
## Files to Create/Modify
- `pkg/eventbus/eventbus.go` - Event bus interface
- `internal/infra/bus/inprocess_bus.go` - In-process implementation
- `internal/infra/bus/kafka_bus.go` - Kafka implementation
- `internal/di/providers.go` - Add event bus provider
- `config/default.yaml` - Add Kafka config

65
docs/content/stories/epic5/5.3-blob-storage.md Normal file
View File

@@ -0,0 +1,65 @@
# Story 5.3: Blob Storage System
## Metadata
- **Story ID**: 5.3
- **Title**: Blob Storage System
- **Epic**: 5 - Infrastructure Adapters
- **Status**: Pending
- **Priority**: Medium
- **Estimated Time**: 5-6 hours
- **Dependencies**: 1.1, 1.5
## Goal
Implement a complete blob storage system using S3 with a clean interface for file upload, download, and management.
## Description
This story implements S3-based blob storage with support for file uploads, downloads, signed URLs, and file deletion. It includes an API endpoint for file uploads.
## Deliverables
### 1. Blob Storage Interface (`pkg/infra/blob/blob.go`)
- `BlobStore` interface with:
- `Upload(ctx context.Context, key string, data []byte) error`
- `Download(ctx context.Context, key string) ([]byte, error)`
- `Delete(ctx context.Context, key string) error`
- `GetSignedURL(ctx context.Context, key string, ttl time.Duration) (string, error)`
- `Exists(ctx context.Context, key string) (bool, error)`
### 2. S3 Implementation (`internal/infra/blob/s3_store.go`)
- AWS S3 client setup
- All interface methods implemented
- Error handling
- Content type detection
### 3. File Upload API
- `POST /api/v1/files/upload` - Upload file
- File validation
- Size limits
- Content type validation
### 4. Configuration
- S3 config in `config/default.yaml`:
- Bucket name
- Region
- Credentials (or use IAM role)
### 5. DI Integration
- Provider function for BlobStore
- Register in DI container
## Acceptance Criteria
- [ ] Blob storage interface is defined
- [ ] S3 implementation works correctly
- [ ] Files can be uploaded and downloaded
- [ ] Signed URLs are generated correctly
- [ ] File upload API works
- [ ] Configuration is loaded from config
- [ ] Blob store is injectable via DI
## Files to Create/Modify
- `pkg/infra/blob/blob.go` - Blob storage interface
- `internal/infra/blob/s3_store.go` - S3 implementation
- `internal/infra/blob/handler.go` - File upload handler
- `internal/di/providers.go` - Add blob store provider
- `config/default.yaml` - Add S3 config

67
docs/content/stories/epic5/5.4-email-notification.md Normal file
View File

@@ -0,0 +1,67 @@
# Story 5.4: Email Notification System
## Metadata
- **Story ID**: 5.4
- **Title**: Email Notification System
- **Epic**: 5 - Infrastructure Adapters
- **Status**: Pending
- **Priority**: High
- **Estimated Time**: 5-6 hours
- **Dependencies**: 1.1, 2.2
## Goal
Implement a complete email notification system with SMTP support, HTML email templates, and integration with identity management.
## Description
This story implements email notifications using SMTP with support for HTML emails and templates. It integrates with the identity service to send verification and password reset emails.
## Deliverables
### 1. Notification Interface (`pkg/notification/notification.go`)
- `Notifier` interface with:
- `SendEmail(ctx context.Context, to, subject, body string) error`
- `SendHTMLEmail(ctx context.Context, to, subject, htmlBody, textBody string) error`
- `SendSMS(ctx context.Context, to, message string) error` (placeholder)
### 2. SMTP Implementation (`internal/infra/email/smtp_notifier.go`)
- SMTP client setup
- HTML email support
- Email templates for:
- Email verification
- Password reset
- Welcome email
### 3. Integration with Identity Service
- Send verification email on registration
- Send password reset email
- Send welcome email
### 4. Configuration
- Email config in `config/default.yaml`:
- SMTP server
- Port
- Username/password
- From address
### 5. DI Integration
- Provider function for Notifier
- Register in DI container
## Acceptance Criteria
- [ ] Notification interface is defined
- [ ] SMTP implementation works correctly
- [ ] HTML emails are sent successfully
- [ ] Email templates work
- [ ] Verification emails are sent on registration
- [ ] Password reset emails are sent
- [ ] Configuration is loaded from config
- [ ] Notifier is injectable via DI
## Files to Create/Modify
- `pkg/notification/notification.go` - Notification interface
- `internal/infra/email/smtp_notifier.go` - SMTP implementation
- `internal/infra/email/templates.go` - Email templates
- `internal/identity/user_service.go` - Integrate email sending
- `internal/di/providers.go` - Add notifier provider
- `config/default.yaml` - Add email config

74
docs/content/stories/epic5/5.5-scheduler-jobs.md Normal file
View File

@@ -0,0 +1,74 @@
# Story 5.5: Scheduler and Background Jobs System
## Metadata
- **Story ID**: 5.5
- **Title**: Scheduler and Background Jobs System
- **Epic**: 5 - Infrastructure Adapters
- **Status**: Pending
- **Priority**: High
- **Estimated Time**: 6-8 hours
- **Dependencies**: 1.1, 5.1
## Goal
Implement a complete scheduler and background job system with cron jobs, job queues, retries, and job status tracking.
## Description
This story implements a scheduler system using Asynq (Redis-backed) that supports cron jobs for periodic tasks and job queues for background processing. Jobs can be registered from modules and tracked.
## Deliverables
### 1. Scheduler Interface (`pkg/scheduler/scheduler.go`)
- `Scheduler` interface with:
- `Cron(spec string, job JobFunc) error` - Schedule cron job
- `Enqueue(queue string, payload any) error` - Enqueue job
- `RegisterJob(name string, handler JobHandler) error` - Register job handler
### 2. Asynq Implementation (`internal/infra/scheduler/asynq_scheduler.go`)
- Redis-backed job queue
- Cron jobs for periodic tasks
- Job retries and backoff
- Job status tracking
- Job result storage
### 3. Job Registry (`internal/infra/scheduler/job_registry.go`)
- Register jobs from modules
- Start job processor on app startup
- Job lifecycle management
### 4. Example Jobs
- Cleanup expired tokens (daily)
- Send digest emails (weekly)
- Database cleanup tasks
### 5. Job Monitoring API
- `GET /api/v1/jobs/status` - Job status endpoint
- Job history and statistics
### 6. Configuration
- Scheduler config in `config/default.yaml`:
- Redis connection (shared with cache)
- Concurrency settings
- Retry settings
### 7. DI Integration
- Provider function for Scheduler
- Register in DI container
## Acceptance Criteria
- [ ] Scheduler interface is defined
- [ ] Cron jobs can be scheduled
- [ ] Jobs can be enqueued
- [ ] Jobs are processed correctly
- [ ] Job retries work
- [ ] Job status is tracked
- [ ] Example jobs run on schedule
- [ ] Job monitoring API works
## Files to Create/Modify
- `pkg/scheduler/scheduler.go` - Scheduler interface
- `internal/infra/scheduler/asynq_scheduler.go` - Asynq implementation
- `internal/infra/scheduler/job_registry.go` - Job registry
- `internal/infra/scheduler/jobs.go` - Example jobs
- `internal/di/providers.go` - Add scheduler provider
- `config/default.yaml` - Add scheduler config

68
docs/content/stories/epic5/5.6-secret-store.md Normal file
View File

@@ -0,0 +1,68 @@
# Story 5.6: Secret Store Integration
## Metadata
- **Story ID**: 5.6
- **Title**: Secret Store Integration
- **Epic**: 5 - Infrastructure Adapters
- **Status**: Pending
- **Priority**: Medium
- **Estimated Time**: 4-5 hours
- **Dependencies**: 0.2
## Goal
Implement secret store integration supporting HashiCorp Vault and AWS Secrets Manager for secure secret management.
## Description
This story implements secret store adapters that can retrieve secrets from external secret management systems, with integration into the configuration system.
## Deliverables
### 1. Secret Store Interface (`pkg/infra/secret/secret.go`)
- `SecretStore` interface with:
- `GetSecret(ctx context.Context, key string) (string, error)`
- `GetSecrets(ctx context.Context, prefix string) (map[string]string, error)`
### 2. Vault Implementation (`internal/infra/secret/vault_store.go`)
- HashiCorp Vault client
- Support KV v2 secrets
- Authentication (token, app role)
- Secret caching
### 3. AWS Secrets Manager (`internal/infra/secret/aws_secrets.go`)
- AWS Secrets Manager client
- Secret retrieval
- Secret caching
### 4. Configuration Integration
- Integrate with config loader
- Overlay secrets on top of file/env config
- Load secrets lazily (cache)
- Secret key resolution
### 5. Configuration
- Secret store config in `config/default.yaml`:
- Provider (vault, aws, none)
- Connection settings
- Cache settings
### 6. DI Integration
- Provider function for SecretStore
- Register in DI container (optional, via config)
## Acceptance Criteria
- [ ] Secret store interface is defined
- [ ] Vault implementation works
- [ ] AWS Secrets Manager implementation works
- [ ] Secrets are loaded into config
- [ ] Secret caching works
- [ ] Configuration integration works
- [ ] Secret store is optional (can be disabled)
## Files to Create/Modify
- `pkg/infra/secret/secret.go` - Secret store interface
- `internal/infra/secret/vault_store.go` - Vault implementation
- `internal/infra/secret/aws_secrets.go` - AWS implementation
- `internal/config/loader.go` - Integrate secret loading
- `internal/di/providers.go` - Add secret store provider
- `config/default.yaml` - Add secret store config

150
docs/content/stories/epic5/5.7-grpc-services.md Normal file
View File

@@ -0,0 +1,150 @@
# Story 5.7: gRPC Service Definitions and Clients
## Metadata
- **Story ID**: 5.7
- **Title**: gRPC Service Definitions and Clients
- **Epic**: 5 - Infrastructure Adapters
- **Status**: Pending
- **Priority**: Medium
- **Estimated Time**: 8-10 hours
- **Dependencies**: 1.7, 3.5
## Goal
Implement gRPC service definitions and clients to enable microservices communication, allowing modules to be extracted as independent services.
## Description
This story implements gRPC service definitions for core services and gRPC clients that implement the service client interfaces. This enables modules to communicate with services over the network when deployed as microservices.
## Deliverables
### 1. gRPC Service Definitions (`api/proto/`)
- Define Protocol Buffer files for core services:
- `identity.proto` - Identity service
- `auth.proto` - Authentication service
- `authz.proto` - Authorization service
- `permission.proto` - Permission service
- `audit.proto` - Audit service
- Use protobuf v3
- Include proper message definitions
- Include service definitions
### 2. gRPC Server Implementations (`internal/services/grpc/server/`)
- Implement gRPC servers for each service:
- `identity_server.go` - Identity gRPC server
- `auth_server.go` - Auth gRPC server
- `authz_server.go` - Authz gRPC server
- Server implementations wrap existing services
- Error handling and validation
- Request/response conversion
### 3. gRPC Client Implementations (`internal/services/grpc/client/`)
- Implement gRPC clients that satisfy service client interfaces:
- `grpc_identity_client.go` - Identity gRPC client
- `grpc_auth_client.go` - Auth gRPC client
- `grpc_authz_client.go` - Authz gRPC client
- Connection pooling
- Retry logic
- Circuit breaker support
- Timeout handling
### 4. gRPC Server Setup
- gRPC server initialization
- Service registration
- Health check service
- Reflection service (development)
- Integration with HTTP server (gRPC-Gateway optional)
### 5. Code Generation
- `Makefile` target for protobuf generation
- Generate Go code from `.proto` files
- Generate gRPC server and client stubs
### 6. Configuration
- gRPC configuration in `config/default.yaml`:
```yaml
grpc:
enabled: false # Enable gRPC server
port: 9090
reflection: true # Enable reflection (dev)
```
### 7. Integration
- Integrate with service factory
- Support switching between local and gRPC clients
- Service registry integration for gRPC services
## Implementation Steps
1. **Install Dependencies**
```bash
go get google.golang.org/grpc
go get google.golang.org/protobuf
go install google.golang.org/protobuf/cmd/protoc-gen-go
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc
```
2. **Define Protocol Buffers**
- Create `api/proto/` directory
- Define `.proto` files for each service
- Define messages and services
3. **Generate gRPC Code**
- Create `Makefile` target
- Generate Go code from protobuf
4. **Implement gRPC Servers**
- Create server implementations
- Wrap existing services
- Handle errors and validation
5. **Implement gRPC Clients**
- Create client implementations
- Implement service client interfaces
- Add connection management
6. **Integrate with Service Factory**
- Update factory to support gRPC clients
- Add gRPC server startup
## Acceptance Criteria
- [ ] gRPC service definitions are created
- [ ] gRPC servers are implemented
- [ ] gRPC clients implement service interfaces
- [ ] Service factory can create gRPC clients
- [ ] gRPC services can be enabled via configuration
- [ ] Code generation works
- [ ] gRPC clients work with service registry
## Related ADRs
- [ADR-0029: Microservices Architecture](../../adr/0029-microservices-architecture.md)
- [ADR-0030: Service Communication Strategy](../../adr/0030-service-communication-strategy.md)
## Implementation Notes
- Use protobuf v3
- Support both unary and streaming RPCs
- Implement proper error handling
- Add OpenTelemetry instrumentation
- Support service versioning
## Testing
```bash
# Generate protobuf code
make generate-proto
# Test gRPC servers
go test ./internal/services/grpc/server/...
# Test gRPC clients
go test ./internal/services/grpc/client/...
```
## Files to Create/Modify
- `api/proto/identity.proto` - Identity service definition
- `api/proto/auth.proto` - Auth service definition
- `api/proto/authz.proto` - Authz service definition
- `internal/services/grpc/server/` - gRPC server implementations
- `internal/services/grpc/client/` - gRPC client implementations
- `internal/services/factory.go` - Add gRPC client support
- `Makefile` - Add protobuf generation
- `config/default.yaml` - Add gRPC configuration

58
docs/content/stories/epic5/README.md Normal file
View File

@@ -0,0 +1,58 @@
# Epic 5: Infrastructure Adapters
## Overview
Implement infrastructure adapters (cache, queue, blob storage, email), make adapters swappable via interfaces, add scheduler/background jobs system, and implement event bus (in-process and Kafka).
## Stories
### 5.1 Cache System (Redis)
- [Story: 5.1 - Cache System](./5.1-cache-system.md)
- **Goal:** Implement a complete Redis-based caching system with a clean interface.
- **Deliverables:** Cache interface, Redis implementation, configuration, DI integration
### 5.2 Event Bus System
- [Story: 5.2 - Event Bus](./5.2-event-bus.md)
- **Goal:** Implement a complete event bus system supporting both in-process and Kafka.
- **Deliverables:** Event bus interface, in-process bus, Kafka bus, core events
### 5.3 Blob Storage System
- [Story: 5.3 - Blob Storage](./5.3-blob-storage.md)
- **Goal:** Implement a complete blob storage system using S3.
- **Deliverables:** Blob storage interface, S3 implementation, file upload API
### 5.4 Email Notification System
- [Story: 5.4 - Email Notification](./5.4-email-notification.md)
- **Goal:** Implement a complete email notification system with SMTP support.
- **Deliverables:** Notification interface, SMTP implementation, email templates, identity integration
### 5.5 Scheduler and Background Jobs System
- [Story: 5.5 - Scheduler & Jobs](./5.5-scheduler-jobs.md)
- **Goal:** Implement a complete scheduler and background job system.
- **Deliverables:** Scheduler interface, Asynq implementation, job registry, example jobs
### 5.6 Secret Store Integration
- [Story: 5.6 - Secret Store](./5.6-secret-store.md)
- **Goal:** Implement secret store integration supporting Vault and AWS Secrets Manager.
- **Deliverables:** Secret store interface, Vault implementation, AWS implementation, config integration
### 5.7 gRPC Service Definitions and Clients
- [Story: 5.7 - gRPC Services](./5.7-grpc-services.md)
- **Goal:** Implement gRPC service definitions and clients to enable microservices communication.
- **Deliverables:** gRPC service definitions, gRPC servers, gRPC clients, code generation
## Deliverables Checklist
- [ ] Cache adapter (Redis) working
- [ ] Event bus (in-process and Kafka) functional
- [ ] Blob storage (S3) adapter
- [ ] Email notification system
- [ ] Scheduler and background jobs
- [ ] Secret store integration (optional)
- [ ] gRPC service definitions and clients
## Acceptance Criteria
- Cache stores and retrieves data correctly
- Events are published and consumed
- Files can be uploaded and downloaded
- Email notifications are sent
- Background jobs run on schedule
- Integration test: full infrastructure stack works