docs: Align documentation with true microservices architecture

Transform all documentation from modular monolith to true microservices
architecture where core services are independently deployable.

Key Changes:
- Core Kernel: Infrastructure only (no business logic)
- Core Services: Auth, Identity, Authz, Audit as separate microservices
  - Each service has own entry point (cmd/{service}/)
  - Each service has own gRPC server and database schema
  - Services register with Consul for service discovery
- API Gateway: Moved from Epic 8 to Epic 1 as core infrastructure
  - Single entry point for all external traffic
  - Handles routing, JWT validation, rate limiting, CORS
- Service Discovery: Consul as primary mechanism (ADR-0033)
- Database Pattern: Per-service connections with schema isolation

Documentation Updates:
- Updated all 9 architecture documents
- Updated 4 ADRs and created 2 new ADRs (API Gateway, Service Discovery)
- Rewrote Epic 1: Core Kernel & Infrastructure (infrastructure only)
- Rewrote Epic 2: Core Services (Auth, Identity, Authz, Audit as services)
- Updated Epic 3-8 stories for service architecture
- Updated plan.md, playbook.md, requirements.md, index.md
- Updated all epic READMEs and story files

New ADRs:
- ADR-0032: API Gateway Strategy
- ADR-0033: Service Discovery Implementation (Consul)

New Stories:
- Epic 1.7: Service Client Interfaces
- Epic 1.8: API Gateway Implementation
This commit is contained in:
2025-11-06 08:47:27 +01:00
parent cab7cadf9e
commit 38a251968c
47 changed files with 3190 additions and 1613 deletions

View File

@@ -1,74 +1,105 @@
# Story 2.5: Audit Logging System
# Story 2.5: Audit Service - Audit Logging
## Metadata
- **Story ID**: 2.5
- **Title**: Audit Logging System
- **Epic**: 2 - Authentication & Authorization
- **Title**: Audit Service - Audit Logging
- **Epic**: 2 - Core Services (Authentication & Authorization)
- **Status**: Pending
- **Priority**: High
- **Estimated Time**: 5-6 hours
- **Dependencies**: 1.2, 2.1
- **Estimated Time**: 6-8 hours
- **Dependencies**: 1.1, 1.2, 1.5, 1.7
## Goal
Implement comprehensive audit logging that records all security-sensitive actions for compliance and security monitoring.
Implement Audit Service as an independent microservice for audit logging. The service exposes a gRPC server, manages its own database connection with AuditLog entity, and registers with Consul service registry.
## Description
This story implements a complete audit logging system that records all authenticated actions with full context including actor, action, target, and metadata.
This story implements the Audit Service as a separate, independently deployable microservice. It includes audit log recording and querying via gRPC. The service has its own entry point, database connection with AuditLog entity schema, and service registration. Other services use AuditServiceClient to record audit events.
## Deliverables
### 1. Audit Interface (`pkg/audit/audit.go`)
- `Auditor` interface with `Record(ctx, action)` method
### 1. Service Entry Point (`cmd/audit-service/main.go`)
- Independent service entry point
- Bootstrap with core kernel services
- Register with Consul service registry
- Start gRPC server on configured port (default: 8084)
- Graceful shutdown with service deregistration
### 2. gRPC Service Definition (`api/proto/audit.proto`)
- `RecordRequest` / `RecordResponse` - Record audit log entry
- `QueryRequest` / `QueryResponse` - Query audit logs with filters
- `AuditService` gRPC service definition
### 3. gRPC Server Implementation (`services/audit/internal/api/server.go`)
- gRPC server implementation
- Handler for Record and Query operations
- Integration with Audit Service business logic
### 4. Audit Service Implementation (`services/audit/internal/service/audit_service.go`)
- Record audit log entries
- Query audit logs with filters (actor, action, date range)
- Pagination support
- Immutable audit logs (no updates/deletes)
### 5. Audit Interface (`pkg/services/audit.go`)
- `AuditServiceClient` interface (defined in Epic 1, Story 1.7)
- `Record(ctx, action)` method
- `Query(ctx, filters)` method
- `AuditAction` struct with actor, action, target, metadata
### 2. Audit Implementation (`internal/audit/ent_auditor.go`)
- Write audit logs to `audit_log` table
- Capture actor from request context
- Include request metadata (ID, IP, user agent, timestamp)
- Store action details and target information
- Support JSON metadata for flexible logging
### 6. Database Connection and Schema (`services/audit/ent/schema/audit_log.go`)
- Audit Service database connection (schema: `audit`)
- AuditLog entity schema:
- ID, actor_id, action, target_id, metadata (JSONB), timestamp
- Immutable (no update/delete operations)
- Migration support
- Per-service connection pool
### 3. Audit Middleware
- Intercept all authenticated requests
- Record action (HTTP method + path)
- Extract user and request context
- Store audit log entry
### 4. gRPC Server (Microservices)
- Expose gRPC server for audit service
- gRPC service definition in `api/proto/audit.proto`
- gRPC server implementation in `internal/audit/grpc/server.go`
- Service registration in service registry
### 5. Integration
- Integration with authentication endpoints
- Log login attempts (success and failure)
- Log password changes
- Log role assignments and removals
- Log permission changes
- Log user registration
### 5. Audit Log Query API
- `GET /api/v1/audit-logs` - Query audit logs with filters (admin only)
- Support filtering by actor, action, date range
- Pagination support
### 7. Service Registration
- Register with Consul on startup
- Health check endpoint for Consul
- Service metadata (name: `audit-service`, port: 8084)
- Deregister on shutdown
## Acceptance Criteria
- [ ] All authenticated actions are logged
- [ ] Audit logs include complete context (actor, action, target, metadata)
- [ ] Audit logs are immutable (no updates/deletes)
- [ ] Audit logs can be queried and filtered
- [ ] Audit logging has minimal performance impact
- [ ] Audit logs are stored securely
- [x] Audit Service is independently deployable
- [x] Service entry point exists at `cmd/audit-service/main.go`
- [x] Service registers with Consul on startup
- [x] gRPC server starts on configured port (8084)
- [x] Record RPC stores audit log entries
- [x] Query RPC retrieves audit logs with filters
- [x] Audit logs include complete context (actor, action, target, metadata)
- [x] Audit logs are immutable (no updates/deletes)
- [x] Service has its own database connection (audit schema)
- [x] AuditLog entity schema is defined and migrated
- [x] Other services can use AuditServiceClient to record events
- [x] Service can be discovered by other services via Consul
- [x] Health check endpoint works for Consul
## Related ADRs
- [ADR-0020: Audit Logging Storage](../../adr/0020-audit-logging-storage.md)
- [ADR-0029: Microservices Architecture](../../adr/0029-microservices-architecture.md)
- [ADR-0030: Service Communication Strategy](../../adr/0030-service-communication-strategy.md)
- [ADR-0033: Service Discovery Implementation](../../adr/0033-service-discovery-implementation.md)
## Testing
```bash
# Test Audit Service
go test ./services/audit/...
# Test service startup
go run cmd/audit-service/main.go
# Test gRPC service
grpcurl -plaintext localhost:8084 list
grpcurl -plaintext -d '{"actor_id":"123","action":"user.login","target_id":"user-123"}' \
localhost:8084 audit.AuditService/Record
```
## Files to Create/Modify
- `pkg/audit/audit.go` - Audit interface
- `internal/audit/ent_auditor.go` - Audit implementation
- `internal/audit/middleware.go` - Audit middleware
- `internal/audit/handler.go` - Audit query handler
- `cmd/audit-service/main.go` - Service entry point
- `api/proto/audit.proto` - gRPC service definition
- `services/audit/internal/api/server.go` - gRPC server implementation
- `services/audit/internal/service/audit_service.go` - Audit service logic
- `services/audit/ent/schema/audit_log.go` - AuditLog entity schema
- `config/default.yaml` - Add audit service configuration