toolz/goplt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: Align documentation with true microservices architecture

Browse Source 
Transform all documentation from modular monolith to true microservices
architecture where core services are independently deployable.

Key Changes:
- Core Kernel: Infrastructure only (no business logic)
- Core Services: Auth, Identity, Authz, Audit as separate microservices
  - Each service has own entry point (cmd/{service}/)
  - Each service has own gRPC server and database schema
  - Services register with Consul for service discovery
- API Gateway: Moved from Epic 8 to Epic 1 as core infrastructure
  - Single entry point for all external traffic
  - Handles routing, JWT validation, rate limiting, CORS
- Service Discovery: Consul as primary mechanism (ADR-0033)
- Database Pattern: Per-service connections with schema isolation

Documentation Updates:
- Updated all 9 architecture documents
- Updated 4 ADRs and created 2 new ADRs (API Gateway, Service Discovery)
- Rewrote Epic 1: Core Kernel & Infrastructure (infrastructure only)
- Rewrote Epic 2: Core Services (Auth, Identity, Authz, Audit as services)
- Updated Epic 3-8 stories for service architecture
- Updated plan.md, playbook.md, requirements.md, index.md
- Updated all epic READMEs and story files

New ADRs:
- ADR-0032: API Gateway Strategy
- ADR-0033: Service Discovery Implementation (Consul)

New Stories:
- Epic 1.7: Service Client Interfaces
- Epic 1.8: API Gateway Implementation
This commit is contained in:
0x1d
2025-11-06 08:47:27 +01:00
parent cab7cadf9e
commit 38a251968c
47 changed files with 3190 additions and 1613 deletions
Download Patch File Download Diff File Expand all files Collapse all files

236
docs/content/requirements.md
View File

@@ -6,7 +6,7 @@
|-----------|----------------------------------------|-------------------|
| **Separation of Concerns (SoC)** | Keeps core services (auth, audit, config) independent from business modules. | Use **layered** or **hexagonal/clean‑architecture** boundaries. |
| **Domain‑Driven Design (DDD) Bounded Contexts** | Allows each module to own its own model & rules while sharing a common identity kernel. | Define a **Core Context** (Identity, Security, Infrastructure) and **Feature Contexts** (Billing, CMS, Chat, …). |
| **Modular Monolith → Micro‑service‑ready** | Start simple (single process) but keep each module in its own package so you can later split to services if needed. | Package each module as an **independent library** with its own **DI container‑module** and **routing**. |
| **microMicroservices Architecture** | Each service is independently deployable from day one. Services communicate via gRPC/HTTP through service clients. | Each service has its own entry point (`cmd/{service}/`), database connection, and deployment configuration. |
| **Plug‑in / Extension‑point model** | Enables customers or internal teams to drop new features without touching core code. | Export **well‑defined interfaces** (e.g., `IUserProvider`, `IPermissionResolver`, `IModuleInitializer`). |
| **API‑First** | Guarantees that any UI (web, mobile, CLI) can be built on top of the same contract. | Publish **OpenAPI/GraphQL schema** as part of the build artefact. |
| **Security‑by‑Design** | The platform will hold user credentials, roles and possibly PII. | Centralize **authentication**, **authorization**, **audit**, **rate‑limiting**, **CORS**, **CSP**, **secure defaults**. |
@@ -50,7 +50,17 @@
---
## REQUIRED BASE MODULES (THE “CORE KERNEL”)
## CORE KERNEL (INFRASTRUCTURE ONLY)
The core kernel provides foundational infrastructure services that all other services depend on. It contains **no business logic** and is purely infrastructure.
## CORE SERVICES (INDEPENDENT MICROSERVICES)
Core services provide business logic and are deployed as separate, independently scalable services. Each service has its own database connection, API endpoints, and deployment configuration.
## INFRASTRUCTURE ADAPTERS (SHARED SERVICES)
These adapters provide infrastructure capabilities that services can use.
| Module | Core responsibilities | Public API / Extension points |
|--------|-----------------------|--------------------------------|
@@ -128,38 +138,65 @@
```
/platform-root
│
├─ /core # ---- Kernel / Base modules ----
│ ├─ /auth
│ │ ├─ src/
│ │ └─ package.json
│ ├─ /identity
│ ├─ /authorization
│ ├─ /audit
│ ├─ /config
│ ├─ /logging
│ ├─ /metrics
│ └─ index.ts (exports all core APIs)
├─ /cmd # ---- Service Entry Points ----
│ ├─ /api-gateway # API Gateway service
│ │ └─ main.go
│ ├─ /auth-service # Auth service
│ │ └─ main.go
│ ├─ /identity-service # Identity service
│ │ └─ main.go
│ ├─ /authz-service # Authorization service
│ │ └─ main.go
│ ├─ /audit-service # Audit service
│ │ └─ main.go
│ └─ /blog-service # Blog feature service
│ └─ main.go
│
├─ /modules # ---- Feature plug‑ins ----
│ ├─ /blog
│ │ ├─ module.yaml # manifest
│ │ ├─ src/
│ │ │ ├─ BlogController.ts
│ │ │ ├─ BlogService.ts
│ │ │ └─ BlogModule.ts (implements IModuleInitializer)
│ │ └─ package.json
├─ /services # ---- Service Implementations ----
│ ├─ /auth/
│ │ ├─ internal/ # Service implementation
│ │ └─ api/ # gRPC/HTTP definitions
│ ├─ /identity/
│ ├─ /authz/
│ ├─ /audit/
│ └─ /blog/
│
├─ /internal # ---- Core Kernel (Infrastructure) ----
│ ├─ /config # Configuration management
│ ├─ /logger # Logging system
│ ├─ /di # Dependency injection
│ ├─ /health # Health checks
│ ├─ /metrics # Metrics collection
│ ├─ /observability # OpenTelemetry integration
│ ├─ /registry # Service registry
│ └─ /pluginloader # Module loader
│
├─ /pkg # ---- Public Interfaces ----
│ ├─ /config # ConfigProvider interface
│ ├─ /logger # Logger interface
│ ├─ /services # Service client interfaces
│ │ ├─ auth.go # AuthServiceClient
│ │ ├─ identity.go # IdentityServiceClient
│ │ ├─ authz.go # AuthzServiceClient
│ │ └─ audit.go # AuditServiceClient
│ └─ /module # IModule interface
│
├─ /modules # ---- Feature Services ----
│ ├─ /blog/
│ │ ├─ go.mod # Service module
│ │ ├─ module.yaml # Service manifest
│ │ ├─ internal/ # Service implementation
│ │ └─ pkg/
│ │ └─ module.go # IModule implementation
│ │
│ ├─ /billing
│ └─ /chat
│ ├─ /billing/
│ └─ /chat/
│
├─ /infra # ---- Infrastructure adapters ----
│ ├─ /orm (typeorm/hibernate/EFCore etc.)
├─ /infra # ---- Infrastructure Adapters ----
│ ├─ /cache (redis)
│ ├─ /queue (rabbit/kafka)
│ ├─ /queue (kafka)
│ └─ /storage (s3/azure‑blob)
│
├─ /gateway (optional API‑gateway layer)
│
├─ /scripts # build / lint / test helpers
│
├─ /ci
@@ -168,42 +205,50 @@
├─ /docs
│ └─ architecture.md
│
├─ package.json (or pom.xml / go.mod)
├─ go.mod # Workspace root
└─ README.md
```
### How it boots
### How Services Boot
```ts
// platform-root/src/main.ts
import { createApp } from '@core/app';
import { loadModules } from '@core/module-loader';
import { CoreModule } from '@core';
Each service has its own entry point and bootstraps independently:
async function bootstrap() {
const app = await createApp();
// 1️⃣ Load core kernel (DI, config, logger)
await app.register(CoreModule);
// 2️⃣ Dynamically discover all `module.yaml` under /modules
const modules = await loadModules(__dirname + '/modules');
// 3️⃣ Initialise each module (order can be defined in manifest)
for (const mod of modules) {
await mod.instance.init(app.builder, app.container);
}
// 4️⃣ Start HTTP / gRPC server
await app.listen(process.env.PORT || 3000);
```go
// cmd/auth-service/main.go
func main() {
// 1️⃣ Load configuration
cfg := config.Load()
// 2️⃣ Initialize core kernel (DI, logger, metrics)
container := di.NewContainer(cfg)
// 3️⃣ Register service implementations
container.Provide(NewAuthService)
container.Provide(NewTokenProvider)
// 4️⃣ Register gRPC server
container.Provide(NewGRPCServer)
// 5️⃣ Register with service registry
container.Provide(NewServiceRegistry)
// 6️⃣ Start service
container.Start()
}
bootstrap().catch(err => {
console.error('❌ Platform failed to start', err);
process.exit(1);
});
// cmd/api-gateway/main.go
func main() {
// API Gateway bootstraps similarly
// Routes requests to backend services via service discovery
}
```
Services communicate via service clients:
- All inter-service communication uses gRPC (primary) or HTTP (fallback)
- Service discovery via service registry
- Each service manages its own database connection
- Services can be deployed independently
---
## KEY DECISIONS YOU MUST TAKE EARLY
@@ -242,41 +287,54 @@ bootstrap().catch(err => {
1. **Create the Core Kernel**
- Set up DI container, config loader, logger, health/metrics endpoint.
- Scaffold `IUserRepository`, `IPermissionResolver`, `ITokenProvider`.
- Infrastructure only - no business logic.
2. **Implement Identity & Auth**
- Choose JWT + Refresh + optional OpenID Connect.
- Add password hashing (bcrypt/argon2) and email verification flow.
2. **Implement API Gateway**
- Request routing to backend services.
- Authentication at edge, rate limiting, CORS.
- Integration with service discovery.
3. **Add Role/Permission Engine**
- Simple RBAC matrix with an extensible `Permission` type.
- Provide a UI admin UI (or API only) to manage roles.
3. **Implement Core Services**
- **Identity Service**: User CRUD, password hashing, email verification.
- **Auth Service**: JWT token generation/validation, refresh tokens.
- **Authz Service**: Permission resolution, RBAC/ABAC.
- **Audit Service**: Immutable audit logging.
4. **Set Up Event Bus & Audit**
- Publish `user.created`, `role.granted` events.
- Store audit entries in an append‑only table (or log to Elastic).
4. **Set Up Service Communication**
- Define service client interfaces.
- Implement gRPC clients (primary) and HTTP clients (fallback).
- Service registry for discovery.
5. **Build the Module Loader**
- Scan `modules/*/module.yaml`, load via `require()`/classpath.
- Register each `IModuleInitializer`.
5. **Set Up Event Bus & Infrastructure**
- Kafka-based event bus.
- Redis cache.
- Shared infrastructure adapters.
6. **Create a Sample Feature Module** – e.g., **Blog**
6. **Build the Module Loader**
- Scan `modules/*/module.yaml` for service modules.
- Register services with service registry.
- Manage service lifecycle.
7. **Create a Sample Feature Service** – e.g., **Blog Service**
- Own entry point (`cmd/blog-service/`).
- Own database connection and schema.
- Use service clients for Auth, Identity, Authz.
- Define its own entities (`Post`, `Comment`).
- Register routes (`/api/v1/blog/posts`).
- Declare required permissions (`blog.post.create`).
7. **Write Integration Tests**
- Spin up an in‑memory DB (SQLite or H2).
- Load core + blog module, assert that a user without `blog.post.create` receives 403.
8. **Write Integration Tests**
- Test service interactions via service clients.
- Spin up services in Docker Compose.
- Test cross-service communication.
8. **Add CI Pipeline**
- Lint → Unit → Integration (Docker Compose with DB + Redis).
- On tag, publish `core` and `blog` packages to your private registry.
9. **Add CI Pipeline**
- Build and test each service independently.
- Docker images for each service.
- Service deployment automation.
9. **Document Extension Points**
- Provide a **Developer Handbook** (README + `docs/extension-points.md`).
10. **Iterate** – add Notification, Scheduler, Multitenancy, API‑Gateway as needed.
10. **Document Service Architecture**
- Service boundaries and responsibilities.
- Service client interfaces.
- Deployment and scaling guides.
---
@@ -298,14 +356,14 @@ Pick the stack you’re most comfortable with; the concepts stay identical.
| Layer | Must‑have components | Why |
|-------|----------------------|-----|
| **Core Kernel** | Config, Logger, DI, Health, Metrics, Error Bus | Foundation for any module. |
| **Security** | Auth (JWT/OIDC), Authorization (RBAC + ABAC), Audit | Guarantees secure, traceable access. |
| **User & Role Management** | User CRUD, Password reset, Role ↔ Permission matrix | The “identity” piece everyone will reuse. |
| **Extension System** | `IModuleInitializer`, `module.yaml`, EventBus, Permission DSL | Enables plug‑ins without touching core. |
| **Infrastructure Adapters** | DB repo, Cache, Queue, Blob storage, Email/SMS | Keeps core agnostic to any concrete tech. |
| **Observability** | Structured logs, Prometheus metrics, OpenTelemetry traces | You can monitor each module individually. |
| **DevOps Boilerplate** | CI pipelines, Dockerfiles, Semantic‑release, Docs | Makes the framework production‑ready out‑of‑the‑box. |
| **Sample Feature Module** | (e.g., Blog) to show how to add routes, permissions, DB entities | Provides a reference implementation for future developers. |
| **Core Kernel** | Config, Logger, DI, Health, Metrics, Error Bus, Observability | Foundation infrastructure for all services. |
| **API Gateway** | Request routing, authentication, rate limiting, CORS | Single entry point for all external traffic. |
| **Core Services** | Identity, Auth, Authz, Audit services (separate services) | Independent, scalable security services. |
| **Service Clients** | gRPC/HTTP clients, service discovery, service registry | Enables service-to-service communication. |
| **Infrastructure Adapters** | Cache, Event Bus, Blob storage, Email/SMS, Scheduler | Shared infrastructure capabilities. |
| **Observability** | Structured logs, Prometheus metrics, OpenTelemetry traces | Cross-service observability and monitoring. |
| **DevOps Boilerplate** | CI pipelines, Dockerfiles, service deployment, Docs | Makes each service production‑ready. |
| **Sample Feature Service** | (e.g., Blog Service) with own entry point, DB, service clients | Provides a reference implementation for future services. |
When you scaffold those pieces **once**, any downstream team can drop a new folder that follows the `module.yaml` contract, implement the initializer, add its own tables & APIs, and instantly get: