378 lines
22 KiB
Markdown
378 lines
22 KiB
Markdown
# Requirements
|
||
|
||
## HIGH‑LEVEL ARCHITECTURAL PRINCIPLES
|
||
|
||
| Principle | Why it matters for a modular platform | How to enforce it |
|
||
|-----------|----------------------------------------|-------------------|
|
||
| **Separation of Concerns (SoC)** | Keeps core services (auth, audit, config) independent from business modules. | Use **layered** or **hexagonal/clean‑architecture** boundaries. |
|
||
| **Domain‑Driven Design (DDD) Bounded Contexts** | Allows each module to own its own model & rules while sharing a common identity kernel. | Define a **Core Context** (Identity, Security, Infrastructure) and **Feature Contexts** (Billing, CMS, Chat, …). |
|
||
| **microMicroservices Architecture** | Each service is independently deployable from day one. Services communicate via gRPC/HTTP through service clients. | Each service has its own entry point (`cmd/{service}/`), database connection, and deployment configuration. |
|
||
| **Plug‑in / Extension‑point model** | Enables customers or internal teams to drop new features without touching core code. | Export **well‑defined interfaces** (e.g., `IUserProvider`, `IPermissionResolver`, `IModuleInitializer`). |
|
||
| **API‑First** | Guarantees that any UI (web, mobile, CLI) can be built on top of the same contract. | Publish **OpenAPI/GraphQL schema** as part of the build artefact. |
|
||
| **Security‑by‑Design** | The platform will hold user credentials, roles and possibly PII. | Centralize **authentication**, **authorization**, **audit**, **rate‑limiting**, **CORS**, **CSP**, **secure defaults**. |
|
||
| **Observability** | You need to know when a plug‑in crashes or misbehaves. | Provide **structured logging**, **metrics**, **tracing**, **health‑checks**, **central error bus**. |
|
||
| **CI/CD‑Ready Boilerplate** | Keeps the framework maintainable and encourages best‑practice adoption. | Include **GitHub Actions / GitLab CI** pipelines that run lint, unit, integration, contract tests, and a publish step. |
|
||
| **Configuration‑as‑Code** | Each module may need its own settings but you want a single source of truth. | Use a **hierarchical config system** (environment > file > secret store). |
|
||
| **Multitenancy (optional)** | If you plan SaaS, the core must be ready to isolate data per tenant. | Provide **tenant‑aware repository abstractions** and **tenant‑scoped middlewares**. |
|
||
| **Versioning & Compatibility** | Modules evolve; the core should never break existing plug‑ins. | Adopt **semantic versioning** + **backwards‑compatibility shim layer** (e.g., deprecation warnings). |
|
||
|
||
---
|
||
|
||
## LAYERED / HEXAGONAL BLUEPRINT
|
||
|
||
```
|
||
+---------------------------------------------------+
|
||
| Presentation |
|
||
| (REST/GraphQL/ gRPC Controllers, UI SDKs, CLI) |
|
||
+-------------------+-------------------------------+
|
||
| Application Services (Use‑cases) |
|
||
| - orchestrate core & feature modules |
|
||
| - enforce policies (RBAC, rate‑limit) |
|
||
+-------------------+-------------------------------+
|
||
| Domain (Business Logic) |
|
||
| - Core Entities (User, Role, Permission) |
|
||
| - Domain Services (PasswordPolicy, TokenMgr) |
|
||
+-------------------+-------------------------------+
|
||
| Infrastructure / Adapters |
|
||
| - Persistence (ORM/NoSQL Repositories) |
|
||
| - External services (Email, SMS, OIDC) |
|
||
| - Event bus (Kafka/Rabbit, In‑process) |
|
||
| - File storage, Cache, Search |
|
||
+---------------------------------------------------+
|
||
| Platform Core (Kernel) |
|
||
| - DI container, Module loader, Config manager |
|
||
| - Cross‑cutting concerns (Logging, Metrics) |
|
||
| - Security subsystem (AuthN/AuthZ, Token Issuer)|
|
||
+---------------------------------------------------+
|
||
```
|
||
|
||
*All layers talk to each other **through interfaces** defined in the Core kernel. Feature modules implement those interfaces and register themselves at startup.*
|
||
|
||
---
|
||
|
||
## CORE KERNEL (INFRASTRUCTURE ONLY)
|
||
|
||
The core kernel provides foundational infrastructure services that all other services depend on. It contains **no business logic** and is purely infrastructure.
|
||
|
||
## CORE SERVICES (INDEPENDENT MICROSERVICES)
|
||
|
||
Core services provide business logic and are deployed as separate, independently scalable services. Each service has its own database connection, API endpoints, and deployment configuration.
|
||
|
||
## INFRASTRUCTURE ADAPTERS (SHARED SERVICES)
|
||
|
||
These adapters provide infrastructure capabilities that services can use.
|
||
|
||
| Module | Core responsibilities | Public API / Extension points |
|
||
|--------|-----------------------|--------------------------------|
|
||
| **Identity‑Management** | - User CRUD (local + federation) <br>- Password hashing, reset flow <br>- Email/Phone verification | `IUserRepository`, `IUserService`, `IExternalIdpProvider` |
|
||
| **Roles & Permissions** | - Role hierarchy (role → permission set) <br>- Permission definitions (string, enum, policy) <br>- Dynamic permission evaluation (ABAC) | `IPermissionResolver`, `IRoleService` |
|
||
| **Authentication** | - JWT / OAuth2 Access & Refresh tokens <br>- OpenID Connect Provider (optional) <br>- Session Management (stateless + optional stateful) | `IAuthService`, `ITokenProvider` |
|
||
| **Authorization Middleware** | - Enforce RBAC/ABAC on each request <br>- Policy DSL (e.g., `hasPermission('project.read') && resource.ownerId == user.id`) | `IAuthorizationHandler` |
|
||
| **Audit & Activity Log** | - Immutable log of security‑relevant actions <br>- Correlation IDs, actor, target, timestamp | `IAuditSink`, `IAuditService` |
|
||
| **Configuration** | - Hierarchical sources (env, files, secret manager) <br>- Validation schema (JSON‑Schema / Yup / JSR‑380) | `IConfigProvider` |
|
||
| **Health & Metrics** | - `/healthz`, `/ready`, `/metrics` endpoints <br>- Export to Prometheus, Grafana, CloudWatch | `IHealthCheck`, `IMetricsRegistry` |
|
||
| **Error Handling** | - Centralized error objects, stack trace masking <br>- Automatic mapping to HTTP status <br>- Exception‑to‑event publishing | `IErrorMapper`, `IErrorBus` |
|
||
| **Logging** | - Structured JSON logs (level, requestId, userId) <br>- pluggable sinks (stdout, file, ELK, Cloud Logging) | `ILoggerFactory` |
|
||
| **Notification** | - Email, SMS, Push (FCM/APNs), Webhooks <br>- Queue‑backed delivery, retries | `INotificationService` |
|
||
| **File / Blob Storage** | - Abstracted bucket API (upload, version, signed URLs) | `IBlobStore` |
|
||
| **Scheduler / Background Jobs** | - Cron‑like tasks, queue workers, retry/back‑off policies | `IJobScheduler`, `IJobProcessor` |
|
||
| **Internationalization (i18n)** | - Message catalog, locale negotiation, runtime translation | `I18nService` |
|
||
| **API Gateway (optional)** | - Rate‑limit, request/response transformation, API‑key handling, request routing to modules | `IGatewayPlugin` |
|
||
| **Multitenancy (optional)** | - Tenant identification (sub‑domain, header, JWT claim) <br>- Tenant‑scoped data isolation primitives | `ITenantResolver`, `ITenantContext` |
|
||
|
||
> **Tip:** Pack each module as a **separate NPM/ Maven / NuGet / Go module** with its own `package.json` / `pom.xml` etc. The platform’s **bootstrapper** loads every module that implements `IModuleInitializer` (or similar) and calls `ConfigureServices` / `RegisterRoutes`.
|
||
|
||
---
|
||
|
||
## EXTENSION‑POINT DESIGN (HOW PLUG‑INS HOOK IN)
|
||
|
||
1. **Module Manifest** – a tiny JSON/YAML file (`module.yaml`) that declares:
|
||
- Module name, version, dependencies (core ≥ 1.2.0, other modules)
|
||
- Public routes (e.g., `/api/v1/blog/**`)
|
||
- Required permissions (auto‑generated from source annotations)
|
||
- UI assets (static folder, React component entry point)
|
||
|
||
2. **Bootstrap Interface**
|
||
```ts
|
||
export interface IModuleInitializer {
|
||
/**
|
||
* Called during platform start‑up.
|
||
* Register services, routes, policies, background jobs.
|
||
*/
|
||
init(app: IApplicationBuilder, container: IServiceContainer): Promise<void>;
|
||
}
|
||
```
|
||
|
||
3. **Dependency Injection (DI) Conventions**
|
||
- Core registers **contracts** (`IUserRepository`, `IPermissionResolver`) as **singletons**.
|
||
- Modules register **implementations** with a **named scope** (e.g., `UserRepository:Local`).
|
||
- Override is possible via **module ordering** or explicit `container.override(...)`.
|
||
|
||
4. **Policy/Permission Extension**
|
||
```ts
|
||
// core lib
|
||
export type Permission = `${string}.${string}`; // e.g., "blog.post.create"
|
||
|
||
// module
|
||
export const BLOG_PERMS = {
|
||
POST_CREATE: 'blog.post.create',
|
||
POST_READ: 'blog.post.read',
|
||
POST_UPDATE: 'blog.post.update',
|
||
POST_DELETE: 'blog.post.delete',
|
||
} as const;
|
||
```
|
||
|
||
5. **Event Bus & Hooks**
|
||
- Central **topic**: `platform.*` (user.created, role.assigned, tenant.created)
|
||
- Modules can **publish** and **subscribe** via `IEventBus`.
|
||
- Provide **synchronous guard hooks** (`beforeUserCreate`, `afterRoleDelete`) for validation & side‑effects.
|
||
|
||
6. **UI Plug‑in System**
|
||
- Serve a **manifest** at `/modules` that front‑end bundles read to render navigation.
|
||
- Encourage **Web Component / Module Federation** pattern for SPA integration.
|
||
|
||
---
|
||
|
||
## SAMPLE REPOSITORY LAYOUT (language‑agnostic)
|
||
|
||
```
|
||
/platform-root
|
||
│
|
||
├─ /cmd # ---- Service Entry Points ----
|
||
│ ├─ /api-gateway # API Gateway service
|
||
│ │ └─ main.go
|
||
│ ├─ /auth-service # Auth service
|
||
│ │ └─ main.go
|
||
│ ├─ /identity-service # Identity service
|
||
│ │ └─ main.go
|
||
│ ├─ /authz-service # Authorization service
|
||
│ │ └─ main.go
|
||
│ ├─ /audit-service # Audit service
|
||
│ │ └─ main.go
|
||
│ └─ /blog-service # Blog feature service
|
||
│ └─ main.go
|
||
│
|
||
├─ /services # ---- Service Implementations ----
|
||
│ ├─ /auth/
|
||
│ │ ├─ internal/ # Service implementation
|
||
│ │ └─ api/ # gRPC/HTTP definitions
|
||
│ ├─ /identity/
|
||
│ ├─ /authz/
|
||
│ ├─ /audit/
|
||
│ └─ /blog/
|
||
│
|
||
├─ /internal # ---- Core Kernel (Infrastructure) ----
|
||
│ ├─ /config # Configuration management
|
||
│ ├─ /logger # Logging system
|
||
│ ├─ /di # Dependency injection
|
||
│ ├─ /health # Health checks
|
||
│ ├─ /metrics # Metrics collection
|
||
│ ├─ /observability # OpenTelemetry integration
|
||
│ ├─ /registry # Service registry
|
||
│ └─ /pluginloader # Module loader
|
||
│
|
||
├─ /pkg # ---- Public Interfaces ----
|
||
│ ├─ /config # ConfigProvider interface
|
||
│ ├─ /logger # Logger interface
|
||
│ ├─ /services # Service client interfaces
|
||
│ │ ├─ auth.go # AuthServiceClient
|
||
│ │ ├─ identity.go # IdentityServiceClient
|
||
│ │ ├─ authz.go # AuthzServiceClient
|
||
│ │ └─ audit.go # AuditServiceClient
|
||
│ └─ /module # IModule interface
|
||
│
|
||
├─ /modules # ---- Feature Services ----
|
||
│ ├─ /blog/
|
||
│ │ ├─ go.mod # Service module
|
||
│ │ ├─ module.yaml # Service manifest
|
||
│ │ ├─ internal/ # Service implementation
|
||
│ │ └─ pkg/
|
||
│ │ └─ module.go # IModule implementation
|
||
│ │
|
||
│ ├─ /billing/
|
||
│ └─ /chat/
|
||
│
|
||
├─ /infra # ---- Infrastructure Adapters ----
|
||
│ ├─ /cache (redis)
|
||
│ ├─ /queue (kafka)
|
||
│ └─ /storage (s3/azure‑blob)
|
||
│
|
||
├─ /scripts # build / lint / test helpers
|
||
│
|
||
├─ /ci
|
||
│ └─ github-actions.yml
|
||
│
|
||
├─ /docs
|
||
│ └─ architecture.md
|
||
│
|
||
├─ go.mod # Workspace root
|
||
└─ README.md
|
||
```
|
||
|
||
### How Services Boot
|
||
|
||
Each service has its own entry point and bootstraps independently:
|
||
|
||
```go
|
||
// cmd/auth-service/main.go
|
||
func main() {
|
||
// 1️⃣ Load configuration
|
||
cfg := config.Load()
|
||
|
||
// 2️⃣ Initialize core kernel (DI, logger, metrics)
|
||
container := di.NewContainer(cfg)
|
||
|
||
// 3️⃣ Register service implementations
|
||
container.Provide(NewAuthService)
|
||
container.Provide(NewTokenProvider)
|
||
|
||
// 4️⃣ Register gRPC server
|
||
container.Provide(NewGRPCServer)
|
||
|
||
// 5️⃣ Register with service registry
|
||
container.Provide(NewServiceRegistry)
|
||
|
||
// 6️⃣ Start service
|
||
container.Start()
|
||
}
|
||
|
||
// cmd/api-gateway/main.go
|
||
func main() {
|
||
// API Gateway bootstraps similarly
|
||
// Routes requests to backend services via service discovery
|
||
}
|
||
```
|
||
|
||
Services communicate via service clients:
|
||
|
||
- All inter-service communication uses gRPC (primary) or HTTP (fallback)
|
||
- Service discovery via service registry
|
||
- Each service manages its own database connection
|
||
- Services can be deployed independently
|
||
|
||
---
|
||
|
||
## KEY DECISIONS YOU MUST TAKE EARLY
|
||
|
||
| Decision | Options | Implications |
|
||
|----------|---------|--------------|
|
||
| **Language / Runtime** | Node.js (NestJS, Fastify), Java (Spring Boot), .NET (ASP.NET Core), Go (Gin/Fiber), Python (FastAPI) | Affects DI framework, module packaging, community libs for auth/OIDC, testing. |
|
||
| **Persistence Strategy** | Relational (PostgreSQL, MySQL) + optional NoSQL (Mongo, Dynamo) | Choose an ORM/Repository pattern that can be swapped per module. |
|
||
| **Auth Protocol** | JWT + Refresh, OAuth2 Authorization Server, OpenID Connect Provider, or integrate with external IdP (Keycloak, Auth0) | Influences token lifetimes, revocation strategy, multi‑tenant support. |
|
||
| **Event Bus** | In‑process EventEmitter (for monolith) → Kafka/Rabbit for scaling | Must expose both sync and async hooks. |
|
||
| **Module Packaging** | NPM packages (private registry) / Maven artifacts / Docker images (for micro‑service extraction) | Define a *semantic version* policy (core ≥ 1.0.0 never forces breaking changes on plug‑ins). |
|
||
| **Multitenancy Model** | Single DB with tenant_id column (shared), Schema‑per‑tenant, or DB‑per‑tenant | Affects repository base class and migrations tooling. |
|
||
| **Internationalisation** | i18next (frontend) + ICU messages in backend, or .NET Resource files | Choose a format that can be merged from modules at build time. |
|
||
| **CI/CD** | GitHub Actions + Docker Buildx + semantic‑release | Automate publishing of core + modules to same artifact registry. |
|
||
| **Testing Strategy** | Unit (Jest, JUnit, xUnit), Integration (Testcontainers), Contract (Pact) | Provide a **core testing harness** that loads a dummy module and asserts the contract of each extension point. |
|
||
|
||
---
|
||
|
||
## COMMON PITFALLS & HOW TO AVOID THEM
|
||
|
||
| Pitfall | Symptoms | Fix / Guardrail |
|
||
|---------|----------|-----------------|
|
||
| **Tight coupling of modules to core implementation** | Module imports internal ORM classes, fails on core upgrade. | Expose **only interfaces** (`IUserRepository`) from core and keep the concrete implementation as a private package. |
|
||
| **Hard‑coded permission strings** | Duplicate names across modules, typos cause silent authorisation bypass. | Provide a **Permission Builder DSL** (`Permission.define('blog.post', ['create', 'read'])`) that generates constants and registers them automatically. |
|
||
| **Global state in modules** | Tests interfere with each other, memory leaks when hot‑reloading. | Enforce **stateless services**; keep per‑request scoped data (e.g., via DI context). |
|
||
| **Schema migrations clash** | Two modules try to add the same column or foreign key. | Adopt a **central migration orchestrator** (e.g., Flyway/DBMate) that loads migration scripts from each module in alphabetical order. |
|
||
| **Authorization checks omitted in new routes** | Security hole for new plug‑in routes. | Provide a **base controller class** that auto‑applies `Authorize` filter, or a compile‑time lint rule that checks every exported route for a permission annotation. |
|
||
| **Vendor lock‑in to a particular IdP** | Hard to replace Keycloak later. | Keep **IdP adapters** behind a `IIdentityProvider` interface; ship at least two (local DB + OIDC). |
|
||
| **Unbounded background jobs** | Queue overflow, OOM, duplicate processing. | Use a **job‑scheduler abstraction** that caps concurrency, persists state, and provides `@Retry` decorator. |
|
||
| **Insufficient observability** | You can’t tell which module caused latency spikes. | Tag every log/metric with `module=<moduleName>` automatically via middleware. |
|
||
| **Version drift between core and modules** | Module built against core 1.0 fails on core 1.5. | Publish a **core compatibility matrix** and enforce `peerDependencies` in package.json; CI should fail on mismatched ranges. |
|
||
|
||
---
|
||
|
||
## QUICK START GUIDE (What to Build First)
|
||
|
||
1. **Create the Core Kernel**
|
||
- Set up DI container, config loader, logger, health/metrics endpoint.
|
||
- Infrastructure only - no business logic.
|
||
|
||
2. **Implement API Gateway**
|
||
- Request routing to backend services.
|
||
- Authentication at edge, rate limiting, CORS.
|
||
- Integration with service discovery.
|
||
|
||
3. **Implement Core Services**
|
||
- **Identity Service**: User CRUD, password hashing, email verification.
|
||
- **Auth Service**: JWT token generation/validation, refresh tokens.
|
||
- **Authz Service**: Permission resolution, RBAC/ABAC.
|
||
- **Audit Service**: Immutable audit logging.
|
||
|
||
4. **Set Up Service Communication**
|
||
- Define service client interfaces.
|
||
- Implement gRPC clients (primary) and HTTP clients (fallback).
|
||
- Service registry for discovery.
|
||
|
||
5. **Set Up Event Bus & Infrastructure**
|
||
- Kafka-based event bus.
|
||
- Redis cache.
|
||
- Shared infrastructure adapters.
|
||
|
||
6. **Build the Module Loader**
|
||
- Scan `modules/*/module.yaml` for service modules.
|
||
- Register services with service registry.
|
||
- Manage service lifecycle.
|
||
|
||
7. **Create a Sample Feature Service** – e.g., **Blog Service**
|
||
- Own entry point (`cmd/blog-service/`).
|
||
- Own database connection and schema.
|
||
- Use service clients for Auth, Identity, Authz.
|
||
- Define its own entities (`Post`, `Comment`).
|
||
|
||
8. **Write Integration Tests**
|
||
- Test service interactions via service clients.
|
||
- Spin up services in Docker Compose.
|
||
- Test cross-service communication.
|
||
|
||
9. **Add CI Pipeline**
|
||
- Build and test each service independently.
|
||
- Docker images for each service.
|
||
- Service deployment automation.
|
||
|
||
10. **Document Service Architecture**
|
||
- Service boundaries and responsibilities.
|
||
- Service client interfaces.
|
||
- Deployment and scaling guides.
|
||
|
||
---
|
||
|
||
## TOOLS & LIBRARIES (starter suggestions per stack)
|
||
|
||
| Stack | Core | Auth | DI / Module | Event Bus | ORM | Validation | Testing |
|
||
|-------|------|------|-------------|-----------|-----|------------|---------|
|
||
| **Node (TypeScript)** | NestJS (or Fastify + `awilix`) | `@nestjs/passport`, `passport-jwt`, `openid-client` | NestJS dynamic modules or `@nestjs-modules/mailer` | `@nestjs/event-emitter` or `KafkaJS` | TypeORM / Prisma | `class-validator` + `class-transformer` | Jest + `supertest`, Testcontainers |
|
||
| **Java** | Spring Boot | Spring Security + `spring-boot-starter-oauth2-resource-server` | Spring Boot `@Configuration` + `ImportBeanDefinitionRegistrar` | Spring Cloud Stream (Kafka) | JPA / Hibernate | Bean Validation (Hibernate Validator) | JUnit5 + Testcontainers |
|
||
| **.NET 8** | ASP.NET Core | `Microsoft.AspNetCore.Authentication.JwtBearer` | `IHostedService` + `Scrutor` for module discovery | MassTransit (Rabbit/Kafka) | EF Core | FluentValidation | xUnit + DockerTestContainers |
|
||
| **Go** | Echo / Fiber | `golang.org/x/oauth2` + `github.com/golang-jwt/jwt/v5` | `uber-go/fx` for DI, module registration | `segmentio/kafka-go` | GORM / Ent | `go-playground/validator` | Testify + Dockertest |
|
||
| **Python** | FastAPI | `fastapi-users` / `Authlib` | `pluggy` (pytest plugins) or custom loader | `aiokafka` | SQLModel / Tortoise ORM | Pydantic | Pytest + pytest‑asyncio, Testcontainers |
|
||
|
||
Pick the stack you’re most comfortable with; the concepts stay identical.
|
||
|
||
---
|
||
|
||
## TL;DR – What You Must Deliver
|
||
|
||
| Layer | Must‑have components | Why |
|
||
|-------|----------------------|-----|
|
||
| **Core Kernel** | Config, Logger, DI, Health, Metrics, Error Bus, Observability | Foundation infrastructure for all services. |
|
||
| **API Gateway** | Request routing, authentication, rate limiting, CORS | Single entry point for all external traffic. |
|
||
| **Core Services** | Identity, Auth, Authz, Audit services (separate services) | Independent, scalable security services. |
|
||
| **Service Clients** | gRPC/HTTP clients, service discovery, service registry | Enables service-to-service communication. |
|
||
| **Infrastructure Adapters** | Cache, Event Bus, Blob storage, Email/SMS, Scheduler | Shared infrastructure capabilities. |
|
||
| **Observability** | Structured logs, Prometheus metrics, OpenTelemetry traces | Cross-service observability and monitoring. |
|
||
| **DevOps Boilerplate** | CI pipelines, Dockerfiles, service deployment, Docs | Makes each service production‑ready. |
|
||
| **Sample Feature Service** | (e.g., Blog Service) with own entry point, DB, service clients | Provides a reference implementation for future services. |
|
||
|
||
When you scaffold those pieces **once**, any downstream team can drop a new folder that follows the `module.yaml` contract, implement the initializer, add its own tables & APIs, and instantly get:
|
||
|
||
* secure authentication,
|
||
* role‑based authorization,
|
||
* logging/metrics,
|
||
* unified config,
|
||
* CI‑ready testing,
|
||
* optional multi‑tenant isolation.
|
||
|
||
That’s the foundation of a **robust, future‑proof platform boilerplate**. Happy building! 🚀 |