319 lines
19 KiB
Markdown
319 lines
19 KiB
Markdown
# Requirements
|
||
|
||
## HIGH‑LEVEL ARCHITECTURAL PRINCIPLES
|
||
|
||
| Principle | Why it matters for a modular platform | How to enforce it |
|
||
|-----------|----------------------------------------|-------------------|
|
||
| **Separation of Concerns (SoC)** | Keeps core services (auth, audit, config) independent from business modules. | Use **layered** or **hexagonal/clean‑architecture** boundaries. |
|
||
| **Domain‑Driven Design (DDD) Bounded Contexts** | Allows each module to own its own model & rules while sharing a common identity kernel. | Define a **Core Context** (Identity, Security, Infrastructure) and **Feature Contexts** (Billing, CMS, Chat, …). |
|
||
| **Modular Monolith → Micro‑service‑ready** | Start simple (single process) but keep each module in its own package so you can later split to services if needed. | Package each module as an **independent library** with its own **DI container‑module** and **routing**. |
|
||
| **Plug‑in / Extension‑point model** | Enables customers or internal teams to drop new features without touching core code. | Export **well‑defined interfaces** (e.g., `IUserProvider`, `IPermissionResolver`, `IModuleInitializer`). |
|
||
| **API‑First** | Guarantees that any UI (web, mobile, CLI) can be built on top of the same contract. | Publish **OpenAPI/GraphQL schema** as part of the build artefact. |
|
||
| **Security‑by‑Design** | The platform will hold user credentials, roles and possibly PII. | Centralize **authentication**, **authorization**, **audit**, **rate‑limiting**, **CORS**, **CSP**, **secure defaults**. |
|
||
| **Observability** | You need to know when a plug‑in crashes or misbehaves. | Provide **structured logging**, **metrics**, **tracing**, **health‑checks**, **central error bus**. |
|
||
| **CI/CD‑Ready Boilerplate** | Keeps the framework maintainable and encourages best‑practice adoption. | Include **GitHub Actions / GitLab CI** pipelines that run lint, unit, integration, contract tests, and a publish step. |
|
||
| **Configuration‑as‑Code** | Each module may need its own settings but you want a single source of truth. | Use a **hierarchical config system** (environment > file > secret store). |
|
||
| **Multitenancy (optional)** | If you plan SaaS, the core must be ready to isolate data per tenant. | Provide **tenant‑aware repository abstractions** and **tenant‑scoped middlewares**. |
|
||
| **Versioning & Compatibility** | Modules evolve; the core should never break existing plug‑ins. | Adopt **semantic versioning** + **backwards‑compatibility shim layer** (e.g., deprecation warnings). |
|
||
|
||
---
|
||
|
||
## LAYERED / HEXAGONAL BLUEPRINT
|
||
|
||
```
|
||
+---------------------------------------------------+
|
||
| Presentation |
|
||
| (REST/GraphQL/ gRPC Controllers, UI SDKs, CLI) |
|
||
+-------------------+-------------------------------+
|
||
| Application Services (Use‑cases) |
|
||
| - orchestrate core & feature modules |
|
||
| - enforce policies (RBAC, rate‑limit) |
|
||
+-------------------+-------------------------------+
|
||
| Domain (Business Logic) |
|
||
| - Core Entities (User, Role, Permission) |
|
||
| - Domain Services (PasswordPolicy, TokenMgr) |
|
||
+-------------------+-------------------------------+
|
||
| Infrastructure / Adapters |
|
||
| - Persistence (ORM/NoSQL Repositories) |
|
||
| - External services (Email, SMS, OIDC) |
|
||
| - Event bus (Kafka/Rabbit, In‑process) |
|
||
| - File storage, Cache, Search |
|
||
+---------------------------------------------------+
|
||
| Platform Core (Kernel) |
|
||
| - DI container, Module loader, Config manager |
|
||
| - Cross‑cutting concerns (Logging, Metrics) |
|
||
| - Security subsystem (AuthN/AuthZ, Token Issuer)|
|
||
+---------------------------------------------------+
|
||
```
|
||
|
||
*All layers talk to each other **through interfaces** defined in the Core kernel. Feature modules implement those interfaces and register themselves at startup.*
|
||
|
||
---
|
||
|
||
## REQUIRED BASE MODULES (THE “CORE KERNEL”)
|
||
|
||
| Module | Core responsibilities | Public API / Extension points |
|
||
|--------|-----------------------|--------------------------------|
|
||
| **Identity‑Management** | - User CRUD (local + federation) <br>- Password hashing, reset flow <br>- Email/Phone verification | `IUserRepository`, `IUserService`, `IExternalIdpProvider` |
|
||
| **Roles & Permissions** | - Role hierarchy (role → permission set) <br>- Permission definitions (string, enum, policy) <br>- Dynamic permission evaluation (ABAC) | `IPermissionResolver`, `IRoleService` |
|
||
| **Authentication** | - JWT / OAuth2 Access & Refresh tokens <br>- OpenID Connect Provider (optional) <br>- Session Management (stateless + optional stateful) | `IAuthService`, `ITokenProvider` |
|
||
| **Authorization Middleware** | - Enforce RBAC/ABAC on each request <br>- Policy DSL (e.g., `hasPermission('project.read') && resource.ownerId == user.id`) | `IAuthorizationHandler` |
|
||
| **Audit & Activity Log** | - Immutable log of security‑relevant actions <br>- Correlation IDs, actor, target, timestamp | `IAuditSink`, `IAuditService` |
|
||
| **Configuration** | - Hierarchical sources (env, files, secret manager) <br>- Validation schema (JSON‑Schema / Yup / JSR‑380) | `IConfigProvider` |
|
||
| **Health & Metrics** | - `/healthz`, `/ready`, `/metrics` endpoints <br>- Export to Prometheus, Grafana, CloudWatch | `IHealthCheck`, `IMetricsRegistry` |
|
||
| **Error Handling** | - Centralized error objects, stack trace masking <br>- Automatic mapping to HTTP status <br>- Exception‑to‑event publishing | `IErrorMapper`, `IErrorBus` |
|
||
| **Logging** | - Structured JSON logs (level, requestId, userId) <br>- pluggable sinks (stdout, file, ELK, Cloud Logging) | `ILoggerFactory` |
|
||
| **Notification** | - Email, SMS, Push (FCM/APNs), Webhooks <br>- Queue‑backed delivery, retries | `INotificationService` |
|
||
| **File / Blob Storage** | - Abstracted bucket API (upload, version, signed URLs) | `IBlobStore` |
|
||
| **Scheduler / Background Jobs** | - Cron‑like tasks, queue workers, retry/back‑off policies | `IJobScheduler`, `IJobProcessor` |
|
||
| **Internationalization (i18n)** | - Message catalog, locale negotiation, runtime translation | `I18nService` |
|
||
| **API Gateway (optional)** | - Rate‑limit, request/response transformation, API‑key handling, request routing to modules | `IGatewayPlugin` |
|
||
| **Multitenancy (optional)** | - Tenant identification (sub‑domain, header, JWT claim) <br>- Tenant‑scoped data isolation primitives | `ITenantResolver`, `ITenantContext` |
|
||
|
||
> **Tip:** Pack each module as a **separate NPM/ Maven / NuGet / Go module** with its own `package.json` / `pom.xml` etc. The platform’s **bootstrapper** loads every module that implements `IModuleInitializer` (or similar) and calls `ConfigureServices` / `RegisterRoutes`.
|
||
|
||
---
|
||
|
||
## EXTENSION‑POINT DESIGN (HOW PLUG‑INS HOOK IN)
|
||
|
||
1. **Module Manifest** – a tiny JSON/YAML file (`module.yaml`) that declares:
|
||
- Module name, version, dependencies (core ≥ 1.2.0, other modules)
|
||
- Public routes (e.g., `/api/v1/blog/**`)
|
||
- Required permissions (auto‑generated from source annotations)
|
||
- UI assets (static folder, React component entry point)
|
||
|
||
2. **Bootstrap Interface**
|
||
```ts
|
||
export interface IModuleInitializer {
|
||
/**
|
||
* Called during platform start‑up.
|
||
* Register services, routes, policies, background jobs.
|
||
*/
|
||
init(app: IApplicationBuilder, container: IServiceContainer): Promise<void>;
|
||
}
|
||
```
|
||
|
||
3. **Dependency Injection (DI) Conventions**
|
||
- Core registers **contracts** (`IUserRepository`, `IPermissionResolver`) as **singletons**.
|
||
- Modules register **implementations** with a **named scope** (e.g., `UserRepository:Local`).
|
||
- Override is possible via **module ordering** or explicit `container.override(...)`.
|
||
|
||
4. **Policy/Permission Extension**
|
||
```ts
|
||
// core lib
|
||
export type Permission = `${string}.${string}`; // e.g., "blog.post.create"
|
||
|
||
// module
|
||
export const BLOG_PERMS = {
|
||
POST_CREATE: 'blog.post.create',
|
||
POST_READ: 'blog.post.read',
|
||
POST_UPDATE: 'blog.post.update',
|
||
POST_DELETE: 'blog.post.delete',
|
||
} as const;
|
||
```
|
||
|
||
5. **Event Bus & Hooks**
|
||
- Central **topic**: `platform.*` (user.created, role.assigned, tenant.created)
|
||
- Modules can **publish** and **subscribe** via `IEventBus`.
|
||
- Provide **synchronous guard hooks** (`beforeUserCreate`, `afterRoleDelete`) for validation & side‑effects.
|
||
|
||
6. **UI Plug‑in System**
|
||
- Serve a **manifest** at `/modules` that front‑end bundles read to render navigation.
|
||
- Encourage **Web Component / Module Federation** pattern for SPA integration.
|
||
|
||
---
|
||
|
||
## SAMPLE REPOSITORY LAYOUT (language‑agnostic)
|
||
|
||
```
|
||
/platform-root
|
||
│
|
||
├─ /core # ---- Kernel / Base modules ----
|
||
│ ├─ /auth
|
||
│ │ ├─ src/
|
||
│ │ └─ package.json
|
||
│ ├─ /identity
|
||
│ ├─ /authorization
|
||
│ ├─ /audit
|
||
│ ├─ /config
|
||
│ ├─ /logging
|
||
│ ├─ /metrics
|
||
│ └─ index.ts (exports all core APIs)
|
||
│
|
||
├─ /modules # ---- Feature plug‑ins ----
|
||
│ ├─ /blog
|
||
│ │ ├─ module.yaml # manifest
|
||
│ │ ├─ src/
|
||
│ │ │ ├─ BlogController.ts
|
||
│ │ │ ├─ BlogService.ts
|
||
│ │ │ └─ BlogModule.ts (implements IModuleInitializer)
|
||
│ │ └─ package.json
|
||
│ │
|
||
│ ├─ /billing
|
||
│ └─ /chat
|
||
│
|
||
├─ /infra # ---- Infrastructure adapters ----
|
||
│ ├─ /orm (typeorm/hibernate/EFCore etc.)
|
||
│ ├─ /cache (redis)
|
||
│ ├─ /queue (rabbit/kafka)
|
||
│ └─ /storage (s3/azure‑blob)
|
||
│
|
||
├─ /gateway (optional API‑gateway layer)
|
||
│
|
||
├─ /scripts # build / lint / test helpers
|
||
│
|
||
├─ /ci
|
||
│ └─ github-actions.yml
|
||
│
|
||
├─ /docs
|
||
│ └─ architecture.md
|
||
│
|
||
├─ package.json (or pom.xml / go.mod)
|
||
└─ README.md
|
||
```
|
||
|
||
### How it boots
|
||
|
||
```ts
|
||
// platform-root/src/main.ts
|
||
import { createApp } from '@core/app';
|
||
import { loadModules } from '@core/module-loader';
|
||
import { CoreModule } from '@core';
|
||
|
||
async function bootstrap() {
|
||
const app = await createApp();
|
||
|
||
// 1️⃣ Load core kernel (DI, config, logger)
|
||
await app.register(CoreModule);
|
||
|
||
// 2️⃣ Dynamically discover all `module.yaml` under /modules
|
||
const modules = await loadModules(__dirname + '/modules');
|
||
|
||
// 3️⃣ Initialise each module (order can be defined in manifest)
|
||
for (const mod of modules) {
|
||
await mod.instance.init(app.builder, app.container);
|
||
}
|
||
|
||
// 4️⃣ Start HTTP / gRPC server
|
||
await app.listen(process.env.PORT || 3000);
|
||
}
|
||
|
||
bootstrap().catch(err => {
|
||
console.error('❌ Platform failed to start', err);
|
||
process.exit(1);
|
||
});
|
||
```
|
||
|
||
---
|
||
|
||
## KEY DECISIONS YOU MUST TAKE EARLY
|
||
|
||
| Decision | Options | Implications |
|
||
|----------|---------|--------------|
|
||
| **Language / Runtime** | Node.js (NestJS, Fastify), Java (Spring Boot), .NET (ASP.NET Core), Go (Gin/Fiber), Python (FastAPI) | Affects DI framework, module packaging, community libs for auth/OIDC, testing. |
|
||
| **Persistence Strategy** | Relational (PostgreSQL, MySQL) + optional NoSQL (Mongo, Dynamo) | Choose an ORM/Repository pattern that can be swapped per module. |
|
||
| **Auth Protocol** | JWT + Refresh, OAuth2 Authorization Server, OpenID Connect Provider, or integrate with external IdP (Keycloak, Auth0) | Influences token lifetimes, revocation strategy, multi‑tenant support. |
|
||
| **Event Bus** | In‑process EventEmitter (for monolith) → Kafka/Rabbit for scaling | Must expose both sync and async hooks. |
|
||
| **Module Packaging** | NPM packages (private registry) / Maven artifacts / Docker images (for micro‑service extraction) | Define a *semantic version* policy (core ≥ 1.0.0 never forces breaking changes on plug‑ins). |
|
||
| **Multitenancy Model** | Single DB with tenant_id column (shared), Schema‑per‑tenant, or DB‑per‑tenant | Affects repository base class and migrations tooling. |
|
||
| **Internationalisation** | i18next (frontend) + ICU messages in backend, or .NET Resource files | Choose a format that can be merged from modules at build time. |
|
||
| **CI/CD** | GitHub Actions + Docker Buildx + semantic‑release | Automate publishing of core + modules to same artifact registry. |
|
||
| **Testing Strategy** | Unit (Jest, JUnit, xUnit), Integration (Testcontainers), Contract (Pact) | Provide a **core testing harness** that loads a dummy module and asserts the contract of each extension point. |
|
||
|
||
---
|
||
|
||
## COMMON PITFALLS & HOW TO AVOID THEM
|
||
|
||
| Pitfall | Symptoms | Fix / Guardrail |
|
||
|---------|----------|-----------------|
|
||
| **Tight coupling of modules to core implementation** | Module imports internal ORM classes, fails on core upgrade. | Expose **only interfaces** (`IUserRepository`) from core and keep the concrete implementation as a private package. |
|
||
| **Hard‑coded permission strings** | Duplicate names across modules, typos cause silent authorisation bypass. | Provide a **Permission Builder DSL** (`Permission.define('blog.post', ['create', 'read'])`) that generates constants and registers them automatically. |
|
||
| **Global state in modules** | Tests interfere with each other, memory leaks when hot‑reloading. | Enforce **stateless services**; keep per‑request scoped data (e.g., via DI context). |
|
||
| **Schema migrations clash** | Two modules try to add the same column or foreign key. | Adopt a **central migration orchestrator** (e.g., Flyway/DBMate) that loads migration scripts from each module in alphabetical order. |
|
||
| **Authorization checks omitted in new routes** | Security hole for new plug‑in routes. | Provide a **base controller class** that auto‑applies `Authorize` filter, or a compile‑time lint rule that checks every exported route for a permission annotation. |
|
||
| **Vendor lock‑in to a particular IdP** | Hard to replace Keycloak later. | Keep **IdP adapters** behind a `IIdentityProvider` interface; ship at least two (local DB + OIDC). |
|
||
| **Unbounded background jobs** | Queue overflow, OOM, duplicate processing. | Use a **job‑scheduler abstraction** that caps concurrency, persists state, and provides `@Retry` decorator. |
|
||
| **Insufficient observability** | You can’t tell which module caused latency spikes. | Tag every log/metric with `module=<moduleName>` automatically via middleware. |
|
||
| **Version drift between core and modules** | Module built against core 1.0 fails on core 1.5. | Publish a **core compatibility matrix** and enforce `peerDependencies` in package.json; CI should fail on mismatched ranges. |
|
||
|
||
---
|
||
|
||
## QUICK START GUIDE (What to Build First)
|
||
|
||
1. **Create the Core Kernel**
|
||
- Set up DI container, config loader, logger, health/metrics endpoint.
|
||
- Scaffold `IUserRepository`, `IPermissionResolver`, `ITokenProvider`.
|
||
|
||
2. **Implement Identity & Auth**
|
||
- Choose JWT + Refresh + optional OpenID Connect.
|
||
- Add password hashing (bcrypt/argon2) and email verification flow.
|
||
|
||
3. **Add Role/Permission Engine**
|
||
- Simple RBAC matrix with an extensible `Permission` type.
|
||
- Provide a UI admin UI (or API only) to manage roles.
|
||
|
||
4. **Set Up Event Bus & Audit**
|
||
- Publish `user.created`, `role.granted` events.
|
||
- Store audit entries in an append‑only table (or log to Elastic).
|
||
|
||
5. **Build the Module Loader**
|
||
- Scan `modules/*/module.yaml`, load via `require()`/classpath.
|
||
- Register each `IModuleInitializer`.
|
||
|
||
6. **Create a Sample Feature Module** – e.g., **Blog**
|
||
- Define its own entities (`Post`, `Comment`).
|
||
- Register routes (`/api/v1/blog/posts`).
|
||
- Declare required permissions (`blog.post.create`).
|
||
|
||
7. **Write Integration Tests**
|
||
- Spin up an in‑memory DB (SQLite or H2).
|
||
- Load core + blog module, assert that a user without `blog.post.create` receives 403.
|
||
|
||
8. **Add CI Pipeline**
|
||
- Lint → Unit → Integration (Docker Compose with DB + Redis).
|
||
- On tag, publish `core` and `blog` packages to your private registry.
|
||
|
||
9. **Document Extension Points**
|
||
- Provide a **Developer Handbook** (README + `docs/extension-points.md`).
|
||
|
||
10. **Iterate** – add Notification, Scheduler, Multitenancy, API‑Gateway as needed.
|
||
|
||
---
|
||
|
||
## TOOLS & LIBRARIES (starter suggestions per stack)
|
||
|
||
| Stack | Core | Auth | DI / Module | Event Bus | ORM | Validation | Testing |
|
||
|-------|------|------|-------------|-----------|-----|------------|---------|
|
||
| **Node (TypeScript)** | NestJS (or Fastify + `awilix`) | `@nestjs/passport`, `passport-jwt`, `openid-client` | NestJS dynamic modules or `@nestjs-modules/mailer` | `@nestjs/event-emitter` or `KafkaJS` | TypeORM / Prisma | `class-validator` + `class-transformer` | Jest + `supertest`, Testcontainers |
|
||
| **Java** | Spring Boot | Spring Security + `spring-boot-starter-oauth2-resource-server` | Spring Boot `@Configuration` + `ImportBeanDefinitionRegistrar` | Spring Cloud Stream (Kafka) | JPA / Hibernate | Bean Validation (Hibernate Validator) | JUnit5 + Testcontainers |
|
||
| **.NET 8** | ASP.NET Core | `Microsoft.AspNetCore.Authentication.JwtBearer` | `IHostedService` + `Scrutor` for module discovery | MassTransit (Rabbit/Kafka) | EF Core | FluentValidation | xUnit + DockerTestContainers |
|
||
| **Go** | Echo / Fiber | `golang.org/x/oauth2` + `github.com/golang-jwt/jwt/v5` | `uber-go/fx` for DI, module registration | `segmentio/kafka-go` | GORM / Ent | `go-playground/validator` | Testify + Dockertest |
|
||
| **Python** | FastAPI | `fastapi-users` / `Authlib` | `pluggy` (pytest plugins) or custom loader | `aiokafka` | SQLModel / Tortoise ORM | Pydantic | Pytest + pytest‑asyncio, Testcontainers |
|
||
|
||
Pick the stack you’re most comfortable with; the concepts stay identical.
|
||
|
||
---
|
||
|
||
## TL;DR – What You Must Deliver
|
||
|
||
| Layer | Must‑have components | Why |
|
||
|-------|----------------------|-----|
|
||
| **Core Kernel** | Config, Logger, DI, Health, Metrics, Error Bus | Foundation for any module. |
|
||
| **Security** | Auth (JWT/OIDC), Authorization (RBAC + ABAC), Audit | Guarantees secure, traceable access. |
|
||
| **User & Role Management** | User CRUD, Password reset, Role ↔ Permission matrix | The “identity” piece everyone will reuse. |
|
||
| **Extension System** | `IModuleInitializer`, `module.yaml`, EventBus, Permission DSL | Enables plug‑ins without touching core. |
|
||
| **Infrastructure Adapters** | DB repo, Cache, Queue, Blob storage, Email/SMS | Keeps core agnostic to any concrete tech. |
|
||
| **Observability** | Structured logs, Prometheus metrics, OpenTelemetry traces | You can monitor each module individually. |
|
||
| **DevOps Boilerplate** | CI pipelines, Dockerfiles, Semantic‑release, Docs | Makes the framework production‑ready out‑of‑the‑box. |
|
||
| **Sample Feature Module** | (e.g., Blog) to show how to add routes, permissions, DB entities | Provides a reference implementation for future developers. |
|
||
|
||
When you scaffold those pieces **once**, any downstream team can drop a new folder that follows the `module.yaml` contract, implement the initializer, add its own tables & APIs, and instantly get:
|
||
|
||
* secure authentication,
|
||
* role‑based authorization,
|
||
* logging/metrics,
|
||
* unified config,
|
||
* CI‑ready testing,
|
||
* optional multi‑tenant isolation.
|
||
|
||
That’s the foundation of a **robust, future‑proof platform boilerplate**. Happy building! 🚀 |