toolz/goplt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: Align documentation with true microservices architecture

Browse Source 
Transform all documentation from modular monolith to true microservices
architecture where core services are independently deployable.

Key Changes:
- Core Kernel: Infrastructure only (no business logic)
- Core Services: Auth, Identity, Authz, Audit as separate microservices
  - Each service has own entry point (cmd/{service}/)
  - Each service has own gRPC server and database schema
  - Services register with Consul for service discovery
- API Gateway: Moved from Epic 8 to Epic 1 as core infrastructure
  - Single entry point for all external traffic
  - Handles routing, JWT validation, rate limiting, CORS
- Service Discovery: Consul as primary mechanism (ADR-0033)
- Database Pattern: Per-service connections with schema isolation

Documentation Updates:
- Updated all 9 architecture documents
- Updated 4 ADRs and created 2 new ADRs (API Gateway, Service Discovery)
- Rewrote Epic 1: Core Kernel & Infrastructure (infrastructure only)
- Rewrote Epic 2: Core Services (Auth, Identity, Authz, Audit as services)
- Updated Epic 3-8 stories for service architecture
- Updated plan.md, playbook.md, requirements.md, index.md
- Updated all epic READMEs and story files

New ADRs:
- ADR-0032: API Gateway Strategy
- ADR-0033: Service Discovery Implementation (Consul)

New Stories:
- Epic 1.7: Service Client Interfaces
- Epic 1.8: API Gateway Implementation
This commit is contained in:
0x1d
2025-11-06 08:47:27 +01:00
parent cab7cadf9e
commit 38a251968c
47 changed files with 3190 additions and 1613 deletions
Download Patch File Download Diff File Expand all files Collapse all files

250
docs/content/playbook.md
View File

@@ -7,7 +7,7 @@
|-----------|-----------------------|------------------------|
| **Hexagonal Architecture** | Go’s package‑level visibility (`internal/`) naturally creates a *boundary* between core and plug‑ins. | Keep all **domain** code in `internal/domain`, expose only **interfaces** in `pkg/`. |
| **Dependency Injection (DI) via Constructors** | Go avoids reflection‑heavy containers; compile‑time wiring is preferred. | Use **uber‑go/fx** (runtime graph) *or* **uber‑go/dig** for optional runtime DI. For a lighter weight solution, use plain **constructor injection** with a small **registry**. |
| **Modular Monolith → Micro‑service‑ready** | A single binary is cheap in Go; later you can extract modules into separate services without breaking APIs. | Each module lives in its own Go **module** (`go.mod`) under `./modules/*`. The core loads them via the **Go plugin** system *or* static registration (preferred for CI stability). |
| **microMicroservices Architecture** | Each service is independently deployable from day one. Services communicate via gRPC/HTTP through service clients. | Each service has its own entry point (`cmd/{service}/`), Go module (`go.mod`), database connection, and deployment. Services discover each other via Consul service registry. |
| **Plugin‑first design** | Go’s `plugin` package allows runtime loading of compiled `.so` files (Linux/macOS). | Provide an **IModule** interface and a **loader** that discovers `*.so` files (or compiled‑in modules for CI). |
| **API‑First (OpenAPI + gin/gorilla)** | Guarantees language‑agnostic contracts. | Generate server stubs from an `openapi.yaml` stored in `api/`. |
| **Security‑by‑Design** | Go’s static typing makes it easy to keep auth data out of the request flow. | Central middleware for JWT verification + context‑based user propagation. |
@@ -18,25 +18,46 @@
---
## 2 CORE KERNEL (What every Go‑platform must ship)
## 2 CORE KERNEL (Infrastructure Only)
| Module | Public Interfaces (exported from `pkg/`) | Recommended Packages | Brief Implementation Sketch |
The core kernel provides foundational infrastructure that all services depend on. It contains **no business logic**.
| Component | Public Interfaces (exported from `pkg/`) | Recommended Packages | Brief Implementation Sketch |
|--------|-------------------------------------------|----------------------|------------------------------|
| **Config** | `type ConfigProvider interface { Get(key string) any; Unmarshal(v any) error }` | `github.com/spf13/viper` | Load defaults (`config/default.yaml`), then env overrides, then optional secret‑store. |
| **Logger** | `type Logger interface { Debug(msg string, fields ...Field); Info(...); Error(...); With(fields ...Field) Logger }` | `go.uber.org/zap` (or `zerolog`) | Global logger is created in `cmd/main.go`; exported via `pkg/logger`. |
| **DI / Service Registry** | `type Container interface { Provide(constructor any) error; Invoke(fn any) error }` | `go.uber.org/dig` (or `fx` for lifecycle) | Core creates a `dig.New()` container, registers core services, then calls `container.Invoke(app.Start)`. |
| **Health & Metrics** | `type HealthChecker interface { Check(ctx context.Context) error }` | `github.com/prometheus/client_golang/prometheus`, `github.com/heptiolabs/healthcheck` | Expose `/healthz`, `/ready`, `/metrics`. |
| **Error Bus** | `type ErrorPublisher interface { Publish(err error) }` | Simple channel‐based implementation + optional Sentry (`github.com/getsentry/sentry-go`) | Core registers a singleton `ErrorBus`. |
| **Auth (JWT + OIDC)** | `type Authenticator interface { GenerateToken(userID string, roles []string) (string, error); VerifyToken(token string) (*TokenClaims, error) }` | `github.com/golang-jwt/jwt/v5`, `github.com/coreos/go-oidc` | Token claims embed `sub`, `roles`, `tenant_id`. Middleware adds `User` to `context.Context`. |
| **Authorization (RBAC/ABAC)** | `type Authorizer interface { Authorize(ctx context.Context, perm Permission) error }` | Custom DSL, `github.com/casbin/casbin/v2` (optional) | Permission format: `"module.resource.action"`; core ships a simple in‑memory resolver and a `casbin` adapter. |
| **Audit** | `type Auditor interface { Record(ctx context.Context, act AuditAction) error }` | Write to append‑only table (Postgres) or Elastic via `olivere/elastic` | Audits include `actorID`, `action`, `targetID`, `metadata`. |
| **Event Bus** | `type EventBus interface { Publish(ctx context.Context, ev Event) error; Subscribe(topic string, handler EventHandler) }` | `github.com/segmentio/kafka-go` (for production) + in‑process fallback | Core ships an **in‑process bus** used by tests and a **Kafka bus** for real deployments. |
| **Persistence (Repository)** | `type UserRepo interface { FindByID(id string) (*User, error); Create(u *User) error; … }` | `entgo.io/ent` (code‑gen ORM) **or** `gorm.io/gorm` | Core provides an `EntClient` wrapper that implements all core repos. |
| **Scheduler / Background Jobs** | `type Scheduler interface { Cron(spec string, job JobFunc) error; Enqueue(q string, payload any) error }` | `github.com/robfig/cron/v3`, `github.com/hibiken/asynq` (Redis‑backed) | Expose a `JobRegistry` where modules can register periodic jobs. |
| **Notification** | `type Notifier interface { Send(ctx context.Context, n Notification) error }` | `github.com/go-mail/mail` (SMTP), `github.com/aws/aws-sdk-go-v2/service/ses`, `github.com/IBM/sarama` (for push) | Core supplies an `EmailNotifier` and a `WebhookNotifier`. |
| **Multitenancy (optional)** | `type TenantResolver interface { Resolve(ctx context.Context) (tenantID string, err error) }` | Header/ sub‑domain parser + JWT claim scanner | Tenant ID is stored in request context and automatically added to SQL queries via Ent’s `Client` interceptor. |
| **Logger** | `type Logger interface { Debug(msg string, fields ...Field); Info(...); Error(...); With(fields ...Field) Logger }` | `go.uber.org/zap` (or `zerolog`) | Global logger is created in each service's `cmd/{service}/main.go`; exported via `pkg/logger`. |
| **DI / Service Registry** | `type Container interface { Provide(constructor any) error; Invoke(fn any) error }` | `go.uber.org/fx` (for lifecycle) | Each service creates its own `fx.New()` container, registers service-specific services. |
| **Health & Metrics** | `type HealthChecker interface { Check(ctx context.Context) error }` | `github.com/prometheus/client_golang/prometheus`, `github.com/heptiolabs/healthcheck` | Each service exposes `/healthz`, `/ready`, `/metrics`. |
| **Error Bus** | `type ErrorPublisher interface { Publish(err error) }` | Simple channel‐based implementation + optional Sentry (`github.com/getsentry/sentry-go`) | Each service registers its own `ErrorBus`. |
| **Service Registry** | `type ServiceRegistry interface { Register(ctx, service) error; Discover(ctx, name) ([]Service, error) }` | `github.com/hashicorp/consul/api` | Consul-based service discovery. Services register on startup, clients discover via registry. |
| **Observability** | `type Tracer interface { StartSpan(ctx, name) (Span, context.Context) }` | `go.opentelemetry.io/otel` | OpenTelemetry integration for distributed tracing across services. |
| **Event Bus** | `type EventBus interface { Publish(ctx context.Context, ev Event) error; Subscribe(topic string, handler EventHandler) }` | `github.com/segmentio/kafka-go` | Kafka-based event bus for asynchronous cross-service communication. |
| **Scheduler / Background Jobs** | `type Scheduler interface { Cron(spec string, job JobFunc) error; Enqueue(q string, payload any) error }` | `github.com/robfig/cron/v3`, `github.com/hibiken/asynq` (Redis‑backed) | Shared infrastructure for background jobs. |
| **Notification** | `type Notifier interface { Send(ctx context.Context, n Notification) error }` | `github.com/go-mail/mail` (SMTP), `github.com/aws/aws-sdk-go-v2/service/ses` | Shared infrastructure for notifications. |
| **Multitenancy (optional)** | `type TenantResolver interface { Resolve(ctx context.Context) (tenantID string, err error) }` | Header/ sub‑domain parser + JWT claim scanner | Tenant ID is stored in request context and automatically added to SQL queries via Ent's `Client` interceptor. |
All *public* interfaces live under `pkg/` so that plug‑ins can import them without pulling in implementation details. The concrete implementations stay in `internal/` (or separate go.mod modules) and are **registered with the container** during bootstrap.
## 2.1 CORE SERVICES (Independent Microservices)
Core business services are implemented as separate, independently deployable services:
| Service | Entry Point | Responsibilities | Service Client Interface |
|--------|-------------|------------------|-------------------------|
| **Auth Service** | `cmd/auth-service/` | JWT token generation/validation, authentication | `AuthServiceClient` in `pkg/services/auth.go` |
| **Identity Service** | `cmd/identity-service/` | User CRUD, password management, email verification | `IdentityServiceClient` in `pkg/services/identity.go` |
| **Authz Service** | `cmd/authz-service/` | Permission resolution, RBAC/ABAC authorization | `AuthzServiceClient` in `pkg/services/authz.go` |
| **Audit Service** | `cmd/audit-service/` | Audit logging, immutable audit records | `AuditServiceClient` in `pkg/services/audit.go` |
| **API Gateway** | `cmd/api-gateway/` | Request routing, authentication, rate limiting, CORS | N/A (entry point) |
Each service:
- Has its own `go.mod` (or shared workspace)
- Manages its own database connection pool and schema
- Exposes gRPC server (and optional HTTP)
- Registers with Consul service registry
- Uses service clients for inter-service communication
All *public* interfaces live under `pkg/` so that services can import them without pulling in implementation details. The concrete implementations stay in `internal/` (for core kernel) or `services/{service}/internal/` (for service implementations) and are **registered with the container** during service bootstrap.
**Note:** Business logic services (Auth, Identity, Authz, Audit) are NOT in the core kernel. They are separate services implemented in Epic 2.
---
@@ -167,13 +188,21 @@ A **code‑gen** tool (`go generate ./...`) can scan each module’s `module.yam
---
## 4 SAMPLE FEATURE MODULE – **Blog**
## 4 SAMPLE FEATURE SERVICE – **Blog Service**
Each feature module is implemented as an independent service:
```
modules/
cmd/
└─ blog-service/
└─ main.go # Service entry point
services/
└─ blog/
├─ go.mod # (module github.com/yourorg/blog)
├─ module.yaml
├─ go.mod # Service dependencies
├─ module.yaml # Service manifest
├─ api/
│ └─ blog.proto # gRPC service definition
├─ internal/
│ ├─ api/
│ │ └─ handler.go
@@ -207,74 +236,165 @@ routes:
permission: blog.post.read
```
### 4.2 Go implementation
### 4.2 Service Entry Point
```go
// pkg/module.go
package blog
// cmd/blog-service/main.go
package main
import (
"github.com/yourorg/platform/pkg/module"
"context"
"github.com/yourorg/platform/internal/config"
"github.com/yourorg/platform/internal/di"
"github.com/yourorg/platform/services/blog/internal/api"
"github.com/yourorg/platform/services/blog/internal/service"
"go.uber.org/fx"
)
type BlogModule struct{}
func (b BlogModule) Name() string { return "blog" }
func (b BlogModule) Init() fx.Option {
return fx.Options(
// Register repository implementation
fx.Provide(NewPostRepo),
// Register service layer
fx.Provide(NewPostService),
// Register HTTP handlers (using Gin)
fx.Invoke(RegisterHandlers),
// Register permissions (optional – just for documentation)
fx.Invoke(RegisterPermissions),
)
func main() {
cfg := config.Load()
fx.New(
// Core kernel services
di.CoreModule(cfg),
// Blog service implementation
fx.Provide(service.NewPostService),
fx.Provide(service.NewPostRepo),
// gRPC server
fx.Provide(api.NewGRPCServer),
// Service registry
fx.Provide(di.ProvideServiceRegistry),
// Start service
fx.Invoke(startService),
).Run()
}
func (b BlogModule) Migrations() []func(*ent.Client) error {
// Ent migration generated in internal/ent/migrate
return []func(*ent.Client) error{
func(c *ent.Client) error { return c.Schema.Create(context.Background()) },
func startService(lc fx.Lifecycle, server *api.GRPCServer, registry registry.ServiceRegistry) {
lc.Append(fx.Hook{
OnStart: func(ctx context.Context) error {
// Register with Consul
registry.Register(ctx, &registry.ServiceInstance{
ID: "blog-service-1",
Name: "blog-service",
Address: "localhost",
Port: 8091,
})
// Start gRPC server
return server.Start()
},
OnStop: func(ctx context.Context) error {
registry.Deregister(ctx, "blog-service-1")
return server.Stop()
},
})
}
```
### 4.3 Service Implementation
```go
// services/blog/internal/service/post_service.go
package service
import (
"context"
"github.com/yourorg/platform/pkg/services"
"github.com/yourorg/platform/services/blog/internal/domain"
)
type PostService struct {
repo *domain.PostRepo
authzClient services.AuthzServiceClient
identityClient services.IdentityServiceClient
auditClient services.AuditServiceClient
}
func NewPostService(
repo *domain.PostRepo,
authzClient services.AuthzServiceClient,
identityClient services.IdentityServiceClient,
auditClient services.AuditServiceClient,
) *PostService {
return &PostService{
repo: repo,
authzClient: authzClient,
identityClient: identityClient,
auditClient: auditClient,
}
}
// Export a variable for the plugin loader
var Module BlogModule
func (s *PostService) CreatePost(ctx context.Context, req *CreatePostRequest) (*Post, error) {
// Check permission via Authz Service
if err := s.authzClient.Authorize(ctx, "blog.post.create"); err != nil {
return nil, err
}
// Get user info via Identity Service
user, err := s.identityClient.GetUser(ctx, req.AuthorID)
if err != nil {
return nil, err
}
// Create post
post, err := s.repo.Create(ctx, &domain.Post{
Title: req.Title,
Content: req.Content,
AuthorID: user.ID,
})
if err != nil {
return nil, err
}
// Audit log via Audit Service
s.auditClient.Record(ctx, &services.AuditAction{
ActorID: user.ID,
Action: "blog.post.create",
TargetID: post.ID,
})
return post, nil
}
```
**Handler registration (Gin example)**
### 4.4 gRPC Handler
```go
// internal/api/handler.go
// services/blog/internal/api/handler.go
package api
import (
"github.com/gin-gonic/gin"
"github.com/yourorg/blog/internal/service"
"github.com/yourorg/platform/pkg/perm"
"github.com/yourorg/platform/pkg/auth"
"context"
"github.com/yourorg/platform/services/blog/api/pb"
"github.com/yourorg/platform/services/blog/internal/service"
)
func RegisterHandlers(r *gin.Engine, svc *service.PostService, authz auth.Authorizer) {
grp := r.Group("/api/v1/blog")
grp.Use(auth.AuthMiddleware()) // verifies JWT, injects user in context
type BlogServer struct {
pb.UnimplementedBlogServiceServer
service *service.PostService
}
// POST /posts
grp.POST("/posts", func(c *gin.Context) {
if err := authz.Authorize(c.Request.Context(), perm.BlogPostCreate); err != nil {
c.JSON(403, gin.H{"error": "forbidden"})
return
}
// decode request, call svc.Create, return 201…
func (s *BlogServer) CreatePost(ctx context.Context, req *pb.CreatePostRequest) (*pb.CreatePostResponse, error) {
post, err := s.service.CreatePost(ctx, &service.CreatePostRequest{
Title: req.Title,
Content: req.Content,
AuthorID: req.AuthorId,
})
// GET /posts/:id (similar)
if err != nil {
return nil, err
}
return &pb.CreatePostResponse{
Post: &pb.Post{
Id: post.ID,
Title: post.Title,
Content: post.Content,
},
}, nil
}
```