741 lines
29 KiB
Markdown
741 lines
29 KiB
Markdown
# Go‑Platform Boilerplate Play‑book
|
||
**“Plug‑in‑friendly SaaS/Enterprise Platform – Go Edition”**
|
||
|
||
## 1 ARCHITECTURAL IMPERATIVES (Go‑flavoured)
|
||
|
||
| Principle | Go‑specific rationale | Enforcement Technique |
|
||
|-----------|-----------------------|------------------------|
|
||
| **Hexagonal Architecture** | Go’s package‑level visibility (`internal/`) naturally creates a *boundary* between core and plug‑ins. | Keep all **domain** code in `internal/domain`, expose only **interfaces** in `pkg/`. |
|
||
| **Dependency Injection (DI) via Constructors** | Go avoids reflection‑heavy containers; compile‑time wiring is preferred. | Use **uber‑go/fx** (runtime graph) *or* **uber‑go/dig** for optional runtime DI. For a lighter weight solution, use plain **constructor injection** with a small **registry**. |
|
||
| **microMicroservices Architecture** | Each service is independently deployable from day one. Services communicate via gRPC/HTTP through service clients. | Each service has its own entry point (`cmd/{service}/`), Go module (`go.mod`), database connection, and deployment. Services discover each other via Consul service registry. |
|
||
| **Plugin‑first design** | Go’s `plugin` package allows runtime loading of compiled `.so` files (Linux/macOS). | Provide an **IModule** interface and a **loader** that discovers `*.so` files (or compiled‑in modules for CI). |
|
||
| **API‑First (OpenAPI + gin/gorilla)** | Guarantees language‑agnostic contracts. | Generate server stubs from an `openapi.yaml` stored in `api/`. |
|
||
| **Security‑by‑Design** | Go’s static typing makes it easy to keep auth data out of the request flow. | Central middleware for JWT verification + context‑based user propagation. |
|
||
| **Observability (OpenTelemetry)** | The Go ecosystem ships first‑class OTEL SDK. | Instrument HTTP, DB, queues, and custom events automatically. |
|
||
| **Configuration‑as‑Code** | Viper + Cobra give hierarchical config and flag parsing. | Load defaults → file → env → secret manager (AWS Secrets Manager / Vault). |
|
||
| **Testing & CI** | `go test` is fast; Testcontainers via **testcontainers-go** can spin up DB, Redis, Kafka. | CI pipeline runs unit, integration, and contract tests on each PR. |
|
||
| **Semantic Versioning & Compatibility** | Go modules already enforce version constraints. | Core declares **minimal required versions** in `go.mod` and uses `replace` for local dev. |
|
||
|
||
---
|
||
|
||
## 2 CORE KERNEL (Infrastructure Only)
|
||
|
||
The core kernel provides foundational infrastructure that all services depend on. It contains **no business logic**.
|
||
|
||
| Component | Public Interfaces (exported from `pkg/`) | Recommended Packages | Brief Implementation Sketch |
|
||
|--------|-------------------------------------------|----------------------|------------------------------|
|
||
| **Config** | `type ConfigProvider interface { Get(key string) any; Unmarshal(v any) error }` | `github.com/spf13/viper` | Load defaults (`config/default.yaml`), then env overrides, then optional secret‑store. |
|
||
| **Logger** | `type Logger interface { Debug(msg string, fields ...Field); Info(...); Error(...); With(fields ...Field) Logger }` | `go.uber.org/zap` (or `zerolog`) | Global logger is created in each service's `cmd/{service}/main.go`; exported via `pkg/logger`. |
|
||
| **DI / Service Registry** | `type Container interface { Provide(constructor any) error; Invoke(fn any) error }` | `go.uber.org/fx` (for lifecycle) | Each service creates its own `fx.New()` container, registers service-specific services. |
|
||
| **Health & Metrics** | `type HealthChecker interface { Check(ctx context.Context) error }` | `github.com/prometheus/client_golang/prometheus`, `github.com/heptiolabs/healthcheck` | Each service exposes `/healthz`, `/ready`, `/metrics`. |
|
||
| **Error Bus** | `type ErrorPublisher interface { Publish(err error) }` | Simple channel‐based implementation + optional Sentry (`github.com/getsentry/sentry-go`) | Each service registers its own `ErrorBus`. |
|
||
| **Service Registry** | `type ServiceRegistry interface { Register(ctx, service) error; Discover(ctx, name) ([]Service, error) }` | `github.com/hashicorp/consul/api` | Consul-based service discovery. Services register on startup, clients discover via registry. |
|
||
| **Observability** | `type Tracer interface { StartSpan(ctx, name) (Span, context.Context) }` | `go.opentelemetry.io/otel` | OpenTelemetry integration for distributed tracing across services. |
|
||
| **Event Bus** | `type EventBus interface { Publish(ctx context.Context, ev Event) error; Subscribe(topic string, handler EventHandler) }` | `github.com/segmentio/kafka-go` | Kafka-based event bus for asynchronous cross-service communication. |
|
||
| **Scheduler / Background Jobs** | `type Scheduler interface { Cron(spec string, job JobFunc) error; Enqueue(q string, payload any) error }` | `github.com/robfig/cron/v3`, `github.com/hibiken/asynq` (Redis‑backed) | Shared infrastructure for background jobs. |
|
||
| **Notification** | `type Notifier interface { Send(ctx context.Context, n Notification) error }` | `github.com/go-mail/mail` (SMTP), `github.com/aws/aws-sdk-go-v2/service/ses` | Shared infrastructure for notifications. |
|
||
| **Multitenancy (optional)** | `type TenantResolver interface { Resolve(ctx context.Context) (tenantID string, err error) }` | Header/ sub‑domain parser + JWT claim scanner | Tenant ID is stored in request context and automatically added to SQL queries via Ent's `Client` interceptor. |
|
||
|
||
## 2.1 CORE SERVICES (Independent Microservices)
|
||
|
||
Core business services are implemented as separate, independently deployable services:
|
||
|
||
| Service | Entry Point | Responsibilities | Service Client Interface |
|
||
|--------|-------------|------------------|-------------------------|
|
||
| **Auth Service** | `cmd/auth-service/` | JWT token generation/validation, authentication | `AuthServiceClient` in `pkg/services/auth.go` |
|
||
| **Identity Service** | `cmd/identity-service/` | User CRUD, password management, email verification | `IdentityServiceClient` in `pkg/services/identity.go` |
|
||
| **Authz Service** | `cmd/authz-service/` | Permission resolution, RBAC/ABAC authorization | `AuthzServiceClient` in `pkg/services/authz.go` |
|
||
| **Audit Service** | `cmd/audit-service/` | Audit logging, immutable audit records | `AuditServiceClient` in `pkg/services/audit.go` |
|
||
| **API Gateway** | `cmd/api-gateway/` | Request routing, authentication, rate limiting, CORS | N/A (entry point) |
|
||
|
||
Each service:
|
||
|
||
- Has its own `go.mod` (or shared workspace)
|
||
- Manages its own database connection pool and schema
|
||
- Exposes gRPC server (and optional HTTP)
|
||
- Registers with Consul service registry
|
||
- Uses service clients for inter-service communication
|
||
|
||
All *public* interfaces live under `pkg/` so that services can import them without pulling in implementation details. The concrete implementations stay in `internal/` (for core kernel) or `services/{service}/internal/` (for service implementations) and are **registered with the container** during service bootstrap.
|
||
|
||
**Note:** Business logic services (Auth, Identity, Authz, Audit) are NOT in the core kernel. They are separate services implemented in Epic 2.
|
||
|
||
---
|
||
|
||
## 3 MODULE (PLUGIN) FRAMEWORK
|
||
|
||
### 3.1 Interface that every module must implement
|
||
|
||
```go
|
||
// pkg/module/module.go
|
||
package module
|
||
|
||
import (
|
||
"context"
|
||
"go.uber.org/fx"
|
||
)
|
||
|
||
// IModule is the contract a feature plug‑in must fulfil.
|
||
type IModule interface {
|
||
// Name returns a unique, human‑readable identifier.
|
||
Name() string
|
||
// Init registers the module's services, routes, jobs, and permissions.
|
||
// The fx.Options returned are merged into the app's lifecycle.
|
||
Init() fx.Option
|
||
// Migrations returns a slice of Ent migration functions (or raw SQL) that
|
||
// the core will run when the platform starts.
|
||
Migrations() []func(*ent.Client) error
|
||
}
|
||
```
|
||
|
||
### 3.2 Registration Mechanics
|
||
|
||
Two ways to get a module into the platform:
|
||
|
||
| Approach | When to use | Pros | Cons |
|
||
|----------|-------------|------|------|
|
||
| **Static registration** – each module imports `core` and calls `module.Register(MyBlogModule{})` in its own `init()` | Development, CI (no `.so` needed) | Simpler, works on Windows; compile‑time type safety | Requires recompiling the binary for new modules |
|
||
| **Runtime `plugin` loading** – compile each module as a `go build -buildmode=plugin -o blog.so ./modules/blog` | Production SaaS where clients drop new modules, or separate micro‑service extraction | Hot‑swap without rebuild | Only works on Linux/macOS; plugins must be compiled with same Go version & same `go.mod` replace graph; debugging harder |
|
||
|
||
**Static registration example**
|
||
|
||
```go
|
||
// internal/registry/registry.go
|
||
package registry
|
||
|
||
import (
|
||
"sync"
|
||
"github.com/yourorg/platform/pkg/module"
|
||
)
|
||
|
||
var (
|
||
mu sync.Mutex
|
||
modules = make(map[string]module.IModule)
|
||
)
|
||
|
||
func Register(m module.IModule) {
|
||
mu.Lock()
|
||
defer mu.Unlock()
|
||
if _, ok := modules[m.Name()]; ok {
|
||
panic("module already registered: " + m.Name())
|
||
}
|
||
modules[m.Name()] = m
|
||
}
|
||
|
||
func All() []module.IModule {
|
||
mu.Lock()
|
||
defer mu.Unlock()
|
||
out := make([]module.IModule, 0, len(modules))
|
||
for _, m := range modules {
|
||
out = append(out, m)
|
||
}
|
||
return out
|
||
}
|
||
```
|
||
|
||
**Plugin loader skeleton**
|
||
|
||
```go
|
||
// internal/pluginloader/loader.go
|
||
package pluginloader
|
||
|
||
import (
|
||
"plugin"
|
||
"github.com/yourorg/platform/pkg/module"
|
||
)
|
||
|
||
func Load(path string) (module.IModule, error) {
|
||
p, err := plugin.Open(path)
|
||
if err != nil {
|
||
return nil, err
|
||
}
|
||
sym, err := p.Lookup("Module")
|
||
if err != nil {
|
||
return nil, err
|
||
}
|
||
mod, ok := sym.(module.IModule)
|
||
if !ok {
|
||
return nil, fmt.Errorf("invalid module type")
|
||
}
|
||
return mod, nil
|
||
}
|
||
```
|
||
|
||
> **Tip:** Ship a tiny CLI (`platformctl modules list`) that scans `./plugins/*.so`, loads each via `Load`, and prints `Name()`. This is a great sanity check for ops.
|
||
|
||
### 3.3 Permissions DSL (compile‑time safety)
|
||
|
||
```go
|
||
// pkg/perm/perm.go
|
||
package perm
|
||
|
||
type Permission string
|
||
|
||
func (p Permission) String() string { return string(p) }
|
||
|
||
var (
|
||
// Core permissions
|
||
SystemHealthCheck Permission = "system.health.check"
|
||
|
||
// Blog module – generated by a small go:generate script
|
||
BlogPostCreate Permission = "blog.post.create"
|
||
BlogPostRead Permission = "blog.post.read"
|
||
BlogPostUpdate Permission = "blog.post.update"
|
||
BlogPostDelete Permission = "blog.post.delete"
|
||
)
|
||
```
|
||
|
||
A **code‑gen** tool (`go generate ./...`) can scan each module’s `module.yaml` for declared actions and emit a single `perm.go` file, guaranteeing no duplicate strings.
|
||
|
||
---
|
||
|
||
## 4 SAMPLE FEATURE SERVICE – **Blog Service**
|
||
|
||
Each feature module is implemented as an independent service:
|
||
|
||
```
|
||
cmd/
|
||
└─ blog-service/
|
||
└─ main.go # Service entry point
|
||
|
||
services/
|
||
└─ blog/
|
||
├─ go.mod # Service dependencies
|
||
├─ module.yaml # Service manifest
|
||
├─ api/
|
||
│ └─ blog.proto # gRPC service definition
|
||
├─ internal/
|
||
│ ├─ api/
|
||
│ │ └─ handler.go
|
||
│ ├─ domain/
|
||
│ │ ├─ post.go
|
||
│ │ └─ post_repo.go
|
||
│ └─ service/
|
||
│ └─ post_service.go
|
||
└─ pkg/
|
||
└─ module.go
|
||
```
|
||
|
||
### 4.1 `module.yaml`
|
||
|
||
```yaml
|
||
name: blog
|
||
version: 0.1.0
|
||
dependencies:
|
||
- core >= 1.3.0
|
||
permissions:
|
||
- blog.post.create
|
||
- blog.post.read
|
||
- blog.post.update
|
||
- blog.post.delete
|
||
routes:
|
||
- method: POST
|
||
path: /api/v1/blog/posts
|
||
permission: blog.post.create
|
||
- method: GET
|
||
path: /api/v1/blog/posts/:id
|
||
permission: blog.post.read
|
||
```
|
||
|
||
### 4.2 Service Entry Point
|
||
|
||
```go
|
||
// cmd/blog-service/main.go
|
||
package main
|
||
|
||
import (
|
||
"context"
|
||
"github.com/yourorg/platform/internal/config"
|
||
"github.com/yourorg/platform/internal/di"
|
||
"github.com/yourorg/platform/services/blog/internal/api"
|
||
"github.com/yourorg/platform/services/blog/internal/service"
|
||
"go.uber.org/fx"
|
||
)
|
||
|
||
func main() {
|
||
cfg := config.Load()
|
||
|
||
fx.New(
|
||
// Core kernel services
|
||
di.CoreModule(cfg),
|
||
|
||
// Blog service implementation
|
||
fx.Provide(service.NewPostService),
|
||
fx.Provide(service.NewPostRepo),
|
||
|
||
// gRPC server
|
||
fx.Provide(api.NewGRPCServer),
|
||
|
||
// Service registry
|
||
fx.Provide(di.ProvideServiceRegistry),
|
||
|
||
// Start service
|
||
fx.Invoke(startService),
|
||
).Run()
|
||
}
|
||
|
||
func startService(lc fx.Lifecycle, server *api.GRPCServer, registry registry.ServiceRegistry) {
|
||
lc.Append(fx.Hook{
|
||
OnStart: func(ctx context.Context) error {
|
||
// Register with Consul
|
||
registry.Register(ctx, ®istry.ServiceInstance{
|
||
ID: "blog-service-1",
|
||
Name: "blog-service",
|
||
Address: "localhost",
|
||
Port: 8091,
|
||
})
|
||
|
||
// Start gRPC server
|
||
return server.Start()
|
||
},
|
||
OnStop: func(ctx context.Context) error {
|
||
registry.Deregister(ctx, "blog-service-1")
|
||
return server.Stop()
|
||
},
|
||
})
|
||
}
|
||
```
|
||
|
||
### 4.3 Service Implementation
|
||
|
||
```go
|
||
// services/blog/internal/service/post_service.go
|
||
package service
|
||
|
||
import (
|
||
"context"
|
||
"github.com/yourorg/platform/pkg/services"
|
||
"github.com/yourorg/platform/services/blog/internal/domain"
|
||
)
|
||
|
||
type PostService struct {
|
||
repo *domain.PostRepo
|
||
authzClient services.AuthzServiceClient
|
||
identityClient services.IdentityServiceClient
|
||
auditClient services.AuditServiceClient
|
||
}
|
||
|
||
func NewPostService(
|
||
repo *domain.PostRepo,
|
||
authzClient services.AuthzServiceClient,
|
||
identityClient services.IdentityServiceClient,
|
||
auditClient services.AuditServiceClient,
|
||
) *PostService {
|
||
return &PostService{
|
||
repo: repo,
|
||
authzClient: authzClient,
|
||
identityClient: identityClient,
|
||
auditClient: auditClient,
|
||
}
|
||
}
|
||
|
||
func (s *PostService) CreatePost(ctx context.Context, req *CreatePostRequest) (*Post, error) {
|
||
// Check permission via Authz Service
|
||
if err := s.authzClient.Authorize(ctx, "blog.post.create"); err != nil {
|
||
return nil, err
|
||
}
|
||
|
||
// Get user info via Identity Service
|
||
user, err := s.identityClient.GetUser(ctx, req.AuthorID)
|
||
if err != nil {
|
||
return nil, err
|
||
}
|
||
|
||
// Create post
|
||
post, err := s.repo.Create(ctx, &domain.Post{
|
||
Title: req.Title,
|
||
Content: req.Content,
|
||
AuthorID: user.ID,
|
||
})
|
||
if err != nil {
|
||
return nil, err
|
||
}
|
||
|
||
// Audit log via Audit Service
|
||
s.auditClient.Record(ctx, &services.AuditAction{
|
||
ActorID: user.ID,
|
||
Action: "blog.post.create",
|
||
TargetID: post.ID,
|
||
})
|
||
|
||
return post, nil
|
||
}
|
||
```
|
||
|
||
### 4.4 gRPC Handler
|
||
|
||
```go
|
||
// services/blog/internal/api/handler.go
|
||
package api
|
||
|
||
import (
|
||
"context"
|
||
"github.com/yourorg/platform/services/blog/api/pb"
|
||
"github.com/yourorg/platform/services/blog/internal/service"
|
||
)
|
||
|
||
type BlogServer struct {
|
||
pb.UnimplementedBlogServiceServer
|
||
service *service.PostService
|
||
}
|
||
|
||
func (s *BlogServer) CreatePost(ctx context.Context, req *pb.CreatePostRequest) (*pb.CreatePostResponse, error) {
|
||
post, err := s.service.CreatePost(ctx, &service.CreatePostRequest{
|
||
Title: req.Title,
|
||
Content: req.Content,
|
||
AuthorID: req.AuthorId,
|
||
})
|
||
if err != nil {
|
||
return nil, err
|
||
}
|
||
|
||
return &pb.CreatePostResponse{
|
||
Post: &pb.Post{
|
||
Id: post.ID,
|
||
Title: post.Title,
|
||
Content: post.Content,
|
||
},
|
||
}, nil
|
||
}
|
||
```
|
||
|
||
**Repository using Ent**
|
||
|
||
```go
|
||
// internal/domain/post_repo.go
|
||
package domain
|
||
|
||
import (
|
||
"context"
|
||
"github.com/yourorg/platform/internal/ent"
|
||
"github.com/yourorg/platform/internal/ent/post"
|
||
)
|
||
|
||
type PostRepo struct{ client *ent.Client }
|
||
|
||
func NewPostRepo(client *ent.Client) *PostRepo { return &PostRepo{client} }
|
||
|
||
func (r *PostRepo) Create(ctx context.Context, p *Post) (*Post, error) {
|
||
entPost, err := r.client.Post.
|
||
Create().
|
||
SetTitle(p.Title).
|
||
SetContent(p.Content).
|
||
SetAuthorID(p.AuthorID).
|
||
Save(ctx)
|
||
if err != nil {
|
||
return nil, err
|
||
}
|
||
return fromEnt(entPost), nil
|
||
}
|
||
```
|
||
|
||
> **Result:** Adding a new feature is just a matter of creating a new folder under `modules/`, implementing `module.IModule`, registering routes, permissions and migrations. The core automatically wires everything together.
|
||
|
||
---
|
||
|
||
## 5 INFRASTRUCTURE ADAPTERS (swap‑able, per‑environment)
|
||
|
||
| Concern | Implementation (Go) | Where it lives |
|
||
|---------|---------------------|----------------|
|
||
| **Database** | `entgo.io/ent` (code‑gen) | `internal/infra/ent/` |
|
||
| **Cache** | `github.com/go-redis/redis/v9` | `internal/infra/cache/` |
|
||
| **Message Queue** | `github.com/segmentio/kafka-go` (Kafka) **or** `github.com/hibiken/asynq` (Redis) | `internal/infra/bus/` |
|
||
| **Blob Storage** | `github.com/aws/aws-sdk-go-v2/service/s3` (or GCS) | `internal/infra/blob/` |
|
||
| **Email** | `github.com/go-mail/mail` (SMTP) | `internal/infra/email/` |
|
||
| **SMS / Push** | Twilio SDK, Firebase Cloud Messaging | `internal/infra/notify/` |
|
||
| **Secret Store** | AWS Secrets Manager (`aws-sdk-go-v2`) or HashiCorp Vault (`github.com/hashicorp/vault/api`) | `internal/infra/secret/` |
|
||
|
||
All adapters expose an **interface** in `pkg/infra/…` and are registered in the DI container as **singletons**.
|
||
|
||
---
|
||
|
||
## 6 OBSERVABILITY STACK
|
||
|
||
| Layer | Library | What it does |
|
||
|-------|---------|--------------|
|
||
| **Tracing** | `go.opentelemetry.io/otel`, `go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp` | Auto‑instrument HTTP, DB (Ent plugin), Kafka, Redis |
|
||
| **Metrics** | `github.com/prometheus/client_golang/prometheus` | Counter/Histogram per request, DB latency, job execution |
|
||
| **Logging** | `go.uber.org/zap` (structured JSON) | Global logger, request‑scoped fields (`request_id`, `user_id`, `tenant_id`) |
|
||
| **Error Reporting** | `github.com/getsentry/sentry-go` (optional) | Capture panics & errors, link to trace ID |
|
||
| **Dashboard** | Grafana + Prometheus + Loki (logs) | Provide ready‑made dashboards in `ops/` folder |
|
||
|
||
**Instrumentation example (HTTP)**
|
||
|
||
```go
|
||
import (
|
||
"net/http"
|
||
"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
|
||
)
|
||
|
||
func main() {
|
||
r := gin.New()
|
||
// Wrap the entire router with OTEL middleware
|
||
wrapped := otelhttp.NewHandler(r, "http-server")
|
||
http.ListenAndServe(":8080", wrapped)
|
||
}
|
||
```
|
||
|
||
**Metrics middleware**
|
||
|
||
```go
|
||
func PromMetrics() gin.HandlerFunc {
|
||
return func(c *gin.Context) {
|
||
start := time.Now()
|
||
c.Next()
|
||
duration := time.Since(start).Seconds()
|
||
method := c.Request.Method
|
||
path := c.FullPath()
|
||
status := fmt.Sprintf("%d", c.Writer.Status())
|
||
requestDuration.WithLabelValues(method, path, status).Observe(duration)
|
||
}
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## 7 CONFIGURATION & ENVIRONMENT
|
||
|
||
```
|
||
config/
|
||
├─ default.yaml # baseline values
|
||
├─ development.yaml
|
||
├─ production.yaml
|
||
└─ secrets/ # git‑ignored, loaded via secret manager
|
||
```
|
||
|
||
**Bootstrap**
|
||
|
||
```go
|
||
func LoadConfig() (*Config, error) {
|
||
v := viper.New()
|
||
v.SetConfigName("default")
|
||
v.AddConfigPath("./config")
|
||
if err := v.ReadInConfig(); err != nil { return nil, err }
|
||
|
||
env := v.GetString("environment") // dev / prod / test
|
||
v.SetConfigName(env)
|
||
v.MergeInConfig() // overrides defaults
|
||
|
||
v.AutomaticEnv() // env vars win
|
||
// optional: secret manager overlay
|
||
return &Config{v}, nil
|
||
}
|
||
```
|
||
|
||
All services receive a `*Config` via DI.
|
||
|
||
---
|
||
|
||
## 8 CI / CD PIPELINE (GitHub Actions)
|
||
|
||
```yaml
|
||
name: CI
|
||
|
||
on:
|
||
push:
|
||
branches: [main]
|
||
pull_request:
|
||
|
||
jobs:
|
||
build:
|
||
runs-on: ubuntu-latest
|
||
env:
|
||
GO111MODULE: on
|
||
steps:
|
||
- uses: actions/checkout@v4
|
||
- name: Set up Go
|
||
uses: actions/setup-go@v5
|
||
with:
|
||
go-version: '1.22'
|
||
- name: Cache Go modules
|
||
uses: actions/cache@v4
|
||
with:
|
||
path: |
|
||
~/.cache/go-build
|
||
~/go/pkg/mod
|
||
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
|
||
- name: Install Tools
|
||
run: |
|
||
go install github.com/vektra/mockery/v2@latest
|
||
go install github.com/golang/mock/mockgen@latest
|
||
- name: Lint
|
||
run: |
|
||
go install golang.org/x/lint/golint@latest
|
||
golint ./...
|
||
- name: Unit Tests
|
||
run: |
|
||
go test ./... -cover -race -short
|
||
- name: Integration Tests (Docker Compose)
|
||
run: |
|
||
docker compose -f ./docker-compose.test.yml up -d
|
||
go test ./... -tags=integration -count=1
|
||
docker compose -f ./docker-compose.test.yml down
|
||
- name: Build Binaries
|
||
run: |
|
||
go build -ldflags="-X main.version=${{ github.sha }}" -o bin/platform ./cmd/platform
|
||
- name: Publish Docker Image
|
||
uses: docker/build-push-action@v5
|
||
with:
|
||
context: .
|
||
push: ${{ github.ref == 'refs/heads/main' }}
|
||
tags: ghcr.io/yourorg/platform:${{ github.sha }}
|
||
```
|
||
|
||
**Key points**
|
||
|
||
* `go test ./... -tags=integration` runs tests that spin up Postgres, Redis, Kafka via **Testcontainers** (`github.com/testcontainers/testcontainers-go`).
|
||
* Linting via `golint` or `staticcheck`.
|
||
* Docker image built from the compiled binary (multi‑stage: `golang:1.22-alpine` → `scratch` or `distroless`).
|
||
* Semantic‑release can be added on top (`semantic-release` action) to tag releases automatically.
|
||
|
||
---
|
||
|
||
## 9 TESTING STRATEGY
|
||
|
||
| Test type | Tools | Typical coverage |
|
||
|-----------|-------|------------------|
|
||
| **Unit** | `testing`, `github.com/stretchr/testify`, `github.com/golang/mock` | Individual services, repositories (use in‑memory DB or mocks). |
|
||
| **Integration** | `testcontainers-go` for Postgres, Redis, Kafka; real Ent client | End‑to‑end request → DB → event bus flow. |
|
||
| **Contract** | `pact-go` or **OpenAPI** validator middleware (`github.com/getkin/kin-openapi`) | Guarantees that modules do not break the published API. |
|
||
| **Load / Stress** | `k6` or `vegeta` scripts in `perf/` | Verify that auth middleware adds < 2 ms latency per request. |
|
||
| **Security** | `gosec`, `zap` for secret detection, OWASP ZAP for API scan | Detect hard‑coded secrets, SQL injection risk. |
|
||
|
||
**Example integration test skeleton**
|
||
|
||
```go
|
||
func TestCreatePost_Integration(t *testing.T) {
|
||
ctx := context.Background()
|
||
// Spin up a PostgreSQL container
|
||
pg, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
|
||
ContainerRequest: testcontainers.ContainerRequest{
|
||
Image: "postgres:15-alpine",
|
||
Env: map[string]string{"POSTGRES_PASSWORD": "secret", "POSTGRES_DB": "test"},
|
||
ExposedPorts: []string{"5432/tcp"},
|
||
WaitingFor: wait.ForListeningPort("5432/tcp"),
|
||
},
|
||
Started: true,
|
||
})
|
||
require.NoError(t, err)
|
||
defer pg.Terminate(ctx)
|
||
|
||
// Build DSN from container host/port
|
||
host, _ := pg.Endpoint(ctx)
|
||
dsn := fmt.Sprintf("postgres://postgres:secret@%s/test?sslmode=disable", host)
|
||
|
||
// Initialize Ent client against that DSN
|
||
client, err := ent.Open("postgres", dsn)
|
||
require.NoError(t, err)
|
||
defer client.Close()
|
||
// Run schema migration
|
||
err = client.Schema.Create(ctx)
|
||
require.NoError(t, err)
|
||
|
||
// Build the whole app using fx, injecting the test client
|
||
app := fx.New(
|
||
core.ProvideAll(),
|
||
fx.Provide(func() *ent.Client { return client }),
|
||
blog.Module.Init(),
|
||
// ...
|
||
)
|
||
// Start the app, issue a HTTP POST request through httptest.Server,
|
||
// assert 201 and DB row existence.
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## 10 COMMON PITFALLS & SOLUTIONS (Go‑centric)
|
||
|
||
| Pitfall | Symptom | Remedy |
|
||
|---------|----------|--------|
|
||
| **Circular imports** (core ↔ module) | `import cycle not allowed` build error | Keep **interfaces only** in `pkg/`. Core implements, modules depend on the interface only. |
|
||
| **Too‑big binary** (all modules compiled in) | Long build times, memory pressure on CI | Use **Go plugins** for truly optional modules; keep core binary minimal. |
|
||
| **Version mismatch with plugins** | Panic: `plugin was built with a different version of package X` | Enforce a **single `replace` directive** for core in each module’s `go.mod` (e.g., `replace github.com/yourorg/platform => ../../platform`). |
|
||
| **Context leakage** (request data not passed) | Logs missing `user_id`, permission checks use zero‑value user | Always store user/tenant info in `context.Context` via middleware; provide helper `auth.FromContext(ctx)`. |
|
||
| **Ent migrations out‑of‑sync** | Startup fails with “column does not exist” | Run `go generate ./...` (ent codegen) and `ent/migrate` automatically at boot **after all module migrations are collected**. |
|
||
| **Hard‑coded permission strings** | Typos go unnoticed → 403 bugs | Use the **generated `perm` package** (see §4.1) and reference constants everywhere. |
|
||
| **Blocking I/O in request path** | 500 ms latency spikes | Off‑load long‑running work to **asynq jobs** or **Kafka consumers**; keep request handlers thin. |
|
||
| **Over‑exposed HTTP handlers** | Missing auth middleware → open endpoint | Wrap the router with a **global security middleware** that checks a whitelist and enforces `Authorizer` for everything else. |
|
||
| **Memory leaks with goroutine workers** | Leak after many module reloads | Use **fx.Lifecycle** to start/stop background workers; always cancel contexts on `Stop`. |
|
||
| **Testing with real DB slows CI** | Pipeline > 10 min | Use **testcontainers** in parallel jobs, cache Docker images, or use in‑memory SQLite for unit tests and only run DB‑heavy tests in a dedicated job. |
|
||
|
||
---
|
||
|
||
## 11 QUICK‑START STEPS (What to code first)
|
||
|
||
1. **Bootstrap repo**
|
||
```bash
|
||
mkdir platform && cd platform
|
||
go mod init github.com/yourorg/platform
|
||
mkdir cmd internal pkg config modules
|
||
touch cmd/main.go
|
||
```
|
||
2. **Create core container** (`internal/di/container.go`) using `dig`. Register Config, Logger, DB, EventBus, Authenticator, Authorizer.
|
||
3. **Add Auth middleware** (`pkg/auth/middleware.go`) that extracts JWT, validates claims, injects `User` into context.
|
||
4. **Implement Permission DSL** (`pkg/perm/perm.go`) and a **simple in‑memory resolver** (`pkg/perm/resolver.go`).
|
||
5. **Write `module` interface** (`pkg/module/module.go`) and a **static registry** (`internal/registry/registry.go`).
|
||
6. **Add first plug‑in** – copy the **Blog** module skeleton from §4.
|
||
`cd modules/blog && go mod init github.com/yourorg/blog && go run ../..` (build & test).
|
||
7. **Wire everything in `cmd/main.go`** using `fx.New(...)`:
|
||
```go
|
||
app := fx.New(
|
||
core.Module, // registers core services
|
||
fx.Invoke(registry.RegisterAllModules), // loads static modules
|
||
fx.Invoke(func(lc fx.Lifecycle, r *gin.Engine) {
|
||
lc.Append(fx.Hook{
|
||
OnStart: func(context.Context) error {
|
||
go r.Run(":8080")
|
||
return nil
|
||
},
|
||
OnStop: func(ctx context.Context) error { return nil },
|
||
})
|
||
}),
|
||
)
|
||
app.Run()
|
||
```
|
||
8. **Run integration test** (`go test ./... -tags=integration`) to sanity‑check DB + routes.
|
||
9. **Add CI workflow** (copy the `.github/workflows/ci.yml` from §8) and push.
|
||
10. **Publish first Docker image** (`docker build -t ghcr.io/yourorg/platform:dev .`).
|
||
|
||
After step 10 you have a **complete, production‑grade scaffolding** that:
|
||
|
||
* authenticates users via JWT (with optional OIDC),
|
||
* authorizes actions through a compile‑time‑checked permission DSL,
|
||
* logs/metrics/traces out of the box,
|
||
* lets any team drop a new `*.so` or static module under `modules/` to add resources,
|
||
* ships with a working CI pipeline and a ready‑to‑run Docker image.
|
||
|
||
---
|
||
|
||
## 12 REFERENCE IMPLEMENTATION (public)
|
||
|
||
If you prefer to start from a **real open‑source baseline**, check out the following community projects that already adopt most of the ideas above:
|
||
|
||
| Repo | Highlights |
|
||
|------|------------|
|
||
| `github.com/go‑microservices/clean‑arch` | Clean‑architecture skeleton, fx DI, Ent ORM, JWT auth |
|
||
| `github.com/ory/hydra` | Full OIDC provider (good for auth reference) |
|
||
| `github.com/segmentio/kafka-go` examples | Event‑bus integration |
|
||
| `github.com/ThreeDotsLabs/watermill` | Pub/sub abstraction usable as bus wrapper |
|
||
| `github.com/hibiken/asynq` | Background job framework (Redis based) |
|
||
|
||
Fork one, strip the business logic, and rename the packages to match *your* `github.com/yourorg/platform` namespace.
|
||
|
||
---
|
||
|
||
## 13 FINAL CHECKLIST (before you ship)
|
||
|
||
- [ ] Core modules compiled & registered in `internal/di`.
|
||
- [ ] `module.IModule` interface and static registry in place.
|
||
- [ ] JWT auth + middleware + `User` context helper.
|
||
- [ ] Permission constants generated & compiled in `pkg/perm`.
|
||
- [ ] Ent schema + migration runner that aggregates all module migrations.
|
||
- [ ] OpenTelemetry tracer/provider wired at process start.
|
||
- [ ] Prometheus metrics endpoint (`/metrics`).
|
||
- [ ] Health endpoints (`/healthz`, `/ready`).
|
||
- [ ] Dockerfile (multi‑stage) and `docker-compose.yml` for dev.
|
||
- [ ] GitHub Actions pipeline (CI) passes all tests.
|
||
- [ ] Sample plug‑in (Blog) builds, loads, registers routes, and passes integration test.
|
||
- [ ] Documentation: `README.md`, `docs/architecture.md`, `docs/extension-points.md`.
|
||
|
||
> **Congratulations!** You now have a **robust, extensible Go platform boilerplate** that can be the foundation for any SaaS, internal toolset, or micro‑service ecosystem you wish to build. Happy coding! |