goplt/docs/content/architecture/module-integration-patterns.md

Module Integration Patterns

Purpose

This document explains how modules integrate with the core platform, focusing on module discovery, initialization, service integration, and communication patterns rather than detailed implementation.

Overview

Modules are independent services that extend the platform's functionality. They integrate with the core platform through well-defined interfaces, service clients, and a standardized initialization process. Each module operates as an independent service while leveraging core platform capabilities.

Key Concepts

• Module: Independent service providing specific functionality
• Module Manifest: YAML file defining module metadata and configuration
• Module Interface: Standard interface all modules implement
• Service Clients: Abstraction for inter-service communication
• Module Registry: Registry tracking all loaded modules

Module Discovery Process

Modules are discovered automatically during application startup by scanning module directories.

sequenceDiagram
    participant Main
    participant ModuleLoader
    participant FileSystem
    participant ModuleManifest
    participant ModuleRegistry
    
    Main->>ModuleLoader: DiscoverModules()
    ModuleLoader->>FileSystem: Scan modules/ directory
    FileSystem-->>ModuleLoader: Module directories
    
    loop For each module directory
        ModuleLoader->>FileSystem: Read module.yaml
        FileSystem-->>ModuleLoader: Module manifest
        ModuleLoader->>ModuleManifest: Parse manifest
        ModuleManifest-->>ModuleLoader: Module metadata
        
        ModuleLoader->>ModuleRegistry: Register module
        ModuleRegistry->>ModuleRegistry: Validate manifest
        ModuleRegistry->>ModuleRegistry: Check dependencies
        ModuleRegistry-->>ModuleLoader: Module registered
    end
    
    ModuleLoader->>ModuleRegistry: Resolve dependencies
    ModuleRegistry->>ModuleRegistry: Build dependency graph
    ModuleRegistry->>ModuleRegistry: Order modules
    ModuleRegistry-->>ModuleLoader: Ordered module list
    ModuleLoader-->>Main: Module list ready

Discovery Steps

1. Directory Scanning: Scan modules/ directory for module subdirectories
2. Manifest Loading: Load module.yaml from each module directory
3. Manifest Parsing: Parse manifest to extract metadata
4. Dependency Extraction: Extract module dependencies from manifest
5. Module Registration: Register module in module registry
6. Dependency Resolution: Build dependency graph and order modules
7. Validation: Validate all dependencies are available

Module Initialization Flow

Modules are initialized in dependency order, ensuring all dependencies are available before module initialization.

sequenceDiagram
    participant Main
    participant ModuleRegistry
    participant Module
    participant DI
    participant Router
    participant ServiceRegistry
    participant DB
    
    Main->>ModuleRegistry: GetOrderedModules()
    ModuleRegistry-->>Main: Ordered module list
    
    loop For each module (dependency order)
        Main->>Module: Init()
        
        Module->>DI: Provide services
        DI->>DI: Register module services
        DI-->>Module: Services registered
        
        Module->>Router: Register routes
        Router->>Router: Add route handlers
        Router-->>Module: Routes registered
        
        Module->>DB: Register migrations
        DB->>DB: Store migration info
        DB-->>Module: Migrations registered
        
        Module->>ServiceRegistry: Register service
        ServiceRegistry->>ServiceRegistry: Register with registry
        ServiceRegistry-->>Module: Service registered
        
        Module->>Module: OnStart hook (optional)
        Module-->>Main: Module initialized
    end
    
    Main->>DB: Run migrations
    DB->>DB: Execute in dependency order
    DB-->>Main: Migrations complete
    
    Main->>Router: Start HTTP server
    Main->>ServiceRegistry: Start service discovery

Initialization Phases

1. Dependency Resolution: Determine module initialization order
2. Service Registration: Register module services in DI container
3. Route Registration: Register HTTP routes
4. Migration Registration: Register database migrations
5. Service Registration: Register module as service in service registry
6. Lifecycle Hooks: Execute OnStart hooks if defined
7. Migration Execution: Run migrations in dependency order
8. Server Startup: Start HTTP and gRPC servers

Module Service Integration

Modules integrate with core services through service client interfaces, ensuring all communication goes through well-defined abstractions.

graph TB
    subgraph "Module Service"
        ModuleHandler[Module Handler]
        ModuleService[Module Service]
        ModuleRepo[Module Repository]
    end
    
    subgraph "Service Clients"
        AuthClient[Auth Service Client]
        IdentityClient[Identity Service Client]
        AuthzClient[Authz Service Client]
        AuditClient[Audit Service Client]
    end
    
    subgraph "Core Services"
        AuthService[Auth Service<br/>:8081]
        IdentityService[Identity Service<br/>:8082]
        AuthzService[Authz Service<br/>:8083]
        AuditService[Audit Service<br/>:8084]
    end
    
    subgraph "Infrastructure"
        EventBus[Event Bus]
        Cache[Cache]
        DB[(Database)]
    end
    
    ModuleHandler --> ModuleService
    ModuleService --> ModuleRepo
    ModuleRepo --> DB
    
    ModuleService -->|gRPC| AuthClient
    ModuleService -->|gRPC| IdentityClient
    ModuleService -->|gRPC| AuthzClient
    ModuleService -->|gRPC| AuditClient
    
    AuthClient --> AuthService
    IdentityClient --> IdentityService
    AuthzClient --> AuthzService
    AuditClient --> AuditService
    
    ModuleService --> EventBus
    ModuleService --> Cache
    
    style ModuleService fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
    style AuthClient fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff

Service Integration Points

1. Authentication: Use Auth Service Client for token validation
2. Identity: Use Identity Service Client for user operations
3. Authorization: Use Authz Service Client for permission checks
4. Audit: Use Audit Service Client for audit logging
5. Event Bus: Publish and subscribe to events
6. Cache: Use cache for performance optimization
7. Database: Direct database access via repositories

Module Data Management

Modules manage their own data while sharing database infrastructure.

graph TD
    subgraph "Module A"
        ModuleA[Module A Service]
        RepoA[Module A Repository]
        SchemaA[Module A Schema<br/>blog_posts]
    end
    
    subgraph "Module B"
        ModuleB[Module B Service]
        RepoB[Module B Repository]
        SchemaB[Module B Schema<br/>billing_subscriptions]
    end
    
    subgraph "Shared Database"
        DB[(PostgreSQL)]
    end
    
    subgraph "Migrations"
        MigrationA[Module A Migrations]
        MigrationB[Module B Migrations]
    end
    
    ModuleA --> RepoA
    RepoA --> SchemaA
    SchemaA --> DB
    
    ModuleB --> RepoB
    RepoB --> SchemaB
    SchemaB --> DB
    
    MigrationA --> DB
    MigrationB --> DB
    
    style ModuleA fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
    style ModuleB fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
    style DB fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff

Data Isolation Patterns

1. Schema Isolation: Each module has its own database schema
2. Table Prefixing: Module tables prefixed with module name
3. Migration Isolation: Each module manages its own migrations
4. Shared Database: Modules share database instance but not schemas
5. Cross-Module Queries: Use service clients, not direct SQL joins

Module Permission System

Modules register permissions that are automatically integrated into the platform's permission system.

sequenceDiagram
    participant Module
    participant ModuleManifest
    participant PermissionGenerator
    participant PermissionRegistry
    participant AuthzService
    
    Module->>ModuleManifest: Define permissions
    ModuleManifest->>ModuleManifest: permissions:
      - blog.post.create
      - blog.post.read
      - blog.post.update
      - blog.post.delete
    
    Module->>PermissionGenerator: Generate permission code
    PermissionGenerator->>PermissionGenerator: Parse manifest
    PermissionGenerator->>PermissionGenerator: Generate constants
    PermissionGenerator-->>Module: Permission code generated
    
    Module->>PermissionRegistry: Register permissions
    PermissionRegistry->>PermissionRegistry: Validate format
    PermissionRegistry->>PermissionRegistry: Store permissions
    PermissionRegistry-->>Module: Permissions registered
    
    AuthzService->>PermissionRegistry: Resolve permissions
    PermissionRegistry-->>AuthzService: Permission list
    AuthzService->>AuthzService: Check permissions

Permission Registration Flow

1. Permission Definition: Define permissions in module.yaml
2. Code Generation: Generate permission constants from manifest
3. Permission Registration: Register permissions during module initialization
4. Permission Validation: Validate permission format and uniqueness
5. Permission Resolution: Permissions available for authorization checks

Module Communication Patterns

Modules communicate with each other through event bus and service clients.

graph TB
    subgraph "Module A"
        ServiceA[Module A Service]
    end
    
    subgraph "Module B"
        ServiceB[Module B Service]
    end
    
    subgraph "Event Bus"
        EventBus[Event Bus<br/>Kafka]
    end
    
    subgraph "Service Clients"
        ClientA[Service Client A]
        ClientB[Service Client B]
    end
    
    subgraph "Module C"
        ServiceC[Module C Service]
    end
    
    ServiceA -->|Publish Event| EventBus
    EventBus -->|Subscribe| ServiceB
    EventBus -->|Subscribe| ServiceC
    
    ServiceA -->|gRPC Call| ClientA
    ClientA --> ServiceB
    
    ServiceB -->|gRPC Call| ClientB
    ClientB --> ServiceC
    
    style ServiceA fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
    style ServiceB fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
    style EventBus fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff

Communication Patterns

Event-Based Communication

sequenceDiagram
    participant ModuleA
    participant EventBus
    participant ModuleB
    participant ModuleC
    
    ModuleA->>EventBus: Publish event
    EventBus->>EventBus: Route to subscribers
    EventBus->>ModuleB: Deliver event
    EventBus->>ModuleC: Deliver event
    
    ModuleB->>ModuleB: Process event
    ModuleC->>ModuleC: Process event
    
    Note over ModuleB,ModuleC: Events processed independently

Service Client Communication

sequenceDiagram
    participant ModuleA
    participant Client
    participant ServiceRegistry
    participant ModuleB
    
    ModuleA->>Client: Call service method
    Client->>ServiceRegistry: Discover Module B
    ServiceRegistry-->>Client: Module B endpoint
    Client->>ModuleB: gRPC call
    ModuleB->>ModuleB: Process request
    ModuleB-->>Client: Response
    Client-->>ModuleA: Return result

Module Route Registration

Modules register their HTTP routes with the platform's router.

sequenceDiagram
    participant Module
    participant Router
    participant AuthzMiddleware
    participant ModuleHandler
    
    Module->>Router: Register routes
    Module->>Router: Define route: /api/v1/blog/posts
    Module->>Router: Define permission: blog.post.create
    Module->>Router: Define handler: CreatePostHandler
    
    Router->>Router: Create route
    Router->>AuthzMiddleware: Register permission check
    Router->>Router: Attach handler
    
    Router->>Router: Route registered
    
    Note over Router: Routes are registered with<br/>permission requirements

Route Registration Process

1. Route Definition: Module defines routes in Init() method
2. Permission Association: Routes associated with required permissions
3. Handler Registration: Handlers registered with router
4. Middleware Attachment: Authorization middleware automatically attached
5. Route Activation: Routes available when HTTP server starts

Integration Points

This module integration integrates with:

• System Behavior Overview: How modules participate in system bootstrap
• Service Orchestration: How modules operate as services
• Operational Scenarios: Module behavior in specific scenarios
• Architecture Modules: Detailed module architecture

Related Documentation

• System Behavior Overview - System-level behavior
• Service Orchestration - Service coordination
• Operational Scenarios - Module usage scenarios
• Architecture Modules - Module architecture details
• Module Requirements - Module requirements and interfaces