goplt/docs/content/architecture/module-integration-patterns.md

# Module Integration Patterns

## Purpose

This document explains how modules integrate with the core platform, focusing on module discovery, initialization, service integration, and communication patterns rather than detailed implementation.

## Overview

Modules are implemented as independent services that extend the platform's functionality. They integrate with core services through service client interfaces, service discovery (Consul), and a standardized initialization process. Each module operates as an independent service with its own entry point, database connection, and deployment while leveraging core platform capabilities.

## Key Concepts

- **Module**: Independent service providing specific functionality
- **Module Manifest**: YAML file defining module metadata and configuration
- **Module Interface**: Standard interface all modules implement
- **Service Clients**: Abstraction for inter-service communication
- **Module Registry**: Registry tracking all loaded modules

## Module Discovery Process

Modules are discovered automatically during application startup by scanning module directories.

```mermaid
sequenceDiagram
    participant Main
    participant ModuleLoader
    participant FileSystem
    participant ModuleManifest
    participant ModuleRegistry
    
    Main->>ModuleLoader: DiscoverModules()
    ModuleLoader->>FileSystem: Scan modules/ directory
    FileSystem-->>ModuleLoader: Module directories
    
    loop For each module directory
        ModuleLoader->>FileSystem: Read module.yaml
        FileSystem-->>ModuleLoader: Module manifest
        ModuleLoader->>ModuleManifest: Parse manifest
        ModuleManifest-->>ModuleLoader: Module metadata
        
        ModuleLoader->>ModuleRegistry: Register module
        ModuleRegistry->>ModuleRegistry: Validate manifest
        ModuleRegistry->>ModuleRegistry: Check dependencies
        ModuleRegistry-->>ModuleLoader: Module registered
    end
    
    ModuleLoader->>ModuleRegistry: Resolve dependencies
    ModuleRegistry->>ModuleRegistry: Build dependency graph
    ModuleRegistry->>ModuleRegistry: Order modules
    ModuleRegistry-->>ModuleLoader: Ordered module list
    ModuleLoader-->>Main: Module list ready
```

### Discovery Steps

1. **Directory Scanning**: Scan `modules/` directory for module subdirectories
2. **Manifest Loading**: Load `module.yaml` from each module directory
3. **Manifest Parsing**: Parse manifest to extract metadata
4. **Dependency Extraction**: Extract module dependencies from manifest
5. **Module Registration**: Register module in module registry
6. **Dependency Resolution**: Build dependency graph and order modules
7. **Validation**: Validate all dependencies are available

## Module Initialization Flow

Modules are initialized in dependency order, ensuring all dependencies are available before module initialization.

```mermaid
sequenceDiagram
    participant Main
    participant ModuleRegistry
    participant Module
    participant DI
    participant Router
    participant ServiceRegistry
    participant DB
    
    Main->>ModuleRegistry: GetOrderedModules()
    ModuleRegistry-->>Main: Ordered module list
    
    loop For each module (dependency order)
        Main->>Module: Init()
        
        Module->>DI: Provide services
        DI->>DI: Register module services
        DI-->>Module: Services registered
        
        Module->>Router: Register routes
        Router->>Router: Add route handlers
        Router-->>Module: Routes registered
        
        Module->>DB: Register migrations
        DB->>DB: Store migration info
        DB-->>Module: Migrations registered
        
        Module->>ServiceRegistry: Register service
        ServiceRegistry->>ServiceRegistry: Register with Consul
        ServiceRegistry-->>Module: Service registered
        
        Module->>Module: OnStart hook (optional)
        Module-->>Main: Module initialized
    end
    
    Main->>DB: Run migrations
    DB->>DB: Execute in dependency order
    DB-->>Main: Migrations complete
    
    Main->>Router: Start HTTP server
    Main->>ServiceRegistry: Start service discovery
```

### Initialization Phases

1. **Dependency Resolution**: Determine module initialization order
2. **Service Registration**: Register module services in DI container
3. **Route Registration**: Register HTTP routes
4. **Migration Registration**: Register database migrations
5. **Service Registration**: Register module as service in service registry
6. **Lifecycle Hooks**: Execute OnStart hooks if defined
7. **Migration Execution**: Run migrations in dependency order
8. **Server Startup**: Start HTTP and gRPC servers

## Module Service Integration

Modules integrate with core services through service client interfaces, ensuring all communication goes through well-defined abstractions.

```mermaid
graph TB
    subgraph "Module Service"
        ModuleHandler[Module Handler]
        ModuleService[Module Service]
        ModuleRepo[Module Repository]
    end
    
    subgraph "Service Clients"
        AuthClient[Auth Service Client]
        IdentityClient[Identity Service Client]
        AuthzClient[Authz Service Client]
        AuditClient[Audit Service Client]
    end
    
    subgraph "Core Services"
        AuthService[Auth Service<br/>:8081]
        IdentityService[Identity Service<br/>:8082]
        AuthzService[Authz Service<br/>:8083]
        AuditService[Audit Service<br/>:8084]
    end
    
    subgraph "Infrastructure"
        EventBus[Event Bus]
        Cache[Cache]
        DB[(Database)]
    end
    
    ModuleHandler --> ModuleService
    ModuleService --> ModuleRepo
    ModuleRepo --> DB
    
    ModuleService -->|gRPC| AuthClient
    ModuleService -->|gRPC| IdentityClient
    ModuleService -->|gRPC| AuthzClient
    ModuleService -->|gRPC| AuditClient
    
    AuthClient --> AuthService
    IdentityClient --> IdentityService
    AuthzClient --> AuthzService
    AuditClient --> AuditService
    
    ModuleService --> EventBus
    ModuleService --> Cache
    
    style ModuleService fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
    style AuthClient fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
```

### Service Integration Points

1. **Authentication**: Use Auth Service Client for token validation
2. **Identity**: Use Identity Service Client for user operations
3. **Authorization**: Use Authz Service Client for permission checks
4. **Audit**: Use Audit Service Client for audit logging
5. **Event Bus**: Publish and subscribe to events
6. **Cache**: Use cache for performance optimization
7. **Database**: Direct database access via repositories

## Module Data Management

Modules manage their own data while sharing database infrastructure.

```mermaid
graph TD
    subgraph "Module A"
        ModuleA[Module A Service]
        RepoA[Module A Repository]
        SchemaA[Module A Schema<br/>blog_posts]
    end
    
    subgraph "Module B"
        ModuleB[Module B Service]
        RepoB[Module B Repository]
        SchemaB[Module B Schema<br/>billing_subscriptions]
    end
    
    subgraph "Shared Database"
        DB[(PostgreSQL)]
    end
    
    subgraph "Migrations"
        MigrationA[Module A Migrations]
        MigrationB[Module B Migrations]
    end
    
    ModuleA --> RepoA
    RepoA --> SchemaA
    SchemaA --> DB
    
    ModuleB --> RepoB
    RepoB --> SchemaB
    SchemaB --> DB
    
    MigrationA --> DB
    MigrationB --> DB
    
    style ModuleA fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
    style ModuleB fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
    style DB fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
```

### Data Isolation Patterns

1. **Schema Isolation**: Each module has its own database schema
2. **Table Prefixing**: Module tables prefixed with module name
3. **Migration Isolation**: Each module manages its own migrations
4. **Shared Database**: Modules share database instance but not schemas
5. **Cross-Module Queries**: Use service clients, not direct SQL joins

## Module Permission System

Modules register permissions that are automatically integrated into the platform's permission system.

```mermaid
sequenceDiagram
    participant Module
    participant ModuleManifest
    participant PermissionGenerator
    participant PermissionRegistry
    participant AuthzService
    
    Module->>ModuleManifest: Define permissions
    ModuleManifest->>ModuleManifest: permissions: blog.post.create, blog.post.read, blog.post.update, blog.post.delete
    
    Module->>PermissionGenerator: Generate permission code
    PermissionGenerator->>PermissionGenerator: Parse manifest
    PermissionGenerator->>PermissionGenerator: Generate constants
    PermissionGenerator-->>Module: Permission code generated
    
    Module->>PermissionRegistry: Register permissions
    PermissionRegistry->>PermissionRegistry: Validate format
    PermissionRegistry->>PermissionRegistry: Store permissions
    PermissionRegistry-->>Module: Permissions registered
    
    AuthzService->>PermissionRegistry: Resolve permissions
    PermissionRegistry-->>AuthzService: Permission list
    AuthzService->>AuthzService: Check permissions
```

### Permission Registration Flow

1. **Permission Definition**: Define permissions in `module.yaml`
2. **Code Generation**: Generate permission constants from manifest
3. **Permission Registration**: Register permissions during module initialization
4. **Permission Validation**: Validate permission format and uniqueness
5. **Permission Resolution**: Permissions available for authorization checks

## Module Communication Patterns

Modules communicate with each other through event bus and service clients.

```mermaid
graph TB
    subgraph "Module A"
        ServiceA[Module A Service]
    end
    
    subgraph "Module B"
        ServiceB[Module B Service]
    end
    
    subgraph "Event Bus"
        EventBus[Event Bus<br/>Kafka]
    end
    
    subgraph "Service Clients"
        ClientA[Service Client A]
        ClientB[Service Client B]
    end
    
    subgraph "Module C"
        ServiceC[Module C Service]
    end
    
    ServiceA -->|Publish Event| EventBus
    EventBus -->|Subscribe| ServiceB
    EventBus -->|Subscribe| ServiceC
    
    ServiceA -->|gRPC Call| ClientA
    ClientA --> ServiceB
    
    ServiceB -->|gRPC Call| ClientB
    ClientB --> ServiceC
    
    style ServiceA fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
    style ServiceB fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
    style EventBus fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
```

### Communication Patterns

#### Event-Based Communication

```mermaid
sequenceDiagram
    participant ModuleA
    participant EventBus
    participant ModuleB
    participant ModuleC
    
    ModuleA->>EventBus: Publish event
    EventBus->>EventBus: Route to subscribers
    EventBus->>ModuleB: Deliver event
    EventBus->>ModuleC: Deliver event
    
    ModuleB->>ModuleB: Process event
    ModuleC->>ModuleC: Process event
    
    Note over ModuleB,ModuleC: Events processed independently
```

#### Service Client Communication

```mermaid
sequenceDiagram
    participant ModuleA
    participant Client
    participant ServiceRegistry
    participant ModuleB
    
    ModuleA->>Client: Call service method
    Client->>ServiceRegistry: Discover Module B
    ServiceRegistry-->>Client: Module B endpoint
    Client->>ModuleB: gRPC call
    ModuleB->>ModuleB: Process request
    ModuleB-->>Client: Response
    Client-->>ModuleA: Return result
```

## Module Route Registration

Modules register their HTTP routes with the platform's router.

```mermaid
sequenceDiagram
    participant Module
    participant Router
    participant AuthzMiddleware
    participant ModuleHandler
    
    Module->>Router: Register routes
    Module->>Router: Define route: /api/v1/blog/posts
    Module->>Router: Define permission: blog.post.create
    Module->>Router: Define handler: CreatePostHandler
    
    Router->>Router: Create route
    Router->>AuthzMiddleware: Register permission check
    Router->>Router: Attach handler
    
    Router->>Router: Route registered
    
    Note over Router: Routes are registered with<br/>permission requirements
```

### Route Registration Process

1. **Route Definition**: Module defines routes in `Init()` method
2. **Permission Association**: Routes associated with required permissions
3. **Handler Registration**: Handlers registered with router
4. **Middleware Attachment**: Authorization middleware automatically attached
5. **Route Activation**: Routes available when HTTP server starts

## Integration Points

This module integration integrates with:

- **[System Behavior Overview](system-behavior.md)**: How modules participate in system bootstrap
- **[Service Orchestration](service-orchestration.md)**: How modules operate as services
- **[Operational Scenarios](operational-scenarios.md)**: Module behavior in specific scenarios
- **[Architecture Modules](architecture-modules.md)**: Detailed module architecture

## Related Documentation

- [System Behavior Overview](system-behavior.md) - System-level behavior
- [Service Orchestration](service-orchestration.md) - Service coordination
- [Operational Scenarios](operational-scenarios.md) - Module usage scenarios
- [Architecture Modules](architecture-modules.md) - Module architecture details
- [Module Requirements](module-requirements.md) - Module requirements and interfaces