docs: update dead links

2025-11-05 11:00:36 +01:00
parent b4f8875a0e
commit 66b0c3b40d
20 changed files with 126 additions and 83 deletions
--- a/docs/content/architecture/data-flow-patterns.md
+++ b/docs/content/architecture/data-flow-patterns.md
@@ -0,0 +1,423 @@
+# Data Flow Patterns
+
+## Purpose
+
+This document describes how data flows through the Go Platform system, covering request/response flows, event flows, cache patterns, and observability data collection.
+
+## Overview
+
+Data flows through the platform in multiple patterns depending on the type of operation. Understanding these patterns helps in debugging, performance optimization, and system design decisions.
+
+## Key Concepts
+
+- **Request Flow**: Data flow from HTTP request to response
+- **Event Flow**: Asynchronous data flow through event bus
+- **Cache Flow**: Data flow through caching layers
+- **Observability Flow**: Telemetry data collection and export
+
+## Request/Response Data Flow
+
+### Standard HTTP Request Flow
+
+Complete data flow from HTTP request to response.
+
+```mermaid
+graph TD
+    Start[HTTP Request] --> Auth[Authentication]
+    Auth -->|Valid| Authz[Authorization]
+    Auth -->|Invalid| Error1[401 Response]
+    
+    Authz -->|Authorized| Handler[Request Handler]
+    Authz -->|Unauthorized| Error2[403 Response]
+    
+    Handler --> Service[Domain Service]
+    Service --> Cache{Cache Check}
+    
+    Cache -->|Hit| CacheData[Return Cached Data]
+    Cache -->|Miss| Repo[Repository]
+    
+    Repo --> DB[(Database)]
+    DB --> Repo
+    Repo --> Service
+    Service --> CacheStore[Update Cache]
+    
+    Service --> EventBus[Publish Events]
+    Service --> Audit[Audit Log]
+    Service --> Metrics[Update Metrics]
+    
+    Service --> Handler
+    Handler --> Response[HTTP Response]
+    CacheData --> Response
+    Error1 --> Response
+    Error2 --> Response
+    
+    Response --> Client[Client]
+    
+    style Auth fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px,color:#fff
+    style Service fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
+    style Cache fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
+```
+
+### Request Data Transformation
+
+How request data is transformed as it flows through the system.
+
+```mermaid
+sequenceDiagram
+    participant Client
+    participant Handler
+    participant Service
+    participant Repo
+    participant DB
+    
+    Client->>Handler: HTTP Request (JSON)
+    Handler->>Handler: Parse JSON
+    Handler->>Handler: Validate request
+    Handler->>Handler: Convert to DTO
+    
+    Handler->>Service: Business DTO
+    Service->>Service: Business logic
+    Service->>Service: Domain entity
+    
+    Service->>Repo: Domain entity
+    Repo->>Repo: Convert to DB model
+    Repo->>DB: SQL query
+    
+    DB-->>Repo: DB result
+    Repo->>Repo: Convert to domain entity
+    Repo-->>Service: Domain entity
+    
+    Service->>Service: Business logic
+    Service->>Service: Response DTO
+    Service-->>Handler: Response DTO
+    
+    Handler->>Handler: Convert to JSON
+    Handler-->>Client: HTTP Response (JSON)
+```
+
+## Event Data Flow
+
+### Event Publishing Flow
+
+How events are published and flow through the event bus.
+
+```mermaid
+graph LR
+    Publisher[Event Publisher] --> Serialize[Serialize Event]
+    Serialize --> Metadata[Add Metadata]
+    Metadata --> EventBus[Event Bus]
+    EventBus --> Topic[Kafka Topic]
+    
+    Topic --> Subscriber1[Subscriber 1]
+    Topic --> Subscriber2[Subscriber 2]
+    Topic --> SubscriberN[Subscriber N]
+    
+    Subscriber1 --> Process1[Process Event]
+    Subscriber2 --> Process2[Process Event]
+    SubscriberN --> ProcessN[Process Event]
+    
+    style EventBus fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
+    style Topic fill:#7b68ee,stroke:#5a4fcf,stroke-width:2px,color:#fff
+```
+
+### Event Data Transformation
+
+How event data is transformed during publishing and consumption.
+
+```mermaid
+sequenceDiagram
+    participant Publisher
+    participant EventBus
+    participant Kafka
+    participant Subscriber
+    
+    Publisher->>Publisher: Domain event
+    Publisher->>EventBus: Publish(event)
+    EventBus->>EventBus: Serialize to JSON
+    EventBus->>EventBus: Add metadata:
+      - trace_id
+      - user_id
+      - timestamp
+      - source
+    
+    EventBus->>Kafka: Send to topic
+    Kafka-->>EventBus: Acknowledged
+    
+    Kafka->>Subscriber: Deliver event
+    Subscriber->>Subscriber: Deserialize JSON
+    Subscriber->>Subscriber: Extract metadata
+    Subscriber->>Subscriber: Domain event
+    Subscriber->>Subscriber: Process event
+```
+
+## Cache Data Flow
+
+### Cache-Aside Pattern Flow
+
+How data flows through cache using the cache-aside pattern.
+
+```mermaid
+graph TD
+    Start[Service Request] --> Check{Cache Hit?}
+    Check -->|Yes| GetCache[Get from Cache]
+    Check -->|No| GetDB[Query Database]
+    
+    GetCache --> Deserialize[Deserialize Data]
+    Deserialize --> Return[Return Data]
+    
+    GetDB --> DB[(Database)]
+    DB --> DBData[Database Result]
+    DBData --> Serialize[Serialize Data]
+    Serialize --> StoreCache[Store in Cache]
+    StoreCache --> Return
+    
+    style Check fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
+    style StoreCache fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
+```
+
+### Cache Invalidation Flow
+
+How cache is invalidated when data changes.
+
+```mermaid
+sequenceDiagram
+    participant Service
+    participant Repository
+    participant DB
+    participant Cache
+    
+    Service->>Repository: Update entity
+    Repository->>DB: Update database
+    DB-->>Repository: Update complete
+    
+    Repository->>Cache: Invalidate(key)
+    Cache->>Cache: Remove from cache
+    Cache-->>Repository: Invalidated
+    
+    Repository-->>Service: Update complete
+    
+    Note over Service,Cache: Next read will fetch from DB and cache
+```
+
+### Cache Write-Through Pattern
+
+How data is written through cache to database.
+
+```mermaid
+sequenceDiagram
+    participant Service
+    participant Cache
+    participant Repository
+    participant DB
+    
+    Service->>Cache: Write data
+    Cache->>Cache: Store in cache
+    Cache->>Repository: Write to database
+    Repository->>DB: Insert/Update
+    DB-->>Repository: Success
+    Repository-->>Cache: Write complete
+    Cache-->>Service: Data written
+```
+
+## Observability Data Flow
+
+### Tracing Data Flow
+
+How distributed tracing data flows through the system.
+
+```mermaid
+graph TD
+    Request[HTTP Request] --> Trace[Start Trace]
+    Trace --> Span1[HTTP Span]
+    
+    Span1 --> Service[Service Call]
+    Service --> Span2[Service Span]
+    
+    Span2 --> DB[Database Query]
+    DB --> Span3[DB Span]
+    
+    Span2 --> gRPC[gRPC Call]
+    gRPC --> Span4[gRPC Span]
+    
+    Span3 --> Aggregate[Collect Spans]
+    Span4 --> Aggregate
+    Aggregate --> Export[Export to Collector]
+    Export --> Collector[OpenTelemetry Collector]
+    Collector --> Backend[Backend Storage]
+    
+    style Trace fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
+    style Aggregate fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
+```
+
+### Metrics Data Flow
+
+How metrics are collected and exported.
+
+```mermaid
+sequenceDiagram
+    participant Service
+    participant MetricsRegistry
+    participant Exporter
+    participant Prometheus
+    participant Grafana
+    
+    Service->>Service: Business operation
+    Service->>MetricsRegistry: Increment counter
+    Service->>MetricsRegistry: Record duration
+    Service->>MetricsRegistry: Set gauge
+    
+    MetricsRegistry->>MetricsRegistry: Aggregate metrics
+    
+    Prometheus->>Exporter: Scrape metrics
+    Exporter->>MetricsRegistry: Get metrics
+    MetricsRegistry-->>Exporter: Metrics data
+    Exporter-->>Prometheus: Prometheus format
+    
+    Prometheus->>Prometheus: Store metrics
+    Grafana->>Prometheus: Query metrics
+    Prometheus-->>Grafana: Metrics data
+    Grafana->>Grafana: Render dashboard
+```
+
+### Log Data Flow
+
+How logs flow through the system to various sinks.
+
+```mermaid
+graph TD
+    Service[Service] --> Logger[Logger]
+    Logger --> Format[Format Log]
+    Format --> Output[Output Log]
+    
+    Output --> Stdout[stdout]
+    Output --> File[File]
+    Output --> LogCollector[Log Collector]
+    
+    LogCollector --> Elasticsearch[Elasticsearch]
+    LogCollector --> CloudLogging[Cloud Logging]
+    
+    Stdout --> Container[Container Logs]
+    
+    style Logger fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
+    style LogCollector fill:#50c878,stroke:#2e7d4e,stroke-width:2px,color:#fff
+```
+
+## Audit Data Flow
+
+How audit logs flow through the system.
+
+```mermaid
+sequenceDiagram
+    participant Service
+    participant AuditClient
+    participant AuditService
+    participant DB
+    participant Archive
+    
+    Service->>Service: Security-sensitive action
+    Service->>AuditClient: Record audit log
+    AuditClient->>AuditClient: Build audit entry:
+      - actor
+      - action
+      - target
+      - metadata
+      - timestamp
+    
+    AuditClient->>AuditService: Store audit log
+    AuditService->>AuditService: Validate entry
+    AuditService->>AuditService: Ensure immutability
+    AuditService->>DB: Insert audit log
+    DB-->>AuditService: Log stored
+    
+    AuditService->>Archive: Archive old logs
+    Archive->>Archive: Long-term storage
+    
+    Note over Service,Archive: Audit logs are immutable
+```
+
+## Cross-Service Data Flow
+
+### Inter-Service Request Flow
+
+How data flows when services communicate via service clients.
+
+```mermaid
+sequenceDiagram
+    participant ServiceA
+    participant ServiceClient
+    participant ServiceRegistry
+    participant ServiceB
+    participant DB
+    
+    ServiceA->>ServiceClient: Call service method
+    ServiceClient->>ServiceRegistry: Discover service
+    ServiceRegistry-->>ServiceClient: Service endpoint
+    
+    ServiceClient->>ServiceB: gRPC request
+    ServiceB->>ServiceB: Process request
+    ServiceB->>DB: Query data
+    DB-->>ServiceB: Data
+    ServiceB->>ServiceB: Business logic
+    ServiceB-->>ServiceClient: gRPC response
+    ServiceClient-->>ServiceA: Return data
+```
+
+### Service-to-Service Event Flow
+
+How events flow between services.
+
+```mermaid
+graph LR
+    ServiceA[Service A] -->|Publish| EventBus[Event Bus]
+    EventBus -->|Route| ServiceB[Service B]
+    EventBus -->|Route| ServiceC[Service C]
+    
+    ServiceB -->|Publish| EventBus
+    EventBus -->|Route| ServiceD[Service D]
+    
+    ServiceC -->|Publish| EventBus
+    EventBus -->|Route| ServiceE[Service E]
+    
+    style EventBus fill:#4a90e2,stroke:#2e5c8a,stroke-width:2px,color:#fff
+```
+
+## Data Flow Patterns Summary
+
+### Request Flow Pattern
+- **Path**: Client → HTTP → Handler → Service → Repository → Database
+- **Response**: Database → Repository → Service → Handler → HTTP → Client
+- **Side Effects**: Cache updates, event publishing, audit logging, metrics
+
+### Event Flow Pattern
+- **Path**: Publisher → Event Bus → Kafka → Subscribers
+- **Characteristics**: Asynchronous, eventual consistency, decoupled
+
+### Cache Flow Pattern
+- **Read**: Cache → (miss) → Database → Cache
+- **Write**: Service → Database → Cache invalidation
+- **Characteristics**: Performance optimization, cache-aside pattern
+
+### Observability Flow Pattern
+- **Tracing**: Service → OpenTelemetry → Collector → Backend
+- **Metrics**: Service → Metrics Registry → Prometheus → Grafana
+- **Logs**: Service → Logger → Collector → Storage
+
+## Integration Points
+
+This data flow patterns document integrates with:
+
+- **[System Behavior Overview](system-behavior.md)**: How data flows fit into system behavior
+- **[Service Orchestration](service-orchestration.md)**: How data flows between services
+- **[Module Integration Patterns](module-integration-patterns.md)**: How data flows through modules
+- **[Operational Scenarios](operational-scenarios.md)**: Data flow in specific scenarios
+- **[Component Relationships](component-relationships.md)**: Component-level data flow
+
+## Related Documentation
+
+- [System Behavior Overview](system-behavior.md) - System-level behavior
+- [Service Orchestration](service-orchestration.md) - Service coordination
+- [Module Integration Patterns](module-integration-patterns.md) - Module integration
+- [Operational Scenarios](operational-scenarios.md) - Operational flows
+- [Component Relationships](component-relationships.md) - Component dependencies
+- [Architecture Overview](architecture.md) - System architecture
+