feat: mock gateway

This commit is contained in:
Patrick Balsiger
2025-10-24 14:24:14 +02:00
parent fba1f162b3
commit 3c3fb886a3
9 changed files with 2851 additions and 0 deletions

425
docs/MOCK_GATEWAY.md Normal file
View File

@@ -0,0 +1,425 @@
# Mock Gateway Implementation
This document describes the mock gateway implementation for the SPORE Gateway project.
## Overview
The mock gateway is a fully-featured simulation of the real SPORE Gateway that implements all functionality (WebSocket, REST API) without requiring actual SPORE nodes. It reuses all types defined for the real gateway, ensuring complete API compatibility.
## Project Structure
```
spore-gateway/
├── cmd/
│ └── mock-gateway/
│ ├── main.go # Mock gateway entry point
│ └── README.md # Detailed usage documentation
├── internal/
│ ├── mock/
│ │ ├── discovery.go # Mock node discovery implementation
│ │ ├── server.go # Mock HTTP server with all endpoints
│ │ ├── websocket.go # Mock WebSocket server
│ │ └── data.go # Mock data generators
│ ├── discovery/ # (Reused types)
│ └── websocket/ # (Reused for upgrader)
├── pkg/
│ ├── client/ # (Reused types)
│ ├── config/ # (Reused types)
│ └── registry/ # (Reused types)
└── scripts/
└── run-mock-gateway.sh # Convenient runner script
```
## Key Features
### 1. Complete REST API Implementation
All endpoints from the real gateway are implemented with realistic mock data:
**Discovery Endpoints**:
- `GET /api/discovery/nodes` - Returns all mock nodes
- `POST /api/discovery/refresh` - Mock refresh operation
- `POST /api/discovery/random-primary` - Selects random primary
- `POST /api/discovery/primary/{ip}` - Sets specific primary
**Cluster Endpoints**:
- `GET /api/cluster/members` - Returns mock cluster members
- `POST /api/cluster/refresh` - Mock cluster refresh
- `GET /api/cluster/node/versions` - Returns node versions
- `POST /api/rollout` - Simulates firmware rollout with progress
**Node Endpoints**:
- `GET /api/node/status` - Mock system status
- `GET /api/node/status/{ip}` - Specific node status
- `GET /api/node/endpoints` - Mock API capabilities
- `POST /api/node/update` - Simulated firmware upload
**Task Endpoints**:
- `GET /api/tasks/status` - Mock task scheduler status
**Monitoring Endpoints**:
- `GET /api/monitoring/resources` - Real-time resource monitoring for all nodes
**Proxy Endpoints**:
- `POST /api/proxy-call` - Mock proxied calls
**Registry Endpoints**:
- `GET /api/registry/health` - Mock registry health
- `GET /api/registry/firmware` - Mock firmware list
- `POST /api/registry/firmware` - Mock firmware upload
- `GET /api/registry/firmware/{name}/{version}` - Mock firmware download
- `PUT /api/registry/firmware/{name}/{version}` - Mock metadata update
- `DELETE /api/registry/firmware/{name}/{version}` - Mock deletion
**WebSocket**:
- `GET /ws` - WebSocket endpoint for real-time updates
**Health**:
- `GET /api/health` - Gateway health check
### 2. WebSocket Implementation
The mock gateway implements full WebSocket functionality with real-time broadcasts:
**Broadcast Types**:
- **Cluster Updates**: Periodic and event-driven cluster state updates
- **Node Discovery**: Node join/leave/update events
- **Firmware Upload Status**: Upload progress notifications
- **Rollout Progress**: Step-by-step rollout progress with percentage
**Message Format**: Identical to real gateway for compatibility
### 3. Mock Node Discovery
Simulates node discovery without UDP:
- Configurable number of nodes (default: 5)
- Realistic node data (IP, hostname, labels, metrics)
- Automatic heartbeat simulation
- Primary node selection
- Node status management (active/inactive)
**Generated Nodes**:
- IPs: `192.168.1.100`, `192.168.1.101`, etc.
- Hostnames: `spore-node-1`, `spore-node-2`, etc.
- Labels: version (matches firmware registry), stable, env, zone, type
- Resources: freeHeap, cpuFreqMHz, flashChipSize
- Latency: Random realistic values
- Firmware Distribution: 40% on v1.0.0, 40% on v1.1.0, 20% on v1.2.0 (beta)
### 4. Type Reuse
The mock gateway reuses all types from the real gateway:
**From `internal/discovery`**:
- `NodeInfo` - Node information structure
- `ClusterStatus` - Cluster state
- `NodeStatus` - Node status enum
- `NodeUpdateCallback` - Callback function type
**From `pkg/client`**:
- `ClusterMember` - Cluster member info
- `ClusterStatusResponse` - Cluster status response
- `TaskStatusResponse` - Task status
- `SystemStatusResponse` - System status
- `CapabilitiesResponse` - API capabilities
- `EndpointInfo` - Endpoint information
- `ParameterInfo` - Parameter details
- `FirmwareUpdateResponse` - Firmware update result
**From `pkg/registry`**:
- `FirmwareRecord` - Firmware metadata
- `GroupedFirmware` - Grouped firmware list
- `FirmwareMetadata` - Firmware metadata for uploads
**From `pkg/config`**:
- `Config` - Configuration structure
This ensures complete API compatibility and type safety.
## Usage
### Quick Start
```bash
# Build and run
go build -o mock-gateway cmd/mock-gateway/main.go
./mock-gateway
# Or use the convenience script
./scripts/run-mock-gateway.sh
```
### Configuration Options
```bash
./mock-gateway [options]
-port string
HTTP server port (default "3001")
-mock-nodes int
Number of mock nodes to simulate (default 5)
-heartbeat-rate int
Heartbeat interval in seconds (default 5)
-log-level string
Log level: debug, info, warn, error (default "info")
-enable-ws bool
Enable WebSocket broadcasts (default true)
-config string
Path to configuration file
```
### Example Usage
```bash
# Development with debug logging
./mock-gateway -log-level debug -mock-nodes 3
# Production-like setup
./mock-gateway -port 3001 -mock-nodes 10 -heartbeat-rate 5
# Testing without WebSocket
./mock-gateway -enable-ws=false
# Large cluster simulation
./mock-gateway -mock-nodes 20 -heartbeat-rate 2
```
### Using the Convenience Script
```bash
# Basic usage
./scripts/run-mock-gateway.sh
# With options
./scripts/run-mock-gateway.sh -p 8080 -n 10 -l debug
# Show help
./scripts/run-mock-gateway.sh --help
```
## Mock Data
### Pre-configured Firmware
**spore-firmware**:
- v1.0.0 - Stable, production (40% of nodes)
- v1.1.0 - Stable, production (40% of nodes)
- v1.2.0 - Beta (20% of nodes)
**sensor-firmware**:
- v2.0.0 - Stable, sensor type
- v2.1.0 - Stable, sensor type
Node labels automatically match the firmware versions in the registry, ensuring consistency between running firmware and available versions.
### Simulated Behaviors
**Firmware Uploads**:
- 2-second simulated processing time
- 90% success rate
- WebSocket status broadcasts
- In-memory storage
**Rollouts**:
- Step-by-step simulation:
1. Label update (500ms)
2. Firmware upload (1s)
3. Completion (500ms)
- WebSocket progress broadcasts
- Node version updates
**Node Updates**:
- Periodic heartbeat simulation
- Random metric updates (latency, heap)
- Automatic status management
**Cluster Changes**:
- Automatic primary selection
- Node discovery events
- WebSocket broadcasts
## API Compatibility
The mock gateway is fully compatible with:
- Frontend applications expecting the real gateway API
- Client libraries using the gateway
- Testing frameworks
- CI/CD pipelines
All responses match the real gateway's format and structure.
## Testing Use Cases
### Frontend Development
```bash
# Run mock gateway for UI development
./mock-gateway -port 3001 -mock-nodes 5 -log-level info
```
### Load Testing
```bash
# Simulate large cluster
./mock-gateway -mock-nodes 50 -heartbeat-rate 1
```
### Integration Testing
```bash
# Headless mode for automated tests
./mock-gateway -log-level error -enable-ws=false
```
### Demonstrations
```bash
# Clean output for demos
./mock-gateway -log-level warn -mock-nodes 8
```
## Implementation Details
### Mock Discovery (`internal/mock/discovery.go`)
**Features**:
- In-memory node storage
- Configurable node count
- Automatic heartbeat simulation
- Primary node management
- Callback system for updates
- Thread-safe operations
**Key Functions**:
- `NewMockNodeDiscovery()` - Creates discovery with N nodes
- `Start()` - Begins heartbeat simulation
- `GetNodes()` - Returns all nodes
- `SetPrimaryNode()` - Sets specific primary
- `SelectRandomPrimaryNode()` - Random selection
- `UpdateNodeVersion()` - Updates node version (for rollouts)
### Mock Server (`internal/mock/server.go`)
**Features**:
- All REST endpoints implemented
- CORS middleware
- JSON middleware
- Logging middleware
- In-memory firmware storage
- Rollout simulation
**Key Functions**:
- `NewMockHTTPServer()` - Creates server
- `setupRoutes()` - Configures all endpoints
- `simulateRollout()` - Simulates rollout process
- All endpoint handlers matching real gateway
### Mock WebSocket (`internal/mock/websocket.go`)
**Features**:
- WebSocket connection management
- Periodic broadcasts
- Event-driven broadcasts
- Client heartbeat (ping/pong)
- Graceful shutdown
**Key Functions**:
- `NewMockWebSocketServer()` - Creates WS server
- `HandleWebSocket()` - Handles connections
- `BroadcastClusterUpdate()` - Cluster updates
- `BroadcastFirmwareUploadStatus()` - Upload status
- `BroadcastRolloutProgress()` - Rollout progress
### Mock Data Generators (`internal/mock/data.go`)
**Functions**:
- `GenerateMockClusterMembers()` - Cluster member data
- `GenerateMockTaskStatus()` - Task scheduler status
- `GenerateMockSystemStatus()` - System status
- `GenerateMockCapabilities()` - API endpoints
- `GenerateMockFirmwareList()` - Firmware registry
- `GenerateMockFirmwareBinary()` - Firmware binary
- `GenerateMockProxyResponse()` - Proxy call responses
- `GenerateMockMonitoringResources()` - Comprehensive resource monitoring data
**Monitoring Data Includes**:
- **CPU Metrics**: Frequency, usage percentage, temperature
- **Memory Metrics**: Total, free, used bytes, usage percentage
- **Network Metrics**: Bytes sent/received, packets sent/received, RSSI, signal quality
- **Flash Metrics**: Total, used, free bytes, usage percentage
- **Summary Statistics**: Aggregate averages across all nodes
## Comparison: Real vs Mock Gateway
| Feature | Real Gateway | Mock Gateway |
|---------|-------------|--------------|
| REST API | ✅ All endpoints | ✅ All endpoints |
| WebSocket | ✅ Real-time | ✅ Real-time |
| Type Safety | ✅ Shared types | ✅ Shared types |
| UDP Discovery | ✅ Real UDP | ❌ Simulated |
| SPORE Node Calls | ✅ Real HTTP | ❌ Mocked |
| Registry Service | ✅ External | ❌ In-memory |
| Firmware Storage | ✅ Disk/Registry | ❌ Memory |
| Node Count | Dynamic (real) | Configurable |
| Setup Required | Hardware + Network | None |
| Dependencies | SPORE nodes + Registry | None |
## Advantages of Mock Gateway
1. **No Hardware Required**: Runs without any SPORE nodes
2. **No Network Setup**: No UDP discovery configuration needed
3. **Fast Iteration**: Instant startup and teardown
4. **Predictable**: Consistent, reproducible behavior
5. **Scalable**: Easily simulate hundreds of nodes
6. **Safe**: No risk to actual hardware
7. **Portable**: Runs anywhere Go runs
8. **Complete**: All features implemented
## When to Use Which
**Use Real Gateway**:
- Production deployment
- Real hardware testing
- Actual firmware updates
- Network topology testing
- Performance benchmarking
**Use Mock Gateway**:
- Frontend development
- API client development
- Automated testing
- Demonstrations
- CI/CD pipelines
- Local development
- Load testing
## Maintenance
The mock gateway is designed to stay in sync with the real gateway:
1. **Types**: All types are shared, so changes propagate automatically
2. **Endpoints**: New endpoints should be added to both implementations
3. **Responses**: Mock responses should mirror real responses
4. **Behaviors**: Simulate realistic timing and error rates
## Future Enhancements
Potential improvements:
- [ ] Configuration file support (currently CLI only)
- [ ] Persistent storage option
- [ ] Configurable error injection
- [ ] Network latency simulation
- [ ] Node failure scenarios
- [ ] Custom firmware metadata
- [ ] API recording/playback
- [ ] Performance metrics
## Contributing
When adding new endpoints to the real gateway:
1. Add corresponding mock implementation in `internal/mock/server.go`
2. Add mock data generators in `internal/mock/data.go` if needed
3. Ensure type reuse from shared packages
4. Update this documentation
5. Test both real and mock implementations
## License
Same as main SPORE Gateway project.

174
docs/QUICKSTART_MOCK.md Normal file
View File

@@ -0,0 +1,174 @@
# Mock Gateway Quick Start Guide
Get started with the SPORE Mock Gateway in under 1 minute!
## TL;DR
```bash
# Build and run
go build -o mock-gateway cmd/mock-gateway/main.go
./mock-gateway
```
That's it! The mock gateway is now running on http://localhost:3001
## What You Get
**5 mock SPORE nodes** ready to use
**All REST API endpoints** working
**WebSocket support** for real-time updates
**Mock firmware registry** with sample firmware
**Simulated rollouts** with progress tracking
## Test It
### 1. Check Health
```bash
curl http://localhost:3001/api/health | jq
```
### 2. List Nodes
```bash
curl http://localhost:3001/api/discovery/nodes | jq
```
### 3. Get Cluster Members
```bash
curl http://localhost:3001/api/cluster/members | jq
```
### 4. List Firmware
```bash
curl http://localhost:3001/api/registry/firmware | jq
```
### 5. Connect WebSocket
```javascript
const ws = new WebSocket('ws://localhost:3001/ws');
ws.onmessage = (event) => console.log(JSON.parse(event.data));
```
## Customize It
```bash
# More nodes
./mock-gateway -mock-nodes 10
# Different port
./mock-gateway -port 8080
# Debug logging
./mock-gateway -log-level debug
# Faster updates
./mock-gateway -heartbeat-rate 2
# All together
./mock-gateway -port 8080 -mock-nodes 10 -log-level debug -heartbeat-rate 2
```
## Use the Script
```bash
# Easy run with default settings
./scripts/run-mock-gateway.sh
# With custom options
./scripts/run-mock-gateway.sh -p 8080 -n 10 -l debug
# See all options
./scripts/run-mock-gateway.sh --help
```
## Connect Your Frontend
Point your frontend to `http://localhost:3001` instead of the real gateway. All endpoints work the same way!
```javascript
// React example
const API_BASE = 'http://localhost:3001';
const WS_URL = 'ws://localhost:3001/ws';
// Fetch nodes
const nodes = await fetch(`${API_BASE}/api/discovery/nodes`).then(r => r.json());
// WebSocket connection
const ws = new WebSocket(WS_URL);
```
## What's Mocked?
✅ Node discovery (5 default nodes: 192.168.1.100-104)
✅ Cluster management
✅ Firmware uploads (90% success rate, 2s delay)
✅ Rollouts (simulated progress: labels → upload → complete)
✅ Task status
✅ System status
✅ Registry operations
✅ WebSocket broadcasts
## Example Workflow
```bash
# 1. Start mock gateway
./mock-gateway
# 2. In another terminal, test the API
curl http://localhost:3001/api/discovery/nodes | jq '.nodes[0]'
# 3. Start a rollout
curl -X POST http://localhost:3001/api/rollout \
-H "Content-Type: application/json" \
-d '{
"firmware": {"name": "spore-firmware", "version": "1.1.0"},
"nodes": [
{"ip": "192.168.1.100", "version": "1.0.0", "labels": {"env": "mock"}},
{"ip": "192.168.1.101", "version": "1.0.0", "labels": {"env": "mock"}}
]
}' | jq
# 4. Watch WebSocket for progress (in browser console)
const ws = new WebSocket('ws://localhost:3001/ws');
ws.onmessage = e => console.log(JSON.parse(e.data));
```
## Next Steps
- 📖 Read [cmd/mock-gateway/README.md](cmd/mock-gateway/README.md) for detailed documentation
- 📖 Read [MOCK_GATEWAY.md](MOCK_GATEWAY.md) for implementation details
- 🔧 Customize the mock data in `internal/mock/data.go`
- 🚀 Use it for frontend development, testing, or demos
## Troubleshooting
**Port already in use?**
```bash
./mock-gateway -port 8080
```
**Need more/fewer nodes?**
```bash
./mock-gateway -mock-nodes 3
```
**Want to see what's happening?**
```bash
./mock-gateway -log-level debug
```
**Build fails?**
```bash
# Make sure you have Go 1.21 or later
go version
# Update dependencies
go mod download
```
## Questions?
- See [MOCK_GATEWAY.md](MOCK_GATEWAY.md) for complete documentation
- See [cmd/mock-gateway/README.md](cmd/mock-gateway/README.md) for usage details
- Check the real gateway's documentation to understand the API
Happy mocking! 🎭

198
docs/monitoring-example.md Normal file
View File

@@ -0,0 +1,198 @@
# Monitoring Resources Endpoint
## Overview
The `/api/monitoring/resources` endpoint provides comprehensive real-time resource monitoring for all nodes in the cluster.
## Endpoint
```
GET /api/monitoring/resources
```
## Response Format
```json
{
"timestamp": "2025-10-24T10:30:45Z",
"nodes": [
{
"timestamp": 1729763445,
"node_ip": "192.168.1.100",
"hostname": "spore-node-1",
"cpu": {
"frequency_mhz": 160,
"usage_percent": 42.5,
"temperature_c": 58.3
},
"memory": {
"total_bytes": 98304,
"free_bytes": 45632,
"used_bytes": 52672,
"usage_percent": 53.6
},
"network": {
"bytes_sent": 3245678,
"bytes_received": 5678901,
"packets_sent": 32456,
"packets_received": 56789,
"rssi_dbm": -65,
"signal_quality_percent": 75.5
},
"flash": {
"total_bytes": 4194304,
"used_bytes": 2097152,
"free_bytes": 2097152,
"usage_percent": 50.0
},
"labels": {
"version": "1.0.0",
"stable": "true",
"env": "production",
"zone": "zone-1",
"type": "spore-node"
}
}
],
"summary": {
"total_nodes": 5,
"avg_cpu_usage_percent": 38.7,
"avg_memory_usage_percent": 51.2,
"avg_flash_usage_percent": 52.8,
"total_bytes_sent": 16228390,
"total_bytes_received": 28394505
}
}
```
## Data Fields
### CPU Metrics
- **frequency_mhz**: Current CPU frequency in MHz (80-240 MHz typical for ESP32)
- **usage_percent**: CPU utilization percentage (0-100%)
- **temperature_c**: CPU temperature in Celsius (45-65°C typical)
### Memory Metrics
- **total_bytes**: Total RAM available (64-128 KB typical)
- **free_bytes**: Free RAM available
- **used_bytes**: Used RAM
- **usage_percent**: Memory utilization percentage
### Network Metrics
- **bytes_sent**: Total bytes transmitted since boot
- **bytes_received**: Total bytes received since boot
- **packets_sent**: Total packets transmitted
- **packets_received**: Total packets received
- **rssi_dbm**: WiFi signal strength in dBm (-30 to -90 typical)
- **signal_quality_percent**: WiFi signal quality (0-100%)
### Flash Metrics
- **total_bytes**: Total flash storage (typically 4MB)
- **used_bytes**: Used flash storage
- **free_bytes**: Free flash storage
- **usage_percent**: Flash utilization percentage
### Node Labels
Each node includes labels that match firmware versions:
- **version**: Current firmware version (e.g., "1.0.0", "1.1.0", "1.2.0")
- **stable**: Whether this is a stable release ("true" or "false")
- **env**: Environment (e.g., "production", "beta")
- **zone**: Deployment zone (e.g., "zone-1", "zone-2", "zone-3")
- **type**: Node type (e.g., "spore-node")
### Summary Statistics
Aggregate metrics across all nodes:
- **total_nodes**: Total number of nodes monitored
- **avg_cpu_usage_percent**: Average CPU usage across all nodes
- **avg_memory_usage_percent**: Average memory usage across all nodes
- **avg_flash_usage_percent**: Average flash usage across all nodes
- **total_bytes_sent**: Combined network traffic sent
- **total_bytes_received**: Combined network traffic received
## Firmware Version Matching
Node labels are automatically synchronized with the firmware available in the registry:
| Version | Registry Status | Node Distribution | Environment |
|---------|----------------|-------------------|-------------|
| 1.0.0 | Stable | 40% of nodes | production |
| 1.1.0 | Stable | 40% of nodes | production |
| 1.2.0 | Beta | 20% of nodes | beta |
This ensures that monitoring data accurately reflects which firmware versions are deployed across the cluster.
## Use Cases
### 1. Real-time Dashboard
Display live resource usage for all nodes in a monitoring dashboard.
### 2. Alerting
Set up alerts based on thresholds:
- CPU usage > 80%
- Memory usage > 90%
- Flash usage > 95%
- WiFi signal quality < 30%
### 3. Capacity Planning
Track resource trends to plan firmware optimizations or hardware upgrades.
### 4. Firmware Rollout Monitoring
Monitor resource usage before, during, and after firmware rollouts to detect issues.
### 5. Network Health
Track WiFi signal quality and network traffic to identify connectivity issues.
## Example Usage
### cURL
```bash
curl http://localhost:3001/api/monitoring/resources
```
### JavaScript (fetch)
```javascript
const response = await fetch('http://localhost:3001/api/monitoring/resources');
const data = await response.json();
console.log(`Monitoring ${data.summary.total_nodes} nodes`);
console.log(`Average CPU: ${data.summary.avg_cpu_usage_percent.toFixed(1)}%`);
console.log(`Average Memory: ${data.summary.avg_memory_usage_percent.toFixed(1)}%`);
data.nodes.forEach(node => {
console.log(`${node.hostname} (${node.labels.version}): CPU ${node.cpu.usage_percent.toFixed(1)}%`);
});
```
### Python
```python
import requests
response = requests.get('http://localhost:3001/api/monitoring/resources')
data = response.json()
print(f"Monitoring {data['summary']['total_nodes']} nodes")
print(f"Average CPU: {data['summary']['avg_cpu_usage_percent']:.1f}%")
print(f"Average Memory: {data['summary']['avg_memory_usage_percent']:.1f}%")
for node in data['nodes']:
print(f"{node['hostname']} ({node['labels']['version']}): "
f"CPU {node['cpu']['usage_percent']:.1f}%")
```
## Mock Gateway Behavior
The mock gateway generates realistic monitoring data with:
- **Dynamic values**: CPU, memory, and network metrics vary on each request
- **Realistic ranges**: Values stay within typical ESP32 hardware limits
- **Signal quality**: WiFi RSSI converted to quality percentage
- **Consistent labels**: Node labels always match firmware registry versions
- **Aggregate summaries**: Automatic calculation of cluster-wide statistics
## Integration with WebSocket
For real-time updates, consider combining this endpoint with the WebSocket connection at `/ws` which broadcasts:
- Node status changes
- Firmware update progress
- Cluster membership changes
The monitoring endpoint provides detailed point-in-time snapshots, while WebSocket provides real-time event streams.