feat: task manager endpoint, updated documentation
This commit is contained in:
437
docs/Development.md
Normal file
437
docs/Development.md
Normal file
@@ -0,0 +1,437 @@
|
||||
# Development & Deployment Guide
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### Required Tools
|
||||
|
||||
- **PlatformIO Core** or **PlatformIO IDE**
|
||||
- **ESP8266 development tools**
|
||||
- **`jq`** for JSON processing in scripts
|
||||
- **Git** for version control
|
||||
|
||||
### System Requirements
|
||||
|
||||
- **Operating System**: Linux, macOS, or Windows
|
||||
- **Python**: 3.7+ (for PlatformIO)
|
||||
- **Memory**: 4GB+ RAM recommended
|
||||
- **Storage**: 2GB+ free space for development environment
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
spore/
|
||||
├── src/ # Source code
|
||||
│ ├── main.cpp # Main application entry point
|
||||
│ ├── ApiServer.cpp # HTTP API server implementation
|
||||
│ ├── ClusterManager.cpp # Cluster management logic
|
||||
│ ├── NetworkManager.cpp # WiFi and network handling
|
||||
│ ├── TaskManager.cpp # Background task management
|
||||
│ └── NodeContext.cpp # Central context and events
|
||||
├── include/ # Header files
|
||||
├── lib/ # Library files
|
||||
├── docs/ # Documentation
|
||||
├── api/ # OpenAPI specification
|
||||
├── examples/ # Example code
|
||||
├── test/ # Test files
|
||||
├── platformio.ini # PlatformIO configuration
|
||||
└── ctl.sh # Build and deployment scripts
|
||||
```
|
||||
|
||||
## PlatformIO Configuration
|
||||
|
||||
### Framework and Board
|
||||
|
||||
The project uses PlatformIO with the following configuration:
|
||||
|
||||
```ini
|
||||
[env:esp01_1m]
|
||||
platform = platformio/espressif8266@^4.2.1
|
||||
board = esp01_1m
|
||||
framework = arduino
|
||||
upload_speed = 115200
|
||||
flash_mode = dout
|
||||
```
|
||||
|
||||
### Key Configuration Details
|
||||
|
||||
- **Framework**: Arduino
|
||||
- **Board**: ESP-01 with 1MB flash
|
||||
- **Upload Speed**: 115200 baud
|
||||
- **Flash Mode**: DOUT (required for ESP-01S)
|
||||
- **Build Type**: Release (optimized for production)
|
||||
|
||||
### Dependencies
|
||||
|
||||
The project requires the following libraries:
|
||||
|
||||
```ini
|
||||
lib_deps =
|
||||
esp32async/ESPAsyncWebServer@^3.8.0
|
||||
bblanchon/ArduinoJson@^7.4.2
|
||||
arkhipenko/TaskScheduler@^3.8.5
|
||||
ESP8266HTTPClient@1.2
|
||||
ESP8266WiFi@1.0
|
||||
```
|
||||
|
||||
## Building
|
||||
|
||||
### Basic Build Commands
|
||||
|
||||
Build the firmware for specific chip:
|
||||
|
||||
```bash
|
||||
# Build for ESP-01 1MB
|
||||
./ctl.sh build target esp01_1m
|
||||
|
||||
# Build for D1 Mini
|
||||
./ctl.sh build target d1_mini
|
||||
|
||||
# Build with verbose output
|
||||
pio run -v
|
||||
```
|
||||
|
||||
### Build Targets
|
||||
|
||||
Available build targets:
|
||||
|
||||
| Target | Description | Flash Size |
|
||||
|--------|-------------|------------|
|
||||
| `esp01_1m` | ESP-01 with 1MB flash | 1MB |
|
||||
| `d1_mini` | D1 Mini with 4MB flash | 4MB |
|
||||
|
||||
### Build Artifacts
|
||||
|
||||
After successful build:
|
||||
|
||||
- **Firmware**: `.pio/build/{target}/firmware.bin`
|
||||
- **ELF File**: `.pio/build/{target}/firmware.elf`
|
||||
- **Map File**: `.pio/build/{target}/firmware.map`
|
||||
|
||||
## Flashing
|
||||
|
||||
### Direct USB Flashing
|
||||
|
||||
Flash firmware to a connected device:
|
||||
|
||||
```bash
|
||||
# Flash ESP-01
|
||||
./ctl.sh flash target esp01_1m
|
||||
|
||||
# Flash D1 Mini
|
||||
./ctl.sh flash target d1_mini
|
||||
|
||||
# Manual flash command
|
||||
pio run --target upload
|
||||
```
|
||||
|
||||
### Flash Settings
|
||||
|
||||
- **Upload Speed**: 115200 baud (optimal for ESP-01)
|
||||
- **Flash Mode**: DOUT (required for ESP-01S)
|
||||
- **Reset Method**: Hardware reset or manual reset
|
||||
|
||||
### Troubleshooting Flashing
|
||||
|
||||
Common flashing issues:
|
||||
|
||||
1. **Connection Failed**: Check USB cable and drivers
|
||||
2. **Wrong Upload Speed**: Try lower speeds (9600, 57600)
|
||||
3. **Flash Mode Error**: Ensure DOUT mode for ESP-01S
|
||||
4. **Permission Denied**: Run with sudo or add user to dialout group
|
||||
|
||||
## Over-The-Air Updates
|
||||
|
||||
### Single Node Update
|
||||
|
||||
Update a specific node:
|
||||
|
||||
```bash
|
||||
# Update specific node
|
||||
./ctl.sh ota update 192.168.1.100 esp01_1m
|
||||
|
||||
# Update with custom firmware
|
||||
./ctl.sh ota update 192.168.1.100 esp01_1m custom_firmware.bin
|
||||
```
|
||||
|
||||
### Cluster-Wide Updates
|
||||
|
||||
Update all nodes in the cluster:
|
||||
|
||||
```bash
|
||||
# Update all nodes
|
||||
./ctl.sh ota all esp01_1m
|
||||
```
|
||||
|
||||
### OTA Process
|
||||
|
||||
1. **Firmware Upload**: Send firmware to target node
|
||||
2. **Verification**: Check firmware integrity
|
||||
3. **Installation**: Install new firmware
|
||||
4. **Restart**: Node restarts with new firmware
|
||||
5. **Verification**: Confirm successful update
|
||||
|
||||
### OTA Requirements
|
||||
|
||||
- **Flash Space**: Minimum 1MB for OTA updates
|
||||
- **Network**: Stable WiFi connection
|
||||
- **Power**: Stable power supply during update
|
||||
- **Memory**: Sufficient RAM for firmware processing
|
||||
|
||||
## Cluster Management
|
||||
|
||||
### View Cluster Status
|
||||
|
||||
```bash
|
||||
# View all cluster members
|
||||
./ctl.sh cluster members
|
||||
|
||||
# View specific node details
|
||||
./ctl.sh cluster members --node 192.168.1.100
|
||||
```
|
||||
|
||||
### Cluster Commands
|
||||
|
||||
Available cluster management commands:
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `members` | List all cluster members |
|
||||
| `status` | Show cluster health status |
|
||||
| `discover` | Force discovery process |
|
||||
| `health` | Check cluster member health |
|
||||
|
||||
### Cluster Monitoring
|
||||
|
||||
Monitor cluster health in real-time:
|
||||
|
||||
```bash
|
||||
# Watch cluster status
|
||||
watch -n 5 './ctl.sh cluster members'
|
||||
|
||||
# Monitor specific metrics
|
||||
./ctl.sh cluster members | jq '.members[] | {hostname, status, latency}'
|
||||
```
|
||||
|
||||
## Development Workflow
|
||||
|
||||
### Local Development
|
||||
|
||||
1. **Setup Environment**:
|
||||
```bash
|
||||
git clone <repository>
|
||||
cd spore
|
||||
pio run
|
||||
```
|
||||
|
||||
2. **Make Changes**:
|
||||
- Edit source files in `src/`
|
||||
- Modify headers in `include/`
|
||||
- Update configuration in `platformio.ini`
|
||||
|
||||
3. **Test Changes**:
|
||||
```bash
|
||||
pio run
|
||||
pio check
|
||||
```
|
||||
|
||||
### Testing
|
||||
|
||||
Run various tests:
|
||||
|
||||
```bash
|
||||
# Code quality check
|
||||
pio check
|
||||
|
||||
# Unit tests (if available)
|
||||
pio test
|
||||
|
||||
# Memory usage analysis
|
||||
pio run --target size
|
||||
```
|
||||
|
||||
### Debugging
|
||||
|
||||
Enable debug output:
|
||||
|
||||
```bash
|
||||
# Serial monitoring
|
||||
pio device monitor
|
||||
|
||||
# Build with debug symbols
|
||||
pio run --environment esp01_1m --build-flags -DDEBUG
|
||||
```
|
||||
|
||||
## Configuration Management
|
||||
|
||||
### Environment Setup
|
||||
|
||||
Create a `.env` file in your project root:
|
||||
|
||||
```bash
|
||||
# API node IP for cluster management
|
||||
export API_NODE=192.168.1.100
|
||||
```
|
||||
|
||||
### Configuration Files
|
||||
|
||||
Key configuration files:
|
||||
|
||||
- **`platformio.ini`**: Build and upload configuration
|
||||
- **`src/Config.cpp`**: Application configuration
|
||||
- **`.env`**: Environment variables
|
||||
- **`ctl.sh`**: Build and deployment scripts
|
||||
|
||||
### Configuration Options
|
||||
|
||||
Available configuration options:
|
||||
|
||||
| Option | Default | Description |
|
||||
|--------|---------|-------------|
|
||||
| `CLUSTER_PORT` | 4210 | UDP discovery port |
|
||||
| `DISCOVERY_INTERVAL` | 1000 | Discovery packet interval (ms) |
|
||||
| `HEALTH_CHECK_INTERVAL` | 1000 | Health check interval (ms) |
|
||||
| `API_SERVER_PORT` | 80 | HTTP API server port |
|
||||
|
||||
## Deployment Strategies
|
||||
|
||||
### Development Deployment
|
||||
|
||||
For development and testing:
|
||||
|
||||
1. **Build**: `pio run`
|
||||
2. **Flash**: `pio run --target upload`
|
||||
3. **Monitor**: `pio device monitor`
|
||||
|
||||
### Production Deployment
|
||||
|
||||
For production systems:
|
||||
|
||||
1. **Build Release**: `pio run --environment esp01_1m`
|
||||
2. **OTA Update**: `./ctl.sh ota update <ip> esp01_1m`
|
||||
3. **Verify**: Check node status via API
|
||||
|
||||
### Continuous Integration
|
||||
|
||||
Automated deployment pipeline:
|
||||
|
||||
```yaml
|
||||
# Example GitHub Actions workflow
|
||||
- name: Build Firmware
|
||||
run: pio run --environment esp01_1m
|
||||
|
||||
- name: Deploy to Test Cluster
|
||||
run: ./ctl.sh ota all esp01_1m --target test
|
||||
|
||||
- name: Deploy to Production
|
||||
run: ./ctl.sh ota all esp01_1m --target production
|
||||
```
|
||||
|
||||
## Monitoring and Debugging
|
||||
|
||||
### Serial Output
|
||||
|
||||
Enable serial monitoring:
|
||||
|
||||
```bash
|
||||
# Basic monitoring
|
||||
pio device monitor
|
||||
|
||||
# With specific baud rate
|
||||
pio device monitor --baud 115200
|
||||
|
||||
# Filter specific messages
|
||||
pio device monitor | grep "Cluster"
|
||||
```
|
||||
|
||||
### API Monitoring
|
||||
|
||||
Monitor system via HTTP API:
|
||||
|
||||
```bash
|
||||
# Check system status
|
||||
curl -s http://192.168.1.100/api/node/status | jq '.'
|
||||
|
||||
# Monitor tasks
|
||||
curl -s http://192.168.1.100/api/tasks/status | jq '.'
|
||||
|
||||
# Check cluster health
|
||||
curl -s http://192.168.1.100/api/cluster/members | jq '.'
|
||||
```
|
||||
|
||||
### Performance Monitoring
|
||||
|
||||
Track system performance:
|
||||
|
||||
```bash
|
||||
# Memory usage over time
|
||||
watch -n 5 'curl -s http://192.168.1.100/api/node/status | jq ".freeHeap"'
|
||||
|
||||
# Task execution status
|
||||
watch -n 10 'curl -s http://192.168.1.100/api/tasks/status | jq ".summary"'
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
1. **Discovery Failures**: Check UDP port 4210 is not blocked
|
||||
2. **WiFi Connection**: Verify SSID/password in Config.cpp
|
||||
3. **OTA Updates**: Ensure sufficient flash space (1MB minimum)
|
||||
4. **Cluster Split**: Check network connectivity between nodes
|
||||
|
||||
### Debug Commands
|
||||
|
||||
Useful debugging commands:
|
||||
|
||||
```bash
|
||||
# Check network connectivity
|
||||
ping 192.168.1.100
|
||||
|
||||
# Test UDP port
|
||||
nc -u 192.168.1.100 4210
|
||||
|
||||
# Check HTTP API
|
||||
curl -v http://192.168.1.100/api/node/status
|
||||
|
||||
# Monitor system resources
|
||||
./ctl.sh cluster members | jq '.members[] | {hostname, status, resources.freeHeap}'
|
||||
```
|
||||
|
||||
### Performance Issues
|
||||
|
||||
Common performance problems:
|
||||
|
||||
- **Memory Leaks**: Monitor free heap over time
|
||||
- **Network Congestion**: Check discovery intervals
|
||||
- **Task Overload**: Review task execution intervals
|
||||
- **WiFi Interference**: Check channel and signal strength
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Code Organization
|
||||
|
||||
1. **Modular Design**: Keep components loosely coupled
|
||||
2. **Clear Interfaces**: Define clear APIs between components
|
||||
3. **Error Handling**: Implement proper error handling and logging
|
||||
4. **Resource Management**: Efficient memory and resource usage
|
||||
|
||||
### Testing Strategy
|
||||
|
||||
1. **Unit Tests**: Test individual components
|
||||
2. **Integration Tests**: Test component interactions
|
||||
3. **System Tests**: Test complete system functionality
|
||||
4. **Performance Tests**: Monitor resource usage and performance
|
||||
|
||||
### Deployment Strategy
|
||||
|
||||
1. **Staged Rollout**: Deploy to test cluster first
|
||||
2. **Rollback Plan**: Maintain ability to rollback updates
|
||||
3. **Monitoring**: Monitor system health during deployment
|
||||
4. **Documentation**: Keep deployment procedures updated
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- **[Architecture Guide](./Architecture.md)** - System architecture overview
|
||||
- **[Task Management](./TaskManagement.md)** - Background task system
|
||||
- **[API Reference](./API.md)** - REST API documentation
|
||||
- **[OpenAPI Specification](../api/)** - Machine-readable API specification
|
||||
Reference in New Issue
Block a user