spore/README.md

# SPORE

> **S**Procket **OR**chestration **E**ngine

SPORE is a cluster engine for ESP8266 microcontrollers that provides automatic node discovery, health monitoring, and over-the-air updates in a distributed network environment.

## Table of Contents

- [Features](#features)
- [Supported Hardware](#supported-hardware)
- [Architecture](#architecture)
- [Cluster Broadcast](#cluster-broadcast)
- [Streaming API](#streaming-api)
- [API Reference](#api-reference)
- [Configuration](#configuration)
- [Development](#development)
- [Current Limitations](#current-limitations)
- [Troubleshooting](#troubleshooting)
- [Documentation](#documentation)
- [Contributing](#contributing)
- [License](#license)
- [Acknowledgments](#acknowledgments)

## Features

- **WiFi Management**: Automatic WiFi STA/AP configuration with MAC-based hostname generation
- **Auto Discovery**: UDP-based node discovery with automatic cluster membership
- **Service Registry**: Dynamic API endpoint discovery and registration
- **Health Monitoring**: Real-time node status tracking with resource monitoring
- **Event System**: Local and cluster-wide event publishing/subscription
- **Cluster Broadcast**: Centralized UDP broadcast of events (CLUSTER_EVENT)
- **Streaming API**: WebSocket bridge for real-time event send/receive
- **Over-The-Air Updates**: Seamless firmware updates across the cluster
- **REST API**: HTTP-based cluster management and monitoring
- **Capability Discovery**: Automatic API endpoint and service capability detection

## Supported Hardware

- **ESP-01 / ESP-01S** (1MB Flash)
- **Wemos D1** (4MB Flash)
- Other ESP8266 boards with 1MB+ flash

## Architecture

SPORE uses a modular architecture with automatic node discovery, health monitoring, and distributed task management.

**Core Components:**
- **Spore Framework**: Main framework class that orchestrates all components
- **Network Manager**: WiFi connection handling and hostname configuration
- **Cluster Manager**: Node discovery, member list management, and health monitoring
- **API Server**: HTTP API server with dynamic endpoint registration
- **Task Scheduler**: Cooperative multitasking system for background operations
- **Node Context**: Central context providing event system and shared resources

**Key Features:**
- **Auto Discovery**: UDP-based node detection on port 4210
- **Health Monitoring**: Continuous status checking via HTTP API
- **Task Scheduling**: Background tasks at configurable intervals
- **Event System**: Local and cluster-wide event publishing/subscription
- **Status Tracking**: Automatic node categorization (ACTIVE, INACTIVE, DEAD)
- **Resource Monitoring**: Memory, CPU, flash, and API endpoint tracking
- **WiFi Fallback**: Automatic access point creation if connection fails

📖 **Detailed Architecture:** See [`docs/Architecture.md`](./docs/Architecture.md) for comprehensive system design and implementation details.

## Quick Start

The Spore framework provides a simple, unified interface for all core functionality:

```cpp
#include <Arduino.h>
#include "Spore.h"

// Create Spore instance with custom labels
Spore spore({
    {"app", "my_app"},
    {"role", "controller"}
});

void setup() {
    spore.setup();
    spore.begin();
}

void loop() {
    spore.loop();
}
```

**Adding Custom Services:**
```cpp
void setup() {
    spore.setup();
    
    // Create and register custom services
    RelayService* relayService = new RelayService(spore.getContext(), spore.getTaskManager(), 2);
    spore.registerService(relayService);
    
    // Or using smart pointers
    auto sensorService = std::make_shared<SensorService>(spore.getContext(), spore.getTaskManager());
    spore.registerService(sensorService);
    
    // Start the API server and complete initialization
    spore.begin();
}
```

**Examples:** See [`examples/base/`](./examples/base/) for basic usage and [`examples/relay/`](./examples/relay/) for custom service integration.

## Cluster Broadcast

Broadcast an event to all peers using the centralized core broadcaster. Services never touch UDP directly; instead they fire a local event that the core transmits as a `CLUSTER_EVENT`.

Usage:

```cpp
// 1) Apply locally via the same event your service already handles
JsonDocument payload;
payload["pattern"] = "rainbow_cycle";
payload["brightness"] = 100;
String payloadStr; serializeJson(payload, payloadStr);
ctx.fire("api/neopattern", &payloadStr);

// 2) Broadcast to peers via the core
JsonDocument envelope;
envelope["event"] = "api/neopattern";
envelope["data"] = payloadStr; // JSON string
String eventJson; serializeJson(envelope, eventJson);
ctx.fire("cluster/broadcast", &eventJson);
```

Notes:
- The core sends subnet-directed broadcasts (e.g., 192.168.1.255) for reliability.
- Peers receive `CLUSTER_EVENT` and forward to local subscribers with `ctx.fire(event, data)`.
- `data` can be a JSON string or nested JSON; receivers handle both.

📖 See the dedicated guide: [`docs/ClusterBroadcast.md`](./docs/ClusterBroadcast.md)

## Streaming API

Real-time event bridge available at `/ws` using WebSocket.

- Send JSON `{ event, payload }` to dispatch events via `ctx.fire`.
- Receive all local events as `{ event, payload }`.

Examples:
```json
{ "event": "api/neopattern/color", "payload": { "color": "#FF0000", "brightness": 128 } }
```
```json
{ "event": "cluster/broadcast", "payload": { "event": "api/neopattern/color", "data": { "color": "#00FF00" } } }
```

📖 See the dedicated guide: [`docs/StreamingAPI.md`](./docs/StreamingAPI.md)

## API Reference

The system provides a comprehensive RESTful API for monitoring and controlling the embedded device. All endpoints return JSON responses and support standard HTTP status codes.

**Core Endpoints:**

| Endpoint | Method | Description |
|----------|--------|-------------|
| `/api/node/status` | GET | System resources and API endpoint registry |
| `/api/node/endpoints` | GET | API endpoints and parameters |
| `/api/cluster/members` | GET | Cluster membership and health status |
| `/api/node/update` | POST | OTA firmware updates |
| `/api/node/restart` | POST | System restart |
| `/api/tasks/status` | GET | Task management and monitoring |
| `/api/tasks/control` | POST | Task control operations |
| `/api/network/status` | GET | WiFi and network status information |
| `/api/network/wifi/scan` | GET/POST | WiFi network scanning and discovery |
| `/api/network/wifi/config` | POST | WiFi configuration management |

**Response Format:** All endpoints return JSON with standardized error handling and HTTP status codes.

📖 **Complete API Documentation:** See [`docs/API.md`](./docs/API.md) for detailed endpoint documentation, examples, and integration guides.

🔧 **OpenAPI Specification:** Machine-readable API spec available in [`api/`](./api/) folder for code generation and tooling integration.

## Configuration

The project uses PlatformIO with Arduino framework and supports multiple ESP8266 boards.

**Key Settings:**
- **Framework**: Arduino
- **Board**: ESP-01 with 1MB flash (default)
- **Upload Speed**: 115200 baud
- **Flash Mode**: DOUT (required for ESP-01S)

**Dependencies:**
- ESPAsyncWebServer, ArduinoJson, TaskScheduler
- ESP8266WiFi and HTTPClient libraries

**Environment Setup:** Create `.env` file for cluster configuration and API node settings.

## Development

**Quick Commands:**
```bash
# Build firmware
./ctl.sh build target esp01_1m

# Flash device
./ctl.sh flash target esp01_1m

# OTA update
./ctl.sh ota update 192.168.1.100 esp01_1m

# Cluster management
./ctl.sh cluster members
```

**Prerequisites:** PlatformIO Core, ESP8266 tools, `jq` for JSON processing

📖 **Complete Development Guide:** See [`docs/Development.md`](./docs/Development.md) for comprehensive build, deployment, and troubleshooting instructions.

## Current Limitations

- WiFi credentials are hardcoded in `Config.cpp` (should be configurable)
- Limited error handling for network failures
- No persistent storage for configuration
- Task monitoring and system health metrics
- Task execution history and performance analytics not yet implemented
- No authentication or security features implemented

## Troubleshooting

### Common Issues

1. **Discovery Failures**: Check UDP port 4210 is not blocked
2. **WiFi Connection**: Verify SSID/password in Config.cpp
3. **OTA Updates**: Ensure sufficient flash space (1MB minimum)
4. **Cluster Split**: Check network connectivity between nodes

### Debug Output

Enable serial monitoring to see cluster activity:

```bash
pio device monitor
```

## Documentation

📚 **Comprehensive documentation is available in the [`docs/`](./docs/) folder:**

- **[API Reference](./docs/API.md)** - Complete API documentation with examples
- **[Architecture Guide](./docs/Architecture.md)** - System design and implementation details
- **[Development Guide](./docs/Development.md)** - Build, deployment, and configuration
- **[Task Management Guide](./docs/TaskManagement.md)** - Background task management system
- **[OpenAPI Spec](./api/)** - Machine-readable API specification

## Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Test thoroughly on ESP8266 hardware
5. Submit a pull request

## License

[Add your license information here]

## Acknowledgments

- Built with [PlatformIO](https://platformio.org/)
- Uses [TaskScheduler](https://github.com/arkhipenko/TaskScheduler) for cooperative multitasking
- [ESPAsyncWebServer](https://github.com/me-no-dev/ESPAsyncWebServer) for HTTP API
- [ArduinoJson](https://arduinojson.org/) for JSON processing