feat: Compose and Nomad deployment
This commit is contained in:
383
README.md
Normal file
383
README.md
Normal file
@@ -0,0 +1,383 @@
|
||||
# SPORE Deployment
|
||||
|
||||
Complete deployment configurations for running the SPORE stack with Docker Compose or HashiCorp Nomad.
|
||||
|
||||
## Overview
|
||||
|
||||
This repository provides two deployment options:
|
||||
|
||||
1. **Docker Compose** - Simple deployment with docker-compose for development and production
|
||||
2. **Nomad** - Production-ready orchestration using HashiCorp Nomad
|
||||
|
||||
Both deployments include all SPORE services with proper networking, volumes, and service discovery.
|
||||
|
||||
## Services
|
||||
|
||||
- **mosquitto**: MQTT broker for message routing (port 1883, 9001)
|
||||
- **spore-gateway**: Node discovery and WebSocket gateway with MQTT integration (port 3001, UDP 4210)
|
||||
- **spore-registry**: Firmware registry service (port 8090)
|
||||
- **spore-ledlab**: LED animation studio (port 8080)
|
||||
- **spore-ui**: Web UI for cluster management (port 3000)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### For Docker Compose
|
||||
- Docker and Docker Compose installed
|
||||
- Available ports: 1883, 3000, 3001, 4210, 8080, 8090
|
||||
|
||||
### For Nomad
|
||||
- HashiCorp Nomad installed
|
||||
- Docker installed (Nomad uses Docker to run containers)
|
||||
- Available ports: 1883, 3000, 3001, 4210, 8080, 8090, 4646
|
||||
|
||||
## Deployment Options
|
||||
|
||||
### Option 1: Docker Compose (Recommended for development)
|
||||
|
||||
Simple, single-node deployment with docker-compose.
|
||||
|
||||
#### Quick Start
|
||||
|
||||
```bash
|
||||
cd spore-deployment
|
||||
|
||||
# Start all services
|
||||
docker compose up -d
|
||||
|
||||
# View logs
|
||||
docker compose logs -f
|
||||
|
||||
# Stop all services
|
||||
docker compose down
|
||||
```
|
||||
|
||||
#### Services Access
|
||||
|
||||
- **Web UI**: http://localhost:3000
|
||||
- **LED Lab**: http://localhost:8080
|
||||
- **Registry API**: http://localhost:8090
|
||||
- **Gateway API**: http://localhost:3001
|
||||
- **MQTT Broker**: tcp://localhost:1883 (WebSocket: ws://localhost:9001)
|
||||
|
||||
#### Data Storage
|
||||
|
||||
All data is persisted in local `./data/` directory:
|
||||
- Registry data: `./data/registry`
|
||||
- MQTT data: `./data/mqtt/`
|
||||
|
||||
#### Management
|
||||
|
||||
```bash
|
||||
# View logs for all services
|
||||
make logs
|
||||
|
||||
# View logs for specific service
|
||||
make gateway-logs
|
||||
make registry-logs
|
||||
make ledlab-logs
|
||||
make ui-logs
|
||||
make mqtt-logs
|
||||
|
||||
# Restart services
|
||||
make restart
|
||||
|
||||
# Clean (stop and remove all data)
|
||||
make clean
|
||||
```
|
||||
|
||||
### Option 2: Nomad (Recommended for production)
|
||||
|
||||
Distributed deployment with HashiCorp Nomad.
|
||||
|
||||
#### Quick Start
|
||||
|
||||
```bash
|
||||
cd spore-deployment
|
||||
|
||||
# Start Nomad in dev mode
|
||||
make nomad-start
|
||||
|
||||
# Build Docker images first
|
||||
cd ../spore-gateway && make docker-build
|
||||
cd ../spore-ledlab && make docker-build
|
||||
cd ../spore-registry && make docker-build
|
||||
cd ../spore-ui && make docker-build
|
||||
|
||||
# Deploy the job
|
||||
make nomad-job-run
|
||||
|
||||
# View status
|
||||
make nomad-status
|
||||
```
|
||||
|
||||
#### Services Access
|
||||
|
||||
Same ports as Docker Compose deployment.
|
||||
|
||||
#### Nomad UI
|
||||
|
||||
Access the Nomad UI at http://localhost:4646 to manage and monitor jobs.
|
||||
|
||||
#### Management
|
||||
|
||||
```bash
|
||||
# View Nomad UI
|
||||
make nomad-ui
|
||||
|
||||
# View job logs
|
||||
make nomad-logs
|
||||
|
||||
# Stop the job
|
||||
make nomad-job-stop
|
||||
|
||||
# Restart Nomad
|
||||
make nomad-stop
|
||||
make nomad-start
|
||||
```
|
||||
|
||||
#### Nomad Configuration
|
||||
|
||||
Nomad configuration is in `config/nomad/nomad.hcl`:
|
||||
- Dev mode with single agent
|
||||
- Docker driver enabled
|
||||
- Host networking for all services
|
||||
- UI enabled on port 4646
|
||||
|
||||
See [nomad/README.md](./nomad/README.md) for detailed Nomad documentation.
|
||||
|
||||
## Common Tasks
|
||||
|
||||
### Building Images
|
||||
|
||||
Both deployment methods require Docker images to be built first.
|
||||
|
||||
```bash
|
||||
# Using make in each project
|
||||
cd ../spore-gateway && make docker-build
|
||||
cd ../spore-ledlab && make docker-build
|
||||
cd ../spore-registry && make docker-build
|
||||
cd ../spore-ui && make docker-build
|
||||
|
||||
# Or build and push all at once from deployment directory
|
||||
make push
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
#### Docker Compose
|
||||
|
||||
Create a `.env` file to override defaults:
|
||||
|
||||
```env
|
||||
LOG_LEVEL=debug
|
||||
MATRIX_WIDTH=32
|
||||
MATRIX_HEIGHT=32
|
||||
FPS=30
|
||||
```
|
||||
|
||||
Then run:
|
||||
```bash
|
||||
docker compose --env-file .env up -d
|
||||
```
|
||||
|
||||
#### Nomad
|
||||
|
||||
Edit `nomad/spore.hcl` to modify environment variables or resource allocations.
|
||||
|
||||
### Data Management
|
||||
|
||||
#### Backup
|
||||
|
||||
```bash
|
||||
# Backup all data
|
||||
tar czf spore-backup.tar.gz ./data/
|
||||
|
||||
# Backup registry only
|
||||
tar czf registry-backup.tar.gz ./data/registry/
|
||||
```
|
||||
|
||||
#### Restore
|
||||
|
||||
```bash
|
||||
# Restore all data
|
||||
tar xzf spore-backup.tar.gz
|
||||
|
||||
# Restore registry only
|
||||
tar xzf registry-backup.tar.gz
|
||||
```
|
||||
|
||||
#### Reset Data
|
||||
|
||||
```bash
|
||||
# Docker Compose
|
||||
make clean
|
||||
|
||||
# Nomad
|
||||
rm -rf nomad-data/
|
||||
```
|
||||
|
||||
## Logs
|
||||
|
||||
### Docker Compose
|
||||
|
||||
```bash
|
||||
# All services
|
||||
docker compose logs -f
|
||||
|
||||
# Specific service
|
||||
docker compose logs -f gateway
|
||||
docker compose logs -f registry
|
||||
```
|
||||
|
||||
### Nomad
|
||||
|
||||
```bash
|
||||
# All job logs
|
||||
make nomad-logs
|
||||
|
||||
# Specific task
|
||||
nomad alloc logs <alloc-id> <task-name>
|
||||
|
||||
# Follow logs
|
||||
nomad alloc logs -f <alloc-id> <task-name>
|
||||
```
|
||||
|
||||
## MQTT Integration
|
||||
|
||||
Both deployments enable MQTT integration for the gateway service.
|
||||
|
||||
### Publishing to MQTT
|
||||
|
||||
```bash
|
||||
# Using mosquitto_pub
|
||||
mosquitto_pub -h localhost -t "spore/cluster/broadcast" -m '{"command":"update"}'
|
||||
|
||||
# Send to specific node
|
||||
mosquitto_pub -h localhost -t "spore/nodes/192.168.1.100/command" -m '{"action":"update"}'
|
||||
```
|
||||
|
||||
### Monitoring MQTT
|
||||
|
||||
```bash
|
||||
# Subscribe to all SPORE topics
|
||||
mosquitto_sub -h localhost -t "spore/#" -v
|
||||
|
||||
# Watch specific topic
|
||||
mosquitto_sub -h localhost -t "spore/cluster/status" -v
|
||||
```
|
||||
|
||||
### MQTT Authentication
|
||||
|
||||
Enable authentication by setting environment variables:
|
||||
|
||||
**Docker Compose:**
|
||||
```yaml
|
||||
gateway:
|
||||
environment:
|
||||
- MQTT_SERVER=tcp://localhost:1883
|
||||
- MQTT_USER=username
|
||||
- MQTT_PASSWORD=password
|
||||
```
|
||||
|
||||
**Nomad:**
|
||||
Edit `nomad/spore.hcl` to add MQTT_USER and MQTT_PASSWORD env vars.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Port Conflicts
|
||||
|
||||
If ports are already in use:
|
||||
|
||||
**Docker Compose:** Edit port mappings in `docker-compose.yml`
|
||||
|
||||
**Nomad:** Edit ports in `nomad/spore.hcl`
|
||||
|
||||
### Services Not Starting
|
||||
|
||||
**Docker Compose:**
|
||||
```bash
|
||||
# Check logs
|
||||
make logs
|
||||
|
||||
# Check container status
|
||||
docker compose ps
|
||||
```
|
||||
|
||||
**Nomad:**
|
||||
```bash
|
||||
# Check job status
|
||||
make nomad-status
|
||||
|
||||
# Check allocation logs
|
||||
nomad alloc logs <alloc-id> <task-name>
|
||||
|
||||
# Check allocation status
|
||||
nomad alloc status <alloc-id>
|
||||
```
|
||||
|
||||
### Network Issues
|
||||
|
||||
Both deployments use host networking for proper UDP broadcast support. If services can't communicate:
|
||||
|
||||
1. Check firewall settings
|
||||
2. Verify ports are not blocked
|
||||
3. Check service logs for connection errors
|
||||
|
||||
### Images Not Found
|
||||
|
||||
Ensure Docker images are built before deploying:
|
||||
|
||||
```bash
|
||||
# Build all images
|
||||
make push
|
||||
|
||||
# Or build individually
|
||||
cd ../spore-gateway && make docker-build
|
||||
cd ../spore-ledlab && make docker-build
|
||||
cd ../spore-registry && make docker-build
|
||||
cd ../spore-ui && make docker-build
|
||||
```
|
||||
|
||||
## Production Deployment
|
||||
|
||||
### Docker Compose
|
||||
|
||||
For production, consider:
|
||||
1. Using environment-specific configuration files
|
||||
2. Setting up proper logging aggregation
|
||||
3. Using Docker secrets for sensitive data
|
||||
4. Implementing health checks
|
||||
5. Setting resource limits
|
||||
|
||||
### Nomad
|
||||
|
||||
For production deployment:
|
||||
1. Use a multi-node Nomad cluster
|
||||
2. Enable ACLs for security
|
||||
3. Configure TLS for all connections
|
||||
4. Use external storage for volumes
|
||||
5. Set up proper monitoring and alerting
|
||||
6. Configure auto-scaling based on load
|
||||
|
||||
## Comparing Deployment Options
|
||||
|
||||
| Feature | Docker Compose | Nomad |
|
||||
|---------|---------------|-------|
|
||||
| Ease of setup | ⭐⭐⭐⭐⭐ | ⭐⭐⭐ |
|
||||
| Scalability | ⭐⭐ | ⭐⭐⭐⭐⭐ |
|
||||
| Multi-node | No | Yes |
|
||||
| Service discovery | Manual | Built-in |
|
||||
| Rolling updates | Manual | Automatic |
|
||||
| Resource management | Basic | Advanced |
|
||||
| Production-ready | ⭐⭐⭐ | ⭐⭐⭐⭐⭐ |
|
||||
|
||||
**Recommendation:**
|
||||
- Use **Docker Compose** for development, testing, or single-node deployments
|
||||
- Use **Nomad** for production, multi-node clusters, or when you need advanced orchestration features
|
||||
|
||||
## Additional Resources
|
||||
|
||||
- [SPORE Documentation](../spore/README.md)
|
||||
- [Nomad Deployment Guide](./nomad/README.md)
|
||||
- [Docker Compose Documentation](https://docs.docker.com/compose/)
|
||||
- [Nomad Documentation](https://www.nomadproject.io/docs)
|
||||
Reference in New Issue
Block a user