235 lines
3.9 KiB
Markdown
235 lines
3.9 KiB
Markdown
# SPORE Nomad Deployment
|
|
|
|
This directory contains Nomad job specifications for deploying the SPORE stack on HashiCorp Nomad.
|
|
|
|
## Prerequisites
|
|
|
|
1. **Install Nomad**: https://www.nomadproject.io/downloads
|
|
2. **Docker**: Nomad uses Docker to run containers
|
|
|
|
Note: Consul is not required. This setup uses Nomad's built-in service discovery.
|
|
|
|
## Quick Start
|
|
|
|
### 1. Start Nomad
|
|
|
|
```bash
|
|
make nomad-start
|
|
```
|
|
|
|
This starts:
|
|
- Nomad at http://localhost:4646
|
|
|
|
### 2. Build Docker Images
|
|
|
|
```bash
|
|
# Build all images
|
|
cd ../spore-gateway && make docker-build
|
|
cd ../spore-ledlab && make docker-build
|
|
cd ../spore-registry && make docker-build
|
|
cd ../spore-ui && make docker-build
|
|
```
|
|
|
|
### 3. Deploy the Job
|
|
|
|
```bash
|
|
make nomad-job-run
|
|
```
|
|
|
|
Or manually:
|
|
|
|
```bash
|
|
nomad job run spore.hcl
|
|
```
|
|
|
|
### 4. Check Status
|
|
|
|
```bash
|
|
# Check job status
|
|
make nomad-status
|
|
|
|
# Or use Nomad UI
|
|
make nomad-ui
|
|
```
|
|
|
|
## Job Specification
|
|
|
|
The `spore.hcl` file defines 5 service groups:
|
|
|
|
1. **mqtt** - MQTT broker (Mosquitto)
|
|
2. **gateway** - Node discovery and WebSocket gateway
|
|
3. **registry** - Firmware registry service
|
|
4. **ledlab** - LED animation studio
|
|
5. **ui** - Web UI for cluster management
|
|
|
|
### Networking
|
|
|
|
- **MQTT, Gateway, LEDLab**: Use host networking for UDP/WebSocket
|
|
- **Registry, UI**: Use bridge networking with mapped ports
|
|
|
|
### Resources
|
|
|
|
Each service has appropriate CPU and memory allocations:
|
|
- MQTT: 100 CPU, 128 MB RAM
|
|
- Gateway: 200 CPU, 256 MB RAM
|
|
- Registry: 200 CPU, 256 MB RAM
|
|
- LEDLab: 300 CPU, 512 MB RAM
|
|
- UI: 100 CPU, 256 MB RAM
|
|
|
|
## Management
|
|
|
|
### View Logs
|
|
|
|
```bash
|
|
make nomad-logs
|
|
```
|
|
|
|
Or for specific task:
|
|
|
|
```bash
|
|
nomad alloc logs <alloc-id> <task-name>
|
|
```
|
|
|
|
### Stop/Start Job
|
|
|
|
```bash
|
|
# Stop
|
|
make nomad-job-stop
|
|
|
|
# Start
|
|
make nomad-job-run
|
|
```
|
|
|
|
### Scale Services
|
|
|
|
```bash
|
|
# Scale a specific group
|
|
nomad job scale -group mqtt -count 1 spore
|
|
```
|
|
|
|
### Update Job
|
|
|
|
Edit `spore.hcl` and run:
|
|
|
|
```bash
|
|
make nomad-job-run
|
|
```
|
|
|
|
Nomad will perform a rolling update if the job is running.
|
|
|
|
## Service Discovery
|
|
|
|
Nomad uses built-in service discovery. Services are discoverable via Nomad's internal DNS:
|
|
|
|
- `spore-gateway.service.nomad`
|
|
- `spore-mqtt.service.nomad`
|
|
- `spore-registry.service.nomad`
|
|
- `spore-ledlab.service.nomad`
|
|
- `spore-ui.service.nomad`
|
|
|
|
## Health Checks
|
|
|
|
All services include health checks that:
|
|
- Verify TCP connectivity
|
|
- Check HTTP endpoints for HTTP-based services
|
|
- Automatically restart unhealthy tasks
|
|
|
|
## Volumes
|
|
|
|
The registry service uses a host volume mount to persist firmware data:
|
|
|
|
```hcl
|
|
volume "registry" {
|
|
type = "host"
|
|
source = "registry"
|
|
read_only = false
|
|
}
|
|
```
|
|
|
|
Volume data is stored in `nomad-data/` directory.
|
|
|
|
## Stopping Everything
|
|
|
|
```bash
|
|
make nomad-stop
|
|
```
|
|
|
|
This stops both Nomad and Consul and cleans up data directories.
|
|
|
|
## Advanced Usage
|
|
|
|
### View Job Status
|
|
|
|
```bash
|
|
nomad job status spore
|
|
```
|
|
|
|
### Inspect Task
|
|
|
|
```bash
|
|
nomad job inspect spore
|
|
```
|
|
|
|
### Watch Job Events
|
|
|
|
```bash
|
|
nomad job status spore -verbose
|
|
```
|
|
|
|
### Connect to Task
|
|
|
|
```bash
|
|
nomad alloc exec <alloc-id> <task-name> /bin/sh
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### Services Not Starting
|
|
|
|
Check logs:
|
|
|
|
```bash
|
|
make nomad-logs
|
|
```
|
|
|
|
### Port Conflicts
|
|
|
|
If ports are in use, modify the job specification in `spore.hcl`:
|
|
|
|
```hcl
|
|
network {
|
|
mode = "bridge"
|
|
port "http" {
|
|
static = 3001 # Change this
|
|
to = 3001
|
|
}
|
|
}
|
|
```
|
|
|
|
### Images Not Found
|
|
|
|
Ensure Docker images are built and available:
|
|
|
|
```bash
|
|
docker images | grep wirelos
|
|
```
|
|
|
|
If using a registry, update the image URLs in `spore.hcl`.
|
|
|
|
|
|
## Production Deployment
|
|
|
|
For production environments:
|
|
|
|
1. **Configure ACLs**: Enable authentication
|
|
2. **Use Nomad Agents**: Deploy on multiple nodes
|
|
3. **TLS**: Enable TLS for all connections
|
|
4. **Resource Limits**: Adjust based on cluster capacity
|
|
5. **Persistence**: Use external storage for volumes
|
|
|
|
## Next Steps
|
|
|
|
- [Nomad Documentation](https://www.nomadproject.io/docs)
|
|
- [SPORE Documentation](../README.md)
|
|
|