feat: Compose and Nomad deployment
This commit is contained in:
234
nomad/README.md
Normal file
234
nomad/README.md
Normal file
@@ -0,0 +1,234 @@
|
||||
# SPORE Nomad Deployment
|
||||
|
||||
This directory contains Nomad job specifications for deploying the SPORE stack on HashiCorp Nomad.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. **Install Nomad**: https://www.nomadproject.io/downloads
|
||||
2. **Docker**: Nomad uses Docker to run containers
|
||||
|
||||
Note: Consul is not required. This setup uses Nomad's built-in service discovery.
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Start Nomad
|
||||
|
||||
```bash
|
||||
make nomad-start
|
||||
```
|
||||
|
||||
This starts:
|
||||
- Nomad at http://localhost:4646
|
||||
|
||||
### 2. Build Docker Images
|
||||
|
||||
```bash
|
||||
# Build all images
|
||||
cd ../spore-gateway && make docker-build
|
||||
cd ../spore-ledlab && make docker-build
|
||||
cd ../spore-registry && make docker-build
|
||||
cd ../spore-ui && make docker-build
|
||||
```
|
||||
|
||||
### 3. Deploy the Job
|
||||
|
||||
```bash
|
||||
make nomad-job-run
|
||||
```
|
||||
|
||||
Or manually:
|
||||
|
||||
```bash
|
||||
nomad job run spore.hcl
|
||||
```
|
||||
|
||||
### 4. Check Status
|
||||
|
||||
```bash
|
||||
# Check job status
|
||||
make nomad-status
|
||||
|
||||
# Or use Nomad UI
|
||||
make nomad-ui
|
||||
```
|
||||
|
||||
## Job Specification
|
||||
|
||||
The `spore.hcl` file defines 5 service groups:
|
||||
|
||||
1. **mqtt** - MQTT broker (Mosquitto)
|
||||
2. **gateway** - Node discovery and WebSocket gateway
|
||||
3. **registry** - Firmware registry service
|
||||
4. **ledlab** - LED animation studio
|
||||
5. **ui** - Web UI for cluster management
|
||||
|
||||
### Networking
|
||||
|
||||
- **MQTT, Gateway, LEDLab**: Use host networking for UDP/WebSocket
|
||||
- **Registry, UI**: Use bridge networking with mapped ports
|
||||
|
||||
### Resources
|
||||
|
||||
Each service has appropriate CPU and memory allocations:
|
||||
- MQTT: 100 CPU, 128 MB RAM
|
||||
- Gateway: 200 CPU, 256 MB RAM
|
||||
- Registry: 200 CPU, 256 MB RAM
|
||||
- LEDLab: 300 CPU, 512 MB RAM
|
||||
- UI: 100 CPU, 256 MB RAM
|
||||
|
||||
## Management
|
||||
|
||||
### View Logs
|
||||
|
||||
```bash
|
||||
make nomad-logs
|
||||
```
|
||||
|
||||
Or for specific task:
|
||||
|
||||
```bash
|
||||
nomad alloc logs <alloc-id> <task-name>
|
||||
```
|
||||
|
||||
### Stop/Start Job
|
||||
|
||||
```bash
|
||||
# Stop
|
||||
make nomad-job-stop
|
||||
|
||||
# Start
|
||||
make nomad-job-run
|
||||
```
|
||||
|
||||
### Scale Services
|
||||
|
||||
```bash
|
||||
# Scale a specific group
|
||||
nomad job scale -group mqtt -count 1 spore
|
||||
```
|
||||
|
||||
### Update Job
|
||||
|
||||
Edit `spore.hcl` and run:
|
||||
|
||||
```bash
|
||||
make nomad-job-run
|
||||
```
|
||||
|
||||
Nomad will perform a rolling update if the job is running.
|
||||
|
||||
## Service Discovery
|
||||
|
||||
Nomad uses built-in service discovery. Services are discoverable via Nomad's internal DNS:
|
||||
|
||||
- `spore-gateway.service.nomad`
|
||||
- `spore-mqtt.service.nomad`
|
||||
- `spore-registry.service.nomad`
|
||||
- `spore-ledlab.service.nomad`
|
||||
- `spore-ui.service.nomad`
|
||||
|
||||
## Health Checks
|
||||
|
||||
All services include health checks that:
|
||||
- Verify TCP connectivity
|
||||
- Check HTTP endpoints for HTTP-based services
|
||||
- Automatically restart unhealthy tasks
|
||||
|
||||
## Volumes
|
||||
|
||||
The registry service uses a host volume mount to persist firmware data:
|
||||
|
||||
```hcl
|
||||
volume "registry" {
|
||||
type = "host"
|
||||
source = "registry"
|
||||
read_only = false
|
||||
}
|
||||
```
|
||||
|
||||
Volume data is stored in `nomad-data/` directory.
|
||||
|
||||
## Stopping Everything
|
||||
|
||||
```bash
|
||||
make nomad-stop
|
||||
```
|
||||
|
||||
This stops both Nomad and Consul and cleans up data directories.
|
||||
|
||||
## Advanced Usage
|
||||
|
||||
### View Job Status
|
||||
|
||||
```bash
|
||||
nomad job status spore
|
||||
```
|
||||
|
||||
### Inspect Task
|
||||
|
||||
```bash
|
||||
nomad job inspect spore
|
||||
```
|
||||
|
||||
### Watch Job Events
|
||||
|
||||
```bash
|
||||
nomad job status spore -verbose
|
||||
```
|
||||
|
||||
### Connect to Task
|
||||
|
||||
```bash
|
||||
nomad alloc exec <alloc-id> <task-name> /bin/sh
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Services Not Starting
|
||||
|
||||
Check logs:
|
||||
|
||||
```bash
|
||||
make nomad-logs
|
||||
```
|
||||
|
||||
### Port Conflicts
|
||||
|
||||
If ports are in use, modify the job specification in `spore.hcl`:
|
||||
|
||||
```hcl
|
||||
network {
|
||||
mode = "bridge"
|
||||
port "http" {
|
||||
static = 3001 # Change this
|
||||
to = 3001
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Images Not Found
|
||||
|
||||
Ensure Docker images are built and available:
|
||||
|
||||
```bash
|
||||
docker images | grep wirelos
|
||||
```
|
||||
|
||||
If using a registry, update the image URLs in `spore.hcl`.
|
||||
|
||||
|
||||
## Production Deployment
|
||||
|
||||
For production environments:
|
||||
|
||||
1. **Configure ACLs**: Enable authentication
|
||||
2. **Use Nomad Agents**: Deploy on multiple nodes
|
||||
3. **TLS**: Enable TLS for all connections
|
||||
4. **Resource Limits**: Adjust based on cluster capacity
|
||||
5. **Persistence**: Use external storage for volumes
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Nomad Documentation](https://www.nomadproject.io/docs)
|
||||
- [SPORE Documentation](../README.md)
|
||||
|
||||
145
nomad/spore.hcl
Normal file
145
nomad/spore.hcl
Normal file
@@ -0,0 +1,145 @@
|
||||
job "spore" {
|
||||
datacenters = ["dc1"]
|
||||
type = "service"
|
||||
|
||||
group "mqtt" {
|
||||
count = 1
|
||||
|
||||
network {
|
||||
mode = "host"
|
||||
}
|
||||
|
||||
task "mqtt" {
|
||||
driver = "docker"
|
||||
|
||||
config {
|
||||
image = "eclipse-mosquitto:2.0"
|
||||
network_mode = "host"
|
||||
}
|
||||
|
||||
resources {
|
||||
cpu = 100
|
||||
memory = 128
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
group "gateway" {
|
||||
count = 1
|
||||
|
||||
network {
|
||||
mode = "host"
|
||||
}
|
||||
|
||||
task "gateway" {
|
||||
driver = "docker"
|
||||
|
||||
config {
|
||||
image = "wirelos/spore-gateway:latest"
|
||||
network_mode = "host"
|
||||
}
|
||||
|
||||
env {
|
||||
LOG_LEVEL = "info"
|
||||
MQTT_SERVER = "tcp://localhost:1883"
|
||||
MQTT_USER = ""
|
||||
MQTT_PASSWORD = ""
|
||||
}
|
||||
|
||||
resources {
|
||||
cpu = 200
|
||||
memory = 256
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
group "registry" {
|
||||
count = 1
|
||||
|
||||
network {
|
||||
mode = "host"
|
||||
}
|
||||
|
||||
task "registry" {
|
||||
driver = "docker"
|
||||
|
||||
config {
|
||||
image = "wirelos/spore-registry:latest"
|
||||
network_mode = "host"
|
||||
|
||||
volumes = [
|
||||
"/opt/nomad/spore/data/registry:/data/registry"
|
||||
]
|
||||
}
|
||||
|
||||
env {
|
||||
REGISTRY_PATH = "/data/registry"
|
||||
DB_PATH = "/data/registry/registry.db"
|
||||
}
|
||||
|
||||
resources {
|
||||
cpu = 200
|
||||
memory = 256
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
group "ledlab" {
|
||||
count = 1
|
||||
|
||||
network {
|
||||
mode = "host"
|
||||
}
|
||||
|
||||
task "ledlab" {
|
||||
driver = "docker"
|
||||
|
||||
config {
|
||||
image = "wirelos/spore-ledlab:latest"
|
||||
network_mode = "host"
|
||||
}
|
||||
|
||||
env {
|
||||
PORT = "8080"
|
||||
UDP_PORT = "4210"
|
||||
GATEWAY_URL = "http://localhost:3001"
|
||||
FILTER_APP_LABEL = "pixelstream"
|
||||
MATRIX_WIDTH = "16"
|
||||
MATRIX_HEIGHT = "16"
|
||||
FPS = "20"
|
||||
}
|
||||
|
||||
resources {
|
||||
cpu = 300
|
||||
memory = 512
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
group "ui" {
|
||||
count = 1
|
||||
|
||||
network {
|
||||
mode = "host"
|
||||
}
|
||||
|
||||
task "ui" {
|
||||
driver = "docker"
|
||||
|
||||
config {
|
||||
image = "wirelos/spore-ui:latest"
|
||||
network_mode = "host"
|
||||
}
|
||||
|
||||
env {
|
||||
PORT = "3000"
|
||||
}
|
||||
|
||||
resources {
|
||||
cpu = 100
|
||||
memory = 256
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user