feat: Compose and Nomad deployment
This commit is contained in:
234
nomad/README.md
Normal file
234
nomad/README.md
Normal file
@@ -0,0 +1,234 @@
|
||||
# SPORE Nomad Deployment
|
||||
|
||||
This directory contains Nomad job specifications for deploying the SPORE stack on HashiCorp Nomad.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. **Install Nomad**: https://www.nomadproject.io/downloads
|
||||
2. **Docker**: Nomad uses Docker to run containers
|
||||
|
||||
Note: Consul is not required. This setup uses Nomad's built-in service discovery.
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Start Nomad
|
||||
|
||||
```bash
|
||||
make nomad-start
|
||||
```
|
||||
|
||||
This starts:
|
||||
- Nomad at http://localhost:4646
|
||||
|
||||
### 2. Build Docker Images
|
||||
|
||||
```bash
|
||||
# Build all images
|
||||
cd ../spore-gateway && make docker-build
|
||||
cd ../spore-ledlab && make docker-build
|
||||
cd ../spore-registry && make docker-build
|
||||
cd ../spore-ui && make docker-build
|
||||
```
|
||||
|
||||
### 3. Deploy the Job
|
||||
|
||||
```bash
|
||||
make nomad-job-run
|
||||
```
|
||||
|
||||
Or manually:
|
||||
|
||||
```bash
|
||||
nomad job run spore.hcl
|
||||
```
|
||||
|
||||
### 4. Check Status
|
||||
|
||||
```bash
|
||||
# Check job status
|
||||
make nomad-status
|
||||
|
||||
# Or use Nomad UI
|
||||
make nomad-ui
|
||||
```
|
||||
|
||||
## Job Specification
|
||||
|
||||
The `spore.hcl` file defines 5 service groups:
|
||||
|
||||
1. **mqtt** - MQTT broker (Mosquitto)
|
||||
2. **gateway** - Node discovery and WebSocket gateway
|
||||
3. **registry** - Firmware registry service
|
||||
4. **ledlab** - LED animation studio
|
||||
5. **ui** - Web UI for cluster management
|
||||
|
||||
### Networking
|
||||
|
||||
- **MQTT, Gateway, LEDLab**: Use host networking for UDP/WebSocket
|
||||
- **Registry, UI**: Use bridge networking with mapped ports
|
||||
|
||||
### Resources
|
||||
|
||||
Each service has appropriate CPU and memory allocations:
|
||||
- MQTT: 100 CPU, 128 MB RAM
|
||||
- Gateway: 200 CPU, 256 MB RAM
|
||||
- Registry: 200 CPU, 256 MB RAM
|
||||
- LEDLab: 300 CPU, 512 MB RAM
|
||||
- UI: 100 CPU, 256 MB RAM
|
||||
|
||||
## Management
|
||||
|
||||
### View Logs
|
||||
|
||||
```bash
|
||||
make nomad-logs
|
||||
```
|
||||
|
||||
Or for specific task:
|
||||
|
||||
```bash
|
||||
nomad alloc logs <alloc-id> <task-name>
|
||||
```
|
||||
|
||||
### Stop/Start Job
|
||||
|
||||
```bash
|
||||
# Stop
|
||||
make nomad-job-stop
|
||||
|
||||
# Start
|
||||
make nomad-job-run
|
||||
```
|
||||
|
||||
### Scale Services
|
||||
|
||||
```bash
|
||||
# Scale a specific group
|
||||
nomad job scale -group mqtt -count 1 spore
|
||||
```
|
||||
|
||||
### Update Job
|
||||
|
||||
Edit `spore.hcl` and run:
|
||||
|
||||
```bash
|
||||
make nomad-job-run
|
||||
```
|
||||
|
||||
Nomad will perform a rolling update if the job is running.
|
||||
|
||||
## Service Discovery
|
||||
|
||||
Nomad uses built-in service discovery. Services are discoverable via Nomad's internal DNS:
|
||||
|
||||
- `spore-gateway.service.nomad`
|
||||
- `spore-mqtt.service.nomad`
|
||||
- `spore-registry.service.nomad`
|
||||
- `spore-ledlab.service.nomad`
|
||||
- `spore-ui.service.nomad`
|
||||
|
||||
## Health Checks
|
||||
|
||||
All services include health checks that:
|
||||
- Verify TCP connectivity
|
||||
- Check HTTP endpoints for HTTP-based services
|
||||
- Automatically restart unhealthy tasks
|
||||
|
||||
## Volumes
|
||||
|
||||
The registry service uses a host volume mount to persist firmware data:
|
||||
|
||||
```hcl
|
||||
volume "registry" {
|
||||
type = "host"
|
||||
source = "registry"
|
||||
read_only = false
|
||||
}
|
||||
```
|
||||
|
||||
Volume data is stored in `nomad-data/` directory.
|
||||
|
||||
## Stopping Everything
|
||||
|
||||
```bash
|
||||
make nomad-stop
|
||||
```
|
||||
|
||||
This stops both Nomad and Consul and cleans up data directories.
|
||||
|
||||
## Advanced Usage
|
||||
|
||||
### View Job Status
|
||||
|
||||
```bash
|
||||
nomad job status spore
|
||||
```
|
||||
|
||||
### Inspect Task
|
||||
|
||||
```bash
|
||||
nomad job inspect spore
|
||||
```
|
||||
|
||||
### Watch Job Events
|
||||
|
||||
```bash
|
||||
nomad job status spore -verbose
|
||||
```
|
||||
|
||||
### Connect to Task
|
||||
|
||||
```bash
|
||||
nomad alloc exec <alloc-id> <task-name> /bin/sh
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Services Not Starting
|
||||
|
||||
Check logs:
|
||||
|
||||
```bash
|
||||
make nomad-logs
|
||||
```
|
||||
|
||||
### Port Conflicts
|
||||
|
||||
If ports are in use, modify the job specification in `spore.hcl`:
|
||||
|
||||
```hcl
|
||||
network {
|
||||
mode = "bridge"
|
||||
port "http" {
|
||||
static = 3001 # Change this
|
||||
to = 3001
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Images Not Found
|
||||
|
||||
Ensure Docker images are built and available:
|
||||
|
||||
```bash
|
||||
docker images | grep wirelos
|
||||
```
|
||||
|
||||
If using a registry, update the image URLs in `spore.hcl`.
|
||||
|
||||
|
||||
## Production Deployment
|
||||
|
||||
For production environments:
|
||||
|
||||
1. **Configure ACLs**: Enable authentication
|
||||
2. **Use Nomad Agents**: Deploy on multiple nodes
|
||||
3. **TLS**: Enable TLS for all connections
|
||||
4. **Resource Limits**: Adjust based on cluster capacity
|
||||
5. **Persistence**: Use external storage for volumes
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Nomad Documentation](https://www.nomadproject.io/docs)
|
||||
- [SPORE Documentation](../README.md)
|
||||
|
||||
Reference in New Issue
Block a user