iot/spore-deployment
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: multiarch build

Browse Source
This commit is contained in:
0x1d
2025-10-27 21:38:20 +01:00
parent 7939ea05f7
commit ee70e99ea1
5 changed files with 535 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

180
docs/DOCKER.md Normal file
View File

@@ -0,0 +1,180 @@
# SPORE Docker Build Guide
This document provides instructions for building and running SPORE services as Docker containers.
## Prerequisites
- Docker installed on your system
- Docker Compose (optional, for running multiple services)
## Building Docker Images
### Using Makefiles
Each project includes a Makefile with Docker targets for easy building and pushing:
```bash
# Build Docker image (single architecture)
make docker-build
# Push Docker image (optional, requires DOCKER_REGISTRY variable)
make docker-push
```
**Optional**: Specify a Docker registry and tag:
```bash
# Build with custom tag
make docker-build IMAGE_TAG=v1.0.0
# Push to custom registry
make docker-push DOCKER_REGISTRY=your-registry.com IMAGE_TAG=v1.0.0
```
### Building Multiarch Images (amd64 & arm64)
For Raspberry Pi and other ARM devices, build multiarch images:
```bash
# One-time setup for buildx
docker run --rm --privileged tonistiigi/binfmt:latest --install all
docker buildx create --name multiarch --use
docker buildx inspect --bootstrap
# Build and push multiarch images
make docker-build-multiarch IMAGE_TAG=latest
```
This builds images for both `linux/amd64` and `linux/arm64` architectures simultaneously.
### Direct Docker Commands
### spore-gateway
From the `spore-gateway` directory:
```bash
docker build -t spore-gateway:latest .
docker run -p 3001:3001 -p 4210:4210/udp spore-gateway:latest
```
### spore-ledlab
From the `spore-ledlab` directory:
```bash
docker build -t spore-ledlab:latest .
docker run -p 8080:8080 --env GATEWAY_URL=http://localhost:3001 spore-ledlab:latest
```
### spore-registry
From the `spore-registry` directory:
```bash
docker build -t spore-registry:latest .
docker run -p 8080:8080 -v registry-data:/data/registry spore-registry:latest
```
Note: The registry data is stored in a Docker volume for persistence.
### spore-ui
From the `spore-ui` directory:
```bash
docker build -t spore-ui:latest .
docker run -p 3000:3000 spore-ui:latest
```
## Environment Variables
### spore-ledlab
- `PORT`: Server port (default: 8080)
- `UDP_PORT`: UDP discovery port (default: 4210)
- `GATEWAY_URL`: Gateway URL (default: http://localhost:3001)
- `FILTER_APP_LABEL`: Filter app label (default: pixelstream)
- `MATRIX_WIDTH`: Matrix width (default: 16)
- `MATRIX_HEIGHT`: Matrix height (default: 16)
### spore-gateway
- HTTP server port: 3001 (override with `-port` flag)
- UDP discovery port: 4210 (override with `-udp-port` flag)
- Log level (default: info, override with `-log-level` flag)
### spore-ui
- `PORT`: Server port (default: 3000)
## Docker Compose (Optional)
You can create a `docker-compose.yml` file to run all services together:
```yaml
version: '3.8'
services:
gateway:
build: ./spore-gateway
ports:
- "3001:3001"
- "4210:4210/udp"
ledlab:
build: ./spore-ledlab
ports:
- "8080:8080"
environment:
- GATEWAY_URL=http://gateway:3001
depends_on:
- gateway
registry:
build: ./spore-registry
ports:
- "8090:8080"
volumes:
- registry-data:/data/registry
ui:
build: ./spore-ui
ports:
- "3000:3000"
volumes:
registry-data:
```
Then run:
```bash
docker-compose up -d
```
## Troubleshooting
### Port Conflicts
If ports are already in use, modify the port mappings in the `docker run` commands or in the `docker-compose.yml` file.
### Registry Data Persistence
The spore-registry service uses a Docker volume to persist data. To backup or restore:
```bash
# Backup
docker run --rm -v registry-data:/data -v $(pwd):/backup alpine tar czf /backup/registry-backup.tar.gz /data
# Restore
docker run --rm -v registry-data:/data -v $(pwd):/backup alpine tar xzf /backup/registry-backup.tar.gz -C /
```
### Network Configuration
When running services in separate containers, make sure they can reach each other. Either:
- Use Docker networking with docker-compose
- Use host networking: `--network host`
- Update service URLs to point to appropriate hosts/containers

235
docs/MULTIARCH.md Normal file
View File

@@ -0,0 +1,235 @@
# Multiarch Docker Images for SPORE
This guide explains how to build and deploy SPORE Docker images that support multiple architectures (amd64 and arm64).
## Overview
SPORE services can run on both x86_64 (amd64) and ARM64 systems (including Raspberry Pi). To support both architectures, we use Docker's buildx feature to create multiarch images.
## Supported Architectures
- **linux/amd64** - Intel/AMD 64-bit processors
- **linux/arm64** - ARM 64-bit processors (Raspberry Pi 4+, Apple Silicon, etc.)
## Setup (One-Time)
### Option 1: Automated Setup Script
```bash
cd spore-deployment
./scripts/setup-multiarch.sh
```
### Option 2: Manual Setup
```bash
# Install QEMU emulation for cross-architecture builds
docker run --rm --privileged tonistiigi/binfmt:latest --install all
# Create and configure the multiarch builder
docker buildx create --name multiarch --use
docker buildx inspect --bootstrap
# Verify setup
docker buildx ls
```
You should see a builder named `multiarch` with support for `linux/amd64` and `linux/arm64`.
## Building Multiarch Images
### All Services at Once
From the `spore-deployment` directory:
```bash
# Build and push all multiarch images
make push-multiarch
# Build without pushing (for local testing)
make build-multiarch
# Build with custom tag
make push-multiarch IMAGE_TAG=v1.0.0
```
### Individual Services
From each service directory:
```bash
# spore-gateway
cd spore-gateway
make docker-build-multiarch
# spore-registry
cd spore-registry
make docker-build-multiarch
# spore-ledlab
cd spore-ledlab
make docker-build-multiarch
# spore-ui
cd spore-ui
make docker-build-multiarch
```
## How It Works
1. **Docker Buildx** - Uses Docker's extended build capabilities
2. **QEMU Emulation** - Allows building ARM images on x86_64 machines
3. **Multi-platform build** - Creates a single image manifest supporting multiple architectures
4. **Automatic selection** - Docker automatically pulls the correct architecture for the host system
## Verifying Multiarch Images
Check which architectures are supported:
```bash
docker buildx imagetools inspect wirelos/spore-gateway:latest
```
Output should show both:
- linux/amd64
- linux/arm64
## Using Multiarch Images
Once pushed to a registry, images are automatically used by Docker based on the host architecture:
**On x86_64 systems:**
```bash
docker pull wirelos/spore-gateway:latest
# Docker automatically uses amd64 version
```
**On Raspberry Pi (ARM64):**
```bash
docker pull wirelos/spore-gateway:latest
# Docker automatically uses arm64 version
```
## Docker Compose
The existing `docker-compose.yml` works automatically with multiarch images:
```bash
cd spore-deployment
make up
```
Docker Compose will pull the correct architecture for your system.
## Nomad Deployment
Multiarch images work automatically with Nomad:
```bash
cd spore-deployment
make nomad-job-run
```
Nomad will schedule pods based on available nodes and their architectures.
## Troubleshooting
### Check Available Platforms
```bash
docker buildx inspect multiarch
```
### Switch Builders
```bash
# Use multiarch builder
docker buildx use multiarch
# Use default builder
docker buildx use default
```
### Remove and Recreate Builder
```bash
docker buildx rm multiarch
docker buildx create --name multiarch --use
docker buildx inspect --bootstrap
```
### Build for Specific Platform Only
```bash
# Build only for ARM64
docker buildx build --platform linux/arm64 -t wirelos/spore-gateway:latest .
# Build only for AMD64
docker buildx build --platform linux/amd64 -t wirelos/spore-gateway:latest .
```
### Common Issues
#### Error: "multiple platforms feature is currently not supported"
- Solution: Enable buildkit: `export DOCKER_CLI_EXPERIMENTAL=enabled`
#### Error: "failed to solve: failed to compute cache key"
- Solution: Make sure all source files are present in build context
#### Slow builds on first run
- Normal: QEMU emulation is slower than native builds
- Subsequent builds use Docker layer cache and are faster
## CI/CD Integration
### GitHub Actions Example
```yaml
name: Build and Push Multiarch
on:
push:
branches: [main]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up QEMU
uses: docker/setup-qemu-action@v1
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
- name: Build and push
run: |
cd spore-deployment
make push-multiarch IMAGE_TAG=${{ github.sha }}
```
## Security Considerations
- Multiarch images are signed for each platform
- When using third-party base images, ensure they also support multiarch
- Test on both architectures before production deployment
## Performance Notes
### Build Performance
- Native builds (build on the target architecture): Fastest
- Cross-compilation (build on different architecture): Slower but convenient
- Emulation via QEMU: Slowest but allows building all architectures on one machine
### Runtime Performance
- No difference: Once the correct architecture image is pulled, performance is the same as single-arch builds
## Additional Resources
- [Docker Buildx Documentation](https://docs.docker.com/buildx/)
- [Docker Multiarch Tutorial](https://www.docker.com/blog/multi-arch-build-and-images-the-simple-way/)
- [QEMU Documentation](https://www.qemu.org/documentation/)