Development Guide¶

This guide helps you set up a development environment for OpenLatch. Target audience: junior to mid-level developers on Linux, macOS, or Windows.

Prerequisites¶

You need the following tools installed:

Tool	Version	Purpose
Python	3.12+	Backend server, tooling
pip	latest	Python package manager
Docker + Docker Compose	latest	Local PostgreSQL database
PlatformIO CLI	6.x	ESP32 firmware build + test
Git	2.x+	Version control
A C++ compiler	gcc/clang	Native firmware tests (runs on your machine)

Install Python 3.12+¶

macOS (Homebrew)Linux (Debian/Ubuntu)Linux (Fedora/RHEL)Windows

brew install python@3.12

sudo apt update
sudo apt install python3.12 python3.12-venv python3-pip

sudo dnf install python3.12 python3-pip

Download from python.org or use winget:

winget install Python.Python.3.12

Make sure to check "Add Python to PATH" during installation.

Install Docker¶

macOSLinux (Debian/Ubuntu)Linux (Fedora/RHEL)Windows

brew install --cask docker
# Then start Docker Desktop from Applications

sudo apt install docker.io docker-compose-v2
sudo usermod -aG docker $USER
# Log out and back in for group change to take effect

sudo dnf install docker docker-compose
sudo systemctl enable --now docker
sudo usermod -aG docker $USER
# Log out and back in for group change to take effect

Download Docker Desktop for Windows or:

winget install Docker.DockerDesktop

Requires WSL 2 enabled. Docker Desktop will guide you through this.

Install PlatformIO CLI¶

PlatformIO is used to build and test the ESP32 firmware. The CLI is sufficient — you don't need the IDE.

pip install platformio

Verify:

pio --version

Note: PlatformIO downloads the ESP32 toolchain automatically on first build. The native test environment (which runs on your computer, no hardware needed) requires only a C++ compiler.

Install a C++ Compiler¶

This is needed for running firmware tests on your development machine (native tests — no ESP32 required).

macOSLinux (Debian/Ubuntu)Linux (Fedora/RHEL)Windows

xcode-select --install

This installs clang. Already done if you have Xcode or Command Line Tools.

sudo apt install build-essential

sudo dnf install gcc-c++ make

Install MSYS2 and then from the MSYS2 terminal:

pacman -S mingw-w64-x86_64-gcc

Or install Visual Studio Build Tools with the "Desktop development with C++" workload.

Project Setup¶

1. Clone the Repository¶

git clone <repository-url>
cd openlatch

2. Start PostgreSQL¶

cd tools
docker compose up -d
cd ..

This starts a PostgreSQL 16 instance on localhost:5432 with: - Database: openlatch - User: postgres - Password: postgres

3. Set Up the Backend¶

cd backend

# Create a virtual environment
python3 -m venv .venv

# Activate it
source .venv/bin/activate    # Linux/macOS
# .venv\Scripts\activate     # Windows (cmd)
# .venv\Scripts\Activate.ps1 # Windows (PowerShell)

# Install dependencies (including dev tools)
pip install -e ".[dev]"

4. Verify Backend Tests Pass¶

cd backend
source .venv/bin/activate  # if not already active
pytest -v

You should see:

tests/test_health.py::test_health_check PASSED

5. Verify Firmware Native Tests Pass¶

cd firmware
pio test -e native

You should see:

39 Tests 0 Failures 0 Ignored
OK

Note: This runs on your computer — no ESP32 hardware needed. The native test environment compiles the firmware logic as a regular C++ program and runs the tests locally.

6. (Optional) Generate Ed25519 Keypair¶

For local development, you can generate a signing keypair:

source backend/.venv/bin/activate
python tools/keygen.py

This prints the keys to stdout. Use --write to save them to files.

Development Workflow¶

TDD (Test-Driven Development)¶

We follow TDD. For every new feature:

Write a failing test first
Write the minimal code to make the test pass
Refactor if needed, keeping tests green
Commit with a meaningful message

Running Tests¶

# Backend
cd backend && source .venv/bin/activate && pytest -v

# Firmware (native tests, no hardware needed)
cd firmware && pio test -e native

# Firmware (on real ESP32, requires hardware)
cd firmware && pio test -e esp32dev

Code Style & Linting¶

Backend (Python):

cd backend && source .venv/bin/activate

# Check formatting
ruff format --check .

# Check lint rules
ruff check .

# Auto-fix
ruff format .
ruff check --fix .

Firmware (C++):

cd firmware

# Static analysis
pio check

Git Commit Messages¶

Write clear, descriptive commit messages. Format:

Short summary of what changed (imperative mood)

Optional longer description explaining why the change was made,
any important design decisions, or context for reviewers.

Good examples: - Add time slot evaluation for overnight events - Fix grace period calculation crossing midnight - Implement Ed25519 signature verification in firmware

Bad examples: - fix stuff - WIP - update

Branching¶

main — stable, CI must pass
Feature branches: feature/add-member-api, feature/wiegand-decoder
Bug fixes: fix/overnight-time-slot

Project Structure¶

openlatch/
├── backend/           Python FastAPI backend
│   ├── app/           Application code
│   │   ├── auth/      Authentication (JWT, passwords)
│   │   ├── models/    SQLAlchemy models + Pydantic schemas
│   │   ├── routes/    API endpoints
│   │   ├── services/  Business logic (rules engine, signing)
│   │   └── crypto/    Ed25519 key management
│   └── tests/         pytest tests
├── firmware/          ESP32 PlatformIO project
│   ├── include/       C++ header files
│   ├── src/           C++ source files
│   └── test/
│       ├── test_native_*/   Tests that run on your computer (one dir per suite)
│       └── test_embedded/   Tests that run on real ESP32
├── proto/             Allow-list protocol specification
├── tools/             Keygen, docker-compose
└── docs/              Architecture, diagrams, this file

Where to Start¶

"I want to..."	Start here
Understand the system	docs/diagrams.md
Work on the backend API	`backend/app/routes/`
Work on access rules	`backend/app/services/rules_engine.py`
Work on firmware logic	`firmware/src/` + `firmware/test/test_native_*/`
Understand the binary protocol	proto/allowlist.md
Change the database schema	`backend/alembic/versions/`

CI/CD Pipeline¶

GitLab CI runs automatically on every push. The pipeline has 4 stages:

lint → test → build → security-scan

Job	Stage	What it does
`backend:lint`	lint	ruff format + ruff check
`firmware:lint`	lint	pio check (C++ static analysis)
`backend:test`	test	pytest with PostgreSQL
`firmware:test:native`	test	pio test -e native
`cross:test`	test	Python signs → C++ verifies
`backend:build`	build	Docker image (main branch only)
`firmware:build`	build	ESP32 binary (main branch only)
`security:backend`	security-scan	pip-audit
`security:firmware`	security-scan	pio check

Your merge request must pass at least lint and test stages.

Common Tasks¶

Add a New API Endpoint¶

Create a test in backend/tests/test_<feature>.py
Add the route in backend/app/routes/<feature>.py
Register the router in backend/app/main.py
Run pytest -v to verify

Add a New Firmware Module¶

Create header in firmware/include/<module>.h
Create source in firmware/src/<module>.cpp
Add native tests in firmware/test/test_native_<module>/test_<module>.cpp
Run pio test -e native to verify

Update the Database Schema¶

Modify the SQLAlchemy model in backend/app/models/

Create a migration:

cd backend
alembic revision --autogenerate -m "description of change"

Apply it:
```
alembic upgrade head
```

Flash Firmware to ESP32¶

cd firmware
pio run -e esp32dev --target upload

Monitor serial output:

pio device monitor

Troubleshooting¶

`pio test -e native` fails with linker errors¶

Make sure you have a C++ compiler installed (see prerequisites). On macOS, run xcode-select --install.

PostgreSQL connection refused¶

Make sure Docker is running and the database container is up:

cd tools && docker compose ps
# If not running:
docker compose up -d

Python `ModuleNotFoundError`¶

Make sure you activated the virtual environment:

cd backend
source .venv/bin/activate  # Linux/macOS

PlatformIO downloads take forever on first run¶

This is normal. PlatformIO downloads the ESP32 toolchain (~500MB) on first build. Subsequent builds are fast.