Skip to content

Development

Development Environment

Start the development environment with bind-mounted source code:

docker compose -f compose.dev.yml up -d --build
docker compose -f compose.dev.yml exec app bash

The development compose file:

  • Builds the dev target of the local Dockerfile, which adds the tests extra and the dev dependency group to the runtime dependencies. The default target is the image that CI publishes, and it contains only the runtime dependencies
  • Mounts the source tree at /app
  • Uses a named volume for .venv (avoids host/container venv conflicts)
  • Runs sleep infinity so you can exec into the container and start the service manually

Inside the container, start the server in debug mode:

sapporo --debug

Local Setup

Install the package with test dependencies:

uv sync --all-extras

Running the Server

Start the service in debug mode for development:

sapporo --debug

The --debug flag enables hot-reloading and verbose logging. Access http://localhost:1122/docs for the interactive Swagger UI.

Testing

Adding Tests

  1. Add tests to the test_<module>.py file corresponding to the target module
  2. Naming convention: test_<target>_<condition>_<expected_result>()
  3. Use hypothesis @given when property-based testing is applicable
  4. Confirm PASS with uv run pytest -v
  5. Confirm lint-clean with uv run ruff check tests/ and uv run ruff format --check tests/
  6. Optionally verify detection power with uv run mutmut run --paths-to-mutate sapporo/<module>.py

Running Tests

# Run all tests
uv run pytest

# Verbose output
uv run pytest -v

# Specific file
uv run pytest tests/unit/test_utils.py

# Specific test
uv run pytest tests/unit/test_utils.py::test_now_str_returns_rfc3339_utc_format

# Run in random order (check for order dependencies)
uv run pytest -p randomly -v

# Exclude slow tests
uv run pytest -m "not slow"

Mutation Testing

# Run according to [tool.mutmut] settings in pyproject.toml
uv run mutmut run

# View results
uv run mutmut results

# Show details of a survived mutant
uv run mutmut show <mutant_id>

Integration Tests

Integration tests require a running sapporo instance with Docker-in-Docker support (e.g., via compose.dev.yml). Each test file exercises a specific workflow engine:

Test File Engine Workflow Type
test_cwltool.py cwltool CWL
test_toil.py toil CWL
test_ep3.py ep3 CWL
test_cromwell.py cromwell WDL
test_nextflow.py nextflow NFL
test_snakemake.py snakemake SMK
test_streamflow.py streamflow CWL
test_ro_crate.py all of the above (multi-engine RO-Crate validation)

test_cancel.py covers cancellation, including that every engine with Docker socket access stops its step containers. test_delete.py, test_auth.py and test_auth_external.py cover deletion and authentication with cwltool.

Run integration tests inside the development container:

docker compose -f compose.dev.yml up -d --build
docker compose -f compose.dev.yml exec app bash

# Start sapporo in background
sapporo --debug &

# Run all integration tests
pytest tests/integration/ -v

# Run a specific engine
pytest tests/integration/test_toil.py -v

# Exclude slow tests (large workflows, roc-validator)
pytest tests/integration/ -v -m "not slow"

Linting and Type Checking

The project uses ruff for linting/formatting and mypy for type checking. Configuration is in pyproject.toml.

# Lint
uv run ruff check .

# Format check
uv run ruff format --check .

# Auto-fix lint issues
uv run ruff check --fix .

# Auto-format
uv run ruff format .

# Type check
uv run mypy

OpenAPI Spec Generation

The OpenAPI specification file (openapi/sapporo-wes-spec-X.X.X.yml) is generated from the FastAPI application. Regenerate it after changing schemas, endpoint descriptions, or the spec version:

uv run sapporo-cli generate-openapi

The generated file should be committed to the repository.

Spec Version

SAPPORO_WES_SPEC_VERSION in sapporo/config.py is the single source of truth for the sapporo-wes spec version. This constant is referenced throughout the codebase to produce the **sapporo-wes-X.X.X extension:** markers in OpenAPI descriptions, info.version, and example values.

To bump the spec version:

  1. Update SAPPORO_WES_SPEC_VERSION in sapporo/config.py
  2. Regenerate the spec: uv run sapporo-cli generate-openapi
  3. Rename the output file if the major/minor version changed

Continuous Integration

GitHub Actions runs the following workflows (.github/workflows/):

Workflow Trigger What it runs
lint push to main, pull requests ruff check, ruff format check, mypy on Python 3.11 to 3.14
pytest push to main, pull requests unit tests on Python 3.11 to 3.14; mutation tests on manual dispatch
latest-deps weekly, manual dispatch uv lock --upgrade, then ruff, mypy and pytest against the newest versions allowed by pyproject.toml
docs push to main mkdocs build and deploy to GitHub Pages
release tag push PyPI publish, Docker image push, GitHub Release

lint and pytest install from uv.lock (uv sync --locked), so they test the same versions that the Docker image gets. latest-deps ignores the lock, so it catches an upstream change (for example a removed import) before it reaches pip install sapporo users. Dependabot opens a weekly pull request that updates uv.lock.

Release Process

Version Management

  • The git tag is the single source of truth for the package version (hatch-vcs)
  • At runtime, the version is obtained via importlib.metadata.version("sapporo")
  • Docker image version is injected by CI from the tag name with --build-arg VERSION=...; the Dockerfile passes it to hatch-vcs through SETUPTOOLS_SCM_PRETEND_VERSION

Release Steps

  1. Create a PR to main and merge
  2. Create a version tag on the main branch and push:
git checkout main && git pull
git tag X.Y.Z
git push origin X.Y.Z
  1. The tag push triggers release.yml automatically:
  2. Publish PyPI package (Trusted Publishing)
  3. Build and push multi-architecture Docker image (ghcr.io)
  4. Create GitHub Release (auto-generated release notes)

Differences from 1.x

  • Changed from Flask to FastAPI
  • Updated base GA4GH WES from 1.0.0 to 1.1.0
  • Reorganized authentication with switchable methods
  • Added SQLite database for faster GET /runs queries
  • Organized Python and Docker toolchain
  • Simplified executable_workflows.json to a list of workflow_urls
  • Full support for automatic run directory cleanup
  • See the sapporo-wes-2.0.0 specification for detailed API changes