Testing And Verification
Last reviewed: 2026-07-05
Canonical Checks
| Command | Status | Purpose |
|---|---|---|
git diff --check |
Implemented Git check | Required by AGENTS.md for whitespace/diff sanity. |
.\.venv\Scripts\python.exe -m pytest |
Implemented | Runs all tests. |
.\.venv\Scripts\python.exe -m pytest tests\unit |
Implemented | Unit tests for models, config, job prep, dashboard, simulator, output, security, operations. |
.\.venv\Scripts\python.exe -m pytest tests\integration |
Implemented | Simulator HTTP and fake Unity ZeroMQ integration tests. |
.\.venv\Scripts\python.exe -m mkdocs build --strict |
Implemented docs check | Required for docs and MkDocs nav changes. |
There is no repository-local Ruff or mypy configuration as of this review.
Test Categories
| Category | Status | Tests |
|---|---|---|
| Config and operations | Implemented | tests/unit/test_config.py, tests/unit/test_operations.py |
| Web App models/client | Implemented | tests/unit/test_webapp_models_and_client.py |
| Workspace, manifest, config rewrite, asset download | Implemented | tests/unit/test_workspace.py, tests/unit/test_job_preparation.py |
| Content server safety | Implemented | tests/unit/test_content_server.py |
| Dashboard routes/manual controls | Implemented/Partial | tests/unit/test_dashboard.py, tests/unit/test_manual_claim.py |
| Simulator state and HTTP API | Implemented/Partial | tests/unit/test_simulator.py, tests/integration/test_simulator_app.py |
| Unity protocol and ZeroMQ client | Implemented | tests/unit/test_unity_protocol.py, tests/integration/test_unity_zmq_client.py |
| Unity process/supervisor | Implemented/Partial | tests/unit/test_unity_process_and_supervisor.py |
| Output validation, upload, cancellation, completion | Implemented | tests/unit/test_output_and_completion.py |
| Health/GPU/system metrics | Implemented | tests/unit/test_health.py |
| Security redaction | Implemented | tests/unit/test_security.py |
Integration Prerequisites
The repo's integration tests use fake local services and do not require real Unity, S3, or Web App. Manual smoke tests against real Unity/Web App require:
- Real or local Web App with worker API and token configured.
- Unity Editor Play Mode or built Unity EXE using the documented ZeroMQ topology.
- Local content certificate when HTTPS content server is enabled.
- ffmpeg/ffprobe if strict media validation is enabled.
Verification Gaps
| Area | Status |
|---|---|
| Real Payload/Web App end-to-end worker API | External |
| Real signed S3 upload with production Web App | External |
| Real Unity Editor/EXE render lifecycle | External |
| Windows service/session behavior with Unity graphics | External/Unknown |
| Long-running auto polling loop | Stubbed in current CLI |