Skip to content

Error Handling And Recovery

Last reviewed: 2026-07-05

Implemented Handling

Failure area Status Behavior Source
Invalid config Implemented Pydantic validation fails; persisted secret keys are rejected. src/anuva_python_server/config/settings.py
Web App request failure Implemented Client records last failure time/error and logs redacted context; caller handles HTTP errors. src/anuva_python_server/webapp/client.py
Manual claim in wrong mode Implemented Raises ManualClaimModeError, mapped by dashboard. src/anuva_python_server/jobs/runner.py
Manual claim while busy Implemented Raises WorkerBusyError unless explicitly allowed. src/anuva_python_server/jobs/runner.py
Claim conflict Implemented/External Backend/simulator returns conflict; dashboard maps Web App errors. src/anuva_python_server/simulator/state.py, src/anuva_python_server/servers/dashboard.py
Required asset failure Implemented Raises RequiredAssetDownloadError; optional assets can be recorded as skipped. src/anuva_python_server/jobs/assets.py
Unsafe asset filename Implemented Model validation rejects path-like filenames. src/anuva_python_server/webapp/models.py
Local content unavailable Implemented Fails active job with CONTENT_SERVER_UNAVAILABLE before render.start. src/anuva_python_server/jobs/runner.py
Invalid ZeroMQ message Implemented Decode/type validation raises protocol/Pydantic errors. src/anuva_python_server/unity/protocol.py
Unity command timeout Implemented ZmqReceiveTimeout; command socket is reset. src/anuva_python_server/unity/zmq_client.py
Missing Unity readiness Implemented UnityReadinessTimeout. src/anuva_python_server/unity/supervisor.py
Unity heartbeat timeout Implemented UnityHeartbeatTimeout; dashboard can mark stale connection disconnected. src/anuva_python_server/unity/supervisor.py, src/anuva_python_server/servers/dashboard.py
Unity process crash Implemented EXE supervisor raises UnityProcessCrashed. src/anuva_python_server/unity/supervisor.py
Unity render timeout Implemented Runner/supervisor timeout paths fail or raise. src/anuva_python_server/jobs/runner.py, src/anuva_python_server/unity/supervisor.py
Unity render failure Implemented render.failed creates failure report with Unity payload. src/anuva_python_server/jobs/runner.py
Output missing/invalid Implemented OutputValidationError with code and diagnostics. src/anuva_python_server/jobs/output.py
Upload failure Implemented Attempts uploadFailed status and sends UPLOAD_FAILED failure report. src/anuva_python_server/jobs/runner.py
Cancellation before render Implemented Cancels active local job without Unity command. src/anuva_python_server/jobs/runner.py
Cancellation during render Implemented Sends render.cancel; may terminate Unity on missing acknowledgement. src/anuva_python_server/jobs/runner.py

Diagnostics

Failure reports include:

  • workerId
  • jobId
  • runner state
  • retrySafe
  • lastUnityEvent
  • expected log paths
  • diagnostics path
  • caller-specific diagnostics

Health snapshots are written at start/end under jobs/<job-id>/diagnostics/system_snapshot_start.json and system_snapshot_end.json.

Redaction

Structured logging and diagnostics should not leak secrets. The implemented redaction utility covers sensitive mapping keys, Authorization headers, cookies, and signed URL query strings. See src/anuva_python_server/utils/security.py and tests/unit/test_security.py.

Recovery Gaps

Gap Status
Continuous retry queue or backoff loop Planned
Automatic Web App outage recovery beyond per-request health tracking Partial
Automatic Unity restart wired into full render loop Partial
Diagnostics bundle upload Planned
Real Windows service recovery validation Unknown
Real Unity crash and GPU failure end-to-end testing Unknown

Related state map: KnownGaps.