Skip to content

Presentation Video Step 3 Implementation Plan

Created: 2026-05-26 02:46 UTC

Goal

Make Step 3 a reliable scene-authoring loop where scene timing, popup voiceover regeneration, scene visual overrides, and PresentationConfig.json recompilation stay in sync without manual workarounds.

Non-goals

  • Do not change Step 1 or Step 2 flows.
  • Do not change narrative generation, final video rendering, or the broader Mastra workflow structure.
  • Do not introduce OpenSpec artifacts.
  • Do not add Payload schema changes unless implementation proves absolutely necessary.

Current Behavior

  • pauseSeconds exists in scene data but is not editable in Step 3.
  • The Step 3 voiceover popup edits local draft text, but Generate Audio currently uses saved scene text.
  • Generate Slides Preview recompiles from persisted scene records and creates a new preview compilation.
  • Generate VO Audio regenerates missing or dirty scene audio, but compiled timing and duration refresh is not consistently aligned to actual TTS timing output.
  • Scene visual overrides already persist and mark scenes dirty, but their recompilation contract needs to remain explicit.
  • Why This Visual still renders in desktop and mobile Step 3.
  • Mobile Step 3 uses a reduced preview and bottom controls rather than a sticky preview at the top of the scrolling editor.

Target Behavior

  • Each scene exposes editable float pauseSeconds, including the final scene.
  • The Edit Voiceover popup generates temporary ElevenLabs audio from the current draft text.
  • The user can play the temporary regenerated clip inside the popup.
  • If the user regenerates again, the previous temporary clip is discarded and replaced.
  • If the user clicks Cancel, the previous committed text and audio remain unchanged and the temporary clip is discarded.
  • If the user clicks Done, the popup commits the new text/audio to the scene, marks preview stale, and does not auto-recompile.
  • Generate Slides Preview recompiles all persisted scene edits into PresentationConfig.json, including voiceover text/audio metadata, pause timing, caption changes, and visual overrides.
  • Generate VO Audio regenerates only dirty or missing scenes and refreshes the latest compiled snapshot using actual TTS timing metadata.
  • Why This Visual is removed from Step 3.
  • Mobile Step 3 uses a vertical flow with the preview visual pinned at the top of the scrolling editor form.

Mermaid Overview

flowchart TD
  select["Select scene"] --> edit["Edit voiceover / caption / pause / visual"]
  edit --> popup["Voiceover popup draft audio"]
  popup --> done{"Done or Cancel?"}
  done -->|Done| persist["Persist scene text/audio + mark preview stale"]
  done -->|Cancel| discard["Discard temporary draft audio"]
  persist --> preview["Generate Slides Preview"]
  preview --> compile["Recompile PresentationConfig.json"]
  compile --> audio["Generate VO Audio for dirty/missing scenes"]
  audio --> refresh["Refresh latest compiled snapshot with actual timing"]

Affected Files And Modules

Primary implementation files

  • src/components/anuvax/presentation-video/presentation-video-page.tsx
  • src/app/(frontend)/app/presentation/actions.ts

Supporting logic likely touched

  • src/lib/presentation-config/schema.ts
  • src/lib/render-worker/service.ts
  • src/lib/voiceover/elevenlabs-timing.ts

Tests

  • tests/integration/presentation-scene-edit-preview.integration.test.ts
  • tests/integration/presentation-assistants.integration.test.ts
  • tests/integration/presentation-compile.integration.test.ts
  • tests/e2e/presentation-video.e2e.spec.ts
  • tests/integration/helpers/mock-runtime.ts

Affected Docs

  • docs/core/PresentationVideoWorkflow.md
  • docs/core/PresentationConfig.md
  • docs/state/ServerActionsMap.md
  • docs/state/CurrentImplementationMap.md
  • docs/state/KnownGaps.md if an existing gap is closed or reduced
  • docs/changes/2026-05-26-0246-presentation-video-step-3/ImplementationLog.md

Data / Schema Impact

  • No Payload collection schema changes are planned.
  • No generate:types or generate:importmap run should be required.
  • Reuse existing scene fields:
  • pauseSec
  • voiceoverAudio
  • voiceoverAudioStatus
  • voiceoverAudioDirty
  • voiceoverDurationSec
  • wordTimestampsActual
  • attachedImages
  • attachedVideo
  • computedStartSec
  • computedEndSec
  • Temporary popup clips should use existing voiceover_audio_assets records and be cleaned up explicitly on replace or cancel.
  • Replaced committed voiceover assets should be deleted after successful commit to avoid orphaned audio records.

Implementation Approach

1. Timing and duration authority

  • Add a shared helper in presentation actions to derive actual scene duration from wordTimestampsActual by taking the max end value.
  • Fall back to the existing estimated duration only when actual word timings are unavailable.
  • Update all timing recomputation paths to use the same helper so computedStartSec, computedEndSec, and voiceoverDurationSec stay aligned.

2. Scene pause editing

  • Extend updatePresentationScene to accept pauseSeconds?: number | null.
  • Persist pauseSeconds into pauseSec.
  • Mark the scene dirty and mark preview stale when pauseSeconds changes.
  • Add float input support in Step 3 with step=0.1 and clamp to the existing schema range 0..10.

3. Popup draft audio workflow

  • Keep committed scene audio unchanged while the popup is open.
  • Add a draft audio generation action that accepts sceneId, current draft text, and an optional previous temporary asset id.
  • Have popup Generate Audio call the draft action using current popup text, not saved scene text.
  • Return temporary audioUrl, temporary audioAssetId, duration, and word timings to the client.
  • Delete the previous temporary asset when a new temporary clip is generated.
  • On Cancel, delete the active temporary asset and leave the scene unchanged.
  • On Done, persist the draft text plus temporary audio asset to the scene, mark voiceoverAudioDirty false, store voiceoverDurationSec, store wordTimestampsActual, and mark preview stale.
  • After successful commit, delete the previously committed scene audio asset if it was replaced.

4. Preview compile contract

  • Keep Generate Slides Preview as the only user action that creates a new presentation_compilations record.
  • Ensure preview recompilation includes:
  • updated pauseSeconds
  • updated committed voiceover text
  • updated committed scene audio asset
  • updated voiceover.audio.durationSeconds
  • updated voiceover.words
  • updated voiceover.words.start/end
  • updated scenes.timing.start/end
  • updated visual overrides
  • Preserve current stale-preview blocking behavior for Generate Video.

5. Deferred bulk audio generation

  • Keep Generate VO Audio limited to scenes with missing audio, dirty audio, or error audio.
  • After bulk generation, recompute scene timing using actual durations.
  • Refresh the latest compilation snapshot in place rather than creating a new preview compilation version.
  • Ensure renderReady becomes true only when all required scenes have audio, valid duration, and are no longer dirty.

6. Layout refresh

  • Rework the Step 3 desktop layout to follow the mockup directionally while preserving existing spacing tokens and current component primitives.
  • Remove Why This Visual from both desktop and mobile layouts.
  • Move mobile Step 3 to a vertical flow with a sticky preview visual pinned at the top of the scrolling editor form.

Ordered Tasks

  • [x] 1. Add shared actual-duration helper and route all timing recomputation through it.
  • [x] 2. Extend updatePresentationScene and Step 3 client state for editable pauseSeconds.
  • [x] 3. Add popup draft-audio actions for generate, discard, and commit.
  • [x] 4. Rewire the popup UI to use draft text/audio state, play generated draft clips, and clean up replaced/canceled clips.
  • [x] 5. Update Step 3 local dirty-state logic so pause and committed popup changes disable Generate Video until preview recompiles.
  • [x] 6. Rework desktop/mobile Step 3 layout and remove Why This Visual.
  • [x] 7. Tighten preview compile so all persisted scene edits are reflected in PresentationConfig.json.
  • [x] 8. Update deferred bulk audio generation to refresh compiled timing and metadata in place.
  • [x] 9. Add or update integration coverage for pause, popup lifecycle, preview recompilation, and visual override propagation.
  • [x] 10. Add or update e2e coverage for Step 3 popup flow, dirty-state recovery, and mobile sticky preview behavior.
  • [x] 11. Update docs and record implementation notes in ImplementationLog.md.

Verification Plan

Run:

  • pnpm lint
  • pnpm typecheck
  • pnpm test:int
  • pnpm test:e2e

Required scenarios

  • Editing pauseSeconds marks preview stale and changes compiled timing after Generate Slides Preview.
  • Popup Generate Audio uses unsaved draft text, not saved scene text.
  • Popup Generate Audio returns a playable temporary clip.
  • Re-generating audio in the popup replaces the previous temporary clip.
  • Canceling the popup preserves prior committed text/audio and removes the temporary clip.
  • Clicking Done commits the new voiceover text/audio and marks preview stale without auto-recompiling.
  • Generate Slides Preview updates:
  • scenes.timing.start
  • scenes.timing.end
  • scenes.voiceover.audio.durationSeconds
  • scenes.voiceover.words
  • scenes.voiceover.words.start/end
  • Generate VO Audio regenerates only dirty/missing scenes and refreshes compiled metadata using actual timings.
  • Scene image overrides appear in compiled config after preview generation.
  • Why This Visual is absent on desktop and mobile.

Docs Parity Checklist

  • [x] Update Step 3 authoring flow in docs/core/PresentationVideoWorkflow.md
  • [x] Update timing/audio metadata source-of-truth in docs/core/PresentationConfig.md
  • [x] Update Step 3 action inventory in docs/state/ServerActionsMap.md
  • [x] Update current scene-editor behavior notes in docs/state/CurrentImplementationMap.md
  • [x] docs/state/KnownGaps.md did not require changes because this work did not close a separately tracked gap entry.
  • [x] Update docs/changes/2026-05-26-0246-presentation-video-step-3/ImplementationLog.md during execution

Acceptance Criteria

  • pauseSeconds is editable as a float for every scene, including the final scene.
  • Popup-generated voiceover audio stays temporary until Done.
  • Popup cancel discards temporary audio and leaves the committed scene untouched.
  • Popup re-generate discards the previous temporary clip and replaces it with the new one.
  • Popup Done commits new voiceover text/audio and marks preview stale without auto-recompiling.
  • Generate Slides Preview recompiles all persisted scene edits into a new preview compilation.
  • Generate VO Audio updates only dirty/missing scenes and refreshes compiled timing/audio metadata.
  • PresentationConfig.json reflects updated timing and word-level audio metadata after the relevant recompilation path.
  • Scene image overrides are present in compiled config after preview generation.
  • Why This Visual is removed from Step 3 and mobile shows a sticky top preview visual.

Assumptions And Defaults

  • No Payload schema change is needed for the popup draft-audio workflow.
  • Temporary popup clips are stored as standard voiceover_audio_assets records and cleaned up explicitly.
  • Replaced committed scene audio assets are deleted after successful commit.
  • Actual TTS duration is derived from the highest word.end in wordTimestampsActual.
  • Estimated duration remains fallback-only for scenes without actual timings.
  • Generate Slides Preview remains the only path that creates a new presentation_compilations row.
  • Deferred bulk audio refresh updates the latest compilation snapshot in place.