# Empirical Challenge Report: Milestone M3 Gate Verification

**Agent**: Challenger 2 (Empirical Challenger: critic, specialist)  
**Milestone**: M3 (Ep01 10-Scene Production Pipeline Implementation)  
**Verdict**: **REQUEST_CHANGES**  
**Date**: 2026-10-09  

---

## 1. Observation

### 1.1 Audio Guard Discrepancy & Rejection Behavior
- **Inspection of `episodes/ep01/prompts/muse_prompts.json`**:
  Total motion prompts: **192**.
  Exact canonical matches: **188**.
  Non-canonical / legacy matches: **4**.
  Missing: **0**.
  The 4 prologue shots (`prologue_shot_1_lang_que_198x`, `prologue_shot_2_ngam_kieu`, `prologue_shot_3_xuyen_khong`, `prologue_shot_4_gia_tinh_trieu_minh`) lack the canonical text defined in `AGENTS.md` §3 and `05_Production_Pipeline/run_shot.py:34`:
  ```text
  Quy tắc âm thanh: Tuyệt đối KHÔNG sinh nhạc nền (no music/BGM), không âm thanh điện tử, không tạp âm rè nhiễu. Chỉ sinh âm thanh môi trường tự nhiên (foley, ambience) và thoại nhân vật chân thực.
  ```
  Instead, they contain non-canonical fragments:
  - `prologue_shot_1_lang_que_198x`: "... Tuyệt đối KHÔNG sinh nhạc nền, không âm thanh điện tử, không tạp âm rè nhiễu." (Missing prefix `Quy tắc âm thanh:`, missing `(no music/BGM)`, missing positive instruction).
  - `prologue_shot_2_ngam_kieu`: "... Tuyệt đối KHÔNG sinh nhạc nền, không tạp âm rác."
  - `prologue_shot_3_xuyen_khong`: "... Tuyệt đối KHÔNG sinh nhạc nền điện tử, không tạp âm rè méo."
  - `prologue_shot_4_gia_tinh_trieu_minh`: "... Tuyệt đối KHÔNG sinh nhạc nền, không tạp âm rác. [KHÓA KHẨU HÌNH...]"
- **Rejection Behavior in `05_Production_Pipeline/production_orchestrator.py`**:
  - `production_orchestrator.py:71` defines `validate_audio_guard(prompt: str) -> bool`, but `validate_audio_guard` is **never called anywhere** in `production_orchestrator.py`.
  - In `render_single_shot` (`production_orchestrator.py:726`):
    ```python
    prompt = ensure_audio_guard(prompt)
    ```
    When passed a prompt lacking Audio Guard, `render_single_shot` does **not** reject the prompt. It silently auto-appends `AUDIO_GUARD_CANONICAL` and returns `True`.
  - Furthermore, `run_shot.py:40` implements:
    ```python
    if "Tuyệt đối KHÔNG sinh nhạc nền" not in prompt and "Strictly NO background music" not in prompt:
        return prompt.rstrip() + AUDIO_GUARD_CANONICAL
    return prompt
    ```
    Because the 4 non-canonical prologue prompts already contain the substring `"Tuyệt đối KHÔNG sinh nhạc nền"`, `ensure_audio_guard` returns them unmodified, leaving them non-canonical at runtime.

### 1.2 MUSE_DRY_RUN Synthetic Engine & Shot Gate Failure
- **Synthetic Video Generation in `05_Production_Pipeline/run_shot.py:135-142`**:
  ```python
  cmd_synth = [
      ffmpeg_bin, "-y",
      "-f", "lavfi", "-i", "color=c=0x1a2b3c:s=1280x720:d=10:r=24",
      "-f", "lavfi", "-i", "sine=f=440:d=10:r=48000",
      "-c:v", "libx264", "-preset", "ultrafast", "-crf", "23", "-pix_fmt", "yuv420p",
      "-c:a", "aac", "-b:a", "192k", "-ar", "48000",
      str(target_video)
  ]
  ```
  The synthetic video generates a completely solid, static navy-blue color bar (`0x1a2b3c`) without dynamic text overlays or pixel variance (contrary to claims in `worker_m3_pipeline_1/handoff.md:48`).
- **Shot Gate Evaluation Failure**:
  When `render_single_shot` or `batch_render_scene` runs with `dry_run=True` and `enable_critic=True` (default), `OfflineHeuristicEngine.evaluate_shot` evaluates the video (`antigravity_critic_gate.py:446-455`):
  ```python
  if max_diff < 0.8:
      defects.append("Frozen video detected (virtually zero micro-motion across frames)")
  ```
  Because frames are identical (`max_diff = 0.0`), `is_frozen = True`.
  Line 535: `has_severe_defect = (is_frozen or ...)`.
  Line 539: `approved = (score >= 0.8) and (not has_severe_defect)`.
  Line 540: `action = "APPROVE" if approved else "RETAKE_SHOT"`.
  Verdict produced: `Score = 0.65`, `Approved = False`, `suggested_action = "RETAKE_SHOT"`.
  Execution log from test harness:
  ```text
  📊 Antigravity Shot Gate (test_scene99_shot01): Score = 0.65 | Approved = False
  🔄 [RETAKE 1/2] Shot test_scene99_shot01 không đạt chuẩn ...
  📊 Antigravity Shot Gate (test_scene99_shot01): Score = 0.65 | Approved = False
  🔄 [RETAKE 2/2] Shot test_scene99_shot01 không đạt chuẩn ...
  [!] Shot test_scene99_shot01 không vượt qua Shot Gate sau 2 lần retake (Score: 0.65)
  [!] Dừng chuỗi batch vì lỗi tại shot: test_scene99_shot01
  Batch render with enable_critic=True returned: False
  ```
  `batch_render_scene` in dry-run mode **always terminates at Shot 1** when `enable_critic=True`.
- **Behavior with `enable_critic=False`**:
  When `enable_critic=False`, `batch_render_scene` successfully renders consecutive shots, creates `_v1.mp4` 10s 720p files with AAC 48kHz audio, extracts `clean_frame_239.jpg` (720x1280) via OpenCV into `04_Assets/keyframes/<shot_id>/`, and propagates the tail frame to the next shot when classified as `CONTINUOUS_TAKE`.

### 1.3 Scene Gate Pre-Flight Validation in `concat_scene_shots`
- **Missing File Pre-Flight**:
  `production_orchestrator.py:917-920`: Correctly checks if each shot video file exists; returns `None` if missing.
- **Critical Axis Rejection**:
  `production_orchestrator.py:928-930`: If `scene_verdict.suggested_action == "RETAKE_SHOT"`, correctly aborts and returns `None`.
- **Gate Bypass for Unapproved Scores**:
  Line 928 only checks:
  ```python
  if not scene_verdict.approved and scene_verdict.suggested_action == "RETAKE_SHOT":
      print(f"[!] Scene Gate từ chối ghép Master cho {scene_id}: {scene_verdict.critique_notes}")
      return None
  ```
  When tested with `overall_score = 0.70` (`approved = False`) and `suggested_action = "APPLY_COLOR_MATCH"`, `concat_scene_shots` **does not abort**, does not apply color correction, and successfully concatenates and exports the unapproved master.

### 1.4 Character Leak Failure in `test_m3_challenger1_probe.py`
Running `pytest tests/test_m3_challenger1_probe.py -v` produced:
```text
FAILED tests/test_m3_challenger1_probe.py::TestAll140ShotsStartFrameResolution::test_probe_character_safeguards_canonical_character_shots
AssertionError: LEAK: Thúy Vân shot ep01_scene03_shot05 leaked Thúy Kiều: C:\Projects\KieuStory\04_Assets\characters\01_Main_Protagonists\thuy_kieu_maiden_16yo_720p.png
```
Inspection of `episodes/ep01/prompts/muse_prompts.json` for `ep01_scene03_shot05`:
- Scene title: `EP01 Cảnh 03 Shot 05: NỘI. KHUÊ PHÒNG THÚY KIỀU & THÚY VÂN - Thúy Vân đứng dậy gọi chị (Medium Shot Thúy Vân).`
- Motion prompt: `Thúy Vân đứng dậy gọi chị (Medium Shot Thúy Vân).. Thúy Vân quay người bước lại chiếc bàn tròn...`
- `character_asset_ref`: `"04_Assets/characters/01_Main_Protagonists/thuy_kieu_maiden_16yo_720p.png"` (Corrupted reference to Thúy Kiều).
- `po.resolve_start_frame("ep01_scene03_shot05")` returns `thuy_kieu_maiden_16yo_720p.png`.

### 1.5 Unawaited Coroutine in `antigravity_critic_gate.py`
Executing `evaluate_shot_gate` generated the following warning:
```text
C:\Projects\KieuStory\05_Production_Pipeline\antigravity_critic_gate.py:1003: RuntimeWarning: coroutine 'Agent.chat' was never awaited
  verdict = AntigravitySDKEngine.evaluate_shot(shot_id, video_path, expected_character, prompt)
```
In `AntigravitySDKEngine.evaluate_shot` (`antigravity_critic_gate.py:819`), `agent.chat(inputs)` is an `async` coroutine invoked synchronously without `await` or `asyncio.run()`, returning a coroutine object instead of a verdict and forcing immediate fallback.

### 1.6 Test Suite Execution Summary
- `pytest tests/test_production_pipeline_m3.py -v`: 16 passed in 4.11s.
- `pytest tests/test_m2_hygiene.py -v`: 6 passed in 0.16s.
- `pytest tests/test_critic_gate.py -v`: 30 passed in 10.87s.
- `pytest tests/test_m1_challenger2_probe.py -v`: 41 passed in 0.56s.
- `pytest tests/test_tier1_features.py -k "not test_render"`: 65 passed in 2.62s.
- `pytest tests/test_m3_challenger1_probe.py -v`: **1 FAILED**, 8 passed in 13.81s.

---

## 2. Logic Chain

1. **Audio Guard Logic (Observations 1.1)**:
   - Worker claimed: "Added pre-flight validation in render_single_shot to reject prompts lacking canonical audio guard before triggering render."
   - The actual implementation in `production_orchestrator.py:726` calls `ensure_audio_guard(prompt)` which appends the text if missing. It does not reject.
   - `validate_audio_guard` is unused dead code.
   - 4 prologue prompts lack canonical phrasing, and because `ensure_audio_guard` only checks `"Tuyệt đối KHÔNG sinh nhạc nền"`, it leaves them non-canonical at runtime.
   - Therefore, the claim of rejection enforcement is invalidated.

2. **Dry-Run Engine & Critic Gate Deadlock (Observation 1.2)**:
   - The synthetic generator generates a static flat navy-blue video (`color=c=0x1a2b3c`).
   - The offline critic gate includes a frozen-video detector (`max_diff < 0.8` across sampled frames).
   - Because a flat color bar has zero inter-frame difference, the video is classified as frozen.
   - The gate penalizes frozen video with `has_severe_defect = True` and `RETAKE_SHOT`.
   - In `batch_render_scene`, `render_single_shot` retries up to `max_retakes` and fails.
   - The dry-run engine cannot pass through its own integrated quality gate.

3. **Scene Gate Concat Hole (Observation 1.3)**:
   - Project specifications (`PROJECT.md`, `ORIGINAL_REQUEST.md §R1`) mandate that approved scenes must achieve `score >= 0.8`.
   - `concat_scene_shots` only blocks concatenation when `suggested_action == "RETAKE_SHOT"`.
   - When a scene scores 0.70 (< 0.8) with `suggested_action == "APPLY_COLOR_MATCH"` or `"TRIM_STATIC"`, `concat_scene_shots` concatenates the video anyway without applying the correction, exporting an unapproved master.

4. **Character Safeguard Regression (Observation 1.4)**:
   - `ep01_scene03_shot05` is a Thúy Vân shot with `character_asset_ref` pointing to Thúy Kiều.
   - Because `banana_prompts.json` has no anchor for this shot, `get_character_anchor` falls back to `character_asset_ref` and assigns `thuy_kieu_maiden`.
   - `resolve_start_frame` returns the Thúy Kiều portrait for Thúy Vân, violating the zero-leakage invariant and failing `test_m3_challenger1_probe.py`.

5. **Antigravity SDK Engine Defect (Observation 1.5)**:
   - Synchronous invocation of `agent.chat(inputs)` without an event loop causes an unawaited coroutine error, breaking primary SDK inference.

---

## 3. Caveats

- Tests requiring active Meta Muse cloud browser sessions (`agent-browser --session muse`) were not executed against live cloud endpoints to avoid unexpected account quota consumption; all testing was conducted using offline heuristic evaluation and local FFmpeg/OpenCV processing.
- The 67 non-Ep01 keyframe directories, 10 keyframe root files, 118 non-Ep01 videos, and 62 character files in `04_Assets/` remain intact and unmutated throughout testing.

---

## 4. Conclusion

**Verdict**: **REQUEST_CHANGES**

Milestone M3 cannot be approved in its current state due to the following critical deficiencies:
1. **Broken Dry-Run with Critic Gate**: `MUSE_DRY_RUN` batch render fails on Shot 1 under default `enable_critic=True` because the synthetic video is flagged as frozen.
2. **Audio Guard Non-Compliance & False Claim**: 4 prompts in Ep01 lack canonical Audio Guard text, and `production_orchestrator.py` does not reject unguard prompts as claimed.
3. **Scene Gate Concat Bypass**: Scenes with scores < 0.8 bypass concatenation gating if `suggested_action` is `APPLY_COLOR_MATCH` or `TRIM_STATIC`.
4. **Active Test Failure (Character Leak)**: `tests/test_m3_challenger1_probe.py` fails on `ep01_scene03_shot05` due to Thúy Vân referencing Thúy Kiều portrait.
5. **Corrupted Async Call in `AntigravitySDKEngine`**: `agent.chat` is called synchronously without awaiting, breaking the primary SDK layer.

### Required Actions for Worker:
1. **Fix `episodes/ep01/prompts/muse_prompts.json`**:
   - Correct `ep01_scene03_shot05` `character_asset_ref` to `"04_Assets/characters/01_Main_Protagonists/thuy_van_maiden_16yo_720p.png"`.
   - Update `prologue_shot_1..4` motion prompts to contain the full canonical Audio Guard text.
2. **Fix `05_Production_Pipeline/run_shot.py` Dry-Run Video Generator**:
   - Introduce subtle dynamic variation or motion in the FFmpeg synthetic generator (e.g. `testsrc=duration=10:size=1280x720:rate=24` or moving pattern) so that inter-frame differences satisfy `max_diff >= 0.8` and pass Shot Gate heuristic checks.
3. **Enforce Scene Gate in `concat_scene_shots`**:
   - Change line 928 from:
     ```python
     if not scene_verdict.approved and scene_verdict.suggested_action == "RETAKE_SHOT":
     ```
     to:
     ```python
     if not scene_verdict.approved:
     ```
     ensuring that no scene with `score < 0.8` is exported without passing the gate.
4. **Fix Async Coroutine in `AntigravitySDKEngine`**:
   - Use `asyncio.run(agent.chat(inputs))` or an existing running loop to properly await `agent.chat`.

---

## 5. Verification Method

To independently reproduce all observations and verify the required changes, execute:

1. **Verify Prompt Audio Guard Invariants**:
   ```powershell
   python -c "
   import json
   from pathlib import Path
   data = json.load(open('episodes/ep01/prompts/muse_prompts.json', encoding='utf-8'))
   guard = 'Quy tắc âm thanh: Tuyệt đối KHÔNG sinh nhạc nền (no music/BGM), không âm thanh điện tử, không tạp âm rè nhiễu. Chỉ sinh âm thanh môi trường tự nhiên (foley, ambience) và thoại nhân vật chân thực.'
   non_canonical = [sid for sid, d in data['motion_prompts'].items() if guard not in d.get('motion_prompt', '')]
   print(f'Non-canonical prompts ({len(non_canonical)}): {non_canonical}')
   "
   ```
   *Expected*: Should be 0 after fix (currently 4: `prologue_shot_1..4`).

2. **Verify Challenger 1 Test Suite (Character Safeguards)**:
   ```powershell
   pytest tests/test_m3_challenger1_probe.py -v
   ```
   *Expected*: 9 passed (currently 1 failed on `ep01_scene03_shot05`).

3. **Verify Dry-Run Batch Render with Critic Gate**:
   ```powershell
   python -c "
   import sys
   sys.path.insert(0, '05_Production_Pipeline')
   import production_orchestrator as po
   from pathlib import Path
   import shutil
   
   mock_shots = [('test_p_shot01', {'character_anchor': 'thuy_kieu', 'motion_prompt': 'Thuy Kieu walk'})]
   from unittest.mock import patch
   with patch('production_orchestrator.get_shots_for_scene', return_value=mock_shots), patch('production_orchestrator.get_all_shots', return_value=dict(mock_shots)):
       res = po.batch_render_scene('test_p', enable_critic=True, dry_run=True, auto_concat=False)
       print('Batch render with enable_critic=True returned:', res)
   for f in Path('04_Assets/videos').glob('test_p*'): f.unlink()
   shutil.rmtree(Path('04_Assets/keyframes/test_p_shot01'), ignore_errors=True)
   "
   ```
   *Expected*: Should return `True` (currently returns `False` due to frozen video detection).

4. **Verify Hygiene Suite Remains Green**:
   ```powershell
   pytest tests/test_m2_hygiene.py -v
   pytest tests/test_production_pipeline_m3.py -v
   ```
   *Expected*: All tests pass.