# TEST_INFRA — Comprehensive Test Infrastructure Specification
## Project: Thập Ngũ Niên (The Fifteen Springs) — AI Cinema Production System

> **Document Status**: Active  
> **Testing Track**: Parallel E2E Testing Track  
> **Scope**: Features F1 through F13 across Test Tiers 1 to 4  
> **Authoritative Specifications**: `ORIGINAL_REQUEST.md`, `PROJECT.md`, `AGENTS.md`

---

## 1. Test Philosophy

### 1.1 Opaque-Box & Requirement-Driven
The testing harness treats all subsystems as opaque components evaluated strictly against explicit functional requirements and acceptance criteria, rather than implementation artifacts:
- **Specification Supremacy**: Every test assertion originates directly from `ORIGINAL_REQUEST.md` (R1–R7) and `PROJECT.md` (F1–F13).
- **Test Integrity**: Zero facade tests. Tests never pass unconditionally or mock away core verification logic. Tests exercise real file parsers, audio processing algorithms, FFmpeg stream integrity, JSON schema validation, and web endpoint routing.
- **Independence & Isolation**: Each test case creates and destroys any temporary files within isolated directories (`tempfile.TemporaryDirectory`) without mutating project assets or depending on execution order.
- **Progressive Testability**: The test suite can run at any phase of milestone development. Tests distinguish between:
  1. *Verified & Passing*: Implemented components meeting exact requirements.
  2. *Implementation Gaps (Pending/Failed)*: Requirements awaiting worker completion or fixing (e.g. EP01 expansion from 48 to 188 shots, complete purge of male crying in line 195, web studio `/api/shots` and `/api/scenes` routes).

---

## 2. Feature Inventory & Coverage Mapping

The test suite covers all 13 features identified in `PROJECT.md` across four rigorous tiers:

| Feature ID | Feature Name | Source Requirement | Tier 1 (Coverage) | Tier 2 (Boundary) | Tier 3 (Interaction) | Tier 4 (Workload) |
|---|---|---|---|---|---|---|
| **F1** | EP01 Screenplay & Pacing Expansion | ORIGINAL_REQUEST §R1 | 5 tests (>=150 shots, >=1500s, 3 Acts) | 5 tests (empty scenes, max limits, resilience) | F1+F2, F1+F3 | Workload 01, 06 |
| **F2** | Đạm Tiên Grave 3-Beat Sequence | ORIGINAL_REQUEST §R2 | 5 tests (C05-A, C05-B, C05-C beats) | 5 tests (premature tears, inversion, missing) | F1+F2, F2+F3 | Workload 07 |
| **F3** | Anti-Male-Tears Script Purge | ORIGINAL_REQUEST §R4 | 5 tests (0% male crying, stoicism) | 5 tests (female crying allowed, casing, punctuation) | F1+F3, F2+F3, F4+F3 | Workload 06 |
| **F4** | Universal Acting Directives in Bible | ORIGINAL_REQUEST §R4 | 5 tests (codified rules in Project Bible) | 5 tests (missing section, word count, character coverage) | F4+F3, F4+F6 | Workload 10 |
| **F5** | Sweet-Spot Start Frame Protocol | ORIGINAL_REQUEST §R3 | 5 tests (720p 1280x720, contextual, zero reverse) | 5 tests (invalid res, missing context, aspect ratio) | F5+F7, F5+F12 | Workload 02, 10 |
| **F6** | Lip-Sync & Off-Screen Dialogue Guard | ORIGINAL_REQUEST §R5 | 5 tests (mandatory closed lips on off-screen) | 5 tests (on-screen allowed, empty dialogue, whitespace) | F4+F6, F6+F7 | Workload 02 |
| **F7** | 188-Shot AI Prompt Registry | ORIGINAL_REQUEST §R3, §R4, §R5 | 5 tests (JSON registries, audio guard) | 5 tests (whitespace in keys, duplicate IDs, syntax) | F5+F7, F6+F7, F7+F11, F7+F12 | Workload 02 |
| **F8** | Audio Continuity & Crossfade Concat | ORIGINAL_REQUEST §R6 | 5 tests (acrossfade 1.0s qsin, RMS inspect) | 5 tests (empty input, single file, nonexistent, delta) | F8+F9, F8+F10 | Workload 04 |
| **F9** | Continuous Stem 2 Festival Ambience | ORIGINAL_REQUEST §R6 | 5 tests (crowd bed asset, 48kHz stereo, 4-stem) | 5 tests (missing fallback, ducking, duration mismatch) | F8+F9, F9+F10 | Workload 03, 04 |
| **F10** | EBU R128 -14 LUFS Normalization | ORIGINAL_REQUEST §R6 | 5 tests (-14 LUFS ±0.5, -1.0 dBTP, LRA 9-11) | 5 tests (extreme quiet, extreme loud, silence, clipping) | F8+F10, F9+F10 | Workload 03 |
| **F11** | Web Review Studio Port 1515 Sync | ORIGINAL_REQUEST §R7 | 5 tests (FastAPI app, /api/shots, /api/scenes) | 5 tests (409 conflict, traversal 404, cache expire) | F7+F11, F11+F12 | Workload 05 |
| **F12** | Automated Production Runner & Concat | ORIGINAL_REQUEST §R7 | 5 tests (CLI flags, start frame resolve, scene order) | 5 tests (invalid shot ID, fallback, code 2 on error) | F7+F12, F11+F12, F12+F13 | Workload 08 |
| **F13** | Master Verification & Assembly Validation | Acceptance Criteria | 5 tests (assemble script, no opencv concat, AAC 48k) | 5 tests (0-byte detect, resolution mismatch, duration) | F12+F13 | Workload 09 |

---

## 3. Test Architecture & Directory Layout

The automated test suite is co-located in `c:\Projects\KieuStory\tests/` using standard Python `unittest` and `pytest` compatibility:

```
c:\Projects\KieuStory\tests/
├── __init__.py                    # Test package marker & shared test utilities
├── test_tier1_features.py         # Tier 1: 65 Feature Coverage Tests (F1-F13, 5 per feature)
├── test_tier2_boundaries.py       # Tier 2: 65 Boundary & Corner Cases (F1-F13, 5 per feature)
├── test_tier3_interactions.py     # Tier 3: 15 Cross-Feature Interaction Tests
├── test_tier4_workloads.py        # Tier 4: 10 Real-World Application Workload Tests
└── run_all_tests.py               # Master Test Runner & Coverage Aggregator CLI
```

### 3.1 Tier Structure
- **Tier 1 (Feature Coverage)**: Validates primary requirement adherence for every feature F1 through F13. Verifies affirmative specifications directly derived from the project bible and request.
- **Tier 2 (Boundary & Corner Cases)**: Pushes each feature against edge conditions: empty strings, malformed schemas, path traversals, extreme decibel levels, silence, clipping, whitespace padding, and negative parameters.
- **Tier 3 (Cross-Feature Interactions)**: Tests pairwise integration across architectural boundaries (Screenplay ↔ Prompts ↔ Audio Engine ↔ Production Orchestrator ↔ Web Studio).
- **Tier 4 (Real-World Application Workloads)**: Simulates end-to-end operational scenarios: parsing full screenplays, testing multi-tone audio through EBU R128 loudness normalization, walking complete Web Studio API surface, and checking feature master assemblies.

---

## 4. Test Runner Instructions

### 4.1 Running All Tests via Master Runner
```powershell
python tests\run_all_tests.py
```
Options:
- `--tier {1,2,3,4}`: Run a specific test tier only.
- `--feature {F1..F13}`: Filter tests targeting a specific feature ID.
- `--verbose`: Enable detailed test execution output.

### 4.2 Running via Pytest
```powershell
pytest tests\ -v
```

### 4.3 Running via Python Standard Library Unittest
```powershell
python -m unittest discover -s tests -p "test_*.py" -v
```

---

## 5. Defect Escalation Protocol

When tests fail due to implementation gaps in milestone deliverables:
1. **Never mutate implementation code**: Test writers operate strictly in read-only mode regarding production source files.
2. **Document exact failure signature**: Capture failing assertion, file path, line number, expected value vs actual value.
3. **Escalate to implementing agent / Milestone Owner**: Include the exact failing test name and reproduction command.