# Comprehensive Analysis: Asset Pipeline, Rendering Optimization & Technical R&D

**Surveyor**: `explorer_survey_pipeline`  
**Date**: 2026-10-01T22:12:00Z  
**Target Milestone**: 2026-10-01T22:03:34Z (Rendering Optimization & Full Asset Pipeline Expansion)

---

## 1. Executive Summary

This investigation surveys the current asset pipeline, rendering shaders, server network synchronization, and test infrastructure in `c:/Projects/FreeExile`.
- **Pipeline Tools**: `tools/asset_pipeline/` has 36 files. Core generators (`animation_pipeline.py`, `design_pipeline_manager.py`) are operational, but R&D modules (`lod_generator.py`, `texture_compressor.py`, `viewport_culling_utils.py`) do **not yet exist**.
- **Line Count Warning**: `animation_pipeline.py` currently has **488 lines** (soft cap: 350, hard cap: 500). Mipmap generation must be modularized into a dedicated helper to prevent violating the hard cap.
- **Metal Shaders**: `monster_palette_instancing.metal` exists in `client/webapp/assets/monsters/` and `SpritePBRNormal.metal` in `client/assets/shaders/`. GPU instancing is ready for 200+ instance draw call batching.
- **Server AoI & Delta Compression**: `SpatialGrid` (64m cells) is present, but animation state filtering, delta compression bitmasks, and HTTP ETag/304 caching are completely unimplemented.
- **DRQ Status**: 18 DRQs exist in `docs/design_requests/approved/`, 0 in `inbox/`. Requirement R1/AC1 requires >= 20 new DRQs across 6 departments.
- **Test Infrastructure**: `tests/unit` has 975 tests (100% operational, pytest 9.1.1 on Python 3.11.9).

---

## 2. Inventory of Existing Pipeline Tools (`tools/asset_pipeline/`)

| File | Lines | Bytes | Current Capabilities & Role |
| :--- | :---: | :---: | :--- |
| `animation_pipeline.py` | 488 | 17,481 | Bakes hero & monster sprite atlases, Sobel normal maps, kinematics manifests. Stride-speed sync, 8-dir heading. Missing: Mipmaps. |
| `design_pipeline_manager.py` | 498 | 20,742 | Manages DRQ lifecycle (Inbox -> In-Progress -> Pending Review -> QA + CEO -> Approved). Missing: Repo root in `sys.path` when run as script. |
| `design_pipeline_types.py` | 66 | 1,861 | Enums: `AssetCategory`, `Priority`, `RequestStatus`, `ApprovalVerdict`. Helper: `is_power_of_two(n)`. |
| `design_pipeline_cli.py` | 134 | 4,043 | CLI dispatcher commands (`scan`, `dispatch`, `submit`, `audit-qa`, `audit-ceo`, `report`). |
| `pbr_texture_synthesizer.py` | 239 | 9,251 | OpenCV-based PBR generator: Sobel tangent-space normal maps, roughness maps, emissive glow masks, MaxRects atlas packer. |
| `normal_map_utils.py` | 47 | 1,551 | Vectorized Sobel filter (`Image.Image` -> tangent-space normal map). |
| `clean_sprite_transparency.py` | 131 | 4,767 | OpenCV GrabCut alpha segmentation to strip white halo/milky artifacts from sprite renders. |
| `produce_savage_monster_assets.py` | 348 | 14,285 | Produces monster concept sheets, skeletons atlas, attachment overlays, and synchronizes assets. |
| `produce_grimdark_monsters_ai.py` | 274 | 10,471 | Catalog of 10 mobs + 2 bosses. Normal map generator. |

---

## 3. R&D Technical Deliverables Analysis & Design Specs

### 3.1. `tools/asset_pipeline/lod_generator.py` (Status: NOT PRESENT)
- **Objective**: Generate 3 Level of Detail (LOD) tiers for sprite atlases using Pillow/Lanczos resampling:
  - **LOD-0 (100%)**: Original full resolution (e.g. 1024x1024 or 1280x384).
  - **LOD-1 (50%)**: Half width and half height (e.g. 512x512).
  - **LOD-2 (25%)**: Quarter width and quarter height (e.g. 256x256).
- **Size Requirement**: LOD-2 file size must be `<= 40%` of original (empirically achieves 10-20% of original PNG size due to 1/16th pixel count).
- **Architecture & Implementation Plan**:
  ```python
  class LODGenerator:
      @staticmethod
      def generate_lods(
          src_path: Path,
          out_dir: Optional[Path] = None,
          scales: Tuple[float, ...] = (1.0, 0.5, 0.25)
      ) -> Dict[str, Path]: ...
  ```
  - CLI support: `--input`, `--output-dir`, `--batch`, `--verify-budget`.
  - Must write companion JSON metadata or update manifest with LOD paths and scale factors.

### 3.2. `tools/asset_pipeline/texture_compressor.py` (Status: NOT PRESENT)
- **Objective**: Batch compress PNG textures into WebP format for web client deployment.
- **Verification**: Pillow 12.3.0 is verified with `features.check('webp') == True`.
- **Target Compression**: Output WebP file size must be `<= 50%` of original PNG size.
- **Architecture & Implementation Plan**:
  - Supports lossy mode (`quality=85-90`) and lossless mode (`lossless=True`).
  - Automatically preserves alpha channel transparency (`RGBA`).
  - Batch directory recursive traversal with glob patterns.
  - Return compression stats: original bytes, compressed bytes, reduction percentage.

### 3.3. `tools/asset_pipeline/viewport_culling_utils.py` (Status: NOT PRESENT)
- **Objective**: Frustum AABB intersection calculator in 2.5D Isometric world space.
- **Mathematical Specification**:
  - Screen coordinate transform:
    $$X_{\text{screen}} = (W_x - W_y) \cdot \frac{T_w}{2} + \frac{V_w}{2}$$
    $$Y_{\text{screen}} = (W_x + W_y) \cdot \frac{T_h}{2} + \frac{V_h}{2} - W_z \cdot H_{\text{scale}}$$
  - Entity AABB: `[min_x, min_y, max_x, max_y]` in screen space.
  - Viewport bounds: `[0 - padding, 0 - padding, V_w + padding, V_h + padding]`.
  - Intersection test: `not (a.max_x < b.min_x or a.min_x > b.max_x or a.max_y < b.min_y or a.min_y > b.max_y)`.
- **Test Requirements**: Unit test suite `tests/unit/test_viewport_culling_utils.py` covering:
  - Entity entirely within viewport -> VISIBLE
  - Entity entirely outside viewport -> CULLED
  - Entity overlapping viewport border -> VISIBLE
  - Elevation offset ($W_z$) shifting screen bounds correctly
  - High-performance batch culling (1,000 entities in < 2ms)

### 3.4. Mipmap Chain Generation in `animation_pipeline.py` (Status: MISSING)
- **Constraint**: `animation_pipeline.py` currently has **488 lines**. Adding 50+ lines would breach the 500 line hard cap!
- **Recommended Architecture**:
  - Create `tools/asset_pipeline/mipmap_utils.py` (or integrate helper in `lod_generator.py`).
  - Function: `generate_mipmap_chain(image: Image.Image, min_dimension: int = 16) -> List[Image.Image]`.
  - In `animation_pipeline.py`, add a compact 10-line hook in `build_atlas_and_manifest`:
    ```python
    # Mipmap generation hook
    from tools.asset_pipeline.mipmap_utils import generate_and_save_mipmaps
    mip_paths = generate_and_save_mipmaps(atlas_path, output_dir)
    manifest_data["mipmaps"] = [p.name for p in mip_paths]
    ```

---

## 4. Metal Shaders & GPU Instancing Guidance

### 4.1. Existing Shaders
- `client/webapp/assets/monsters/monster_palette_instancing.metal` (56 lines):
  - Struct `MonsterInstanceData`: `modelMatrix` (float4x4), `elementalTint` (float4), `emissiveGlow` (float), `paletteLUTIndex` (uint).
  - Vertex shader: indexes instances via `instances[instanceID]`.
  - Fragment shader: color matrix palette shift and emissive lighting.
- `client/assets/shaders/SpritePBRNormal.metal` (90 lines):
  - Normal map tangent-space unpacking, Blinn-Phong specular, rim lighting, ghost alpha for i-frames.

### 4.2. Guidance for 200+ Instance Rendering in 1 Draw Call
To achieve 120 FPS ProMotion with 200+ monsters on screen:
1. **Shared Vertex & Index Buffers**: All monsters of a genus share a quad mesh (6 vertices or 4 vertices indexed).
2. **Dynamic Ring Buffer for Instances**: Allocate `MTLBuffer` for $N = 256$ instances ($256 \times 96\text{ bytes} \approx 24.5\text{ KB}$).
3. **Texture Atlas / Texture2DArray**: Bind single atlas texture containing all monster sprites.
4. **Single Draw Call**:
   ```objc
   [renderEncoder setRenderPipelineState:instancedPipelineState];
   [renderEncoder setVertexBuffer:instanceBuffer offset:0 atIndex:1];
   [renderEncoder setFragmentTexture:monsterAtlas atIndex:0];
   [renderEncoder drawPrimitives:MTLPrimitiveTypeTriangle vertexStart:0 vertexCount:6 instanceCount:activeMonsterCount];
   ```
5. **Draw Call Budget**: Reduces draw calls from 200 down to **1 draw call** for all monsters sharing the atlas.

---

## 5. Server AoI Filtering, Delta Compression & Asset Manifest Caching

### 5.1. Server AoI Animation State Filtering
- **Current State**: `server/world/spatial_grid.py` has 64m cell grid. `get_entities_in_aoi()` returns set of entity IDs in 9 adjacent cells.
- **Requirement**: Server must only broadcast animation state updates to clients whose observer entity resides within the same 9-cell neighborhood.
- **Specification**:
  - Server maintains `visible_entities_per_client: Dict[int, Set[int]]`.
  - When entity $E$ enters client $C$'s AoI: trigger `EntitySpawnPacket` with full state.
  - While entity $E$ is in $C$'s AoI: broadcast filtered animation updates.
  - When entity $E$ exits $C$'s AoI: trigger `EntityDespawnPacket`.

### 5.2. Delta Compression for Animation State
- **Problem**: 30Hz tick rate with 50 entities = 1,500 state packets/sec. Sending full packets (48 bytes) = 72 KB/sec per client.
- **Solution**: Bitmask delta encoding:
  - Header byte: `Bitmask (uint8)`
    - `Bit 0 (0x01)`: Position changed ($\Delta x, \Delta y$ as int16 millimeters)
    - `Bit 1 (0x02)`: Animation clip changed (clip_id uint8)
    - `Bit 2 (0x04)`: Heading angle changed (uint8: 0..255 for 360°)
    - `Bit 3 (0x08)`: Speed scale changed (uint8: percentage)
    - `Bit 4 (0x10)`: Combat status / i-frame changed (uint8 bitflags)
  - Result: Unchanged entities take **0 bytes** (omitted) or **1 byte** bitmask. Average tick payload drops to 8 bytes/entity (83% bandwidth reduction).

### 5.3. HTTP ETag Caching Layer for Asset Manifests
- **Problem**: Client fetches multiple manifests (`hero_anim_manifest.json`, `char_*_manifest.json`) on zone load.
- **Solution**:
  - Server generates strong ETag: `ETag: W/"<sha256(manifest_content)[:16]>"`
  - Headers: `Cache-Control: public, max-age=86400, must-revalidate`
  - On incoming `If-None-Match`: Compare hash -> Return `304 Not Modified` with empty body.
  - Reduces payload to 0 bytes and eliminates JSON reparsing.

---

## 6. Design Requests (DRQ) Inventory & Gaps

- **Current Approved**: 18 DRQs in `docs/design_requests/approved/`.
- **Current Inbox**: 0 DRQs in `docs/design_requests/inbox/`.
- **Requirement R1 / AC1**: At least **20 new DRQs** across 6 departments:
  1. Systems (`lead_systems_designer`): 4 DRQs (Passive Tree nodes, Crafting UI, Map Device, Boss Arena)
  2. Narrative (`quest_narrative_designer`): 4 DRQs (NPC avatars, dialogue portraits, Quest scrolls, cutscenes)
  3. Economy (`poe2_game_designer` / `economy`): 4 DRQs (Bloodstones, Bone Relics, Dust, Trade Bazaar)
  4. Security (`ciso_agent`): 3 DRQs (Security shield, alert badges, forensic telemetry)
  5. Client (`web_client_engineer` / `ios_client_engineer`): 3 DRQs (Loading screen, login background, parallax layers)
  6. Server (`server_engineer`): 2 DRQs (Agent Orb activation VFX, harvest report overlays)

---

## 7. Technical Report Requirements (`RENDERING_OPTIMIZATION_TECH_REPORT.md`)

- Target Path: `docs/research/RENDERING_OPTIMIZATION_TECH_REPORT.md`
- Target Length: >= 300 lines (adhering to GEMINI.md soft cap 400, hard cap 600).
- Must include:
  1. Executive Summary & Hardware Budget (iOS 120 FPS / 8.33ms target).
  2. Sprite LOD Resampling Benchmarks (LOD 100%, 50%, 25% file sizes & memory savings).
  3. WebP Batch Compression Benchmarks (PNG vs WebP byte comparisons).
  4. 2.5D Viewport Frustum Culling Benchmarks (draw call reductions).
  5. Apple Metal Instancing Architecture & Math (200+ instances in 1 draw call).
  6. Server AoI Animation State Filtering & Delta Compression Benchmarks (bandwidth reduction).
  7. HTTP Manifest ETag Caching Benchmarks (latency & transfer bytes).