# HANDOFF REPORT: TEXTURE PACKING, RENDERING & ASSET PROCESSING PIPELINE

**Agent**: `explorer_survey_3` (Explorer Subagent)  
**Parent Orchestrator**: `orchestrator_22` (`34037784-62e1-41f8-bfe6-912696fdec14`)  
**Date**: 2026-10-04  
**Working Directory**: `c:\Projects\FreeExile\.agents\teamwork\explorer_survey_3`  
**Associated Analysis**: [`analysis.md`](file:///c:/Projects/FreeExile/.agents/teamwork/explorer_survey_3/analysis.md)

---

## 1. Observation

### 1.1. Existing Asset Pipeline Tools & Non-PoT Dimensions
- `tools/asset_pipeline/animation_pipeline.py`:
  - Lines 320-324:
    ```python
    total_frames_count = sum(cfg["frames"] for cfg in anim_configs.values())
    cols = min(8, total_frames_count)
    rows = math.ceil(total_frames_count / cols)
    atlas_w = cols * fw
    atlas_h = rows * fh
    atlas_img = Image.new("RGBA", (atlas_w, atlas_h), (0, 0, 0, 0))
    ```
  - For heroes (`fw=160, fh=192`), `atlas_w = 8 * 160 = 1280`, `atlas_h = 5 * 192 = 960`.
  - Resulting atlas `hero_anim_atlas.png` is **$1280 \times 960$**, which violates the Power-of-Two mandate ($1024 \times 1024$ or $2048 \times 2048$).
- `tools/asset_pipeline/texture_compressor.py`:
  - Implements WebP compression via Pillow (`img.save(dst, format="WEBP", quality=80, method=6)`).
  - Lines 1-162: No support for ASTC 4x4 or `.astc` format output.
- `tools/asset_pipeline/lod_generator.py`:
  - Lines 20-24: Implements 3 LOD tiers (100%, 50%, 25%) via Lanczos resampling.
  - Test `test_lod2_size_budget` passes with $\text{LOD-2} \le 40\%$.
- `tools/asset_pipeline/pbr_texture_synthesizer.py`:
  - Lines 131-188: Implements `SpriteAtlasPacker` using First-Fit Decreasing Height (FFDH) shelf bin packing into a fixed power-of-two canvas (`max_atlas_size = 2048`), computing normalized UV coordinates `[u0, v0, u1, v1]`.

### 1.2. Normal Map Synthesis & Verbatim Channel Inversion Failure
- `tools/asset_pipeline/produce_all_missing_skills_vfx.py`:
  - Lines 151-155:
    ```python
    norm_x = (dx / mag) * 0.5 + 0.5
    norm_y = (dy / mag) * 0.5 + 0.5
    norm_z = (dz / mag) * 0.5 + 0.5
    normal_bgr = np.stack([(norm_x * 255).astype(np.uint8), (norm_y * 255).astype(np.uint8), (norm_z * 255).astype(np.uint8), alpha], axis=-1)
    return normal_bgr
    ```
  - Saved using `cv2.imwrite(str(t / "savage_primal_skills_vfx_atlas_normal.png"), normal)`.
- In OpenCV `cv2.imwrite`, channel 0 is Blue and channel 2 is Red. Here, channel 0 received `norm_x * 255` instead of `norm_z * 255`.
- **Verbatim Test Failure** when running `pytest tests/e2e/test_asset_campaign_and_pipeline_e2e.py -v`:
  ```
  FAILED tests/e2e/test_asset_campaign_and_pipeline_e2e.py::TestTier4RealWorldScenarios::test_t4_client_assets_normal_map_blue_channel
  AssertionError: Normal map savage_primal_skills_vfx_atlas_normal.png B-mean 127.0 <= 128
  assert 126.97726917266846 > 128.0
  ```
- Contrast with `tools/asset_pipeline/pbr_texture_synthesizer.py` line 59:
  ```python
  # Pack into BGR format for OpenCV (B=Z, G=Y, R=X)
  normal_bgr = np.dstack((nz * 255.0, ny * 255.0, nx * 255.0)).astype(np.uint8)
  ```
  And `tools/asset_pipeline/m4_normal_utils.py` lines 36-46:
  ```python
  r = ((nx * 0.5 + 0.5) * 255).astype(np.uint8)
  g = ((ny * 0.5 + 0.5) * 255).astype(np.uint8)
  b = ((nz * 0.5 + 0.5) * 255).astype(np.uint8)
  r[alpha == 0] = 128
  g[alpha == 0] = 128
  b[alpha == 0] = 255
  Image.merge("RGBA", (Image.fromarray(r), Image.fromarray(g), Image.fromarray(b), Image.fromarray(alpha)))
  ```

### 1.3. Dilation Padding & Benchmark Observations
- Bilinear interpolation between opaque foreground $(R, G, B, 1.0)$ and transparent black $(0, 0, 0, 0)$ darkens RGB on edges before alpha blending occurs.
- Shaders `client/cocos/assets/resources/shaders/sprite_pbr.effect` and `client/assets/shaders/SpritePBRNormal.metal`:
  - `SpritePBRNormal.metal` line 78-80 evaluates rim lighting:
    `float NdotV = max(dot(normal, V), 0.0); float rimFactor = pow(1.0 - NdotV, uniforms.rimPower);`
    Because $V = (0, 0, 1)$, $N \cdot V = N_z$. If edge normal or color bleeds toward black $(0, 0, 0)$, rim lighting and diffuse lighting multiply dark values, resulting in severe black halos.
- Empirical test on $1024 \times 1024$ RGBA canvas:
  - `cv2.inpaint` (Telea): **2,901.8 ms**
  - Iterative morphological dilation (`cv2.dilate` on 3 color channels, 8 iterations): **90.8 ms**
  - Normalized box-filter dilation (`cv2.filter2D` on RGB and mask count, 8 iterations): **540.9 ms**

### 1.4. ASTC 4x4 Requirements
- Apple Metal supports `MTLPixelFormatASTC_4x4_sRGB` and `MTLPixelFormatASTC_4x4_LDR`.
- Footprint is 8.0 bpp (16 bytes per $4 \times 4$ block), providing 75% memory savings compared to uncompressed RGBA8888.
- Canonical 16-byte container header:
  `[0x13, 0xAB, 0xA1, 0x5C]` (magic) + `[4, 4, 1]` (block sizes) + 3 bytes width + 3 bytes height + 3 bytes depth.

---

## 2. Logic Chain

1. **Premise**: In 2.5D ARPGs on iOS Metal and WebGL, hardware mipmapping and ASTC compression require Power-of-Two textures ($1024 \times 1024$ or $2048 \times 2048$).
   - *Supported by Observation 1.1*: `hero_anim_atlas.png` is currently $1280 \times 960$, causing non-integer mipmap divisions and preventing hardware texture batching.
2. **Premise**: In tangent-space normal mapping, the $Z$ component $N_z$ points outward from the surface toward the camera ($N_z > 0$).
   - *Supported by Observation 1.2*: Converting $N_z \in (0, 1]$ via $(N_z \times 0.5 + 0.5) \times 255$ guarantees Blue $\in [128, 255]$, yielding a Blue channel mean $> 128.0$.
3. **Premise**: The test failure on `savage_primal_skills_vfx_atlas_normal.png` (B-mean 127.0 $\le$ 128) is caused by OpenCV's BGR channel order.
   - *Supported by Observation 1.2*: In `produce_all_missing_skills_vfx.py:154`, stacking `norm_x` in channel 0 caused OpenCV's `cv2.imwrite` to save $N_x$ into Blue and $N_z$ into Red. Fixing channel order to `(nz, ny, nx, alpha)` for `cv2.imwrite` or `(nx, ny, nz, alpha)` for PIL solves this completely.
4. **Premise**: Bilinear filtering and mipmaps interpolate color values into transparent areas. If transparent pixels have $RGB=(0,0,0)$, interpolated edge texels become dark, creating black borders on dark backgrounds.
   - *Supported by Observation 1.3*: Extending the edge colors of opaque pixels into transparent pixels for 4–8 pixels while leaving Alpha=0 (Dilation Padding) ensures interpolated color remains pure foreground color, completely eliminating dark fringes.
5. **Premise**: 8-iteration morphological dilation (`cv2.dilate`) executes in ~90ms per $1024 \times 1024$ image (over 30x faster than inpaint), making it ideal for the automated asset pipeline.
   - *Supported by Observation 1.3*: Benchmark showed 90.8 ms vs 2,901.8 ms.
6. **Premise**: ASTC 4x4 requires power-of-two dimensions and 16-byte headers.
   - *Supported by Observation 1.4*: Writing the canonical header and integrating with `astcenc` allows zero-copy hardware texture uploads in Metal iOS.

---

## 3. Caveats

- `astcenc` executable is not installed globally on the current Windows machine. A Python-based fallback generator writing canonical 16-byte `.astc` headers and exporting WebP companion files should be used when building on headless environments lacking the ARM CLI.
- Character class animations currently only contain single-direction frames ($1280 \times 960$ with 33 frames). Porting to full 8-directional motion matching requires baking $8 \times 8 = 64$ frames per action state into $2048 \times 2048$ atlases.
- No other caveats.

---

## 4. Conclusion

1. **Texture Atlas Dimensions**: All skill and VFX atlases must be exported at $1024 \times 1024$ PoT, and character/monster atlases at $2048 \times 2048$ PoT. The current $1280 \times 960$ dimensions in `animation_pipeline.py` must be upgraded to $2048 \times 2048$ padded canvases with updated JSON manifests.
2. **Channel Fix for Normal Maps**: In all OpenCV-based normal map generators, array channels passed to `cv2.imwrite` must be ordered as `(nz * 255, ny * 255, nx * 255, alpha)`. Transparent pixels ($A=0$) must be clamped to flat tangent normal `(128, 128, 255)`. This will immediately resolve the failure in `test_t4_client_assets_normal_map_blue_channel`.
3. **Dilation Padding Implementation**: Implement `apply_dilation_padding()` using the 8-iteration `cv2.dilate` method in `animation_pipeline.py`, `produce_all_missing_skills_vfx.py`, and `savage_vfx_factory.py` immediately prior to saving diffuse textures and computing normal maps.
4. **ASTC 4x4 Support**: Create `tools/asset_pipeline/astc_compressor.py` supporting both `astcenc` execution and direct 16-byte `.astc` binary container generation.
5. **Directory Structure**: Adopt the unified hierarchy under `client/cocos/assets/resources/vfx/skills/`, `monsters/archetypes/`, and `characters/` with matching UV JSON descriptors.

---

## 5. Verification Method

To independently verify all findings and validate fixes:

1. **Run Unit Tests for Asset Pipeline**:
   ```powershell
   pytest tests/unit/test_asset_pipeline_tools.py -v
   ```
   *Expected*: 9 passed.

2. **Run E2E Asset Campaign Tests**:
   ```powershell
   pytest tests/e2e/test_asset_campaign_and_pipeline_e2e.py -v
   ```
   *Observe*: `test_t4_client_assets_normal_map_blue_channel` fails on `savage_primal_skills_vfx_atlas_normal.png` with B-mean 127.0 $\le$ 128, validating our diagnosis.

3. **Verify Blue Channel Mean with Python One-Liner**:
   ```powershell
   python -c "from PIL import Image, ImageStat; stat = ImageStat.Stat(Image.open('client/webapp/assets/vfx/savage_primal_skills_vfx_atlas_normal.png')); print('Blue Mean:', stat.mean[2])"
   ```
   *Result*: Displays `126.98` (failing). After applying `(nz, ny, nx, alpha)` channel stacking, Blue Mean will be $> 230.0$.

4. **Verify Dilation Padding Performance & Accuracy**:
   ```powershell
   python -c "import cv2, numpy as np; img = np.zeros((1024, 1024, 4), dtype=np.uint8); img[200:800, 200:800, :3] = [200, 150, 100]; img[200:800, 200:800, 3] = 255; mask = (img[:, :, 3] > 0).astype(np.uint8); k = cv2.getStructuringElement(cv2.MORPH_RECT, (3, 3)); rgb = img[:, :, :3].copy(); [exec('dm = cv2.dilate(mask, k); f = (dm == 1) & (mask == 0); [rgb[:,:,c].__setitem__(f, cv2.dilate(rgb[:,:,c], k)[f]) for c in range(3)]; mask = dm') for _ in range(8)]; print('Dilated fringe pixel:', rgb[195, 205], 'Original alpha preserved:', img[195, 205, 3])"
   ```
   *Expected*: `Dilated fringe pixel: [200 150 100] Original alpha preserved: 0`.