# TEST INFRASTRUCTURE SPECIFICATION (TEST_INFRA.md) # DSCons ERP Mem0 Long-Term Knowledge Base Overhaul & Verification Framework **Author**: `teamwork_preview_test_writer_e2e` (Specialist / QA) **Workspace**: `c:\Projects\DSCons` **Date**: 2026-10-05T20:46:00Z **Target Milestone**: E2E Testing Track — Automated Verification Test Suite **Authoritative Request**: `c:\Projects\DSCons\.agents\teamwork\ORIGINAL_REQUEST.md` (`## 2026-10-05T20:25:13Z`) **Project Blueprint**: `c:\Projects\DSCons\PROJECT.md` --- ## 1. Executive Summary & Verification Philosophy This document defines the comprehensive end-to-end (E2E) verification architecture for the overhaul of the Mem0 long-term memory engine backing the **DSCons ERP System** (Công ty TNHH Xây Dựng Định Sơn — MST `0202111150`). ### 1.1. Core Verification Directives 1. **Zero Synthetic Data / Strict Ground Truth**: - No mocks, fake memory fixtures, or artificial simulator shims. - All tests interact directly with the active Windows Background Service on `http://127.0.0.1:8765/sse` backed by Embedded Qdrant at `C:\Users\Admin\.gemini\antigravity\mem0_data\qdrant`. 2. **Opaque-Box Requirement-Driven Testing**: - Tests evaluate system behavior purely through observable semantic retrieval responses (`mem0_search`, `mem0_get_all`, `mem0_status`). - Internal storage mechanics are verified via outputs, ranking, cosine similarity scores, and metadata assertions. 3. **Negative Baseline Formulation (TDD Closed-Loop)**: - Prior to Milestone M1 (Selective Cleanup) and Milestone M2 (Memory Ingestion), the test suite MUST fail on contaminated queries (e.g. `"frontend theme"` retrieving FreeExile NPC HUD, `"DSCons rule"` retrieving obsolete logging duplicates). - Post M1 & M2 execution, the test suite must transition deterministically to 100% Passed. 4. **Zero-Wipe & Cross-Project Preservation**: - Tests assert that the 449 non-DSCons memory entries (FreeExile game design, VoLamWeb infrastructure rules, and global Antigravity directives) remain 100% intact. --- ## 2. Infrastructure Architecture & Service Topology ``` ┌─────────────────────────────────────────────────────────────────────────────┐ │ DSCONS ERP MEM0 SERVICE TOPOLOGY │ ├─────────────────────────────────────────────────────────────────────────────┤ │ │ │ FastMCP Background Service (port 8765) │ │ Endpoint: http://127.0.0.1:8765/sse │ │ ▲ │ │ │ JSON-RPC over Server-Sent Events (SSE) │ │ ▼ │ │ mcp.client.sse.sse_client / ClientSession │ │ ▲ │ │ │ Async queries: mem0_search(query, limit=5), mem0_status() │ │ ▼ │ │ Automated Verification Test Suite │ │ c:\Projects\DSCons\tests\test_mem0_verification.py │ │ - Standalone CLI Runner (exit code 0/1 + Rich Scorecard) │ │ - Pytest Parameterized Engine (pytest -v) │ │ │ └─────────────────────────────────────────────────────────────────────────────┘ ``` ### 2.1. Vector Embedder Mechanics (`PureVectorEmbedder`) The local embedding model operates as a 768-dimensional semantic hash engine: - Word-level SHA-256 hash: $+1.0$ weight per token. - Character 3-gram MD5 hash: $+0.5$ weight per character tri-gram (for fuzzy typo and morphological similarity). - Cosine normalization: $\vec{v} / \|\vec{v}\|_2$. **Test Implications**: - Queries must match both English technical terminology and Vietnamese domain phrasing. - Expected score threshold for verified domain memories is calibrated at $\mathbf{\ge 0.55}$ (typical top hits score $0.60 - 0.75$). --- ## 3. Verification Catalog: 8 Core Queries The test suite evaluates 8 distinct queries across all 5 knowledge pillars of DSCons ERP: | Query ID | Query String | Core Domain Pillar | Mandatory vs Domain | Expected Top Hit & Context | | :---: | :--- | :--- | :---: | :--- | | **Q1** | `"DSCons rule"` | Area 1: Root Standards & Core Laws | **Mandatory (R3)** | Fact 1.1 / 1.2: 7 Core Laws, Zero Synthetic Data, Hexagonal DDD, MST 0202111150 | | **Q2** | `"frontend theme"` | Area 3: Frontend UI/UX Standards | **Mandatory (R3)** | Fact 3.1 / 3.2: Dark Slate `#0b0f19`, Inter, JetBrains Mono, Bloomberg density | | **Q3** | `"backend architecture"` | Area 2: Backend & DB Architecture | **Mandatory (R3)** | Fact 2.1 / 2.2: 4-layer Clean Architecture, Ports & Adapters, NUMERIC(18,4) | | **Q4** | `"AEC pricing norms"` | Area 4: AEC Business & BoQ | **Domain Pillar** | Fact 4.2 / 4.4: Circular 38 Larsen pile $VL = 0$, MR/T ratio 40–65%, 4-tier markup | | **Q5** | `"CAD takeoff geometry"` | Area 4: AEC CAD Engineering | **Domain Pillar** | Fact 4.7: TCVN3 decoding, 1px stroke-width, clean Bounding Box, New vs Renovation | | **Q6** | `"database precision"` | Area 2: Database & Ledgers | **Domain Pillar** | Fact 2.2: NUMERIC(18,4), Python Decimal, Double-entry ledger, Pessimistic locking | | **Q7** | `"testing workflow"` | Area 5: Testing & Quality | **Domain Pillar** | Fact 5.1 / 5.2: Deep Matrix D1–D6 on `task.md`, Autonomous TDD $\ge 80\%$, pytest 100% | | **Q8** | `"MCP tools protocol"` | Area 1: Ecosystem & Tooling | **Domain Pillar** | Fact 1.4: Mandatory MCP matrix (`lsp-mcp`, `chrome-devtools-mcp`, `markitdown`) | --- ## 4. Quantitative Pass / Fail Criteria To pass the verification gate, all 5 validation gates must evaluate to `PASS`: ### Gate 1: Top-1 Similarity Score ($\ge 0.55$) - For every query $Q \in \{Q_1, \dots, Q_8\}$, the top-ranked result from `mem0_search(query=Q, limit=5)` must have: $$\text{Score}_{\text{Top-1}}(Q) \ge 0.55$$ ### Gate 2: 100% DSCons Domain Dominance - **Top-1 Strict Requirement**: The #1 hit for all 8 queries MUST belong strictly to the DSCons domain. - Identification: Text contains `[DSCons`, `DSCons ERP`, or metadata contains `project: "DSCons"`. - **Top-3 Dominance Requirement**: At least 2 of the top 3 hits ($\ge 66.7\%$) must belong to DSCons. - **Zero Foreign Precedence**: No foreign project record (e.g. FreeExile, VoLamWeb) may rank higher than the first DSCons record. ### Gate 3: 0% Contamination Gate (Strict Negative Assertions) Zero tolerance for obsolete remnants or cross-project leakage: 1. **Zero Legacy Log Duplicates**: - `app/core/logging_config` or UUIDs `bab3dbc0-2f3c-49ad-b416-a9f4f8077dbf`, `5dcc1be8-7216-47a3-8285-dc5b81ffe324` must NOT appear in any search results. 2. **Zero Legacy Contract Hallucinations**: - Outdated artificial numbers (e.g. `"cache 71 hạng mục"`, `"4.753 tỷ"`) must NOT appear. 3. **Zero FreeExile Contamination in Top-3**: - For all 8 DSCons queries, no FreeExile game terminology (`Feral NPC`, `Dialogue HUD`, `Savage Infinite Atlas`, `Combat UI`, `Cocos Creator`) may appear in the Top-3 hits. 4. **Zero Corrupted Metadata**: - No records with mismatched metadata (such as record `8d9178bc-0f7c-429f-b7ee-1af803df4c15` where project was labeled DSCons but content was FreeExile netcode). ### Gate 4: Complete 5-Pillar Knowledge Coverage When inspecting the memory catalog via `mem0_get_all` or aggregate search: - **Area 1 (Root Standards)**: $\ge 5$ facts verified. - **Area 2 (Backend & DB)**: $\ge 5$ facts verified. - **Area 3 (Frontend UI/UX)**: $\ge 5$ facts verified. - **Area 4 (AEC & BoQ Business)**: $\ge 7$ facts verified. - **Area 5 (Testing & Workflow)**: $\ge 5$ facts verified. - **Total Newly Ingested Invariants**: Exactly 27 atomic facts. ### Gate 5: Zero-Wipe Preservation Gate - The total memory count in Mem0 must equal: $$\text{Total Points} = 449 \text{ (Preserved non-DSCons)} + 27 \text{ (New DSCons Invariants)} = 476$$ - Non-DSCons entries (FreeExile: 339, VoLamWeb: 7, Developer Preferences: 2, Global Directives: 20, Headers: 81) must remain completely unmolested. --- ## 5. Test Suite Implementation Specification The automated test suite is housed at `c:\Projects\DSCons\tests\test_mem0_verification.py`. ### 5.1. Execution Modes 1. **Interactive CLI Runner**: ```bash python tests/test_mem0_verification.py ``` Outputs a colorized, structured ASCII scorecard table summarizing: - Query name and query string. - Top-1 score and score pass/fail badge. - Domain dominance status. - Contamination check status. - Overall verdict per query and final exit code (0 for PASS, 1 for FAIL). 2. **Automated Pytest Harness**: ```bash pytest tests/test_mem0_verification.py -v ``` Integrates into the CI/CD pipeline and regression test suites with 8 distinct test methods: - `test_q1_dscons_root_rules()` - `test_q2_frontend_theme()` - `test_q3_backend_architecture()` - `test_q4_aec_pricing_norms()` - `test_q5_cad_takeoff_geometry()` - `test_q6_database_precision()` - `test_q7_testing_workflow()` - `test_q8_mcp_tools_protocol()` Plus global assertions: - `test_zero_contamination_audit()` - `test_five_pillar_coverage_and_preservation()` --- ## 6. Scorecard Format Specification Upon execution, the runner outputs a structured scorecard in the following schema: ``` ======================================================================================== DSCONS ERP MEM0 VERIFICATION SCORECARD ======================================================================================== Query ID | Query String | Top-1 Score | Domain Dominance | Contamination | Verdict ----------+------------------------+-------------+------------------+---------------+-------- Q1 (Man) | DSCons rule | 0.6821 | 100% DSCons | CLEAN (0%) | PASS Q2 (Man) | frontend theme | 0.6145 | 100% DSCons | CLEAN (0%) | PASS Q3 (Man) | backend architecture | 0.6534 | 100% DSCons | CLEAN (0%) | PASS Q4 (Dom) | AEC pricing norms | 0.5982 | 100% DSCons | CLEAN (0%) | PASS Q5 (Dom) | CAD takeoff geometry | 0.6310 | 100% DSCons | CLEAN (0%) | PASS Q6 (Dom) | database precision | 0.6719 | 100% DSCons | CLEAN (0%) | PASS Q7 (Dom) | testing workflow | 0.6205 | 100% DSCons | CLEAN (0%) | PASS Q8 (Dom) | MCP tools protocol | 0.6418 | 100% DSCons | CLEAN (0%) | PASS ======================================================================================== OVERALL VERDICT: ALL 8 QUERIES PASSED (100% Pass Rate) | EXIT CODE: 0 ======================================================================================== ``` --- ## 7. Hand-off Verification Runbook To execute independent verification of the test suite: 1. Ensure the Mem0 FastMCP service is active on `http://127.0.0.1:8765/sse`. 2. Run `python tests/test_mem0_verification.py`. 3. Check exit code: `echo $LASTEXITCODE` (PowerShell) or `echo $?` (Bash). 4. Review stdout scorecard table.