Kj*1dZddlmZddlZddlZddlZddlZddlmZddl m Z e ej drej deejjjZed z d z d z d z Zed z d z ZdZdZd#dZd$dZ d%d&d Zd'd!Zed"kreedS)(uZ AutoPOE2 - Offset Registry Builder (TASK-RE-PIPELINE-2026 / Phase C4) Chuyển kết quả trích xuất PyGhidra (data/re_out//extraction.json) thành Offset Registry ở HAI định dạng: 1. bin/Release/offsets_.toml - build-keyed, C++ OffsetRegistry nạp trực tiếp (không cần SQLite dependency trong C++ Core). 2. data/poe2_offsets.db (SQLite WAL) - SSoT tầng Python với provenance đầy đủ. INVARIANT được thực thi: INV-BUILD-01 : tệp TOML gắn khóa build_time_date_stamp + build_section_layout_hash; C++ sẽ từ chối dùng nếu client đang chạy không khớp fingerprint. INV-SIG-UNIQUE : CHỈ signature có match_count == 1 được phép vào registry. INV-OFFSET-01 : không có bất kỳ địa chỉ heap tuyệt đối nào (registry chỉ chứa RVA/offset tĩnh suy từ binary trên đĩa). Rule 14 : mọi record kèm provenance (build_id, sha256, tool, thời điểm). Ngữ nghĩa cuối cùng (ô nào là InGameState / LifeComponent ...) là bước HUMAN-IN-THE-LOOP: tool đánh dấu target='unreviewed' cho tới khi được xác nhận bằng dữ liệu live (captures/*_RAM.json, memprobe_report.txt). Cách dùng: python tools/re_pipeline/build_registry.py --extraction data/re_out/6A9E477A/extraction.json ) annotationsN)Path)List reconfigureutf-8encodingdatare_out6A9E477Azextraction.jsonzpoe2_offsets.dbaw CREATE TABLE IF NOT EXISTS builds ( build_id TEXT PRIMARY KEY, time_date_stamp TEXT NOT NULL, size_of_image TEXT NOT NULL, file_size INTEGER NOT NULL, file_sha256 TEXT NOT NULL, section_layout_hash TEXT NOT NULL, binary_path TEXT NOT NULL, first_seen TEXT NOT NULL, last_verified TEXT ); CREATE TABLE IF NOT EXISTS signatures ( id INTEGER PRIMARY KEY AUTOINCREMENT, build_id TEXT NOT NULL REFERENCES builds(build_id), name TEXT NOT NULL, pattern TEXT NOT NULL, rip_offset INTEGER NOT NULL, match_count INTEGER NOT NULL, unique_ok INTEGER NOT NULL, pattern_bytes INTEGER, target_address TEXT NOT NULL, target_block TEXT NOT NULL, fan_in INTEGER, instruction_addr TEXT, mnemonic TEXT, cross_build_ok INTEGER DEFAULT 0, provenance TEXT NOT NULL ); CREATE TABLE IF NOT EXISTS offsets ( id INTEGER PRIMARY KEY AUTOINCREMENT, build_id TEXT NOT NULL REFERENCES builds(build_id), kind TEXT NOT NULL, target TEXT NOT NULL, payload TEXT NOT NULL, validator_id TEXT, source TEXT NOT NULL, discovered_at TEXT NOT NULL, last_verified TEXT, hit_count INTEGER DEFAULT 0, miss_count INTEGER DEFAULT 0 ); CREATE INDEX IF NOT EXISTS idx_offsets_build ON offsets(build_id, kind); CREATE INDEX IF NOT EXISTS idx_signatures_build ON signatures(build_id, unique_ok); ).dataz.rdata_RDATA address_strstr image_baseintreturnc(t|d|z S)u>Chuyển địa chỉ ảnh (VA) Ghidra trả về thành RVA.)r)rrs 8C:\Projects\AutoPOE2\tools\re_pipeline\build_registry.pyrva_from_addressrcs {B  * ,,recorddict size_of_imageboolct|dpd}|tvrdS tt|d|}n#tt t f$rYdSwxYwd|cxko|kncS)uOINV-BUILD-01: registry chỉ chứa slot .data/.rdata/_RDATA nằm trong image.blockFaddressr)rgetIN_IMAGE_DATA_BLOCKSrKeyError TypeError ValueError)rrrrrvas ris_in_image_data_slotr'hs  7##)r * *E (((us6)#455zBB i ,uu  # # # #m # # # ##s#AA0/A0extraction_pathrdb_path toml_path min_fan_incH tj|d}|ddkr3t d|dt jdS|d}|d }|d }tt|d d |d }|d} tt|dd fd|dD} | st dt d|j ddtj t|} | t| d| d||d|d|d|d|d|d||f | d|f| d|ft!| dD]8\} } | d |d!|d"| d#| d$| d%d&| d'd| d(| d)| d*| d+| d,| d-d&tj| |t|d.d/0f| d1|d2d3tj| d$| d%d&d4t%| d)d5d6d/0d7d8|dd&d&f :| | n#| wxYwd9d:|d;|dd<|dd=|d>| d?d@dAdB|ddC|ddAg }g}| D])} | d*dDkr|| d)*t!|ddEdD]/\} }|dF| dGt%|d50|r|dAt!| ddEdD]U\} } |dH| dI| d$|dJ| dI| d%d&V| r|dA|j dd|dK|dKzdt dL|t dM|t dN|dOt1| dPt1|t dQt dRt dSd&S)TNrrschemazautopoe2.re_extraction/v1u"[ERROR] Schema không hỗ trợ: file provenanceresultbuild_idrr generated_attoolrc g|]}|drh|ddkrO|dr:t|ddklt|}|S)unique match_countr0 signaturefan_inr)r!rr').0crr+rs r z"build_registry..s 55?? uu]33q88QUU;=O=O8 h"" # #z 1 1 !!Z ? ? 2 1 1 1r candidatesuL[WARN] Không có signature UNIQUE nào - registry rỗng phần tri thức.uM Hãy tăng --max-candidates/--pattern-window khi chạy re_extract.py.T)parentsexist_okzPRAGMA journal_mode=WAL;z8INSERT OR REPLACE INTO builds VALUES (?,?,?,?,?,?,?,?,?)time_date_stamp file_size file_sha256section_layout_hash binary_pathz)DELETE FROM signatures WHERE build_id = ?z&DELETE FROM offsets WHERE build_id = ?)startzINSERT INTO signatures (build_id,name,pattern,rip_offset,match_count,unique_ok,pattern_bytes,target_address,target_block,fan_in,instruction_addr,mnemonic,cross_build_ok,provenance) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?)sig__03dr9 rip_offsetrr8 pattern_bytesr rr:instruction_addressmnemonic)r5r4 extractionF) ensure_asciizINSERT INTO offsets (build_id,kind,target,payload,validator_id,source,discovered_at,last_verified,hit_count,miss_count) VALUES (?,?,?,?,?,?,?,?,?,?) aob_pattern unreviewed0xX)patternrJ target_rvaValidateInGameState static_rez?# AutoPOE2 Offset Registry - INV-BUILD-01 build-keyed knowledgez# build_id = z# binary_sha256 = z# section_layout_hash = z# generated_at = z# tool = zF# INV-SIG-UNIQUE: moi pattern duoi day chi khop DUNG 1 lan trong .textz:# INV-OFFSET-01: khong co dia chi heap tuyet doi nao o dayrzbuild_time_date_stamp = zbuild_section_layout_hash = r static_root_z = 0x aob_pattern_z = aob_rip_offset_ z[OK] SQLite registry : z[OK] TOML registry : z build_id=z unique_signatures=z static_roots=zB [HUMAN-IN-THE-LOOP] Ngu nghia cac slot dang la 'unreviewed' -zB can doi chuan live (memprobe_report.txt, captures/*_RAM.json)zD truoc khi gan ten InGameState/LifeComponent/... vao cot target.)jsonloads read_textr!printsysstderrrrparentmkdirsqlite3connect executescript SQL_SCHEMAexecute enumeratedumpsrcommitcloseappend write_textjoinlen)r(r)r*r+payloadr1r2r3r4r5r>connindexrlines static_rootsrrrs ` @@rbuild_registryrwvsj22G2DDEEG{{8 ;;; J7;;x3H3HJJQTQ[\\\\q&J X F*%HS -..33J>*L 6?DJ788"==M,'J _ \]]] ]^^^ N555 ?3w<< ( (DH :&&& /000 F,-?+;'=)01=)   @8+NNN ={KKK&z;;;0 0 ME6 LL7 1811e111;'JJ|Q//=)JJ//9%7OJJx((JJ455JJz**J!%|'*?';';==%*    4 LL/! J$*;$7'-zz,'B'B'_,@@HZ0E%FHH3\33+T++PD B:.?#@BBJz2G'HJJ  E!L33 '?g % %   y 1 2 2 2' SbS(9CCC__{ ]E]]0@j0Y0Y]]]^^^^ R":crc?!<<<PP v CECCf[.ACCDDD NuNNL!1L1LNNOOOO R 4$777 5))D07CCC -G - -... /I / /000  .8 . .J . .l++ . ./// NOOO NOOO PQQQ 1s G?M..NcFtjd}|dtt|dtt |ddd|d t d d |}t|j }| s;td |tj tdtj dStj|d}|dd}|jrt|jnt$dz dz d|dz }t'|t|j||jS)Nz+AutoPOE2 Offset Registry Builder (Phase C4)) descriptionz --extraction)defaultz--dbz--tomluITệp TOML xuất ra (mặc định: bin/Release/offsets_.toml))rzhelpz --min-fan-inruGLọc signature theo fan-in tối thiểu (0 = giữ tất cả unique))typerzr{u([ERROR] Không thấy tệp extraction: r.uE Hãy chạy: python tools/re_pipeline/re_extract.py trước.rrr1r3binReleaseoffsets_z.toml)argparseArgumentParser add_argumentrDEFAULT_EXTRACTION DEFAULT_DBr parse_argsrrNis_filer`rarbr]r^r_toml REPO_ROOTrwdbr+)parserargsr(rrr3r*s rmainr s  $AF 4F0G0GHHH J888 $ X S! V     D4?++O  " " $ $ JJJQTQ[\\\\ U\_\fggggqj22G2DDEEG|$Z0H#'9TYEI%(B8(B(B(BB /4==)T_ U UUr__main__)rrrrrr)rrrrrrrr)r) r(rr)rr*rr+rrr)rr)__doc__ __future__rrr]rerapathlibrtypingrhasattrstdoutr__file__resolvercrrrrhr"rr'rwr__name__ SystemExitrrrs2#"""""   73:}%%-JG,,, DNN " " $ $ + 2 9 '(2Z?BSS  "3 3 / l5---- $ $ $ $&'S S S S S lVVVV< z *TTVV  r