Hj,-dZddlmZddlZddlmcmZddl Z ddl Z ddl Z ddl m Z ddlmZddlmZddlmZmZddlZddlmZmZdd lmZmZdd lmZdd lm Z dd l!m"Z"dd l#m$Z$edZ%Gdde j&Z'Gdde j&Z(Gdde j&Z)e*dkre j+dSdS)zComprehensive test suite for Multimodal OCR Engine, Image Preprocessing, and its integration into Document Management and Banking UNC workflows. ) annotationsN)date)Decimal)Path) MagicMockpatch)Image ImageDraw)FILE_MAGIC_SIGNATURESAgentSecurityGuard)UncClassifierAndRouter)DocumentExtractorMixin)ImagePreprocessor)MultimodalOcrServicezTc:\Projects\DSCons\Finance\UNC-2026\Techcombank-2026\DebitNote26.06.2026_Minh An.pdfc:eZdZdZd dZd dZd dZd dZd dZd S) TestImagePreprocessorzKUnit tests for Pure Python / Pillow / Numpy image preprocessing algorithms.returnNonecHtjddd}tj|}t dddD] }|d|fd |fgd d !|d dd}tj|dd}| t|d dtj |dd\}}| |tj| |jd| |jddS)z8Test text skew angle estimation and automated deskewing.RGB)iXi,rrcolor(i2i&)rrr)fillwidthg@T)expand fillcolorg$@g?) max_angle angle_stepg?)deltarN)r newr Drawrangelinerotaterestimate_skew_angleassertAlmostEqualabsdeskewassertIsInstance assertGreaterr height)selfimgdrawyskewedangledeskewed rotated_bys 6C:\Projects\DSCons\tests\test_multimodal_ocr_engine.pytest_estimate_skew_and_deskewz3TestImagePreprocessor.test_estimate_skew_and_deskew's/izAAA~c""r3## D DA IIAwa) I C C C CCHH"5fY\]]] s5zz3c::: 17$[^___* h 444 8>1--- 8?A.....ctjddd}tj|}|dddt j|}d |d d d d f<tj|}tj |}tj |}| |j |j d S) z3Test auto-contrast enhancement and noise reduction.r)r>)r?r?r)rPzCONG TRINH 2026)drArArrr<N) r r&r r'textnparray fromarrayrenhance_contrastdenoise assertEqualsize)r2r3r4arr noisy_imgenhanceddenoiseds r:!test_enhance_contrast_and_denoisez7TestImagePreprocessor.test_enhance_contrast_and_denoise<sizAAA~c"" (-O DDDhsmmBrE2b5LOC(( $5i@@$,X66 11111r<cttjddd}tj|}|gddt j|}tj|}ttj |}| | dd hd S) zCTest Otsu global binarization produces pure black and white pixels.r)rArA)r>r>r>r)rRr@r@)rrrrBrrN) r r&r r' rectangleradaptive_binarizerErFsetunique assertTrueissubset)r2r3r4bin_imgbin_arr unique_valss r:test_adaptive_binarizez,TestImagePreprocessor.test_adaptive_binarizeKsizAAA~c"" '''l;;;#5c::(7##")G,,--   ,,aX6677777r<cts|dt}t j|dd}|t|d||dj d||dj ddS)z8Test PyMuPDF rasterization of real AEC / Bank PDF pages.,Real Debit Note file not found on filesystem) max_pagesdpiriN) REAL_DEBIT_NOTE_PATHexistsskipTest read_bytesrrender_pdf_to_imagesrJlenr0r r1)r2 pdf_bytesimagess r:test_render_pdf_to_imagesz/TestImagePreprocessor.test_render_pdf_to_imagesWs#**,, J MMH I I I(3355 "7 QTWXXX Va((( 6!9?C000 6!9+S11111r<ctjddd}tj|d}||d|t|dd S) z5Test image serialization to standard base64 data URI.r)rrrrJPEGformatzdata:image/jpeg;base64,rN)r r&rimage_to_data_urirW startswithr0rh)r2r3data_uris r:test_image_to_data_uriz,TestImagePreprocessor.test_image_to_data_uribspix???$6s6JJJ ++,EFFGGG 3x=="-----r<Nrr) __name__ __module__ __qualname____doc__r;rPr\rkrsr<r:rr$s~UU////* 2 2 2 2 8 8 8 8 2 2 2 2......r<rc2eZdZdZd dZd dZd dZd dZdS) TestMultimodalOcrServicez9Unit and functional tests for the Multimodal OCR Service.rrc,t|_dS)N)rservice)r2s r:setUpzTestMultimodalOcrService.setUpms+-- r<cts|dt}|j|dd}||dd||dd||dd ||d d ||d d ||dd||dd||dd||dd||dd||dd|d|dddS)zcVerify 100% extraction accuracy on actual Techcombank Debit Note (Core Law 1: Zero Synthetic Data).r^zDebitNote26.06.2026_Minh An.pdfpdf bank_code TECHCOMBANKref_noFT26177032980455amountg HAamount_in_wordsuJBa trăm chín mươi triệu sáu trăm bốn mươi bốn nghìn đồngsender_account07898888 sender_namezVND-TGTT-CT TNHH XD DINH SONreceiver_account 1058177339 receiver_namezCONG TY CO PHAN MINH AN BG 688 receiver_bankzNGOAI THUONG VN (VCB)txn_datez 2026-06-26 directionoutflowz0CTY TNHH XD DINH SON TRA TIEN CHO MINH AN BG 688 descriptionN) rcrdrerfr}extract_banking_unc_datarJgetassertIn)r2ridatas r:(test_extract_banking_unc_real_debit_notezATestMultimodalOcrService.test_extract_banking_unc_real_debit_noteps#**,, J MMH I I I(3355 |44  -    +.. >>> (++-?@@@ (++[999 "3447C D D D "233Z@@@ -002PQQQ "455|DDD /224TUUU /224KLLL *--|<<< +.. ::: H$((S`bdJeJefffffr<cd}|j|ddd|}||dd|d|d d ||d d ||d d||dddS)zHTest heuristic metadata extractor for construction procurement dossiers.uTRUNG TÂM DỊCH VỤ SỰ NGHIỆP CÔNG XÃ KIẾN MINH Số: IB2600471890 THÔNG BÁO MỜI THẦU Về việc: Gói thầu số 05: Xây dựng cống hộp đoạn từ nhà ông Đèn đến nhà bà Lanh Ngày ban hành: 22/08/2026 Đã ký số bởi Giám đốc trung tâmzutf-8zTBMT_CongHop_OngDen.txttxt) file_contentfilename file_format text_content document_code IB2600471890uGói thầu số 05document_titlerdocument_grouplegal document_typeTBsignature_statusfully_executedN)r}extract_document_metadataencoderJrr)r2 sample_textmetas r:(test_extract_document_metadata_heuristiczATestMultimodalOcrService.test_extract_document_metadata_heuristics : |55$++G44.$ 6   /22NCCC +TXX6F-K-KLLL "233W=== /22D999 "4557GHHHHHr<cd}|tj|dd}|tj|d|tj|dd}|tj|dd}d }|tj|d |tj|d d S) zQEnsure webp and tiff image formats pass AgentSecurityGuard Zero-Trust validation.sPNG  IHDRpngs JFIFjpgjpegsRIFF$WEBPVP8webpsII*sMM*tifftifN)rWr validate_magic_bytes)r2 png_bytes jpg_bytes webp_bytestiff_letiff_bes r:$test_security_magic_bytes_for_imagesz=TestMultimodalOcrService.test_security_magic_bytes_for_imagess;  *? 5QQRRR4  *? 5QQRRR *? 6RRSSS4  *? FSSTTT-, *?PPQQQ *?OOPPPPPr<Nrt)rurvrwrxr~rrrryr<r:r{r{jsxCC....gggg2IIII.QQQQQQr<r{c*eZdZdZddZddZddZdS) )TestOcrIntegrationWithBankingAndDocumentszOIntegration tests verifying OCR hooks in banking router and document extractor.rrctts|dt}t jd|\}}}||||d|| ddddd S) aTest UncClassifierAndRouter.detect_unc_and_bank detects Techcombank from content bytes. Note: When the classifier already identifies the bank via keyword matching (is_unc=True AND bank_code is set), it correctly skips the expensive OCR deep inspection path. In that case, ocr_matched will be False because OCR was unnecessary, not because it failed. r^zDocument_Scan_Receipt_01.pdf)rcontentr confidencerg?z;Detection confidence should be >= 0.8 for automated routingN) rcrdrerfr detect_unc_and_bankrWrJassertGreaterEqualr)r2riis_uncrdetailss r:"test_unc_classifier_deep_ocr_matchzLTestOcrIntegrationWithBankingAndDocuments.test_unc_classifier_deep_ocr_matchs$**,, J MMH I I I(3355 %;%O3& & & " 7  M222  KK a ( (# I     r<czt}tjddd}tj}||d|}td}|j dd | | | t|d }| |t|r|d Sd S#|r|wwxYw) zQTest DocumentExtractorMixin handles images gracefully using MultimodalOcrService.r)rArrrPNGrnz(storage/secure_vault/test_ocr_sample.pngT)parentsexist_okrN)rr r&ioBytesIOsavegetvaluerparentmkdir write_bytes extract_textstrr/rdunlink)r2 extractorr3img_bufpng_data temp_img_pathextracted_texts r:test_extractor_mixin_with_imagezITestOcrIntegrationWithBankingAndDocuments.test_extractor_mixin_with_images6*,, iy@@@*,, '''##%%GHH ""4$"???!!(+++ '&33C 4F4FNNN  ! !.# 6 6 6##%% '$$&&&&& ' '}##%% '$$&&&& 's %>D+D:cts|dt}t }|t td|}||dd| |dddt|j d dpt|j d d}|5}| 5}| d dddn #1swxYwY|ddddS#1swxYwYdS) z^Test ingest_statement_file parses real Techcombank Debit Note via OCR and inserts transaction.r^N) file_pathrrrr total_parsedrr_get_connection_open_connectionzNDELETE FROM erp_bank_transactions WHERE reference_number = 'FT26177032980455';)rcrdrerfr ingest_statement_filerrJrrgetattr_clientcursorexecutecommit)r2rirouterresget_connconncurs r:/test_ingest_statement_file_with_real_debit_notezYTestOcrIntegrationWithBankingAndDocuments.test_ingest_statement_file_with_real_debit_notes#**,, J MMH I I I(3355 '))**.//"+   --}===  : :A>>>6>+;TBBwgfn^prvFwFw XZZ 4 # d                KKMMM                   s6 E,E4 E,E E,E E,,E03E0Nrt)rurvrwrxrrrryr<r:rrsVYY    8'''',r<r__main__),rx __future__rbuiltins @py_builtins_pytest.assertion.rewrite assertionrewrite @pytest_arrosunittestdatetimerdecimalrpathlibr unittest.mockrrnumpyrEPILr r 3app.modules.agents.application.agent_security_guardr r 9app.modules.banking.application.unc_classifier_and_routerr :app.modules.core.application.document_processing.extractorr3app.modules.core.application.ocr.image_preprocessorr7app.modules.core.application.ocr.multimodal_ocr_servicerrcTestCaserr{rrumainryr<r:rs:#""""" ******** RQQQQQXXXXXXt[ C.C.C.C.C.H-C.C.C.LIQIQIQIQIQx0IQIQIQXNNNNN0ANNNb zHMOOOOOr<