jo\dZddlZddlmZmZGddZdddZd dd Zd dd Zd Z d Z dZ dZ dZ dZdZdZdZdZdZdZdZdZdZdZdZdZdZdZd Zd!Zd"Zd#Z d$Z!dd%d&d'Z"dS)(aF Copyright (C) 2023 Artifex Software, Inc. This file is part of PyMuPDF. PyMuPDF is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. PyMuPDF is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. You should have received a copy of the GNU Affero General Public License along with MuPDF. If not, see Alternative licensing terms are available from the licensor. For commercial licensing, see or contact Artifex Software, Inc., 39 Mesa Street, Suite 108A, San Francisco, CA 94129, USA, for further information. --------------------------------------------------------------------- PyMuPDF table cell-span resolution (opt-in extension). Provides SpanCell and resolve_spans (plus the _span_* helpers), which reconstruct a detected table's merged-cell (colspan / rowspan) structure. Re-exported by pymupdf.table; never runs on the default find_tables() path. Reuses the word-selection helpers of pymupdf._table_refine. N)_refine_is_vertical_or_rotated_refine_page_wordsceZdZdZddZdS)SpanCellaOne reconstructed table cell after span resolution (PyMuPDF extension). ``bbox`` is the placement's ``(x0, y0, x1, y1)`` union rect, ``text`` the page text it claims (lines joined by ``\n``), and ``colspan``/``rowspan`` how many grid columns/rows it covers. resolve_spans always sets a real bbox; a caller padding its own grid may construct SpanCells with ``bbox=None``. ``tag`` is the cell's HTML tag (``"td"``/``"th"``), defaulting to ``"td"``. resolve_spans leaves it at the default; find_tables(refine=True) overwrites it from the resolved header region so Table.to_html() can serialize the grid directly, and a caller building its own grid may set it too.tdcL||_||_||_||_||_dSN)bboxtextcolspanrowspantag)selfr r r r rs BC:\Projects\DSCons\.venv\Lib\site-packages\pymupdf\_table_spans.py__init__zSpanCell.__init__7s)    N)r)__name__ __module__ __qualname____doc__rrrrr*s4 D Drrg@) tolerancecg}t|D]6}|r||dz |kr|t|7|S)zGreedy 1-D clustering of edge coordinates into slot boundaries. Keeps a sorted value only when it is more than ``tolerance`` from the last kept boundary, so the first value of each run is the retained boundary.)sortedappendfloat)valuesr boundariesvalues r_span_clustered_boundariesr!@s_ J((  %*R.0I== %,,'''' r?cd}tt|dz D]4}||||dzzdz }||z |cxkr ||zkrn/|dz }5td|S)zKHow many [boundaries[i], boundaries[i+1]] slots the span start..end covers.r@)rangelenmax)startendrrcountindexmidpoints r_span_covered_slot_countr.Ms Es:*++u% 519(==D 9  ; ; ; ;C)O ; ; ; ; ; QJE q%==rc,g}tt|dz D]N}||||dzzdz }|j|z |cxkr|j|zkrn9||O|sdSt |t |dzfS)zIThe half-open column-slot range (first, last+1) a rect's x-extent covers.r$r%)rr$)r&r'x0x1rminr()rect x_boundariesrhitsr,r-s r_span_slot_ranger6Ws Ds<((1,-- ',uqy*AASH 7Y ( A A A Adg .A A A A A A KK    v IIs4yy1} %%rcJ|d|dko|d|dkSNrr$rabs r_span_ranges_intersectr<cs% Q4!A$; &1Q4!A$;&rct|j|cxkot|jknco3t|j|cxkot|jkncSr )rr0r1y0y1)xyr3s r_span_point_in_rectrBgsg >>Q 0 0 0 0%.. 0 0 0 0 ZU47^^q5Z5Z5Z5ZERVRYNN5Z5Z5Z5ZZrc tjtd|Dtd|Dtd|Dtd|DS)Nc3$K|] }|jV dSr )r0.0r3s r z#_span_rect_union..m$ & &DG & & & & & &rc3$K|] }|jV dSr r>rEs rrGz#_span_rect_union..nrHrc3$K|] }|jV dSr )r1rEs rrGz#_span_rect_union..orHrc3$K|] }|jV dSr r?rEs rrGz#_span_rect_union..prHr)pymupdfRectr2r()rectss r_span_rect_unionrQks~ < & & & & &&& & & & & &&& & & & & &&& & & & & &&&   rclt|j|jt|j|jz Sr )r2r1r(r0r9s r_span_x_overlaprSts% qtQT??Sqt__ ,,rcnt||}|dkrdStd|j}||z dkS)aTrue if a text span overlaps a cell by enough x to signal a merge. A few points of overlap is noise (a span drifting across a column line with trailing whitespace or a currency glyph); require >2pt and >=15% of the cell width before treating it as a merged-cell signal.r%Fr"g333333?)rSr(width) span_rect cell_rectoverlap cell_widths r_span_substantial_x_overlaprZxsB i33G#~~uS)/**J Z 4 ''rc|sgSg}|dx}}|ddD])}||dzkr|}|||f|x}}*|||f|S)z>Collapse a sorted index list into (start, end) inclusive runs.rr$N)r)indicesrangesr)r*r,s r_span_contiguous_rangesr^s  F!*EC C!G  C  ucl### MM5#, Mrcg}t|dD]a\}}}||kr |r||ddkr||||gf;|d\}}}|t||||gzf|d<b|S)zIMerge overlapping (start, end, text) intervals, accumulating their texts.c"|d|dfSr8ritems rz'_span_merge_intervals..sQa?Qrkeyrr$)rrr() intervalsmergedr)r*r prev_startprev_endtextss r_span_merge_intervalsrks F"92Q2QRRRFFsD C<<  A.. MM5#v. / / / &,Rj# He #h"4"4etfnEr Mrcld|dgD}d|S)z=Space-joined non-empty span texts of a get_text('dict') line.cg|]n}t|dpd8t|dpdoS)r )strgetstriprFspans r z#_span_line_text..sx     txx%2 & & , , . .  DHHV   "##))++   rspans )rpjoin)linepartss r_span_line_textrzs@  HHWb))   E 88E??rc|d}|rtj|}|js|Sd|dgD}d|D}|sdSt |S)zKBounding rect of a dict line (its bbox, else the union of its span bboxes).r cg|]@}|dtj|dpgAS)r )rprNrOrrs rrtz#_span_line_rect..sU     88F    TXXf%%+,,   rruc g|] }|j | Sr)is_emptyrEs rrtz#_span_line_rect..s 9 9 9d4= 9T 9 9 9rN)rprNrOr~rQ)rxr r3rPs r_span_line_rectrs 88F  D |D!!} K  HHWb))   E : 9e 9 9 9E t E " ""rct|dd}||Sg}|ddgD]~}|ddvr|dgD]M}t|st |}|s$t |}|6|||fN t|d|n#t$rYnwxYw|S)zVertical/rotated text lines as (rect, text), cached on the page. Selects non-horizontal lines whose reading order get_text('dict') already preserves._span_vertical_lines_cacheNdictblockstypeNrlines) getattrget_textrprrzrrsetattr Exception)pagecachedrblockrxr r3s r_span_vertical_text_linesrs& T7 > >F  Ev&&**8R88 ' ' 99V  I - - IIgr** ' 'D1$77 "4((D "4((D| LL$ & & & & ' 2E::::      LsC C C c |sdSg}t|D]\}}t|jt|jzdz t|jt|jzdz t |r|||f|sdSd}|D]<\}\}} } } }|| zdz | | zdz t fd|Dr|dz }=|tdt|z dkrdStd|DdkrdSd d t|d DS) a"Text of vertical lines centered in rect, when they dominate the rect. Returns the stacked line text only if vertical lines are centered in the rect, cover >=60% of the rect's selected words, and carry >=2 tokens; else None so the caller falls back to horizontal line synthesis.N?rc3@K|]\}}t|VdSr )rB)rF line_rect_cxcys rrGz/_span_vertical_text_for_rect..s4UU,)Q"2r955UUUUUUrr$g333333?c3^K|](\}}t|V)dSr )r'splitrFrr s rrGz/_span_vertical_text_for_rect..s6 7 7D3tzz||   7 7 7 7 7 7r c3 K|] \}}|V dSr rrs rrGz/_span_vertical_text_for_rect..s: At rct|djt|dj|djzdz fS)Nrr)rr0r>r?ras rrcz._span_vertical_text_for_rect..s;eDGJ//%ad1gj9PTW8W2X2X1XYrrd)rrr0r1r>r?rBranyr(r'sumrwr)rr3selected_words candidatesrr vertical_hitsrwx0wy0wx1wy1rrs @@r_span_vertical_text_for_rectrs tJ4T::11 4IL!!E),$7$773 >IL!!E),$7$773 > r2t , , 1   y$/ 0 0 0 tM&4" "Cc3Ci3 Ci3  UUUUU*UUU U U  Q Ms1c.11222S88t 7 7J 7 7 777!;;t 99 YY     rc |sdSd|D}t|t|dz}td|dz}g}t|dD]\}}}}||zdz } d} |} |D]1} t| t | d z } | | kr| } | } 2| || ||fgd e| d ||ft| d }t | d |d z z| z|z | d <g}t|d D]J} |ddt| d dDKd|S)a Join center-point-selected cell words into text, re-synthesizing lines. ``words`` are (y0, x0, y1, text) tuples; they are grouped into lines by an adaptive median-height nearest-line rule, each line's words ordered by x, and the lines joined by newlines.rnc>g|]\}}}}td||z S)g?r()rFr>rr?s rrtz,_span_words_to_line_text..s.;;;\RBs3R  ;;;rrr%g?c"|d|dfSr8rras rrcz*_span_words_to_line_text.. sDGT!W;MrrdNcenter_y)rwordsrr$c,t|dS)Nr)rras rrcz*_span_words_to_line_text..suT*5E/F/Frrvc3 K|] \}}|V dSr rrs rrGz+_span_words_to_line_text..s&"g"gGAt4"g"g"g"g"g"grc|dSrrras rrcz*_span_words_to_line_text..s ^bcd^err)rr'r(absrrrw)rheights median_heightline_thresholdrr>r0r?r r best_line best_distancerxdistancer+ text_liness r_span_words_to_line_textrs r;;U;;;G7OOCLLA$56Mmd233N E"5.M.MNNNZZBD2g_ &  ) )D2d:&6 7 7788H=(( (   LLbRJ<@@ A A A '!!2t*---Ig&''!&y'_x1r?r s r_span_word_line_tupler!s9 BCT "IIuRyy%))SYY 77rcg}t|D]g\}}|\}}}}} t| s/||zdz} ||zdz} t| | |r|||fh|S)z(index, word) pairs whose center lies in rect, index into ``page_words``. The index is what lets resolve_spans claim each page word for exactly one placement (an earlier cell's word is not re-claimed by a later one).r) enumeraterorqrBr) page_wordsr3selectedr,rrrrrr rrs r_span_select_words_in_rectr's H ,,++ t#' S#sD4yy    Ci3 Ci3  r2t , , + OOUDM * * * Orc^t|||}||Std|DS)zIText for a rect: vertical-line text if it dominates, else line synthesis.Nc2g|]\}}t|Sr)r)rFrrs rrtz-_span_words_text_for_rect..=s%$_$_$_WQ%:4%@%@$_$_$_r)rr)rr3r vertical_texts r_span_words_text_for_rectr8s=0t^LLM  #$_$_P^$_$_$_ ` ``rcfdt||D}|D]\}}|t|||S)zKText of rect's words, skipping words already claimed and claiming the rest.c&g|] \}}|v ||fSrr)rFr,r claimed_wordss rrtz,_span_claim_text_in_rect..Bs6 E4  % %  % % %r)raddr)rr3rrrr,rs ` r_span_claim_text_in_rectr@ss5j$GGH !!q%    $T4 : ::rc`t|dd}||Sg}|ddgD]}|dgD]}|dgD]}t|dpd}t |d krL|d }|sdt j|}|jr| ||f t|d|n#t$rYnwxYw|S) zPage text spans as (rect, text), length>=2, cached on the page. These drive merged-cell detection: a single span whose x-extent crosses a grid column line signals cells the line grid split but text joins._span_text_spans_cacheNrrrrur rnrr ) rrrprorqr'rNrOr~rrr) rrrurrxrsr r r3s r_span_text_spansrLsb T3T : :F  Ev&&**8R88 + +IIgr** + +D"-- + +488F++1r2288::t99q==xx''|D))= dD\**** + + .6666      Ls D D+*D+c t|dkrgStd|D}td|D}g}|D]\ } j jzdz }||dz ks ||dzkr* fdt |D}t|dkrYt |D]#\}} | |kr||| |f$t|S)aNColumn ranges within one grid row that a single text span crosses. ``entries`` are the row's cell rects (in column order). A text span centered in the row band and substantially overlapping >=2 adjacent cells marks those cells for merging; overlapping intervals are merged. Returns (start, end, texts) with end>start.rc3$K|] }|jV dSr rJrFentrys rrGz+_span_crossing_intervals..r$//e//////rc3$K|] }|jV dSr rMrs rrGz+_span_crossing_intervals..srrr%r"c:g|]\}}t||Sr)rZ)rFpositionrrVs rrtz,_span_crossing_intervals..ys=   %*9e<<    r) r'r2r(r>r?rr^rrk) entries text_spansrow_y0row_y1rf span_textr hit_positionsr)r*rVs @r_span_crossing_intervalsris< 7||a //w/// / /F //w/// / /FI * : : 9L9</36 fsl " "h#&=&=     #,W#5#5   }   ! ! 1-@@ : :JE3U{{  %i!8999 : ! + ++rc||krdS|D]?}|ddvr |d|ks |d|kr&t||drdS@dS)NFrole>header header_leaf header_grouprowcolsTr<)row_idxrbase body_startothers r_span_is_leaf_header_rangers{*u = I I I  <7 " "eElj&@&@  !$f 6 6 55  4rc g}t|D]{\}}td|DD]\\}}tj|}t||} ||||| | d| dz ||krdndd]||D]4} | ddkrt | d| d ||rd nd | d<5|S) a Describe every grid cell by row/slot-range/role for the strict-colspan gate. Header cells split into ``header_leaf`` (no header cell below within body) vs ``header_group`` (spans over a lower leaf), which the reject rules compare against the body split.cg|]}||Sr rrFcells rrtz*_span_build_base_cells..s(R(R(R$AQAQAQAQrr$rbodyr)rrr3rr rrrrrr)rrNrOr6rr) cellsr4rrrrrrr3rrbs r_span_build_base_cellsrs D!%((   '(R(R#(R(R(RSS  NHd<%%D#D,77D KK" (  #Awa0&-&;&;FF         <8 # # *$u+tF|T:VV MM V Krc^tfd|D}|D]fd|D}|st|dkr|ddkr:tfd|D}tfd|D}|dkr|dkrdSd S) zDTrue if some body row splits the column range ``cols`` into >1 cell.cPh|]"}|ddk|dk|d#S)rrrr)rFrbrs r z._span_body_rows_split_under..s=mmmV 8N8NSWX]S^blSlSlU SlSlSlrcpg|]2}|ddk|dkt|d0|3S)rrrrr)rFrbrrs rrtz/_span_body_rows_split_under..sX   F|v%%$u+*@*@E[\`bfgmbnEoEo*@ *@*@*@rr$rrc3\K|]&}td|ddV'dSrrNrrFrbrs rrGz._span_body_rows_split_under..s9KKdCQfa99KKKKKKrc3\K|]&}td|ddV'dSr$rNr2rs rrGz._span_body_rows_split_under..s9IID#d1gtF|A77IIIIIIrTF)rr'r2r()rrr body_rowsr5 covered_start covered_endrs` ` @r_span_body_rows_split_underrsmmmmmmmnnI              t99>>d1gfo55 KKKKdKKKKK IIIIDIIIII DG # # tAw(>(>44 5rcfd|D}|sdSt|dkr|ddkrdStfd|D}tfd|D}|dko |dkS)zGTrue if leaf-header cells split the column range ``cols`` into >1 cell.cXg|]&}|ddkt|d$|'S)rrrrrs rrtz1_span_header_leafs_split_over..s? r r rTT&\]%B%BG]^bdhiodpGqGq%BD%B%B%BrFr$rrc3\K|]&}td|ddV'dSrrrs rrGz0_span_header_leafs_split_over..s9GG$DGT&\!_55GGGGGGrc3\K|]&}td|ddV'dSrrrs rrGz0_span_header_leafs_split_over..s9EEc$q'4<?33EEEEEEr)r'r2r()rrr5rrs` r_span_header_leafs_split_overrs r r r rT r r rD u 4yyA~~$q'&/T11uGGGG$GGGGGMEEEEEEEEEK DG # > tAw(>>rc|d|dz dkrdS||kr)t||||sdSt|||rdSdSt||rdSdS)zJWhether to reject a candidate merge over ``cols`` as a (reason,) mismatch.r$r)Frn)T.header_leaf_colspan_changed_against_body_split)T.body_colspan_changed_against_header_leaf_split)rrrrrrrs r#_span_reject_colspan_mismatch_mergers AwaAy)'4zJJ 9 &tT: > > JIIy$T400FEE 9rcg}|||dzD]6}t||}|t|||7|S)Nr$)rrr)rrr)r*rrjrrs r_span_cell_texts_for_entriesr s[ Eq)DD*:u== .tUEBBCCCC Lrcl|dkrdSdt|||||D}t| S)zAllow an otherwise-rejected header merge when a covered part is empty. A header leaf that would be rejected against a body split is still merged if one of the merged parts has no text (an empty header slot the body fills).rFc6g|]}|Sr)rq)rFr s rrtzD_span_allow_header_colspan_merge_with_empty_part..s ooo4$**,,ooor)r all)reasonrrr)r*r part_textss r0_span_allow_header_colspan_merge_with_empty_partrsK AAAuoo+GgW\^acm+n+noooJ: rF)header_row_countstrict_colspanc Xt|}g}g}|D]`}|D][}|tj|} || j| jg|| j| jg\at|} t|} |r-tdtt|pd|nd} |rt|| | ng} t|}t|}t}g}t!|D]\}}g}d|D}t#||}d|D}d}|t|kr5||}|||\}}t'|||dz} |rUt)| | }t+||| | \}}|rt-||||||rd}|r||} |dz }n|dz}n|dz}n ||} |dz }| jrt1| j| j| }t1| j| j| }|t5t7| t9|| |||| |t|k5|||S) aResolve a detected table's merged-cell (colspan/rowspan) structure. ``cells`` is a row-major grid: a list of rows, each a list of ``[x0, y0, x1, y1]`` cell rectangles (``None`` for a gap) -- the same grid shape :func:`refine_grid` accepts. Returns a row-major grid of :class:`SpanCell` placements, ragged where cells span: each placement carries its union ``bbox``, the page ``text`` it claims, and its ``colspan``/``rowspan`` (how many clustered column/row slots it covers). Every page word is claimed by at most one placement. ``header_row_count`` is the number of leading header rows (default a conservative 1); it only matters together with ``strict_colspan``. ``strict_colspan`` (default False), when set, refuses a merge whose colspan would contradict the header/body column split. This is a PyMuPDF extension; it reads page text/graphics but does not mutate the page. Nr$rc:g|]}|tj|Sr )rNrOrs rrtz!resolve_spans..!s'JJJ$9I7<%%9I9I9Irc i|] \}}}|||f Srr)rFr)r*rjs r z!resolve_spans..#s%UUU6GeS%ec5\UUUrr)rrrr)r*rF)r r r r )r'rNrOextendr0r1r>r?r!r(r2intrrrsetrrrprQr6rrr~r.rrtupler) rrrrrowsx_edgesy_edgesrrr3r4 y_boundariesr base_cellsrrr placementsr placement_rowrrfintervals_by_startrinterval end_position_crossing_textscandidate_colsrejectrr r s r resolve_spansr(s;" u::DGG// / /D|<%%D NNDGTW- . . . NNDGTW- . . . .  /.g66L-g66LBFMQC 0 5A66==>>>AJLZb'|ZHHH`bJ!$''J#D))JEEMJ!%((3)3)  JJ#JJJ,WjAA UU9UUUW%%)--h77H#08- o'r*syB ,58     CF7: & & & & &'''[[[--- ( ( (      ###$:   F!!!B888 "aaa ; ; ;:,,,B   @( ? ? ?   48[[[[[[[r