+j&ddlmZddlmZddlmZddlZddlZddl m Z ddlm Z ddl m Z mZdd lmZmZmZmZmZejeZdZdZd[dZd\dZd\dZd\dZd\dZd\dZd\dZd\dZ d\dZ!dZ"dZ# d]d^d#Z$d_d(Z%d`d,Z&dadbd1Z'dcd3Z(ddd7Z)ded:Z*dfd;Z+dgd=Z, dhdidCZ-djdDZ.dkdGZ/ dhdidHZ0 dldidJZ1 dldidKZ2djdLZ3dmdNZ4dndRZ5dodTZ6dpdWZ7GdXdYeZ8dS)q) annotations)Callable)EnumN)pairwise_distances)Tensor)is_torchdynamo_compilinglogging)_convert_to_batch_tensor_convert_to_float_tensor_convert_to_tensornormalize_embeddings to_scipy_cooarbreturntuple[Tensor, Tensor]c|jr|js||fS|jr|js||fS||fS)aW Converts the dense tensor to sparse COO if exactly one of the two inputs is sparse, so that mixed inputs can reuse the all-sparse computations. The lone dense operand in such a mix typically still holds mostly zeros (e.g. sparse embeddings encoded with ``convert_to_sparse_tensor=False``), making the conversion cheap, whereas densifying the sparse operand would allocate a second full-size dense tensor. Args: a (Tensor): The first tensor. b (Tensor): The second tensor. Returns: tuple[Tensor, Tensor]: The two tensors, using the same layout. ) is_sparse to_sparserrs SC:\Projects\DSCons\.venv\Lib\site-packages\sentence_transformers\util\similarity.py_match_layoutsrsY { 1; !++--{ 1; {{}}a a4Kc"t||S) Computes the cosine similarity between two tensors. Args: a (Union[list, np.ndarray, Tensor]): The first tensor. b (Union[list, np.ndarray, Tensor]): The second tensor. Returns: Tensor: Matrix with res[i][j] = cos_sim(a[i], b[j]) )cos_simrs rpytorch_cos_simr-s 1a==rlist | np.ndarray | Tensorct|}t|}t|}t|}tj||ddS)rrr )r rtorchmm transposeto_denserra_normb_norms rrr;sf !##A ##A !! $ $F !! $ $F 8FF,,Q22 3 3 < < > >>rcdt|}t|}|js|jrIt|}t|}||zdSt t|t|S)a Computes the pairwise cosine similarity cos_sim(a[i], b[i]). Args: a (Union[list, np.ndarray, Tensor]): The first tensor. b (Union[list, np.ndarray, Tensor]): The second tensor. Returns: Tensor: Vector with res[i] = cos_sim(a[i], b[i]) dim)r rrsumr$pairwise_dot_scorer%s rpairwise_cos_simr.Ns !##A ##A {_ak_%a((%a(($$$,,55777!"6q"9"9;OPQ;R;RSS\\^^^rct|}t|}tj||ddS)a Computes the dot-product dot_prod(a[i], b[j]) for all i and j. Args: a (Union[list, np.ndarray, Tensor]): The first tensor. b (Union[list, np.ndarray, Tensor]): The second tensor. Returns: Tensor: Matrix with res[i][j] = dot_prod(a[i], b[j]) rr )r r!r"r#r$rs r dot_scorer0esJ !##A ##A 8Aq{{1a(( ) ) 2 2 4 44rct|}t|}||zdS)a Computes the pairwise dot-product dot_prod(a[i], b[i]). Args: a (Union[list, np.ndarray, Tensor]): The first tensor. b (Union[list, np.ndarray, Tensor]): The second tensor. Returns: Tensor: Vector with res[i] = dot_prod(a[i], b[i]) r)r*)r r,r$rs rr-r-vsB !##A ##A E;;2;   ' ' ) ))rc t|}t|}|js|jrtdt ||\}}t |}t |}t ||d}tj|  |j  Stj ||d S)a Computes the manhattan similarity (i.e., negative distance) between two tensors. Handles sparse tensors without converting to dense when possible. Args: a (Union[list, np.ndarray, Tensor]): The first tensor. b (Union[list, np.ndarray, Tensor]): The second tensor. Returns: Tensor: Matrix with res[i][j] = -manhattan_distance(a[i], b[j]) z8Using scipy for sparse Manhattan similarity computation. manhattan)metricg?p)r rlogger warning_oncerrrr! from_numpyfloattodevicer$cdist)rra_coob_coodists r manhattan_simrAs !##A ##A{ 4ak 4VWWWa##1QQ!%{CCC&&,,..11!(;;DDFFF AqC(((113333rct|}t|}t||\}}tjtj||z d S)a< Computes the manhattan similarity (i.e., negative distance) between pairs of tensors. Args: a (Union[list, np.ndarray, Tensor]): The first tensor. b (Union[list, np.ndarray, Tensor]): The second tensor. Returns: Tensor: Vector with res[i] = -manhattan_distance(a[i], b[i]) r)r*)r rr!r,absr$rs rpairwise_manhattan_simrDsc !##A ##A !Q  DAq IeiA&&B / / / 8 8 : : ::rct|}t|}|js|jr&t||\}}tj||zdd}tj||zdd}tj|| }|d|zz |z}tj |d}tj | Stj ||d S) a Computes the euclidean similarity (i.e., negative distance) between two tensors. Handles sparse tensors without converting to dense when possible. Args: a (Union[list, np.ndarray, Tensor]): The first tensor. b (Union[list, np.ndarray, Tensor]): The second tensor. Returns: Tensor: Matrix with res[i][j] = -euclidean_distance(a[i], b[j]) r r*rgming@r5) r rrr!sparser,r$ unsqueezematmultclampsqrtr=)rr a_norm_sq b_norm_sq dot_product squared_dists r euclidean_simrSs= !##A ##A{)ak)a##1L$$QU$22;;==GGJJ L$$QU$22;;==GGJJ l1accee,,5577 !1{?2Y> {z&_canonicalize_side..*s!/T/T/T!0B10E0E/T/T/TrT) batch_firstN`zA` must be a tensor, an ndarray, or a list of per-item masks, not .)rolisttupler!nnutilsrnn pad_sequencerprqr r TypeErrortype__name__rt)rjrkrvs r_canonicalize_sider"s$u && x~!../T/Tt/T/T/Tbf.gg D"* % % !$'' *T6":":  m m mW[\`WaWaWj m m m    !T 2 22rFrdtruncateboolc |jd|kr|S|r|jd|kr |dd|fStd|d|jdd|d)aMatch a mask's token span to the padded embeddings. The chunk loops pass ``truncate=True``, since a batch-wide mask legitimately spans more than one chunk's local padding. Caller-facing inputs must match exactly: a mismatched mask is a construction bug, not a layout to guess at.r).Nrz ` covers z( tokens, but the padded embeddings have r)shape ValueError)rkrdrvrs r_fit_mask_widthr4sv z"~ !DJrNU**C%K  hhh 2hh`ehhh i iirtorch.device | Nonect|tr|jSt|tr*|r(t|dtr |djSdS)zOThe device multi-vector embeddings live on, or None for numpy and empty inputs.rN)rorr<rrjs r_embeddings_devicer?s^*f%%!  *d##$ $z*Q-QW7X7X$!}## 4rr< torch.devicetuple[Tensor, Tensor | None]c|j|kr||}| |j|kr||}||fS)zMove one side's padded embeddings and mask onto the scoring device. The mask is checked separately, since a caller can hand in a cpu mask alongside gpu embeddings and the masked_fill kernel rejects that.)r<r;)rjrkr<s r _to_devicerHsOF""]]6**  DK611wwv t rrcendc| |||nd}t||||\}}t|||\}}|t||jd|d}||fS)a Pad one chunk of a side onto the scoring device, with its mask narrowed to the chunk's local width. Padding the whole batch upfront instead would let one long outlier size the full ``(batch, max_len, dim)`` tensor: multi-GB of mostly padding on ragged input.Nr T)r)_pad_multi_vector_inputsrrr)rjrkrcrr<rv chunk_maskchunks r _pad_chunkrSs{%)$4eCi$J0E#I1F SSE:"5*f==E:$ZQPTUUU * rczt|ts |jdStd|DdS)zLThe trailing embedding dimension, read without materializing a padded batch.r)c3^K|](}t|t|dV)dS)rNrar}items r z!_embedding_dim..ks7BB$D BT!WBBBBBBrr )rorrnextrs r_embedding_dimrgsA j$ ' '$## BB*BBBA F FFra_maskc|<|ddSt|trt jd|DStt|ddS)zPer-item count of real query tokens for MeanMaxSim scoring: the mask sum when a mask is given, the item lengths for ragged lists, and the non-zero rows for padded tensors.Nr)r*r rGcHg|]}tt|d S)r )r_rars rrz'_query_token_counts..ts(===4STA..===r) rr,rMrorr!tensor_zero_row_maskr )rrs r_query_token_countsrns{{}}  R ((..1.555!T?|==1===>>> ,Q// 0 0 5 5 7 7 ; ; ; C C I Ia I P PPrb_maskchunk_elements int | Nonelength_normalizestr | torch.device | Nonec 8t||d\}}t||d\}}t|dkst|dkrZt|pt|}tjt|t|tj|S|rt ||nd}t||\}} | t| |j dd} t|} ||nt} t|tr d|D} n|j dg| z} |t|t| d}t|p|j}|tj|n|} t|| | \}} t!| |j d|j dzt#|z| }g}|D]T\}}t%||||| d\}}|t)||| ||Ut|dkr|dntj|d}|7|||j|j dz }|S) a Computes the MaxSim (late-interaction) score between two collections of multi-vector embeddings. For each query in ``a`` and document in ``b``, the score is the sum over query tokens of the maximum similarity to any document token: ``sum_i max_j (a_i . b_j)``. This is the scoring function used by ColBERT-style models. Args: a (Union[list, np.ndarray, Tensor]): Query embeddings. Either a 3D tensor of shape ``(batch_a, num_query_tokens, embedding_dim)`` (pre-padded), a list of 2D tensors of shape ``(num_query_tokens_i, embedding_dim)`` (variable-length, padded internally), or a single 2D tensor or array (one query, e.g. from a singular encode call, scored as a batch of one). b (Union[list, np.ndarray, Tensor]): Document embeddings. Either a 3D tensor of shape ``(batch_b, num_doc_tokens, embedding_dim)``, a list of 2D tensors of shape ``(num_doc_tokens_i, embedding_dim)``, or a single 2D tensor or array (one document, scored as a batch of one). a_mask (Tensor, optional): Boolean or float mask for query tokens, shape ``(batch_a, num_query_tokens)``, ``(num_query_tokens,)`` alongside a single 2D ``a``, or a list of per-item 1D masks (padded to the batch). Tokens with a 0 / False entry are excluded from the sum. Defaults to None (use all tokens). b_mask (Tensor, optional): Boolean or float mask for document tokens, shape ``(batch_b, num_doc_tokens)``, ``(num_doc_tokens,)`` alongside a single 2D ``b``, or a list of per-item 1D masks. Tokens with a 0 / False entry are excluded from the max. Defaults to None (use all tokens). chunk_elements (int, optional): Element budget for the padded ``(chunk, d_tokens, dim)`` documents plus the 4D ``(batch_a, chunk, q_tokens, d_tokens)`` scoring intermediate. Documents are greedily packed into chunks (padded per chunk, so a long outlier only sizes its own chunk) that stay under the budget, with a floor of one document per chunk. Defaults to None, which applies a 100M-element budget (at most ~400 MB, half that in bf16 / fp16). With very large query batches that floor can still be big: shard queries externally if a single document exceeds memory. length_normalize (bool, optional): Divide each score by the number of real (unmasked) query tokens, yielding MeanMaxSim: scores land in the per-token similarity range (about ``[-1, 1]`` for normalized embeddings) independent of the query length. Defaults to False. device (str, torch.device, optional): Device to run the scoring on, moving each budget-sized document chunk there rather than the whole corpus. The returned scores stay on the documents' device either way. Defaults to None (the documents' device). Returns: Tensor: Matrix with ``res[i][j]`` = MaxSim(a[i], b[j]), shape ``(batch_a, batch_b)``, always float32 (half precision inputs are accumulated in float32 to keep nearby scores distinct), on the documents' device: CPU documents score on the CPU even against GPU queries. rrrdtyper<Nr c,g|]}t|Srzrr}documents rrzmaxsim..s;;;X3x==;;;rr*r<r)rrarr!zerosfloat32rrrr_MAXSIM_CHUNK_ELEMENT_BUDGETrorr_r<rrirrr`_maxsim_score_documentsr;catrrJ)rrrrrrr< result_devicequery_token_counts a_mask_padded num_documentsbudgetdocument_widthsscoring_devicerb score_chunksd_startd_endchunk_b chunk_b_maskscoress rmaxsimrxsd#1fh77IAv"1fh77IAv 1vv{{c!ffkk*1--F1CA1F1F {3q663q66}UUUU;KU,Q777QU/6::A} ' qwqz8LL FFM-9^^?[F!T7;;;;;71:,6  _)=)=xHH 'q))5QXM-3-?U\&)))]N!!]NCCA}?AGAJ,CnUVFWFW,WY_ ` `FL pp *1fgunV^ _ _3Aw |\\__`mnnoooo #L 1 1Q 6 6\!__EIlXYabstNrFr)r*rr!onesrr<_fill_empty_document_scoresrreinsum masked_fill_rJfinforrHr_valuesr:r,anyrrrremptyrs rrrsj wqzQ 171:qwqzAHUUU* K AGAJemAH U U UW\   \/1a 0 0FV[[]]44Q77AA!DDDekRXR^F_F_Fcddd ZZBZ   & , , . .F &**1--- ZZBZ  F ,Vfkkmm6G6GB6G6O6O5OPP MrrrctsH|jjdkr8|r$tdt dd||t S)aOverwrite the scores of documents flagged in ``empty`` (no unmasked token), whose every token sat at the masked-fill dtype minimum and whose sum therefore poisons downstream losses. The fill is unconditional: reading ``empty`` on the host costs a device synchronization per scoring call and is a data-dependent branch that :func:`torch.compile` cannot capture, so only the warning reads it, and only where that is free. cpuaEncountered documents without any scoreable token during multi-vector similarity scoring, e.g. because MultiVectorMask (via keep_only_token_ids or skiplist_words) masked out every token, or because a document embedding has zero token vectors. These documents score about z.0ezD (before any normalization), ranking them below every real document.)rr<rrr7r8_EMPTY_DOCUMENT_SCORE masked_fill)rrs rrrs{ $ % % %,*;u*D*D*D + f         e%: ; ;;rc t||d\}}t||d\}}t|t|kr0tdt|dt|dt|dkr?t|pt|}t jdtj|S|rt||nd}||nt} t|tr d |Dn|j d gt|z} t|tr d |Dn|j d gt|z} |t|t| d}|t|t| d}t|p"t|pt jd }|t j|n|} t| } t|}t!| | |z| | |z }g}|D]k\}}t#||||| d\}}t#||||| d\}}|t'|||||lt|d kr|dnt j|d}|$|||j|jz }|S)a Computes the pairwise MaxSim (late-interaction) score between each query-document pair. For each ``i``, computes the MaxSim score between ``a[i]`` and ``b[i]``: the sum over query tokens of the maximum similarity to any document token. This is the pairwise analogue of :func:`maxsim`. Args: a (Union[list, np.ndarray, Tensor]): Query embeddings. Either a 3D tensor of shape ``(batch, num_query_tokens, embedding_dim)`` (pre-padded), a list of 2D tensors with shape ``(num_query_tokens_i, embedding_dim)``, or a single 2D tensor or array (one query, e.g. from a singular encode call, scored as a batch of one). b (Union[list, np.ndarray, Tensor]): Document embeddings. Either a 3D tensor of shape ``(batch, num_doc_tokens, embedding_dim)``, a list of 2D tensors with shape ``(num_doc_tokens_i, embedding_dim)``, or a single 2D tensor or array (one document, scored as a batch of one). a_mask (Tensor, optional): Boolean or float mask for query tokens, shape ``(batch, num_query_tokens)``, ``(num_query_tokens,)`` alongside a single 2D ``a``, or a list of per-item 1D masks (padded to the batch). Tokens with a 0 / False entry are excluded from the sum. Defaults to None (use all tokens). b_mask (Tensor, optional): Boolean or float mask for document tokens, shape ``(batch, num_doc_tokens)``, ``(num_doc_tokens,)`` alongside a single 2D ``b``, or a list of per-item 1D masks. Tokens with a 0 / False entry are excluded from the max. Defaults to None (use all tokens). chunk_elements (int, optional): Element budget for the padded ``(chunk, q_tokens, dim)`` queries and ``(chunk, d_tokens, dim)`` documents plus the ``(chunk, q_tokens, d_tokens)`` scoring intermediate. Pairs are greedily packed into chunks (padded per chunk, so a long outlier only sizes its own chunk) that stay under the budget, with a floor of one pair per chunk. Defaults to None, which applies a 100M-element budget (at most ~400 MB, half that in bf16 / fp16). length_normalize (bool, optional): Divide each score by the number of real (unmasked) query tokens, yielding MeanMaxSim: scores land in the per-token similarity range (about ``[-1, 1]`` for normalized embeddings) independent of the query length. Defaults to False. device (str, torch.device, optional): Device to run the scoring on, moving each budget-sized chunk of pairs there rather than everything. The returned scores stay on the documents' device either way. Defaults to None (the documents' device). Returns: Tensor: Vector with ``res[i]`` = MaxSim(a[i], b[i]), shape ``(batch,)``, always float32 (half precision inputs are accumulated in float32 to keep nearby scores distinct), on the documents' device: CPU documents score on the CPU even against GPU queries. rrzF`a` and `b` must hold the same number of items to score as pairs, got z and rrrNc,g|]}t|Srzr)r}querys rrz#maxsim_pairwise..Js...5CJJ...rr c,g|]}t|Srzrrs rrz#maxsim_pairwise..Ks777s8}}777rr)r[r*r)rrarrr!rrrrrorrrr_r<rrirr`_maxsim_score_pairsr;rr)rrrrrrr<rrr query_widthsrr query_widthr+rbrrcrchunk_a chunk_a_maskrrrs rmaxsim_pairwiser s`#1fh77IAv"1fh77IAv 1vvQ kUXYZU[U[ k kbefgbhbh k k k    1vv{{*1--F1CA1F1F {1EM-HHHH;KU,Q777QU-9^^?[F2bstNr)r*rrs rrrs4 wqzQ 171:UZIII*5;qwqz_`_g+h+h+hjoppp \.!Q / /FV[[]]44Q777V\9R9R9VWWW ZZBZ   & , , . .F & ZZBZ  F ,Vfkkmm6G6GB6G6O6O5OPP Mrpaddedc^|d|jS)aDerive a ``(B, T)`` mask from a pre-padded ``(B, T, D)`` token-embedding tensor by treating all-zero rows as padding. Real token embeddings normally carry information in at least one dim, so a fully-zero row is the all-but-impossible-except-by-pad signal. This matches the zero padding of :func:`_pad_tensor_list`. Returned in the input dtype (1.0 for real tokens, 0.0 for pad). Boundary: a real all-zero row is indistinguishable from padding here and is masked out of scoring. L2-normalized pipelines cannot produce one, but token pooling can (a cluster mean of antipodal vectors cancels to zero). Masking such a row scores it as 0 rather than dtype-min garbage, the preferable failure mode. A document whose rows are *all* zero instead reads as empty and takes the :func:`_fill_empty_document_scores` sentinel. Any future module that intentionally emits zero rows (e.g. learned pruning) must pass explicit masks instead of relying on this detection. r)r*)rr;r)rs rrrs( ::":    . ..rtensors list[Tensor]lengthsc|djjdkr,tjjj|ddSt|}t| |krtj |Stj |d}t|}| |j}tj|d|z }tj||jtj|||z }tjtjt||j||}|t||z|jd}|d||z|z||t|||jdS) aStack per-item ``(tokens_i, dim)`` embeddings into a padded ``(batch, max_tokens, dim)``. ``pad_sequence`` copies item by item, which on an accelerator is one kernel launch per item: for a 10k document corpus that padding can outweigh the MaxSim that follows it by two orders of magnitude. Both alternatives here launch a constant number of kernels instead. On CPU there are no launches to amortize and ``pad_sequence`` is the fastest of the three, so it keeps that side. rrT)r padding_valuer*r<) output_sizer))r<rr!rrrrrYr_rHstackrrar;cumsumarangerepeat_interleave new_zerosr index_copy_view) rrmax_lenflattotalstarts positionsrowsrs r_pad_tensor_listrsqz&&x~!..wDXY.ZZZ'++--  G 7;;==W$${7### 9W! $ $ $D IIEjj%%G \'1 % % /F U4;777%:QRXZaot:u:u:uuI  "5<G T[#Q#Q#QSZhm n n nD ^^CLL72DJrN C CF q$.94d;;; ;;s7||Wdjn = ==rinputsct|trd|D}d|D}tjd|D}t ||}|tj|jd|j}|d| |jdk |j }||fSt|}| s| }|'|dkrt|}||fS) acPad a list of variable-length multi-vector tensors into a single 3D tensor with a mask. Returns the padded tensor and either the user-provided mask or one derived from the input. For a list input the mask comes from the per-row lengths. For a 3D tensor input without a mask the mask comes from all-zero rows (see :func:`_zero_row_mask`). c,g|]}t|Srzr{r}rLs rrz,_pad_multi_vector_inputs..s!999Q%a((999rcbg|],}|r|n|-Srz)is_floating_pointr:rs rrz,_pad_multi_vector_inputs..s5NNN++--<1117799NNNrc(g|]}|jdSr)rrs rrz,_pad_multi_vector_inputs..s<<>> similarity_fn = SimilarityFunction.to_similarity_fn("cosine") >>> similarity_scores = similarity_fn(embeddings1, embeddings2) >>> similarity_scores tensor([[0.3952, 0.0554], [0.0992, 0.1570]]) The provided function 4 is not supported. Use one of the supported values: r)r"COSINEr DOT_PRODUCTr0 MANHATTANrA EUCLIDEANrSMAXSIMr MEAN_MAXSIMrrpossible_valuesr's rto_similarity_fnz#SimilarityFunction.to_similarity_fnDs.11DEE "4"; ; ;N "4"@ @ @  "4"> > > "4"> > > "4"; ; ;M "4"@ @ @  V%8 V VoAoQoQoSoS V V V   rct|}|tjkrtS|tjkrtS|tjkrt S|tjkrtS|tj krtS|tj krtStd|dtd)a1 Converts a similarity function into a pairwise similarity function. The pairwise similarity function returns the diagonal vector from the similarity matrix, i.e. it only computes the similarity(a[i], b[i]) for each i in the range of the input tensors, rather than computing the similarity between all pairs of a and b. Args: similarity_function (Union[str, SimilarityFunction]): The name or enum value of the similarity function. Returns: Callable[[Union[list, np.ndarray, Tensor], Union[list, np.ndarray, Tensor]], Tensor]: The pairwise similarity function. The MaxSim family also accepts further keyword arguments, see :func:`maxsim_pairwise`. Raises: ValueError: If the provided similarity function is not supported. Example: >>> pairwise_fn = SimilarityFunction.to_similarity_pairwise_fn("cosine") >>> similarity_scores = pairwise_fn(embeddings1, embeddings2) >>> similarity_scores tensor([0.3952, 0.1570]) r+r,r)r"r-r.r.r-r/rDr0rUr1rr2rrr3r4s rto_similarity_pairwise_fnz,SimilarityFunction.to_similarity_pairwise_fnns411DEE "4"; ; ;# # "4"@ @ @% % "4"> > >) ) "4"> > >) ) "4"; ; ;" " "4"@ @ @' ' V%8 V VoAoQoQoSoS V V V   r list[str]c$dtDS)a| Returns a list of possible values for the SimilarityFunction enum. Returns: list: A list of possible values for the SimilarityFunction enum. Example: >>> possible_values = SimilarityFunction.possible_values() >>> possible_values ['cosine', 'dot', 'euclidean', 'manhattan', 'maxsim', 'meanmaxsim'] cg|] }|j Srz)rr|s rrz6SimilarityFunction.possible_values..s444A444r)r"rzrrr3z"SimilarityFunction.possible_valuess54!34444rN)r'r(rr))rr8)r __module__ __qualname____doc__r-r.DOTr0r/r1r2 staticmethodr5r7r3rzrrr"r".s  FK CII FK' ' ' \' R* * * \* X 5 5 5\ 5 5 5rr")rrrrrr)rrrrrr)rrrrrrr ) rVrWrXrYrZrYr[rYrr\)rjrrkrlrrm)rjrrkrurvrwrrm)F) rkrrdrYrvrwrrrr)rjrrr)rjrrkrlr<rrr)rjrrkrlrcrYrrYr<rrvrwrr)rjrrrY)rrrrlrr)NNNFN)rrrrrrlrrlrrrrr<rrr) rrrrrrlrrlrr)rrrrrr)NNNTN)rrrr)rrrrrr)rrrkrlrr)rrrrrr)9 __future__rcollections.abcrenumrnumpyrpr!sklearn.metricsrrtransformers.utilsrr rr r r rr get_loggerrr7rrrr.r0r-rArDrSrUrrrirtrrrrrrrrrrrrrrrrrr r"rzrrrGs""""""$$$$$$ ......@@@@@@@@  H % %*    ????&____.5555"****"44448;;;;$))))@CCCC( +]^     F     3333$jjjjj(GGGGQQQQ! !%"(,YYYYYx""""J<<<<.! !%"(,WWWWWz! !%!(,F! !%!(,40////$>>>>2:*!*!*!*!Zz5z5z5z5z5z5z5z5z5z5r