ej-'VdZddlZddlmZddlmZmZddlm Z m Z m Z m Z m Z ddlmZmZmZddlmZmZejeZd Zd ed Dd ed DzZGdde ZGdde dZeGdde ZdgZdS)z Processor class for PaliGemma. N) BatchFeature) ImageInputis_valid_image)MultiModalDataProcessingKwargsProcessorMixin TextKwargsUnpack) AddedTokenPreTokenizedInput TextInput)auto_docstringloggingzcg|] }d|dd S)z4>.0is `C:\Projects\DSCons\.venv\Lib\site-packages\transformers\models\paligemma\processing_paligemma.py r$s"555Aq555icg|] }d|dd S)z3rrrs rrr$s"8]8]8]Q8]8]8]rcNeZdZUdZeezeezeezdzed<dS)PaliGemmaTextKwargsa suffix (`str`, `list[str]`, `list[list[str]]`): The suffixes or batch of suffixes to be encoded. Only necessary for finetuning. See https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/paligemma/README.md for more information. If your prompt is " What is on the image", the suffix corresponds to the expected prediction "a cow sitting on a bench". Nsuffix)__name__ __module__ __qualname____doc__rr list__annotations__rrrrr'sJ ) )DO ;dCT>U UX\ \\\\\\rrc.eZdZUeed<dddddidZdS)PaliGemmaProcessorKwargs text_kwargsF)paddingreturn_mm_token_type_ids data_formatchannels_first)r' images_kwargsN)rr r!rr$ _defaultsrrrr&r&1sG$$$$(-  + IIIrr&F)totalc :eZdZeZ dfd Ze ddedzdee ze eze e zde ede ffd Z dfd Z ddedzdee ze eze e zdzde effd Zd ed edefd Zdd ZefdZxZS)PaliGemmaProcessorNc t|dstd|j|_t|ds]ttdd}d|gi}|||t|_t|_n|j|_|j|_| td|_ d|_ t|||dS) Nimage_seq_lengthz;Image processor is missing an `image_seq_length` attribute. image_tokenFT) normalizedspecialadditional_special_tokens) chat_template)hasattr ValueErrorr2r IMAGE_TOKENadd_special_tokensconvert_tokens_to_idsimage_token_idr3 add_tokens EXTRA_TOKENS add_bos_token add_eos_tokensuper__init__)selfimage_processor tokenizerr7kwargsr3 tokens_to_add __class__s rrCzPaliGemmaProcessor.__init__Bs(:;; \Z[[ [ / @y-00 5$[UDQQQK8;-HM  ( ( 7 7 7"+"A"A+"N"ND *D  "+":D (4D \***"' "'  )=QQQQQrimagestextrGreturnc |ddx}2t|ttfs|g}fd|D}||d<d|d<jt fdjji|}|dd }tj d||d |}d |vr@tj |d }d |tj |d dk<||d<t||S)a Returns: [`BatchFeature`]: A [`BatchFeature`] with the following fields: - **input_ids** -- List of token ids to be fed to a model. Returned when `text` is not `None`. If `suffix` is provided, the `input_ids` will also contain the suffix input ids. - **attention_mask** -- List of indices specifying which tokens should be attended to by the model (when `return_attention_mask=True` or if *"attention_mask"* is in `self.model_input_names` and if `text` is not `None`). - **pixel_values** -- Pixel values to be fed to a model. Returned when `images` is not `None`. - **labels** -- Labels compatible with training if `suffix` is not None rNc.g|]}|jjzSr)rF eos_token)rsamplerDs rrz/PaliGemmaProcessor.__call__..ts#MMMFft~77MMMr text_pairTreturn_token_type_idstokenizer_init_kwargsr'return_tensorsrJrKtoken_type_ids input_idsirlabels)data tensor_typer)pop isinstancer#tuple _merge_kwargsr&rF init_kwargsgetrB__call__nparrayr) rDrJrKrGrrTresultrXrIs ` rrazPaliGemmaProcessor.__call__^s8&jj400 0F =ftUm44 " MMMMfMMMF"(F; *.&'## $  "&."<     .223CDD!!EdEEfEE v % %Xf[122F>BF28F#3455: ;%F8 ^DDDDrc |}tjd ||d|^}}}||td|DsTtdt |r|ggt |z}nt|ttfrrt |dr]t |t |kr0tdt |dt |dd |D}nkt|ttfr@t|dttfrt |ddr|}ntd fd t||D}npg}|D]i}| tt tz}||d|jjz||dzd zj|}||ddfS)NrUc3(K|] }t|vVdSN)r:)rrPs r z;PaliGemmaProcessor.prepare_inputs_layout..s'@@{f,@@@@@@raLYou are passing both `text` and `images` to `PaliGemmaProcessor`. The processor expects special image tokens in the text, as many tokens as there are images per each text. It is recommended to add `` tokens in the very beginning of your text. For this call, we will infer how many images each text has and add special tokens.rz Received z images for zK prompts. Each prompt should be associated with an image or list of images.cg|]}|gSrr)rimages rrz.s(E(E(EU%(E(E(ErzAimages must be an image, list of images or list of list of imagescg|]D\}}tt|trt|ndzjj|dES) )r:r\r#lenrF bos_token)rprompt image_listrDs rrz.sw* #JPT9U9U&\c*ooo[\]D_c_m_wDyDDDrrnr)rBprepare_inputs_layoutanyloggerwarningrror\r#r]r9ziprfindr:appendrFrp) rDrJrKrG_images_per_samplenew_textrPlast_image_endrIs ` rrsz(PaliGemmaProcessor.prepare_inputs_layoutsu85778\T\\U[\\q  $"2@@4@@@@@& <"&))j*0 SYY(>%%u 66 j>&QR);T;T j6{{c$ii//(XF XXTXXX)F(Ef(E(E(E%%ve}55j"6!9tUm<<j'vay|44j )/%%$%hiii.1$8I.J.J "F%+\\+%>%>[AQAQ%QNOO/$.2JJVTbTcTcMddgkk tT4''rc tjd||d||td|tddSdS)NrUzF`images` are expected as arguments to a `PaliGemmaProcessor` instance.z]You are using PaliGemma without a text prefix. It will perform as a picture-captioning model.r)rBvalidate_inputsr9ru warning_once)rDrJrKrGrIs rrz"PaliGemmaProcessor.validate_inputssn CvDCCFCCC >eff f <   o      s5 RRRRRR8%)Z^(E(ET!(E++d9o=EV@WW(E12 (E  (E(E(E(E(E^(ET-(-(-(-(-(-(b%)aeT!++d9o=EV@WWZ^^)*  888SV8888----$HHHHXHHHHHrr0)r"numpyrbfeature_extraction_utilsr image_utilsrrprocessing_utilsrrr r r tokenization_utils_baser r rutilsrr get_loggerrrur:ranger?rr&r0__all__rrrrs44444455555555POOOOOOOOO,,,,,,,,  H % % 55t5558]8]RWRWX[R\R\8]8]8]] ]]]]]*]]]     /u    `H`H`H`H`H`H`H`HF r