˲jlUdZddlZddlZddlZddlZddlZddlZddlZddlZddl Z ddl m Z m Z m Z mZddlmZddlmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZ m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'ddl(m)Z)gdZ*e&dZ+e&d Z,e&d Z-e&d Z.d Z/d Z0Gdde"Z1Gddej2Z3eZ4eZ5ee5ge$eee4ffZ6eeee4ge5fZ7eZ8ee4ge8fZ9ee8ge4fZ:edee;ge;fZee5ge$ee$e1efeffZ?GddeZ@ejAZBiZCee%ee@feDd<GddeZEiZFee%eeEfeDd<iZGee;e%efeDd< ejHIdZJdZKddlLmMZMeMeJeMdkZNeNse jOdeP[Mn#ejHjQ$rd xZNZKYnwxYwd ZRgZSeeeDd!<ddddd"d#e%ed$e6d%e7d&ee;d'ee9d(ee:d)ee?d*dfd+ZTd#e%ed&e;d*dfd,ZUe)d-eP. dddddd"d#e%ed$e6d%e7d/ee<d0ee=d&ee;d'ee9d(ee:d)ee?d*dfd1ZVddddd"d#e%ed$e6d%e7d&ee;d'ee9d(ee:d)ee?d*dfd2ZWejXd3Gd4d5ee+ZYe&d6e7ZZejXd3Gd8d9eeZe+fZ[ejXd3Gd:d;Z\dZ^d?eed@e4d*e$edffdAZ_defeffdZ ddce5dfeee5gefd*ee$e>effdZ dde>dce5dfeee5gefd*ee$e>effdZddtduedefdce5dve5dfeee5gefd*e5f dZde>d*e;fdZdede>d*efdZdS)a Contains utility functions for working with nested python data structures. A *pytree* is Python nested data structure. It is a tree in the sense that nodes are Python collections (e.g., list, tuple, dict) and the leaves are Python values. Furthermore, a pytree should not contain reference cycles. pytrees are useful for working with nested collections of Tensors. For example, one can use `tree_map` to map a function over all Tensors inside some nested collection of Tensors and `tree_leaves` to get a flat list of all Tensors inside some nested collection. pytrees are helpful for implementing nested collection support for PyTorch APIs. This pytree implementation is not very performant due to Python overhead To improve the performance we can move parts of the implementation to C++. N) defaultdictdeque namedtuple OrderedDict)Enum)AnyCallablecast DefaultDictDequeDict FrozenSetGenericHashableIterableListMapping NamedTupleOptionalroverloadProtocolSequenceTupleTypeTypeVarUnion) deprecated)PyTreeContext FlattenFunc UnflattenFuncDumpableContextToDumpableContextFnFromDumpableContextFnTreeSpecLeafSpeckeystrkey_getregister_pytree_node tree_flattentree_flatten_with_pathtree_unflatten tree_iter tree_leavestree_leaves_with_pathtree_structuretree_maptree_map_with_path tree_map_ tree_map_onlytree_map_only_tree_alltree_any tree_all_only tree_any_onlytreespec_dumpstreespec_loadstreespec_pprintTSURNO_SERIALIZED_TYPE_NAME_FOUNDcFeZdZdefdZdedefdZdefdZ de de fdZ dS) KeyEntryreturncdSNselfs DC:\Projects\Historical\venv\Lib\site-packages\torch/utils/_pytree.py__hash__zKeyEntry.__hash__f othercdSrGrH)rJrOs rK__eq__zKeyEntry.__eq__irMrNcdSrGrHrIs rK__str__zKeyEntry.__str__lrMrNparentcdSrGrH)rJrTs rKgetz KeyEntry.getorMrNN) __name__ __module__ __qualname__intrLobjectboolrQstrrSrrVrHrNrKrDrDes #     F t          # #      rNrDc(eZdZdedeffd ZxZS) EnumEncoderobjrEc~t|tr|jSt|SrG) isinstancervaluesuperdefault)rJr` __class__s rKrezEnumEncoder.defaultts3 c4  9 wws###rN)rWrXrYr[r]re __classcell__rfs@rKr_r_ssK$6$c$$$$$$$$$$rNr_r%.cPeZdZUeeed<eed<eed<ee ed<dS)NodeDeftype flatten_fn unflatten_fnflatten_with_keys_fnN) rWrXrYrr__annotations__r r!rFlattenWithKeysFuncrHrNrKrjrjsI s)OOO"#6777777rNrjSUPPORTED_NODESc\eZdZUeeed<eed<eeed<ee ed<dS)_SerializeNodeDeftypserialized_type_nameto_dumpable_contextfrom_dumpable_contextN) rWrXrYrrror]rr#r$rHrNrKrsrssO cNNN!"56666#$9::::::rNrsSUPPORTED_SERIALIZED_TYPESSERIALIZED_TYPE_TO_PYTHON_TYPEoptreeT)Versionz0.13.0zoptree is installed but the version is too old to support PyTorch Dynamo in C++ pytree. C++ pytree support is disabled. Please consider upgrading optree using `python3 -m pip install --upgrade 'optree>=0.13.0'`.F_cxx_pytree_pending_importsrurvrwrnclsrlrmrurvrwrnrEc \t5|tvrt|d dddn #1swxYwYt|||||||tsdSt r#ddlm}|||||||dS|||f}|||d} t || fdS)aRegister a container-like type as pytree node. Args: cls: the type to register flatten_fn: A callable that takes a pytree and returns a flattened representation of the pytree and additional context to represent the flattened pytree. unflatten_fn: A callable that takes a flattened version of the pytree, additional context, and returns an unflattened pytree. serialized_type_name: A keyword argument used to specify the fully qualified name used when serializing the tree spec. to_dumpable_context: An optional keyword argument to custom specify how to convert the context of the pytree to a custom json dumpable representation. This is used for json serialization, which is being used in torch.export right now. from_dumpable_context: An optional keyword argument to custom specify how to convert the custom json dumpable representation of the context back to the original context. This is used for json deserialization, which is being used in torch.export right now. flatten_with_keys_fn: An optional keyword argument to specify how to access each pytree leaf's keypath when flattening and tree-mapping. Like ``flatten_fn``, but in place of a List[leaf], it should return a List[(keypath, leaf)]. z& is already registered as pytree node.Nr}rA) _cxx_pytree)rurvrw) _NODE_REGISTRY_LOCKrq ValueError_private_register_pytree_node_cxx_pytree_exists_cxx_pytree_importedrr|append) r~rlrmrurvrwrncxxargskwargss rKr)r)skD MM / ! !KKKLL L "MMMMMMMMMMMMMMM" 1/31 ;(((((( ))   !5 3"7 *     Z.$8#6%:   $**D&>:::::s 155c dt|tt|ttt dS)a Registers a namedtuple as a valid pytree node. By default namedtuples are valid pytree nodes, but they are not serializable. This API provides the argument `serialized_type_name` which allows these namedtuples to be serialized. Args: cls: the dataclass type to register serialized_type_name: The serialized name for the dataclass. This is required if you want to serialize the pytree TreeSpec containing this namedtuple. r}N)r_namedtuple_flatten_namedtuple_unflatten_namedtuple_serialize_namedtuple_deserialize_namedtuple_flatten_with_keys)r~rus rK_register_namedtuplers:"" 115:rNzy`torch.utils._pytree._register_pytree_node` is deprecated. Please use `torch.utils._pytree.register_pytree_node` instead.)category to_str_fnmaybe_from_str_fnc r||tjdtdt|||||||dS)aRegister a container-like type as pytree node for the Python pytree only. Args: cls: the type to register flatten_fn: A callable that takes a pytree and returns a flattened representation of the pytree and additional context to represent the flattened pytree. unflatten_fn: A callable that takes a flattened version of the pytree, additional context, and returns an unflattened pytree. serialized_type_name: A keyword argument used to specify the fully qualified name used when serializing the tree spec. to_dumpable_context: An optional keyword argument to custom specify how to convert the context of the pytree to a custom json dumpable representation. This is used for json serialization, which is being used in torch.export right now. from_dumpable_context: An optional keyword argument to custom specify how to convert the custom json dumpable representation of the context back to the original context. This is used for json deserialization, which is being used in torch.export right now. flatten_with_keys_fn: An optional keyword argument to specify how to access each pytree leaf's keypath when flattening and tree-mapping. Like ``flatten_fn``, but in place of a List[leaf], it should return a List[(keypath, leaf)]. Nzx`to_str_fn` and `maybe_from_str_fn` is deprecated. Please use `to_dumpable_context` and `from_dumpable_context` instead.) stacklevelr})warningswarn FutureWarningr) r~rlrmrrrurvrwrns rK_register_pytree_noder*snR 1 =  T      " 1/31rNcTt5|tvrtj|dt ||||}|t|<|du|duz rt d|d|t }t||||}|t|<|t|<ddddS#1swxYwYdS)zThis is an internal function that is used to register a pytree node type for the Python pytree only. End-users should use :func:`register_pytree_node` instead. zM is already registered as pytree node. Overwriting the previous registration.Nz7Both to_dumpable_context and from_dumpable_context for z must be None or registered.) rrqrrrjrrBrsrxry) r~rlrmrurvrwrnnode_defserialize_node_defs rKrrfsV CC / ! ! M999    3 L:NOO' 4 ',AT,I J )#)))  '#@ .   !    +="3'?B&';<7CCCCCCCCCCCCCCCCCCsBBB!$B!)frozencBeZdZUeed<defdZdeedefdZ dS) SequenceKeyidxrEcd|jdSN[]rrIs rKrSzSequenceKey.__str__ 48    rNsequencec||jSrGr)rJrs rKrVzSequenceKey.gets!!rNN) rWrXrYrZror]rSrr=rVrHrNrKrrs` HHH!!!!!"HQK"A""""""rNrK)boundcFeZdZUeed<defdZdeeefdefdZ dS) MappingKeykeyrEcd|jdSrrrIs rKrSzMappingKey.__str__rrNmappingc||jSrGr)rJrs rKrVzMappingKey.getstx  rNN) rWrXrYrror]rSrr=rVrHrNrKrrsd FFF!!!!!!71a4=!Q!!!!!!rNrc6eZdZUeed<defdZdedefdZdS) GetAttrKeynamerEcd|jS)N.)rrIs rKrSzGetAttrKey.__str__s49rNr`c,t||jSrG)getattrr)rJr`s rKrVzGetAttrKey.getssDI&&&rNN)rWrXrYr]rorSrrVrHrNrKrrs\ III's's''''''rNrdc$t|dfSrGlistrs rK_tuple_flattenrs 77D=rNc\t|\}}dt|D|fS)Nc6g|]\}}t||fSrHr.0ivs rK z,_tuple_flatten_with_keys..' > > >DAq[^^Q  > > >rN)r enumeratervaluescontexts rK_tuple_flatten_with_keysr6%Q''OFG > >If,=,= > > > GGrNrrc t|SrG)tuplerrs rK_tuple_unflattenrs ==rNc |dfSrGrHrs rK _list_flattenrs d7NrNc\t|\}}dt|D|fS)Nc6g|]\}}t||fSrHrrs rKrz+_list_flatten_with_keys..rrN)rrrs rK_list_flatten_with_keysrs4#A&&OFG > >If,=,= > > > GGrNc t|SrGrrs rK_list_unflattenrs <<rNct|t|fSrGrrkeysrs rK _dict_flattenr+   T!&&((^^ ++rNc^t|\}}dt||D|fS)Nc6g|]\}}t||fSrHrrkrs rKrz+_dict_flatten_with_keys..' @ @ @41aZ]]A  @ @ @rN)rziprs rK_dict_flatten_with_keysrs8$A&&OFG @ @3w+?+? @ @ @' IIrNc<tt||SrG)dictrrs rK_dict_unflattenrs GV$$ % %%rNc>t|t|fSrG)rrkrs rKrrs 77DGG rNcht|\}}dt|j|D|fS)Nc6g|]\}}t||fSrH)r)rfieldrs rKrz1_namedtuple_flatten_with_keys..s)MMMHE1*U  Q MMMrN)rr_fieldsrs rKrrs?*!,,OFGMMGOV0L0LMMM rNc2tt||SrG)r rrs rKrrs  GGV, - --rNc|tvrtd|dt|}|j}|tkrtd|d|S)Nz-Can't serialize TreeSpec of namedtuple class zc because we didn't register a serializated_type_name. Please register using `_register_namedtuple`.za because we couldn't find a serializated_type_name. Please register using `_register_namedtuple`.)rxNotImplementedErrorrurB)rrrus rKrrs000! &G & & &   4G<-B<<<! &G & & &   rNdumpable_contextcX|tvrtd|dt|}|S)Nz/Can't deserialize TreeSpec of namedtuple class z. because we couldn't find a serializated name.)ryr)rrts rKrrsI===! <>N < < <   ))9 :C JrNct|t|fSrGrrs rK_ordereddict_flattenrrrNc^t|\}}dt||D|fS)Nc6g|]\}}t||fSrHrrs rKrz2_ordereddict_flatten_with_keys..rrN)rrrs rK_ordereddict_flatten_with_keysrs8+1--OFG @ @3w+?+? @ @ @' IIrNcPtdt||DS)Nc3$K|] \}}||fV dSrGrH)rrrcs rK z)_ordereddict_unflatten..!s*KK UU|KKKKKKrN)rrrs rK_ordereddict_unflattenrs+ KKc'66J6JKKK K KKrNc<t|\}}||j|gfSrG)rdefault_factory)rr dict_contexts rK_defaultdict_flattenr(s'(++FL A%|4 44rNcht|\}}|\}}dt||D|fS)Nc6g|]\}}t||fSrHrrs rKrz2_defaultdict_flatten_with_keys..2s' E E E41aZ]]A  E E ErN)rr)rrr_rs rK_defaultdict_flatten_with_keysr-sA+1--OFGOA| E E3|V+D+D E E Ew NNrNcH|\}}t|t||SrG)rr)rrrrs rK_defaultdict_unflattenr5s(%,!O\  (M(M N NNrNc0|\}}|j|j|d}|S)N)default_factory_moduledefault_factory_namer)rXrY)rrrjson_defaultdicts rK_defaultdict_serializer =s0$+!O\"1"< / <$ rNc:t|tsJt|hdksJ|d}|d}t|tsJt|tsJt j|}t ||}|d}||gS)N>rr r r r r)rbrsetr] importlib import_moduler)rr r modulerrs rK_defaultdict_deserializerGs & - --- -  %%%     ..FG+,BC ,c 2 222 2 *C 0 000 0  $%; < .brrN)rrrs rK_deque_flatten_with_keysr^rrNc$t||S)Nr)rrs rK_deque_unflattenres  ( ( ((rNzbuiltins.tuple)rurnz builtins.listz builtins.dictzcollections.namedtuplezcollections.OrderedDictzcollections.defaultdictzcollections.dequeSTANDARD_DICT_TYPES BUILTIN_TYPEStreect|}|j}t|dks|dtkrdSt |dd}t |tsdSt d|DS)NrArFrc3FK|]}t|tkVdSrG)rkr])rentrys rKrz*_is_namedtuple_instance..s-66etE{{c!666666rN)rk __bases__lenrrrball)rrtbasesfieldss rK_is_namedtuple_instancer's{ t**C ME 5zzQ%(e++u S)T * *F fe $ $u 66v666 6 66rNcLt|rtSt|SrG)r'rrk)rs rK_get_node_typer)s$t$$ ::rNis_leafcL|dur ||pt|tvSrG)r)rqrr*s rK_is_leafr-s8 4  1GGDMM n 77 7rN)initreqreprc<eZdZUeed<eed<eded<ejdZ e ed<ejdZ e ed<ejdZ e ed<dd Z dd e d efdZd efdZdedeed d fdZded eefdZdeed efdZd S)r%rkrchildren_specsF)r. num_nodes num_leaves num_childrenrENcRtd|jDd}td|jD}t|j}t|d|t|d|t|d|dS)Nc3$K|] }|jV dSrG)r3rspecs rKrz)TreeSpec.__post_init__..s$HHDHHHHHHrNrA)startc3$K|] }|jV dSrG)r4r8s rKrz)TreeSpec.__post_init__..s$IITIIIIIIrNr3r4r5)sumr2r#r[ __setattr__)rJr3r4r5s rK __post_init__zTreeSpec.__post_init__sHHD4GHHHPQRRR IIT5HIIIII 4.// 4i8884z:::4>>>>>rNrindentc8d|jjd|jd}d}|jdkrkdz ||jdz }||jdkrdndz }|dfd |jddDz }|d }||zS) Nz TreeSpec(z, z, [rrrrA,cLg|] }ddzz|z!S)  )__repr__)rchildr?s rKrz%TreeSpec.__repr__..sC3<'%..*@*@@rNz]))rkrWrr5r2rEjoin)rJr? repr_prefixchildren_specs_str repr_suffixs ` rKrEzTreeSpec.__repr__sNty'9NNT\NNN "$  q aKF $"5a"8"A"A&"I"I I  ):Q)>)>##B F  #((!%!4QRR!8##   1444 [((rNc.|jdko |jdkS)NrA)r3r4rIs rKr*zTreeSpec.is_leafs~";t!';;rNrsubtreesc ^|r|dSt}|jtvr||jkrt d|jd|dt |j}|\}}t||j kr(t d|j dt|d||j krt d|jdn|jtvo|tv}||jkr|st d|jd|dt|j kr(t d|j dtd|r|jtur|j n |j d}|} t} t| } | | krS| | } | | } d}| r|d | z }| r|d | z }t d |dfd | D}n\t |j}|\}}||j kr1|jtur#t d |jd|j d|dt!||jD]\}}|||dS)NzType mismatch; expected z , but got rzNode arity mismatch; expected z+Node context mismatch for custom node type zNode type mismatch; expected rArz; missing key(s): z; extra key(s): zNode keys mismatchc g|] }| SrHrH)rrrs rKrz2TreeSpec._flatten_up_to_helper..'s D D Dsc D D DrNz$Node context mismatch for node type z ; expected )r*rr)rkrrrqrlr#r5rrrr differencerrr2_flatten_up_to_helper)rJrrL node_typerl child_pytreesrboth_standard_dictr expected_keys got_key_setexpected_key_set missing_keys extra_keysmessage child_pytree child_specs ` rKrPzTreeSpec._flatten_up_to_helpers <<>>  OOD ! ! ! F"4(( 9M ) )DI%% F $ FF7@FFF)3>J%/Z%5%5 "M7=!!T%666 S $ 1SS=@=O=OSSS$,&& P$)PPP' 00UYBU5U DI%%.@% F $ FF7@FFF4yyD--- J $ 1JJ=@YYJJJ " y 33LLa  !- !$ii #&}#5#5 "222#3#>#>{#K#KL!,!7!78H!I!IJ G#G#F #F#FF!C#Bj#B#BB$%D'%D%D%DEEE D D D Dm D D D ,Y7B )3D)9)9& wt|++ ..$KtyKK$(LKK>EKKK ),M4;N(O(O E E $L*  , ,\8 D D D D E ErNc6g}||||SrG)rP)rJrrLs rK flatten_up_tozTreeSpec.flatten_up_to7s"!# ""4222rNleavesc t|ttfst|}t||jkr+t dt|d|jd|d|r|dSt|jj }d}d}g}|j D]>}||jz }| | ||||}?|||j S)Nz0treespec.unflatten(leaves): `leaves` has length z, but the spec refers to a pytree that holds z items (z).r)rbrrr#r4rr*rqrkrmr2r unflattenr)rJr^rmr:endrRr[s rKr`zTreeSpec.unflatten<s!&4-00 "&\\F v;;$/ ) )#3v;;##>Bo#####  <<>> !9 &ty1>  -  J :( (C  !5!5fU3Y6G!H!H I I IEE|M4<888rNrENr)rWrXrYrrorr dataclassesrr3rZr4r5r>r]rEr\r*rrPr]rr`rHrNrKr%r%sz III $$$$&[&E222Is222'k'U333J333) )u555L#555????))s)3)))) <<<<<JE&JEDLJETJEJEJEJEX&T&\ 9 9&999999rNc:eZdZdfd ZddZd dedefdZxZS) r&rENcNtddgdSrG)rd__init__)rJrfs rKrgzLeafSpec.__init__Ws% tR(((((rNct|ddt|ddt|dddS)Nr3rAr4r5r)r[r=rIs rKr>zLeafSpec.__post_init__ZsP4a0004q111433333rNrr?cdSN*rH)rJr?s rKrEzLeafSpec.__repr___srNrbrc) rWrXrYrgr>rZr]rErgrhs@rKr&r&Vsx))))))4444 s3rNr&r^ct|r|tSt|}t|j}||\}}fd|D}t |||S)Nr*c4g|]}t|S)rn_tree_flatten_helper)rrFr*r^s rKrz(_tree_flatten_helper..vs6AFUFG<<<rN)r-r _LEAF_SPECr)rqrlr%)rr^r*rQrlrRrr2s `` rKrqrqhs g&&& dt$$I +6J'Z--M7JWN Iw 7 77rNc2g}t|||}||fS)zkFlattens a pytree into a list of values and a TreeSpec that can be used to reconstruct the pytree. rnrp)rr*r^r9s rKr*r*}s'F fg > > >D 4<rNtreespecct|ts tdt|d||S)zqGiven a list of values and a TreeSpec, builds a pytree. This is the inverse operation of `tree_flatten`. zftree_unflatten(leaves, treespec): Expected `treespec` to be instance of TreeSpec but got item of type r)rbr% TypeErrorrkr`)r^rts rKr,r,s_ h ) )  K9=h K K K      f % %%rNc#Kt||r|VdSt|}t|j}||\}}|D]}t ||Ed{VdS)z,Get an iterator over the leaves of a pytree.rnN)r-r)rqrlr-)rr*rQrlrRrrFs rKr-r-s g&&& 9 "4(( $Y/: %:d++ q# 9 9E 888 8 8 8 8 8 8 8 8 9 9rNc>tt||S)z!Get a list of leaves of a pytree.rn)rr-r,s rKr.r.s  $000 1 11rNc0t||dS)zGet the TreeSpec for a pytree.rnrA)r*r,s rKr0r0s g . . .q 11rNrnfuncrestsct||\}|gfd|Dz}t|g|RS)aMap a multi-input function over pytree args to produce a new pytree. See also :func:`tree_map_`. >>> tree_map(lambda x: x + 1, {'x': 7, 'y': (42, 64)}) {'x': 8, 'y': (43, 65)} >>> tree_map(lambda x: x is None, {'x': 7, 'y': (42, 64), 'z': None}) {'x': False, 'y': (False, False), 'z': True} If multiple inputs are given, the structure of the tree is taken from the first input; subsequent inputs need only have ``tree`` as a prefix: >>> tree_map(lambda x, y: [x] + y, [5, 6], [[7, 9], [1, 2]]) [[5, 7, 9], [6, 1, 2]] Args: func (callable): A function that takes ``1 + len(rests)`` arguments, to be applied at the corresponding leaves of the pytrees. tree (pytree): A pytree to be mapped over, with each leaf providing the first positional argument to function ``func``. rests (tuple of pytree): A tuple of pytrees, each of which has the same structure as ``tree`` or has ``tree`` as a prefix. is_leaf (callable, optional): An extra leaf predicate function that will be called at each flattening step. The function should have a single argument with signature ``is_leaf(node) -> bool``. If it returns :data:`True`, the whole subtree being treated as a leaf. Otherwise, the default pytree registry will be used to determine a node is a leaf or not. If the function is not specified, the default pytree registry will be used. Returns: A new pytree with the same structure as ``tree`` but with the value at each leaf given by ``func(x, *xs)`` where ``x`` is the value at the corresponding leaf in ``tree`` and ``xs`` is the tuple of values at corresponding nodes in ``rests``. rnc:g|]}|SrHr]rrrts rKrztree_map..'EEE!H22155EEErN)r*r`maprzrr*r{r^ flat_argsrts @rKr1r1sbN$D':::FHEEEEuEEEEI   c$3333 4 44rNct||\}|gfd|Dz}tt|g|Rd|S)aTLike :func:`tree_map`, but do an inplace call on each leaf and return the original tree. See also :func:`tree_map`. Args: func (callable): A function that takes ``1 + len(rests)`` arguments, to be applied at the corresponding leaves of the pytrees. tree (pytree): A pytree to be mapped over, with each leaf providing the first positional argument to function ``func``. rests (tuple of pytree): A tuple of pytrees, each of which has the same structure as ``tree`` or has ``tree`` as a prefix. is_leaf (callable, optional): An extra leaf predicate function that will be called at each flattening step. The function should have a single argument with signature ``is_leaf(node) -> bool``. If it returns :data:`True`, the whole subtree being treated as a leaf. Otherwise, the default pytree registry will be used to determine a node is a leaf or not. If the function is not specified, the default pytree registry will be used. Returns: The original ``tree`` with the value at each leaf is given by the side-effect of function ``func(x, *xs)`` (not the return value) where ``x`` is the value at the corresponding leaf in ``tree`` and ``xs`` is the tuple of values at values at corresponding nodes in ``rests``. rnc:g|]}|SrHr~rs rKrztree_map_..rrNrr)r*rrrs @rKr3r3se8$D':::FHEEEEuEEEEI #d Y   **** KrN __type_or_types_or_predcdSrGrHrs rKmap_onlyrCrNcdSrGrHrs rKrrrrNcdSrGrHrs rKrrrrNcdSrGrHrs rKrr%rrNcdSrGrHrs rKrr*rrNcvtttfs*tjdkr/tt jrdtdtffd n!trntddttgtfdttgtfffd }|S)a Suppose you are writing a tree_map over tensors, leaving everything else unchanged. Ordinarily you would have to write: def go(t): if isinstance(t, Tensor): return ... else: return t With this function, you only need to write: @map_only(Tensor) def go(t): return ... You can also directly use 'tree_map_only' rxrEc$t|SrGrb)rrs rKpredzmap_only..predIsa!899 9rNz9Argument must be a type, a tuple of types, or a callable.rzchtjdtdtffd }|S)NrrEc4|r |S|SrGrH)rrzrs rKwrappedz*map_only..wrapper..wrappedRs%tAww tAwwHrN) functoolswrapsr=r)rzrrs` rKwrapperzmap_only..wrapperQsQ    q S          rN) rbrkrsys version_infotypes UnionTyperr\callablervr r=)rrrs` @rKrr/s*)D%=99 U G## . @ @ $ :C :D : : : : : : : ) * *U&STTThsCx(XseSj-A NrNcdSrGrHrrzrr*s rKr4r4] CrNcdSrGrHrs rKr4r4grrNcdSrGrHrs rKr4r4qrrNcdSrGrHrs rKr4r4{rrNcRtt||||SNrn)r1rrs rKr4r4s- 5H455d;;T7 S S SSrNcdSrGrHrs rKr5r5rrNcdSrGrHrs rKr5r5rrNcdSrGrHrs rKr5r5rrNcdSrGrHrs rKr5r5rrNcRtt||||Sr)r3rrs rKr5r5s- 6X566t<.:LL1Z?-K-KLttAwwLLLLLLrN)r-r$rrrr*rs`` rKr8r8> $000I LLLLL LLL L LLrNcdSrGrHrs rKr9r9rrNcdSrGrHrs rKr9r9rrNcdSrGrHrs rKr9r9 rrNc`t||}tfd|DS)Nrnc3NK|]}t||V dSrGrrs rKrz tree_any_only..rrN)r-rrs`` rKr9r9rrNct|tsJt||r |g|jzS|rdSt |}||jkrdSt|j}||\}}t||j ks ||j krdSg}t||j D]!\}} t|| |} | || z }dS|Sr)rbr%r-r4r*r)rkrqrlr#r5rrr2_broadcast_to_and_flatten) rrtr*rQrlrRctxresultrFr[flats rKrr)s h ) ))) )g&&&,v+++tt$$IHM!!t +6J#D))M3 =X222cX=M6M6MtF 0GHHz( GLLL   dNFF44 MrNcJeZdZUdZeeed<eed<eded<dS)_TreeSpecSchemaa _TreeSpecSchema is the schema used to serialize the TreeSpec It contains the following fields: - type: A string name of the type. null for the case of a LeafSpec. - context: Any format which is json dumpable - children_spec: A list of children serialized specs. rkr children_specN) rWrXrY__doc__rr]ror"rrHrNrKrrKsL 3- )******rNrcHeZdZUeegefed<eegefed<dS) _ProtocolFntreespec_to_jsonjson_to_treespecN)rWrXrYr r%r"rorHrNrKrrZsCz?:;;;;0(:;;;;;;rNr_SUPPORTED_PROTOCOLSc|rtddgS|jtvrt d|jdt|j}|j}|t krt d|jd|jD tj |j t}n<#t$r}td|d}~wwxYw||j }d|j D}t|||S)Nz Serializing  in pytree is not registered.z%No registered serialization name for zZ found. Please update your _register_pytree_node call with a `serialized_type_name` kwarg.)r~zUnable to serialize context. Please make the context json dump-able, or register a custom serializer using _register_pytree_node.c,g|]}t|SrH)_treespec_to_json)rrFs rKrz%_treespec_to_json..s!SSS%&u--SSSrN)r*rrkrxrrurBrvjsondumpsrr_rvr2)rtrruserialized_contexte child_schemass rKrrbsT/tT2...}666! G8= G G G   4HMB-B<<<! aHM a a a   -5 !%H,<+!N!N!N     A   0CCHDTUUSS8;RSSSM /1C] S SSs B11 C;C  C json_schemac|d(|d t|ddkrtS|dtvrtd|ddt|d}t|}|j> t j|d}n=#t$r}td|d}~wwxYw||d}d|dD}t|||S) NrkrrrzDeserializing rzUnable to deserialize context. Please make the context json load-able, or register a custom serializer using _register_pytree_node.c,g|]}t|SrH)_json_to_treespec)r child_strings rKrz%_json_to_treespec..s.,8,''rN) r#rrryrrxrwrloadsrvr%)rrtrrexr2s rKrrs9F#  " *  O, - - 2 26"@@@! O[0 O O O    )V)< =C3C8/7 jY!788GG   A   %::;y;QRR>{KKK  DH D D $%9%>%>%@%@ A A D D  rNceZdZdefdZdS) _DummyLeafrEcdSrjrHrIs rKrEz_DummyLeaf.__repr__rlrNN)rWrXrYr]rErHrNrKrrs/#rNrcxtdt|jD|}t|S)Nc*g|]}tSrH)r)rrs rKrz#treespec_pprint..s:::!:::rN)r,ranger4r0)rt dummy_trees rKr<r<s@::uX%899:::J   rNzC`pytree_to_str` is deprecated. Please use `treespec_dumps` instead.c t|SrG)r:)rts rK pytree_to_strrs ( # ##rNzC`str_to_pytree` is deprecated. Please use `treespec_loads` instead.rc t|SrG)r;)rs rK str_to_pytreers $  rNrrcg}|D]$}|t|%|D]$}|t|%|S)zpGet a flat list of arguments to this function A slightly faster version of tree_leaves((args, kwargs)) )extendr-r)rrr^as rKarg_tree_leavesrsj F $$ ill#### ]]__$$ ill#### MrNcht||\}}ttd|||fS)aFlattens a pytree like :func:`tree_flatten`, but also returns each leaf's key path. Args: tree: a pytree to flatten. If it contains a custom type, that type must be registered with an appropriate `tree_flatten_with_path_fn` when registered with :func:`register_pytree_node`. is_leaf: An extra leaf predicate function that will be called at each flattening step. The function should have a single argument with signature ``is_leaf(node) -> bool``. If it returns :data:`True`, the whole subtree being treated as a leaf. Otherwise, the default pytree registry will be used to determine a node is a leaf or not. If the function is not specified, the default pytree registry will be used. Returns: A tuple where the first element is a list of (key path, leaf) pairs, and the second element is a :class:`TreeSpec` representing the structure of the flattened tree. rH)r*r_generate_key_paths)rr*rrts rKr+r+s7(tW--KAx #Bg66 7 7 AArNc>ttd||S)a8Gets the leaves of a pytree like ``tree_leaves`` and returns each leaf's key path. Args: tree: a pytree. If it contains a custom type, that type must be registered with an appropriate `tree_flatten_with_path_fn` when registered with :func:`register_pytree_node`. is_leaf: An extra leaf predicate function that will be called at each flattening step. The function should have a single argument with signature ``is_leaf(node) -> bool``. If it returns :data:`True`, the whole subtree being treated as a leaf. Otherwise, the default pytree registry will be used to determine a node is a leaf or not. If the function is not specified, the default pytree registry will be used. Returns: A list of (key path, leaf) pairs. rH)rrr,s rKr/r/s$ #Bg66 7 77rNkey_pathc#6K|r||r||fVdSt|}t|}|s||fVdS|j}|r4||\}}|D]!\}} t g||R| |Ed{V"dSt d|d)Nz.Did not find a flatten_with_keys_fn for type: zF. Please pass a flatten_with_keys_fn argument to register_pytree_node.)r)rqrVrnrr) rrr*rQhandlerflatten_with_keys key_childrenrrcs rKrr&s 774==nt$$I!!),,G n4  ++D11 a  G GDAq*>X>q>>1gFF F F F F F F F F G G SY S S S   rNct||\}tt|}|fd|Dz}fdt|DS)aLike :func:`tree_map`, but the provided callable takes an additional key path argument. Args: func: A function that takes ``2 + len(rests)`` arguments, to be applied at the corresponding leaves of the pytrees. The first positional argument to ``func`` is the key path of the leaf in question. The second positional argument is the value of the leaf. tree: A pytree to be mapped over, with each leaf providing the first positional argument to function ``func``. rests: A tuple of pytrees, each of which has the same structure as ``tree`` or has ``tree`` as a prefix. is_leaf: An extra leaf predicate function that will be called at each flattening step. The function should have a single argument with signature ``is_leaf(node) -> bool``. If it returns :data:`True`, the whole subtree being treated as a leaf. Otherwise, the default pytree registry will be used to determine a node is a leaf or not. If the function is not specified, the default pytree registry will be used. Returns A new pytree with the same structure as ``tree`` but with the value at each leaf given by ``func(keypath, x, *xs)`` where ``keypath`` is the key path at the corresponding leaf in ``tree``, ``x`` is the value at that leaf, and ``xs`` is the tuple of values at corresponding nodes in ``rests``. c:g|]}|SrHr~rs rKrz&tree_map_with_path..bs'*T*T*T8+A+A!+D+D*T*T*TrNc3"K|] }|V dSrGrH)rxsrzs rKrz%tree_map_with_path..cs)KKBddBiKKKKKKrN)r+rrr`)rzrr*r{keypath_leavesall_keypath_leavesrts` @rKr2r2Cs{: 6dGDDNH#~.//N'*T*T*T*Te*T*T*TT   KKKK#7I2JKKK K KKrNkpc@dd|DS)z9Given a key path, return a pretty-printed representation.rc,g|]}t|SrH)r])rrs rKrzkeystr..hs'''qCFF'''rN)rG)r s rKr'r'fs# 77''B''' ( ((rNr`c:|D]}||}|S)zAGiven an object and a key path, return the value at the key path.)rV)r`r rs rKr(r(ks' eeCjj JrN)NNrG)rrdrrimportlib.metadatarr threadingrr collectionsrrrrenumrtypingrr r r r r rrrrrrrrGenericOrderedDictrrrrrrrtyping_extensionsr__all__r=r>r?r@rrBrD JSONEncoderr_rrr r!r"r#r$r] ToStrFuncMaybeFromStrFuncKeyPathrprjLockrrqrorsrxrymetadataversion_optree_versionrtorch._vendor.packaging.versionr{_cxx_pytree_dynamo_traceablerrPackageNotFoundErrorrr|r)rrr dataclassrrrrrrrrrrrrrrrrrrrrr_odict_flatten_odict_unflattenrrrr rrrrrrr frozensetrrkrr\r'r)r-r%r&rrrqr*r,r-r.r0r1r3Type2Type3rrTypeAnyFn2Fn3FnFnAny MapOnlyFnrr4r5r6r7r8r9rrrrrZrrr:r;rr<rrrr+r/rr2r'r(rHrNrKr.s)#"  CCCCCCCCCCCC0)(((((   F GCLL GCLL GCLL GCLL+,' ?      x    $$$$$$"$$$  xtCy''9!::; (3-169: y/9: /!2G!;< j$s),c1 2 SE8E#w2C,D#EEF # xtE(C-4H/I3/N)OOP88888j888%in&&,.d3i()...;;;;; ;;;BDDc,=!=>CCC79S$s)^ 4999 (00::O777777#*7?#;#;wwx?P?P#P '   j      .>>>8== #5#5#5>")+T#Y++++/9==A:>E;E;E; cE;E; E; #3- E; ""56 E;$$9:E;##67E; E;E;E;E;P c 8 E &*48 4+/9==A:>444 c44 4 " 4  01 4#3-4""564$$9:4##674 444  4x+/9==A:>)C)C)C c)C)C )C #3- )C ""56 )C$$9:)C##67)C )C)C)C)CXd###"""""'!*""$#" GCx   d###!!!!!A!!$#!d###'''''''$#'eCHo%S 70B*CH S#XH 4hm$ %w ./HHHHXc]WsCxT#Y5cG);#<HtCyHU4hm8L3Mw3V-WHHHH HSMGS ,T#s(^,d3i.@(A,,,,J CH~J 4hm$ %w ./JJJJ&HSM&G&S#X&&&&:%S 70B*C 4hm$ %w ./.(3-.'.j.... 7     (o',.sCx8,U49gCU=V,,,,J#s(#J 4hm$ %w ./JJJJL SML LS!LLLL&)5KS15eDIwH SzH 4hm$ %w ./HHHH)Xc])W)s)))) )1 (0 (0 1-1627 2.27 ,1 (1y ; $((Yt_"+ D$ KeD"" y 7#7$77776HXvhn-E$FRVD$UCCCL9L9L9L9L9L9L9DCL9^     x   XZZ 3788 8 I8hx~. /8 8888.37     hx~. /  49h      &8C= &H & & & & &3799 9 hx~. /9c]9999&3722 2 hx~. /2 #Y22223722 2 hx~. /2222237 )5)5)5 38 )5 )5 )5hx~. / )5  )5)5)5)5`37  38   hx~. /   D d1gtAw d1gtAwQ'(wDIuT#Y^4eoEFGGDIuT#Y^445Gad }a aAg"# qc1f# aS(C5#:.. /  eAqDkiAq#I6O  eAq!Gn3q!QPS|CT9U  d1g)Bq#vJ2G   g)E#J2G  hud{&; %PS*@U +"7HcUD[,A#AB+uSz++++\ 37 !!W QV* hx~. /     37 "1a4[ aCi. hx~. /     37 "1a7^ aAsl  hx~. /     37 %seTk2 * hx~. /    37 TT"7HcUD[,A#ABT *T Thx~. / T  TTTT 37 !!W QV* hx~. /     37 "1a4[ aCi. hx~. /     37 "1a7^ aAsl  hx~. /     37 %seTk2 * hx~. /    37 UU"7HcUD[,A#ABU *U Uhx~. / U  UUUU37%% C5$; % %hx~. /% %%%%37%% C5$; % %hx~. /% %%%% 37 !W QW+ hx~. /     37 1a4[ aDj/ hx~. /     37 1a7^ aAtm  hx~. /    37 MMM +M Mhx~. / M  MMMM 37 !W QW+ hx~. /     37 1a4[ aDj/ hx~. /     37 1a7^ aAtm  hx~. /    37 MMM +M Mhx~. / M  MMMM*37 hx~. /d3i D  + + + + + + + +<<<<<*<<< 02d3 +,111!T!T_!T!T!T!TH 2? 2x 2 2 2 2F&+&79JKKQX##,sx h3 I $H$$$$ $  I        6 V S    37BB B hx~. /B 4gsl# $h ./BBBB43788 8 hx~. /8 % 8888037    hx~. / eGSL!"     B37 L L L 38  L  L  Lhx~. / L  L L L LF)w)3)))) 'csHH32H3