NjrdZddlZddlmZddlmZddlmZmZ dej de fdZ dej de e ee ej ffdZdej de efd Zdej de fd ZdS) z NOTE: This file must be imported like ``import torch.distributed.fsdp._traversal_utils`` and not like ``from torch.distributed.fsdp._traversal_utils import ...`` to avoid circular imports. For brevity, we may import the file as ``traversal_utils``. N _get_registry) _FSDPState_get_module_fsdp_statemodulereturnc0t|}|dSd|vS)zA Returns if ``module`` can compose with ``fully_shard``. NT replicater)rregistrys UC:\Projects\DSCons\.venv\Lib\site-packages\torch\distributed\fsdp\_traversal_utils.py _composabler "s' V$$Ht h &&c*g}g}t}t}tj|g}|r|}||t |s;t t|D]}||vr| |t|}|C||vr?||| || ||||fS)a Returns a tuple containing: 1. A list of the ``_FSDPState`` instances in the module tree rooted at ``module`` without any duplicates and following the ``module.modules()`` traversal order (which is assumed to be depth-first). 2. A corresponding list of the modules owning the states in the first list. For the wrapper code path, both returned lists are the same, each containing all ``FullyShardedDataParallel`` instances. For the composable code path, this returns a list of all composable state instances and a list of the corresponding fully sharded modules. See [Note: Fully Sharded Module]. NOTE: The traversal does not proceed into any module annotated by an incompatible API (e.g. ``replicate``). ) set collectionsdequepopleftaddr reversedlistchildren appendleftrappend) r fsdp_states fsdp_modulesvisited_fsdp_statesvisited_modulesr submodule child_moduleoptional_states r _get_fsdp_states_with_modulesr!0s1&%'K$&L,/55'*eeO +6*;VH*E*EE  +MMOO I&&&9%%  $T)*<*<*>*>%?%?@@ / /L?22  .../ ::  %.@S*S*S  # #N 3 3 3   ~ . . .    * * *  +  $$rc*t|\}}|S)z*See :func:`_get_fsdp_states_with_modules`.)r!)rr_s r _get_fsdp_statesr$`s26::NK rc8dt|D}|S)z Returns all ``FlatParamHandle`` s in the module tree rooted at ``module`` following the rules in :func:`_get_fsdp_state`. c*g|]}|j |jS)N)_handle).0 fsdp_states r z%_get_fsdp_handles..ks.    )  ) ) )r)r$)rhandless r _get_fsdp_handlesr,fs/ *622G Nr)__doc__rtorch.nnnn&torch.distributed._composable.contractr$torch.distributed.fsdp._common_utilsrrModuleboolr tuplerr!r$r,rr r6s@@@@@@SSSSSSSS&' 'd''''-% I-% 4 T")_ ,--%-%-%-%`RY4 +; bi D      r