[jddlmZddlmZdggddggddZer&d d lmZd dlmZm Z m Z d d l m Z d dlm Z mZmZmZmZd Sdd lZeeed eeeje<d S)) TYPE_CHECKING) _LazyModuleDistributedConfig)is_fsdp_enabledis_fsdp_managed_moduleverify_fsdp_planDistributedMixin)gather_full_state_dict)initialize_fully_sharded_data_parallelismload_optimizer_distributed!save_model_checkpoint_distributedsave_optimizer_distributed)configuration_utilsfsdpmixinutils)r)r N__file__) module_spec)typingrrr_import_structurerrrrrr rr r r r rrsys__name__globals__spec__modulesOC:\Projects\DSCons\.venv\Lib\site-packages\transformers\distributed\__init__.pyr!sG! 00 M M M !  rPOOOOOOOOO''''''JJJ'K''))J2GIZhpqqqCKr