*j;ddlmZddlZddlZddlZddlmZddlmZm Z ddl m Z ddl m ZddlmZddlmZdd lmZdd lmZmZmZmZejeZeGd d eZdS) ) annotationsN)Callable) dataclassfield)Union)parse)TrainingArguments) __version__) ParallelMode) BatchSamplersDefaultBatchSamplerMultiDatasetBatchSamplersMultiDatasetDefaultBatchSamplercTeZdZUdZgdZedddiZded<eej dd iZ d ed <ee j dd iZ d ed<eeddiZded<eeddiZded<edddiZded<fdZeddZfdZxZS)BaseTrainingArgumentsa BaseTrainingArguments extends :class:`~transformers.TrainingArguments` with additional arguments specific to Sentence Transformers. See :class:`~transformers.TrainingArguments` for the complete list of available arguments. Args: output_dir (`str`): The output directory where the model checkpoints will be written. prompts (`Union[Dict[str, Dict[str, str]], Dict[str, str], str]`, *optional*): The prompts to use for each column in the training, evaluation and test datasets. Four formats are accepted: 1. `str`: A single prompt to use for all columns in the datasets, regardless of whether the training/evaluation/test datasets are :class:`datasets.Dataset` or a :class:`datasets.DatasetDict`. 2. `Dict[str, str]`: A dictionary mapping column names to prompts, regardless of whether the training/evaluation/test datasets are :class:`datasets.Dataset` or a :class:`datasets.DatasetDict`. 3. `Dict[str, str]`: A dictionary mapping dataset names to prompts. This should only be used if your training/evaluation/test datasets are a :class:`datasets.DatasetDict` or a dictionary of :class:`datasets.Dataset`. 4. `Dict[str, Dict[str, str]]`: A dictionary mapping dataset names to dictionaries mapping column names to prompts. This should only be used if your training/evaluation/test datasets are a :class:`datasets.DatasetDict` or a dictionary of :class:`datasets.Dataset`. batch_sampler (Union[:class:`~sentence_transformers.base.sampler.BatchSamplers`, `str`, :class:`~sentence_transformers.base.sampler.DefaultBatchSampler`, Callable[[...], :class:`~sentence_transformers.base.sampler.DefaultBatchSampler`]], *optional*): The batch sampler to use. See :class:`~sentence_transformers.base.sampler.BatchSamplers` for valid options. Defaults to ``BatchSamplers.BATCH_SAMPLER``. multi_dataset_batch_sampler (Union[:class:`~sentence_transformers.base.sampler.MultiDatasetBatchSamplers`, `str`, :class:`~sentence_transformers.base.sampler.MultiDatasetDefaultBatchSampler`, Callable[[...], :class:`~sentence_transformers.base.sampler.MultiDatasetDefaultBatchSampler`]], *optional*): The multi-dataset batch sampler to use. See :class:`~sentence_transformers.base.sampler.MultiDatasetBatchSamplers` for valid options. Defaults to ``MultiDatasetBatchSamplers.PROPORTIONAL``. router_mapping (`Dict[str, str] | Dict[str, Dict[str, str]]`, *optional*): A mapping of dataset column names to Router routes, like "query" or "document". This is used to specify which Router submodule to use for each dataset. Two formats are accepted: 1. `Dict[str, str]`: A mapping of column names to routes. 2. `Dict[str, Dict[str, str]]`: A mapping of dataset names to a mapping of column names to routes for multi-dataset training/evaluation. learning_rate_mapping (`Dict[str, float] | None`, *optional*): A mapping of parameter name regular expressions to learning rates. This allows you to set different learning rates for different parts of the model, e.g., `{'SparseStaticEmbedding\.*': 1e-3}` for the SparseStaticEmbedding module. This is useful when you want to fine-tune specific parts of the model with different learning rates. )accelerator_config fsdp_config deepspeedgradient_checkpointing_kwargslr_scheduler_kwargslearning_rate_mappingpromptsrouter_mappingNhelpzThe prompts to use for each column in the datasets. Either 1) a single string prompt, 2) a mapping of column names to prompts, 3) a mapping of dataset names to prompts, or 4) a mapping of dataset names to a mapping of column names to prompts.)defaultmetadataz;Union[str, None, dict[str, str], dict[str, dict[str, str]]]rzThe batch sampler to use.zRUnion[BatchSamplers, str, DefaultBatchSampler, Callable[..., DefaultBatchSampler]] batch_samplerz'The multi-dataset batch sampler to use.zvUnion[MultiDatasetBatchSamplers, str, MultiDatasetDefaultBatchSampler, Callable[..., MultiDatasetDefaultBatchSampler]]multi_dataset_batch_samplerzA mapping of dataset column names to Router routes, like "query" or "document". Either 1) a mapping of column names to routes or 2) a mapping of dataset names to a mapping of column names to routes for multi-dataset training/evaluation. )default_factoryrrzA mapping of parameter name regular expressions to learning rates. This allows you to set different learning rates for different parts of the model, e.g., {'SparseStaticEmbedding\.*': 1e-3} for the SparseStaticEmbedding module.z"Union[str, None, dict[str, float]]rcTtttdkrdndS)N5.0.0) parse_versiontransformers_versionVC:\Projects\DSCons\.venv\Lib\site-packages\sentence_transformers\base\training_args.pyzBaseTrainingArguments.vs' 6J(K(K}]dOeOe(e(eknr&zThis argument is deprecated and will be removed in the future. If you're on Transformers v5+, then you should use `warmup_steps` instead as it also works with float values.z float | None warmup_ratioctttdkr@|j8|jdkr-|j|_d|_tdnMt |jtr3d|jcxkrdkr!nn|jdkr|j|_d|_t t |j trt|j n|j |_ t |j trt|j n|j |_ t |jtr5 t!j|j|_n#t j$rYnwxYw|j|jni|_t |jtrB t!j|j|_n"#t j$rt)dwxYw|j|jni|_t |jtrB t!j|j|_n"#t j$rt)dwxYwd|_d |_|jt2jkr&|jd krtd nH|jt2jkr3|js,|jd krtd d|_|jd krB|jdkr9|js4| r"td dSdSdSdSdS)Nr!raThe `warmup_ratio` argument is deprecated in Transformers v5+, and will also be removed from Sentence Transformers once support for Transformers v4 is dropped. Since you're using Transformers v5+, please use `warmup_steps` (as a float) to specify the warmup ratio instead.r"g?zThe `learning_rate_mapping` argument must be a dictionary mapping parameter name regular expressions to learning rates. A stringified dictionary also works.zThe `router_mapping` argument must be a dictionary mapping dataset column names to Router routes, like 'query' or 'document'. A stringified dictionary also works.TFunusedzCurrently using DataParallel (DP) for multi-gpu training, while DistributedDataParallel (DDP) is recommended for faster training. See https://sbert.net/docs/sentence_transformer/training/distributed.html for more information.zWhen using DistributedDataParallel (DDP), it is recommended to set `dataloader_drop_last=True` to avoid hanging issues with an uneven last batch. Setting `dataloader_drop_last=True`.aWhen using `dataloader_num_workers > 0` with spawned worker processes, it is recommended to also set `dataloader_persistent_workers=True`. Every spawned worker imports Sentence Transformers before it can produce a batch, which takes several seconds per worker, and without persistent workers that cost is paid again on every epoch and every evaluation. This commonly makes training slower than `dataloader_num_workers=0`.)!r#r$r) warmup_stepsloggerwarning isinstancefloatsuper __post_init__rstrr rrrjsonloadsJSONDecodeErrorr ValueErrorrprediction_loss_onlyddp_broadcast_buffers parallel_moder NOT_DISTRIBUTED output_dir DISTRIBUTEDdataloader_drop_lastdataloader_num_workersdataloader_persistent_workers_workers_are_spawned)self __class__s r'r2z#BaseTrainingArguments.__post_init__}s - . .-2H2H H H ,1Ba1G1G$($5!$(!t$+U33 &d>O8U8U8U8URU8U8U8U8U8UZ^ZkorZrZr$($5!$%! 2rgsT"""""" $$$$$$((((((((444444KKKKKK<<<<<<333333  8 $ $ g"g"g"g"g"9g"g" g"g"g"r&