@j ddlmZddlmZmZmZmZddlmZddl m Z m Z gZ eGddZ eGdd e Ze d id d d gddddddddddddddddddddd d!dd"dd#d$d%d&d'dd()Zd*e_ed+id d d gddddddddddddddddddddd d!dd"dd#d$d%d,e jd'dd(-Zd.e_ed/id d d gddddddddddddddddddddd d!dd"dd#d$d%d,e jd'dd(0Zd1e_ed2id d d gddddddddddddddddddddd d!dd"dd#d$d%d,e jd'dd(0Zd3e_e d4id d d gddddd5dddddddd6dddddd7dd d!d d"dd#d8d%d&d'dd(9Zd:e_ed;id d d gddddd5dddddddd6dddddd7dd d!d d"dd#d8d%d,e jd'dd(0Zde_ed?id d d gddddd5dddddddd6dddddd7dd d!d d"dd#d8d%d,e jd'dd(0Zd@e_e dAid dBd gdddCdd5dddddddd6dddddd7dd d!d d"dCd#d d%d&d'dCd(9ZdDe_edEid dBd gdddCdd5dddddddd6dddddd7dd d!d d"dCd#d d%d,e jd'dCd(0ZdFe_edGid dBd gdddCdd5dddddddd6dddddd7dd d!d d"dCd#d d%d,e jd'dCd(0ZdHe_edIid dBd gdddCdd5dddddddd6dddddd7dd d!d d"dCd#d d%d,e jd'dCd(0ZdJe_e dKid dBd gdddCdd5dd dddddd6dddd dd7dd d!d d"dCd#d d%d&d'dCd(9ZdLe_e dMid d d gddddddddddddddddddddd d!dd"dd#d$d%d&d'dd(9ZdNe_e dOid dBd gddddd5dd dddddd6dddd dd7dd d!d d"dCd#d d%d&d'dCd(9ZdPe_e dQid dBd gdddddRdd ddddddSdddd ddTdd d!d d"dCd#d d%d&d'dCd(9Z dUe _edVid dBd gddddd5dd dddddd6dddd dd7ddd!d d"dCd#dd%d,e jd'dCd(0Z!dWe!_edXid dBd gdddddRdd ddddddSdddd ddTddd!d d"dCd#dd%d,e jd'dCd(0Z"dYe"_edZid d d gddddddd dddddddddd ddddd!d d"dd#dd%d[e j#d'dd\d(]Z$d^e$_ed_id d d gddddddd dddddddddd ddddd!d d"dd#dd%d`e j%d'ddad(]Z&dbe&_edcid d d gddddddd dddddddddd ddddd!d d"dd#dd%dde j'd'dd\d(]Z(dee(_edfid d d gddddddd dddddddddd ddddd!d d"dd#dd%dge j)d'dd(0Z*dhe*_ediid d d gddddddd dddddddddd ddddd!d d"dd#dd%dje j+d'ddkd(]Z,dle,_e dmid d d gddddddddddddddddndodpdqdddddd d!dd"dd#d$d%d&idrd'dsZ-dte-_e duid d d gddddddddddddddddndodpdqdddddd d!dd"dd#d$d%d&idrd'dsZ.dve._e dwid dBd gddddd5dddddddd6dddndodpdqdddd7dd d!dd"dCd#d$d%d&idrd'dCsZ/dxe/_e dyid dBd gdddCdd5dd dddddd6dddd dd7dd d!d d"dCd#d d%d&d(d'dCsZ0dze0_e d{id dBd gdddCddRddddddddSdddd ddTdd d!d d"dCd#d d%d&d(d'dCsZ1d|e1_e d}id dBd gdddCdd~ddddddddSdddd dddd d!d d"dCd#d d%d&d(d'dCsZ2de2_eGddeZ3e3did dBd gdddCdd5dd dddddd6dddd dd7ddd!d d"dCd#dd%d`e j4d'dCd(0Z5de5_d&S)) dataclass)AnyDictOptionalTuple)Module)alignerutilsceZdZUdZeed<eeefed<eed<e ed<eed<e defdZ d Z d d de fd Zd S) Wav2Vec2BundleuData class that bundles associated information to use pretrained :py:class:`~torchaudio.models.Wav2Vec2Model`. This class provides interfaces for instantiating the pretrained model along with the information necessary to retrieve pretrained weights and additional data to be used with the model. Torchaudio library instantiates objects of this class, each of which represents a different pretrained model. Client code should access pretrained models via these instances. Please see below for the usage and the available values. Example - Feature Extraction >>> import torchaudio >>> >>> bundle = torchaudio.pipelines.HUBERT_BASE >>> >>> # Build the model and load pretrained weight. >>> model = bundle.get_model() Downloading: 100%|███████████████████████████████| 360M/360M [00:06<00:00, 60.6MB/s] >>> >>> # Resample audio to the expected sampling rate >>> waveform = torchaudio.functional.resample(waveform, sample_rate, bundle.sample_rate) >>> >>> # Extract acoustic features >>> features, _ = model.extract_features(waveform) _path_params _sample_rate_normalize_waveform _model_typereturnc|jS)zUSample rate of the audio that the model is trained on. :type: float )rselfs TC:\Projects\Historical\venv\Lib\site-packages\torchaudio/pipelines/_wav2vec2/impl.py sample_ratezWav2Vec2Bundle.sample_rate1s   c6tj|j|SN)r _get_state_dictrr dl_kwargss rrzWav2Vec2Bundle._get_state_dict9s$TZ;;;rNrctj|j|j}||}|||jrtj|d}||S)aConstruct the model and load the pretrained weight. The weight file is downloaded from the internet and cached with :func:`torch.hub.load_state_dict_from_url` Args: dl_kwargs (dictionary of keyword arguments): Passed to :func:`torch.hub.load_state_dict_from_url`. Returns: Variation of :py:class:`~torchaudio.models.Wav2Vec2Model`. For the models listed below, an additional layer normalization is performed on the input. For all other models, a :py:class:`~torchaudio.models.Wav2Vec2Model` instance is returned. - WAV2VEC2_LARGE_LV60K - WAV2VEC2_ASR_LARGE_LV60K_10M - WAV2VEC2_ASR_LARGE_LV60K_100H - WAV2VEC2_ASR_LARGE_LV60K_960H - WAV2VEC2_XLSR53 - WAV2VEC2_XLSR_300M - WAV2VEC2_XLSR_1B - WAV2VEC2_XLSR_2B - HUBERT_LARGE - HUBERT_XLARGE - HUBERT_ASR_LARGE - HUBERT_ASR_XLARGE - WAVLM_LARGE T)normalize_waveform) r _get_modelrrrload_state_dictr _extend_modeleval)rrmodel state_dicts r get_modelzWav2Vec2Bundle.get_model=su< !14<@@)))44  j)))  # H'$GGGE  r)__name__ __module__ __qualname____doc__str__annotations__rrfloatboolpropertyrrrr(rrr r s: JJJ #s(^ !U!!!X!<<<&*$$$f$$$$$$rr cxeZdZUdZeedfed<dZeedfed<ddded eedffd Z d Z d S) Wav2Vec2ASRBundleuData class that bundles associated information to use pretrained :py:class:`~torchaudio.models.Wav2Vec2Model`. This class provides interfaces for instantiating the pretrained model along with the information necessary to retrieve pretrained weights and additional data to be used with the model. Torchaudio library instantiates objects of this class, each of which represents a different pretrained model. Client code should access pretrained models via these instances. Please see below for the usage and the available values. Example - ASR >>> import torchaudio >>> >>> bundle = torchaudio.pipelines.HUBERT_ASR_LARGE >>> >>> # Build the model and load pretrained weight. >>> model = bundle.get_model() Downloading: 100%|███████████████████████████████| 1.18G/1.18G [00:17<00:00, 73.8MB/s] >>> >>> # Check the corresponding labels of the output. >>> labels = bundle.get_labels() >>> print(labels) ('-', '|', 'E', 'T', 'A', 'O', 'N', 'I', 'H', 'S', 'R', 'D', 'L', 'U', 'M', 'W', 'C', 'F', 'G', 'Y', 'P', 'B', 'V', 'K', "'", 'X', 'J', 'Q', 'Z') >>> >>> # Resample audio to the expected sampling rate >>> waveform = torchaudio.functional.resample(waveform, sample_rate, bundle.sample_rate) >>> >>> # Infer the label probability distribution >>> emissions, _ = model(waveform) >>> >>> # Pass emission to decoder >>> # `ctc_decode` is for illustration purpose only >>> transcripts = ctc_decode(emissions, labels) ._labelsr _remove_aux_axis-blankr<rc|g|jRS)auThe output class labels. The first is blank token, and it is customizable. Args: blank (str, optional): Blank token. (default: ``'-'``) Returns: Tuple[str, ...]: For models fine-tuned on ASR, returns the tuple of strings representing the output class labels. Example >>> from torchaudio.pipelines import HUBERT_ASR_LARGE as bundle >>> bundle.get_labels() ('-', '|', 'E', 'T', 'A', 'O', 'N', 'I', 'H', 'S', 'R', 'D', 'L', 'U', 'M', 'W', 'C', 'F', 'G', 'Y', 'P', 'B', 'V', 'K', "'", 'X', 'J', 'Q', 'Z') )r5)rr<s r get_labelszWav2Vec2ASRBundle.get_labelss,% %%%rcBtj|j||jSr)r rrr9rs rrz!Wav2Vec2ASRBundle._get_state_dicts$TZDrr2rrr4r4ds%%N38_(1eCHo111 &&&& sCx &&&&0SSSSSrr4zwav2vec2_fairseq_base_ls960.pthextractor_mode group_normextractor_conv_layer_config)) rDr8r7rGrGrGrDr7r7rHextractor_conv_biasFencoder_embed_dimiencoder_projection_dropoutg?encoder_pos_conv_kernelencoder_pos_conv_groupsencoder_num_layers encoder_num_headsencoder_attention_dropoutencoder_ff_interm_featuresi encoder_ff_interm_dropoutgencoder_dropoutencoder_layer_norm_firstencoder_layer_dropg? aux_num_outNi>Wav2Vec2)rrrrraWav2vec 2.0 model ("base" architecture), pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset :cite:`7178964` (the combination of "train-clean-100", "train-clean-360", and "train-other-500"), not fine-tuned. Originally published by the authors of *wav2vec 2.0* :cite:`baevski2020wav2vec` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2Bundle` for the usage. z)wav2vec2_fairseq_base_ls960_asr_ll10m.pth)rrr5rrra9Wav2vec 2.0 model ("base" architecture with an extra linear module), pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset :cite:`7178964` (the combination of "train-clean-100", "train-clean-360", and "train-other-500"), and fine-tuned for ASR on 10 minutes of transcribed audio from *Libri-Light* dataset :cite:`librilight` ("train-10min" subset). Originally published by the authors of *wav2vec 2.0* :cite:`baevski2020wav2vec` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z)wav2vec2_fairseq_base_ls960_asr_ls100.pth)r5rrraWav2vec 2.0 model ("base" architecture with an extra linear module), pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset :cite:`7178964` (the combination of "train-clean-100", "train-clean-360", and "train-other-500"), and fine-tuned for ASR on 100 hours of transcribed audio from "train-clean-100" subset. Originally published by the authors of *wav2vec 2.0* :cite:`baevski2020wav2vec` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z)wav2vec2_fairseq_base_ls960_asr_ls960.pthaWav2vec 2.0 model ("base" architecture with an extra linear module), pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset :cite:`7178964` (the combination of "train-clean-100", "train-clean-360", and "train-other-500"), and fine-tuned for ASR on the same audio with the corresponding transcripts. Originally published by the authors of *wav2vec 2.0* :cite:`baevski2020wav2vec` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z wav2vec2_fairseq_large_ls960.pthiig?)rrraWav2vec 2.0 model ("large" architecture), pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset :cite:`7178964` (the combination of "train-clean-100", "train-clean-360", and "train-other-500"), not fine-tuned. Originally published by the authors of *wav2vec 2.0* :cite:`baevski2020wav2vec` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2Bundle` for the usage. z*wav2vec2_fairseq_large_ls960_asr_ll10m.ptha:Wav2vec 2.0 model ("large" architecture with an extra linear module), pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset :cite:`7178964` (the combination of "train-clean-100", "train-clean-360", and "train-other-500"), and fine-tuned for ASR on 10 minutes of transcribed audio from *Libri-Light* dataset :cite:`librilight` ("train-10min" subset). Originally published by the authors of *wav2vec 2.0* :cite:`baevski2020wav2vec` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z*wav2vec2_fairseq_large_ls960_asr_ls100.ptha%Wav2vec 2.0 model ("large" architecture with an extra linear module), pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset :cite:`7178964` (the combination of "train-clean-100", "train-clean-360", and "train-other-500"), and fine-tuned for ASR on 100 hours of transcribed audio from the same dataset ("train-clean-100" subset). Originally published by the authors of *wav2vec 2.0* :cite:`baevski2020wav2vec` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z*wav2vec2_fairseq_large_ls960_asr_ls960.pthaWav2vec 2.0 model ("large" architecture with an extra linear module), pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset :cite:`7178964` (the combination of "train-clean-100", "train-clean-360", and "train-other-500"), and fine-tuned for ASR on the same audio with the corresponding transcripts. Originally published by the authors of *wav2vec 2.0* :cite:`baevski2020wav2vec` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z wav2vec2_fairseq_large_lv60k.pth layer_normTafWav2vec 2.0 model ("large-lv60k" architecture), pre-trained on 60,000 hours of unlabeled audio from *Libri-Light* dataset :cite:`librilight`, not fine-tuned. Originally published by the authors of *wav2vec 2.0* :cite:`baevski2020wav2vec` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2Bundle` for the usage. z*wav2vec2_fairseq_large_lv60k_asr_ll10m.pthaWav2vec 2.0 model ("large-lv60k" architecture with an extra linear module), pre-trained on 60,000 hours of unlabeled audio from *Libri-Light* dataset :cite:`librilight`, and fine-tuned for ASR on 10 minutes of transcribed audio from the same dataset ("train-10min" subset). Originally published by the authors of *wav2vec 2.0* :cite:`baevski2020wav2vec` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z*wav2vec2_fairseq_large_lv60k_asr_ls100.pthaWav2vec 2.0 model ("large-lv60k" architecture with an extra linear module), pre-trained on 60,000 hours of unlabeled audio from *Libri-Light* dataset :cite:`librilight`, and fine-tuned for ASR on 100 hours of transcribed audio from *LibriSpeech* dataset :cite:`7178964` ("train-clean-100" subset). Originally published by the authors of *wav2vec 2.0* :cite:`baevski2020wav2vec` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z*wav2vec2_fairseq_large_lv60k_asr_ls960.ptha+Wav2vec 2.0 model ("large-lv60k" architecture with an extra linear module), pre-trained on 60,000 hours of unlabeled audio from *Libri-Light* :cite:`librilight` dataset, and fine-tuned for ASR on 960 hours of transcribed audio from *LibriSpeech* dataset :cite:`7178964` (the combination of "train-clean-100", "train-clean-360", and "train-other-500"). Originally published by the authors of *wav2vec 2.0* :cite:`baevski2020wav2vec` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z!wav2vec2_fairseq_large_xlsr53.ptha Wav2vec 2.0 model ("base" architecture), pre-trained on 56,000 hours of unlabeled audio from multiple datasets ( *Multilingual LibriSpeech* :cite:`Pratap_2020`, *CommonVoice* :cite:`ardila2020common` and *BABEL* :cite:`Gales2014SpeechRA`), not fine-tuned. Originally published by the authors of *Unsupervised Cross-lingual Representation Learning for Speech Recognition* :cite:`conneau2020unsupervised` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2Bundle` for the usage. zhubert_fairseq_base_ls960.pthaHuBERT model ("base" architecture), pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset :cite:`7178964` (the combination of "train-clean-100", "train-clean-360", and "train-other-500"), not fine-tuned. Originally published by the authors of *HuBERT* :cite:`hsu2021hubert` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2Bundle` for the usage. zhubert_fairseq_large_ll60k.pthacHuBERT model ("large" architecture), pre-trained on 60,000 hours of unlabeled audio from *Libri-Light* dataset :cite:`librilight`, not fine-tuned. Originally published by the authors of *HuBERT* :cite:`hsu2021hubert` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2Bundle` for the usage. zhubert_fairseq_xlarge_ll60k.pthi0iaiHuBERT model ("extra large" architecture), pre-trained on 60,000 hours of unlabeled audio from *Libri-Light* dataset :cite:`librilight`, not fine-tuned. Originally published by the authors of *HuBERT* :cite:`hsu2021hubert` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2Bundle` for the usage. z(hubert_fairseq_large_ll60k_asr_ls960.ptha HuBERT model ("large" architecture), pre-trained on 60,000 hours of unlabeled audio from *Libri-Light* dataset :cite:`librilight`, and fine-tuned for ASR on 960 hours of transcribed audio from *LibriSpeech* dataset :cite:`7178964` (the combination of "train-clean-100", "train-clean-360", and "train-other-500"). Originally published by the authors of *HuBERT* :cite:`hsu2021hubert` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z)hubert_fairseq_xlarge_ll60k_asr_ls960.pthaHuBERT model ("extra large" architecture), pre-trained on 60,000 hours of unlabeled audio from *Libri-Light* dataset :cite:`librilight`, and fine-tuned for ASR on 960 hours of transcribed audio from *LibriSpeech* dataset :cite:`7178964` (the combination of "train-clean-100", "train-clean-360", and "train-other-500"). Originally published by the authors of *HuBERT* :cite:`hsu2021hubert` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z%wav2vec2_voxpopuli_base_10k_asr_de.pt )r r7r8#)r5rrr9rawav2vec 2.0 model ("base" architecture), pre-trained on 10k hours of unlabeled audio from *VoxPopuli* dataset :cite:`voxpopuli` ("10k" subset, consisting of 23 languages), and fine-tuned for ASR on 282 hours of transcribed audio from "de" subset. Originally published by the authors of *VoxPopuli* :cite:`voxpopuli` under CC BY-NC 4.0 and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z%wav2vec2_voxpopuli_base_10k_asr_en.pt)r r7r8awav2vec 2.0 model ("base" architecture), pre-trained on 10k hours of unlabeled audio from *VoxPopuli* dataset :cite:`voxpopuli` ("10k" subset, consisting of 23 languages), and fine-tuned for ASR on 543 hours of transcribed audio from "en" subset. Originally published by the authors of *VoxPopuli* :cite:`voxpopuli` under CC BY-NC 4.0 and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z%wav2vec2_voxpopuli_base_10k_asr_es.ptr`awav2vec 2.0 model ("base" architecture), pre-trained on 10k hours of unlabeled audio from *VoxPopuli* dataset :cite:`voxpopuli` ("10k" subset, consisting of 23 languages), and fine-tuned for ASR on 166 hours of transcribed audio from "es" subset. Originally published by the authors of *VoxPopuli* :cite:`voxpopuli` under CC BY-NC 4.0 and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z%wav2vec2_voxpopuli_base_10k_asr_fr.pt+awav2vec 2.0 model ("base" architecture), pre-trained on 10k hours of unlabeled audio from *VoxPopuli* dataset :cite:`voxpopuli` ("10k" subset, consisting of 23 languages), and fine-tuned for ASR on 211 hours of transcribed audio from "fr" subset. Originally published by the authors of *VoxPopuli* :cite:`voxpopuli` under CC BY-NC 4.0 and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z%wav2vec2_voxpopuli_base_10k_asr_it.pt%r6awav2vec 2.0 model ("base" architecture), pre-trained on 10k hours of unlabeled audio from *VoxPopuli* dataset :cite:`voxpopuli` ("10k" subset, consisting of 23 languages), and fine-tuned for ASR on 91 hours of transcribed audio from "it" subset. Originally published by the authors of *VoxPopuli* :cite:`voxpopuli` under CC BY-NC 4.0 and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. zwavlm_base.pthencoder_max_distancei encoder_num_bucketsi@WavLM)rrraBWavLM Base model ("base" architecture), pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset :cite:`7178964`, not fine-tuned. Originally published by the authors of *WavLM* :cite:`chen2022wavlm` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2Bundle` for the usage. zwavlm_base_plus.pthaWavLM Base+ model ("base" architecture), pre-trained on 60,000 hours of Libri-Light dataset :cite:`librilight`, 10,000 hours of GigaSpeech :cite:`GigaSpeech2021`, and 24,000 hours of *VoxPopuli* :cite:`voxpopuli`, not fine-tuned. Originally published by the authors of *WavLM* :cite:`chen2022wavlm` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2Bundle` for the usage. zwavlm_large.pthaWavLM Large model ("large" architecture), pre-trained on 60,000 hours of Libri-Light dataset :cite:`librilight`, 10,000 hours of GigaSpeech :cite:`GigaSpeech2021`, and 24,000 hours of *VoxPopuli* :cite:`voxpopuli`, not fine-tuned. Originally published by the authors of *WavLM* :cite:`chen2022wavlm` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2Bundle` for the usage. zwav2vec2_xlsr_300m.ptha(XLS-R model with 300 million parameters, pre-trained on 436,000 hours of unlabeled audio from multiple datasets ( *Multilingual LibriSpeech* :cite:`Pratap_2020`, *CommonVoice* :cite:`ardila2020common`, *VoxLingua107* :cite:`valk2021voxlingua107`, *BABEL* :cite:`Gales2014SpeechRA`, and *VoxPopuli* :cite:`voxpopuli`) in 128 languages, not fine-tuned. Originally published by the authors of *XLS-R* :cite:`babu2021xls` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2Bundle` for usage details. zwav2vec2_xlsr_1b.ptha&XLS-R model with 1 billion parameters, pre-trained on 436,000 hours of unlabeled audio from multiple datasets ( *Multilingual LibriSpeech* :cite:`Pratap_2020`, *CommonVoice* :cite:`ardila2020common`, *VoxLingua107* :cite:`valk2021voxlingua107`, *BABEL* :cite:`Gales2014SpeechRA`, and *VoxPopuli* :cite:`voxpopuli`) in 128 languages, not fine-tuned. Originally published by the authors of *XLS-R* :cite:`babu2021xls` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2Bundle` for usage details. zwav2vec2_xlsr_2b.pthiia&XLS-R model with 2 billion parameters, pre-trained on 436,000 hours of unlabeled audio from multiple datasets ( *Multilingual LibriSpeech* :cite:`Pratap_2020`, *CommonVoice* :cite:`ardila2020common`, *VoxLingua107* :cite:`valk2021voxlingua107`, *BABEL* :cite:`Gales2014SpeechRA`, and *VoxPopuli* :cite:`voxpopuli`) in 128 languages, not fine-tuned. Originally published by the authors of *XLS-R* :cite:`babu2021xls` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2Bundle` for usage details. c eZdZdZGddejZGddejZdde e d e d e e d fffd Z dddde d efdZdde e d e d ee effdZd efdZd efdZxZS)Wav2Vec2FABundleuData class that bundles associated information to use pretrained :py:class:`~torchaudio.models.Wav2Vec2Model` for forced alignment. This class provides interfaces for instantiating the pretrained model along with the information necessary to retrieve pretrained weights and additional data to be used with the model. Torchaudio library instantiates objects of this class, each of which represents a different pretrained model. Client code should access pretrained models via these instances. Please see below for the usage and the available values. Example - Feature Extraction >>> import torchaudio >>> >>> bundle = torchaudio.pipelines.MMS_FA >>> >>> # Build the model and load pretrained weight. >>> model = bundle.get_model() Downloading: 100%|███████████████████████████████| 1.18G/1.18G [00:05<00:00, 216MB/s] >>> >>> # Resample audio to the expected sampling rate >>> waveform = torchaudio.functional.resample(waveform, sample_rate, bundle.sample_rate) >>> >>> # Estimate the probability of token distribution >>> emission, _ = model(waveform) >>> >>> # Generate frame-wise alignment >>> alignment, scores = torchaudio.functional.forced_align( >>> emission, targets, input_lengths, target_lengths, blank=0) >>> ceZdZdZdS)Wav2Vec2FABundle.TokenizerzInterface of the tokenizerNr)r*r+r,r2rr Tokenizerrks((((rrmceZdZdZdS)Wav2Vec2FABundle.AlignerzInterface of the alignerNrlr2rrAlignerros&&&&rrp*r:starr<r.c^t|}||ng||RS)aGet the labels corresponding to the feature dimension of emission. The first is blank token, and it is customizable. Args: star (str or None, optional): Change or disable star token. (default: ``"*"``) blank (str, optional): Change the blank token. (default: ``'-'``) Returns: Tuple[str, ...]: For models fine-tuned on ASR, returns the tuple of strings representing the output class labels. Example >>> from torchaudio.pipelines import MMS_FA as bundle >>> bundle.get_labels() ('-', 'a', 'i', 'e', 'n', 'o', 'u', 't', 's', 'r', 'm', 'k', 'l', 'd', 'g', 'h', 'y', 'b', 'p', 'w', 'c', 'v', 'j', 'z', 'f', "'", 'q', 'x', '*') >>> bundle.get_labels(star=None) ('-', 'a', 'i', 'e', 'n', 'o', 'u', 't', 's', 'r', 'm', 'k', 'l', 'd', 'g', 'h', 'y', 'b', 'p', 'w', 'c', 'v', 'j', 'z', 'f', "'", 'q', 'x') r;)superr>)rrrr<labels __class__s rr>zWav2Vec2FABundle.get_labelss7*##%#00vv?V?T??:rTNr with_starctj|j|j}tj|j||j}||tj||j d|}| |S)a'Construct the model and load the pretrained weight. The weight file is downloaded from the internet and cached with :func:`torch.hub.load_state_dict_from_url` Args: with_star (bool, optional): If enabled, the last dimension of output layer is extended by one, which corresponds to `star` token. dl_kwargs (dictionary of keyword arguments): Passed to :func:`torch.hub.load_state_dict_from_url`. Returns: Variation of :py:class:`~torchaudio.models.Wav2Vec2Model`. .. note:: The model created with this method returns probability in log-domain, (i.e. :py:func:`torch.nn.functional.log_softmax` is applied), whereas the other Wav2Vec2 models returns logit. T)r!apply_log_softmax append_star) r r"rrrrr9r#r$rr%)rrwrr&r's rr(zWav2Vec2FABundle.get_model5s( !14<@@*4:y$BWXX  j)))# d&>RVdm      rc^dt|||DS)aMGet the mapping from token to index (in emission feature dim) Args: star (str or None, optional): Change or disable star token. (default: ``"*"``) blank (str, optional): Change the blank token. (default: ``'-'``) Returns: Tuple[str, ...]: For models fine-tuned on ASR, returns the tuple of strings representing the output class labels. Example >>> from torchaudio.pipelines import MMS_FA as bundle >>> bundle.get_dict() {'-': 0, 'a': 1, 'i': 2, 'e': 3, 'n': 4, 'o': 5, 'u': 6, 't': 7, 's': 8, 'r': 9, 'm': 10, 'k': 11, 'l': 12, 'd': 13, 'g': 14, 'h': 15, 'y': 16, 'b': 17, 'p': 18, 'w': 19, 'c': 20, 'v': 21, 'j': 22, 'z': 23, 'f': 24, "'": 25, 'q': 26, 'x': 27, '*': 28} >>> bundle.get_dict(star=None) {'-': 0, 'a': 1, 'i': 2, 'e': 3, 'n': 4, 'o': 5, 'u': 6, 't': 7, 's': 8, 'r': 9, 'm': 10, 'k': 11, 'l': 12, 'd': 13, 'g': 14, 'h': 15, 'y': 16, 'b': 17, 'p': 18, 'w': 19, 'c': 20, 'v': 21, 'j': 22, 'z': 23, 'f': 24, "'": 25, 'q': 26, 'x': 27} ci|]\}}|| Sr2r2).0iks r z-Wav2Vec2FABundle.get_dict..esTTTA1TTTr)rrr<) enumerater>)rrrr<s rget_dictzWav2Vec2FABundle.get_dictRs1&UT4??E?+R+R!S!STTTTrcNtj|S)zIInstantiate a Tokenizer. Returns: Tokenizer )r rmrrs r get_tokenizerzWav2Vec2FABundle.get_tokenizergs  111rc,tjdS)zFInstantiate an Aligner. Returns: Aligner rr;)r rprs r get_alignerzWav2Vec2FABundle.get_aligneros Q''''r)rqr:)T)r)r*r+r,r ITokenizerrmIAlignerrprr-rr>r0rr(rr@rrr __classcell__)rvs@rririsu  D)))))G&)))''''''"''';;x};3;sTWx;;;;;;0T4f:UUXc]UUtCQTH~UUUU*2y2222(W((((((((rrizQhttps://dl.fbaipublicfiles.com/mms/torchaudio/ctc_alignment_mling_uroman/model.pta Trained on 31K hours of data in 1,130 languages from *Scaling Speech Technology to 1,000+ Languages* :cite:`pratap2023scaling`. Published by the authors of *Scaling Speech Technology to 1,000+ Languages* :cite:`pratap2023scaling` under [`CC-BY-NC 4.0 License `__]. Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2FABundle` for usage details. .. note:: Unlike other Wav2Vec2 bundles, this model does not have a token for word boundary (like `|`). This makes the post-processing of alignments slightly different. )6 dataclassesrtypingrrrrtorch.nnrr r __all__r r4 WAV2VEC2_BASEr,_get_en_labelsWAV2VEC2_ASR_BASE_10MWAV2VEC2_ASR_BASE_100HWAV2VEC2_ASR_BASE_960HWAV2VEC2_LARGEWAV2VEC2_ASR_LARGE_10MWAV2VEC2_ASR_LARGE_100HWAV2VEC2_ASR_LARGE_960HWAV2VEC2_LARGE_LV60KWAV2VEC2_ASR_LARGE_LV60K_10MWAV2VEC2_ASR_LARGE_LV60K_100HWAV2VEC2_ASR_LARGE_LV60K_960HWAV2VEC2_XLSR53 HUBERT_BASE HUBERT_LARGE HUBERT_XLARGEHUBERT_ASR_LARGEHUBERT_ASR_XLARGE_get_de_labelsVOXPOPULI_ASR_BASE_10K_DE_get_vp_en_labelsVOXPOPULI_ASR_BASE_10K_EN_get_es_labelsVOXPOPULI_ASR_BASE_10K_ES_get_fr_labelsVOXPOPULI_ASR_BASE_10K_FR_get_it_labelsVOXPOPULI_ASR_BASE_10K_IT WAVLM_BASEWAVLM_BASE_PLUS WAVLM_LARGEWAV2VEC2_XLSR_300MWAV2VEC2_XLSR_1BWAV2VEC2_XLSR_2Bri_get_mms_labelsMMS_FAr2rrrs !!!!!!------------  TTTTTTT Tn DSDSDSDSDSDSDS DSN + , %( ( (   u  S  %c  "3  "2  b! " R# $ $S% & %d' ( $S) * 3+ , #E- . d/ 0 t1 4= @  *) 5 , %( ( (   u  S  %c  "3  "2  b! " R# $ $S% & %d' ( $S) * 3+ , #E- . d/ 0 r1 4 !E " "?   B !+*/,%( ( (  u S %c "3 "2 b!" R#$ $S%& %d'( $S)* 3+, #E-. d/0 r14 !E " "?   D "+*/,%( ( (  u S %c "3 "2 b!" R#$ $S%& %d'( $S)* 3+, #E-. d/0 r14 !E " "?   B " &,%( ( (  u T %c "3 "2 b!" R#$ $S%& %d'( $S)* 3+, #E-. c/0 t14=@ +*0,%( ( (  u T %c "3 "2 b!" R#$ $S%& %d'( $S)* 3+, #E-. c/0 r14 !E " "?   B ",+0,%( ( (  u T %c "3 "2 b!" R#$ $S%& %d'( $S)* 3+, #E-. c/0 r14 !E " "?   B #,+0,%( ( (  u T %c "3 "2 b!" R#$ $S%& %d'( $S)* 3+, #E-. c/0 r14 !E " "?   B #&~&,%( ( (  t T %c "3 "2 b!" R#$ $S%& %d'( $S)* 3+, #D-. c/0 t14=@  100,%( ( (  t T %c "3 "2 b!" R#$ $S%& %d'( $S)* 3+, #D-. c/0 r14 !E " "?   B ($!2 10,%( ( (  t T %c "3 "2 b!" R#$ $S%& %d'( $S)* 3+, #D-. c/0 r14 !E " "? ! ! !B )%!2 10,%( ( (  t T %c "3 "2 b!" R#$ $S%& %d'( $S)* 3+, #D-. c/0 r14 !E " "? ! ! !B )%!.',%( ( (  t T %c "3 "2 b!" R#$ $S%& %d'( $S)* 3+, #D-. c/0 t14=@ n#,%( ( (  u S %c "3 "2 b!" R#$ $S%& %d'( $S)* 3+, #E-. d/0 t14= @  ~$,%( ( (  u T %c "3 "2 b!" R#$ $S%& %d'( $S)* 3+, #D-. c/0 t14= @  %,%( ( (  u T %c "3 "2 b!" R#$ $S%& %d'( $S)* 3+, #D-. c/0 t14= @  %$.,%( ( (  u T %c "3 "2 b!" R#$ $S%& %d'( $S)* 3+, #D-. c/0 r14 !E " "?   B &%/,%( ( (  u T %c "3 "2 b!" R#$ $S%& %d'( $S)* 3+, #D-. c/0 r14 !E " "?   B  .-+,%( ( (  u S %c "3 "2 b!" R#$ $S%& %d'( $S)* 3+, #E-. c/0 r14 !E " ""A!!!D %!.-+,%( ( (  u S %c "3 "2 b!" R#$ $S%& %d'( $S)* 3+, #E-. c/0 r14 $E # % %"A!!!D %!.-+,%( ( (  u S %c "3 "2 b!" R#$ $S%& %d'( $S)* 3+, #E-. c/0 r14 !E " ""A!!!D %!.-+,%( ( (  u S %c "3 "2 b!" R#$ $S%& %d'( $S)* 3+, #E-. c/0 r14 !E " "?   B %!.-+,%( ( (  u S %c "3 "2 b!" R#$ $S%& %d'( $S)* 3+, #E-. c/0 r14 !E " "A!!!D %!^,%( ( (  u S %c "3 "2 b!" R#$ %& s'( $S)* %d+, $S-. 3/0 #E12 d34 t58A!!! D  !.,%( ( (  u S %c "3 "2 b!" R#$ %& s'( $S)* %d+, $S-. 3/0 #E12 d34 t58A!!!D n,%( ( (  u T %c "3 "2 b!" R#$ %& s'( $S)* %d+, $S-. 3/0 #D12 d34 t58A!!! D  $^,%( ( (  t T %c "3 "2 b!" R#$ $S%& %d'( $S)* 3+, #D-. c/0 t14=@$">,%( ( (  t T %c "3 "2 b!" R#$ $S%& %d'( $S)* 3+, #D-. c/0 t14=@"">,%( ( (  t T %c "3 "2 b!" R#$ $S%& %d'( $S)* 3+, #D-. c/0 t14=@$ A(A(A(A(A((A(A( A(H  W,%( ( (  t T %c "3 "2 b!" R#$ $S%& %d'( $S)* 3+, #D-. c/0 r14 "E ! # #?   B r