@j57bddlZddlZddlmZmZddlmZddlmZddl m Z m Z m Z ddl Z ddlZddlmZddlmZmZmZgZdeje je jjzZed d ezZd ZGd d e jjZ Gdde jjZ!GddeZ"GddeZ#Gdde jje"Z$Gdde#Z%eGddZ&e&deeddddd d!d"d#d$d%d& Z'd'e'_(dS)(N)ABCabstractmethod) dataclass)partial)CallableListTuple) module_utils)emformer_rnnt_baseRNNTRNNTBeamSearch( g?ctj||tjk||tjk<||tjktjz ||tjk<|SN)torchlogmathexs SC:\Projects\Historical\venv\Lib\site-packages\torchaudio/pipelines/rnnt_pipeline.py_piecewise_linear_logrsJIaDF m,,Aa$&jMqDF{^df,Aa46kN Hc$eZdZfdZdZxZS)_FunctionalModulecVt||_dSr)super__init__ functional)selfr __class__s rrz_FunctionalModule.__init__s$ $rc,||Sr)r r!inputs rforwardz_FunctionalModule.forwardsu%%%r__name__ __module__ __qualname__rr& __classcell__r"s@rrrsG%%%%%&&&&&&&rrc$eZdZfdZdZxZS)_GlobalStatsNormalizationctt|5}tj|}dddn #1swxYwY|dtj|d|dtj|ddS)Nmean invstddev) rropenjsonloadsreadregister_bufferrtensor)r!global_stats_pathfblobr"s rrz"_GlobalStatsNormalization.__init__$s  # $ $ (:affhh''D ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( VU\$v,%?%?@@@ [%,tK7H*I*IJJJJJs'A$$A(+A(c&||jz |jzSr)r0r1r$s rr&z!_GlobalStatsNormalization.forward-s !T^33rr'r,s@rr.r.#sLKKKKK4444444rr.c\eZdZedejdeejejffdZdS)_FeatureExtractorr%returncdS)XGenerates features and length output from the given input tensor. Args: input (torch.Tensor): input tensor. Returns: (torch.Tensor, torch.Tensor): torch.Tensor: Features, with shape `(length, *)`. torch.Tensor: Length, with shape `(1,)`. Nr$s r__call__z_FeatureExtractor.__call__2rN)r(r)r*rrTensorr rBrArrr=r=1sP  el  uU\5<5O/P    ^    rr=c:eZdZedeedefdZdS)_TokenProcessortokensr>c dS)zDecodes given list of tokens to text sequence. Args: tokens (List[int]): list of tokens to decode. Returns: str: Decoded text sequence. NrA)r!rGkwargss rrBz_TokenProcessor.__call__CrCrN)r(r)r*rrintstrrBrArrrFrFBsD  tCy  s    ^    rrFc~eZdZdZdejjddffd Zdejde ejejffdZ xZ S)_ModuleFeatureExtractorz``torch.nn.Module``-based feature extraction pipeline. Args: pipeline (torch.nn.Module): module that implements feature extraction logic. pipeliner>NcVt||_dSr)rrrN)r!rNr"s rrz _ModuleFeatureExtractor.__init__Ws$   rr%ct||}tj|jdg}||fS)r@r)rNrr7shape)r!r%featureslengths rr&z_ModuleFeatureExtractor.forward[s9==''x~a0122r) r(r)r*__doc__rnnModulerrDr r&r+r,s@rrMrMPs !!T!!!!!! U\ eEL%,4N.O        rrMcDeZdZdZdeddfdZd deededefd Z dS) _SentencePieceTokenProcessorztSentencePiece-model-based token processor. Args: sp_model_path (str): path to SentencePiece model. sp_model_pathr>Nc(tjdstdddl}|||_|j|j|jh|_ dS)N sentencepiecez2SentencePiece is not available. Please install it.r) model_file) r is_module_available RuntimeErrorr[SentencePieceProcessorsp_modelunk_ideos_idpad_idpost_process_remove_list)r!rYspms rrz%_SentencePieceTokenProcessor.__init__ts/@@ USTT T####22m2LL M " " M " " M " ") %%%rTrGlstripcfd|ddD}dj|dd}|r|S|S)aXDecodes given list of tokens to text sequence. Args: tokens (List[int]): list of tokens to decode. lstrip (bool, optional): if ``True``, returns text sequence with leading whitespace removed. (Default: ``True``). Returns: str: Decoded text sequence. c&g|] }|jv |SrA)rd).0 token_indexr!s r z9_SentencePieceTokenProcessor.__call__..s-   'DLi9i9iK9i9i9irNu▁ )joinr` id_to_piecereplacerf)r!rGrffiltered_hypo_tokens output_strings` rrBz%_SentencePieceTokenProcessor.__call__s    +1!"":    9 9:N O OPPXXYacfgg  ! '')) ) r)T) r(r)r*rTrKrrrJboolrBrArrrXrXmsr  c  d     !!tCy!$!#!!!!!!rrXceZdZUdZGddeZGddeZee d<e ge fe d<ee d<ee d <e e d <e e d <e e d <e e d <e e d<e e d<e e d<e e d<de fdZ ede fdZede fdZede fdZede fdZede fdZede fdZdefdZdefdZdefdZdefdZdS) RNNTBundleuDataclass that bundles components for performing automatic speech recognition (ASR, speech-to-text) inference with an RNN-T model. More specifically, the class provides methods that produce the featurization pipeline, decoder wrapping the specified RNN-T model, and output token post-processor that together constitute a complete end-to-end ASR inference pipeline that produces a text sequence given a raw waveform. It can support non-streaming (full-context) inference as well as streaming inference. Users should not directly instantiate objects of this class; rather, users should use the instances (representing pre-trained models) that exist within the module, e.g. :data:`torchaudio.pipelines.EMFORMER_RNNT_BASE_LIBRISPEECH`. Example >>> import torchaudio >>> from torchaudio.pipelines import EMFORMER_RNNT_BASE_LIBRISPEECH >>> import torch >>> >>> # Non-streaming inference. >>> # Build feature extractor, decoder with RNN-T model, and token processor. >>> feature_extractor = EMFORMER_RNNT_BASE_LIBRISPEECH.get_feature_extractor() 100%|███████████████████████████████| 3.81k/3.81k [00:00<00:00, 4.22MB/s] >>> decoder = EMFORMER_RNNT_BASE_LIBRISPEECH.get_decoder() Downloading: "https://download.pytorch.org/torchaudio/models/emformer_rnnt_base_librispeech.pt" 100%|███████████████████████████████| 293M/293M [00:07<00:00, 42.1MB/s] >>> token_processor = EMFORMER_RNNT_BASE_LIBRISPEECH.get_token_processor() 100%|███████████████████████████████| 295k/295k [00:00<00:00, 25.4MB/s] >>> >>> # Instantiate LibriSpeech dataset; retrieve waveform for first sample. >>> dataset = torchaudio.datasets.LIBRISPEECH("/home/librispeech", url="test-clean") >>> waveform = next(iter(dataset))[0].squeeze() >>> >>> with torch.no_grad(): >>> # Produce mel-scale spectrogram features. >>> features, length = feature_extractor(waveform) >>> >>> # Generate top-10 hypotheses. >>> hypotheses = decoder(features, length, 10) >>> >>> # For top hypothesis, convert predicted tokens to text. >>> text = token_processor(hypotheses[0][0]) >>> print(text) he hoped there would be stew for dinner turnips and carrots and bruised potatoes and fat mutton pieces to [...] >>> >>> >>> # Streaming inference. >>> hop_length = EMFORMER_RNNT_BASE_LIBRISPEECH.hop_length >>> num_samples_segment = EMFORMER_RNNT_BASE_LIBRISPEECH.segment_length * hop_length >>> num_samples_segment_right_context = ( >>> num_samples_segment + EMFORMER_RNNT_BASE_LIBRISPEECH.right_context_length * hop_length >>> ) >>> >>> # Build streaming inference feature extractor. >>> streaming_feature_extractor = EMFORMER_RNNT_BASE_LIBRISPEECH.get_streaming_feature_extractor() >>> >>> # Process same waveform as before, this time sequentially across overlapping segments >>> # to simulate streaming inference. Note the usage of ``streaming_feature_extractor`` and ``decoder.infer``. >>> state, hypothesis = None, None >>> for idx in range(0, len(waveform), num_samples_segment): >>> segment = waveform[idx: idx + num_samples_segment_right_context] >>> segment = torch.nn.functional.pad(segment, (0, num_samples_segment_right_context - len(segment))) >>> with torch.no_grad(): >>> features, length = streaming_feature_extractor(segment) >>> hypotheses, state = decoder.infer(features, length, 10, state=state, hypothesis=hypothesis) >>> hypothesis = hypotheses[0] >>> transcript = token_processor(hypothesis[0]) >>> if transcript: >>> print(transcript, end=" ", flush=True) he hoped there would be stew for dinner turn ips and car rots and bru 'd oes and fat mut ton pieces to [...] ceZdZdZdS)RNNTBundle.FeatureExtractorz:Interface of the feature extraction part of RNN-T pipelineNr(r)r*rTrArrFeatureExtractorrxsHHHHrrzceZdZdZdS)RNNTBundle.TokenProcessorz7Interface of the token processor part of RNN-T pipelineNryrArrTokenProcessorr|sEEEErr} _rnnt_path_rnnt_factory_func_global_stats_path_sp_model_path_right_padding_blank _sample_rate_n_fft_n_mels _hop_length_segment_length_right_context_lengthr>c|}tj|j}t j|}||||Sr) r torchaudioutils_download_assetr~rloadload_state_dicteval)r!modelpath state_dicts r _get_modelzRNNTBundle._get_models`''))//@@Z%%  j)))  rc|jS)zSSample rate (in cycles per second) of input waveforms. :type: int )rr!s r sample_ratezRNNTBundle.sample_rates   rc|jS)z7Size of FFT window to use. :type: int )rrs rn_fftzRNNTBundle.n_ffts {rc|jS)z`Number of mel spectrogram features to extract from input waveforms. :type: int )rrs rn_melszRNNTBundle.n_mels s |rc|jS)zdNumber of samples between successive frames in input expected by model. :type: int )rrs r hop_lengthzRNNTBundle.hop_lengths rc|jS)zTNumber of frames in segment in input expected by model. :type: int )rrs rsegment_lengthzRNNTBundle.segment_lengths ##rc|jS)zcNumber of frames in right contextual block in input expected by model. :type: int )rrs rright_context_lengthzRNNTBundle.right_context_length%s ))rcT|}t||jS)zOConstructs RNN-T decoder. Returns: RNNTBeamSearch )rr r)r!rs r get_decoderzRNNTBundle.get_decoder-s% !!eT[111rc tjj}t t jtj j j j j tdtdt|tfdS)zzConstructs feature extractor for non-streaming (full-context) ASR. Returns: FeatureExtractor rrrrc.|ddSNrlr transposers rz2RNNTBundle.get_feature_extractor..BAKK1,=,=rc0t|tzSrr_gainrs rrz2RNNTBundle.get_feature_extractor..C,A!e),L,Lrc`tjj|dddjfS)Nr)rrUr padr)rr!s rrz2RNNTBundle.get_feature_extractor..Es)EH,?,C,CA1aQUQdGe,f,frrrrrrMrrU Sequential transformsMelSpectrogramrrrrrr.r! local_paths` rget_feature_extractorz RNNTBundle.get_feature_extractor6s  %55d6MNN & H  %44 $ 0 4;cgcr5""="=>>!"L"LMM)*55!"f"f"f"fgg     rc dtj|j}t t jtj |j |j |j |j tdtdt|S)zvConstructs feature extractor for streaming (simultaneous) ASR. Returns: FeatureExtractor rc.|ddSrrrs rrz.Urrc0t|tzSrrrs rrz.Vrrrrs rget_streaming_feature_extractorz*RNNTBundle.get_streaming_feature_extractorIs  %55d6MNN & H  %44 $ 0 4;cgcr5""="=>>!"L"LMM)*55      rchtj|j}t |S)zQConstructs token processor. Returns: TokenProcessor )rrrrrXrs rget_token_processorzRNNTBundle.get_token_processor[s+  %55d6IJJ +J777rN)r(r)r*rTr=rzrFr}rK__annotations__rr rJrpropertyrrrrrrr rrrrrArrrvrvsFFPIIIII,IIIFFFFFFFFOOO T**** KKK KKK LLLD!S!!!X!sXX C   X $$$$X$*c***X*2^2222 '7    & 1A    $8^888888rrvz(models/emformer_rnnt_base_librispeech.pti) num_symbolsz2pipeline-assets/global_stats_rnnt_librispeech.jsonz.pipeline-assets/spm_bpe_4096_librispeech.modelii>iP) r~rrrrrrrrrrraASR pipeline based on Emformer-RNNT, pretrained on *LibriSpeech* dataset :cite:`7178964`, capable of performing both streaming and non-streaming inference. The underlying model is constructed by :py:func:`torchaudio.models.emformer_rnnt_base` and utilizes weights trained on LibriSpeech using training script ``train.py`` `here `__ with default arguments. Please refer to :py:class:`RNNTBundle` for usage instructions. ))r3rabcrr dataclassesr functoolsrtypingrrr rrtorchaudio._internalr torchaudio.modelsr r r __all__log10iinfoint16max_decibelpowrrrUrVrr.r=rFrMrXrvEMFORMER_RNNT_BASE_LIBRISPEECHrTrArrrs ########!!!!!!(((((((((( ------FFFFFFFFFF  JDJ{u{5;77;<< < Bx     &&&&&&&& 4 4 4 4 4 4 4 4        "      c         eho/@   :(!(!(!(!(!?(!(!(!V I8I8I8I8I8I8I8 I8X",9w1tDDDKC    " " " *&&&r