@j ddlmZmZddlmZmZddlZddlmZ ddlm Z ddlm Z GddeZ Gdd e Z dd e d eed efd ZGddeZdZdZGddeZdS))ABCabstractmethod)DictListN)Tensor) TokenSpancReZdZedeedeeefdZdS) ITokenizer transcriptreturncdS)a Tokenize the given transcript (list of word) .. note:: The toranscript must be normalized. Args: transcript (list of str): Transcript (list of word). Returns: (list of int): List of token sequences Nselfr s WC:\Projects\Historical\venv\Lib\site-packages\torchaudio/pipelines/_wav2vec2/aligner.py__call__zITokenizer.__call__ N)__name__ __module__ __qualname__rrstrrrrrr r sL  49  d3i    ^    rr c^eZdZdeeeffdZdeedeeefdZdS) Tokenizer dictionaryc||_dSNr)rrs r__init__zTokenizer.__init__s $rr r c fd|DS)Nc,g|]}fd|DS)c*g|]}j|Srr).0crs r z1Tokenizer.__call__... s 222#222rr)r#wordrs rr%z&Tokenizer.__call__.. s.JJJt2222T222JJJrrrs` rrzTokenizer.__call__sJJJJzJJJJrN) rrrrrintrrrrrrrrsm%4S>%%%%K49Kd3iKKKKKKrremissiontokensblankc|j}|d}tj|gtj|}t j|||\}}|}|d|d}}||fS)Nr)dtypedevicer*)r- unsqueezetorchtensorint32F forced_alignexp)r(r)r*r-targetsaligned_tokensscoress r_align_emission_and_tokensr9#s{ _F!!!$$HlF85;vFFFG^HgUKKKNF ZZ\\F+A.q FN 6 !!rc beZdZededeeedeeefdZdS)IAlignerr(r)r cdS)aGenerate list of time-stamped token sequences Args: emission (Tensor): Sequence of token probability distributions in log-domain. Shape: `(time, tokens)`. tokens (list of integer sequence): Tokenized transcript. Output from :py:class:`torchaudio.pipelines.Wav2Vec2FABundle.Tokenizer`. Returns: (list of TokenSpan sequence): Tokens with time stamps and scores. Nr)rr(r)s rrzIAligner.__call__0rrN) rrrrrrr'rrrrrr;r;/sX    d3i  T$y/EZ    ^    rr;ct|t|ksJd}g}|D]'}|||||z||z }(|S)Nr)lensumappend)list_lengthsiretls r _unflattenrF?sg u::W % % % % A C  5QU#$$$ Q Jrcd|DS)Ncg|] }|D]}| Srr)r#rAitems rr%z_flatten..Js% < < )r#tss rr%z$Aligner.__call__..Ws!;!;!;b#b''!;!;!;r) ndim ValueErrorshaper9rKr*r3 merge_tokensrF)rr(r)r7r8spanss rrzAligner.__call__Qsz =A  VhnVVWW W!;HhvFVFVX\Xb!c!c~v66%!;!;F!;!;!;<<r\s~######## !!!!!!++++++        "KKKKK KKK " " "c "3 " " " "      s    === = = = = =h = = = = =r