˲j+ddlZddlmZddlZddlmZddlmZmZddl m Z m Z ddl m Z ddlmZgd ZGd d eZGd d eZGddeZGddeZGdde eZdS)N)Any)Tensor) functionalinit) ParameterUninitializedParameter)LazyModuleMixin)Module)BilinearIdentity LazyLinearLinearc@eZdZdZdededdffd ZdedefdZxZS) r aA placeholder identity operator that is argument-insensitive. Args: args: any argument (unused) kwargs: any keyword argument (unused) Shape: - Input: :math:`(*)`, where :math:`*` means any number of dimensions. - Output: :math:`(*)`, same shape as the input. Examples:: >>> m = nn.Identity(54, unused_argument1=0.1, unused_argument2=False) >>> input = torch.randn(128, 20) >>> output = m(input) >>> print(output.size()) torch.Size([128, 20]) argskwargsreturnNcHtdSNsuper__init__)selfrr __class__s HC:\Projects\Historical\venv\Lib\site-packages\torch/nn/modules/linear.pyrzIdentity.__init__+s inputc|Srrrs rforwardzIdentity.forward.s r) __name__ __module__ __qualname____doc__rrrr! __classcell__rs@rr r s~(cSTVrr c eZdZUdZddgZeed<eed<eed< ddedededdffd Z dd Z d edefd Z de fd Z xZS)ra/Applies an affine linear transformation to the incoming data: :math:`y = xA^T + b`. This module supports :ref:`TensorFloat32`. On certain ROCm devices, when using float16 inputs this module will use :ref:`different precision` for backward. Args: in_features: size of each input sample out_features: size of each output sample bias: If set to ``False``, the layer will not learn an additive bias. Default: ``True`` Shape: - Input: :math:`(*, H_{in})` where :math:`*` means any number of dimensions including none and :math:`H_{in} = \text{in\_features}`. - Output: :math:`(*, H_{out})` where all but the last dimension are the same shape as the input and :math:`H_{out} = \text{out\_features}`. Attributes: weight: the learnable weights of the module of shape :math:`(\text{out\_features}, \text{in\_features})`. The values are initialized from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})`, where :math:`k = \frac{1}{\text{in\_features}}` bias: the learnable bias of the module of shape :math:`(\text{out\_features})`. If :attr:`bias` is ``True``, the values are initialized from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})` where :math:`k = \frac{1}{\text{in\_features}}` Examples:: >>> m = nn.Linear(20, 30) >>> input = torch.randn(128, 20) >>> output = m(input) >>> print(output.size()) torch.Size([128, 30]) in_features out_featuresweightTNbiasrc\||d}t||_||_t t j||ffi||_|r%t t j|fi||_n| dd| dSNdevicedtyper,) rrr)r*rtorchemptyr+r,register_parameterreset_parameters)rr)r*r,r0r1factory_kwargsrs rrzLinear.__init__]s%+U;; &( K{3 F F~ F F     2!%+l"M"Mn"M"MNNDII  # #FD 1 1 1 rc tj|jtjd|jYtj|j\}}|dkrdtj|z nd}tj|j| |dSdS)N)arr )rkaiming_uniform_r+mathsqrtr,_calculate_fan_in_and_fan_outuniform_)rfan_in_bounds rr5zLinear.reset_parametersrs dkTYq\\:::: 9 :4;GGIFA-3aZZA &))))QE M$)eVU 3 3 3 3 3 ! rrcBtj||j|jSr)Flinearr+r,r s rr!zLinear.forward|sxt{DI666rc:d|jd|jd|jduS)Nz in_features=, out_features=, bias=)r)r*r,rs r extra_reprzLinear.extra_reprs0pd.ppt?PppY]YbjnYnppprTNNrNr"r#r$r% __constants__int__annotations__rboolrr5r!strrIr&r's@rrr2s##J#N3M NNN               *44447V77777qCqqqqqqqqrrc 8eZdZ ddedededdffd ZxZS) NonDynamicallyQuantizableLinearTNr)r*r,rcTt|||||dS)N)r,r0r1r)rr)r*r,r0r1rs rrz(NonDynamicallyQuantizableLinear.__init__s;  Du      rrJ)r"r#r$rNrPrr&r's@rrSrSss                      rrSc eZdZUdZgdZeed<eed<eed<eed< ddededed ed df fd Z dd Z d eded efdZ d e fdZ xZS)r aApplies a bilinear transformation to the incoming data: :math:`y = x_1^T A x_2 + b`. Args: in1_features: size of each first input sample in2_features: size of each second input sample out_features: size of each output sample bias: If set to False, the layer will not learn an additive bias. Default: ``True`` Shape: - Input1: :math:`(*, H_{in1})` where :math:`H_{in1}=\text{in1\_features}` and :math:`*` means any number of additional dimensions including none. All but the last dimension of the inputs should be the same. - Input2: :math:`(*, H_{in2})` where :math:`H_{in2}=\text{in2\_features}`. - Output: :math:`(*, H_{out})` where :math:`H_{out}=\text{out\_features}` and all but the last dimension are the same shape as the input. Attributes: weight: the learnable weights of the module of shape :math:`(\text{out\_features}, \text{in1\_features}, \text{in2\_features})`. The values are initialized from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})`, where :math:`k = \frac{1}{\text{in1\_features}}` bias: the learnable bias of the module of shape :math:`(\text{out\_features})`. If :attr:`bias` is ``True``, the values are initialized from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})`, where :math:`k = \frac{1}{\text{in1\_features}}` Examples:: >>> m = nn.Bilinear(20, 30, 40) >>> input1 = torch.randn(128, 20) >>> input2 = torch.randn(128, 30) >>> output = m(input1, input2) >>> print(output.size()) torch.Size([128, 40]) ) in1_features in2_featuresr*rVrWr*r+TNr,rcl||d}t||_||_||_t t j|||ffi||_|r%t t j|fi||_ n| dd| dSr.) rrrVrWr*rr2r3r+r,r4r5) rrVrWr*r,r0r1r6rs rrzBilinear.__init__s%+U;; ((( K|\B U Un U U     2!%+l"M"Mn"M"MNNDII  # #FD 1 1 1 rcdtj|jdz }t j|j| ||jt j|j| |dSdS)Nr )r;r<r+sizerr>r,)rrAs rr5zBilinear.reset_parametersslDIdk..q11222 dkE65111 9 M$)eVU 3 3 3 3 3 ! rinput1input2cDtj|||j|jSr)rCbilinearr+r,)rr[r\s rr!zBilinear.forwardsz&&$+tyAAArc Jd|jd|jd|jd|jduS)Nz in1_features=z, in2_features=rFrG)rVrWr*r,rHs rrIzBilinear.extra_reprsW ND- N Nd>O N N - N N6:it6K N N rrJrKrLr's@rr r s$##JEDDM NNN               04444 BfBfBBBBB C        rr cbeZdZUdZeZeed<eed< d dede ddffd Z d fd Z d d Z xZ S) raA :class:`torch.nn.Linear` module where `in_features` is inferred. In this module, the `weight` and `bias` are of :class:`torch.nn.UninitializedParameter` class. They will be initialized after the first call to ``forward`` is done and the module will become a regular :class:`torch.nn.Linear` module. The ``in_features`` argument of the :class:`Linear` is inferred from the ``input.shape[-1]``. Check the :class:`torch.nn.modules.lazy.LazyModuleMixin` for further documentation on lazy modules and their limitations. Args: out_features: size of each output sample bias: If set to ``False``, the layer will not learn an additive bias. Default: ``True`` Attributes: weight: the learnable weights of the module of shape :math:`(\text{out\_features}, \text{in\_features})`. The values are initialized from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})`, where :math:`k = \frac{1}{\text{in\_features}}` bias: the learnable bias of the module of shape :math:`(\text{out\_features})`. If :attr:`bias` is ``True``, the values are initialized from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})` where :math:`k = \frac{1}{\text{in\_features}}` r+r,TNr*rc||d}tdddtdi||_||_|rtdi||_dSdS)Nr/rFr)rrrr+r*r,)rr*r,r0r1r6rs rrzLazyLinear.__init__ s{%+U;; Au%%%,>>~>> (  A.@@@@DIII A Arc|s-|jdkr$tdSdSdS)Nr)has_uninitialized_paramsr)rr5)rrs rr5zLazyLinear.reset_parameterssP,,.. '43Cq3H3H GG $ $ & & & & & ' '3H3Hrcp|rtj5|jd|_|j|j|jf|j |j|jf| ddddS#1swxYwYdSdS)N) rcr2no_gradshaper)r+ materializer*r,r5r s rinitialize_parametersz LazyLinear.initialize_parameterss  ( ( * * ( ( (#(;r?  ''):D???%%'''  ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( (sA4B))B-0B-rJrK)r"r#r$r%r cls_to_becomerrOrNrPrr5rir&r's@rrrs8M """"    HL A A A'+ A  A A A A A A''''''((((((((rr)r;typingrr2rtorch.nnrrCrtorch.nn.parameterrrlazyr moduler __all__r rrSr rrrrrqs  ********@@@@@@@@!!!!!!   v8NqNqNqNqNqVNqNqNql      f    Q Q Q Q Q vQ Q Q h8(8(8(8(8(&8(8(8(8(8(r