j|UddlmZmZmZmZddlZddlmcmZ ddlm Z ddl m Z gZ eeed<ejjGddZdS))DictListOptionalTupleN)Tensor)2_scripted_functional_optimizer_deprecation_warning__all__ceZdZ ddeededeeefdeeefd ed ed efd Zd ee efdZ dS)_FunctionalRprop{Gz?g?g333333?gư>2Fparamslretas step_sizesforeachmaximize_allow_empty_param_listcztdd|i|_||_||_||_||_t |dkr|stdd|i|_tj ttj tttj ffi|_dS)N) stacklevelrrz%optimizer got an empty parameter listr)rdefaultsrrrrlen ValueError param_grouptorchjitannotaterrstrstate)selfrrrrrrrs YC:\Projects\Historical\venv\Lib\site-packages\torch/distributed/optim/functional_rprop.py__init__z_FunctionalRprop.__init__s ;aHHHH "   $   v;;!  $; DEE E%f-Y''U\4U\@Q;R-R(SUWXX gradientsc*|jd}g}g}g}g}g}|jd}|j\} } |j\} } t |t |kr6t ddt |dzdt |zd} t ||D]\}}|| tj|z} | || |||j vrii|j |<|j |}tj d|d <tj |tj |d <tj|||d <|j |}| |d | |d | |d tj5t!j|||||| | | | |j|j| ddddS#1swxYwYdS)NrrzEthe gradients passed in does not equal to the size of the parameters!zParams length: z. zGradients length: Fgstep) memory_formatprev step_size) step_size_min step_size_maxetaminusetaplusrr has_complex)rrrrrrzipr is_complexappendr"tensor zeros_likepreserve_format full_likeno_gradFrproprr)r#r'rparams_with_gradgradsprevsr state_stepsrr/r0r-r.r1paramgradientr"s r$r)z_FunctionalRprop.step6s!(+  ]4  I''+$ } v;;#i.. ( (W3CKK33347s9~~778   "6955 2 2OE8#u/666  ''... X&&& **(*DJu% Ju-E$)L$5$5E&M$)$4U-B%%%E&M*/2)F)FE+& 5) U6]+++!!% "4555""5=111 ]__   G ++! '                      s+HH H N)r r rFFF) __name__ __module__ __qualname__rrfloatrboolr%rr)r&r$r r s $.*4(-YYV Y YE5L! Y %,' Y  YY"&YYYY85d8F#34555555r&r )typingrrrrrtorch.optim._functionaloptim _functionalr:r,torch.distributed.optim._deprecation_warningrr r!__annotations__rscriptr rGr&r$rOs............. ######### cRRRRRRRRRRr&