[j%lUdZddlmZddlZddlmZddlmZddl Z ddl m Z ddl m Z dd lmZmZd d lmZd d lmZe jeZd dddZdddZe jjd)d*dZedGddZdaded<e jj d+d Z!d,d!Z"d-d(Z#dS).zSonicMoE integration: fused MoE using CuteDSL kernels from `kernels-community/sonic-moe`. Provides `sonicmoe_experts_forward` registered as "sonicmoe" in the ExpertsInterface. Requirements: CUDA, `kernels`, `nvidia-cutlass-dsl`, has_gate=True. ) annotationsN)Callable) dataclass)version)logging)is_kernels_availablemaybe_import_error)lazy_load_kernel)to_localswiglugeglureglu)silugelureluz4.6.0z0.1.10)znvidia-cutlass-dslzapache-tvm-ffiF raise_errorboolreturnc tstd|Stjstd|Stjd}|dkrtd|d|St D]\}} tj |}n1#tj j $rtd|d |cYcSwxYwtj |tj |krtd|d |d |d |cSd S)u+Whether the sonic-moe kernel can be loaded in this environment: `kernels` installed, a CUDA GPU on Hopper (SM90) or newer, and `nvidia-cutlass-dsl` / `apache-tvm-ffi` no newer than the versions the kernel was validated against (`SONICMOE_DEPENDENCIES`) — newer releases may break its CuteDSL / TVM FFI dispatch. A one-glance gate for callers, including external stacks. With `raise_error=True` (used by the loader) it re-raises the specific `ImportError` instead of returning `False`. `@assume_constant_result` makes dynamo evaluate this once at trace time and inline the bool (its probe is untraceable — an `lru_cache`'d import check, the `get_device_capability` pybind, and `importlib.metadata` version lookups), so it stays compile-safe if called from a compiled region. zSsonic-moe requires the `kernels` package. Use a different `experts_implementation`.rzdsonic-moe kernel requires CUDA, but CUDA is not available. Use a different `experts_implementation`.r z`sonic-moe requires a Hopper (SM90+) or newer GPU, but the current device has compute capability z-.x. Use a different `experts_implementation`.zsonic-moe requires `z`, but it is not installed.z` >= z, but z is installed.T) r r torchcuda is_availableget_device_capabilitySONICMOE_DEPENDENCIESitems importlibmetadatarPackageNotFoundErrorparse)rmajor distribution min_version installeds PC:\Projects\DSCons\.venv\Lib\site-packages\transformers\integrations\sonicmoe.pyis_sonicmoe_loadabler).s ! ! ! a#     : " " $ $ ! r#     J , , . .q 1E qyy! O O O O#    &;%@%@%B%B! k  !*22<@@II!6   %P|PPP^i       = # #gmK&@&@ @ @%f|ff+ffYfff'    A 4s-C  )C;:C;T)frozenc(eZdZUdZded<ded<dS)SonicMoEzAEntry points exposed by the `kernels-community/sonic-moe` kernel.typeactivation_type_enumrmoe_general_routing_inputsN)__name__ __module__ __qualname____doc____annotations__r(r,r,^s0KK((((((r6r,zSonicMoE | None _SONICMOENonecptdStdtd}|tdt t |dddd}t |dd}d d |fd|ffD}|r&td d |d t ||adS)a Load sonic-moe once into the `_SONICMOE` module global. `@allow_in_graph` makes `torch.compile` treat the untraceable hub download + dynamic import as a single opaque node instead of tracing into it; it returns `None` (proxyable) and populates the global, which `load_sonicmoe_kernel` then returns. Under NO circumstances may this function return a value: an `@allow_in_graph` fx node's return must be proxyable, and returning the bundle (e.g. from the warm-cache short-circuit) breaks torch.compile with `Unsupported: torch.* op returned non-Tensor`. Raises `ImportError` if CUDA/hardware requirements are not met, or if the kernel or required symbols are not found. NTrz sonic-moeu}Failed to load the sonic-moe kernel — check that `kernels-community/sonic-moe` has a build matching the current torch/CUDA.enumsActivationTyper/cg|] \}}|| SNr5).0nameattrs r( z)_load_sonicmoe_kernel..s- D$ < <rs #"""""$$$$$$!!!!!! IIIIIIII))))))%%%%%%  H % %Wg > >07(SS%,,,,&%,^ $))))))))" !!!!....b 999999r6