+j hddlmZddlmZddlZddlmZmZddlmZGddej Z dS)) annotations)IterableN)Tensornn) SparseEncodercReZdZddfd Zdd ZddZddZeddZxZ S) FlopsLossNmodelr threshold float | NonereturnNonecdt||_||_dS)a FlopsLoss implements a regularization technique to promote sparsity in sparse encoder models. It calculates the squared L2 norm of the mean embedding vector, which helps reduce the number of floating-point operations (FLOPs) required during inference by encouraging more zero values in the embeddings. It can use a threshold to ignore embeddings with too few non-zero (active) elements. This loss is used as a regularization component within other losses like :class:`SpladeLoss` rather than being used as a standalone loss function. Args: model: SparseEncoder model to be regularized threshold: Optional threshold for the number of non-zero (active) elements in the embeddings. If specified, only embeddings with more than this number of non-zero (active) elements will be considered. This can help to ignore embeddings that are too sparse and may not contribute meaningfully to the loss. References: - For further details, see: https://huggingface.co/papers/2004.05665 for the general FLOPS loss and https://huggingface.co/papers/2504.14839 for FLOPS with thresholds, a.k.a. FLOPS with l0 masking. Relations: - Used as a component within :class:`SpladeLoss` to regularize both query and document embeddings Example: - This loss is typically used within the :class:`SpladeLoss` class, which combines it with other loss components. N)super__init__r r )selfr r __class__s _C:\Projects\DSCons\.venv\Lib\site-packages\sentence_transformers\sparse_encoder\losses\flops.pyrzFlopsLoss.__init__ s-4  "sentence_featuresIterable[dict[str, Tensor]]labelsrc td)NzcFlopsLoss is not intended to be used directly. Use it as a regularizer within the SpladeLoss class.)NotImplementedError)rrrs rforwardzFlopsLoss.forward*s! q   r embeddings torch.Tensorc|jO|dkd}||jk}||dz}t jt j|ddzS)Nr)dim)r sumfloat unsqueezetorchmean)rrl0_normmasks rcompute_loss_from_embeddingsz&FlopsLoss.compute_loss_from_embeddings/sv > %!Q+++22Gdn,3355D#dnnQ&7&77JyJA666!;<<r;s""""""$$$$$$ DDDDDD88888 88888r