vllm.models.qwen4_exp.nvidia.ops.hc

NVIDIA HyperConnection kernels for Qwen4Exp.

hc_up_gate_mix(lowrank, xn, w_up)

hc_gate_mix(xn, F.linear(lowrank, w_up), 4) in one SM100 kernel.

The gate stays on chip in fp32 (the chain rounds it to bf16) and the sigmoid uses tanh.approx; results are within one bf16 ULP of an fp32 reference. Only hc_count 4, hidden size 2560 and rank 320 are supported.

Source code in vllm/models/qwen4_exp/nvidia/ops/hc.py

def hc_up_gate_mix(
    lowrank: torch.Tensor, xn: torch.Tensor, w_up: torch.Tensor
) -> torch.Tensor:
    """``hc_gate_mix(xn, F.linear(lowrank, w_up), 4)`` in one SM100 kernel.

    The gate stays on chip in fp32 (the chain rounds it to bf16) and the
    sigmoid uses ``tanh.approx``; results are within one bf16 ULP of an fp32
    reference.
    Only hc_count 4, hidden size 2560 and rank 320 are supported.
    """
    out = xn.new_empty(xn.shape[0], xn.shape[1] // 4)
    torch.ops._C.qwen4_exp_hc_up_gate_mix(out, lowrank, xn, w_up)
    return out