Summary
In affected versions of vLLM, GGUF dequantize CUDA kernels truncate tensor dimensions, leaving parts of output tensors uninitialized while still exposed to callers.
In multi-tenant inference setups, these uninitialized regions can contain residual GPU data from other users' requests, leading to cross-tenant information disclosure when GGUF dequantization is used.
Affected context
This vulnerability appears in vLLM deployments that use GGUF quantized models and rely on the GGUF dequantize kernels implemented in csrc/quantization/gguf/gguf_kernel.cu.
It is most relevant in multi-tenant inference or serving environments where different users' workloads share the same GPU memory pool and tensor buffers may be reused across requests.
Single-tenant setups or those with strict memory isolation see reduced risk, but are not completely immune if sensitive data is reused within the same process or service.
Exploit preconditions
- Attack surface: LLM inference APIs or services backed by vLLM using GGUF quantization.
- Required access: Ability to send inference requests or otherwise trigger GGUF dequantization and read the resulting output tensors (e.g., via application-level responses or debugging interfaces).
- Trigger point: Execution of the GGUF dequantize CUDA kernels where tensor dimensions are truncated and only part of the output buffer is initialized.
- Why exploitation is possible: The output tensor is allocated at full size with
torch::empty(uninitialized memory), but the kernel processes only a subset of elements due to integer truncation, leaving the remainder as residual GPU memory that can contain data from prior allocations.
Why this happens
The bug stems from incorrect handling of tensor dimensions in the GGUF dequantize CUDA kernels, where integer truncation leads to a mismatch between the allocated tensor size and the number of elements actually processed.
This maps to CWE-681: Incorrect Conversion between Numeric Types, since dimension calculations are affected by type conversion or truncation, causing the kernel to iterate over fewer elements than the buffer size.
As a result, large portions of the output tensor remain uninitialized, yet are treated as valid output and may be propagated up the stack.
When this uninitialized buffer contains remnants of previous GPU allocations, sensitive data can be exposed to other tenants, matching CWE-200: Exposure of Sensitive Information to an Unauthorized Actor in the context of shared GPU memory.
Potential impact
- An attacker may observe residual tensor data from other users' inference requests in shared GPU environments.
- Sensitive model inputs, intermediate representations, or outputs could be partially reconstructed from leaked tensor regions.
- Cross-tenant data isolation guarantees in multi-tenant LLM serving platforms can be undermined.
- Attackers could infer patterns about other workloads or models running on the same GPU.
General mitigation guidance
Mitigation generally focuses on eliminating uninitialized memory exposure and ensuring numeric correctness in tensor dimension handling.
Recommended patterns include:
- Aligning kernel iteration logic with the allocated tensor size by using consistent, non-truncating numeric types for dimension and index calculations.
- Explicitly initializing output tensors before use, especially in kernels that write conditionally or may skip elements, to avoid propagating residual data.
- Introducing defensive checks that validate tensor sizes and kernel launch parameters, failing closed when mismatches are detected.
- Designing multi-tenant GPU serving architectures with strong memory hygiene: aggressively clearing or reinitializing large buffers between requests, and minimizing reuse of raw, uninitialized device memory.
- Applying least-privilege and data minimization principles at the API layer, exposing only necessary outputs and avoiding direct user access to low-level tensor buffers.
How teams detect issues like this
Detection commonly involves a mix of code-level and runtime analyses:
- Static analysis rules that flag mismatches between allocated tensor sizes and loop bounds or kernel indices, especially where integer casts or truncations occur.
- Code reviews focused on CUDA kernels and tensor operations, looking for use of
torch::emptyor equivalent uninitialized allocations followed by partial writes. - Runtime testing and fuzzing that fill GPU memory with known patterns, invoke dequantize paths, and verify that outputs do not contain residual or unexpected data.
- Policy-based checks in CI/CD pipelines that require explicit initialization for large buffers and consistent numeric types for dimension calculations.
- Optional use of GPU memory debugging tools and sanitizers to detect uninitialized reads or cross-allocation data reuse during test runs.

