Summary
SGLang's /v1/rerank endpoint can reach remote code execution when it loads a model file that defines a malicious tokenizer.chat_template.
Because Jinja2 templates are rendered with an unsandboxed jinja2.Environment(), attacker-controlled templates can execute arbitrary code during request handling.
Affected context
The vulnerability appears in SGLang deployments that:
- Expose the
/v1/rerankAPI endpoint - Load model artifacts that include a
tokenizer.chat_templatefield - Render Jinja2-based chat templates using a plain, unsandboxed
jinja2.Environment()
It becomes relevant when model files originate from locations that are not fully trusted, such as external registries, shared model hubs, or user-uploaded artifacts.
Exploit preconditions
- Attack surface: The /v1/rerank HTTP API combined with the model loading mechanism.
- Required access: Ability to supply or influence a model file that SGLang will load (e.g., uploading a model, pointing the service to a particular model path, or compromising the model supply chain).
- Trigger point: Loading a model file containing a crafted
tokenizer.chat_templatefollowed by a request that causes the template to be rendered by the reranking endpoint. - Why exploitation is possible: The Jinja2 template engine runs with a full, unsandboxed environment, allowing template expressions to access powerful Python objects and execute logic beyond simple formatting.
Why this happens
The core issue is the use of a powerful templating engine (Jinja2) as if it were a harmless string interpolation mechanism, without sandboxing or constraining available features.
Jinja2 templates rendered through a generic jinja2.Environment() can access attributes, filters, and constructs that amount to executing attacker-supplied code when the template content is not trusted.
This maps to CWE-94: Improper Control of Generation of Code ('Code Injection'), where text interpreted by a runtime (here, the template engine) is influenced by untrusted model metadata and executed with the application's privileges.
In this case, the tokenizer.chat_template field from model files is treated as trusted logic, not as untrusted data, creating a code injection channel.
Potential impact
- Arbitrary code execution with the privileges of the SGLang process
- Full compromise of the host running the reranking service or its container
- Access to API keys, model weights, and sensitive in-memory data
- Tampering with inference results or reranking behavior
- Lateral movement within the environment where SGLang is deployed
General mitigation guidance
Effective mitigation focuses on treating templates from model artifacts as untrusted input and strictly limiting what the template engine can do:
- Prefer sandboxed or restricted Jinja2 environments, disabling access to dangerous attributes, filters, and Python internals.
- Consider using data-only formatting mechanisms (e.g., simple variable substitution) instead of a general-purpose template language for untrusted sources.
- Enforce a fixed, application-controlled set of templates and avoid loading template logic from model files whenever possible.
- Apply allowlist-based validation on template content (e.g., only permit variable interpolation and simple control flow) before rendering.
- Run the reranking service under least-privilege accounts and apply OS/container sandboxing to limit the blast radius if a template is abused.
- Separate template configuration from model artifacts so that model downloads cannot automatically introduce new executable template logic.
How teams detect issues like this
Detection typically combines code-level inspection with configuration and runtime monitoring:
- Use static analysis or code search to identify calls to
jinja2.Environment()orEnvironment.from_string()/get_template()where the template content can originate from model metadata or other untrusted sources. - During code review, flag any pattern where templates are loaded from external files, model configs, or user-controlled data without explicit sandboxing or allowlisting.
- Add automated CI checks to enforce the use of a hardened, shared template environment instead of ad-hoc Jinja2 environments.
- Log template sources and model identifiers used by the reranking endpoint, and monitor for unexpected or newly introduced templates.
- Optionally, use security testing tools or fuzzing to exercise template rendering paths with crafted templates to ensure sandbox policies are effective.

