Severity:

HIGH RISK

(7.3)

Prompt Injection / Injection in localGPT LLM Prompt Handler

CVE-2026-5002

Deferred

Vulnerability type

Category:

Prompt Injection / Injection

CWE:

CWE-74, CWE-707

Impact:

LLM Prompt Manipulation and Potential Data Exposure

Severity:

HIGH RISK

(7.3)

CVSS version:

3.1

Catch this CVE while code is being written

Detect vulnerable code directly in pull requests, so CVEs are fixed before merge.

Summary

A prompt injection vulnerability in the _route_using_overviews function of backend/server.py in PromtEngineer localGPT allows crafted input to influence how prompts are constructed and routed to the underlying LLM.

When attacker-controlled content is incorporated into prompts without adequate sanitization or isolation, remote users may be able to override system instructions, exfiltrate sensitive data, or cause unintended actions by the model.

Affected context

The issue arises in deployments of localGPT where user-supplied text, document overviews, or request parameters are passed into the LLM Prompt Handler.

Specifically, the _route_using_overviews function in backend/server.py appears to combine user-facing content with system or application-level instructions before requesting a response from the language model.

In environments where responses can influence downstream processing, access internal knowledge bases, or interact with other tools, this blending of trusted and untrusted content without clear separation increases the risk that an attacker can subvert the intended behavior of the assistant.

Exploit preconditions

  • Attack surface: Remote, via HTTP or API endpoints that call the LLM Prompt Handler (e.g., chat, query, or retrieval endpoints).
  • Required access: Ability to submit or control input that becomes part of the "overviews" or prompt text, such as user queries, uploaded documents, or metadata fields.
  • Trigger point: Invocation of _route_using_overviews in backend/server.py, which constructs prompts using attacker-controlled content and forwards them to the LLM.
  • Why exploitation is possible: The code path treats untrusted content as part of trusted instructions or context without sufficient neutralization, encoding, or isolation, allowing attacker input to alter prompt semantics or override higher-priority guidance.

Why this happens

At its core, this vulnerability is an injection bug in prompt construction logic.

The _route_using_overviews function appears to combine user-derived "overviews" with system prompts or routing instructions in a single, unstructured text channel.

This maps to CWE-74: Improper Neutralization of Special Elements in Output Used by a Downstream Component, because untrusted input is allowed to shape the structure and meaning of the prompt that is consumed by the downstream LLM.

Additionally, it reflects CWE-707: Improper Neutralization, since there is no clear delimiting, encoding, or role separation between trusted and untrusted segments.

Conceptually, the bug class is: "treating user content as if it were safe to mix directly with control instructions for a generative model," enabling the model to interpret attacker text as commands rather than mere data.

Potential impact

  • Attackers may coerce the model into ignoring or bypassing system and developer instructions.
  • Sensitive information present in retrieved overviews or internal context may be exposed or summarized against policy.
  • The model can be induced to generate misleading, harmful, or policy-breaking outputs that appear to come from a trusted assistant.
  • If responses are used by automated agents, pipelines, or tools, attacker-driven model output could indirectly affect external systems or data flows.
  • Logs, analytics, or other observability data connected to model responses may be polluted with attacker-controlled content.

General mitigation guidance

Effective mitigation focuses on isolating untrusted content from control instructions and reducing the authority of model responses:

  • Role and channel separation: Structure prompts so that system and developer instructions are clearly separated from user content, using role-based message formats where possible.
  • Context delimiting: Wrap user-provided overviews and text in clear markers or templates (e.g., quoting, fenced blocks, explicit "data-only" sections) so the model is guided to treat them as content, not commands.
  • Input constraints: Apply strict validation and normalization to user inputs, limiting length, disallowing unexpected control-like patterns where feasible, and avoiding direct interpolation of raw text into instruction fields.
  • Least-privilege model usage: Avoid granting a single prompt end-to-end control over powerful tools or sensitive data; use intermediary components to filter and validate model outputs before enacting them.
  • Policy-aware prompt design: Encode explicit instructions telling the model to treat user content as untrusted data, to ignore attempts to modify core policies, and to avoid executing arbitrary instructions from retrieved text.
  • Defense-in-depth: Combine prompt-level defenses with external controls such as access controls on data sources, response post-processing, and guardrails that detect and block high-risk outputs.

How teams detect issues like this

Detection centres on surfacing unsafe prompt construction and high-risk model behaviors:

  • Static analysis and code scanning:
  • Search for string concatenation or f-string interpolation where user input or document text is merged directly into system prompts or routing templates.
  • Flag functions like _route_using_overviews that construct prompts from mixed trust sources without clear role or channel separation.
  • Enforce patterns where prompt-building uses structured message types (system/user/assistant) instead of free-form strings.
  • Code review signals:
  • Pay attention to paths where retrieved documents, summaries, or "overviews" are appended near instructions such as "ignore previous directions" or tooling commands.
  • Look for any logic that allows user-controlled fields to influence which prompt template, chain, or tool is selected.
  • Runtime monitoring:
  • Log prompts and responses (with privacy controls) and analyze for patterns of instruction override, data exfiltration attempts, or unusual routing behavior.
  • Use model-specific safety tooling or guardrail frameworks to detect and block responses that indicate successful injection (e.g., the model acknowledging it has overridden system instructions).
  • Policy enforcement:
  • Integrate automated checks into CI/CD to enforce LLM security guidelines, such as "no direct interpolation of untrusted content into system prompts."

Sources

SourceLink
github.comgithub.com/August829/CVEP/issues/9
vuldb.comvuldb.com/submit/779122
vuldb.comvuldb.com/vuln/353889
vuldb.comvuldb.com/vuln/353889/cti

Frequently asked

questions

Yes. As described, an attacker can exploit the issue remotely by sending crafted input to any API or user interface that triggers the _route_using_overviews function, assuming that their content is incorporated into the prompt construction.

The attacker typically only needs the ability to supply or influence the text that becomes part of the overviews or user query handled by the LLM Prompt Handler.

No privileged access to the host or the underlying model is required if the application already exposes a network-accessible interface.

The most robust fix is to redesign prompt construction so that system and routing instructions are completely separated from user inputs and overviews, using structured roles and explicit delimiting.

Avoid interpolating raw user text directly into instruction fields; instead, treat it as data that the model should analyze within a constrained context.

Establish an LLM security design standard that treats all external content as untrusted, enforces strict role separation in prompts, and requires threat modeling for any feature that lets untrusted text affect model behavior.

Combine secure prompt engineering patterns with validation, monitoring, and guardrails so that even if injection is attempted, it is less likely to succeed or cause harm.

Find out what's already
exploitable in your codebase.