Summary
A prompt injection vulnerability in the _route_using_overviews function of backend/server.py in PromtEngineer localGPT allows crafted input to influence how prompts are constructed and routed to the underlying LLM.
When attacker-controlled content is incorporated into prompts without adequate sanitization or isolation, remote users may be able to override system instructions, exfiltrate sensitive data, or cause unintended actions by the model.
Affected context
The issue arises in deployments of localGPT where user-supplied text, document overviews, or request parameters are passed into the LLM Prompt Handler.
Specifically, the _route_using_overviews function in backend/server.py appears to combine user-facing content with system or application-level instructions before requesting a response from the language model.
In environments where responses can influence downstream processing, access internal knowledge bases, or interact with other tools, this blending of trusted and untrusted content without clear separation increases the risk that an attacker can subvert the intended behavior of the assistant.
Exploit preconditions
- Attack surface: Remote, via HTTP or API endpoints that call the LLM Prompt Handler (e.g., chat, query, or retrieval endpoints).
- Required access: Ability to submit or control input that becomes part of the "overviews" or prompt text, such as user queries, uploaded documents, or metadata fields.
- Trigger point: Invocation of
_route_using_overviewsinbackend/server.py, which constructs prompts using attacker-controlled content and forwards them to the LLM. - Why exploitation is possible: The code path treats untrusted content as part of trusted instructions or context without sufficient neutralization, encoding, or isolation, allowing attacker input to alter prompt semantics or override higher-priority guidance.
Why this happens
At its core, this vulnerability is an injection bug in prompt construction logic.
The _route_using_overviews function appears to combine user-derived "overviews" with system prompts or routing instructions in a single, unstructured text channel.
This maps to CWE-74: Improper Neutralization of Special Elements in Output Used by a Downstream Component, because untrusted input is allowed to shape the structure and meaning of the prompt that is consumed by the downstream LLM.
Additionally, it reflects CWE-707: Improper Neutralization, since there is no clear delimiting, encoding, or role separation between trusted and untrusted segments.
Conceptually, the bug class is: "treating user content as if it were safe to mix directly with control instructions for a generative model," enabling the model to interpret attacker text as commands rather than mere data.
Potential impact
- Attackers may coerce the model into ignoring or bypassing system and developer instructions.
- Sensitive information present in retrieved overviews or internal context may be exposed or summarized against policy.
- The model can be induced to generate misleading, harmful, or policy-breaking outputs that appear to come from a trusted assistant.
- If responses are used by automated agents, pipelines, or tools, attacker-driven model output could indirectly affect external systems or data flows.
- Logs, analytics, or other observability data connected to model responses may be polluted with attacker-controlled content.
General mitigation guidance
Effective mitigation focuses on isolating untrusted content from control instructions and reducing the authority of model responses:
- Role and channel separation: Structure prompts so that system and developer instructions are clearly separated from user content, using role-based message formats where possible.
- Context delimiting: Wrap user-provided overviews and text in clear markers or templates (e.g., quoting, fenced blocks, explicit "data-only" sections) so the model is guided to treat them as content, not commands.
- Input constraints: Apply strict validation and normalization to user inputs, limiting length, disallowing unexpected control-like patterns where feasible, and avoiding direct interpolation of raw text into instruction fields.
- Least-privilege model usage: Avoid granting a single prompt end-to-end control over powerful tools or sensitive data; use intermediary components to filter and validate model outputs before enacting them.
- Policy-aware prompt design: Encode explicit instructions telling the model to treat user content as untrusted data, to ignore attempts to modify core policies, and to avoid executing arbitrary instructions from retrieved text.
- Defense-in-depth: Combine prompt-level defenses with external controls such as access controls on data sources, response post-processing, and guardrails that detect and block high-risk outputs.
How teams detect issues like this
Detection centres on surfacing unsafe prompt construction and high-risk model behaviors:
- Static analysis and code scanning:
- Search for string concatenation or f-string interpolation where user input or document text is merged directly into system prompts or routing templates.
- Flag functions like
_route_using_overviewsthat construct prompts from mixed trust sources without clear role or channel separation. - Enforce patterns where prompt-building uses structured message types (system/user/assistant) instead of free-form strings.
- Code review signals:
- Pay attention to paths where retrieved documents, summaries, or "overviews" are appended near instructions such as "ignore previous directions" or tooling commands.
- Look for any logic that allows user-controlled fields to influence which prompt template, chain, or tool is selected.
- Runtime monitoring:
- Log prompts and responses (with privacy controls) and analyze for patterns of instruction override, data exfiltration attempts, or unusual routing behavior.
- Use model-specific safety tooling or guardrail frameworks to detect and block responses that indicate successful injection (e.g., the model acknowledging it has overridden system instructions).
- Policy enforcement:
- Integrate automated checks into CI/CD to enforce LLM security guidelines, such as "no direct interpolation of untrusted content into system prompts."

