/

CVE-2026-30304

Severity:

CRITICAL RISK

(9.6)

Improper Input Validation / AI Prompt Injection Leading to

CVE-2026-30304

Analyzed

Vulnerability type

Category:

Improper Input Validation / AI Prompt Injection Leading to Command Execution

CWE:

CWE-20

Impact:

Arbitrary Command Execution on the User's System

Severity:

CRITICAL RISK

(9.6)

CVSS version:

3.1

Catch this CVE while code is being written

Detect vulnerable code directly in pull requests, so CVEs are fixed before merge.

Summary

AI Code's automatic terminal execution feature misclassifies attacker-influenced commands as "safe" based on model judgment alone, without robust technical safeguards.

By using prompt injection, an attacker can wrap destructive shell commands in persuasive natural language so the model labels them as safe, causing them to run without any user confirmation and enabling arbitrary command execution on the user's system.

Affected context

This vulnerability affects environments where:

  • AI Code is configured to automatically execute "safe" terminal commands, and
  • The AI model's natural-language reasoning is treated as the primary safety check for classifying commands as safe vs. destructive.

It is relevant whenever user-visible AI interactions can be influenced by untrusted text (e.g., code snippets, documentation, chat content, comments, or remote data) and those interactions directly drive shell command execution with relaxed or bypassed user approval.

Exploit preconditions

  • Attack surface: Natural-language prompts or content that the AI model consumes (chat messages, embedded instructions, code comments, or external documents).
  • Required access: Ability to provide or influence text that the AI Code assistant reads while auto-command execution is enabled.
  • Trigger point: The AI model is prompted (directly or indirectly) to generate and classify terminal commands, and then executes those labeled as "safe" without user confirmation.
  • Why exploitation is possible: The system relies on the model's subjective determination of "safe" vs. "destructive" commands, which can be manipulated through prompt injection patterns that override or confuse its instructions, leading to misclassification of harmful commands as safe.

Why this happens

The core issue is improper input validation (CWE-20) applied to AI-generated commands: natural-language reasoning by the model is treated as a security control, but there is no strong, deterministic validation layer enforcing what commands are allowed to run without confirmation.

The product design assumes the model can reliably distinguish safe from unsafe terminal commands, yet the classification logic is susceptible to prompt injection. Attackers can craft instructions that:

  • Reframe malicious actions as safe or necessary,
  • Override system-level guidance in the prompt, and
  • Induce the model to tag destructive commands as "safe."

Instead of using a strict, rule-based allowlist or dedicated policy engine, the system delegates safety decisions to a general-purpose model that processes untrusted text, causing untrusted input to directly influence the safety classification of shell commands.

Potential impact

  • Execution of arbitrary shell commands on the user's machine without explicit approval.
  • Deletion, modification, or exfiltration of local files and source code repositories.
  • Installation of malware, backdoors, or unauthorized remote-access tools.
  • Credential theft through access to configuration files, SSH keys, or tokens.
  • Unauthorized network operations, such as scanning, pivoting, or data exfiltration to attacker-controlled endpoints.

General mitigation guidance

Effective mitigation focuses on decoupling AI reasoning from command authorization and enforcing strict, deterministic controls on what can run automatically:

  • Replace model-judged "safe vs. destructive" decisions with an explicit, narrowly scoped allowlist of auto-executable commands and argument patterns.
  • Introduce a policy engine or rule-based filter that inspects commands independently of prompts and blocks or requires approval for anything outside a constrained set.
  • Treat all commands derived from content that might include untrusted text as untrusted; require explicit user confirmation for these commands regardless of model judgment.
  • Reduce privileges for the environment in which commands execute (e.g., non-admin users, sandboxing, limited filesystem and network access) to constrain damage if misclassification occurs.
  • Provide clear UI indicators and fine-grained controls so users can disable auto-execution, restrict it to read-only operations, or scope it to specific project directories.
  • Design prompts and system instructions so that safety rules are enforced in infrastructure, not only in natural-language guidance to the model.

How teams detect issues like this

Detection strategies should target both design and runtime behaviors:

  • Design and code review: Flag any feature where a model's natural-language judgment is used as the primary or sole control for classifying commands as safe, especially when it bypasses user confirmation.
  • Static analysis / policy checks: Search for code paths that:
  • Route model-generated strings directly to shell execution, and
  • Depend on a textual "safe/destructive" label from the model without a strong validation layer or allowlist.
  • Runtime monitoring: Log all AI-triggered command executions, including:
  • The original prompt or context,
  • The model's classification (safe vs. requires approval),
  • Whether user confirmation occurred.

Analyze these logs for patterns where destructive commands are executed under a "safe" label.

  • Security testing / red-teaming: Conduct structured prompt-injection tests to validate whether crafted instructions can cause misclassification, using test harnesses that exercise the auto-execution path.

Sources

SourceLink
github.comgithub.com/Secsys-FDU/LLM-Tool-Calling-CVEs/issues/2
marketplace.visualstudio.commarketplace.visualstudio.com/items?itemName=tianguaduizhang.claude-dev-china

Frequently asked

questions

Yes, in many usage scenarios. If an attacker can influence any text that the AI Code assistant consumes—such as shared project files, documentation, code comments, or chat content—they may trigger malicious commands to execute automatically on the user's machine, provided auto-execution of "safe" commands is enabled.

The attacker needs the ability to control or inject text into the context that the AI model processes. This might be access to a repository, a collaboration channel, or any integration where untrusted content is fed into the assistant. Direct shell access to the victim's machine is not required; the AI assistant becomes the execution bridge.

The safest pattern is to eliminate trust in the model as a security decision-maker for command execution. Use a strict allowlist and independent validation layer to govern which commands (if any) can run automatically, and require explicit user approval for all others—especially those that can modify state, access sensitive data, or perform network operations.

Treat AI models as untrusted helpers, not as security or policy engines. Design systems so that:

  • AI outputs are always validated by deterministic, rule-based controls before affecting the host environment.
  • Untrusted text inputs cannot directly shape what commands are allowed to run without human approval.
  • Privileges for AI-driven operations are minimized, logs are collected, and AI features that execute code or commands default to safe, non-destructive behavior with explicit user consent.
Find out what's already
exploitable in your codebase.