Summary
MetaGPT up to version 0.8.1 contains a code injection vulnerability in the check_solution function used by the HumanEvalBenchmark and MBPPBenchmark components.
When attacker-controlled content is incorporated into dynamically evaluated code, a remote attacker may execute arbitrary code in the environment running the benchmarks.
Affected context
The issue arises in environments running MetaGPT's HumanEval and MBPP benchmark components where:
- User- or model-generated solutions are programmatically evaluated by
check_solution - The benchmark framework constructs and executes code based on dynamic inputs
- There is insufficient neutralization or validation of content that becomes part of executable code
This typically impacts systems that automatically run code for evaluation or testing of generated solutions.
Exploit preconditions
- Attack surface: Remote interface or workflow that accepts code or solution submissions processed by MetaGPT benchmarks.
- Required access: Ability to supply or manipulate the solution or related input that
check_solutionincorporates into executable code. - Trigger point: Execution of the benchmark evaluation process, where
check_solutionbuilds and runs code to assess correctness. - Why exploitation is possible: Untrusted input is interpolated into code strings or evaluation contexts without proper escaping, isolation, or policy enforcement, enabling injection of arbitrary statements.
Why this happens
The root cause is the construction and execution of dynamic code that incorporates untrusted content without sufficient neutralization or confinement.
In the context of check_solution for HumanEval/MBPP, attacker-controlled data is likely concatenated into source code or passed to an evaluator (such as exec, eval, or equivalent dynamic execution mechanisms).
This aligns with:
- CWE-94: Improper Control of Generation of Code, where external input influences generated code that is then executed.
- CWE-74: Improper Neutralization of Special Elements, where special characters or tokens in the input are not escaped, allowing them to break out of the intended structure and inject new behavior.
The insecure pattern is treating evaluation-time input as trusted and executable, instead of treating it as data subject to strict validation and sandboxing.
Potential impact
- Execution of arbitrary code in the context of the process running MetaGPT benchmarks
- Access to sensitive data available to the evaluation environment (e.g., environment variables, local files, credentials)
- Lateral movement within the host or CI agent running the benchmark suite
- Tampering with benchmark results, logs, or artifacts
- Potential use of the compromised environment as a pivot point for broader infrastructure attacks
General mitigation guidance
Risk reduction focuses on eliminating direct code execution paths from untrusted input and constraining evaluation environments:
- Avoid dynamically constructing code strings from user- or model-controlled input; prefer explicit, parameterized calling patterns instead of
eval/exec-style evaluation. - If dynamic execution is unavoidable, use strict whitelisting and structural validation to ensure input matches expected patterns (e.g., function bodies without imports or system calls).
- Introduce sandboxed execution environments with strong process-level isolation, restricted file system access, and minimal privileges for any code being evaluated.
- Treat benchmark input as untrusted: sanitize or neutralize special characters that can alter control flow or break out of expected code templates.
- Apply least-privilege principles so that the benchmark process has only the access it strictly needs, reducing impact if injected code runs.
- Separate evaluation infrastructure from production systems to prevent cross-environment compromise.
How teams detect issues like this
Detection can combine code-level analysis and runtime monitoring:
- Use static analysis to flag calls to dynamic execution APIs (e.g.,
exec,eval,compile,ast.literal_evalmisuse) where arguments are derived from external or model-generated input. - Incorporate secure coding checks in code review to identify string concatenation or f-string interpolation that builds executable code from untrusted data.
- Enforce secure coding policies in CI that fail builds when prohibited patterns (e.g., unguarded
execon variable content) are detected. - Apply runtime monitoring or sandbox auditing to detect unexpected system calls, file operations, or network activity originating from benchmark evaluation processes.
- Use automated review tools that understand data flow to trace untrusted inputs from interfaces (APIs, CLI, files) into dynamic code execution sinks.

