Summary
Amazon SageMaker Python SDK versions prior to 3.2.0 and 2.256.0 expose the ModelBuilder HMAC signing key in cleartext within the response to the DescribeTrainingJob API.
An attacker with permissions to call this API and modify objects in the training job's S3 output location may be able to upload arbitrary artifacts that are executed on subsequent training job runs.
Affected context
This vulnerability affects environments where the Amazon SageMaker Python SDK (pre-3.2.0 and pre-2.256.0) is used with ModelBuilder-based training workflows.
The issue manifests when:
- The
DescribeTrainingJobAPI is used to inspect training jobs, and - The response includes the ModelBuilder HMAC signing key in cleartext, and
- The same principal (or a compromised one) can also write to the S3 output location configured for the training job.
In such settings, the HMAC key can be abused to sign or validate arbitrary artifacts that SageMaker may trust and execute in later training invocations.
Exploit preconditions
- Attack surface: AWS SageMaker control plane via the
DescribeTrainingJobAPI and the S3 output bucket for training jobs. - Required access:
- Ability to invoke
DescribeTrainingJoband read its response, and - Permissions to create, overwrite, or modify artifacts in the training job's S3 output location.
- Trigger point: A subsequent reuse of the training job configuration or ModelBuilder pipeline that pulls and executes artifacts from the S3 output location, trusting signatures derived from the exposed HMAC key.
- Why exploitation is possible: The HMAC signing key, intended to provide integrity or authenticity for artifacts, is exposed in cleartext in an API response. An attacker with both API read access and S3 write access can use the key to introduce arbitrary artifacts that appear valid to the training pipeline, leading to execution of attacker-controlled content.
Why this happens
This vulnerability stems from exposing a sensitive cryptographic secret—the ModelBuilder HMAC signing key—in cleartext within an API response that is not strictly limited to highly privileged, internal-only consumption.
Conceptually, this maps to CWE-319: Cleartext Transmission of Sensitive Information, but in a broader sense it is an example of leaking a signing secret across a trust boundary where it can be combined with existing permissions to subvert integrity protections.
Instead of keeping signing keys confined to secure storage mechanisms and backend components, the key is included in the DescribeTrainingJob response, effectively distributing the secret to any caller with that API permission.
Once the key is available to an attacker with S3 write access, the integrity model is broken: the attacker can craft or alter artifacts, sign or otherwise align them with expectations derived from the key, and have them accepted and executed by the training job on subsequent runs.
Potential impact
- An attacker may upload malicious model artifacts or training outputs that are executed or consumed by subsequent training jobs.
- The attacker could embed arbitrary code in training containers or preprocessing scripts, potentially leading to arbitrary code execution in the training environment.
- Model integrity can be compromised, resulting in data poisoning, backdoored models, or manipulated training outcomes.
- Sensitive training data or environment credentials accessible during training could be exfiltrated via malicious artifacts.
- Downstream systems that consume trained models could be impacted by corrupted or malicious models, causing incorrect predictions or enabling further compromise.
General mitigation guidance
Effective mitigation focuses on preventing sensitive signing keys from being exposed and on compartmentalizing permissions around training artifacts:
- Ensure cryptographic keys used for signing or validating artifacts are never returned in API responses; keep them confined to secure backend components or dedicated key management systems.
- Apply strict least-privilege policies so that principals that can call inspection APIs (like
DescribeTrainingJob) do not automatically have write access to S3 output locations, and vice versa. - Treat training output locations as sensitive: limit write access, enforce strong IAM boundaries, and use separate identities for read, write, and execution roles.
- Favor design patterns where integrity of artifacts is validated using keys that are only available inside the execution environment and not exposed through public or semi-public APIs.
- Introduce defense-in-depth controls such as explicit allow-lists or manifests for executable artifacts, and avoid automatically executing any artifact solely based on its presence or expected structure.
How teams detect issues like this
Detection typically involves both configuration review and monitoring for suspicious interactions:
- Configuration and IAM review:
- Examine IAM policies to identify principals that have both
sagemaker:DescribeTrainingJoband write access to the relevant S3 output buckets. - Review SDK usage to confirm whether sensitive fields (like signing keys) are handled or logged outside trusted boundaries.
- Static and policy analysis:
- Use policy-as-code tools (e.g., IAM linter or CSPM tools) to flag roles that combine SageMaker describe permissions with broad S3 write permissions to training output paths.
- Incorporate static analysis or code review checks that highlight handling of cryptographic keys or sensitive configuration values in application logic.
- Runtime monitoring:
- Monitor CloudTrail logs for patterns such as repeated
DescribeTrainingJobcalls followed by unusual S3 write operations to training outputs. - Track changes to S3 objects in training output locations and alert on unexpected writers or anomalous modification patterns.
- Automated code and configuration review tools may be integrated into CI/CD pipelines to flag risky permission combinations or secret-handling patterns.

