/

CVE-2026-31234

Severity:

CRITICAL RISK

(9.8)

Unsafe Deserialization in Horovod KVStore HTTP Server

CVE-2026-31234

Deferred

Vulnerability type

Category:

Unsafe Deserialization

CWE:

CWE-502

Impact:

Remote Code Execution

Severity:

CRITICAL RISK

(9.8)

CVSS version:

3.1

Catch this CVE while code is being written

Detect vulnerable code directly in pull requests, so CVEs are fixed before merge.

Summary

Horovod through 0.28.1 contains an unsafe deserialization vulnerability in its KVStore HTTP server, where arbitrary data written over HTTP is deserialized with cloudpickle.loads() on worker nodes.

Because the KVStore endpoint lacks authentication and integrity checks, a remote attacker can inject a malicious pickle payload that is later deserialized by a worker, leading to arbitrary code execution in distributed training environments.

Affected context

This vulnerability affects deployments using Horovod's KVStore HTTP server for distributed task coordination or parameter sharing.

The risk is present when:

  • KVStore is exposed over a network interface reachable by untrusted parties (e.g., shared clusters, cloud environments, flat internal networks), and
  • Workers retrieve values from KVStore that are deserialized using cloudpickle.loads() without verifying the origin or integrity of the stored data.

Even environments assumed to be "internal only" may be affected if network segmentation and access controls are weak or misconfigured.

Exploit preconditions

  • Attack surface: Horovod KVStore HTTP server endpoint (typically over HTTP).
  • Required access: Ability to send HTTP PUT requests to the KVStore server; no authentication or prior trust relationship is required.
  • Trigger point: A Horovod worker issues an HTTP GET to read a key from KVStore and deserializes the returned value with cloudpickle.loads().
  • Why exploitation is possible:
  • KVStore accepts arbitrary PUTs from any client and stores attacker-supplied data.
  • There is no authentication, authorization, or integrity protection on the stored values.
  • Workers blindly deserialize stored values as Python objects using cloudpickle.loads(), which can execute attacker-controlled code during deserialization.

Why this happens

The core issue is the combination of unauthenticated write access and unsafe deserialization of untrusted data.

Horovod's KVStore HTTP server allows arbitrary clients to write values via PUT, and worker processes later read those values and pass them directly to cloudpickle.loads() for deserialization.

This behavior maps to CWE-502: Unsafe Deserialization, where serialized data from an untrusted or unverifiable source is treated as trusted object state.

Because pickle/cloudpickle formats can embed code execution primitives, deserializing attacker-controlled payloads allows execution of arbitrary code within the worker process.

Potential impact

  • Remote execution of arbitrary Python code within Horovod worker processes.
  • Compromise of the distributed training job, including manipulation of model parameters and training data.
  • Data exfiltration from worker nodes, including access credentials, training datasets, and model artifacts.
  • Lateral movement within the cluster by using compromised workers as a pivot point.
  • Supply-chain style poisoning of trained models, leading to degraded performance or malicious behavior in downstream deployments.

General mitigation guidance

Effective mitigation focuses on both deserialization safety and network exposure reduction:

  • Replace unsafe deserialization (pickle/cloudpickle.loads) with safer serialization formats (e.g., JSON, protocol buffers) that do not support arbitrary code execution.
  • Introduce authentication and authorization controls on KVStore endpoints so only trusted Horovod components can write and read values.
  • Enforce integrity and origin validation for values read from KVStore (e.g., keyed MACs or signatures checked by workers before processing).
  • Restrict network exposure of KVStore to trusted hosts and subnets (e.g., internal-only bindings, firewall rules, service meshes with mTLS).
  • Run Horovod workers with least privilege, limiting filesystem and network access so that a compromise has constrained blast radius.
  • Treat distributed coordination channels as untrusted by default, explicitly validating and sanitizing all data retrieved from shared services.

How teams detect issues like this

Detection can combine code-focused and environment-focused approaches:

  • Static analysis: Flag uses of pickle.loads, cloudpickle.loads, or equivalent deserializers when applied to data coming from networked services or other untrusted channels.
  • Code review signals: Look for KVStore or similar coordination services that accept unauthenticated writes and for worker code that deserializes those values without validation.
  • Runtime monitoring: Deploy network monitoring to detect unexpected or anomalous PUT requests to KVStore endpoints, especially from non-worker hosts or unusual IP ranges.
  • Policy enforcement: Use security policies in CI/CD that disallow unsafe deserialization in new code and require authenticated, integrity-protected communication for distributed components.
  • Automated tooling: Integrate SAST/IAST tools capable of tracking data flow from HTTP handlers to deserialization sinks and alerting on unsafe paths.

Sources

SourceLink
github.comgithub.com/horovod/horovod
notion.sonotion.so/CVE-2026-31234-35d1e139318881d585cde508b9d2453c

Frequently asked

questions

Yes. As long as an attacker can reach the Horovod KVStore HTTP endpoint over the network, they can send malicious PUT requests and potentially achieve code execution on workers that consume the poisoned values.

No explicit authentication is required by the vulnerable KVStore component. However, practical exploitation depends on network-level access: an attacker must be able to send HTTP requests to the KVStore service from their vantage point.

The most robust fix is to remove unsafe deserialization from the data path altogether—switching from pickle/cloudpickle to a safe, non-executable serialization format—and enforce strong authentication and integrity checks on all KVStore operations.

Treat all data received over the network as untrusted, avoid general-purpose executable serialization formats for inter-service communication, require authentication and authorization for coordination services, and mandate security reviews for any component that both accepts remote input and deserializes or executes it.

Find out what's already
exploitable in your codebase.