Summary
A server-side request forgery (SSRF) flaw in Crawl4AI's Docker API allows the streaming crawl endpoint to fetch attacker-supplied URLs without destination validation.
By abusing the /crawl/stream path or enabling streaming on /crawl, a remote unauthenticated client can make the server access internal, private, or link-local resources and stream back their contents.
Affected context
This vulnerability occurs in deployments exposing the Crawl4AI Docker API server, specifically when the streaming crawl functionality is enabled and reachable from untrusted networks.
The issue is present in versions prior to 0.9.0 where destination validation is applied only to the non-streaming /crawl path, leaving the /crawl/stream handler and streaming mode requests unprotected.
It becomes risky when the Docker API has network visibility into internal services, private infrastructure addresses, or link-local endpoints that are not meant to be directly exposed to external clients.
Exploit preconditions
- Attack surface: HTTP API endpoints
/crawl/streamand/crawlwithcrawler_config.stream=true - Required access: Ability to send HTTP POST requests to the Crawl4AI Docker API from an untrusted or external client; no authentication required in vulnerable configurations
- Trigger point: Submitting a crawl configuration containing a seed URL pointing to an internal, private, or link-local address
- Why exploitation is possible: The streaming crawl handler (
handle_stream_crawl_request) forwards seed URLs directly to the crawler without applying SSRF destination checks, allowing the server to act as a proxy to otherwise non-public network locations
Why this happens
The underlying bug is an inconsistent enforcement of SSRF protections between different code paths in the Docker API server.
While the non-streaming /crawl handler performs destination validation to restrict which hosts may be fetched, the streaming path passes user-supplied seed URLs straight into the crawler without applying the same checks.
This maps to CWE-918: Server-Side Request Forgery (SSRF), where a web-facing service initiates network requests on behalf of a client to arbitrary destinations because input-derived URLs are trusted and not constrained by a robust allowlist or network access policy.
Potential impact
- Unauthorized access to internal HTTP services that are not intended to be internet-facing
- Exposure of sensitive configuration endpoints, metadata services, or admin panels reachable from the Docker host's network
- Use of the crawler server as a pivot point for internal network mapping and reconnaissance
- Possible interaction with cloud provider metadata APIs, which could lead to credential or token leakage
- Increased blast radius if the Docker host has broad network reach into private infrastructure segments
General mitigation guidance
Effective mitigation focuses on treating outbound requests triggered by client-controlled URLs as untrusted and constraining where the server may connect.
Common defensive patterns include:
- Applying uniform SSRF destination validation across all endpoints and code paths that initiate outbound requests, including streaming handlers and background jobs
- Restricting allowed destinations to an explicit allowlist or well-defined domain set, and denying access to internal ranges, link-local addresses, and metadata endpoints
- Separating network environments so that crawler services have minimal visibility into sensitive internal services and only the required outbound egress
- Enforcing least-privilege network and container configurations, such as limiting accessible subnets and disabling unnecessary DNS resolution paths
- Designing APIs such that clients specify high-level crawl intents or resource identifiers instead of raw arbitrary URLs, reducing the need to trust direct user-supplied network locations
How teams detect issues like this
Organizations can detect this type of SSRF risk by:
- Using static analysis or code searching to identify endpoints and handlers that accept user-supplied URLs or hostnames and initiate outbound HTTP requests without centralized validation
- Reviewing API implementations for differences in validation logic between streaming versus non-streaming paths, or between synchronous and asynchronous request handlers
- Implementing security code review checklists that highlight SSRF concerns anywhere external input influences network destinations, including in crawler configurations and job parameters
- Enforcing infrastructure policies (e.g., via container/network security platforms) that restrict outbound connections from application containers to approved destinations and log denied attempts for investigation
- Leveraging automated application security testing tools that probe APIs for SSRF behaviors by submitting URLs pointing to controlled endpoints and private address ranges

