lexiforest/curl_cffi
Curl Cffi
curl_cffi is a Python HTTP client built on libcurl that focuses on browser fingerprint impersonation to evade anti-bot detection. By replacing default TLS handshake and HTTP/2 settings with those extracted from real browsers like Chrome and Firefox, it allows HTTP requests that closely mimic actual browser traffic, reducing the likelihood of being blocked by services that fingerprint automated clients.
Beyond fingerprint impersonation, curl_cffi offers a dual API supporting both synchronous and asynchronous execution, with per-request proxy assignment, automatic retry with exponential backoff, and persistent session-based connection pooling. It also includes WebSocket connectivity, enabling real-time two‑way communication over an existing session—features that collectively give developers fine‑grained control over request behavior while maintaining a simple, requests‑like interface.
Features
- TLS Fingerprint Impersonators - A Python HTTP client that impersonates browser TLS fingerprints to evade anti-bot detection systems.
- Anti-Bot Evasion - Evades bot detection by imitating the exact TLS and HTTP/2 fingerprints of popular browsers.
- HTTP Request Clients - Provides a synchronous and asynchronous HTTP client with a requests-like interface for executing web requests.
- Web Scraping Evasion Tools - Sends HTTP requests that mimic real browsers to scrape content from sites that block automated tools.
- Request Retries - Automatically retries failed HTTP requests with configurable backoff to handle transient network errors.
- Connection Pooling - Maintains a persistent session object that reuses TCP connections and preserves TLS state across requests.
- Automated Retry Sessions - Combines automatic retry with persistent session management for reliable data fetching from web services.
- Proxy and Fingerprint Rotation - Spreads requests across multiple proxies to hide the source IP and reduce the risk of blocking.
- Per-Request Proxy Assignments - Allows each HTTP request to use a different proxy server independently of the session configuration.
- Exponential Backoff Retries - Implements exponential backoff retry logic to automatically re-attempt failed requests with increasing delays.
- Dual API HTTP Clients - Ships both blocking and async request interfaces built on the same underlying libcurl engine.
