1 repositorio
Logic for directing requests to specific handler functions based on custom metadata labels assigned to the request.
Distinct from Request Routing: Distinct from general request routing by using internal labels rather than URL paths or HTTP methods to dispatch logic.
Explore 1 awesome GitHub repository matching web development · Label-Based. Refine with filters or upvote what's useful.
Crawlee-python is a web crawling framework for building scalable scrapers using Python. It serves as a comprehensive tool for web scraping automation, providing a system to extract structured data from websites using both lightweight HTTP requests and headless browser automation. The framework is distinguished by its anti-bot evasion capabilities, which include browser fingerprint impersonation and tiered proxy rotation to bypass detection systems and solve challenges such as Cloudflare. It also incorporates artificial intelligence for autonomous website navigation and schema-based data extra
Maps specific request labels to dedicated handler functions to ensure pages are processed by the appropriate logic.