1 repository
Awesome GitHub RepositoriesRequest Context Sharers
Mechanisms for sharing data between independent collector instances.
Distinguishing note: Focuses on cross-collector state sharing.
Explore 1 awesome GitHub repository matching web development · Request Context Sharers. Refine with filters or upvote what's useful.
Awesome Request Context Sharers GitHub Repositories
Găsește cele mai bune repo-uri cu AI.Vom căuta cele mai potrivite repository-uri folosind AI.
- gocolly/colly
gocolly/colly
25,101Vezi pe GitHubColly is a high-performance web scraping framework designed for the automated extraction of structured data from websites. It provides a programmable toolkit that manages the complexities of large-scale data collection, including concurrent request orchestration, automatic cookie handling, and robots.txt compliance. By utilizing an asynchronous execution model, the engine maintains high throughput while preventing resource exhaustion during recursive or distributed crawling tasks. The framework is distinguished by its modular, event-driven architecture, which allows developers to hook into sp
Passes custom data between different collector instances by storing values in the request context.
Gocrawlercrawlingframework
