awesome-repositories.com
博客
awesome-repositories.com

通过 AI 驱动的搜索,发现最优秀的开源仓库。

探索精选搜索开源替代品自托管软件博客网站地图
项目关于排名机制媒体报道MCP 服务器
法律隐私政策服务条款
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
·

9 个仓库

Awesome GitHub RepositoriesHTTP Content Retrievers

Utilities for fetching web content via HTTP requests to be used as sources in data pipelines.

Distinct from Data Retrieval Pipelines: Focuses on HTTP-specific retrieval as a pipeline source, whereas Data Retrieval Pipelines is the broader system for normalization.

Explore 9 awesome GitHub repositories matching data & databases · HTTP Content Retrievers. Refine with filters or upvote what's useful.

Awesome HTTP Content Retrievers GitHub Repositories

用 AI 发现最棒的仓库。我们将通过 AI 为您搜索最匹配的仓库。
  • netflix/falcorNetflix 的头像

    Netflix/falcor

    10,572在 GitHub 上查看↗

    Falcor is a JavaScript library that models remote data as a single virtual JSON graph, providing a path-based query engine for efficient client-side data retrieval and updates. It represents multiple remote data sources as a unified document where entities are accessed via globally unique identity paths. The system distinguishes itself by treating the remote data model as a virtual JSON resource, allowing the client to query specific paths without managing individual endpoints. It uses a reference-aware graph model to handle many-to-many relationships and prevents data duplication. Network ef

    Fetches JSON graph data from remote servers using HTTP or HTTPS protocols.

    JavaScript
    在 GitHub 上查看↗10,572
  • xianhu/learnpythonxianhu 的头像

    xianhu/LearnPython

    8,484在 GitHub 上查看↗

    LearnPython is a programming tutorial consisting of a collection of practical code examples used to demonstrate Python language features and programming patterns. It serves as a comprehensive learning resource that implements core language concepts through functional code. The project provides specialized guides and samples covering several key domains. These include asynchronous network programming with event loops and coroutines, data visualization using numerical datasets for 2D and 3D plots, and web scraping for fetching content and automating login flows. It also features instructions on

    Fetches web content via HTTP requests and processes responses as plain text or raw bytes.

    Jupyter Notebooklearning-pythonpythonpython-flask
    在 GitHub 上查看↗8,484
  • bblanchon/arduinojsonbblanchon 的头像

    bblanchon/ArduinoJson

    7,176在 GitHub 上查看↗

    ArduinoJson is a C++ library for parsing and manipulating JSON data and MessagePack binary streams on microcontrollers with limited memory and processing power. It provides the core primitives necessary for embedded data serialization and parsing, enabling devices to exchange structured data over serial or network interfaces. The library is distinguished by its focus on microcontroller memory management, employing strategies such as pool-based allocation, string deduplication, and non-owning string views to minimize RAM usage. It further optimizes for constrained environments by allowing cons

    Provides utilities to fetch JSON documents from remote servers via HTTP requests.

    C++arduinoarduino-libraryc-plus-plus
    在 GitHub 上查看↗7,176
  • bitfield/scriptbitfield 的头像

    bitfield/script

    6,991在 GitHub 上查看↗

    This project is a Go shell scripting library and framework designed for writing automation scripts and CLI tools. It provides a concurrent data pipeline system for chaining sources, filters, and sinks to process text and JSON streams. The library distinguishes itself through a comprehensive toolkit for shell-like operations, including a text processing engine for regular expression filtering and frequency analysis, a filesystem utility toolkit for recursive search and path manipulation, and an integrated HTTP client wrapper for building data pipelines that fetch web content. The capability s

    Provides tools for retrieving web data using GET or POST requests to feed into processing pipelines.

    Gocatcurlcut
    在 GitHub 上查看↗6,991
  • balloonwj/cppguideballoonwj 的头像

    balloonwj/CppGuide

    6,030在 GitHub 上查看↗

    CppGuide is a curated collection of educational resources and practical guides focused on C++ server development, Linux kernel internals, concurrent programming, network protocols, and security exploitation. It provides structured learning paths for backend developers, covering everything from interview preparation to building high-performance network servers and understanding operating system fundamentals. The guide distinguishes itself by offering in-depth, hands-on tutorials that walk through real-world implementations, including building a Redis-like server from scratch, designing custom

    Retrieves web content via HTTP requests for testing and debugging endpoints.

    在 GitHub 上查看↗6,030
  • remitchell/python-scrapingREMitchell 的头像

    REMitchell/python-scraping

    4,714在 GitHub 上查看↗

    该项目是一个 Python 网络爬虫库和自动化数据收集套件。它提供了从网站提取结构化数据的工具,实现了用于导航站点链接的网络爬虫,并能解析 HTML DOM 结构以隔离特定的元素和属性。 该工具包包含一个用于处理非结构化文本和清洗原始 Web 内容以提取有意义信息的流水线。它还具备图像数据提取功能,并支持集成外部 API 以从远程端点检索结构化数据。 该系统涵盖了广泛的功能领域,包括自动化 Web 数据提取、网络爬虫工作流,以及使用代理和验证码(CAPTCHA)求解器绕过爬取障碍的技术。

    Provides HTTP content retrievers to fetch raw web content as sources for data pipelines.

    Jupyter Notebook
    在 GitHub 上查看↗4,714
  • kkdai/youtubekkdai 的头像

    kkdai/youtube

    3,903在 GitHub 上查看↗

    This project is a Go library and command-line utility designed for the retrieval and local archival of remote video content. It provides a programmatic interface for fetching media streams, allowing users to extract metadata and download video files directly to local storage. The library distinguishes itself through its ability to resolve playback restrictions by performing algorithmic transformations on obfuscated authentication tokens. This signature decryption process enables the tool to bypass standard access limitations, while its interface-driven design allows for the selection of speci

    Provides utilities for fetching web content via HTTP requests to be used as sources in data pipelines.

    Gogoyoutube
    在 GitHub 上查看↗3,903
  • jsoma/tabletopjsoma 的头像

    jsoma/tabletop

    3,771在 GitHub 上查看↗

    Tabletop is a JavaScript library and data parser designed to retrieve data from public Google Sheets and convert it into structured JSON objects. It functions as a client-side tool for fetching remote spreadsheet data and transforming rows into lists of objects or arrays for use in web applications. The library enables the use of Google Sheets as a lightweight database, allowing for dynamic content management where application data can be updated by editing a spreadsheet. It supports selective worksheet retrieval to limit the amount of transferred data and provides programmatic interfaces for

    Fetches raw spreadsheet content from remote URLs via asynchronous HTTP requests to use as data sources.

    JavaScript
    在 GitHub 上查看↗3,771
  • gpac/gpacgpac 的头像

    gpac/gpac

    3,205在 GitHub 上查看↗

    GPAC is an open-source multimedia framework built around a pluggable filter graph pipeline, where modular processing units called filters connect into a directed graph to handle media workflows. At its core, the framework centers all media packaging and manipulation on the ISO Base Media File Format (ISOBMFF), with specialized tools for reading, writing, fragmenting, and encrypting MP4 and related containers. It also provides a declarative scene graph composition system for describing interactive multimedia scenes using MPEG-4 BIFS, X3D, SVG, or VRML syntax, alongside a hardware-accelerated re

    Downloads remote media content via HTTP and feeds it as raw data blocks into the processing pipeline.

    Catsc3broadcastcenc
    在 GitHub 上查看↗3,205
  1. Home
  2. Data & Databases
  3. Data Retrieval Pipelines
  4. HTTP Content Retrievers

探索子标签

  • Graph Data RetrievalFetching structured graph-based data over HTTP protocols. **Distinct from HTTP Content Retrievers:** f3_mt2 focuses on pipeline sources; this specifically targets the retrieval of virtual JSON graph paths.