12 个仓库
Techniques for retrieving large datasets in sequential chunks.
Distinguishing note: Focuses on cursor-based API retrieval patterns.
Explore 12 awesome GitHub repositories matching web development · API Pagination. Refine with filters or upvote what's useful.
PostHog is a comprehensive product analytics and feature management platform designed to capture, process, and visualize user behavior data. It provides a unified suite for tracking application events, managing feature rollouts, and monitoring system health through session recordings and error tracking. By leveraging a columnar-storage-optimized architecture, the platform enables high-performance aggregation and filtering across massive event datasets. What distinguishes PostHog is its integrated approach to data pipelines and application control. It features a robust event ingestion system t
Retrieves large datasets using cursor-based links to manage memory and connection efficiency.
This project is a comprehensive link management and marketing attribution platform designed for creating, tracking, and analyzing shortened URLs. It functions as a centralized hub for marketing analytics, providing tools to monitor link performance, visualize conversion funnels, and manage affiliate programs through a unified dashboard. The platform distinguishes itself by integrating advanced attribution modeling and partner management directly into the link infrastructure. It supports complex marketing workflows, including automated commission calculations, fraud detection, and payout distr
Retrieves large collections of resources in manageable chunks using standard pagination parameters.
DataEase is an open-source, self-hosted business intelligence platform designed for building interactive data visualizations and managing analytical reporting. It provides a centralized environment where users can construct dashboards through a drag-and-drop interface, connecting to diverse data sources including relational databases, data warehouses, and external APIs. The platform distinguishes itself through its focus on embedded analytics and enterprise-grade governance. It allows for the seamless integration of charts, dashboards, and management modules into third-party web applications
Retrieves large datasets by mapping pagination parameters to request headers or query strings.
Got is a promise-based HTTP request library for Node.js that supports HTTP/2 and streaming. It provides a system for making network requests with a focus on asynchronous control flow and type-safe API client development. The library is distinguished by its middleware-based request lifecycle, which uses interceptors and plugins to modify request options and response data. It includes a configurable automatic retry mechanism with backoff strategies, a built-in HTTP response cache, and a cookie-jar system for maintaining persistent sessions. Broad capabilities cover data handling through duplex
Implements a mechanism for retrieving multiple pages of API results by following links or updating parameters.
Cachet is a self-hosted, open-source status page system designed to communicate service uptime, incident history, and infrastructure performance to end users. It provides a centralized dashboard for managing the operational lifecycle of system components, tracking service disruptions, and scheduling maintenance windows. The platform distinguishes itself through a comprehensive RESTful API that enables programmatic status page management and automated incident reporting. It supports deep integration with external monitoring tools, allowing for the synchronization of performance metrics and the
Implements pagination to manage large datasets and ensure efficient data retrieval.
This is a tool for searching, downloading, and archiving articles and engagement metadata from WeChat official accounts. It functions as a web-based content scraper and data exporter, allowing for the automated retrieval of social media content and the collection of performance metrics. The project distinguishes itself through a system that captures session credentials and authentication cookies from desktop clients via a local proxy to access private engagement data. It utilizes a concurrent proxy-pool fetching mechanism to download large volumes of content while avoiding rate limits, and it
Retrieves complete article histories by iteratively requesting sequential pages of data.
This project is a comprehensive Node.js software development kit designed for integrating large language models into applications. It serves as a foundational client for interacting with REST and WebSocket services, enabling developers to implement chat functionality, multimodal content generation, and autonomous agent orchestration. The library provides a structured framework for defining executable tools and enforcing JSON schemas, ensuring that model outputs remain programmatically compatible with downstream systems. The SDK distinguishes itself through its robust request orchestration and
Implements asynchronous pagination for efficient retrieval of large API datasets.
The AWS SDK for Go is a software development kit and cloud infrastructure library used to programmatically interact with AWS cloud services. It provides a programmatic interface for authenticating requests, executing operations against regional endpoints, and automating the creation and management of cloud resources. The toolkit utilizes model-driven code generation to create service clients and employs a session-based configuration system to share credentials and global settings across those clients. It implements signature-based request authentication to verify identity and integrity throug
Implements cursor-based retrieval patterns to process large datasets from AWS services in sequential chunks.
Elsa Core is a workflow engine framework designed for defining, executing, and managing long-running business processes. It functions as a distributed workflow orchestrator and event-driven trigger system, capable of operating as a multi-tenant platform with secure data isolation. The project distinguishes itself through a flexible approach to workflow definitions, supporting a visual drag-and-drop designer, programmatic C# definitions, and portable JSON specifications. It provides a highly extensible architecture allowing for the development of custom activities and the use of a dynamic expr
Implements techniques for retrieving large datasets in sequential chunks via API query parameters.
TubeArchivist is a self-hosted YouTube video archiving system and metadata indexer. It functions as a personal media library and download manager that allows users to create a searchable offline collection of videos, channels, and playlists. The system distinguishes itself by indexing subtitles, comments, and channel information for full-text search and retrieval. It features automated media synchronization to track subscriptions and playlists, ensuring new content is automatically queued and downloaded as it is published. The project provides a broad set of capabilities for digital asset ma
Implements pagination for large API datasets to improve loading performance via query parameters.
This is a tool for downloading videos, images, and audio from the Douyin social media platform using shareable URLs or profile links. It can download individual posts, entire user profiles including all posts and liked content, collections, and music tracks, with options for watermark-free and high-quality output. The tool also supports live stream recording, comment collection, and keyword-based content search with JSONL export. The project distinguishes itself through an integrated REST API server that accepts download and transcription requests, tracks job status, and exposes health check
Launches a browser to manually solve CAPTCHAs when API pagination is blocked during profile downloads.
Pagy 是一个与框架无关的 Ruby 分页库,用于将大型数据集拆分为编号页面。它提供了多种数据拆分实现,包括用于标准集合的基于偏移量的切片,以及用于海量数据集的高性能键集(keyset)分页。 该库包含一个分页 UI 生成器,用于渲染可访问的 HTML 导航栏,以及一个专用的 REST API 分页工具,可生成符合 RFC 标准的链接头和 JSON 元数据。它还具有用于拆分 Elasticsearch、Meilisearch 和 Typesense 等搜索引擎结果的专用系统,以及用于将记录组织成日历单位的基于时间的分页功能。 其广泛的功能涵盖了数组和数据库集合的数据管理,包括记录计数记忆化和页面大小重新平衡。该项目还提供全面的 UI 和可访问性工具,例如 ARIA 标签、从右到左(RTL)布局支持以及与各种 CSS 框架的集成。安全性通过分页参数清理和针对越界请求的错误处理来解决。 该系统可通过全局默认值的中央配置进行扩展,并允许通过模块预置(prepending)覆盖核心逻辑。
Provides comprehensive tools for implementing API pagination including markers and metadata headers.