12 repositorios
Techniques for retrieving large datasets in sequential chunks.
Distinguishing note: Focuses on cursor-based API retrieval patterns.
Explore 12 awesome GitHub repositories matching web development · API Pagination. Refine with filters or upvote what's useful.
PostHog is a comprehensive product analytics and feature management platform designed to capture, process, and visualize user behavior data. It provides a unified suite for tracking application events, managing feature rollouts, and monitoring system health through session recordings and error tracking. By leveraging a columnar-storage-optimized architecture, the platform enables high-performance aggregation and filtering across massive event datasets. What distinguishes PostHog is its integrated approach to data pipelines and application control. It features a robust event ingestion system t
Retrieves large datasets using cursor-based links to manage memory and connection efficiency.
This project is a comprehensive link management and marketing attribution platform designed for creating, tracking, and analyzing shortened URLs. It functions as a centralized hub for marketing analytics, providing tools to monitor link performance, visualize conversion funnels, and manage affiliate programs through a unified dashboard. The platform distinguishes itself by integrating advanced attribution modeling and partner management directly into the link infrastructure. It supports complex marketing workflows, including automated commission calculations, fraud detection, and payout distr
Retrieves large collections of resources in manageable chunks using standard pagination parameters.
DataEase is an open-source, self-hosted business intelligence platform designed for building interactive data visualizations and managing analytical reporting. It provides a centralized environment where users can construct dashboards through a drag-and-drop interface, connecting to diverse data sources including relational databases, data warehouses, and external APIs. The platform distinguishes itself through its focus on embedded analytics and enterprise-grade governance. It allows for the seamless integration of charts, dashboards, and management modules into third-party web applications
Retrieves large datasets by mapping pagination parameters to request headers or query strings.
Got is a promise-based HTTP request library for Node.js that supports HTTP/2 and streaming. It provides a system for making network requests with a focus on asynchronous control flow and type-safe API client development. The library is distinguished by its middleware-based request lifecycle, which uses interceptors and plugins to modify request options and response data. It includes a configurable automatic retry mechanism with backoff strategies, a built-in HTTP response cache, and a cookie-jar system for maintaining persistent sessions. Broad capabilities cover data handling through duplex
Implements a mechanism for retrieving multiple pages of API results by following links or updating parameters.
Cachet is a self-hosted, open-source status page system designed to communicate service uptime, incident history, and infrastructure performance to end users. It provides a centralized dashboard for managing the operational lifecycle of system components, tracking service disruptions, and scheduling maintenance windows. The platform distinguishes itself through a comprehensive RESTful API that enables programmatic status page management and automated incident reporting. It supports deep integration with external monitoring tools, allowing for the synchronization of performance metrics and the
Implements pagination to manage large datasets and ensure efficient data retrieval.
This is a tool for searching, downloading, and archiving articles and engagement metadata from WeChat official accounts. It functions as a web-based content scraper and data exporter, allowing for the automated retrieval of social media content and the collection of performance metrics. The project distinguishes itself through a system that captures session credentials and authentication cookies from desktop clients via a local proxy to access private engagement data. It utilizes a concurrent proxy-pool fetching mechanism to download large volumes of content while avoiding rate limits, and it
Retrieves complete article histories by iteratively requesting sequential pages of data.
This project is a comprehensive Node.js software development kit designed for integrating large language models into applications. It serves as a foundational client for interacting with REST and WebSocket services, enabling developers to implement chat functionality, multimodal content generation, and autonomous agent orchestration. The library provides a structured framework for defining executable tools and enforcing JSON schemas, ensuring that model outputs remain programmatically compatible with downstream systems. The SDK distinguishes itself through its robust request orchestration and
Implements asynchronous pagination for efficient retrieval of large API datasets.
The AWS SDK for Go is a software development kit and cloud infrastructure library used to programmatically interact with AWS cloud services. It provides a programmatic interface for authenticating requests, executing operations against regional endpoints, and automating the creation and management of cloud resources. The toolkit utilizes model-driven code generation to create service clients and employs a session-based configuration system to share credentials and global settings across those clients. It implements signature-based request authentication to verify identity and integrity throug
Implements cursor-based retrieval patterns to process large datasets from AWS services in sequential chunks.
Elsa Core is a workflow engine framework designed for defining, executing, and managing long-running business processes. It functions as a distributed workflow orchestrator and event-driven trigger system, capable of operating as a multi-tenant platform with secure data isolation. The project distinguishes itself through a flexible approach to workflow definitions, supporting a visual drag-and-drop designer, programmatic C# definitions, and portable JSON specifications. It provides a highly extensible architecture allowing for the development of custom activities and the use of a dynamic expr
Implements techniques for retrieving large datasets in sequential chunks via API query parameters.
TubeArchivist is a self-hosted YouTube video archiving system and metadata indexer. It functions as a personal media library and download manager that allows users to create a searchable offline collection of videos, channels, and playlists. The system distinguishes itself by indexing subtitles, comments, and channel information for full-text search and retrieval. It features automated media synchronization to track subscriptions and playlists, ensuring new content is automatically queued and downloaded as it is published. The project provides a broad set of capabilities for digital asset ma
Implements pagination for large API datasets to improve loading performance via query parameters.
This is a tool for downloading videos, images, and audio from the Douyin social media platform using shareable URLs or profile links. It can download individual posts, entire user profiles including all posts and liked content, collections, and music tracks, with options for watermark-free and high-quality output. The tool also supports live stream recording, comment collection, and keyword-based content search with JSONL export. The project distinguishes itself through an integrated REST API server that accepts download and transcription requests, tracks job status, and exposes health check
Launches a browser to manually solve CAPTCHAs when API pagination is blocked during profile downloads.
Pagy es una biblioteca de paginación para Ruby, agnóstica al framework, utilizada para dividir grandes conjuntos de datos en páginas numeradas. Proporciona múltiples implementaciones para dividir datos, incluyendo segmentación basada en desplazamiento (offset) para colecciones estándar y paginación basada en claves (keyset) de alto rendimiento para conjuntos de datos masivos. La biblioteca incluye un generador de UI de paginación para renderizar barras de navegación HTML accesibles y una herramienta dedicada para la paginación de API REST que produce encabezados de enlace compatibles con RFC y metadatos JSON. También cuenta con sistemas especializados para dividir resultados de motores de búsqueda como Elasticsearch, Meilisearch y Typesense, así como paginación basada en tiempo para organizar registros en unidades de calendario. Sus capacidades cubren la gestión de datos para arrays y colecciones de bases de datos, incluyendo la memorización del conteo de registros y el reequilibrio del tamaño de página. El proyecto también proporciona herramientas integrales de UI y accesibilidad, como etiquetado ARIA, soporte para diseño de derecha a izquierda e integración con varios frameworks CSS. La seguridad se aborda mediante la sanitización de parámetros de paginación y el manejo de errores para solicitudes fuera de rango. El sistema es extensible mediante una configuración central para valores predeterminados globales y permite anular la lógica central mediante la anteposición de módulos.
Provides comprehensive tools for implementing API pagination including markers and metadata headers.