12 dépôts
Techniques for retrieving large datasets in sequential chunks.
Distinguishing note: Focuses on cursor-based API retrieval patterns.
Explore 12 awesome GitHub repositories matching web development · API Pagination. Refine with filters or upvote what's useful.
PostHog is a comprehensive product analytics and feature management platform designed to capture, process, and visualize user behavior data. It provides a unified suite for tracking application events, managing feature rollouts, and monitoring system health through session recordings and error tracking. By leveraging a columnar-storage-optimized architecture, the platform enables high-performance aggregation and filtering across massive event datasets. What distinguishes PostHog is its integrated approach to data pipelines and application control. It features a robust event ingestion system t
Retrieves large datasets using cursor-based links to manage memory and connection efficiency.
This project is a comprehensive link management and marketing attribution platform designed for creating, tracking, and analyzing shortened URLs. It functions as a centralized hub for marketing analytics, providing tools to monitor link performance, visualize conversion funnels, and manage affiliate programs through a unified dashboard. The platform distinguishes itself by integrating advanced attribution modeling and partner management directly into the link infrastructure. It supports complex marketing workflows, including automated commission calculations, fraud detection, and payout distr
Retrieves large collections of resources in manageable chunks using standard pagination parameters.
DataEase is an open-source, self-hosted business intelligence platform designed for building interactive data visualizations and managing analytical reporting. It provides a centralized environment where users can construct dashboards through a drag-and-drop interface, connecting to diverse data sources including relational databases, data warehouses, and external APIs. The platform distinguishes itself through its focus on embedded analytics and enterprise-grade governance. It allows for the seamless integration of charts, dashboards, and management modules into third-party web applications
Retrieves large datasets by mapping pagination parameters to request headers or query strings.
Got is a promise-based HTTP request library for Node.js that supports HTTP/2 and streaming. It provides a system for making network requests with a focus on asynchronous control flow and type-safe API client development. The library is distinguished by its middleware-based request lifecycle, which uses interceptors and plugins to modify request options and response data. It includes a configurable automatic retry mechanism with backoff strategies, a built-in HTTP response cache, and a cookie-jar system for maintaining persistent sessions. Broad capabilities cover data handling through duplex
Implements a mechanism for retrieving multiple pages of API results by following links or updating parameters.
Cachet is a self-hosted, open-source status page system designed to communicate service uptime, incident history, and infrastructure performance to end users. It provides a centralized dashboard for managing the operational lifecycle of system components, tracking service disruptions, and scheduling maintenance windows. The platform distinguishes itself through a comprehensive RESTful API that enables programmatic status page management and automated incident reporting. It supports deep integration with external monitoring tools, allowing for the synchronization of performance metrics and the
Implements pagination to manage large datasets and ensure efficient data retrieval.
This is a tool for searching, downloading, and archiving articles and engagement metadata from WeChat official accounts. It functions as a web-based content scraper and data exporter, allowing for the automated retrieval of social media content and the collection of performance metrics. The project distinguishes itself through a system that captures session credentials and authentication cookies from desktop clients via a local proxy to access private engagement data. It utilizes a concurrent proxy-pool fetching mechanism to download large volumes of content while avoiding rate limits, and it
Retrieves complete article histories by iteratively requesting sequential pages of data.
This project is a comprehensive Node.js software development kit designed for integrating large language models into applications. It serves as a foundational client for interacting with REST and WebSocket services, enabling developers to implement chat functionality, multimodal content generation, and autonomous agent orchestration. The library provides a structured framework for defining executable tools and enforcing JSON schemas, ensuring that model outputs remain programmatically compatible with downstream systems. The SDK distinguishes itself through its robust request orchestration and
Implements asynchronous pagination for efficient retrieval of large API datasets.
The AWS SDK for Go is a software development kit and cloud infrastructure library used to programmatically interact with AWS cloud services. It provides a programmatic interface for authenticating requests, executing operations against regional endpoints, and automating the creation and management of cloud resources. The toolkit utilizes model-driven code generation to create service clients and employs a session-based configuration system to share credentials and global settings across those clients. It implements signature-based request authentication to verify identity and integrity throug
Implements cursor-based retrieval patterns to process large datasets from AWS services in sequential chunks.
Elsa Core is a workflow engine framework designed for defining, executing, and managing long-running business processes. It functions as a distributed workflow orchestrator and event-driven trigger system, capable of operating as a multi-tenant platform with secure data isolation. The project distinguishes itself through a flexible approach to workflow definitions, supporting a visual drag-and-drop designer, programmatic C# definitions, and portable JSON specifications. It provides a highly extensible architecture allowing for the development of custom activities and the use of a dynamic expr
Implements techniques for retrieving large datasets in sequential chunks via API query parameters.
TubeArchivist is a self-hosted YouTube video archiving system and metadata indexer. It functions as a personal media library and download manager that allows users to create a searchable offline collection of videos, channels, and playlists. The system distinguishes itself by indexing subtitles, comments, and channel information for full-text search and retrieval. It features automated media synchronization to track subscriptions and playlists, ensuring new content is automatically queued and downloaded as it is published. The project provides a broad set of capabilities for digital asset ma
Implements pagination for large API datasets to improve loading performance via query parameters.
This is a tool for downloading videos, images, and audio from the Douyin social media platform using shareable URLs or profile links. It can download individual posts, entire user profiles including all posts and liked content, collections, and music tracks, with options for watermark-free and high-quality output. The tool also supports live stream recording, comment collection, and keyword-based content search with JSONL export. The project distinguishes itself through an integrated REST API server that accepts download and transcription requests, tracks job status, and exposes health check
Launches a browser to manually solve CAPTCHAs when API pagination is blocked during profile downloads.
Pagy est une bibliothèque de pagination Ruby agnostique aux frameworks, utilisée pour diviser de grands jeux de données en pages numérotées. Elle fournit de multiples implémentations pour diviser les données, incluant le découpage basé sur les offsets pour les collections standard et la pagination par clés (keyset pagination) haute performance pour les jeux de données massifs. La bibliothèque inclut un générateur d'interface utilisateur de pagination pour le rendu de barres de navigation HTML accessibles et un outil dédié pour la pagination d'API REST qui produit des en-têtes de lien conformes à la RFC et des métadonnées JSON. Elle propose également des systèmes spécialisés pour diviser les résultats provenant de moteurs de recherche comme Elasticsearch, Meilisearch et Typesense, ainsi qu'une pagination temporelle pour organiser les enregistrements en unités calendaires. Les capacités étendues couvrent la gestion des données pour les tableaux et les collections de base de données, incluant la mémoïsation du nombre d'enregistrements et le rééquilibrage de la taille des pages. Le projet fournit également des outils complets d'interface utilisateur et d'accessibilité, tels que le marquage ARIA, le support des mises en page de droite à gauche et l'intégration avec divers frameworks CSS. La sécurité est traitée via la désinfection des paramètres de pagination et la gestion des erreurs pour les requêtes hors limites. Le système est extensible via une configuration centrale pour les valeurs par défaut globales et permet de surcharger la logique principale via le pré-pendant de modules.
Provides comprehensive tools for implementing API pagination including markers and metadata headers.