5 مستودعات
Executes finite data processing tasks to completion for repetitive reporting or analysis.
Distinct from Kubernetes Batch Jobs: Distinct from Kubernetes Batch Jobs: focuses on the platform's internal batch execution capability, not Kubernetes-native resources.
Explore 5 awesome GitHub repositories matching data & databases · Batch Processing Jobs. Refine with filters or upvote what's useful.
Hazelcast is a distributed data platform that combines an in-memory data grid with a stream processing engine to support real-time analytics and event-driven applications. It functions as a partitioned, distributed key-value store that replicates data across cluster nodes to provide low-latency access and high availability. The platform also serves as a distributed SQL query engine, allowing users to execute standard SQL statements against both in-memory datasets and external data sources. What distinguishes Hazelcast is its use of a distributed consensus subsystem to maintain strongly consis
Processes finite datasets to perform repetitive tasks like report generation.
Pinot is a distributed, columnar analytical database designed for high-concurrency, low-latency query processing. It functions as a real-time OLAP datastore, enabling interactive, user-facing analytics by ingesting and querying massive datasets from both streaming and batch sources. The system architecture relies on a centralized controller for cluster coordination and a distributed segment-based storage model to ensure horizontal scalability. The platform distinguishes itself through a hybrid ingestion pipeline that unifies real-time event streams and historical batch data into a single quer
Launches a standalone job that builds segments from source data and pushes them to the cluster.
هذا المشروع عبارة عن دليل شامل ومجموعة من أفضل الممارسات لاختبار تطبيقات الواجهة الخلفية لـ Node.js. يوفر مجموعة منسقة من الأنماط وأمثلة مرجعية لكتابة اختبارات الوحدة والتكامل والمكونات الموثوقة. يتميز المشروع باستراتيجيات محددة لتكامل الواجهة الخلفية، بما في ذلك طرق مفصلة لاختبار عقد API مقابل مواصفات OpenAPI والمخططات المشتركة. ويقدم توجيهاً متخصصاً حول إدارة اختبار طابور الرسائل، مع التركيز على التكرار (idempotency)، والمرونة، ومزامنة الأحداث غير المتزامنة. يغطي الدليل نطاقاً واسعاً من مجالات القدرات، بما في ذلك عزل حالة قاعدة البيانات والتنظيف، ومحاكاة التبعيات الخارجية عبر معترضات الشبكة (network interceptors) والبدائل المكتوبة (type-safe stubs)، وأتمتة البنية التحتية للاختبار المعبأة (containerized). كما يتناول تحسين الأداء لمجموعات الاختبار من خلال استخدام أقراص RAM ومحركات الذاكرة لتخزين البيانات.
Provides methods to test message queue consumers using batches with mixed failures to ensure system resilience.
This project is a Python software development kit and framework for building applications that integrate with large language models. It serves as a multimodal content generator and vector embedding library, enabling the production and editing of text, images, audio, and video. The toolkit provides specialized capabilities for adapting base models through supervised and reinforcement training. It further distinguishes itself by offering tools for orchestrating complex workflows, including stateful chat sessions, the enforcement of structured output via schemas, and the integration of external
Implements batch processing jobs for handling large volumes of requests with configurable dataset inputs and outputs.
Good Job is a background job processor for Ruby on Rails that utilizes a PostgreSQL database as its primary storage engine. By leveraging relational database transactions, it ensures persistent and reliable task execution, integrating directly with the Active Job framework to handle asynchronous operations and recurring job scheduling within existing application environments. The system distinguishes itself through an in-process execution model that allows background workers to run within the same process as the web server, simplifying deployment by removing the need for separate worker servi
Allows grouping multiple tasks into a single collection to track collective progress and trigger lifecycle callbacks.