13 مستودعات
Sequential processing of tasks using a queue to manage system resource consumption.
Distinct from Batch Processing: Focuses on the queueing mechanism for file processing rather than bulk data operations
Explore 13 awesome GitHub repositories matching data & databases · Task Queues. Refine with filters or upvote what's useful.
ImageOptim is a macOS image optimizer and GUI image compressor designed to reduce image file sizes for web and disk storage. It functions as a lossless image optimizer that removes unnecessary metadata and optimizes encoding to reduce file sizes without losing pixel data. The application handles macOS media asset management and web image preparation by shrinking files to improve website loading speeds and reduce storage usage. It provides both lossless and lossy compression options to maintain visual quality while reducing the disk space used by images.
Manages a queue of images to process them sequentially and prevent system resource exhaustion.
TubeArchivist is a self-hosted YouTube video archiving system and metadata indexer. It functions as a personal media library and download manager that allows users to create a searchable offline collection of videos, channels, and playlists. The system distinguishes itself by indexing subtitles, comments, and channel information for full-text search and retrieval. It features automated media synchronization to track subscriptions and playlists, ensuring new content is automatically queued and downloaded as it is published. The project provides a broad set of capabilities for digital asset ma
Processes pending items in a sequential download queue to retrieve files for offline storage.
downkyicore is a cross-platform desktop application designed for the automated batch downloading of Bilibili videos, including support for high-resolution 8K and HDR content. It functions as a batch media downloader that manages multiple files and collections simultaneously through an automated queue system. The project integrates FFmpeg for media processing to separate audio and video streams and remove watermarks. It uses specialized mechanisms to bypass access restrictions and rate limits by employing random identifiers and mobile referrers during the parsing process. The system includes
Manages a sequential queue of download tasks using a local database to handle batch processing and retries.
Osmedeus is a security workflow orchestration engine that coordinates AI agents, shell commands, and scanning tools through declarative YAML pipelines. It functions as a distributed security scanner, a declarative workflow automator, and an AI agent framework for security, enabling automated multi-step security analysis with conditional branching, parallel execution, and distributed workers. The engine distinguishes itself through a hybrid runner model that executes workflow steps on the local host, inside Docker containers, or over SSH to remote machines, selected per step or module. It supp
Enqueues scan tasks for later execution with configurable concurrency limits.
This repository is a collection of YAML automations and scripts for Home Assistant that coordinate devices and services into context‑aware home routines. Its design centers on combining multiple sensor inputs—motion, bed presence, garage door state—into a single occupancy signal that gates automations, reducing false triggers. The collection delivers actionable mobile notifications with script‑bound buttons for snoozing or escalating alerts, and integrates an external vision service to classify camera snapshots so package deliveries are detected and announced. What distinguishes this configur
Processes tasks one at a time from a persistent queue with per-task time thresholds and delay guards.
Parabolic is a graphical frontend for the yt-dlp engine, serving as a web media downloader and extractor. It provides a visual interface for saving high-quality video and audio content from various web platforms into local files. The application functions as a multi-format media exporter, allowing content to be saved into diverse audio and video containers. It includes a media metadata manager to capture and store associated information, such as subtitles and technical metadata, alongside the downloaded files. The system supports batch content acquisition through concurrent download manageme
Coordinates multiple simultaneous download tasks using a worker queue to optimize system resource consumption.
هذا تطبيق Windows للتعرف التلقائي على الكلام يقوم بنسخ الصوت المنطوق من ملفات الفيديو إلى ملفات ترجمة SRT ذات طابع زمني. يعمل كمولد ترجمة وأداة ترجمة تحول كلام الوسائط إلى نص متزامن. يعمل البرنامج كناسخ وسائط دفعي، مما يسمح بالمعالجة المتزامنة لملفات صوت وفيديو متعددة لتوليد ترجمات بشكل جماعي. يتضمن سير عمل ترجمة لتحويل النسخ بين لغات مختلفة لإنشاء ملفات ثنائية اللغة أو مترجمة محلياً. يوفر النظام أيضاً قدرات تنقيح النصوص، باستخدام التعبيرات النمطية وفلاتر مخصصة لتنظيف النسخ عن طريق إزالة كلمات الحشو والأنماط غير المرغوب فيها. يتم دعم ذلك من خلال واجهة مستخدم رسومية أصلية لـ Windows.
Uses a task queue to process multiple media files sequentially and manage system resources.
Rudder Server عبارة عن منصة بيانات عملاء وخط أنابيب توجيه أحداث مصمم لجمع وتحويل وتوجيه بيانات أحداث العملاء من مصادر مختلفة إلى مستودعات البيانات وأدوات الأعمال. يعمل كمحلل هوية عملاء، يربط المعرفات من مصادر متعددة لبناء رسم بياني موحد للهوية وملفات تعريف سلوكية شاملة للعملاء. يتميز النظام بقدرات ETL العكسية، التي تدفع شرائح العملاء والجماهير المعالجة من مستودعات البيانات مرة أخرى إلى تطبيقات الطرف الثالث التشغيلية. كما يوفر مستوى بيانات حاوية لنشر Kubernetes، مما يتيح إدارة البنية التحتية للبيانات ككود. تغطي المنصة مجموعة واسعة من قدرات إدارة البيانات، بما في ذلك تحويل الأحداث في الوقت الفعلي، والتحقق من المخطط عبر كتالوجات البيانات، وحوكمة الخصوصية. تشمل هذه أدوات لإدارة موافقة المستخدم، وفرض إقامة البيانات داخل مناطق جغرافية محددة، وإخفاء معلومات التعريف الشخصية أثناء النقل. تتم إدارة تثبيت ونشر مكونات مستوى البيانات باستخدام مخططات Helm.
Schedules bulk data uploads and synchronization tasks using a priority queue to manage system load.
UniExtract2 هي مجموعة من الأدوات المصممة لاستخراج الأرشيف العالمي، وفك الضغط المجمع، وتحليل تنسيق الملفات. تسترجع الملفات من تنسيقات مضغوطة مختلفة، ومثبتات البرامج، وصور الأقراص، وأرشيفات الألعاب إلى أدلة محلية. يتضمن المشروع محلل تنسيق ملفات يحدد أنواع الملفات عن طريق مسح المحتويات الداخلية والرؤوس دون الحاجة إلى استخراج كامل. كما يتميز بفك تشفير كلمة مرور الأرشيف الذي يحاول استعادة الوصول إلى الأرشيفات المحمية باستخدام قائمة محددة مسبقاً من كلمات المرور الشائعة. تدعم الأداة سير عمل فك الضغط المجمع من خلال معالجة طابور الدفعات واستخراج الدليل العودي للتعامل مع هياكل الأرشيف المتداخلة. كما توفر القدرة على تحليل المحتويات الداخلية لمثبتات البرامج وصور الأقراص دون تنفيذ عملية التثبيت.
Uses a task queue to manage the sequential decompression of multiple archive files.
Abogen is a text-to-speech audiobook generator that transforms digital documents and subtitle files into audiobooks. It utilizes language models to perform text normalization, rewriting contractions and punctuation to produce more natural speech synthesis. The system features a voice profile mixer that blends multiple voice models using adjustable weight ratios to create personalized synthetic voices. It also includes an automated export system that sends completed audio files and metadata to a remote Audiobookshelf server via a web API. The project manages the end-to-end audiobook productio
Manages a queue of document processing tasks with support for per-file configuration overrides.
Ripme is a batch media downloader and web media scraper designed for extracting images and videos from image-hosting platforms and social media sites. It functions as an image gallery downloader and a network client capable of retrieving full albums and paginated content. The project includes a custom media ripper framework that allows for the definition of new extraction rules to support websites lacking native support. It features a proxy-enabled network layer for routing requests through HTTP or SOCKS servers and supports session-based content retrieval using authentication cookies and cus
Scans web pages for linked galleries and adds them to a download queue for batch processing.
ncmdump is a proprietary audio cache converter and binary stream decryption utility. It decrypts raw audio data from specialized music cache files and transforms them into standard audio formats. The project functions both as a standalone tool and a cross-language conversion library. It exposes its internal decryption and conversion logic as a dynamic library, allowing the capabilities to be embedded into external applications written in different programming languages. The utility provides batch media processing through recursive directory traversal, enabling the identification and conversi
Manages a queue of target files for sequential processing to optimize resource usage.
TubeSync is a containerized media management tool and self-hosted archiver designed to automate the downloading and organization of video content from online sources, such as YouTube channels and playlists, into a local library for offline access. It functions as a download manager and metadata generator, utilizing a web interface to manage video subscriptions and synchronization settings. The system features a rule-based content filter that evaluates video metadata against user-defined conditions to determine which items enter the download queue. To handle restricted or age-gated content, it
Manages download tasks in a database, processed sequentially by a worker process with real-time updates.