Awesome GitHub RepositoriesBatch Processing Engines

Faceswap is a comprehensive framework for automated media manipulation and neural face synthesis. It provides a modular pipeline that manages the entire lifecycle of facial feature extraction, deep learning model training, and image conversion. By coordinating complex computer vision workflows, the system enables users to map facial identities between source and destination datasets while maintaining structural alignment and lighting consistency across video frames. The project distinguishes itself through a highly extensible plugin-based architecture that handles hardware-accelerated process

Airflow is a platform for programmatically authoring, scheduling, and monitoring complex data pipelines. It functions as a workflow automation engine that manages the lifecycle of recurring business processes by executing code-defined task dependencies. By representing workflows as directed acyclic graphs, the system ensures that task execution order and data flow are explicitly defined and reliably maintained across distributed computing environments. The platform distinguishes itself through a highly modular, provider-based architecture that decouples core orchestration logic from external

Feast is an open-source feature store for machine learning that provides a central platform for defining, storing, and serving features across both training and inference workflows. It operates as a declarative system where feature definitions are written as code in Python files, synchronized to a central registry, and made available for low-latency online retrieval or point-in-time correct historical joins for training datasets. The project abstracts storage behind a pluggable architecture, allowing offline and online backends to be swapped without changing retrieval logic, and coordinates ma

ZenML is an extensible machine learning orchestration framework designed to manage the end-to-end lifecycle of data pipelines and AI agent workflows. It functions as a durable orchestrator that executes machine learning tasks as directed acyclic graphs, ensuring that every step is containerized for consistent performance across local, cloud, and hybrid infrastructure. By decoupling pipeline code from underlying compute and storage backends, the platform allows developers to define infrastructure-agnostic stacks that remain portable across diverse environments. The project distinguishes itself

AiNiee is an LLM-based localization tool that automates the translation of games, books, subtitles, and documents across multiple languages. It operates as a batch processing engine, translating entire folders of files in parallel while preserving directory structure, and includes a glossary management system that enforces terminology consistency using AI-powered glossaries, forbidden terms, and user-defined text substitution rules. The tool differentiates itself through key architectural decisions: it distributes translation requests across multiple API keys to bypass rate limits and acceler

YT-Spammer-Purge هو أداة لإدارة محتوى YouTube عبر API وأداة للإشراف، مصممة لاكتشاف وحذف التعليقات المزعجة (spam) والاحتيالية عبر الفيديوهات والقنوات. يعمل كأداة متخصصة للتصفية والإدارة لتحديد المحتوى الإشكالي وحظر المستخدمين المسيئين. تستخدم الأداة تصفية المحتوى المؤتمتة واكتشاف النصوص القائم على الأنماط للإبلاغ عن التعليقات غير المرغوب فيها باستخدام التعبيرات النمطية (regex) وتحليل مستوى الأحرف. ويدعم عملية مراجعة من مرحلتين، حيث يتم تخزين التعليقات المشبوهة في قائمة انتظار للمراجعة اليدوية قبل الحذف النهائي. يتضمن النظام قدرات إدارية لمعالجة التعليقات بشكل جماعي عبر الحسابات بالكامل، وتدقيق إجراءات الإشراف من خلال سجلات مفصلة، وتقييد صلاحيات المشرفين لمنع سوء الاستخدام الإداري.