awesome-repositories.com
Blog
awesome-repositories.com

Entdecke die besten Open-Source-Repositories mit KI-gestützter Suche.

EntdeckenKuratierte SuchenOpen-Source-AlternativenSelf-hosted SoftwareBlogSitemap
ProjektÜber unsRanking-MethodikPresseMCP-Server
RechtlichesDatenschutzAGB
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
·

14 Repos

Awesome GitHub RepositoriesTask Queues

Sequential processing of tasks using a queue to manage system resource consumption.

Distinct from Batch Processing: Focuses on the queueing mechanism for file processing rather than bulk data operations

Explore 14 awesome GitHub repositories matching data & databases · Task Queues. Refine with filters or upvote what's useful.

Awesome Task Queues GitHub Repositories

Finde die besten Repos mit KI.Wir suchen mit KI nach den am besten passenden Repositories.
  • imageoptim/imageoptimAvatar von ImageOptim

    ImageOptim/ImageOptim

    9,873Auf GitHub ansehen↗

    ImageOptim is a macOS image optimizer and GUI image compressor designed to reduce image file sizes for web and disk storage. It functions as a lossless image optimizer that removes unnecessary metadata and optimizes encoding to reduce file sizes without losing pixel data. The application handles macOS media asset management and web image preparation by shrinking files to improve website loading speeds and reduce storage usage. It provides both lossless and lossy compression options to maintain visual quality while reducing the disk space used by images.

    Manages a queue of images to process them sequentially and prevent system resource exhaustion.

    HTML
    Auf GitHub ansehen↗9,873
  • yaobiao131/downkyicoreAvatar von yaobiao131

    yaobiao131/downkyicore

    7,450Auf GitHub ansehen↗

    downkyicore is a cross-platform desktop application designed for the automated batch downloading of Bilibili videos, including support for high-resolution 8K and HDR content. It functions as a batch media downloader that manages multiple files and collections simultaneously through an automated queue system. The project integrates FFmpeg for media processing to separate audio and video streams and remove watermarks. It uses specialized mechanisms to bypass access restrictions and rate limits by employing random identifiers and mobile referrers during the parsing process. The system includes

    Manages a sequential queue of download tasks using a local database to handle batch processing and retries.

    C#avaloniabilibili
    Auf GitHub ansehen↗7,450
  • tubearchivist/tubearchivistAvatar von tubearchivist

    tubearchivist/tubearchivist

    7,561Auf GitHub ansehen↗

    TubeArchivist is a self-hosted YouTube video archiving system and metadata indexer. It functions as a personal media library and download manager that allows users to create a searchable offline collection of videos, channels, and playlists. The system distinguishes itself by indexing subtitles, comments, and channel information for full-text search and retrieval. It features automated media synchronization to track subscriptions and playlists, ensuring new content is automatically queued and downloaded as it is published. The project provides a broad set of capabilities for digital asset ma

    Processes pending items in a sequential download queue to retrieve files for offline storage.

    Pythonarchivedownload-managerdownload-videos
    Auf GitHub ansehen↗7,561
  • j3ssie/osmedeusAvatar von j3ssie

    j3ssie/Osmedeus

    6,425Auf GitHub ansehen↗

    Osmedeus is a security workflow orchestration engine that coordinates AI agents, shell commands, and scanning tools through declarative YAML pipelines. It functions as a distributed security scanner, a declarative workflow automator, and an AI agent framework for security, enabling automated multi-step security analysis with conditional branching, parallel execution, and distributed workers. The engine distinguishes itself through a hybrid runner model that executes workflow steps on the local host, inside Docker containers, or over SSH to remote machines, selected per step or module. It supp

    Enqueues scan tasks for later execution with configurable concurrency limits.

    Go
    Auf GitHub ansehen↗6,425
  • ccostan/home-assistantconfigAvatar von CCOSTAN

    CCOSTAN/Home-AssistantConfig

    5,176Auf GitHub ansehen↗

    This repository is a collection of YAML automations and scripts for Home Assistant that coordinate devices and services into context‑aware home routines. Its design centers on combining multiple sensor inputs—motion, bed presence, garage door state—into a single occupancy signal that gates automations, reducing false triggers. The collection delivers actionable mobile notifications with script‑bound buttons for snoozing or escalating alerts, and integrates an external vision service to classify camera snapshots so package deliveries are detected and announced. What distinguishes this configur

    Processes tasks one at a time from a persistent queue with per-task time thresholds and delay guards.

    Pythonaialexaautomation
    Auf GitHub ansehen↗5,176
  • nickvisionapps/parabolicAvatar von NickvisionApps

    NickvisionApps/Parabolic

    5,106Auf GitHub ansehen↗

    Parabolic is a graphical frontend for the yt-dlp engine, serving as a web media downloader and extractor. It provides a visual interface for saving high-quality video and audio content from various web platforms into local files. The application functions as a multi-format media exporter, allowing content to be saved into diverse audio and video containers. It includes a media metadata manager to capture and store associated information, such as subtitles and technical metadata, alongside the downloaded files. The system supports batch content acquisition through concurrent download manageme

    Coordinates multiple simultaneous download tasks using a worker queue to optimize system resource consumption.

    C#csharpdownloaderflathub
    Auf GitHub ansehen↗5,106
  • wxbool/video-srt-windowsAvatar von wxbool

    wxbool/video-srt-windows

    5,037Auf GitHub ansehen↗

    Dies ist eine Windows-Anwendung für automatische Spracherkennung, die gesprochenes Audio aus Videodateien in zeitgestempelte SRT-Untertiteldateien transkribiert. Sie dient als Untertitelgenerator und Übersetzungstool, das Medien-Sprache in synchronisierten Text umwandelt. Die Software fungiert als Batch-Medien-Transkribierer, der die gleichzeitige Verarbeitung mehrerer Audio- und Videodateien ermöglicht, um Untertitel in großen Mengen zu generieren. Sie enthält einen Übersetzungsworkflow zur Konvertierung von Transkriptionen zwischen verschiedenen Sprachen für die Erstellung zweisprachiger oder lokalisierter Dateien. Das System bietet zudem Textverfeinerungsfunktionen unter Verwendung regulärer Ausdrücke und benutzerdefinierter Filter, um Transkripte durch das Entfernen von Füllwörtern und unerwünschten Mustern zu bereinigen. Dies wird durch eine native grafische Windows-Benutzeroberfläche unterstützt.

    Uses a task queue to process multiple media files sequentially and manage system resources.

    Goffmpeggogolang
    Auf GitHub ansehen↗5,037
  • rudderlabs/rudder-serverAvatar von rudderlabs

    rudderlabs/rudder-server

    4,437Auf GitHub ansehen↗

    Rudder Server ist eine Customer Data Platform (CDP) und Event-Routing-Pipeline, die darauf ausgelegt ist, Kundendaten zu sammeln, zu transformieren und von verschiedenen Quellen an Data Warehouses und Business-Tools weiterzuleiten. Es fungiert als Customer-Identity-Resolver, der Identifikatoren aus mehreren Quellen verknüpft, um einen einheitlichen Identitätsgraphen und umfassende verhaltensbasierte Kundenprofile zu erstellen. Das System zeichnet sich durch Reverse-ETL-Funktionen aus, die verarbeitete Kundensegmente und Zielgruppen aus Data Warehouses zurück in operative Drittanbieteranwendungen pushen. Es bietet zudem eine containerisierte Datenebene für Kubernetes-Deployments, was die Verwaltung der Dateninfrastruktur als Code ermöglicht. Die Plattform deckt eine breite Palette von Datenmanagement-Funktionen ab, einschließlich Echtzeit-Event-Transformation, Schema-Validierung via Datenkatalogen und Privacy-Governance. Dazu gehören Tools zur Verwaltung der Benutzereinwilligung, zur Durchsetzung der Datenresidenz innerhalb spezifischer geografischer Regionen und zur Maskierung personenbezogener Daten während der Übertragung. Installation und Deployment der Datenebenen-Komponenten werden mittels Helm-Charts verwaltet.

    Schedules bulk data uploads and synchronization tasks using a priority queue to manage system load.

    Gobigquerycdpcustomer-data
    Auf GitHub ansehen↗4,437
  • bioruebe/uniextract2Avatar von Bioruebe

    Bioruebe/UniExtract2

    4,344Auf GitHub ansehen↗

    UniExtract2 ist eine Suite von Tools, die für universelle Archiv-Extraktion, Batch-Dekomprimierung und Dateiformatanalyse entwickelt wurde. Sie ruft Dateien aus verschiedenen komprimierten Formaten, Software-Installern, Disk-Images und Spielarchiven in lokale Verzeichnisse ab. Das Projekt enthält einen Dateiformatanalysator, der Dateitypen durch Scannen interner Inhalte und Header identifiziert, ohne eine vollständige Extraktion zu erfordern. Es bietet zudem einen Archiv-Passwort-Entschlüsseler, der versucht, den Zugriff auf geschützte Archive mithilfe einer vordefinierten Liste gängiger Passwörter wiederherzustellen. Das Tool unterstützt Bulk-Dekomprimierungs-Workflows durch Batch-Queue-Verarbeitung und rekursive Verzeichnisextraktion, um verschachtelte Archivstrukturen zu handhaben. Es bietet zudem die Möglichkeit, die internen Inhalte von Software-Installern und Disk-Images zu analysieren, ohne den Installationsprozess auszuführen.

    Uses a task queue to manage the sequential decompression of multiple archive files.

    AutoItdecompressionextractorfile-analysis
    Auf GitHub ansehen↗4,344
  • denizsafak/abogenAvatar von denizsafak

    denizsafak/abogen

    4,135Auf GitHub ansehen↗

    Abogen is a text-to-speech audiobook generator that transforms digital documents and subtitle files into audiobooks. It utilizes language models to perform text normalization, rewriting contractions and punctuation to produce more natural speech synthesis. The system features a voice profile mixer that blends multiple voice models using adjustable weight ratios to create personalized synthetic voices. It also includes an automated export system that sends completed audio files and metadata to a remote Audiobookshelf server via a web API. The project manages the end-to-end audiobook productio

    Manages a queue of document processing tasks with support for per-file configuration overrides.

    Pythonaudiobookaudiobookscontent-creation
    Auf GitHub ansehen↗4,135
  • ripmeapp/ripmeAvatar von RipMeApp

    RipMeApp/ripme

    4,041Auf GitHub ansehen↗

    Ripme is a batch media downloader and web media scraper designed for extracting images and videos from image-hosting platforms and social media sites. It functions as an image gallery downloader and a network client capable of retrieving full albums and paginated content. The project includes a custom media ripper framework that allows for the definition of new extraction rules to support websites lacking native support. It features a proxy-enabled network layer for routing requests through HTTP or SOCKS servers and supports session-based content retrieval using authentication cookies and cus

    Scans web pages for linked galleries and adds them to a download queue for batch processing.

    Javaalbumarchivalarchive
    Auf GitHub ansehen↗4,041
  • neoapplications/neo-backupAvatar von NeoApplications

    NeoApplications/Neo-Backup

    3,691Auf GitHub ansehen↗

    Neo-Backup is an Android application backup tool and rooted device data manager. It creates encrypted copies of application data and system files on rooted devices to preserve application state and settings. The project functions as a backup scheduler, allowing for the automation of periodic data preservation tasks based on custom application lists and fixed timetables. To protect sensitive information, it utilizes password-based encryption to secure backup archives. The system provides capabilities for device-wide data backup and recovery, including the ability to restore applications and t

    Implements a sequential task queue to manage multiple data extraction jobs without overloading device resources.

    Kotlin
    Auf GitHub ansehen↗3,691
  • taurusxin/ncmdumpAvatar von taurusxin

    taurusxin/ncmdump

    3,385Auf GitHub ansehen↗

    ncmdump is a proprietary audio cache converter and binary stream decryption utility. It decrypts raw audio data from specialized music cache files and transforms them into standard audio formats. The project functions both as a standalone tool and a cross-language conversion library. It exposes its internal decryption and conversion logic as a dynamic library, allowing the capabilities to be embedded into external applications written in different programming languages. The utility provides batch media processing through recursive directory traversal, enabling the identification and conversi

    Manages a queue of target files for sequential processing to optimize resource usage.

    C++
    Auf GitHub ansehen↗3,385
  • meeb/tubesyncAvatar von meeb

    meeb/tubesync

    2,625Auf GitHub ansehen↗

    TubeSync is a containerized media management tool and self-hosted archiver designed to automate the downloading and organization of video content from online sources, such as YouTube channels and playlists, into a local library for offline access. It functions as a download manager and metadata generator, utilizing a web interface to manage video subscriptions and synchronization settings. The system features a rule-based content filter that evaluates video metadata against user-defined conditions to determine which items enter the download queue. To handle restricted or age-gated content, it

    Manages download tasks in a database, processed sequentially by a worker process with real-time updates.

    Pythonmedia-serverplexpvr
    Auf GitHub ansehen↗2,625
  1. Home
  2. Data & Databases
  3. Batch Processing
  4. Task Queues

Unter-Tags erkunden

  • Gallery Discovery QueuesQueues that automatically discover and add related web galleries for batch processing. **Distinct from Task Queues:** Distinct from general task queues by focusing on the discovery of linked galleries as the queue source.