9 repository-uri
Pipelines that intercept and modify file attributes or metadata during data transfer processes.
Explore 9 awesome GitHub repositories matching data & databases · Metadata Transformation Pipelines. Refine with filters or upvote what's useful.
This project is a command-line storage manager that provides a unified interface for performing file operations across local filesystems and diverse cloud storage providers. It functions as a cross-platform storage abstraction, utilizing a modular backend architecture to map heterogeneous cloud storage APIs into a standard set of file system operations. This allows for consistent data management and movement regardless of the underlying storage service. The tool serves as a network data transfer engine designed for automated data migration and cloud storage synchronization. It distinguishes i
Intercepts and modifies file attributes during transfer to match the requirements of the destination storage backend.
FastMCP is a Python framework designed for building servers that expose functions, resources, and prompts to AI models using the Model Context Protocol. It simplifies the development process by automatically deriving tool metadata, input schemas, and documentation directly from Python function signatures and type hints. The framework provides a unified container for managing these components, allowing developers to build modular applications that integrate seamlessly with AI assistants. The project distinguishes itself through its support for interactive, server-defined user interface compone
Intercepts and modifies tool schemas or namespaces during transmission to resolve naming conflicts and refine how tools are presented to clients.
Jackett is a self-hosted background service that functions as a BitTorrent tracker aggregator and proxy. It enables automated media management applications to query multiple torrent indexers simultaneously by translating standardized search requests into site-specific formats and consolidating the resulting data into a single, unified feed. The service distinguishes itself through an adapter-based architecture that handles the complexities of disparate tracker interfaces and security protocols. It integrates with external proxy services to bypass anti-bot challenges and maintain persistent ac
Intercepts and modifies metadata during data transfer processes to ensure accurate content matching.
YoutubeDownloader is a desktop application designed to retrieve and archive video and audio content from online platforms. It enables users to download media files directly to local storage, providing options to select specific quality levels and file formats to suit local playback requirements. The application distinguishes itself through its ability to access restricted or private content by utilizing personal account credentials. By managing session authentication, it allows for the retrieval of media that is not accessible to the general public. Furthermore, it incorporates automated work
Injects descriptive metadata and stream information into media containers after the download completes.
Nightingale is a Prometheus-compatible monitoring and alerting platform designed to centralize telemetry management across multiple time-series databases. It functions as a multi-source alerting engine and metric data pipeline that ingests telemetry via remote write protocols and triggers alarms based on data from sources such as Prometheus, Elasticsearch, Loki, and ClickHouse. The system is distinguished by its automated alert healing system, which executes predefined scripts and RPC-based corrective actions when monitoring thresholds are breached. It supports distributed alert processing, a
Transforms alert data through a sequence of relabeling, filtering, and metadata enrichment steps before notification.
Sub-Store is a proxy subscription management server that aggregates multiple subscription links into a single unified stream for distribution to various clients. It functions as a transformation pipeline that filters, modifies, and reformats proxy node metadata. The system acts as a cross-platform format converter to ensure compatibility across diverse client applications. It includes an encryption decryption gateway that uses private keys to handle age-standard encrypted subscription content and a cache-layered aggregator to reduce external requests. The server provides capabilities for dyn
Provides a pipeline to intercept and modify proxy node metadata during the data transfer process.
VidBee is a self-hosted media download manager that wraps the yt-dlp engine to download videos and audio from over 1000 websites. It functions as both a desktop client and a Fastify-based web service, managing downloads through a persistent queue with pause, resume, retry, and real-time progress tracking. The application uses cookie-based authentication to access login-gated, age-restricted, or subscriber-only content by importing browser cookies or Netscape-format cookie files. The application distinguishes itself through automated download workflows, including RSS and Atom feed monitoring t
Allows users to select output container formats like MP4, MKV, or WebM for downloads.
node-ytdl-core este o bibliotecă JavaScript pentru Node.js concepută pentru a extrage metadate și a face streaming de conținut video și audio de pe YouTube. Servește ca un downloader media și fetcher de stream-uri, permițând utilizatorilor să preia detalii video și date media din surse remote. Biblioteca oferă capabilități specializate pentru extracția video, inclusiv capacitatea de a parsa URL-urile media pentru identificatori unici și de a analiza formatele disponibile. Permite selectarea și filtrarea fluxurilor video și audio specifice pe baza criteriilor de calitate și rezoluție. Proiectul gestionează traficul de rețea prin evitarea limitelor de rată și gestionarea cookie-urilor de autentificare. Utilizează fluxuri lizibile (readable streams) pentru piping-ul datelor și suportă cereri de tip byte-range pentru a prelua segmente specifice ale fișierelor media.
Allows users to filter and select specific video or audio streams based on quality, container, and resolution criteria.
Nali este o suită de instrumente de linie de comandă pentru rezolvarea adreselor IP la locații geografice și identificarea furnizorilor de rețele de livrare a conținutului folosind baze de date offline. Acesta funcționează ca un instrument de geolocalizare IP offline și un resolver de baze de date care mapează adresele la locații fizice și proprietari de rețea fără a necesita o conexiune la internet activă. Proiectul se distinge printr-o abordare offline-first a analizei rețelei, utilizând furnizori de baze de date pluggable și caching-ul metadatelor fișierelor locale pentru a asigura confidențialitatea datelor și independența față de API-urile externe. Include un utilitar dedicat pentru identificarea furnizorilor de rețele de livrare a conținutului și un sistem pentru gestionarea și actualizarea fișierelor de date geografice locale. Setul de instrumente suportă atât fluxuri de lucru interactive, cât și automatizate, prezentând o buclă read-eval-print pentru căutări manuale secvențiale și un procesor de metadate care citește fluxuri de adrese IP din input-ul standard. Acest lucru permite integrarea metadatelor geografice și ale furnizorului în pipeline-urile shell. Configurarea pentru depozitele de date și fișierele de configurare este gestionată prin variabile de mediu ale sistemului.
Appends geographic or provider metadata to IP addresses and domains received via standard input.