Awesome GitHub RepositoriesVideo Frame Processing

Qwen2.5-VL es un transformador multimodal autorregresivo diseñado para procesar secuencias intercaladas de tokens de texto y visuales. Integra incrustaciones de características visuales en un espacio de modelo de lenguaje compartido para realizar razonamiento intermodal y generar respuestas coherentes o código de diseño estructurado. El proyecto se distingue por el mapeo de visión-lenguaje-acción, lo que le permite percibir interfaces visuales y traducir esa percepción en comandos accionables para operar pantallas digitales y hardware robótico. Emplea codificación de imagen de resolución dinámica y indexación de video de fotogramas temporales para manejar diversos tamaños de imagen y secuencias visuales de larga duración. El modelo cubre una amplia superficie de capacidades, incluyendo reconocimiento óptico de caracteres multilingüe para la digitalización de documentos, conexión espacial para localizar objetos a través de cuadros delimitadores y el análisis de contenido de video de larga duración. También admite razonamiento matemático multimodal para resolver problemas utilizando gráficos y diagramas, y extiende su comprensión a una longitud de contexto de un millón de tokens.

This project is a collection of pre-trained machine learning models and conversion pipelines designed for running inference directly in the browser using TensorFlow.js. It provides a library of ready-to-use models for computer vision, audio classification, and natural language processing tasks. The suite includes specialized tools for transforming Python-based Keras models into JSON formats compatible with web environments. It enables the deployment of these models by fetching architectures and weight shards via HTTP for client-side execution. The project covers a broad range of capabilities

AnimateAnyone is an appearance-preserving video synthesizer designed for character animation from a single static image. It functions as a diffusion image-to-video generator that transforms a source image into a high-fidelity video sequence while maintaining consistent character identity, clothing, and visual details across all frames. The system enables video-driven character reenactment by transferring motions, facial expressions, and body movements from a reference video onto a static character. It employs pose-guided video generation to control movement via skeleton keypoints and pose sig

YOLOv10 is a PyTorch computer vision library and real-time vision framework designed for locating and identifying multiple objects in images and video streams. It functions as an end-to-end object detector that optimizes for high-speed deployment and detection precision. The project is distinguished by an NMS-free detection architecture that predicts a single bounding box per object, eliminating the need for non-maximum suppression post-processing to reduce inference latency. It further optimizes for edge hardware through scalable weights and a quantization-friendly structure that facilitates

SmarterStreaming is a cross-platform SDK for hardware-accelerated audio and video capture, encoding, and streaming. It provides a complete media pipeline for low-latency RTMP and RTSP streaming, and includes an embedded lightweight RTSP server that can serve live feeds directly from the source device to local network clients without requiring a separate server. The SDK also integrates with GB28181 surveillance platforms, enabling compliant device registration and streaming for standardized video monitoring systems. The project distinguishes itself through a set of integrated capabilities for

Faceai is a computer vision toolkit designed for facial analysis, identity recognition, and image processing. It provides integrated engines for detecting human faces in static images and live video streams, matching facial encodings against identity databases, and mapping facial landmarks to understand geometric structure and alignment. The project enables real-time augmented reality applications, such as applying virtual makeup and digital accessories by scaling assets to detected facial coordinates. It also includes a suite for digital image restoration capable of removing noise, erasing w

Este proyecto es una guía de ingeniería multimedia basada en C y un framework de procesamiento multimedia diseñado para gestionar códecs, frames y paquetes dentro del ecosistema de FFmpeg y Libav. Proporciona documentación técnica y patrones de implementación para transcodificación, remuxing y redimensionamiento de datos de vídeo y audio. El proyecto incluye un entorno de desarrollo en contenedores que encapsula las bibliotecas multimedia y toolchains necesarias dentro de una imagen virtual para garantizar entornos de compilación consistentes. El framework cubre una gama de flujos de trabajo de ingeniería multimedia, incluyendo streaming de bitrate adaptativo, remuxing de contenedores multimedia y extracción de metadatos. También proporciona capacidades para el procesamiento de streams, como la sincronización de audio-vídeo y el redimensionamiento de resolución, junto con herramientas para registrar detalles de códecs y tiempos.

This project is a cross-platform mobile camera framework and real-time computer vision library. It provides a high-performance interface for mobile applications to handle hardware control, media capture, and live camera frame processing. The framework includes a dedicated system for running AI models and custom analysis on live camera streams using high-performance worklets. It also functions as a real-time detection and decoding system for QR codes and barcodes. Broad capabilities cover the capture of high-resolution photos and videos with controls for zoom, HDR, and frame rates. The projec

FreeMoCap is an open-source markerless motion capture system that reconstructs 3D human pose from video. It uses a multi-camera setup with ChArUco board calibration to accurately triangulate body landmarks, and it also supports single-camera recording for simpler captures. The system outputs skeleton joint data and generates interactive Jupyter notebooks for each recording, enabling users to explore and analyse motion data directly. Built around hardware-synchronised video capture and MediaPipe-based 2D pose detection, FreeMoCap supports both calibrated multi-camera recording and real-time 2D

jetson-inference is a set of libraries and tools for executing optimized deep learning models on embedded GPU hardware. Its primary purpose is to enable real-time computer vision and AI inference at the edge with low latency and high throughput. The project distinguishes itself through high-performance streaming analytics and the ability to execute concurrent AI pipelines on auto-grade silicon. It provides specialized support for multi-sensor stream processing, utilizing zero-copy data transport to load camera frames directly into GPU memory. The codebase covers a broad surface of capabiliti

Gifski is a native macOS desktop application and video processing utility designed to convert video files into high-quality animated GIFs. It functions as a video-to-GIF converter that allows for the adjustment of frame rates and dimensions to balance visual fidelity with file size. The tool features capabilities for animation loop generation, including the creation of bounce effects by reversing video playback. It employs palette-based color quantization and error-diffusion dithering to maintain color accuracy and reduce banding in the final output. The application integrates with macOS sys

BasicSR is a PyTorch-based image restoration toolbox and framework designed for training and deploying deep learning models to upscale, denoise, and deblur images and videos. It serves as a comprehensive system for image super-resolution and video quality restoration, providing the necessary infrastructure to recover fine visual details and increase pixel density. The project distinguishes itself through specialized toolkits for facial image enhancement and high-fidelity face synthesis, as well as a dedicated video quality restoration suite that utilizes deformable convolutions and generative

ASCII-generator is a tool for converting images and videos into text-based ASCII art. It functions as an image-to-ASCII converter and a video-to-ASCII processor that maps pixel intensity and color to specific alphanumeric characters. The system generates stylized visual representations by transforming visual files into grayscale or colored ASCII art text files. It can render static images into text art or process video files into a sequence of ASCII art frames for animation. The rendering process involves translating image pixels into text grids and mapping brightness values to characters ba

Depth-Anything is a monocular depth estimation foundation model that produces dense per-pixel depth maps from a single RGB image. It is built on a DINOv2 Vision Transformer encoder backbone and trained on 62 million unlabeled images using a teacher-student pseudo-labeling framework, enabling robust generalization across diverse scenes without task-specific training. The model outputs both relative depth maps, which capture the ordering of scene points, and metric depth maps with real-world units after fine-tuning on datasets like NYUv2 or KITTI. The project distinguishes itself through its ab

Backgroundremover is an AI-powered tool that removes backgrounds from both images and videos, accessible through a command-line interface and a Python API. At its core, it uses a pre-trained deep learning model to classify each pixel as foreground or background, producing a binary mask for removal. The tool distinguishes itself through multiple integration methods and output capabilities. It can process images and videos via Unix pipeline data streams, operate as an HTTP API server, or be called programmatically within Python scripts. Users can choose among different AI models to balance proc

tensorrtx is a computer vision inference engine and model implementation library designed for graphics processor acceleration. It provides a framework for optimizing deep learning models through a GPU inference optimizer, a deep learning model converter for transforming weights from frameworks like TensorFlow and PyTorch, and a custom plugin library to implement operations not natively supported by the TensorRT API. The project distinguishes itself through a comprehensive collection of pre-defined network implementations, ranging from various YOLO versions and DETR transformers for object det

Video-retalking is an AI lip synchronization framework and talking head video editor designed to match the mouth movements of a subject in a video to a target audio track. It utilizes a deep learning pipeline to synchronize speech with video recordings. The system employs a two-stage generation process that separates coarse lip movement from high-resolution detail refinement. It incorporates identity-aware face refinement and expression template alignment to maintain photorealistic skin textures and ensure visual consistency across video frames. The toolset covers facial expression modificat

SAMURAI is a zero-shot visual tracking model that adapts the Segment Anything architecture for video object segmentation. It uses a first-frame prompt, such as a bounding box or mask, to initialize tracking, then employs a motion-aware memory mechanism that stores and updates temporal motion features across frames to guide mask refinement. An online memory update strategy continuously refreshes this memory with new frame predictions, while temporal motion encoding computes optical flow between consecutive frames to inform object boundary and occlusion handling. The system is designed for real

I've made everythign less reliant on torch.compile for VRAM efficiency, so things should work better even without it. Also figured workaround for some issues when using compile that made first run use drastically more VRAM, issue I battled with myself a lot.

ByteTrack is a multi-object tracking framework that implements the ByteTrack algorithm, an ECCV 2022 method designed to recover occluded objects and reduce trajectory fragmentation. The core innovation of the project is its association algorithm, which processes every detection box—including low-confidence ones—by using separate high and low score thresholds, Kalman filter motion prediction, and Hungarian algorithm matching to produce consistent object identities across video frames. The project distinguishes itself by its comprehensive approach to handling occlusions and fragmented trajector