Awesome GitHub RepositoriesVideo Frame Processing

Qwen2.5-VL एक ऑटोरग्रेसिव मल्टीमॉडल ट्रांसफार्मर है जिसे टेक्स्ट और विजुअल टोकन के इंटरलीव्ड अनुक्रमों को संसाधित करने के लिए डिज़ाइन किया गया है। यह क्रॉस-मॉडल तर्क करने और सुसंगत प्रतिक्रियाएं या संरचित लेआउट कोड उत्पन्न करने के लिए विजुअल फीचर एम्बेडिंग को एक साझा भाषा मॉडल स्थान में एकीकृत करता है। प्रोजेक्ट विजन-लैंग्वेज-एक्शन मैपिंग के माध्यम से खुद को अलग करता है, जिससे यह विजुअल इंटरफेस को समझने और उस धारणा को डिजिटल स्क्रीन और रोबोटिक हार्डवेयर को संचालित करने के लिए कार्रवाई योग्य कमांड में अनुवाद करने की अनुमति देता है। यह विविध छवि आकारों और लंबी अवधि के वीडियो अनुक्रमों को संभालने के लिए डायनामिक-रिज़ॉल्यूशन छवि एन्कोडिंग और टेम्पोरल-फ्रेम वीडियो इंडेक्सिंग का उपयोग करता है। मॉडल दस्तावेज़ डिजिटलीकरण के लिए बहुभाषी ऑप्टिकल कैरेक्टर रिकग्निशन, बाउंडिंग बॉक्स के माध्यम से वस्तुओं का पता लगाने के लिए स्थानिक ग्राउंडिंग, और लंबे वीडियो सामग्री के विश्लेषण सहित क्षमताओं के एक व्यापक क्षेत्र को कवर करता है। यह चार्ट और आरेखों का उपयोग करके समस्याओं को हल करने के लिए मल्टीमॉडल गणितीय तर्क का भी समर्थन करता है, और अपनी समझ को दस लाख टोकन की कॉन्टेक्स्ट लंबाई तक विस्तारित करता है।

This project is a collection of pre-trained machine learning models and conversion pipelines designed for running inference directly in the browser using TensorFlow.js. It provides a library of ready-to-use models for computer vision, audio classification, and natural language processing tasks. The suite includes specialized tools for transforming Python-based Keras models into JSON formats compatible with web environments. It enables the deployment of these models by fetching architectures and weight shards via HTTP for client-side execution. The project covers a broad range of capabilities

AnimateAnyone is an appearance-preserving video synthesizer designed for character animation from a single static image. It functions as a diffusion image-to-video generator that transforms a source image into a high-fidelity video sequence while maintaining consistent character identity, clothing, and visual details across all frames. The system enables video-driven character reenactment by transferring motions, facial expressions, and body movements from a reference video onto a static character. It employs pose-guided video generation to control movement via skeleton keypoints and pose sig

YOLOv10 is a PyTorch computer vision library and real-time vision framework designed for locating and identifying multiple objects in images and video streams. It functions as an end-to-end object detector that optimizes for high-speed deployment and detection precision. The project is distinguished by an NMS-free detection architecture that predicts a single bounding box per object, eliminating the need for non-maximum suppression post-processing to reduce inference latency. It further optimizes for edge hardware through scalable weights and a quantization-friendly structure that facilitates

SmarterStreaming is a cross-platform SDK for hardware-accelerated audio and video capture, encoding, and streaming. It provides a complete media pipeline for low-latency RTMP and RTSP streaming, and includes an embedded lightweight RTSP server that can serve live feeds directly from the source device to local network clients without requiring a separate server. The SDK also integrates with GB28181 surveillance platforms, enabling compliant device registration and streaming for standardized video monitoring systems. The project distinguishes itself through a set of integrated capabilities for

Faceai is a computer vision toolkit designed for facial analysis, identity recognition, and image processing. It provides integrated engines for detecting human faces in static images and live video streams, matching facial encodings against identity databases, and mapping facial landmarks to understand geometric structure and alignment. The project enables real-time augmented reality applications, such as applying virtual makeup and digital accessories by scaling assets to detected facial coordinates. It also includes a suite for digital image restoration capable of removing noise, erasing w

This project is a C-based media engineering guide and multimedia processing framework designed for managing codecs, frames, and packets within the FFmpeg and Libav ecosystem. It provides technical documentation and implementation patterns for transcoding, remuxing, and resizing video and audio data. The project includes a containerized development environment that encapsulates required media libraries and toolchains within a virtual image to ensure consistent build environments. The framework covers a range of multimedia engineering workflows, including adaptive bitrate streaming, media cont

This project is a cross-platform mobile camera framework and real-time computer vision library. It provides a high-performance interface for mobile applications to handle hardware control, media capture, and live camera frame processing. The framework includes a dedicated system for running AI models and custom analysis on live camera streams using high-performance worklets. It also functions as a real-time detection and decoding system for QR codes and barcodes. Broad capabilities cover the capture of high-resolution photos and videos with controls for zoom, HDR, and frame rates. The projec

FreeMoCap is an open-source markerless motion capture system that reconstructs 3D human pose from video. It uses a multi-camera setup with ChArUco board calibration to accurately triangulate body landmarks, and it also supports single-camera recording for simpler captures. The system outputs skeleton joint data and generates interactive Jupyter notebooks for each recording, enabling users to explore and analyse motion data directly. Built around hardware-synchronised video capture and MediaPipe-based 2D pose detection, FreeMoCap supports both calibrated multi-camera recording and real-time 2D

jetson-inference is a set of libraries and tools for executing optimized deep learning models on embedded GPU hardware. Its primary purpose is to enable real-time computer vision and AI inference at the edge with low latency and high throughput. The project distinguishes itself through high-performance streaming analytics and the ability to execute concurrent AI pipelines on auto-grade silicon. It provides specialized support for multi-sensor stream processing, utilizing zero-copy data transport to load camera frames directly into GPU memory. The codebase covers a broad surface of capabiliti

Gifski is a native macOS desktop application and video processing utility designed to convert video files into high-quality animated GIFs. It functions as a video-to-GIF converter that allows for the adjustment of frame rates and dimensions to balance visual fidelity with file size. The tool features capabilities for animation loop generation, including the creation of bounce effects by reversing video playback. It employs palette-based color quantization and error-diffusion dithering to maintain color accuracy and reduce banding in the final output. The application integrates with macOS sys

BasicSR is a PyTorch-based image restoration toolbox and framework designed for training and deploying deep learning models to upscale, denoise, and deblur images and videos. It serves as a comprehensive system for image super-resolution and video quality restoration, providing the necessary infrastructure to recover fine visual details and increase pixel density. The project distinguishes itself through specialized toolkits for facial image enhancement and high-fidelity face synthesis, as well as a dedicated video quality restoration suite that utilizes deformable convolutions and generative

ASCII-generator is a tool for converting images and videos into text-based ASCII art. It functions as an image-to-ASCII converter and a video-to-ASCII processor that maps pixel intensity and color to specific alphanumeric characters. The system generates stylized visual representations by transforming visual files into grayscale or colored ASCII art text files. It can render static images into text art or process video files into a sequence of ASCII art frames for animation. The rendering process involves translating image pixels into text grids and mapping brightness values to characters ba

Depth-Anything is a monocular depth estimation foundation model that produces dense per-pixel depth maps from a single RGB image. It is built on a DINOv2 Vision Transformer encoder backbone and trained on 62 million unlabeled images using a teacher-student pseudo-labeling framework, enabling robust generalization across diverse scenes without task-specific training. The model outputs both relative depth maps, which capture the ordering of scene points, and metric depth maps with real-world units after fine-tuning on datasets like NYUv2 or KITTI. The project distinguishes itself through its ab

Backgroundremover is an AI-powered tool that removes backgrounds from both images and videos, accessible through a command-line interface and a Python API. At its core, it uses a pre-trained deep learning model to classify each pixel as foreground or background, producing a binary mask for removal. The tool distinguishes itself through multiple integration methods and output capabilities. It can process images and videos via Unix pipeline data streams, operate as an HTTP API server, or be called programmatically within Python scripts. Users can choose among different AI models to balance proc

tensorrtx is a computer vision inference engine and model implementation library designed for graphics processor acceleration. It provides a framework for optimizing deep learning models through a GPU inference optimizer, a deep learning model converter for transforming weights from frameworks like TensorFlow and PyTorch, and a custom plugin library to implement operations not natively supported by the TensorRT API. The project distinguishes itself through a comprehensive collection of pre-defined network implementations, ranging from various YOLO versions and DETR transformers for object det

Video-retalking is an AI lip synchronization framework and talking head video editor designed to match the mouth movements of a subject in a video to a target audio track. It utilizes a deep learning pipeline to synchronize speech with video recordings. The system employs a two-stage generation process that separates coarse lip movement from high-resolution detail refinement. It incorporates identity-aware face refinement and expression template alignment to maintain photorealistic skin textures and ensure visual consistency across video frames. The toolset covers facial expression modificat

SAMURAI is a zero-shot visual tracking model that adapts the Segment Anything architecture for video object segmentation. It uses a first-frame prompt, such as a bounding box or mask, to initialize tracking, then employs a motion-aware memory mechanism that stores and updates temporal motion features across frames to guide mask refinement. An online memory update strategy continuously refreshes this memory with new frame predictions, while temporal motion encoding computes optical flow between consecutive frames to inform object boundary and occlusion handling. The system is designed for real

I've made everythign less reliant on torch.compile for VRAM efficiency, so things should work better even without it. Also figured workaround for some issues when using compile that made first run use drastically more VRAM, issue I battled with myself a lot.

ByteTrack is a multi-object tracking framework that implements the ByteTrack algorithm, an ECCV 2022 method designed to recover occluded objects and reduce trajectory fragmentation. The core innovation of the project is its association algorithm, which processes every detection box—including low-confidence ones—by using separate high and low score thresholds, Kalman filter motion prediction, and Hungarian algorithm matching to produce consistent object identities across video frames. The project distinguishes itself by its comprehensive approach to handling occlusions and fragmented trajector