mrousavy/react-native-vision-camera
React Native Vision Camera
This project is a cross-platform mobile camera framework and real-time computer vision library. It provides a high-performance interface for mobile applications to handle hardware control, media capture, and live camera frame processing.
The framework includes a dedicated system for running AI models and custom analysis on live camera streams using high-performance worklets. It also functions as a real-time detection and decoding system for QR codes and barcodes.
Broad capabilities cover the capture of high-resolution photos and videos with controls for zoom, HDR, and frame rates. The project supports drawing visual overlays, such as shapes and shaders, over live feeds and includes utilities for managing camera hardware permissions and pixel format configurations.
Features
- Cross-Platform Camera Components - Acts as a high-performance cross-platform interface for integrating camera hardware and media capture into mobile apps.
- Camera Feed Decoders - Detects and decodes QR codes and barcodes in real-time directly from the live camera video stream.
- Computer Vision Libraries - Serves as a high-performance library for running AI models and real-time frame analysis on mobile camera streams.
- Real-Time Object Detection - Runs real-time object and face detection using AI models on live camera frames within React Native.
- Pixel-Buffer Memory Mapping - Maps raw camera pixel data into memory buffers accessible by both native code and JavaScript worklets for high-speed processing.
- Worklet Threading - Executes JavaScript functions on dedicated high-performance background threads to analyze camera frames without blocking the UI.
- Mobile Photo and Video Capture - Supports recording high-resolution images and video clips with customizable resolutions, aspect ratios, and frame rates.
- Real-Time Model Inference on Frames - Runs AI models and custom logic on individual camera frames for tasks like facial recognition and object detection.
- Device Hardware Integrations - Integrates native mobile application logic with camera hardware to optimize image quality and device selection.
- Camera Capture Controllers - Provides low-level controls for device selection, zoom, HDR, and night modes to optimize image capture.
- Mobile Media Capture - Enables recording high-resolution photos and videos with control over zoom, HDR, and frame rates.
- Native Bindings - Provides high-performance synchronous communication between JavaScript and native C++ camera modules using the JavaScript Interface.
- Scanning Workflows - Provides the operational workflow for capturing and processing barcode data from a live camera feed.
- Real-Time Visual Stream Processors - Streams camera frames to JavaScript worklet functions for high-performance analysis while the camera is active.
- Camera Frame Processors - Provides a system for manipulating pixel formats and applying visual shaders to live video streams.
- Frame Dropping Strategies - Manages high-frequency camera streams by discarding frames when the processing thread cannot keep up with hardware.
- Render-to-Texture Buffers - Binds camera output directly to GPU textures for low-latency previews and real-time shader overlays.
- Live Stream Overlays - Enables drawing shapes, text, and shaders directly onto the live camera stream for real-time visual feedback.
- QR Code Scanner Libraries - Provides a real-time detection and decoding system for QR codes and barcodes within a mobile camera feed.
- Visual Feed Overlays - Supports drawing shapes, text, or shaders over a real-time camera feed to provide visual feedback.
