ngxson/smolvlm-realtime-webcam
Smolvlm Realtime Webcam
This is a webcam-based client for a local llama.cpp server that enables real-time object detection and vision-language model inference directly from a browser. It captures frames from the user's webcam at configurable intervals and sends them to a locally running inference server for analysis, displaying both detection results and textual scene descriptions as they are produced.
The application distinguishes itself by combining object detection with vision-language scene description in a single real-time interface, all processed through a local llama.cpp server for private, offline operation. It handles frame processing requests asynchronously to keep the UI responsive, and allows users to adjust the frame-capture interval from 100 milliseconds to 2 seconds to balance responsiveness with performance.
The tool provides a complete pipeline for real-time webcam analysis, from capturing live frames to sending them to a vision-language model that generates textual descriptions and identifies objects in the video feed. Configuration options focus on adjusting the timing between successive captures and detection requests.
Features
- Local Object Detection - Sends captured webcam frames to a local AI server for object detection and displays results.
- Real-Time Object Detection - Captures live webcam frames and sends them to a local AI server to identify objects in the video feed.
- Vision-Language Models - Sends captured frames to a vision-language model that generates textual descriptions and object detection results.
- Vision-Language Inference - Sends camera frames to a local inference server and displays textual scene descriptions from a vision-language model.
- Vision-Language Model Analyses - A real-time webcam application that sends captured frames to a local vision-language model for object detection and scene description.
- Webcam Frame Captures - Accesses the user's webcam hardware to continuously capture frames for immediate processing and display.
- llama.cpp Backend Servers - Communicates with a local llama.cpp server to process webcam frames through a vision-language model.
- Web Chat Frontends - A web interface that communicates with a local llama.cpp inference server to process webcam frames through a vision-language model.
- Configurable Capture Intervals - Captures webcam frames at user-defined intervals ranging from 100ms to 2 seconds.
