yangchris11/samurai
Samurai
SAMURAI is a zero-shot visual tracking model that adapts the Segment Anything architecture for video object segmentation. It uses a first-frame prompt, such as a bounding box or mask, to initialize tracking, then employs a motion-aware memory mechanism that stores and updates temporal motion features across frames to guide mask refinement. An online memory update strategy continuously refreshes this memory with new frame predictions, while temporal motion encoding computes optical flow between consecutive frames to inform object boundary and occlusion handling.
The system is designed for real-time inference, optimizing model forward passes and memory operations to achieve tracking speeds on standard hardware. It operates without requiring any task-specific fine-tuning or training, enabling zero-shot tracking of unseen objects in video. SAMURAI also handles objects that enter the video after the initial frame, detecting and following them once they appear.
The project provides a custom video demo runner that accepts a video file or frame directory and a first-frame bounding box to produce tracking results. It supports semi-supervised video object segmentation, generating segmentation predictions using a pre-trained model and ground-truth masks from the first frame.
