kenshohara/3D-ResNets-PyTorch
3D ResNets PyTorch
This project is a PyTorch implementation of 3D residual networks designed for video action recognition. It provides a spatiotemporal architecture that analyzes both spatial frames and temporal motion to classify human activities within video clips.
The system includes a distributed model training framework to accelerate learning across multiple compute nodes. It supports the deployment and fine-tuning of pre-trained model weights, allowing the adaptation of existing networks to specific new datasets.
The codebase covers the full pipeline for spatiotemporal learning, including video dataset preprocessing tools for converting raw files into image sequences, action inference capabilities, and metrics for calculating recognition accuracy.
Features
- Action Recognition - Classifies human activities and behaviors within video sequences using deep 3D residual networks.
- Action Recognition Training - Provides a framework for training neural networks to learn and recognize complex motion features in large-scale video datasets.
- 3D - Implements 3D residual networks specifically designed for classifying human actions in video sequences.
- 3D Convolutional Networks - Implements 3D convolutional networks to process spatiotemporal video data for action recognition.
- Spatiotemporal - Captures both appearance and motion patterns by analyzing video frames as single volumetric data blocks.
- Video Action Inference - Provides video action inference capabilities to predict activity class probabilities from trained models.
- Residual Connection Implementations - Employs residual connection implementations to mitigate vanishing gradients in deep spatiotemporal networks.
- Action Recognition Models - Implements an action recognition model based on 3D residual networks for identifying human activities in video.
- Data Parallelism - Provides data parallelism to split large video datasets across multiple compute nodes for synchronized training.
- Video Sequence Preprocessing - Provides video sequence preprocessing utilities to transform raw video into training-ready image frames.
- Distributed Video Model Training - Implements distributed video model training to scale the processing of extensive datasets across multiple GPU nodes.
- Pre-trained Weight Loading - Implements pre-trained weight loading for the immediate deployment of existing model checkpoints.
- Distributed Training - Ships a distributed training system for scaling spatiotemporal network learning across multiple compute nodes.
- Classification Fine-Tuning - Supports classification fine-tuning to adapt pre-trained weights to specific action recognition datasets.
- Vision Dataset Preparation - Provides tools for converting raw video files into formatted image sequences and annotations for training.
- Model Fine-Tuning - Provides mechanisms for optimizing pre-trained 3D ResNet weights on task-specific video datasets.
- Vision Model Fine-Tuning - Supports adapting pre-trained 3D ResNet weights to specific new datasets through specialized fine-tuning of the classification layers.
- Computer Vision Models - Action recognition using 3D residual networks.
- Computer Vision Research - Spatiotemporal 3D CNNs for video action recognition.
- Model Implementations - 3D ResNets for action recognition.
- Video Representation Learning - Provides 3D ResNet architectures for spatiotemporal feature learning.
- Facial, Action and Pose Recognition - Listed in the “Facial, Action and Pose Recognition” section of the The Incredible Pytorch awesome list.
