timerring/bilive
Bilive
Bilive is a multimodal AI video pipeline and live stream recording tool designed to capture real-time broadcasts and automate the creation of highlight clips. It functions as a multi-platform stream orchestrator capable of distributing looped pre-recorded content and managing the automated upload of processed video clips to various destinations.
The system distinguishes itself through AI-driven content generation, using comment density to detect high-energy segments and multimodal models to automatically produce descriptive titles and synchronized subtitles. It further utilizes image-to-image AI models to transform video screenshots into stylized cover art for thumbnails.
Technical capabilities include real-time stream recording with fragment merging to fix network-induced gaps, speech-to-text transcription, and the rendering of comment-based subtitle overlays. The project also provides asynchronous parallel upload queues and dynamic template interpolation for customizing video metadata.
Features
- Multimodal Inference Pipelines - Orchestrates a multimodal AI pipeline to generate titles, subtitles, and cover art from video and audio data.
- Live Stream Recorders - Captures live broadcast video and user comments in real time to create high-quality archives.
- AI Transcription APIs - Provides AI-driven speech-to-text transcription using local GPU acceleration or external APIs to create synchronized subtitles.
- Multimodal AI Pipeline Orchestration - Coordinates a multimodal AI chain that combines speech transcription, title generation, and thumbnail creation.
- Speech Transcription - Converts spoken audio from live streams into written text using GPU-accelerated local models or APIs.
- AI Highlight Extractors - Automatically detects high-energy highlights using comment density and generates engaging titles via AI.
- Highlight Detection - Automatically detects high-energy video highlights by analyzing live chat comment density and multimodal data.
- Comment Volume Analysis - Analyzes live chat comment volume to automatically detect high-energy segments for highlight extraction.
- Automated Subtitle Generators - Automates the end-to-end process of transcribing spoken audio and embedding synchronized subtitles into video.
- Multi-Platform Stream Distribution - Orchestrates the distribution of looped content and processed clips across multiple streaming destinations.
- Video Clip Extraction - Extracts short, high-energy highlight segments from long-form live streams for social media distribution.
- Video Fragment Merging - Detects and merges interrupted stream segments to fix recording gaps caused by network instability.
- Social Media Publishers - Automates the publishing of processed highlight clips to social media platforms using customizable title and description templates.
- Content Title Generators - Uses multimodal language models to analyze video segments and automatically produce descriptive titles.
- Image-to-Image Translation - Transforms extracted video screenshots into stylized thumbnail cover art using image-to-image diffusion models.
- Video Publishing Integrations - Provides connectors for automatically publishing processed video recordings and highlight clips to external platforms.
- Bullet Comment Rendering - Renders scrolling, overlay-style bullet comments from data files directly onto the video output.
- Subtitle Rendering Layers - Renders synchronized on-screen subtitle overlays derived from speech-to-text transcriptions and live comment data.
- Burned-In Subtitle Rendering - Permanently burns structured comment data and synchronized subtitles directly into the video output frames.
- Social Media Cover Generators - Generates stylized professional cover art for social media platforms by transforming video screenshots.
- Upload Automation - Automates the end-to-end capture and upload of processed highlight clips via configurable pipelines.
- AI Thumbnail Generators - Transforms extracted video screenshots into stylized cover art for thumbnails using image-to-image AI models.
- Looped Stream Distribution - Distributes a constant automated loop of pre-recorded content across multiple streaming platforms simultaneously.
- Media & Communication - Automated Bilibili live stream recording and management system.
