RayVentura/ShortGPT
ShortGPT
ShortGPT is an automated short-form video creation framework that combines large language model-driven scripting with neural voice synthesis, visual asset retrieval, and programmatic video editing. The project provides a modular pipeline architecture that chains script generation, voiceover synthesis, caption rendering, and video assembly into automated workflows, enabling the production of complete short videos from a topic prompt.
The framework distinguishes itself through an LLM-oriented editing language that controls video assembly and rendering tasks programmatically, and a multilingual dubbing pipeline that transcribes, translates, and re-voices video audio into different languages with new captions. It integrates multiple neural text-to-speech engines supporting over 30 languages, and sources background footage and images from external media APIs based on script content.
The system covers the full video production lifecycle, including script writing, visual asset sourcing, voiceover generation, timed caption overlay, and automated video editing. It also supports video dubbing workflows that produce captioned videos in target languages.
Features
- Short-Form Video Generation - Generates complete short-form videos from a topic prompt using AI-driven scripting and assembly.
- Video Editing Agents - Controls video assembly and editing tasks programmatically through a natural-language editing language interpreted by a large language model.
- AI Video Dubbing Tools - Transcribes, translates, and re-voices a video's audio into a different language to produce a new captioned video.
- AI Video Script Generation - Writes a full video script from a topic or prompt using a large language model.
- Full Pipeline Automators - Generates a full video script from a topic and then sources visuals, records voiceover, and renders the final clip automatically.
- Audio Transcription & Re-Synthesis Pipelines - Transcribes source audio, translates text, and re-synthesizes voiceover in a target language.
- Voice Synthesis - Generates human-like voice audio in multiple languages by integrating neural voice synthesis engines.
- Video Assembly Languages - Controls video assembly and editing through a domain-specific language interpreted by a large language model.
- Text-to-Speech Synthesis - Converts written scripts into human-like spoken audio in over 30 languages using neural text-to-speech engines.
- Multi-Language Speech Generators - Converts written scripts into spoken audio in over 30 languages using neural TTS engines.
- Multi-Voice Synthesis Engines - Generates voiceovers by routing text through multiple neural TTS engines supporting 30+ languages.
- Timed Caption Compositors - Produces timed subtitle tracks by synchronizing text with audio timestamps and compositing onto video frames.
- Modular Pipeline Architectures - Chains discrete processing stages (script, voice, footage, captions) into a linear automation pipeline.
- AI Video Editing Automation - Controls video editing tasks programmatically using an LLM-oriented editing language to assemble and render complete videos.
- LLM-Orchestrated Video Pipelines - Orchestrates the complete video creation pipeline using LLMs for scripting, editing, and dubbing.
- Timed Caption Compositors - Creates and overlays timed captions onto the final video automatically.
- Media Data APIs - Retrieves background footage and images by querying external media APIs based on script content.
- Domain-Specific Editing Languages - Ships a domain-specific language interpreted by LLMs to control video assembly and rendering tasks.
- API-Based Background Sourcers - Retrieves relevant images and video clips from online APIs to match the script's content.
- API-Based Video Asset Sourcers - Retrieves relevant images and video clips from online APIs to match script content for automated video production.
- API-Based Visual Sourcers - Retrieves images and video footage from online sources to incorporate relevant visuals into content.
- Audio and Subtitle Tools - AI framework to automate short-form video creation.
