Projects sharing features with Jianchang512 Stt

Whishper is a graphical user interface for transcribing audio and video files into text using the Whisper model. It serves as a speech-to-text tool and subtitle file generator that converts spoken content into editable text and timed subtitle formats. The project features an integrated transcription and translation interface, allowing users to refine automated results and convert transcribed text into different languages. It includes a visual editor for correcting speech recognition errors, adjusting segment timecodes, and performing bilingual translation reviews. The system handles the full

AI-Media2Doc is a self-hosted media transcription and content repurposing platform. It converts audio and video files into structured documents, searchable transcripts, and notes using a local speech-to-text pipeline and language model integration. The system enables the creation of illustrated articles by extracting video frames based on subtitle timestamps. Users can control the style and structure of generated text through custom behavioral prompts and integrate external language models via configurable API endpoints. The platform covers automated speech transcription, multimedia content

Vibe is a cross-platform transcription tool that converts spoken audio into text by running Whisper neural models directly on your device, with no cloud dependency. It can transcribe audio from files, microphones, system output, and network streams, and supports both batch processing of multiple files and real-time captioning from continuous input. Beyond basic transcription, Vibe identifies and labels different speakers through speaker diarization, and offers a choice of Command-Line Interface or HTTP API for automated and remote workflows. It also includes plugins to export transcripts to c

SmartSub is a cross-platform desktop application for AI-driven video transcription and subtitle generation. It converts audio and video files into text subtitles using local AI models and incorporates hardware acceleration to increase processing speed. The tool features a subtitle translator that leverages large language models, such as OpenAI and DeepSeek, to convert subtitles between different languages. It includes a visual editor for proofreading and polishing transcribed text, paired with a video preview for frame-accurate synchronization. The software supports batch processing of multi

autoMate is an AI agent tool server and multi-model AI gateway that exposes local system tools and data to chat clients via a standard protocol. It functions as a computer use agent capable of controlling desktop interfaces and browsers to automate local system tasks through natural language. The project serves as a SaaS integration platform, bridging AI agents to third-party services such as GitHub, Notion, Slack, and Jira. It also operates as a personal knowledge management system that stores markdown notes and files using a hybrid keyword and full-text search memory system. The software f

ESPnet is a comprehensive speech processing toolkit and PyTorch-based trainer designed for building end-to-end speech recognition, synthesis, and translation models. It provides a structured framework for developing automatic speech recognition systems using transducer and encoder-decoder architectures, alongside engines for text-to-speech synthesis and speech translation pipelines. The project distinguishes itself through a recipe-based workflow execution system that ensures experimental reproducibility by running standardized sequences of scripts for data preparation and model training. It

Streamer-Sales is a digital avatar video generator and large language model sales agent framework. It functions as a multimodal AI pipeline that synthesizes talking-head videos using text-to-speech and face animation to create virtual spokespeople. The system differentiates itself by combining retrieval-augmented generation with agentic web searching to ground product descriptions and sales scripts in factual data and real-time information. It further integrates a multimodal interaction workflow that coordinates automatic speech recognition and text-to-speech for real-time voice conversations

This project is an AI-driven suite of tools designed to repurpose long-form video content into short-form clips. It integrates a speech-to-text engine for automated transcription, a highlighting system that ranks engaging segments based on emotional hooks, and a video processor that converts horizontal footage into vertical formats. The system distinguishes itself through intelligent video cropping that utilizes face tracking and motion smoothing to keep subjects centered. It also employs an analysis system to extract viral highlights by scoring segments for engagement and practical value. T

Fonoster is a conversational AI framework and multi-tenant communications platform as a service. It serves as a programmable voice gateway and SIP telephony platform, enabling the creation of voice-based assistants and automated communication workflows using large language models. The project distinguishes itself through a vendor-agnostic speech integration engine that abstracts speech-to-text and text-to-speech providers. It features a multi-tenant architecture that isolates telephony resources and user identities into distinct organizational workspaces. The system covers a broad range of t

Kimi-Audio is a large language model audio foundation model designed to understand audio input and generate high-fidelity speech responses in real time. It functions as a unified system encompassing a text-to-speech synthesis engine and a speech-to-text transcription tool. The project enables real-time audio conversations through a multi-modal conversation loop and chunk-wise streaming detokenization to reduce playback latency. It provides controls over speech speed, accent, and emotional tone during conversational audio generation. The system covers audio intelligence capabilities, includin

Omniparse is a multimodal content parser and generative AI ingestion engine designed to convert documents, images, and multimedia into a uniform format. It functions as a data preprocessing pipeline that transforms diverse raw data sources into structured markdown to improve the performance of large language model workflows. The system extracts text and structural data from PDFs, images, audio, and video files. It includes a web crawler that converts dynamic website content into clean markdown and a multimodal transformation process that maps disparate input formats into a unified data schema

This project is a framework for developing multimodal AI agents that function as programmable participants in real-time communication rooms. It enables the construction of agents that can see, hear, and speak by integrating speech-to-text, large language models, and text-to-speech pipelines to facilitate low-latency, natural conversations. The system is distinguished by its advanced orchestration of real-time media and conversational flow, including support for full-duplex speech, preemptive response generation, and sophisticated interruption management. It further differentiates itself throu

This project is an open source Linux GPU kernel driver implemented as a loadable kernel module. It functions as a GPU firmware loader, providing the low-level driver services necessary to enable direct communication between the operating system and graphics processing units. The driver utilizes a dual-module architecture that separates GPL-licensed kernel code from proprietary firmware blobs. This system extracts and links signed binary firmware images into the kernel modules at driver load time. The project provides driver support for Turing-architecture GPUs and all subsequent newer hardwa

LocalAI is a local generative AI platform and inference engine designed to host large language, vision, and audio models on private hardware. It functions as an API compatible gateway that mimics proprietary service endpoints, allowing existing third-party software to integrate with a self-hosted backend. The platform distinguishes itself as a distributed AI model orchestrator, capable of scaling inference across machine clusters using VRAM-aware routing and hardware coordination. It provides a unified interface for diverse open-source backends and supports self-hosted RAG infrastructure thro

Linly-Dubbing is an automated video dubbing pipeline designed for multilingual video localization. It converts spoken content in videos into another language by coordinating speech-to-text transcription, text translation, and text-to-speech synthesis. The system distinguishes itself through AI-driven lip synchronization and animation, which aligns facial expressions and mouth movements to the synthesized voiceover. It also utilizes audio source separation to isolate vocals from background music and noise, allowing for clean voice replacement while preserving original background audio. The br

musicdl is a command line music downloader and library manager designed for searching and downloading audio tracks and playlists from streaming platforms to local storage. It functions as a tool for music library archiving, allowing for the bulk acquisition of media and the organization of local audio collections. The project includes an AI lyric transcriber that uses machine learning models to generate text lyrics from audio files, supporting synchronized playback where lyrics are highlighted based on playback timestamps. To maintain access to streaming platforms, it employs a network proxy

This Python SDK provides a comprehensive toolkit for synthetic audio generation, voice cloning, and the development of conversational AI agents. It enables the creation of lifelike spoken audio from text, the replication of human voices through custom cloning, and the deployment of real-time voice agents capable of interacting with external large language models. The library distinguishes itself through deep integration of conversational AI capabilities, including the design of agent personas and the execution of real-time actions via APIs. It supports professional-grade audio production thro

KrillinAI is an AI video localization pipeline and toolset designed to automate the process of transcribing, translating, and dubbing video content into multiple languages. It provides a command-line interface to chain these stages into a single production workflow, coordinating speech-to-text transcription, translation, and audio generation. The system features a translation framework that uses large language models to maintain professional terminology and natural semantics rather than literal word replacement. It includes a dubbing tool that utilizes text-to-speech and voice cloning to gene

This project is a multimodal translation framework and large language model capable of speech-to-speech, speech-to-text, and text-to-text translation across nearly 100 languages. It provides a real-time speech translation engine and a comprehensive toolkit for converting spoken audio between languages. The system is distinguished by its ability to preserve the original speaker's tone, pace, and prosody during translation. It utilizes a specialized on-device inference toolkit that converts model checkpoints into C-based libraries, enabling low-latency execution on mobile and edge hardware with

Bella is an AI companion system featuring a conversational interface for interacting with local and cloud artificial intelligence models. It integrates a local model manager to automate the download and organization of machine learning weights and a speech-to-text transcription engine to enable hands-free interaction. The project includes an emotion visualization system that uses cross-fading video playback to represent the agent's internal state. This is driven by a state-driven emotion mapping system and an interaction-based affinity system that tracks user engagement frequency to trigger s

Autocut is a text-based video editor and automatic speech recognition tool. It allows users to cut and merge video clips by modifying a text transcript instead of using a traditional timeline. The system operates as an FFmpeg video processor and subtitle manipulation utility. It converts spoken audio into text and compacts subtitle files into simplified formats, enabling the removal of unwanted video segments by deleting corresponding sentences from a transcription file. The project covers automated video transcription, non-linear video cutting, and subtitle file management. It supports hard

mlx-audio is an audio processing toolkit built on Apple MLX that provides speech transcription, text-to-speech synthesis, voice cloning, and audio source separation using local models. It offers an OpenAI-compatible REST API and web interface for running audio generation and transcription tasks, enabling drop-in integration with existing tools that follow that endpoint structure. The toolkit supports text-prompted audio source separation, allowing specific sounds to be isolated from mixed recordings based on natural language descriptions. It also provides voice cloning from a short reference

Julius is a high-performance, open-source speech recognition engine designed for large vocabulary continuous speech recognition. It functions as a comprehensive framework utilizing Hidden Markov Model-based acoustic modeling and N-gram language models to convert live or recorded audio into text. The engine is built to support real-time streaming and provides a network-accessible service that allows external applications to manage recognition sessions and receive transcription results through programmatic commands. The engine distinguishes itself through its modular architecture and support fo

uncaptcha2 is an audio captcha bypass tool and ReCaptcha solver designed to circumvent bot detection systems. It functions as a speech-to-text captcha solver that converts audio security challenges into text to automate the completion of digit-based verification steps. The tool enables automated web scraping by removing security barriers and bypassing ReCaptcha challenges. It facilitates automated form submission by programmatically solving the verification steps required to access restricted websites.

Autosub is a command-line media processor and automatic subtitle generator that converts audio streams from video and audio files into timed text overlays. It functions as an AI speech-to-text converter that uses OpenAI Whisper to generate synchronized subtitles. The tool includes a language translation pipeline to convert transcribed speech into target languages, enabling multilingual video captioning. It manages the process from audio-stream extraction to the serialization of final subtitle files for local storage. The system covers audio-to-text transcription, time-stamped text mapping, a