# multimodal-art-projection/yue **Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/multimodal-art-projection-yue).** 6,292 stars · 744 forks · Python · Apache-2.0 ## Links - GitHub: https://github.com/multimodal-art-projection/YuE - Homepage: https://map-yue.github.io - awesome-repositories: https://awesome-repositories.com/repository/multimodal-art-projection-yue.md ## Topics `ai` `audio-generation` `deep-learning` `foundation-models` `gpt` `huggingface` `llama` `llms` `music-generation` `style-transfers` `voice-cloning` ## Description YuE: Open Full-song Music Generation Foundation Model, something similar to Suno.ai but open ## Tags ### Development Tools & Productivity - [Lyrics-to-Full-Song Generators](https://awesome-repositories.com/f/development-tools-productivity/integration-metadata-retrievers/media-metadata-retrievers/lyric-retrieval/ai-generated-lyrics/lyrics-to-full-song-generators.md) — Turns written lyrics into a complete song with vocals and accompaniment by processing each labeled section like verse and chorus. ([source](https://cdn.jsdelivr.net/gh/multimodal-art-projection/yue@main/README.md)) ### Artificial Intelligence & ML - [Genre-Conditioned Generation](https://awesome-repositories.com/f/artificial-intelligence-ml/ai-music-asset-creation/genre-transitions/genre-conditioned-generation.md) — Generates music matching user-defined combinations of genre, instrument, mood, and vocal style tags. - [Open-Source Models](https://awesome-repositories.com/f/artificial-intelligence-ml/ai-music-composition/open-source-models.md) — Provides an open-source alternative to commercial AI music services for generating full songs with vocals and instrumental tracks. - [Multi-Resolution Audio Tokenizers](https://awesome-repositories.com/f/artificial-intelligence-ml/audio-tokenization/multi-resolution-audio-tokenizers.md) — Tokenizes audio into discrete tokens at multiple temporal scales for efficient autoregressive generation. - [Multi-Scale Audio Tokenizers](https://awesome-repositories.com/f/artificial-intelligence-ml/audio-tokenization/multi-scale-audio-tokenizers.md) — Tokenizes audio into discrete tokens at multiple temporal resolutions for efficient generation. - [Phoneme-to-Note Alignment Mechanisms](https://awesome-repositories.com/f/artificial-intelligence-ml/audio-transcription/audio-transcript-aligners/lyric-to-audio-alignment/phoneme-to-note-alignment-mechanisms.md) — Aligns phoneme-level lyric timing with generated musical notes using a cross-attention mechanism. - [Autoregressive Transformers](https://awesome-repositories.com/f/artificial-intelligence-ml/autoregressive-transformers.md) — Generates music tokens sequentially using a transformer decoder with causal masking and shared latent conditioning. - [Vocal-Accompaniment Dual Generators](https://awesome-repositories.com/f/artificial-intelligence-ml/dual-generator-architectures/vocal-accompaniment-dual-generators.md) — Generates separate vocal and accompaniment audio tracks simultaneously using a shared latent representation. - [Dual-Track Latent Decoders](https://awesome-repositories.com/f/artificial-intelligence-ml/generative-ai-resources/diffusion-visual-models/generative-ai-models/latent-space-generative-models/shared-latent-spaces/dual-track-latent-decoders.md) — Generates vocal and accompaniment tracks simultaneously from a shared latent representation using separate decoders. - [Style-Based Music Generation](https://awesome-repositories.com/f/artificial-intelligence-ml/generative-music-agents/style-based-music-generation.md) — Creates new songs that mimic the musical style and structure of a provided reference audio clip. - [Genre and Vocal Style Generators](https://awesome-repositories.com/f/artificial-intelligence-ml/music-genre-classifiers/genre-and-vocal-style-generators.md) — Produces music across a wide range of genres and vocal styles, from metal to jazz to rap. ([source](https://map-yue.github.io/)) - [Genre-Tag Conditioned Music Generators](https://awesome-repositories.com/f/artificial-intelligence-ml/music-genre-classifiers/genre-tag-conditioned-music-generators.md) — Creates music that matches a user-defined combination of genre, instrument, mood, and vocal style tags. ([source](https://cdn.jsdelivr.net/gh/multimodal-art-projection/yue@main/README.md)) - [Music Generation Models](https://awesome-repositories.com/f/artificial-intelligence-ml/open-models/music-generation-models.md) — An open-source foundation model that generates complete songs with vocals and accompaniment from lyrics and genre tags. - [Phoneme-to-Note Cross-Attention Mechanisms](https://awesome-repositories.com/f/artificial-intelligence-ml/sequence-alignment-models/phoneme-based-alignment/phoneme-to-note-cross-attention-mechanisms.md) — Aligns phoneme-level lyric timing with generated musical notes using a cross-attention mechanism between text and audio tokens. - [Song Section Structural Prompts](https://awesome-repositories.com/f/artificial-intelligence-ml/structural-prompt-design/song-section-structural-prompts.md) — Encodes labeled lyric sections as structural prompts that guide the transformer decoder to follow song structure. - [Audio Reference Style Conditioning](https://awesome-repositories.com/f/artificial-intelligence-ml/transformer-architectures/diffusion-transformers/speech-latent/style-conditioned-diffusion-decoders/audio-reference-style-conditioning.md) — Conditions generation on a reference audio clip by extracting and injecting style embeddings into the model. - [Lyric-to-Audio Alignment](https://awesome-repositories.com/f/artificial-intelligence-ml/audio-transcription/audio-transcript-aligners/lyric-to-audio-alignment.md) — Aligns phoneme-level lyric timing with generated musical notes using cross-attention for structured song composition. ### Part of an Awesome List - [Text-to-Music Generators](https://awesome-repositories.com/f/awesome-lists/media/music-and-audio-generation/text-to-sound-effect-generation/text-to-music-generators.md) — Generates complete songs with vocals and accompaniment from user-provided lyrics and genre tags. - [Multi-Language Lyric Generators](https://awesome-repositories.com/f/awesome-lists/media/lyrics-management/multi-language-lyric-generators.md) — Produces songs with lyrics in languages such as English, Mandarin, Cantonese, Japanese, and Korean. ([source](https://cdn.jsdelivr.net/gh/multimodal-art-projection/yue@main/README.md)) ### Business & Productivity Software - [Music Generation Alternatives](https://awesome-repositories.com/f/business-productivity-software/open-source-productivity-alternatives/music-generation-alternatives.md) — An open-source alternative to Suno.ai for generating full songs with both vocal and instrumental tracks. - [Multi-Language Lyric Generation](https://awesome-repositories.com/f/business-productivity-software/digital-library-search/song-searches/multi-language-lyric-generation.md) — Produces songs with lyrics in English, Mandarin, Cantonese, Japanese, and Korean across diverse genres. ### Graphics & Multimedia - [Lyric Section Structure Composers](https://awesome-repositories.com/f/graphics-multimedia/lyric-composition-tools/lyric-section-structure-composers.md) — Divides lyrics into labeled sections like verse and chorus, then generates music that follows that structure. ([source](https://cdn.jsdelivr.net/gh/multimodal-art-projection/yue@main/README.md)) - [Section-Structured Lyrics](https://awesome-repositories.com/f/graphics-multimedia/lyric-composition-tools/section-structured-lyrics.md) — Divides lyrics into labeled sections like verse and chorus to generate music that follows the specified structure. - [Genre-Conditioned Generation Embeddings](https://awesome-repositories.com/f/graphics-multimedia/music-metadata-retrieval/genre-hierarchies/genre-metadata-propagation/genre-conditioned-generation-embeddings.md) — Embeds genre tags as learnable vectors that modulate the generation process across all layers. - [Genre-Conditioned Transformer Modulation](https://awesome-repositories.com/f/graphics-multimedia/music-metadata-retrieval/genre-hierarchies/genre-metadata-propagation/genre-conditioned-transformer-modulation.md) — Embeds genre tags as learnable vectors that modulate transformer activations across all layers. - [Dual-Track Generation](https://awesome-repositories.com/f/graphics-multimedia/media-processing-analysis/audio-processing-systems/audio-processing/speech-to-text-pipelines/audio-processing-pipelines/generative-audio-pipelines/dual-track-generation.md) — Generates separate vocal and accompaniment audio tracks simultaneously using a shared latent representation. ### User Interface & Experience - [Song Section Structure Encoders](https://awesome-repositories.com/f/user-interface-experience/content-dividers/content-sectioning/resume-section-structures/song-section-structure-encoders.md) — Encodes labeled lyric sections as structural prompts that guide the music generation process. - [Audio Style Embedding Injections](https://awesome-repositories.com/f/user-interface-experience/css-styling/style-injections/audio-style-embedding-injections.md) — Extracts style embeddings from a reference audio clip and injects them into the generation process via cross-attention. ### Content Management & Publishing - [Multilingual Music Generation](https://awesome-repositories.com/f/content-management-publishing/content-management-systems/content-architecture-modeling/documentation-tooling/generation-publishing/documentation-generators/multilingual-generation/multilingual-audio-generators/multilingual-music-generation.md) — Produces songs with lyrics in English, Mandarin, Cantonese, Japanese, and Korean across diverse genres.