Projects sharing features with Magenta

PyTorch implementation of Image-to-Image Translation Using Conditional Adversarial Networks.

We have a reimplementation of the UNIT method that is more performant. It is avaiable at Imaginaire

This repository serves as a centralized collection of state-of-the-art deep learning architectures and reference implementations designed for research and application development. It provides a comprehensive toolkit for computer vision and natural language processing, offering pre-built models and training pipelines for tasks ranging from image classification and object detection to complex sequence modeling. The project distinguishes itself by providing a flexible execution harness that manages the entire training lifecycle, including data ingestion and backpropagation. It supports scalable

This is a PyTorch implementation of the paper NIMA: Neural IMage Assessment (accepted at IEEE Transactions on Image Processing) by Hossein Talebi and Peyman Milanfar. You can learn more from this post at Google Research Blog.

pix2pixHD is a conditional generative adversarial network designed to transform semantic label maps into high-resolution photorealistic images. It functions as a high-resolution image synthesizer and an image-to-image translation model capable of producing synthetic images at 2048x1024 resolution. The system includes a semantic image editor that allows for the modification of high-resolution visuals by updating the underlying semantic label maps. This enables interactive image editing and the generation of photorealistic images based on source images or discrete label maps. The framework pro

This project is an unsupervised image restoration tool that uses a convolutional neural network as a structural prior to reconstruct images from noisy or incomplete data. It functions as a neural network image prior, utilizing the inherent biases of the network architecture to restore pixels without the need for a pre-trained dataset or external learning. The system performs zero-shot image restoration by treating the network architecture itself as a regularization term. It uses a randomly initialized encoder-decoder structure and iterative gradient descent to minimize pixel-wise loss, recove

This project is a deep learning framework designed for training and deploying image-to-image translation models. It serves as a research platform for experimenting with neural network architectures that transform visual content between distinct stylistic domains, supporting both paired and unpaired training data. The framework distinguishes itself through its support for cycle-consistency constraints, which allow for image translation between domains without requiring corresponding paired examples. It provides a structured pipeline that utilizes adversarial loss optimization, where generator

This is a PyTorch-based computer vision library for detecting 2D and 3D facial landmark coordinates. It functions as a facial landmark detector and reconstruction tool, utilizing deep learning to identify precise geometric points on human faces from image datasets. The library allows for the selection of specific detection backends to balance accuracy and processing speed. It supports the integration of precomputed bounding box files, which enables the system to bypass the initial detection phase and proceed directly to landmark extraction. The toolkit includes capabilities for batch image p

Muzic is a deep learning platform and framework for AI-driven music analysis, composition, and synthesis. It functions as a music generation framework and analysis tool, utilizing large language models and autonomous agents to orchestrate the creation and interpretation of symbolic and audio music. The project is distinguished by its cross-modal capabilities, mapping natural language and symbolic music into a shared joint embedding space for zero-shot classification and information retrieval. It employs a variety of specialized architectures, including diffusion frameworks for audio synthesis

DeepDream is a deep learning image processor and convolutional neural network art generator designed to synthesize psychedelic imagery and visualize how neural networks interpret visual data. It functions as a tool for generating generative AI art by amplifying patterns recognized by a pre-trained model to produce dream-like effects. The project utilizes a TensorFlow image visualizer to explore how different layers of a neural network perceive images. This is achieved through algorithmic image manipulation and deep learning visualization techniques that transform standard photographs into sty

This Python SDK provides a comprehensive toolkit for synthetic audio generation, voice cloning, and the development of conversational AI agents. It enables the creation of lifelike spoken audio from text, the replication of human voices through custom cloning, and the deployment of real-time voice agents capable of interacting with external large language models. The library distinguishes itself through deep integration of conversational AI capabilities, including the design of agent personas and the execution of real-time actions via APIs. It supports professional-grade audio production thro

Jukebox is a generative audio model and AI music synthesis tool designed to create high-fidelity music samples and singing voices. It functions as a deep learning system that synthesizes raw audio conditioned on genre and artist metadata, utilizing a neural audio codec to convert raw audio into discrete codes for generative modeling and reconstruction. The system enables musical style steering and AI music composition by conditioning generation on specific artists, genres, and lyrics. It supports audio priming, allowing existing wave files to guide the creation of new musical sequences, and p

This project serves as a comprehensive, curated directory of resources, tools, and platforms dedicated to the generative artificial intelligence ecosystem. It functions as a central hub for developers and researchers to discover the frameworks, models, and services necessary for building, deploying, and managing intelligent software applications. The directory distinguishes itself by providing a structured index of specialized tooling across several technical domains. It covers the full lifecycle of generative AI, including the development of autonomous agent systems, the implementation of re

Audiocraft is a deep learning audio library and machine learning framework designed for training, fine-tuning, and evaluating generative models for music and sound effects. It functions as a text-to-music generative model and a neural audio codec, providing the tools necessary to compress audio signals into discrete representations and synthesize high-fidelity waveforms from textual descriptions. The framework is distinguished by its ability to combine multiple conditioning signals, allowing for the generation of audio based on text prompts, melodic excerpts, or style-based audio clips. It al

StyleGAN2 is a TensorFlow generative adversarial network and image synthesis model designed to produce high-resolution synthetic visual content. It functions as a deep learning architecture that learns patterns from image datasets to synthesize new images. The project includes a latent space projection tool for mapping existing images to latent vectors to analyze their representation within a generative model. It also provides an image quality evaluation framework to measure the visual fidelity and diversity of synthetic outputs. The system covers the full generative pipeline, including imag

StyleGAN is a TensorFlow-based generative adversarial network framework designed for the synthesis of high-resolution synthetic imagery. It utilizes a style-based generator architecture to create realistic visual assets from latent vectors, focusing on the production of high-fidelity images. The system incorporates style mixing and stochastic noise injection to control visual attributes and fine-grained details. It uses adaptive instance normalization and progressive resolution upsampling to manage image quality and variety across different resolutions. The framework covers the full lifecycl

This project is a diffusion model training framework and image synthesis pipeline. It provides the tools necessary to train generative models to learn image data distributions through an iterative denoising process. The framework includes a generative model evaluation tool consisting of automated scripts used to measure the quality and accuracy of produced samples. The system covers model training pipelines and performance evaluation for generative diffusion models.

CycleGAN is a generative adversarial network framework designed for unpaired image-to-image translation. It enables the conversion of images between two distinct visual domains using datasets that do not require direct one-to-one matching examples. The project implements a deep learning style transfer tool capable of artistic style transfer, object transfiguration, and domain-to-domain conversion. It uses a dual-generator architecture and cycle-consistency loss to ensure that images translated to a target domain and back recover their original state. The framework covers core machine learnin

ParlAI is a conversational AI research framework designed for training, evaluating, and sharing dialogue models using a unified interface for datasets and agents. It functions as a PyTorch-based training platform and a dialogue data collection system, providing a centralized model zoo for the distribution of versioned pretrained agents. The project distinguishes itself through a knowledge-grounded retrieval system that combines dense and sparse indexing to ground responses in external information. It also provides a comprehensive infrastructure for gathering human-AI interaction data via inte

PyTorch-GAN is a research-oriented framework providing a collection of modular implementations for generative adversarial network architectures. It serves as a toolkit for training and evaluating models that utilize adversarial minimax optimization to produce synthetic data, offering a structured environment for exploring complex generative tasks within the PyTorch ecosystem. The library distinguishes itself through a comprehensive suite of image synthesis and manipulation capabilities, including super-resolution, inpainting, and cross-domain style translation. It supports advanced training m

Fairseq is a PyTorch toolkit for sequence-to-sequence modeling, specializing in neural machine translation, automatic speech recognition, and large-scale language model training. It provides a framework for processing and aligning diverse data sources, including text, audio, and video, to support tasks such as speech-to-text conversion and multimodal sequence learning. The project is distinguished by its distributed training capabilities, which utilize parameter sharding, mixed-precision training, and CPU offloading to handle models that exceed single-device memory. It also includes specializ

Open-Sora-Plan is a text-to-video framework and distributed video training system. It utilizes a diffusion transformer architecture and large language model components to transform written descriptions or image prompts into high-quality video sequences. The system features a distributed infrastructure designed for large-scale video training and inference. It employs sequence parallelism to split high-resolution or long-duration video samples across multiple GPUs and uses a sparse attention mechanism to increase processing speed. The project includes capabilities for both text-to-video and im

This is a TensorFlow-based encoder-decoder framework and model library used for mapping input sequences to output sequences. It functions as a deep learning sequence mapper designed to transform sequential data from one domain to another. The library provides tools for implementing sequence-to-sequence modeling across multiple domains, including neural machine translation, automatic text summarization, and image captioning generation. The framework incorporates recurrent neural networks and utilizes attention-based contextualization to weight input sequences. It supports multiple decoding st

This project is an educational resource and comprehensive guide for implementing and deploying deep learning models using the PyTorch framework. It provides a structured learning curriculum consisting of tutorials and notebooks that cover neural network architectures, data pipelines, and model optimization across multiple AI domains. The curriculum includes practical implementation guides for building convolutional networks, transformers, and recurrent models. It specifically focuses on workflows for computer vision, including image classification, object detection, and segmentation, as well

This project is a comprehensive educational curriculum and structured learning path covering the full lifecycle of large language models. It provides a guided progression through the theory, architecture, training, and deployment of these models. The curriculum includes specialized guides on transformer architecture, model training tutorials, and frameworks for designing autonomous agents. It also provides dedicated resources for studying model safety and ethics. The material covers a wide range of technical capabilities, including distributed training strategies, parameter-efficient fine-tu