microsoft/TRELLIS
TRELLIS
TRELLIS is a 3D generative AI model and latent diffusion framework designed to transform natural language descriptions or reference images into textured 3D assets. It operates as a text-to-3D asset generator that utilizes structured latent representations to produce high-quality 3D meshes, Gaussians, and Radiance Fields.
The system functions as a multi-format 3D decoder, converting internal representations into standard exchange formats such as GLB and PLY. It also serves as a 3D asset editing tool, enabling the modification of specific regions of generated objects through targeted text or image-based prompts.
The framework covers a broad range of capabilities including cross-modal conditioning and diffusion-based latent generation. It supports large-scale model training across single or multi-node GPU configurations and provides workflows for creating visual variations of existing assets.
Features
- Text-to-3D Generators - Transforms natural language descriptions or reference images into high-quality textured 3D assets.
- Diffusion-Based 3D Generators - Uses a diffusion process to predict 3D latent representations from text or image embeddings.
- Latent Space Generative Models - Represents complex 3D geometry and texture as compressed tensors to enable scalable generation.
- 3D Asset Generators - Synthesizes detailed 3D objects with textures using text descriptions or image conditions.
- Generative 3D Modeling - Implements a deep learning framework for synthesizing 3D meshes, Gaussians, and Radiance Fields.
- Distributed GPU Training - Scales model training across multiple GPU nodes using synchronized gradient updates.
- Latent-to-Pixel Decoding - Converts internal compressed 3D representations into standard exchange formats like GLB and PLY.
- Asset Variation Generators - Creates new versions of existing 3D assets that remain visually aligned with specified text prompts.
- Large-Scale Model Training - Supports large-scale generative model training across single or multi-node GPU configurations.
- Large Scale Training - Trains generative 3D models across single or multi-node GPU setups for improved quality.
- Multi-Format Latent Decoding - Translates learned latent vectors into diverse outputs including 3D Gaussians, meshes, and radiance fields.
- 3D Scene Exporters - Provides a pipeline to export internal 3D representations into standard exchange formats such as GLB and PLY.
- 3D Asset Exporters - Converts generated 3D assets into standard industry file types such as GLB and PLY.
- Global Asset Variations - Provides capabilities to modify generated 3D objects to create overall visual variations.
- Local Region Editing - Enables modification of specific regions of a 3D model using targeted prompts to refine details.
- Prompt-Based Geometry Editing - Provides tools for modifying specific regions of 3D objects through targeted text or image-based prompts.
- 3D Representation Conversions - Converts internal 3D latent representations into various formats like meshes, 3D Gaussians, and Radiance Fields.
