microsoft/TRELLIS.2
TRELLIS.2
TRELLIS.2 is a generative image-to-3D system that creates high-resolution 3D assets with physically based rendering materials from 2D images. It utilizes a sparse voxel representation to handle complex topologies and internal structures without relying on iso-surface fields.
The project features a structured latent space representation that maps geometry and texture attributes to maintain visual fidelity. It employs an optimization-free geometry reconstruction process to decode latent representations directly into voxel grids and includes a PBR texture generator for synthesizing base color, roughness, and metallic maps.
The system covers 3D content generation and pipeline automation, including bidirectional conversion between textured surface meshes and sparse volumetric grids. It supports the export of generated assets into industry-standard GLB files.
The framework provides capabilities for training generative models from scratch or fine-tuning them on custom datasets.
Features
- 3D Asset Generators - Provides a generative system that synthesizes high-resolution 3D meshes and textures directly from 2D image inputs.
- Latent Space Encoders - Maps 3D geometry and texture attributes into a compressed coordinate system to maintain high visual fidelity.
- 3D Reconstruction - Decodes latent representations directly into voxel grids to generate 3D shapes without iterative refinement.
- Physically Based Rendering Materials - Applies photorealistic surface attributes including color, roughness, and opacity using physically based rendering materials.
- Physically Based Rendering - Calculates realistic surface attributes to simulate photorealistic light interaction on generated 3D assets.
- Generative PBR Map Generators - Provides a tool for creating photorealistic base color, roughness, and metallic maps for 3D rendering.
- Mesh and Voxel Conversions - Provides bidirectional transformation between textured surface meshes and sparse volumetric grids without requiring optimization.
- Non-Manifold Topology Modeling - Supports complex 3D topologies including open surfaces and enclosed interior structures without relying on iso-surface fields.
- Automated PBR Synthesis - Generates photorealistic surface attributes like roughness and metallic maps to enable realistic lighting.
- Generative PBR Textures - Synthesizes photorealistic base color, roughness, and metallic maps to create realistic 3D surface attributes.
- Sparse Voxel Representations - Utilizes a sparse voxel representation to handle complex topologies and internal structures without iso-surface fields.
- Mesh-Voxel Transformations - Transforms textured surface meshes into sparse volumetric grids and back without using optimization or rendering.
- Generative Model Training Tools - Provides capabilities for training generative 3D models from scratch or fine-tuning them on custom datasets.
- 3D Asset Exporters - Implements capabilities to convert internal 3D representations into industry-standard GLB files while preserving PBR materials.
- 3D Asset Pipelines - Provides a pipeline to convert generated 3D representations into industry-standard GLB files for professional software.
- Multi-Channel PBR Mapping - Generates independent maps for base color, metallic, and roughness to support industry-standard PBR material workflows.
