xxlong0/Wonder3D
Wonder3D
Wonder3D ist ein diffusionsbasiertes System für die 3D-Rekonstruktion aus einem einzelnen Bild. Es generiert hochdetaillierte 3D-Meshes aus einem einzelnen Eingabebild, indem es konsistente Multi-View-Normal-Maps und Farbbilder erzeugt.
Die Pipeline fungiert als Multi-View-Normal-Map-Generator und Extraktor für texturierte Meshes. Sie nutzt domänenübergreifende Multi-View-Synthese zur Erstellung sichtabhängiger Maps, die anschließend durch Radiance-Fusion und speichereffiziente Oberflächenrekonstruktion in 3D-Geometrie umgewandelt werden.
Das Projekt deckt 3D-Mesh-Generierung, Multi-View-Generierung und texturierte 3D-Modellierung ab. Es enthält zudem Funktionen zum Training von Diffusionsmodellen, um die Konsistenz der generierten sichtabhängigen Maps zu optimieren.
Features
- Single-Image 3D Reconstructions - Generates high-detail textured 3D meshes from a single input image using multi-view normal maps.
- Diffusion-Based 3D Generators - Implements a diffusion-based system to generate consistent multi-view normal and color maps for 3D asset creation.
- Multi-View Image Generators - Creates consistent multi-perspective images from a single source to facilitate 3D reconstruction.
- Surface Reconstruction - Extracts smooth 3D surfaces from radiance fields using a memory-efficient reconstruction method.
- 3D Mesh Generators - Programmatically creates 3D polygonal meshes with surfaces and textures from image data.
- Volumetric Mesh Extractors - Converts generated multi-view imagery into 3D meshes using radiance fusion and efficient reconstruction.
- Volumetric Mesh Extraction - Converts generated multi-view normals and colors into 3D meshes using high-quality normal fusion.
- Radiance-Based Mesh Conversion - Translates radiance and normal information into high-quality textured 3D geometry via fusion.
- View-Dependent Map Synthesis - Produces coordinated color and normal maps that maintain the identity of the original input image.
- Normal Map Extractions - Extracts surface orientation data from generated normal maps to build a coherent 3D mesh.
- Multi-View Consistency Attention - Uses shared attention layers to ensure spatial and geometric consistency across generated viewpoints.
- Diffusion Model Training - Provides workflows to optimize attention and cross-domain modules for improving view-dependent map consistency.
- Memory-Efficient - Provides a memory-optimized pipeline to generate smooth 3D meshes from multi-view data.
- Image-to-3D Texture Engines - Transfers detailed surface textures and colors from a single image onto a reconstructed 3D model.
