TripoSG

TripoSG: High-Fidelity 3D Shape Synthesis using Large-Scale Rectified Flow Models

By Tripo

TripoSG is an advanced high-fidelity, high-quality and high-generalizability image-to-3D generation foundation model. It leverages large-scale rectified flow transformers, hybrid supervised training, and a high-quality dataset to achieve state-of-the-art performance in 3D shape generation.

✨ Key Features

- High-Fidelity Generation: Produces meshes with sharp geometric features, fine surface details, and complex structures
- Semantic Consistency: Generated shapes accurately reflect input image semantics and appearance
- Strong Generalization: Handles diverse input styles including photorealistic images, cartoons, and sketches
- Robust Performance: Creates coherent shapes even for challenging inputs with complex topology

🔬 Technical Highlights

- Large-Scale Rectified Flow Transformer: Combines RF's linear trajectory modeling with transformer architecture for stable, efficient training
- Advanced VAE Architecture: Uses Signed Distance Functions (SDFs) with hybrid supervision combining SDF loss, surface normal guidance, and eikonal loss
- High-Quality Dataset: Trained on 2 million meticulously curated Image-SDF pairs, ensuring superior output quality
- Efficient Scaling: Implements architecture optimizations for high performance even at smaller model scales

🔥 Updates

* [2025-04] Release TripoSG-scribble, a CFG-distilled, 512 token model for fast shape prototyping from scribble+prompt! Try the online demo here.
* [2025-03] Release of TripoSG 1.5B parameter rectified flow model and VAE trained on 2048 latent tokens, along with inference code and interactive demo

🔨 Installation

Clone the repo:
bash
git clone https://github.com/VAST-AI-Research/TripoSG.git
cd TripoSG

Create a conda environment (optional):
bash
conda create -n tripoSG python=3.10
conda activate tripoSG

Install dependencies:
bash
pytorch (select correct CUDA version)
pip install torch torchvision --index-url https://download.pytorch.org/whl/{your-cuda-version}

other dependencies
pip install -r requirements.txt

💡 Quick Start

Generate a 3D mesh from an image:
bash
python -m scripts.inference_triposg --image-input assets/example_data/hjswed.png --output-path ./output.glb

Limiting the number of faces:
bash
python -m scripts.inference_triposg --image-input assets/example_data/hjswed.png --faces 5000 --output-path ./output.glb

or from scribble+prompt:
bash
 python -m scripts.inference_triposg_scribble --image-input assets/example_scribble_data/cat_with_wings.png --prompt "a cat with wings" --scribble-conf 0.3 --output-path output.glb

The required model weights will be automatically downloaded:
- TripoSG (image condition) model from VAST-AI/TripoSG → pretrained_weights/TripoSG
= TripoSG-scribble (scribble+prompt condition) model from VAST-AI/TripoSG-scribble → pretrained_weights/TripoSG-scribble
- RMBG model from briaai/RMBG-1.4 → pretrained_weights/RMBG-1.4

💻 System Requirements

- CUDA-enabled GPU with at least 8GB VRAM

📝 Tips

- If you want to use the full VAE module (including the encoder part), you need to uncomment the Line-15 in triposg/models/autoencoders/autoencoder_kl

……（后略，见原文）