Skip to content

The official PyTorch implementation of Consistent3D (CVPR 2024)

License

Notifications You must be signed in to change notification settings

sail-sg/Consistent3D

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

6 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Consistent3D: Towards Consistent High-Fidelity Text-to-3D Generation with Deterministic Sampling Prior (CVPR 2024)

results

This is an official PyTorch implementation of Consistent3D. See the paper here. If you find our Consistent3D helpful or heuristic to your projects, please cite this paper and also star this repository. Thanks!

@misc{wu2024consistent3d,
      title={Consistent3D: Towards Consistent High-Fidelity Text-to-3D Generation with Deterministic Sampling Prior}, 
      author={Zike Wu and Pan Zhou and Xuanyu Yi and Xiaoding Yuan and Hanwang Zhang},
      year={2024},
      eprint={2401.09050},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Acknowledgement: This repo is based on the following amazing project: threestudio.

Requirements

All experiments were conducted using PyTorch 1.13.0, CUDA 11.7.1, and CuDNN 8.5.0. We strongly recommend to use the provided Dockerfile to build an image to reproduce our experiments. Please check out threestudio repository for alternative installation.

Quickstart

# --------- Coarse NeRF Optimization Stage --------- #
python launch.py --config configs/consistency-coarse.yaml --train --gpu 0 \
system.prompt_processor.prompt="a delicious hamburger"

# ------------- Mesh Refinement Stage -------------- #
# Geometry Refinement (Optional)
python launch.py --config configs/consistency-refine.yaml --train --gpu 0 \
system.prompt_processor.prompt="a delicious hamburger" \
system.geometry_convert_from=path/to/coarse/dir/ckpts/last.ckpt

# Texture Refinement
python launch.py --config configs/consistency-texture.yaml --train --gpu 0 \
system.prompt_processor.prompt="a delicious hamburger" \
system.geometry_convert_from=path/to/refine/dir/ckpts/last.ckpt

Export Mesh

Given the generated results, we can further export the corresponding mesh.

# this uses default mesh-exporter configurations which exports obj+mtl
python launch.py --config path/to/texture/dir/configs/parsed.yaml --export --gpu 0 resume=path/to/texture/dir/ckpts/last.ckpt system.exporter_type=mesh-exporter
# specify system.exporter.fmt=obj to get obj with vertex colors
# you may also add system.exporter.save_uv=false to accelerate the process, suitable for a quick peek of the result
python launch.py --config path/to/texture/dir/configs/parsed.yaml --export --gpu 0 resume=path/to/texture/dir/ckpts/last.ckpt system.exporter_type=mesh-exporter system.exporter.fmt=obj

Tips for Improving Quality

Here are some tips that may help you improve the generation quality:

  • Increase batch size. Large batch sizes help convergence and improve the 3D consistency of the geometry.
  • Train longer. This helps if you can already obtain reasonable results and would like to enhance the details. See examples in configs with -long suffix.
  • Try different seeds. Try different seed by setting seed=N when you suffer from the Janus face problem.

Please check out threestudio repository for more tips.

About

The official PyTorch implementation of Consistent3D (CVPR 2024)

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages