76 lines
1.5 KiB
Markdown
76 lines
1.5 KiB
Markdown
# How to run
|
|
|
|
## DreamGaussion & MVDream
|
|
|
|
Please install PyTorch & Nvidia CUDA toolkit first.
|
|
|
|
I run my code on PyTorch 2.1.0 with 11.8 CUDA.
|
|
|
|
|
|
And run these code to build the environment to generate 3D Pokemon models.
|
|
```
|
|
git clone https://github.com/dreamgaussian/dreamgaussian.git
|
|
|
|
cd dreamgaussian
|
|
|
|
# a modified gaussian splatting (+ depth, alpha rendering)
|
|
git clone --recursive https://github.com/ashawkey/diff-gaussian-rasterization
|
|
pip install ./diff-gaussian-rasterization
|
|
|
|
# simple-knn
|
|
pip install ./simple-knn
|
|
|
|
# nvdiffrast
|
|
pip install git+https://github.com/NVlabs/nvdiffrast/
|
|
|
|
# kiuikit
|
|
pip install git+https://github.com/ashawkey/kiuikit
|
|
|
|
# To use MVdream, also install:
|
|
pip install git+https://github.com/bytedance/MVDream
|
|
```
|
|
|
|
## Generate 3D Pokemon models
|
|
|
|
After building the environment.
|
|
|
|
We provide a script to generate multi type & multi animal type's Pokemon models.
|
|
|
|
```
|
|
cd ../
|
|
python generate.py
|
|
```
|
|
|
|
It will generate many 3D pokemon models in `output/` directory.
|
|
|
|
And plese run the script to transform the mesh model to cloud points.
|
|
|
|
```
|
|
pip install -r requirements
|
|
python mesh_to_points.py
|
|
```
|
|
|
|
## Camera Calibration
|
|
|
|
You can find the chess pattern provided by Opencv https://github.com/opencv/opencv/blob/3.4/doc/pattern.png
|
|
|
|
Please record a video and run calibration scripts
|
|
|
|
```
|
|
python camera_calibration.py my_video.py
|
|
```
|
|
|
|
it will save intristic matrix & distortion matrix.
|
|
|
|
## Visualize
|
|
|
|
Then you can link your webcamera. and run this script to show your Pokemon.
|
|
|
|
```
|
|
python main.py
|
|
```
|
|
|
|
Here is ourt DEMO https://youtu.be/nOnE82SKFAM
|
|
|
|
|