Skip to content

Latest commit

 

History

History
65 lines (49 loc) · 3.88 KB

plenoxel.md

File metadata and controls

65 lines (49 loc) · 3.88 KB

Plenoxel

Plenoxel has shown dramatic improvements in training time by directly optimizing sparse voxel grids. The primary version has been released in Link, nevertheless, there are several excessive parts that are not used in the code, resulting in difficulties when modifying the code. Here, we reorganized the code so that the proposed model can be conveninently compared with other baselines.

Properties

✅ Single GPU only!

✅ nearly 30mins for training

Details

We found several interesting implementation details in the official code, which are not explained in the paper.

  • For blender scenes, the scale of scene is reduced to 2/3.
  • There are two strategies to resample the sparse voxel grids: "sigma" and "weights". Refer to the code Link and Link for details.
  • We have observed that the data loading phase of tanks_and_temples dataset is slightly different from NeRF++. In detail, they have changed the scale of scenes by 0.95 after normalizing the poses. This results in a slightly better performance in all scenes.
  • When evaluating on the tanks_ant_temples dataset, we found a strong dependency of the final performance on the seed.

Setup

Before running the code, users should install an additional CUDA package.

export LD_LIBRARY_PATH=[path to environment]/lib/python[version]/site-packages/torch/lib/:$LD_LIBRARY_PATH
# Example: export LD_LIBRARY_PATH=/home/abcd/anaconda3/envs/env_name/lib/python3.7/site-packages/torch/lib/:$LD_LIBRARY_PATH
pip install .

Since the CUDA implementation only supports 32bits training, it is strongly recommended to train with 32 bits; several functions could cause side-effects.

Reproduced Performance

Due to limited storage of our Google Cloud, we do not provide the pretrained model for the plenoxel model(< 7GB each). Instead, we provide scripts in scripts/plenoxel_torch, which can completely reproduce the performance reported below in 30 minutes.

Blender

Chair Drums Ficus Hotdog Lego Materials Mic Ship
PSNR (Test) 34.11 25.36 32.10 36.67 34.43 29.18 33.32 29.70
SSIM (Test) 0.9768 0.9329 0.9766 0.9808 0.9755 0.9494 0.9849 0.8891
LPIPS (Test) 0.02968 0.06453 0.02465 0.03343 0.02666 0.05490 0.01363 0.1339
PSNR (All) 35.40 27.66 34.10 38.66 37.38 32.76 34.74 32.47
SSIM (All) 0.9820 0.9524 0.9854 0.9867 0.9851 0.9737 0.9886 0.9232
LPIPS (All) 0.02311 0.05495 0.01749 0.02429 0.01802 0.03649 0.010930 0.1077

LLFF (NeRF-Real)

Fern Flower Fortress Horns Leaves Orchids Room Trex
PSNR (Test) 25.55 27.85 31.19 27.56 21.45 20.46 30.54 26.41
SSIM (Test) 0.8325 0.8649 0.8842 0.8545 0.7607 0.6815 0.9409 0.8885
LPIPS (Test) 0.2195 0.1781 0.1802 0.2328 0.1952 0.2664 0.1860 0.2382
PSNR (All) 27.81 30.96 32.48 28.74 23.94 24.19 33.82 28.65
SSIM (All) 0.8811 0.9129 0.9000 0.8723 0.8341 0.7929 0.9574 0.9222
LPIPS (All) 0.1817 0.1328 0.1655 0.2144 0.1556 0.2111 0.1628 0.1900

Tanks and Temples

M60 Train Truck Playground
PSNR (Test) 17.94 17.63 22.26 22.13
SSIM (Test) 0.6759 0.6132 0.7390 0.6900
LPIPS (Test) 0.4449 0.4417 0.3745 0.4422
PSNR (All) 25.71 22.41 24.01 25.02
SSIM (All) 0.8228 0.7115 0.7817 0.7283
LPIPS (All) 0.2927 0.3656 0.3112 0.3824