[MMDet3D] How to install

What is MMDet3D?

While PyTorch is unarguably one of the ground-winning libraries in the field of deep learning, it does not provide built-in 3D perception models by nature. And therefore, mostly it is up to researchers to implement all modules(pre- and post-processings, anchors, etc.) of complicated 3D perception pipeline. Clearly, it is not impossible to implement on their own, but it is time-consuming job and could easily distract them from the core job, which is to find research novelties. See the exemplary perception module below and imagine when one needs to code individual components of the model themselves.

Architecture overview of NeRF-Det

MMDetection3D (MMDet3D) is an open-source toolbox, specifically designed to supply built-in models for 3D object detection tasks. It provides a collection of SOTA algorithms and models so that researchers could purely focus on what they are interested in, instead of trouble shooting through individual components of 3D perception system.

One can find the list of provided models in the link.

How to Install MMDet3D?

The general installation guide is well explained in their official website.

However the detail of the instruction for individual systems could vary.

Therefore, I will take this chance to describe how I installed MMDet3D v1.4.0 in my system(Ubuntu20.04, GTX3090).

The following is the command lines I used based on conda environment named openmmlab with python3.8.

# build virtual environment of python3.8 with name openmmlab using conda
$conda create --name openmmlab python=3.8 -y
$conda activate openmmlab

# install torch-relavant libraries
$conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.3 -c pytorch

# use the package manager mim to install openmmlab-relavant libraries with specific versions.
$pip install -U openmim
$mim install mmengine==0.10.3
$mim install mmcv==2.1.0
$mim install mmdet==3.3.0

# install mmdetection3d ver 1.4.0
$git clone https://github.com/open-mmlab/mmdetection3d.git -b dev-1.4.0
$cd mmdetection3d
$pip install -v -e .
# "-v" means verbose, or more output
# "-e" means installing a project in edtiable mode,
# thus any local modifications made to the code will take effect without reinstallation.

# install additional libraries required to run particular models with sparse convolution.
$pip install cumm-cu113
$pip install spconv-cu113

Note that I specified cuda version when isntalling cumm and spconv, i.e., cumm-cu113. This is because those libraries require you to specify which CUDA version the virtual environment is running on. Mine was 11.3 as cudatoolkit=11.3, and thus, I gave the command pip install cumm-cu113.

Once you complete the installation instructions above, your virtual environment must have libraries installed as below.

conda activate openmmlab
pip list | grep mm

How to Ensure MMDet3D is Successfully Installed?

To confirm it is well installed, MMDet3D provides some demo scripts.

By conducting inference on a sample data with pre-trained 3D detector, one can verify the installation of MMDet3D.

Assuming you installed MMDet3D at your project ROOT, first we download pre-trained weights for a 3D detector, PointPillar.

mim download mmdet3d -config pointpillars_hv_secfpn_8xb6–160e_kitti-3d-car  -dest .

This command should get the config file and pre-trained weight for PointPillar installed at the current working directory.

• config: pointpillars_hv_secfpn_8xb6-160e_kitti-3d-car.py
• weight: hv_pointpillars_secfpn_6x8_160e_kitti-3d-car_20220331_134606-d42d15ed.pth
• If you want to download it at another directory, simply give the path of the directory to the argument, —dest.

Once downloaded, we are ready to conduct the demo to run PointPillar on a sample data by the command below.

cd mmdetection3d
python demo/pcd_demo.py demo/data/kitti/000008.bin pointpillars_hv_secfpn_8xb6–160e_kitti-3d-car.py hv_pointpillars_secfpn_6x8_160e_kitti-3d-car_20220331_134606-d42d15ed.pth --show

If all aforementioned requirements were fulfilled, this should plot the point cloud of the given sample data along with 3D bounding boxes predicted by PointPillar, as shown below.

Reference

[1] MMDet3D