Develop Tools & CodePaper and LLMs

Learned Point Cloud Compression for Classification

Our codec demonstrates the potential of specialized codecs for machine analysis of point clouds, and provides a basis for extension to more complex tasks and datasets ...

Tags:

Pricing Type

  • Pricing Type: Free
  • Price Range Start($):

GitHub Link

The GitHub link is https://github.com/multimedialabsfu/learned-point-cloud-compression-for-classification

Introduce

Researchers from Simon Fraser University have developed a specialized point cloud codec aimed at efficient machine vision tasks like classification. The codec, built upon PointNet, offers a remarkable trade-off between compression and accuracy, achieving a 94% reduction in BD-bitrate compared to non-specialized codecs on the ModelNet40 dataset. Two lightweight encoder configurations deliver similar BD-bitrate reductions of 93% and 92%, with minor accuracy drops of 3% and 5%, respectively. The codec showcases the potential of tailored codecs for point cloud analysis, suggesting prospects for complex tasks and datasets in the future. The project provides code, instructions for installation, dataset handling, training, and evaluation procedures for the codec’s application.

Our codec demonstrates the potential of specialized codecs for machine analysis of point clouds, and provides a basis for extension to more complex tasks and datasets in the future.

Content

Abstract: Deep learning is increasingly being used to perform machine vision tasks such as classification, object detection, and segmentation on 3D point cloud data. However, deep learning inference is computationally expensive. The limited computational capabilities of end devices thus necessitate a codec for transmitting point cloud data over the network for server-side processing. Such a codec must be lightweight and capable of achieving high compression ratios without sacrificing accuracy. Motivated by this, we present a novel point cloud codec that is highly specialized for the machine task of classification. Our codec, based on PointNet, achieves a significantly better rate-accuracy trade-off in comparison to alternative methods. In particular, it achieves a 94% reduction in BD-bitrate over non-specialized codecs on the ModelNet40 dataset. For low-resource end devices, we also propose two lightweight configurations of our encoder that achieve similar BD-bitrate reductions of 93% and 92% with 3% and 5% drops in top-1 accuracy, while consuming only 0.470 and 0.048 encoder-side kMACs/point, respectively. Our codec demonstrates the potential of specialized codecs for machine analysis of point clouds, and provides a basis for extension to more complex tasks and datasets in the future. Use either poetry or virtualenv to install python dependencies. Download and repair the ModelNet40 dataset (it has incorrect OFF headers!) by running: During training, the torch_geometric.datasets.modelnet.ModelNet dataset class also downloads and processes its own format of the ModelNet40 dataset into the directory ${paths.datasets}/modelnet/dataset=modelnet40,format=pt. For compatibility with our scripts, we recommend that you also follow the same directory structure. [OPTIONAL] For evaluating input compression codecs: We use CompressAI Trainer for training. See the walkthrough for more details. To train a single-task classification compression model, use: To train a multi-task input reconstruction compression model, use: To train vanilla PointNet (i.e. no compression), use: Please see scripts/run.sh for more examples, including how to train different codec configurations/layer sizes. CompressAI Trainer automatically evaluates models during training. To see the Aim experiment tracker/dashboard, please follow these instructions. To save the results to JSON, modify scripts/save_json_from_aim_query.py to query your specific models. (By default, it is currently set up to generate all the trained codec curves shown in our paper.) Then, run: In our paper, we also evaluated “input compression codec” performance. To reproduce these results, please follow the procedure outlined in the paper, and make use of the following scripts: NOTE: Please modify RUN_HASHES to point to your own trained models.


Learned Point Cloud Compression for Classification

Related