Task-Oriented Compact Representation of 3D Point Clouds via A Matrix Optimization-Driven Network

Yue QIAN, Junhui HOU, Qijian ZHANG, Yiming ZENG, Sam KWONG, Ying HE

Research output: Journal PublicationsJournal Article (refereed)peer-review


This paper explores the task-oriented compact representation of 3D point clouds, which should maintain the performance of subsequent applications applied to such compact point clouds as much as possible. Designing from the perspective of matrix optimization, we propose MOPS-Net, a novel deep learning-based method that is distinguishable from existing approaches due to its interpretability and flexibility. The matrix optimization problem is challenging due to the discrete and combinatorial nature of the sampling matrix. Therefore, we tackle the challenges by relaxing the binary constraint of the sampling matrix and formulating a constrained and differentiable optimization problem. We then design a deep neural network to mimic the matrix optimization by exploring both the local and global structures of the input data. MOPS-Net can be end-to-end trained with a task network and is permutation-invariant, making it robust to the input. We also extend MOPS-Net such that a single network after one-time training is capable of handling arbitrary downsampling ratios. Extensive experimental results show that MOPS-Net can achieve favorable performance against state-of-the-art deep learning-based methods over various tasks, including classification, reconstruction, and registration. Besides, we validate the robustness of MOPS-Net on noisy data.
Original languageEnglish
JournalIEEE Transactions on Circuits and Systems for Video Technology
Publication statusE-pub ahead of print - 25 Apr 2023
Externally publishedYes

Bibliographical note

Publisher Copyright:


  • Classification
  • Deep learning
  • Feature extraction
  • Image coding
  • Optimization
  • Point cloud
  • Point cloud compression
  • Reconstruction
  • Registration
  • Sampling
  • Task analysis
  • Three-dimensional displays


Dive into the research topics of 'Task-Oriented Compact Representation of 3D Point Clouds via A Matrix Optimization-Driven Network'. Together they form a unique fingerprint.

Cite this