Task-Oriented Compact Representation of 3D Point Clouds via A Matrix Optimization-Driven Network

Yue QIAN, Junhui HOU, Qijian ZHANG, Yiming ZENG, Sam KWONG, Ying HE

Research output: Journal PublicationsJournal Article (refereed)peer-review


This paper explores the task-oriented compact representation of 3D point clouds, which should maintain the performance of subsequent applications applied to such compact point clouds as much as possible. Designing from the perspective of matrix optimization, we propose MOPS-Net, a novel deep learning-based method that is distinguishable from existing approaches due to its interpretability and flexibility. The matrix optimization problem is challenging due to the discrete and combinatorial nature of the sampling matrix. Therefore, we tackle the challenges by relaxing the binary constraint of the sampling matrix and formulating a constrained and differentiable optimization problem. We then design a deep neural network to mimic the matrix optimization by exploring both the local and global structures of the input data. MOPS-Net can be end-to-end trained with a task network and is permutation-invariant, making it robust to the input. We also extend MOPS-Net such that a single network after one-time training is capable of handling arbitrary downsampling ratios. Extensive experimental results show that MOPS-Net can achieve favorable performance against state-of-the-art deep learning-based methods over various tasks, including classification, reconstruction, and registration. Besides, we validate the robustness of MOPS-Net on noisy data.
Original languageEnglish
Pages (from-to)6981-6995
Number of pages15
JournalIEEE Transactions on Circuits and Systems for Video Technology
Issue number11
Early online date25 Apr 2023
Publication statusPublished - 1 Nov 2023
Externally publishedYes

Bibliographical note

Publisher Copyright:
© 2023 IEEE.


  • Point cloud
  • classification
  • deep learning
  • optimization
  • reconstruction
  • registration
  • sampling


Dive into the research topics of 'Task-Oriented Compact Representation of 3D Point Clouds via A Matrix Optimization-Driven Network'. Together they form a unique fingerprint.

Cite this