3DTA: No-Reference 3D Point Cloud Quality Assessment with Twin Attention

  • Linxia Zhu
  • , Jun Cheng
  • , Xu Wang
  • , Honglei Su* (Corresponding Author)
  • , Huan Yang
  • , Hui Yuan
  • , Jari Korhonen
  • *Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Point clouds are rapidly gaining popularity in many practical applications, and point cloud quality assessment (PCQA) is an important research topic that helps us measure and improve the visual experience in applications using point clouds. Research on full-reference (FR) PCQAs has recently made impressive progress, and research on no-reference (NR) PCQAs has also gradually increased. However, the performance of the prior NR PCQA methods still suffers from weak generalization ability and lower accuracy than the FR metrics in general. In this work, we propose a two-stage sampling method that can reasonably represent a whole point cloud, making it possible to efficiently calculate the point cloud quality. For quality prediction, we designed a twin-attention-based transformer PCQA model (3DTA), which uses the data of the two-stage sampling method as input and directly outputs the predicted quality score. Our model is accurate and widely applicable, and it has a simple and flexible structure. Experimental results show that in most cases, the proposed 3DTA model substantially outperforms the benchmark NR methods. The accuracy of the proposed method is competitive even against that of the FR method, which makes 3DTA a strong candidate for the PCQA task, regardless of the reference availability. The code of the proposed model is publicly available at <uri>https://github.com/philox12358/3DTA-PCQA</uri>.

Original languageEnglish
Pages (from-to)10489 - 10502
Number of pages14
JournalIEEE Transactions on Multimedia
Volume26
Early online date30 May 2024
DOIs
Publication statusPublished - 14 Nov 2024

Funding

This work was supported in part by the National Natural Science Foundation of China under Grants (62222110 and 62172259), in part by the Shandong Provincial Natural Science Foundation, China, under Grants (ZR2022MF275,ZR2022ZD38, ZR2021MF025, and ZR2022QF076), and in part by the Taishan Scholar Project of Shandong Province (tsqn202103001) and the Major Scientific and Technological Innovation Project of Shandong Province under Grant 2020CXGC010109.

FundersFunder number
National Natural Science Foundation of China62222110, 62172259
Shandong Provincial Natural Science FoundationZR2022MF275, ZR2022ZD38, ZR2021MF025, ZR2022QF076

    Keywords

    • deep learning
    • Feature extraction
    • Measurement
    • Point cloud compression
    • point cloud quality assessment
    • point convolution
    • Predictive models
    • Quality assessment
    • Task analysis
    • Three-dimensional displays

    Fingerprint

    Dive into the research topics of '3DTA: No-Reference 3D Point Cloud Quality Assessment with Twin Attention'. Together they form a unique fingerprint.

    Cite this