Gated Transformer Representing Region Importance for Image Quality Assessment

Deep neural networks, particularly convolutional neural networks (CNNs), have shown significant promise in image quality assessment (IQA), yet the underlying workings of these models in IQA remain partially unexplored. This study unveils a novel positionally masked transformer, shedding light on how various regions of an image influence its overall quality. Surprisingly, the findings reveal that half of an image may exert only a marginal influence on image quality, while the remaining half proves vital. This observation has been extended to other CNN-based IQA models, unearthing a consistent pattern where specific image regions significantly shape overall quality. In a stride to understand these phenomena, three semantic measures: saliency, frequency, and objectness, have been identified, exhibiting a strong correlation with the importance of image regions in IQA. Building upon these insights, a new gated operation has been proposed, representing the fluctuating significance of regions in image quality. A gate, integrable into a transformer encoder for IQA, serves to pinpoint the crucial spatial regions, enhancing their impact by amplifying attention weights. The resulting gated transformer has been rigorously tested on publicly available IQA datasets, demonstrating exceptional performance and reinforcing the innovative nature of this approach. The success of this study paves the way for more intricate and insightful analyses of IQA.