Geometry-based spherical JND modeling for 360$^\circ$ display
360$^\circ$ videos have received widespread attention due to its realistic and immersive experiences for users. To date, how to accurately model the user perceptions on 360$^\circ$ display is still a challenging issue. In this paper, we exploit the visual characteristics of 360$^\circ$ projection and display and extend the popular just noticeable difference (JND) model to spherical JND (SJND). First, we propose a quantitative 2D-JND model by jointly considering spatial contrast sensitivity, luminance adaptation and texture masking effect. In particular, our model introduces an entropy-based region classification and utilizes different parameters for different types of regions for better modeling performance. Second, we extend our 2D-JND model to SJND by jointly exploiting latitude projection and field of view during 360$^\circ$ display. With this operation, SJND reflects both the characteristics of human vision system and the 360$^\circ$ display. Third, our SJND model is more consistent with user perceptions during subjective test and also shows more tolerance in distortions with fewer bit rates during 360$^\circ$ video compression. To further examine the effectiveness of our SJND model, we embed it in Versatile Video Coding (VVC) compression. Compared with the state-of-the-arts, our SJND-VVC framework significantly reduced the bit rate with negligible loss in visual quality.
Code (0)
등록된 구현이 없습니다.
Tasks
Video CompressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
SGAT4PASS: Spherical Geometry-Aware Transformer for PAnoramic Semantic Segmentation
As an important and challenging problem in computer vision, PAnoramic Semantic Segmentation (PASS) gives complete scene perception based on an ultra-wide angle of view. Usually, prevalent PASS methods with 2D panoramic i…
Semantic SegmentationLightweight and Robust Multi-Channel End-to-End Speech Recognition with Spherical Harmonic Transform
This paper presents SHTNet, a lightweight spherical harmonic transform (SHT) based framework, which is designed to address cross-array generalization challenges in multi-channel automatic speech recognition (ASR) through…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)channel selectionspeech-recognition+1Pano-AVQA: Grounded Audio-Visual Question Answering on 360$^\circ$ Videos
360$^\circ$ videos convey holistic views for the surroundings of a scene. It provides audio-visual cues beyond pre-determined normal field of views and displays distinctive spatial relations on a sphere. However, previou…
Audio-visual Question AnsweringQuestion AnsweringRelationVisual Question Answering+1PanoGRF: Generalizable Spherical Radiance Fields for Wide-baseline Panoramas
Achieving an immersive experience enabling users to explore virtual environments with six degrees of freedom (6DoF) is essential for various applications such as virtual reality (VR). Wide-baseline panoramas are commonly…
Depth Estimation3D Scene Geometry Estimation from 360$^\circ$ Imagery: A Survey
This paper provides a comprehensive survey on pioneer and state-of-the-art 3D scene geometry estimation methodologies based on single, two, or multiple images captured under the omnidirectional optics. We first revisit t…
Simultaneous Localization and MappingStereo MatchingSurvey