Mapped Convolutions
We present a versatile formulation of the convolution operation that we term a "mapped convolution." The standard convolution operation implicitly samples the pixel grid and computes a weighted sum. Our mapped convolution decouples these two components, freeing the operation from the confines of the image grid and allowing the kernel to process any type of structured data. As a test case, we demonstrate its use by applying it to dense inference on spherical data. We perform an in-depth study of existing spherical image convolution methods and propose an improved sampling method for equirectangular images. Then, we discuss the impact of data discretization when deriving a sampling function, highlighting drawbacks of the cube map representation for spherical data. Finally, we illustrate how mapped convolutions enable us to convolve directly on a mesh by projecting the spherical image onto a geodesic grid and training on the textured mesh. This method exceeds the state of the art for spherical depth estimation by nearly 17%. Our findings suggest that mapped convolutions can be instrumental in expanding the application scope of convolutional neural networks.
Code (1)
Tasks
Depth EstimationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
DualConvMesh-Net: Joint Geodesic and Euclidean Convolutions on 3D Meshes
We propose DualConvMesh-Nets (DCM-Net) a family of deep hierarchical convolutional networks over 3D geometric data that combines two types of convolutions. The first type, geodesic convolutions, defines the kernel weight…
3D Semantic SegmentationScene SegmentationSemantic SegmentationEcoFlow: Efficient Convolutional Dataflows for Low-Power Neural Network Accelerators
Dilated and transposed convolutions are widely used in modern convolutional neural networks (CNNs). These kernels are used extensively during CNN training and inference of applications such as image segmentation and high…
Generative Adversarial NetworkImage GenerationImage SegmentationSemantic SegmentationFast Discrete Fourier Transform algorithms requiring less than 0(NlogN) multiplications
In the paper it is shown that there exist infinite classes of fast DFT algorithms having multiplicative complexity lower than O(NlogN), i.e. smaller than their arithmetical complexity. The derivation starts with nesting …
End-to-End Learned Block-Based Image Compression with Block-Level Masked Convolutions and Asymptotic Closed Loop Training
Learned image compression research has achieved state-of-the-art compression performance with auto-encoder based neural network architectures, where the image is mapped via convolutional neural networks (CNN) into a late…
Image CompressionVideo CompressionGrassmann Graph Embedding
Geometric deep learning that employs the geometric and topological features of data has attracted increasing attention in deep neural networks. Learning the intrinsic structure property of data is a crucial step for dime…
Dimensionality ReductionGraph Embedding