SelectionConv: Convolutional Neural Networks for Non-rectilinear Image Data
Convolutional Neural Networks have revolutionized vision applications. There are image domains and representations, however, that cannot be handled by standard CNNs (e.g., spherical images, superpixels). Such data are usually processed using networks and algorithms specialized for each type. In this work, we show that it may not always be necessary to use specialized neural networks to operate on such spaces. Instead, we introduce a new structured graph convolution operator that can copy 2D convolution weights, transferring the capabilities of already trained traditional CNNs to our new graph network. This network can then operate on any data that can be represented as a positional graph. By converting non-rectilinear data to a graph, we can apply these convolutions on these irregular image domains without requiring training on large domain-specific datasets. Results of transferring pre-trained image networks for segmentation, stylization, and depth prediction are demonstrated for a variety of such data forms.
Code (1)
Tasks
Depth EstimationDepth PredictionSuperpixelsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Interpolated SelectionConv for Spherical Images and Surfaces
We present a new and general framework for convolutional neural network operations on spherical (or omnidirectional) images. Our approach represents the surface as a graph of connected points that doesn't rely on a parti…
Semantic SegmentationStyle Transfer3D Object Detection from a Single Fisheye Image Without a Single Fisheye Training Image
Existing monocular 3D object detection methods have been demonstrated on rectilinear perspective images and fail in images with alternative projections such as those acquired by fisheye cameras. Previous works on object …
2D Object Detection3D Object DetectionMonocular 3D Object DetectionObject+2On Retrospective k-space Subsampling schemes For Deep MRI Reconstruction
Acquiring fully-sampled MRI $k$-space data is time-consuming, and collecting accelerated data can reduce the acquisition time. Employing 2D Cartesian-rectilinear subsampling schemes is a conventional approach for acceler…
MRI ReconstructionArtwork painting identification method for panorama based on adaptive rectilinear projection and optimized ASIFT
In the paper, the authors present an artwork painting identification method for panorama based on adaptive rectilinear projection and optimized ASIFT (Affine Scale-Invariant Feature Transform). Firstly, the authors use t…
Image EnhancementHEAL-ViT: Vision Transformers on a spherical mesh for medium-range weather forecasting
In recent years, a variety of ML architectures and techniques have seen success in producing skillful medium range weather forecasts. In particular, Vision Transformer (ViT)-based models (e.g. Pangu-Weather, FuXi) have s…
Weather Forecasting