paper-with-me

Papers

Sphere2Vec: A General-Purpose Location Representation Learning over a Spherical Surface for Large-Scale Geospatial Predictions

2023-06-30 · Gengchen Mai, Yao Xuan, Wenyun Zuo, Yutong He, Jiaming Song, Stefano Ermon, Krzysztof Janowicz, Ni Lao

Generating learning-friendly representations for points in space is a fundamental and long-standing problem in ML. Recently, multi-scale encoding schemes (such as Space2Vec and NeRF) were proposed to directly encode any point in 2D/3D Euclidean space as a high-dimensional vector, and has been successfully applied to various geospatial prediction and generative tasks. However, all current 2D and 3D location encoders are designed to model point distances in Euclidean space. So when applied to large-scale real-world GPS coordinate datasets, which require distance metric learning on the spherical surface, both types of models can fail due to the map projection distortion problem (2D) and the spherical-to-Euclidean distance approximation error (3D). To solve these problems, we propose a multi-scale location encoder called Sphere2Vec which can preserve spherical distances when encoding point coordinates on a spherical surface. We developed a unified view of distance-reserving encoding on spheres based on the DFS. We also provide theoretical proof that the Sphere2Vec preserves the spherical surface distance between any two points, while existing encoding schemes do not. Experiments on 20 synthetic datasets show that Sphere2Vec can outperform all baseline models on all these datasets with up to 30.8% error rate reduction. We then apply Sphere2Vec to three geo-aware image classification tasks - fine-grained species recognition, Flickr image recognition, and remote sensing image classification. Results on 7 real-world datasets show the superiority of Sphere2Vec over multiple location encoders on all three tasks. Further analysis shows that Sphere2Vec outperforms other location encoder models, especially in the polar regions and data-sparse areas because of its nature for spherical surface distance preservation. Code and data are available at https://gengchenmai.github.io/sphere2vec-website/.

📄 PDF Abstract BibTeX arXiv:2306.17624

Code (1)

gengchenmai/sphere2vec pytorch

Tasks

image-classificationImage ClassificationMetric LearningNeRFRemote Sensing Image ClassificationRepresentation Learning

Methods 이 논문이 사용한 방법론

fail 설명 없음
GPS Greedy Policy Search (GPS) is a simple algorithm that learns a policy for test-time data augmentation based on the predictive performance on a validation set. GPS starts with…

Similar Papers 제목 키워드 기반

Sphere2Vec: Multi-Scale Representation Learning over a Spherical Surface for Geospatial Predictions

2022-01-25 · Gengchen Mai, Yao Xuan, Wenyun Zuo, Krzysztof Janowicz 외

Generating learning-friendly representations for points in a 2D space is a fundamental and long-standing problem in machine learning. Recently, multi-scale encoding schemes (such as Space2Vec) were proposed to directly e…

image-classificationImage ClassificationRepresentation Learning

Sphere2Vec: Self-Supervised Location Representation Learning on Spherical Surfaces

2021-09-29 · Gengchen Mai, Yao Xuan, Wenyun Zuo, Yutong He 외

Location encoding is valuable for a multitude of tasks where both the absolute positions and local contexts (image, text, and other types of metadata) of spatial objects are needed for accurate predictions. However, most…

image-classificationImage ClassificationRepresentation LearningSelf-Supervised Learning

SphereNet: Learning Spherical Representations for Detection and Classification in Omnidirectional Images

2018-09-01 · ECCV 2018 9 · Benjamin Coors, Alexandru Paul Condurache, Andreas Geiger

Omnidirectional cameras offer great benefits over classical cameras wherever a wide field of view is essential, such as in virtual reality applications or in autonomous robots. Unfortunately, standard convolutional neura…

General Classificationimage-classificationImage Classificationobject-detection+1

SatCLIP: Global, General-Purpose Location Embeddings with Satellite Imagery

2023-11-28 · Konstantin Klemmer, Esther Rolf, Caleb Robinson, Lester Mackey 외

Geographic information is essential for modeling tasks in fields ranging from ecology to epidemiology. However, extracting relevant location characteristics for a given task can be challenging, often requiring expensive …

Contrastive Learning

Single Image Calibration of Multi-axial Imaging Systems

2013-06-01 · CVPR 2013 6 · Amit Agrawal, Srikumar Ramalingam

Imaging systems consisting of a camera looking at multiple spherical mirrors (reflection) or multiple refractive spheres (refraction) have been used for wide-angle imaging applications. We describe such setups as multi-a…

Camera CalibrationPose Estimation