Comparing apples to apples in the evaluation of binary coding methods
We discuss methodological issues related to the evaluation of unsupervised binary code construction methods for nearest neighbor search. These issues have been widely ignored in literature. These coding methods attempt to preserve either Euclidean distance or angular (cosine) distance in the binary embedding space. We explain why when comparing a method whose goal is preserving cosine similarity to one designed for preserving Euclidean distance, the original features should be normalized by mapping them to the unit hypersphere before learning the binary mapping functions. To compare a method whose goal is to preserves Euclidean distance to one that preserves cosine similarity, the original feature data must be mapped to a higher dimension by including a bias term in binary mapping functions. These conditions ensure the fair comparison between different binary code methods for the task of nearest neighbor search. Our experiments show under these conditions the very simple methods (e.g. LSH and ITQ) often outperform recent state-of-the-art methods (e.g. MDSH and OK-means).
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Comparing Apples to Apples: Generating Aspect-Aware Comparative Sentences from User Reviews
It is time-consuming to find the best product among many similar alternatives. Comparative sentences can help to contrast one item from others in a way that highlights important features of an item that stand out. Given …
Towards Apples to Apples for AI Evaluations: From Real-World Use Cases to Evaluation Scenarios
AI measurement science has a wide variety of methodologies and measurements for comparing AI systems, resulting in what often appear to be "apples-to-oranges" comparisons across AI evaluations. To move toward "apples-to-…
Apples to Apples: Learning Semantics of Common Entities Through a Novel Comprehension Task
Understanding common entities and their attributes is a primary requirement for any system that comprehends natural language. In order to enable learning about common entities, we introduce a novel machine comprehension …
BenchmarkingPart-Of-Speech TaggingReading ComprehensionSemantic Textual Similarity+1Design of an Intelligent Vision Algorithm for Recognition and Classification of Apples in an Orchard Scene
Apple is one of the remarkable fresh fruit that contains a high degree of nutritious and medicinal value. Hand harvesting of apples by seasonal farmworkers increases physical damages on the surface of these fruits, which…
MarketingLocalizing Small Apples in Complex Apple Orchard Environments
The localization of fruits is an essential first step in automated agricultural pipelines for yield estimation or fruit picking. One example of this is the localization of apples in images of entire apple trees. Since th…
ObjectObject Proposal Generation