Flexible Tool Selection through Low-dimensional Attribute Alignment of Vision and Language
Flexible tool selection reflects a complex cognitive ability that distinguishes humans from other species, yet computational models that capture this ability remain underdeveloped. We developed a framework using low-dimensional attribute representations to bridge visual tool perception and linguistic task understanding. We constructed a comprehensive dataset (ToolNet) containing 115 common tools labeled with 13 carefully designed attributes spanning physical, functional, and psychological properties, paired with natural language scenarios describing tool usage. Visual encoders (ResNet or ViT) extract attributes from tool images while fine-tuned language models (GPT-2, LLaMA, DeepSeek) derive required attributes from task descriptions. Our approach achieves 74% accuracy in tool selection tasks-significantly outperforming direct tool matching (20%) and smaller multimodal models (21%-58%), while approaching performance of much larger models like GPT-4o (73%) with substantially fewer parameters. Ablation studies revealed that manipulation-related attributes (graspability, hand-relatedness, elongation) consistently prove most critical across modalities. This work provides a parameter-efficient, interpretable solution that mimics human-like tool cognition, advancing both cognitive science understanding and practical applications in tool selection tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
AttributeMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
MMM-fair: An Interactive Toolkit for Exploring and Operationalizing Multi-Fairness Trade-offs
Fairness-aware classification requires balancing performance and fairness, often intensified by intersectional biases. Conflicting fairness definitions further complicate the task, making it difficult to identify univers…
IVFS: Simple and Efficient Feature Selection for High Dimensional Topology Preservation
Feature selection is an important tool to deal with high dimensional data. In unsupervised case, many popular algorithms aim at maintaining the structure of the original data. In this paper, we propose a simple and effec…
feature selectionVocal Bursts Intensity PredictionA Copula Graphical Model for Multi-Attribute Data using Optimal Transport
Motivated by modern data forms such as images and multi-view data, the multi-attribute graphical model aims to explore the conditional independence structure among vectors. Under the Gaussian assumption, the conditional …
AttributeOn Irrelevance of Attributes in Flexible Prediction
This paper analyses properties of conceptual hierarchy obtained via incremental concept formation method called "flexible prediction" in order to determine what kind of "relevance" of participating attributes may be requ…
AttributeGeneral ClassificationPredictionQuick and Robust Feature Selection: the Strength of Energy-efficient Sparse Training for Autoencoders
Major complications arise from the recent increase in the amount of high-dimensional data, including high computational costs and memory requirements. Feature selection, which identifies the most relevant and informative…
ClusteringDenoisingDimensionality ReductionFeature Importance+1