Collage Prompting: Budget-Friendly Visual Recognition with GPT-4V
Recent advancements in generative AI have suggested that by taking visual prompt, GPT-4V can demonstrate significant proficiency in image recognition task. Despite its impressive capabilities, the financial cost associated with GPT-4V's inference presents a substantial barrier for its wide use. To address this challenge, our work introduces Collage Prompting, a budget-friendly prompting approach that concatenates multiple images into a single visual input. With collage prompt, GPT-4V is able to perform image recognition on several images simultaneously. Based on the observation that the accuracy of GPT-4V's image recognition varies significantly with the order of images within the collage prompt, our method further learns to optimize the arrangement of images for maximum recognition accuracy. A graph predictor is trained to indicate the accuracy of each collage prompt, then we propose an optimization method to navigate the search space of possible image arrangements. Experiment results across various datasets demonstrate the cost-efficiency score of collage prompt is much larger than standard prompt. Additionally, collage prompt with learned arrangement achieves clearly better accuracy than collage prompt with random arrangement in GPT-4V's visual recognition.
Code (0)
등록된 구현이 없습니다.
Tasks
NavigateSimilar Papers 제목 키워드 기반
Self-Reasoning Agentic Framework for Narrative Product Grid-Collage Generation
Narrative-driven product photography has become a prevalent paradigm in modern marketing, as coherent visual storytelling helps convey product value and establishes emotional engagement with consumers. However, existing …
Visual StorytellingImage GenerationEffect of Mammalian Tissue Source on the Molecular and Macroscopic Characteristics of UV-Cured Type I Collagen Hydrogel Networks
The tissue source of type I collagen is critical to ensure scalability and regulation-friendly clinical translation of new medical device prototypes. However, the selection of a commercial source of collagen that fulfils…
Vocal Bursts Type PredictionA collagen-based theranostic wound dressing with visual, long-lasting infection detection capability
Continuous wound monitoring is one strategy to minimise infection severity and inform prompt variations in therapeutic care following infection diagnosis. However, integration of this functionality in therapeutic wound d…
Monomer-induced customisation of UV-cured atelocollagen hydrogel networks
The covalent functionalisation of type I atelocollagen with either 4-vinylbenzyl or methacrylamide residues is presented as a simple synthetic strategy to achieve customisable, cell-friendly UV-cured hydrogel networks wi…
Vocal Bursts Type PredictionSoftCollage: A Differentiable Probabilistic Tree Generator for Image Collage
Image collage task aims to create an informative and visual-aesthetic visual summarization for an image collection. While several recent works exploit tree-based algorithm to preserve image content better, all of the…