The Contribution of Knowledge in Visiolinguistic Learning: A Survey on Tasks and Challenges
Recent advancements in visiolinguistic (VL) learning have allowed the development of multiple models and techniques that offer several impressive implementations, able to currently resolve a variety of tasks that require the collaboration of vision and language. Current datasets used for VL pre-training only contain a limited amount of visual and linguistic knowledge, thus significantly limiting the generalization capabilities of many VL models. External knowledge sources such as knowledge graphs (KGs) and Large Language Models (LLMs) are able to cover such generalization gaps by filling in missing knowledge, resulting in the emergence of hybrid architectures. In the current survey, we analyze tasks that have benefited from such hybrid approaches. Moreover, we categorize existing knowledge sources and types, proceeding to discussion regarding the KG vs LLM dilemma and its potential impact to future hybrid approaches.
Code (0)
등록된 구현이 없습니다.
Tasks
Knowledge GraphsSimilar Papers 제목 키워드 기반
A survey on knowledge-enhanced multimodal learning
Multimodal learning has been a field of increasing interest, aiming to combine various modalities in a single joint representation. Especially in the area of visiolinguistic (VL) learning multiple models and techniques h…
Conditional Image GenerationFactual Visual Question AnsweringFairnessImage Captioning+13AutoSurvey: Large Language Models Can Automatically Write Surveys
This paper introduces AutoSurvey, a speedy and well-organized methodology for automating the creation of comprehensive literature surveys in rapidly evolving fields like artificial intelligence. Traditional survey paper …
RetrievalSurveyImage Search With Text Feedback by Visiolinguistic Attention Learning
Image search with text feedback has promising impacts in various real-world applications, such as e-commerce and internet search. Given a reference image and text feedback from user, the goal is to retrieve images that n…
AttributeDeep AttentionImage RetrievalMultimodal Deep LearningA survey of benchmarking frameworks for reinforcement learning
Reinforcement learning has recently experienced increased prominence in the machine learning community. There are many approaches to solving reinforcement learning problems with new techniques developed constantly. When …
Benchmarkingreinforcement-learningReinforcement LearningReinforcement Learning (RL)+1A Survey on Table-and-Text HybridQA: Concepts, Methods, Challenges and Future Directions
Table-and-text hybrid question answering (HybridQA) is a widely used and challenging NLP task commonly applied in the financial and scientific domain. The early research focuses on migrating other QA task methods to Hybr…
Question AnsweringSurvey