Grounding and Evaluation for Large Language Models: Practical Challenges and Lessons Learned (Survey)
With the ongoing rapid adoption of Artificial Intelligence (AI)-based systems in high-stakes domains, ensuring the trustworthiness, safety, and observability of these systems has become crucial. It is essential to evaluate and monitor AI systems not only for accuracy and quality-related metrics but also for robustness, bias, security, interpretability, and other responsible AI dimensions. We focus on large language models (LLMs) and other generative AI models, which present additional challenges such as hallucinations, harmful and manipulative content, and copyright infringement. In this survey article accompanying our KDD 2024 tutorial, we highlight a wide range of harms associated with generative AI systems, and survey state of the art approaches (along with open challenges) to address these harms.
Code (0)
등록된 구현이 없습니다.
Tasks
SurveyMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Text-controlled Motion Mamba: Text-Instructed Temporal Grounding of Human Motion
Human motion understanding is a fundamental task with diverse practical applications, facilitated by the availability of large-scale motion capture datasets. Recent studies focus on text-motion tasks, such as text-based …
MambaMotion GenerationQuestion AnsweringTowards Understanding Visual Grounding in Visual Language Models
Visual grounding refers to the ability of a model to identify a region within some visual input that matches a textual description. Consequently, a model equipped with visual grounding capabilities can target a wide rang…
multimodal generationReferring ExpressionVisual GroundingWhen Visual Grounding Meets Gigapixel-level Large-scale Scenes: Benchmark and Approach
Visual grounding refers to the process of associating natural language expressions with corresponding regions within an image. Existing benchmarks for visual grounding primarily operate within small-scale scenes with…
Scene UnderstandingVisual GroundingMotivGraph-SoIQ: Integrating Motivational Knowledge Graphs and Socratic Dialogue for Enhanced LLM Ideation
Large Language Models (LLMs) hold substantial potential for accelerating academic ideation but face critical challenges in grounding ideas and mitigating confirmation bias for further refinement. We propose integrating m…
Knowledge GraphsTowards Visual Text Grounding of Multimodal Large Language Model
Despite the existing evolution of Multimodal Large Language Models (MLLMs), a non-neglectable limitation remains in their struggle with visual text grounding, especially in text-rich images of documents. Document images,…
BenchmarkingLanguage ModelingLanguage ModellingLarge Language Model+5