GRILLBot: An Assistant for Real-World Tasks with Neural Semantic Parsing and Graph-Based Representations
GRILLBot is the winning system in the 2022 Alexa Prize TaskBot Challenge, moving towards the next generation of multimodal task assistants. It is a voice assistant to guide users through complex real-world tasks in the domains of cooking and home improvement. These are long-running and complex tasks that require flexible adjustment and adaptation. The demo highlights the core aspects, including a novel Neural Decision Parser for contextualized semantic parsing, a new "TaskGraph" state representation that supports conditional execution, knowledge-grounded chit-chat, and automatic enrichment of tasks with images and videos.
Code (0)
등록된 구현이 없습니다.
Tasks
Semantic ParsingSimilar Papers 제목 키워드 기반
GRILLBot: A multi-modal conversational agent for complex real-world tasks
We present GRILLBot, an open-source multi-modal task-oriented voice assistant to help users perform complex tasks, focusing on the domains of cooking and home improvement. GRILLBot curates and leverages web information e…
ManagementNavigateQuestion AnsweringTask PlanningGRILLBot In Practice: Lessons and Tradeoffs Deploying Large Language Models for Adaptable Conversational Task Assistants
We tackle the challenge of building real-world multimodal assistants for complex real-world tasks. We describe the practicalities and challenges of developing and deploying GRILLBot, a leading (first and second prize win…
Code GenerationManagementQuestion AnsweringWorld KnowledgeLiveClawBench: Benchmarking LLM Agents on Complex, Real-World Assistant Tasks
OpenClaw-style personal assistants extend LLM agents from isolated tool use to open-ended, stateful, and personalized software environments. Evaluating these assistants is fundamentally a fidelity problem: benchmarks mus…
RAmBLA: A Framework for Evaluating the Reliability of LLMs as Assistants in the Biomedical Domain
Large Language Models (LLMs) increasingly support applications in a wide range of domains, some with potential high societal impact such as biomedicine, yet their reliability in realistic use cases is under-researched. I…
Semantic SimilaritySemantic Textual SimilarityTEOChat: A Large Vision-Language Assistant for Temporal Earth Observation Data
Large vision and language assistants have enabled new capabilities for interpreting natural images. These approaches have recently been adapted to earth observation data, but they are only able to handle single image inp…
Change DetectionEarth ObservationInstruction FollowingQuestion Answering+3