When Life Gives You Lemons, Make Cherryade: Converting Feedback from Bad Responses into Good Labels
Deployed dialogue agents have the potential to integrate human feedback to continuously improve themselves. However, humans may not always provide explicit signals when the chatbot makes mistakes during interactions. In this work, we propose Juicer, a framework to make use of both binary and free-form textual human feedback. It works by: (i) extending sparse binary feedback by training a satisfaction classifier to label the unlabeled data; and (ii) training a reply corrector to map the bad replies to good ones. We find that augmenting training with model-corrected replies improves the final dialogue model, and we can further improve performance by using both positive and negative replies through the recently proposed Director model.
Code (0)
등록된 구현이 없습니다.
Tasks
ChatbotSimilar Papers 제목 키워드 기반
When Life Gives You Lemons, Squeeze Your Way Through: Understanding Citrus Avoidance Behaviour by Free-Ranging Dogs in India
Palatability of food is driven by multiple factors like taste, smell, texture, freshness, etc. and can be very variable across species. There are classic examples of local adaptations leading to speciation, driven by foo…
When Life Gives You AI, Will You Turn It Into A Market for Lemons? Understanding How Information Asymmetries About AI System Capabilities Affect Market Outcomes and Adoption
AI consumer markets are characterized by severe buyer-supplier market asymmetries. Complex AI systems can appear highly accurate while making costly errors or embedding hidden defects. While there have been regulatory ef…
The Market for Lemons and the Regulator's Signalling Problem
The Market for Lemons is a classic model of asymmetric information first studied by Nobel Prize economist George Akerlof. It shows that information asymmetry between the seller and buyer may result in market collapse or …
Lemonade from Lemons: Information Design and Adverse Selection
A seller posts a price for a single object. The seller's and buyer's values may be interdependent. We characterize the set of payoff vectors across all information structures. Simple feasibility and individual-rationalit…
Capability Advertisement as a Market for Lemons: A Trust Layer for Heterogeneous Agent Networks
Large language model (LLM) agents have begun to delegate work to one another. Protocols such as the Model Context Protocol (MCP) and the Agent2Agent protocol (A2A) let an agent publish what it can do and let others call …