Combining Data-driven Supervision with Human-in-the-loop Feedback for Entity Resolution
The distribution gap between training datasets and data encountered in production is well acknowledged. Training datasets are often constructed over a fixed period of time and by carefully curating the data to be labeled. Thus, training datasets may not contain all possible variations of data that could be encountered in real-world production environments. Tasked with building an entity resolution system - a model that identifies and consolidates data points that represent the same person - our first model exhibited a clear training-production performance gap. In this case study, we discuss our human-in-the-loop enabled, data-centric solution to closing the training-production performance divergence. We conclude with takeaways that apply to data-centric learning at large.
Code (0)
등록된 구현이 없습니다.
Tasks
Entity ResolutionSimilar Papers 제목 키워드 기반
A Human-in/on-the-Loop Framework for Accessible Text Generation
Plain Language and Easy-to-Read formats in text simplification are essential for cognitive accessibility. Yet current automatic simplification and evaluation pipelines remain largely automated, metric-driven, and fail to…
Text SimplificationText GenerationBeyond Self-Play: Hierarchical Reasoning for Continuous Motion in Closed-Loop Traffic Simulation
Closed-loop traffic simulation requires agents that are both scalable and behaviorally realistic. Recent self-play reinforcement learning approaches demonstrate strong scalability, but their equilibrium strategies fail t…
Multi-agent Reinforcement LearningIMPACT-Scribe: Interactive Temporal Action Segmentation with Boundary Scribbles and Query Planning
Dense temporal annotation of procedural activity videos is vital for action understanding and embodied intelligence but remains labor-intensive due to reactive tools. Each correction is treated as an isolated edit, limit…
Action UnderstandingAction SegmentationTask-oriented grasping for dexterous robots using postural synergies and reinforcement learning
In this paper, we address the problem of task-oriented grasping for humanoid robots, emphasizing the need to align with human social norms and task-specific objectives. Existing methods, employ a variety of open-loop and…
Reinforcement LearningA knowledge-augmented dataset of high-risk driving scenarios with LLM annotations for autonomous driving
Safe autonomous driving requires both rapid responses to common high-risk events and deeper reasoning over rare, extreme long-tail scenarios in traffic safety. These scenarios are severely under-represented in naturalist…
Autonomous Driving