Domain adaptation in practice: Lessons from a real-world information extraction pipeline
Advances in transfer learning and domain adaptation have raised hopes that once-challenging NLP tasks are ready to be put to use for sophisticated information extraction needs. In this work, we describe an effort to do just that – combining state-of-the-art neural methods for negation detection, document time relation extraction, and aspectual link prediction, with the eventual goal of extracting drug timelines from electronic health record text. We train on the THYME colon cancer corpus and test on both the THYME brain cancer corpus and an internal corpus, and show that performance of the combined systems is unacceptable despite good performance of individual systems. Although domain adaptation shows improvements on each individual system, the model selection problem is a barrier to improving overall pipeline performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Domain AdaptationLink PredictionModel SelectionNegationNegation DetectionRelation ExtractionTransfer LearningSimilar Papers 제목 키워드 기반
The Road of Adaptive AI for Precision in Cybersecurity
Cybersecurity's evolving complexity presents unique challenges and opportunities for AI research and practice. This paper shares key lessons and insights from designing, building, and operating production-grade GenAI pip…
Domain Adaptation of a State of the Art Text-to-SQL Model: Lessons Learned and Challenges Found
There are many recent advanced developments for the Text-to-SQL task, where the Picard model is one of the the top performing models as measured by the Spider dataset competition. However, bringing Text-to-SQL systems to…
Domain AdaptationLanguage ModelingLanguage ModellingText to SQL+1Better Practices for Domain Adaptation
Distribution shifts are all too common in real-world applications of machine learning. Domain adaptation (DA) aims to address this by providing various frameworks for adapting models to the deployment data without using …
BenchmarkingDomain AdaptationSource-Free Domain AdaptationTest-time Adaptation+1Engineering RAG Systems for Real-World Applications: Design, Development, and Evaluation
Retrieval-Augmented Generation (RAG) systems are emerging as a key approach for grounding Large Language Models (LLMs) in external knowledge, addressing limitations in factual accuracy and contextual relevance. However, …
Optical Character Recognition (OCR)RAGRetrievalRetrieval-augmented Generation+1Lessons from Usable ML Deployments and Application to Wind Turbine Monitoring
Through past experiences deploying what we call usable ML (one step beyond explainable ML, including both explanations and other augmenting information) to real-world domains, we have learned three key lessons. First, ma…
Decision Making