Assessment of Amazon Comprehend Medical: Medication Information Extraction
In November 27, 2018, Amazon Web Services (AWS) released Amazon Comprehend Medical (ACM), a deep learning based system that automatically extracts clinical concepts (which include anatomy, medical conditions, protected health information (PH)I, test names, treatment names, and medical procedures, and medications) from clinical text notes. Uptake and trust in any new data product relies on independent validation across benchmark datasets and tools to establish and confirm expected quality of results. This work focuses on the medication extraction task, and particularly, ACM was evaluated using the official test sets from the 2009 i2b2 Medication Extraction Challenge and 2018 n2c2 Track 2: Adverse Drug Events and Medication Extraction in EHRs. Overall, ACM achieved F-scores of 0.768 and 0.828. These scores ranked the lowest when compared to the three best systems in the respective challenges. To further establish the generalizability of its medication extraction performance, a set of random internal clinical text notes from NYU Langone Medical Center were also included in this work. And in this corpus, ACM garnered an F-score of 0.753.
Code (0)
등록된 구현이 없습니다.
Tasks
AnatomyMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Comprehend Medical: a Named Entity Recognition and Relationship Extraction Web Service
Comprehend Medical is a stateless and Health Insurance Portability and Accountability Act (HIPAA) eligible Named Entity Recognition (NER) and Relationship Extraction (RE) service launched under Amazon Web Services (AWS) …
Anatomynamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+2How well it works: Benchmarking performance of GPT models on medical natural language processing tasks
Importance: The ability of large language models (LLMs) to generate high-quality, human-like text has been accompanied with speculation about their application in healthcare, alongside ethical and safety concerns. Object…
BenchmarkingDevelopment and validation of a short form of the medication literacy scale for Chinese College Students
Medication literacy is integral to health literacy, pivotal for medication safety and adherence. It denotes an individual's capacity to discern, comprehend, and convey medication-related information. Existing scales, how…
FormThe Medical Deconfounder: Assessing Treatment Effects with Electronic Health Records
The treatment effects of medications play a key role in guiding medical prescriptions. They are usually assessed with randomized controlled trials (RCTs), which are expensive. Recently, large-scale electronic health reco…
Accurate clinical and biomedical Named entity recognition at scale
We introduce an agile, production-grade clinical and biomedical Named entity recognition (NER) algorithm based on a modified BiLSTM-CNN-Char DL architecture built on top of Apache Spark. Our NER implementation establishe…
Clinical Concept ExtractionDe-identificationnamed-entity-recognitionNamed Entity Recognition+2