Toward Reliable Ad-hoc Scientific Information Extraction: A Case Study on Two Materials Datasets
We explore the ability of GPT-4 to perform ad-hoc schema based information extraction from scientific literature. We assess specifically whether it can, with a basic prompting approach, replicate two existing material science datasets, given the manuscripts from which they were originally manually extracted. We employ materials scientists to perform a detailed manual error analysis to assess where the model struggles to faithfully extract the desired information, and draw on their insights to suggest research directions to address this broadly important task.
Code (1)
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
SoMeSci- A 5 Star Open Data Gold Standard Knowledge Graph of Software Mentions in Scientific Articles
Knowledge about software used in scientific investigations is important for several reasons, for instance, to enable an understanding of provenance and methods involved in data handling. However, software is usually not …
ArticlesEntity DisambiguationEntity Linkingnamed-entity-recognition+4Multimodal Approach for Metadata Extraction from German Scientific Publications
Nowadays, metadata information is often given by the authors themselves upon submission. However, a significant part of already existing research papers have missing or incomplete metadata information. German scientific …
Multimodal Deep LearningHuman-artificial intelligence teaming for scientific information extraction from data-driven additive manufacturing research using large language models
Data-driven research in Additive Manufacturing (AM) has gained significant success in recent years. This has led to a plethora of scientific literature to emerge. The knowledge in these works consists of AM and Artificia…
Reducing Hallucinations in LLM-based Scientific Literature Analysis Using Peer Context Outlier Detection
Reducing hallucinations in Large Language Models (LLMs) is essential for improving the accuracy of data extraction from large text corpora. Current methods, like prompt engineering and chain-of-thought prompting, focus o…
Prompt EngineeringOutlier DetectionEntity Recognition and Relation Extraction from Scientific and Technical Texts in Russian
This paper is devoted to the study of methods for information extraction (entity recognition and relation classification) from scientific texts on information technology. Scientific publications provide valuable informat…
Keyword ExtractionRelationRelation ClassificationRelation Extraction