Shallow Synthesis of Knowledge in GPT-Generated Texts: A Case Study in Automatic Related Work Composition

2024-02-19 · Anna Martin-Boyle, Aahan Tyagi, Marti A. Hearst, Dongyeop Kang

Numerous AI-assisted scholarly applications have been developed to aid different stages of the research process. We present an analysis of AI-assisted scholarly writing generated with ScholaCite, a tool we built that is designed for organizing literature and composing Related Work sections for academic papers. Our evaluation method focuses on the analysis of citation graphs to assess the structural complexity and inter-connectedness of citations in texts and involves a three-way comparison between (1) original human-written texts, (2) purely GPT-generated texts, and (3) human-AI collaborative texts. We find that GPT-4 can generate reasonable coarse-grained citation groupings to support human users in brainstorming, but fails to perform detailed synthesis of related works without human intervention. We suggest that future writing assistant tools should not be used to draft text independently of the human author.

📄 PDF Abstract BibTeX arXiv:2402.12255

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Adam 설명 없음

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Multi-Head Attention 설명 없음

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

How well can machine-generated texts be identified and can language models be trained to avoid identification?

2023-10-25 · Sinclair Schneider, Florian Steuber, Joao A. G. Schneider, Gabi Dreo Rodosek

With the rise of generative pre-trained transformer models such as GPT-3, GPT-NeoX, or OPT, distinguishing human-generated texts from machine-generated ones has become important. We refined five separate language models …

Linguistic Acceptability Text Generation

Knowledge-Rich Context Extraction and Ranking with KnowPipe

2012-05-01 · LREC 2012 5 · Anne-Kathrin Schumann

This paper presents ongoing Phd thesis work dealing with the extraction of knowledge-rich contexts from text corpora for terminographic purposes. Although notable progress in the field has been made over recent years, th…

FakePolisher: Making DeepFakes More Detection-Evasive by Shallow Reconstruction

2020-06-13 · Yihao Huang, Felix Juefei-Xu, Run Wang, Qing Guo 외

At this moment, GAN-based image generation methods are still imperfect, whose upsampling design has limitations in leaving some certain artifact patterns in the synthesized image. Such artifact patterns can be easily exp…

DeepFake Detection Face Swapping Fake Image Detection Image Generation

FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios

2023-07-25 · I-Chun Chern, Steffi Chern, Shiqi Chen, Weizhe Yuan 외

The emergence of generative pre-trained models has facilitated the synthesis of high-quality text, but it has also posed challenges in identifying factual errors in the generated text. In particular: (1) A wider range of…

Code Generation Fact Checking Mathematical Reasoning

Function approximation by deep networks

2019-05-30 · H. N. Mhaskar, T. Poggio

We show that deep networks are better than shallow networks at approximating functions that can be expressed as a composition of functions described by a directed acyclic graph, because the deep networks can be designed …