a survey on GPT-3

2022-12-01 · Mingyu Zong, Bhaskar Krishnamachari

This paper provides an introductory survey to GPT-3. We cover some of the historical development behind this technology, some of the key features of GPT-3, and discuss the machine learning model and the datasets used. We survey both academic and commercial efforts applying GPT-3 in diverse domains such as developing conversational AI chatbots, software development, creative work, domain knowledge, and business productivity. We discuss some of the challenges that GPT-3 faces such as the problems of training complexity, bias, and hallucination/incorrect answers. We also discuss the future research opportunities in this area.

📄 PDF Abstract BibTeX arXiv:2212.00857

Code (0)

등록된 구현이 없습니다.

Tasks

HallucinationSurvey

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음

Attention 설명 없음

Cosine Annealing Cosine Annealing is a type of learning rate schedule that has the effect of starting with a large learning rate that is relatively rapidly decreased to a minimum value before…

Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…

Linear Warmup With Cosine Annealing Linear Warmup With Cosine Annealing is a learning rate schedule where we increase the learning rate linearly for $n$ updates and then anneal according to a cosine schedule…

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

{Dispute@FaQ-s}How to file a dispute with Expedia? How to file a dispute with Expedia? To file a complaint against Expedia, first try contacting their customer service directly. You can reach them by phone at…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

SurveyX: Academic Survey Automation via Large Language Models

2025-02-20 · Xun Liang, Jiawei Yang, Yezhaohui Wang, Chen Tang 외

Large Language Models (LLMs) have demonstrated exceptional comprehension capabilities and a vast knowledge base, suggesting that LLMs can serve as efficient tools for automated survey generation. However, recent research…

Survey

Survey Response Generation: Generating Closed-Ended Survey Responses In-Silico with Large Language Models

2025-10-13 · Georg Ahnert, Anna-Carolina Haensch, Barbara Plank, Markus Strohmaier arxiv

Many in-silico simulations of human survey responses with large language models (LLMs) focus on generating closed-ended survey responses, whereas LLMs are typically trained to generate open-ended text instead. Previous r…

Response Generation

Trends in Publishing Blockchain Surveys: A Bibliometric Perspective

2021-09-20 · Hira Ahmad, Muhammad Ahtazaz Ahsan, Adnan Noor Mian

A large number of survey papers are being published in blockchain since the first survey appeared in 2017. A person entering into the field of blockchain is faced with the issue of which blockchain surveys to read and wh…

Survey

AutoSurvey: Large Language Models Can Automatically Write Surveys

2024-06-10 · Yidong Wang, Qi Guo, Wenjin Yao, Hongbo Zhang 외

This paper introduces AutoSurvey, a speedy and well-organized methodology for automating the creation of comprehensive literature surveys in rapidly evolving fields like artificial intelligence. Traditional survey paper …

Retrieval Survey

SurveyForge: On the Outline Heuristics, Memory-Driven Generation, and Multi-dimensional Evaluation for Automated Survey Writing

2025-03-06 · Xiangchao Yan, Shiyang Feng, Jiakang Yuan, Renqiu Xia 외

Survey paper plays a crucial role in scientific research, especially given the rapid growth of research publications. Recently, researchers have begun using LLMs to automate survey generation for better efficiency. Howev…

Articles Survey