paper-with-me

홈 › Papers

From Natural Language to Simulations: Applying GPT-3 Codex to Automate Simulation Modeling of Logistics Systems

2022-02-24 · Ilya Jackson, Maria Jesus Saenz

Our work is the first attempt to apply Natural Language Processing to automate the development of simulation models of systems vitally important for logistics. We demonstrated that the framework built on top of the fine-tuned GPT-3 Codex, a Transformer-based language model, could produce functionally valid simulations of queuing and inventory control systems given the verbal description. In conducted experiments, GPT-3 Codex demonstrated convincing expertise in Python as well as an understanding of the domain-specific vocabulary. As a result, the language model could produce simulations of a single-product inventory-control system and single-server queuing system given the domain-specific context, a detailed description of the process, and a list of variables with the corresponding values. The demonstrated results, along with the rapid improvement of language models, open the door for significant simplification of the workflow behind the simulation model development, which will allow experts to focus on the high-level consideration of the problem and holistic thinking.

📄 PDF Abstract BibTeX arXiv:2202.12107

Code (1)

jackil1993/gpt3_scm 공식 구현

Tasks

Language ModelingLanguage Modellingvalid

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음
Attention 설명 없음
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Cosine Annealing Cosine Annealing is a type of learning rate schedule that has the effect of starting with a large learning rate that is relatively rapidly decreased to a minimum value before…
Linear Warmup With Cosine Annealing Linear Warmup With Cosine Annealing is a learning rate schedule where we increase the learning rate linearly for $n$ updates and then anneal according to a cosine schedule…
Residual Connection 설명 없음
Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…
Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Similar Papers 제목 키워드 기반

CodexDB: Generating Code for Processing SQL Queries using GPT-3 Codex

2022-04-19 · Immanuel Trummer

CodexDB is an SQL processing engine whose internals can be customized via natural language instructions. CodexDB is based on OpenAI's GPT-3 Codex model which translates text into code. It is a framework on top of GPT-3 C…

How Effective Are Neural Networks for Fixing Security Vulnerabilities

2023-05-29 · Yi Wu, Nan Jiang, Hung Viet Pham, Thibaud Lutellier 외

Security vulnerability repair is a difficult task that is in dire need of automation. Two groups of techniques have shown promise: (1) large code language models (LLMs) that have been pre-trained on source code for tasks…

Code CompletionProgram Repair

Improving Automated Program Repair with Domain Adaptation

2022-12-21 · Armin Zirak, Hadi Hemati

Automated Program Repair (APR) is defined as the process of fixing a bug/defect in the source code, by an automated tool. APR tools have recently experienced promising results by leveraging state-of-the-art Neural Langua…

Domain AdaptationProgram RepairZero-Shot Learning

Automated Code generation for Information Technology Tasks in YAML through Large Language Models

2023-05-02 · Saurabh Pujar, Luca Buratti, Xiaojie Guo, Nicolas Dupuis 외

The recent improvement in code generation capabilities due to the use of large language models has mainly benefited general purpose programming languages. Domain specific languages, such as the ones used for IT Automatio…

Code Generation

Few-Shot Semantic Parsing with Language Models Trained On Code

2022-01-16 · ACL ARR January 2022 1 · Anonymous

Large language models can perform semantic parsing with little training data, when prompted with in-context examples. It has been shown that this can be improved by formulating the problem as paraphrasing into canonical …

Semantic Parsing