paper-with-me

홈 › Papers

AviationGPT: A Large Language Model for the Aviation Domain

2023-11-29 · Liya Wang, Jason Chou, Xin Zhou, Alex Tien, Diane M Baumgartner

The advent of ChatGPT and GPT-4 has captivated the world with large language models (LLMs), demonstrating exceptional performance in question-answering, summarization, and content generation. The aviation industry is characterized by an abundance of complex, unstructured text data, replete with technical jargon and specialized terminology. Moreover, labeled data for model building are scarce in this domain, resulting in low usage of aviation text data. The emergence of LLMs presents an opportunity to transform this situation, but there is a lack of LLMs specifically designed for the aviation domain. To address this gap, we propose AviationGPT, which is built on open-source LLaMA-2 and Mistral architectures and continuously trained on a wealth of carefully curated aviation datasets. Experimental results reveal that AviationGPT offers users multiple advantages, including the versatility to tackle diverse natural language processing (NLP) problems (e.g., question-answering, summarization, document writing, information extraction, report querying, data cleaning, and interactive data exploration). It also provides accurate and contextually relevant responses within the aviation domain and significantly improves performance (e.g., over a 40% performance gain in tested cases). With AviationGPT, the aviation industry is better equipped to address more complex research problems and enhance the efficiency and safety of National Airspace System (NAS) operations.

📄 PDF Abstract BibTeX arXiv:2311.17686

Code (0)

등록된 구현이 없습니다.

Tasks

Language ModelingLanguage ModellingLarge Language ModelQuestion Answering

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음
Attention 설명 없음
Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…
Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…
BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…
Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…
Position-Wise Feed-Forward Layer 설명 없음

Similar Papers 제목 키워드 기반

There is No Big Brother or Small Brother: Knowledge Infusion in Language Models for Link Prediction and Question Answering

2023-01-10 · Ankush Agarwal, Sakharam Gawade, Sachin Channabasavarajendra, Pushpak Bhattacharyya

The integration of knowledge graphs with deep learning is thriving in improving the performance of various natural language processing (NLP) tasks. In this paper, we focus on knowledge-infused link prediction and questio…

Knowledge GraphsLink PredictionPredictionQuestion Answering

Adapting Sentence Transformers for the Aviation Domain

2023-05-16 · Liya Wang, Jason Chou, Dave Rouck, Alex Tien 외

Learning effective sentence representations is crucial for many Natural Language Processing (NLP) tasks, including semantic search, semantic textual similarity (STS), and clustering. While multiple transformer models hav…

DenoisingNatural Language InferenceSemantic Textual SimilaritySentence+3

KITLM: Domain-Specific Knowledge InTegration into Language Models for Question Answering

2023-08-07 · Ankush Agarwal, Sakharam Gawade, Amar Prakash Azad, Pushpak Bhattacharyya

Large language models (LLMs) have demonstrated remarkable performance in a wide range of natural language tasks. However, as these models continue to grow in size, they face significant challenges in terms of computation…

Language ModelingLanguage ModellingMulti-hop Question AnsweringQuestion Answering

CAMB: A comprehensive industrial LLM benchmark on civil aviation maintenance

2025-08-28 · Feng Zhang, Chengjie Pang, Yuehan Zhang, Chenyu Luo arxiv

Civil aviation maintenance is a domain characterized by stringent industry standards. Within this field, maintenance procedures and troubleshooting represent critical, knowledge-intensive tasks that require sophisticated…

Prompt Engineering

Pre-Flight: A Benchmark for Evaluating Large Language Models on Aviation Operational Knowledge

2026-07-02 · Alex Brooker, Tim Hughes arxiv

Large language models (LLMs) are increasingly proposed for aviation business operations, from documentation and training generation to customer facing assistants. General purpose benchmarks do not measure whether a model…

General Knowledge