paper-with-me

홈 › Papers

Classifying Math KCs via Task-Adaptive Pre-Trained BERT

2021-05-24 · Jia Tracy Shen, Michiharu Yamashita, Ethan Prihar, Neil Heffernan, Xintao Wu, Sean McGrew, Dongwon Lee

Educational content labeled with proper knowledge components (KCs) are particularly useful to teachers or content organizers. However, manually labeling educational content is labor intensive and error-prone. To address this challenge, prior research proposed machine learning based solutions to auto-label educational content with limited success. In this work, we significantly improve prior research by (1) expanding the input types to include KC descriptions, instructional video titles, and problem descriptions (i.e., three types of prediction task), (2) doubling the granularity of the prediction from 198 to 385 KC labels (i.e., more practical setting but much harder multinomial classification problem), (3) improving the prediction accuracies by 0.5-2.3% using Task-adaptive Pre-trained BERT, outperforming six baselines, and (4) proposing a simple evaluation measure by which we can recover 56-73% of mispredicted KC labels. All codes and data sets in the experiments are available at:https://github.com/tbs17/TAPT-BERT

📄 PDF Abstract BibTeX arXiv:2105.11343

Code (0)

등록된 구현이 없습니다.

Tasks

MathPredictionTask 2

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.
WordPiece 설명 없음
Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…
Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…
Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Refunds@Expedia|||How do I get a full refund from Expedia? “How do I get a full refund from Expedia? How do I get a full refund from Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Quick Help &…

Similar Papers 제목 키워드 기반

BERT Goes Brrr: A Venture Towards the Lesser Error in Classifying Medical Self-Reporters on Twitter

2021-06-01 · NAACL (SMM4H) 2021 6 · Alham Fikri Aji, Made Nindyatama Nityasya, Haryo Akbarianto Wibowo, Radityo Eko Prasojo 외

This paper describes our team’s submission for the Social Media Mining for Health (SMM4H) 2021 shared task. We participated in three subtasks: Classifying adverse drug effect, COVID-19 self-report, and COVID-19 symptoms.…

Hyperparameter Optimization

MathBERT: A Pre-trained Language Model for General NLP Tasks in Mathematics Education

2021-06-02 · Jia Tracy Shen, Michiharu Yamashita, Ethan Prihar, Neil Heffernan 외

Since the introduction of the original BERT (i.e., BASE BERT), researchers have developed various customized BERT models with improved performance for specific domains and tasks by exploiting the benefits of transfer lea…

Knowledge TracingLanguage ModelingLanguage ModellingMath+1

MathBERT: A Pre-Trained Model for Mathematical Formula Understanding

2021-05-02 · Shuai Peng, Ke Yuan, Liangcai Gao, Zhi Tang

Large-scale pre-trained models like BERT, have obtained a great success in various Natural Language Processing (NLP) tasks, while it is still a challenge to adapt them to the math-related tasks. Current pre-trained model…

Headline GenerationInformation RetrievalMathRetrieval+2

DomURLs_BERT: Pre-trained BERT-based Model for Malicious Domains and URLs Detection and Classification

2024-09-13 · Abdelkader El Mahdaouy, Salima Lamsiyah, Meryem Janati Idrissi, Hamza Alami 외

Detecting and classifying suspicious or malicious domain names and URLs is fundamental task in cybersecurity. To leverage such indicators of compromise, cybersecurity vendors and practitioners often maintain and update b…

Language ModelingLanguage ModellingMasked Language ModelingMulti-class Classification

Classifying Long Clinical Documents with Pre-trained Transformers

2021-05-14 · Xin Su, Timothy Miller, Xiyu Ding, Majid Afshar 외

Automatic phenotyping is a task of identifying cohorts of patients that match a predefined set of criteria. Phenotyping typically involves classifying long clinical documents that contain thousands of tokens. At the same…

Sentence