Text Difficulty Study: Do machines behave the same as humans regarding text difficulty?

2022-08-14 · Bowen Chen, Xiao Ding, Li Du, Qin Bing, Ting Liu

Given a task, human learns from easy to hard, whereas the model learns randomly. Undeniably, difficulty insensitive learning leads to great success in NLP, but little attention has been paid to the effect of text difficulty in NLP. In this research, we propose the Human Learning Matching Index (HLM Index) to investigate the effect of text difficulty. Experiment results show: (1) LSTM has more human-like learning behavior than BERT. (2) UID-SuperLinear gives the best evaluation of text difficulty among four text difficulty criteria. (3) Among nine tasks, some tasks' performance is related to text difficulty, whereas some are not. (4) Model trained on easy data performs best in easy and medium data, whereas trains on a hard level only perform well on hard data. (5) Training the model from easy to hard leads to fast convergence.

📄 PDF Abstract BibTeX arXiv:2208.14509

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

WordPiece 설명 없음

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

Adam 설명 없음

Tanh Activation 설명 없음

Do humans and machines have the same eyes? Human-machine perceptual differences on image classification

2023-04-18 · Minghao Liu, Jiaheng Wei, Yang Liu, James Davis

Trained computer vision models are assumed to solve vision tasks by imitating human behavior learned from training labels. Most efforts in recent vision research focus on measuring the model task performance using standa…

image-classification Image Classification

Human vs. supervised machine learning: Who learns patterns faster?

2020-11-30 · Niklas Kühl, Marc Goutier, Lucas Baier, Clemens Wolff 외

The capabilities of supervised machine learning (SML), especially compared to human abilities, are being discussed in scientific research and in the usage of SML. This study provides an answer to how learning performance…

BIG-bench Machine Learning

Byzantine-Resilient Non-Convex Stochastic Gradient Descent

2020-12-28 · ICLR 2021 1 · Zeyuan Allen-Zhu, Faeze Ebrahimian, Jerry Li, Dan Alistarh

We study adversary-resilient stochastic distributed optimization, in which $m$ machines can independently compute stochastic gradients, and cooperate to jointly optimize over their local objective functions. However, an …

Distributed Optimization

Assessing Quality Estimation Models for Sentence-Level Prediction

2018-08-01 · COLING 2018 8 · Hoang Cuong, Jia Xu

This paper provides an evaluation of a wide range of advanced sentence-level Quality Estimation models, including Support Vector Regression, Ride Regression, Neural Networks, Gaussian Processes, Bayesian Neural Networks,…

Gaussian Processes Machine Translation Prediction regression+1

Personalized Game Difficulty Prediction Using Factorization Machines

2022-09-06 · Jeppe Theiss Kristensen, Christian Guckelsberger, Paolo Burelli, Perttu Hämäläinen

The accurate and personalized estimation of task difficulty provides many opportunities for optimizing user experience. However, user diversity makes such difficulty estimation hard, in that empirical measurements from s…

Diversity Prediction