Automatic Construction of Large Readability Corpora

2016-12-01 · WS 2016 12 · Jorge Alberto Wagner Filho, Rodrigo Wilkens, Aline Villavicencio

This work presents a framework for the automatic construction of large Web corpora classified by readability level. We compare different Machine Learning classifiers for the task of readability assessment focusing on Portuguese and English texts, analysing the impact of variables like the feature inventory used in the resulting corpus. In a comparison between shallow and deeper features, the former already produce F-measures of over 0.75 for Portuguese texts, but the use of additional features results in even better results, in most cases. For English, shallow features also perform well as do classic readability formulas. Comparing different classifiers for the task, logistic regression obtained, in general, the best results, but with considerable differences between the results for two and those for three-classes, especially regarding the intermediary class. Given the large scale of the resulting corpus, for evaluation we adopt the agreement between different classifiers as an indication of readability assessment certainty. As a result of this work, a large corpus for Brazilian Portuguese was built, including 1.7 million documents and about 1.6 billion tokens, already parsed and annotated with 134 different textual attributes, along with the agreement among the various classifiers.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Text Classification Text Simplification

Methods 이 논문이 사용한 방법론

Logistic Regression Logistic Regression, despite its name, is a linear model for classification rather than regression. Logistic regression is also known in the literature as logit regression,…

Analysing Zero-Shot Readability-Controlled Sentence Simplification

2024-09-30 · Abdullah Barayan, Jose Camacho-Collados, Fernando Alva-Manchego

Readability-controlled text simplification (RCTS) rewrites texts to lower readability levels while preserving their meaning. RCTS models often depend on parallel corpora with readability annotations on both source and ta…

Sentence Text Simplification

Learning Syntactic Dense Embedding with Correlation Graph for Automatic Readability Assessment

2021-07-09 · ACL 2021 5 · Xinying Qiu, Yuan Chen, Hanwu Chen, Jian-Yun Nie 외

Deep learning models for automatic readability assessment generally discard linguistic features traditionally used in machine learning models for the task. We propose to incorporate linguistic features into neural networ…

Feature Optimization for Predicting Readability of Arabic L1 and L2

2018-07-01 · WS 2018 7 · Hind Saddiki, Nizar Habash, Violetta Cavalli-Sforza, Muhamed Al Khalil

Advances in automatic readability assessment can impact the way people consume information in a number of domains. Arabic, being a low-resource and morphologically complex language, presents numerous challenges to the ta…

Language Modeling Language Modelling

Measuring text readability with machine comprehension: a pilot study

2019-08-01 · WS 2019 8 · Marc Benzahra, Fran{\c{c}}ois Yvon

This article studies the relationship between text readability indice and automatic machine understanding systems. Our hypothesis is that the simpler a text is, the better it should be understood by a machine. We thus ex…

Reading Comprehension

Developing a Monolingual Sentence Simplification Corpus for Urdu

2020-07-01 · WS 2020 7 · Yusra Anees, Sadaf Abdul Rauf, Nauman Iqbal, Abdul Basit Siddiqi

Complex sentences are a hurdle in the learning process of language learners. Sentence simplification aims to convert a complex sentence into its simpler form such that it is easily comprehensible. To build such automated…