Weakly Supervised Learners for Correction of AI Errors with Provable Performance Guarantees
We present a new methodology for handling AI errors by introducing weakly supervised AI error correctors with a priori performance guarantees. These AI correctors are auxiliary maps whose role is to moderate the decisions of some previously constructed underlying classifier by either approving or rejecting its decisions. The rejection of a decision can be used as a signal to suggest abstaining from making a decision. A key technical focus of the work is in providing performance guarantees for these new AI correctors through bounds on the probabilities of incorrect decisions. These bounds are distribution agnostic and do not rely on assumptions on the data dimension. Our empirical example illustrates how the framework can be applied to improve the performance of an image classifier in a challenging real-world task where training data are scarce.
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Neural Quality Estimation of Grammatical Error Correction
Grammatical error correction (GEC) systems deployed in language learning environments are expected to accurately correct errors in learners{'} writing. However, in practice, they often produce spurious corrections and fa…
Grammatical Error CorrectionMachine TranslationRe-RankingAUTOMATIC PRONUNCIATION MISTAKE DETECTOR PROJECT REPORT
Given the drawbacks of traditional English pronunciation correction systems, such as failure to provide timely feedback and correct learners' pronunciation errors, slow improvement of learners' English proficiency, and e…
Mistake Detectionspeech-recognitionSpeech RecognitionSpelling Correction for Russian: A Comparative Study of Datasets and Methods
We develop a minimally-supervised model for spelling correction and evaluate its performance on three datasets annotated for spelling errors in Russian. The first corpus is a dataset of Russian social media data that was…
Cross-corpusMachine TranslationRe-RankingSpelling Correction+1Exploring word embeddings and phonological similarity for the unsupervised correction of language learner errors
The presence of misspellings and other errors or non-standard word forms poses a considerable challenge for NLP systems. Although several supervised approaches have been proposed previously to normalize these, annotated …
Grapheme-to-Phoneme ConversionLanguage AcquisitionWord Embeddings