paper-with-me

홈 › Papers

Face Identification Proficiency Test Designed Using Item Response Theory

2021-06-22 · Géraldine Jeckeln, Ying Hu, Jacqueline G. Cavazos, Amy N. Yates, Carina A. Hahn, Larry Tang, P. Jonathon Phillips, Alice J. O'Toole

Measures of face-identification proficiency are essential to ensure accurate and consistent performance by professional forensic face examiners and others who perform face-identification tasks in applied scenarios. Current proficiency tests rely on static sets of stimulus items, and so, cannot be administered validly to the same individual multiple times. To create a proficiency test, a large number of items of "known" difficulty must be assembled. Multiple tests of equal difficulty can be constructed then using subsets of items. We introduce the Triad Identity Matching (TIM) test and evaluate it using Item Response Theory (IRT). Participants view face-image "triads" (N=225) (two images of one identity, one image of a different identity) and select the different identity. In Experiment 1, university students (N=197) showed wide-ranging accuracy on the TIM test, and IRT modeling demonstrated that the TIM items span various difficulty levels. In Experiment 2, we used IRT-based item metrics to partition the test into subsets of specific difficulties. Simulations showed that subsets of the TIM items yielded reliable estimates of subject ability. In Experiments 3a and 3b, we found that the student-derived IRT model reliably evaluated the ability of non-student participants and that ability generalized across different test sessions. In Experiment 3c, we show that TIM test performance correlates with other common face-recognition tests. In summary, the TIM test provides a starting point for developing a framework that is flexible and calibrated to measure proficiency across various ability levels (e.g., professionals or populations with face-processing deficits).

📄 PDF Abstract BibTeX arXiv:2106.15323

Code (0)

등록된 구현이 없습니다.

Tasks

Face IdentificationFace Recognition

Similar Papers 제목 키워드 기반

Student achievement and French sentence repetition test scores

2014-05-01 · LREC 2014 5 · Deryle Lonsdale, Benjamin Millard

Sentence repetition (SR) tests are one way of probing a language learner{'}s oral proficiency. Test-takers listen to a set of carefully engineered sentences of varying complexity one-by-one, and then try to repeat them b…

Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Sentencespeech-recognition+1

Jump-Starting Item Parameters for Adaptive Language Tests

2021-11-01 · EMNLP 2021 11 · Arya D. McCarthy, Kevin P. Yancey, Geoff T. LaFlair, Jesse Egbert 외

A challenge in designing high-stakes language assessments is calibrating the test item difficulties, either a priori or from limited pilot test data. While prior work has addressed ‘cold start’ estimation of item difficu…

Language AcquisitionMulti-Task LearningSkills Assessment

Textual complexity as a predictor of difficulty of listening items in language proficiency tests

2016-12-01 · COLING 2016 12 · Anastassia Loukina, Su-Youn Yoon, Jennifer Sakano, Youhua Wei 외

In this paper we explore to what extent the difficulty of listening items in an English language proficiency test can be predicted by the textual properties of the prompt. We show that a system based on multiple text com…

Reading Comprehension

Item Development and Scoring for Japanese Oral Proficiency Testing

2012-05-01 · LREC 2012 5 · Hitokazu Matsushita, Deryle Lonsdale

This study introduces and evaluates a computerized approach to measuring Japanese L2 oral proficiency. We present a testing and scoring method that uses a type of structured speech called elicited imitation (EI) to evalu…

Language ModelingLanguage ModellingSpeech Recognition

Machine Learning--Driven Language Assessment

2020-01-01 · TACL 2020 1 · Burr Settles, Geoffrey T. LaFlair, Masato Hagiwara

We describe a method for rapidly creating language proficiency assessments, and provide experimental evidence that such tests can be valid, reliable, and secure. Our approach is the first to use machine learning and natu…

BIG-bench Machine LearningLanguage AcquisitionSkills Assessmentvalid