Angler: Helping Machine Translation Practitioners Prioritize Model Improvements
Machine learning (ML) models can fail in unexpected ways in the real world, but not all model failures are equal. With finite time and resources, ML practitioners are forced to prioritize their model debugging and improvement efforts. Through interviews with 13 ML practitioners at Apple, we found that practitioners construct small targeted test sets to estimate an error's nature, scope, and impact on users. We built on this insight in a case study with machine translation models, and developed Angler, an interactive visual analytics tool to help practitioners prioritize model improvements. In a user study with 7 machine translation experts, we used Angler to understand prioritization practices when the input space is infinite, and obtaining reliable signals of model quality is expensive. Our study revealed that participants could form more interesting and user-focused hypotheses for prioritization by analyzing quantitative summary statistics and qualitatively assessing data by reading sentences.
Code (1)
Tasks
Machine TranslationmodelTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Boosting propagule transport models with individual-specific data from mobile apps
Management of invasive species and pathogens requires information about the traffic of potential vectors. Such information is often taken from vector traffic models fitted to survey data. Here, user-specific data collect…
ManagementCan machine learning predict citizen-reported angler behavior?
Prediction of angler behaviors, such as catch rates and angler pressure, is essential to maintaining fish populations and ensuring angler satisfaction. Angler behavior can partly be tracked by online platforms and mobile…
ManagementPredictionWebsite visits can predict angler presence using machine learning
Understanding and predicting recreational angler effort is important for sustainable fisheries management. However, conventional methods of measuring angler effort, such as surveys, can be costly and limited in both time…
ManagementSustainable Recreational Fishing Using a Novel Electrical Muscle Stimulation (EMS) Lure and Ensemble Network Algorithm to Maximize Catch and Release Survivability
With 200-700 million anglers in the world, sportfishing is nearly five times more common than commercial trawling. Worldwide, hundreds of thousands of jobs are linked to the sportfishing industry, which generates billion…
Combining Sequence Distillation and Transfer Learning for Efficient Low-Resource Neural Machine Translation Models
In neural machine translation (NMT), sequence distillation (SD) through creation of distilled corpora leads to efficient (compact and fast) models. However, its effectiveness in extremely low-resource (ELR) settings has …
Low Resource Neural Machine TranslationLow-Resource Neural Machine TranslationMachine TranslationNMT+3