What kinds of errors do reference resolution models make and what can we learn from them?
Referring resolution is the task of identifying the referent of a natural language expression, for example “the woman behind the other woman getting a massage”. In this paper we investigate which are the kinds of referring expressions on which current transformer based models fail. Motivated by this analysis we identify the weakening of the spatial natural constraints as one of its causes and propose a model that aims to restore it. We evaluate our proposed model on different datasets for the task showing improved performance on the most challenging kinds of referring expressions. Finally we present a thorough analysis of the kinds errors that are improved by the new model and those that are not and remain future challenges for the task.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Underreporting of errors in NLG output, and what to do about it
We observe a severe under-reporting of the different kinds of errors that Natural Language Generation systems make. This is a problem, because mistakes are an important indicator of where systems should still be improved…
PositionText GenerationThe Curious Case of Control
Children acquiring English make systematic errors on subject control sentences (Chomsky, 1969) possibly due to heuristics based on semantic roles (Maratsos, 1974).Given the advanced fluency of large generative language m…
ObjectGradations of Error Severity in Automatic Image Descriptions
Earlier research has shown that evaluation metrics based on textual similarity (e.g., BLEU, CIDEr, Meteor) do not correlate well with human evaluation scores for automatically generated text. We carried out an experiment…
Decision-Theoretic Question Generation for Situated Reference Resolution: An Empirical Study and Computational Model
Dialogue agents that interact with humans in situated environments need to manage referential ambiguity across multiple modalities and ask for help as needed. However, it is not clear what kinds of questions such agents …
Question GenerationQuestion-Generationslot-fillingSlot FillingWord Embeddings as Features for Supervised Coreference Resolution
A common reason for errors in coreference resolution is the lack of semantic information to help determine the compatibility between mentions referring to the same entity. Distributed representations, which have been sho…
coreference-resolutionCoreference ResolutionDimensionality ReductionMachine Translation+3