The First Multilingual Surface Realisation Shared Task (SR'18): Overview and Evaluation Results
We report results from the SR{'}18 Shared Task, a new multilingual surface realisation task organised as part of the ACL{'}18 Workshop on Multilingual Surface Realisation. As in its English-only predecessor task SR{'}11, the shared task comprised two tracks with different levels of complexity: (a) a shallow track where the inputs were full UD structures with word order information removed and tokens lemmatised; and (b) a deep track where additionally, functional words and morphological information were removed. The shallow track was offered in ten, and the deep track in three languages. Systems were evaluated (a) automatically, using a range of intrinsic metrics, and (b) by human judges in terms of readability and meaning similarity. This report presents the evaluation results, along with descriptions of the SR{'}18 tracks, data and evaluation methods. For full descriptions of the participating systems, please see the separate system reports elsewhere in this volume.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
The First Multilingual Surface Realisation Shared Task (SRâ18): Overview and Evaluation Results
LORIA / Lorraine University at Multilingual Surface Realisation 2019
This paper presents the LORIA / Lorraine University submission at the Multilingual Surface Realisation shared task 2019 for the shallow track. We outline our approach and evaluate it on 11 languages covered by the shared…
The Second Multilingual Surface Realisation Shared Task (SR'19): Overview and Evaluation Results
We report results from the SR{'}19 Shared Task, the second edition of a multilingual surface realisation task organised as part of the EMNLP{'}19 Workshop on Multilingual Surface Realisation. As in SR{'}18, the shared ta…
The Third Multilingual Surface Realisation Shared Task (SR’20): Overview and Evaluation Results
This paper presents results from the Third Shared Task on Multilingual Surface Realisation (SR’20) which was organised as part of the COLING’20 Workshop on Multilingual Surface Realisation. As in SR’18 and SR’19, the sha…
Underspecified Universal Dependency Structures as Inputs for Multilingual Surface Realisation
In this paper, we present the datasets used in the Shallow and Deep Tracks of the First Multilingual Surface Realisation Shared Task (SR{'}18). For the Shallow Track, data in ten languages has been released: Arabic, Czec…
Natural Language UnderstandingText Generation