A Parallel Evaluation Data Set of Software Documentation with Document Structure Annotation
This paper accompanies the software documentation data set for machine translation, a parallel evaluation data set of data originating from the SAP Help Portal, that we released to the machine translation community for research purposes. It offers the possibility to tune and evaluate machine translation systems in the domain of corporate software documentation and contributes to the availability of a wider range of evaluation scenarios. The data set comprises of the language pairs English to Hindi, Indonesian, Malay and Thai, and thus also increases the test coverage for the many low-resource language pairs. Unlike most evaluation data sets that consist of plain parallel text, the segments in this data set come with additional metadata that describes structural information of the document context. We provide insights into the origin and creation, the particularities and characteristics of the data set as well as machine translation results.
Code (1)
Tasks
Machine TranslationTranslationSimilar Papers 제목 키워드 기반
Documentation of Machine Learning Software
Machine Learning software documentation is different from most of the documentations that were studied in software engineering research. Often, the users of these documentations are not software experts. The increasing i…
BIG-bench Machine LearningVerifiable Source Code Documentation in Controlled Natural Language
Writing documentation about software internals is rarely considered a rewarding activity. It is highly time-consuming and the resulting documentation is fragile when the software is continuously evolving in a multi-devel…
Evaluating Repository-level Software Documentation via Question Answering and Feature-Driven Development
Software documentation is crucial for repository comprehension. While Large Language Models (LLMs) advance documentation generation from code snippets to entire repositories, existing benchmarks have two key limitations:…
Question AnsweringCIAO - Code In Architecture Out - Automated Software Architecture Documentation with Large Language Models
Software architecture documentation is essential for system comprehension, yet it is often unavailable or incomplete. While recent LLM-based techniques can generate documentation from code, they typically address local a…
LLM-Based Code Documentation Generation and Multi-Judge Evaluation
High-quality source code documentation is vital yet often neglected, especially in critical domains like healthcare where reliability and maintainability are essential. We presented an AI powered framework that automates…
Code Documentation GenerationPrompt Engineering