Creating and Managing a large annotated parallel corpora of Indian languages
This paper presents the challenges in creating and managing large parallel corpora of 12 major Indian languages (which is soon to be extended to 23 languages) as part of a major consortium project funded by the Department of Information Technology (DIT), Govt. of India, and running parallel in 10 different universities of India. In order to efficiently manage the process of creation and dissemination of these huge corpora, the web-based (with a reduced stand-alone version also) annotation tool ILCIANN (Indian Languages Corpora Initiative Annotation Tool) has been developed. It was primarily developed for the POS annotation as well as the management of the corpus annotation by people with differing amount of competence and at locations physically situated far apart. In order to maintain consistency and standards in the creation of the corpora, it was necessary that everyone works on a common platform which was provided by this tool.
Code (0)
등록된 구현이 없습니다.
Tasks
ManagementPOSSimilar Papers 제목 키워드 기반
Command-line utilities for managing and exploring annotated corpora
ROOTS: a toolkit for easy, fast and consistent processing of large sequential annotated data collections
The development of new methods for given speech and natural language processing tasks usually consists in annotating large corpora of data before applying machine learning techniques to train models or to extract informa…
DiversityManagementBuilding The Sense-Tagged Multilingual Parallel Corpus
Sense-annotated parallel corpora play a crucial role in natural language processing. This paper introduces our progress in creating such a corpus for Asian languages using English as a pivot, which is the first such corp…
Creating and using large monolingual parallel corpora for sentential paraphrase generation
Projecting named entity recognizers without annotated or parallel corpora
Named entity recognition (NER) is a well-researched task in the field of NLP, which typically requires large annotated corpora for training usable models. This is a problem for languages which lack large annotated corpor…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NER