Labeling of Query Words using Conditional Random Field
This paper describes our approach on Query Word Labeling as an attempt in the shared task on Mixed Script Information Retrieval at Forum for Information Retrieval Evaluation (FIRE) 2015. The query is written in Roman script and the words were in English or transliterated from Indian regional languages. A total of eight Indian languages were present in addition to English. We also identified the Named Entities and special symbols as part of our task. A CRF based machine learning framework was used for labeling the individual words with their corresponding language labels. We used a dictionary based approach for language identification. We also took into account the context of the word while identifying the language. Our system demonstrated an overall accuracy of 75.5% for token level language identification. The strict F-measure scores for the identification of token level language labels for Bengali, English and Hindi are 0.7486, 0.892 and 0.7972 respectively. The overall weighted F-measure of our system was 0.7498.
Code (0)
등록된 구현이 없습니다.
Tasks
Information RetrievalLanguage IdentificationRetrievalSimilar Papers 제목 키워드 기반
Hybrid semi-Markov CRF for Neural Sequence Labeling
This paper proposes hybrid semi-Markov conditional random fields (SCRFs) for neural sequence labeling in natural language processing. Based on conventional conditional random fields (CRFs), SCRFs have been designed for t…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NERImage Labeling with Markov Random Fields and Conditional Random Fields
Most existing methods for object segmentation in computer vision are formulated as a labeling task. This, in general, could be transferred to a pixel-wise label assignment task, which is quite similar to the structure of…
SegmentationSemantic SegmentationFew-Shot Sequence Labeling with Label Dependency Transfer and Pair-wise Embedding
While few-shot classification has been widely explored with similarity based methods, few-shot sequence labeling poses a unique challenge as it also calls for modeling the label dependencies. To consider both the item si…
Few-Shot Learningnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+1Masked Conditional Random Fields for Sequence Labeling
Conditional Random Field (CRF) based neural models are among the most performant methods for solving sequence labeling problems. Despite its great success, CRF has the shortcoming of occasionally generating illegal seque…
TAGMulti-Task Learning of Query Intent and Named Entities using Transfer Learning
Named entity recognition (NER) has been studied extensively and the earlier algorithms were based on sequence labeling like Hidden Markov Models (HMM) and conditional random fields (CRF). These were followed by neural ne…
Multi-Task Learningnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+2