A critical survey on measuring success in rank-based keyword assignment to documents
Evaluation approaches for unsupervised rank-based keyword assignment are nearly as numerous as are the existing systems. The prolific production of each newly used metric (or metric twist) seems to stem from general dis-satisfaction with the previous one and the source of that dissatisfaction has not previously been discussed in the literature. The difficulty may stem from a poor specification of the keyword assignment task in view of the rank-based approach. With a more complete specification of this task, we aim to show why the previous evaluation metrics fail to satisfy researchers{'} goals to distinguish and detect good rank-based keyword assignment systems. We put forward a characterisation of an ideal evaluation metric, and discuss the consistency of the evaluation metrics with this ideal, finding that the average standard normalised cumulative gain metric is most consistent with this ideal.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
No Stupid Questions: An Analysis of Question Query Generation for Citation Recommendation
Existing techniques for citation recommendation are constrained by their adherence to article contents and metadata. We leverage GPT-4o-mini's latent expertise as an inquisitive assistant by instructing it to ask questio…
Citation RecommendationEnhanced Semantic Graph Based Approach With Sentiment Analysis For User Interest Retrieval From Social Sites
Blogs and social networking sites serve as a platform to the users for expressing their interests, ideas and thoughts. Targeted marketing uses the recommendation systems for suggesting their services and products to the …
MarketingRecommendation SystemsRetrievalSentiment AnalysisTextRank: Bringing Order into Texts
In this paper, we introduce TextRank – a graph-based ranking model for text processing and show how this model can be successfully used in natural language applications. In particular, we propose two innovative unsupervi…
SentenceN-Grams TextRank A Novel Domain Keyword Extraction Technique
The rapid growth of the internet has given us a wealth of information and data spread across the web. However, as the data begins to grow we simultaneously face the grave problem of an Information Explosion. An abundance…
Keyword ExtractionManagementA survey and taxonomy of loss functions in machine learning
Most state-of-the-art machine learning techniques revolve around the optimisation of loss functions. Defining appropriate loss functions is therefore critical to successfully solving problems in this field. In this surve…
regressionSurvey