홈 › Papers

Towards explainable evaluation of language models on the semantic similarity of visual concepts

2022-09-08 · COLING 2022 10 · Maria Lymperaiou, George Manoliadis, Orfeas Menis Mastromichalakis, Edmund G. Dervakos, Giorgos Stamou

Recent breakthroughs in NLP research, such as the advent of Transformer models have indisputably contributed to major advancements in several tasks. However, few works research robustness and explainability issues of their evaluation strategies. In this work, we examine the behavior of high-performing pre-trained language models, focusing on the task of semantic similarity for visual vocabularies. First, we address the need for explainable evaluation metrics, necessary for understanding the conceptual quality of retrieved instances. Our proposed metrics provide valuable insights in local and global level, showcasing the inabilities of widely used approaches. Secondly, adversarial interventions on salient query semantics expose vulnerabilities of opaque metrics and highlight patterns in learned linguistic representations.

📄 PDF Abstract BibTeX arXiv:2209.03723

Code (0)

등록된 구현이 없습니다.

Tasks

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Adam 설명 없음

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Multi-Head Attention 설명 없음

Residual Connection 설명 없음

Explainable Deepfake Detection Challenge

2026-07-23 · Abhijeet Narang, Kartik Kuckreja, Shreya Ghosh, Muhammad Haris Khan 외 arxiv

Deepfake detection is moving beyond binary classification decisions toward systems that can also explain the visual evidence supporting those decisions. This transition is important for real-world verification settings, …

Explanation Generation Binary Classification Image Classification Semantic Similarity

Efficient Explainable Face Verification based on Similarity Score Argument Backpropagation

2023-04-26 · Marco Huber, Anh Thi Luu, Philipp Terhörst, Naser Damer

Explainable Face Recognition is gaining growing attention as the use of the technology is gaining ground in security-critical applications. Understanding why two faces images are matched or not matched by a given face re…

Face Recognition Face Verification

Contrastive Visual Semantic Pretraining Magnifies the Semantics of Natural Language Representations

2022-03-14 · ACL 2022 5 · Robert Wolfe, Aylin Caliskan

We examine the effects of contrastive visual semantic pretraining by comparing the geometry and semantic properties of contextualized English language representations formed by GPT-2 and CLIP, a zero-shot multimodal imag…

Image Captioning Semantic Textual Similarity Sentence Sentence Embeddings+1

Explainable Novel Category Discovery in Semantic Concept Space

2026-07-05 · Ifrat Ikhtear Uddin, Yang Zhou, KC Santosh, Longwei Wang arxiv

Novel category discovery aims to identify unseen classes from unlabeled data by transferring knowledge from labeled categories, but most existing methods perform discovery in opaque latent feature spaces. As a result, th…

Integrating Language Guidance into Vision-based Deep Metric Learning

2022-03-16 · CVPR 2022 1 · Karsten Roth, Oriol Vinyals, Zeynep Akata

Deep Metric Learning (DML) proposes to learn metric spaces which encode semantic similarities as embedding space distances. These spaces should be transferable to classes beyond those seen during training. Commonly, DML …

Metric Learning