Papers Probing Language Models
“Probing Language Models” 태그가 달린 논문 20편 · 필터 해제
Linguistically Grounded Analysis of Language Models using Shapley Head Values
Understanding how linguistic knowledge is encoded in language models is crucial for improving their generalisation capabilities. In this paper, we investigate the processing of morphosyntactic phenomena, by leveraging a …
Probing Language ModelsProbing Language Models on Their Knowledge Source
Large Language Models (LLMs) often encounter conflicts between their learned, internal (parametric knowledge, PK) and external knowledge provided during inference (contextual knowledge, CK). Understanding how LLMs models…
Probing Language ModelsProbing Language Models for Pre-training Data Detection
Large Language Models (LLMs) have shown their impressive capabilities, while also raising concerns about the data contamination problems due to privacy issues and leakage of benchmark datasets in the pre-training phase. …
Probing Language ModelsUnveiling LLMs: The Evolution of Latent Representations in a Dynamic Knowledge Graph
Large Language Models (LLMs) demonstrate an impressive capacity to recall a vast range of factual knowledge. However, understanding their underlying reasoning and internal mechanisms in exploiting this knowledge remains …
Claim VerificationCommon Sense ReasoningKnowledge GraphsKnowledge Probing+3Probing Language Models' Gesture Understanding for Enhanced Human-AI Interaction
The rise of Large Language Models (LLMs) has affected various disciplines that got beyond mere text generation. Going beyond their textual nature, this project proposal aims to investigate the interaction between LLMs an…
Probing Language ModelsText GenerationSocial Bias Probing: Fairness Benchmarking for Language Models
While the impact of social biases in language models has been recognized, prior methods for bias evaluation have been limited to binary association tests on small datasets, limiting our understanding of bias complexities…
BenchmarkingFairnessProbing Language ModelsProbing Representations for Document-level Event Extraction
The probing classifiers framework has been employed for interpreting deep neural network models for a variety of natural language processing (NLP) applications. Studies, however, have largely focused on sentencelevel NLP…
Document-level Event ExtractionEvent ExtractionProbing Language ModelsSentenceTable-GPT: Table-tuned GPT for Diverse Table Tasks
Language models, such as GPT-3.5 and ChatGPT, demonstrate remarkable abilities to follow diverse human instructions and perform a wide range of tasks. However, when probing language models using a range of basic table-un…
Probing Language ModelsProbing Large Language Models from A Human Behavioral Perspective
Large Language Models (LLMs) have emerged as dominant foundational models in modern NLP. However, the understanding of their prediction processes and internal mechanisms, such as feed-forward networks (FFN) and multi-hea…
MemorizationProbing Language ModelsQuantifying and Analyzing Entity-level Memorization in Large Language Models
Large language models (LLMs) have been proven capable of memorizing their training data, which can be extracted through specifically designed prompts. As the scale of datasets continues to grow, privacy risks arising fro…
Language ModelingLanguage ModellingMemorizationProbing Language ModelsLanguage Models Trained on Media Diets Can Predict Public Opinion
Public opinion reflects and shapes societal behavior, but the traditional survey-based tools to measure it are limited. We introduce a novel approach to probe media diet models -- language models adapted to online news, …
Probing Language ModelsSurveyKAMEL : Knowledge Analysis with Multitoken Entities in Language Models
Large language models (LMs) have been shown to capture large amounts of relational knowledge from the pre-training corpus. These models can be probed for this factual knowledge by using cloze-style prompts as demonstrat…
Knowledge GraphsProbing Language ModelsUniversal and Independent: Multilingual Probing Framework for Exhaustive Model Interpretation and Evaluation
Linguistic analysis of language models is one of the ways to explain and describe their reasoning, weaknesses, and limitations. In the probing part of the model interpretability research, studies concern individual langu…
Probing Language ModelsDo ever larger octopi still amplify reporting biases? Evidence from judgments of typical colour
Language models (LMs) trained on raw texts have no direct access to the physical world. Gordon and Van Durme (2013) point out that LMs can thus suffer from reporting bias: texts rarely report on common facts, instead foc…
Common Sense ReasoningPhysical Commonsense ReasoningProbing Language ModelsWinoDict: Probing language models for in-context word acquisition
We introduce a new in-context learning paradigm to measure Large Language Models' (LLMs) ability to learn novel words during inference. In particular, we rewrite Winograd-style co-reference resolution problems by replaci…
In-Context LearningProbing Language ModelsDiscontinuous Constituency and BERT: A Case Study of Dutch
In this paper, we set out to quantify the syntactic capacity of BERT in the evaluation regime of non-context free patterns, as occurring in Dutch. We devise a test suite based on a mildly context-sensitive formalism, fro…
Probing Language ModelsThe neural architecture of language: Integrative modeling converges on predictive processing
The neuroscience of perception has recently been revolutionized with an integrative modeling approach in which computation, brain function, and behavior are linked across many datasets and many computational models. By r…
Language ModellingProbing Language ModelsReading ComprehensionProbing Language Models for Understanding of Temporal Expressions
We present three Natural Language Inference (NLI) challenge sets that can evaluate NLI models on their understanding of temporal expressions. More specifically, we probe these models for three temporal properties: (a) th…
Natural Language InferenceProbing Language ModelsRelationProbing Toxic Content in Large Pre-Trained Language Models
Large pre-trained language models (PTLMs) have been shown to carry biases towards different social groups which leads to the reproduction of stereotypical and toxic content by major NLP systems. We propose a method based…
Probing Language ModelsSentenceQuantifying Gender Bias Towards Politicians in Cross-Lingual Language Models
Recent research has demonstrated that large pre-trained language models reflect societal biases expressed in natural language. The present paper introduces a simple method for probing language models to conduct a multili…
Language ModelingLanguage ModellingProbing Language Models