Scaling up Discovery of Latent Concepts in Deep NLP Models
Despite the revolution caused by deep NLP models, they remain black boxes, necessitating research to understand their decision-making processes. A recent work by Dalvi et al. (2022) carried out representation analysis through the lens of clustering latent spaces within pre-trained models (PLMs), but that approach is limited to small scale due to the high cost of running Agglomerative hierarchical clustering. This paper studies clustering algorithms in order to scale the discovery of encoded concepts in PLM representations to larger datasets and models. We propose metrics for assessing the quality of discovered latent concepts and use them to compare the studied clustering algorithms. We found that K-Means-based concept discovery significantly enhances efficiency while maintaining the quality of the obtained concepts. Furthermore, we demonstrate the practicality of this newfound efficiency by scaling latent concept discovery to LLMs and phrasal concepts.
Code (1)
Tasks
ClusteringDecision MakingSimilar Papers 제목 키워드 기반
LatexBlend: Scaling Multi-concept Customized Generation with Latent Textual Blending
Customized text-to-image generation renders user-specified concepts into novel contexts based on textual prompts. Scaling the number of concepts in customized generation meets a broader demand for user creation, whereas …
Computational EfficiencyDenoisingImage GenerationText to Image Generation+1Your most telling friends: Propagating latent ideological features on Twitter using neighborhood coherence
Multidimensional scaling in networks allows for the discovery of latent information about their structure by embedding nodes in some feature space. Ideological scaling for users in social networks such as Twitter is an e…
Generating Counterfactual Trajectories with Latent Diffusion Models for Concept Discovery
Trustworthiness is a major prerequisite for the safe application of opaque deep learning models in high-stakes domains like medicine. Understanding the decision-making process not only contributes to fostering trust but …
counterfactualImage GenerationNxPlain: Web-based Tool for Discovery of Latent Concepts
The proliferation of deep neural networks in various domains has seen an increased need for the interpretability of these models, especially in scenarios where fairness and trust are as important as model performance. A …
FairnessSentenceConceptX: A Framework for Latent Concept Analysis
The opacity of deep neural networks remains a challenge in deploying solutions where explanation is as important as precision. We present ConceptX, a human-in-the-loop framework for interpreting and annotating latent rep…