In-Context Clustering with Large Language Models

2025-10-09 · Ying Wang, Mengye Ren, Andrew Gordon Wilson arxiv

We propose In-Context Clustering (ICC), a flexible LLM-based procedure for clustering data from diverse distributions. Unlike traditional clustering algorithms constrained by predefined similarity measures, ICC flexibly captures complex relationships among inputs through an attention mechanism. We show that pretrained LLMs exhibit impressive zero-shot clustering capabilities on text-encoded numeric data, with attention matrices showing salient cluster patterns. Spectral clustering using attention matrices offers surprisingly competitive performance. We further enhance the clustering capabilities of LLMs on numeric and image data through fine-tuning using the Next Token Prediction (NTP) loss. Moreover, the flexibility of LLM prompting enables text-conditioned image clustering, a capability that classical clustering methods lack. Our work extends in-context learning to an unsupervised setting, showcasing the effectiveness and flexibility of LLMs for clustering. Our code is available at https://agenticlearning.ai/icc.

📄 PDF Abstract BibTeX arXiv:2510.08466

Code (0)

등록된 구현이 없습니다.

Tasks

Image Clustering

Context-Aware Clustering using Large Language Models

2024-05-02 · Sindhu Tipirneni, Ravinarayana Adkathimar, Nurendra Choudhary, Gaurush Hiranandani 외

Despite the remarkable success of Large Language Models (LLMs) in text understanding and generation, their potential for text clustering tasks remains underexplored. We observed that powerful closed-source LLMs provide g…

Clustering Language Modeling Language Modelling Text Augmentation+2

Contextual Phenotyping of Pediatric Sepsis Cohort Using Large Language Models

2025-05-14 · Aditya Nagori, Ayush Gautam, Matthew O. Wiens, Vuong Nguyen 외

Clustering patient subgroups is essential for personalized care and efficient resource use. Traditional clustering methods struggle with high-dimensional, heterogeneous healthcare data and lack contextual understanding. …

Clustering Large Language Model

Topic Modeling with Contextualized Word Representation Clusters

2020-10-23 · Laure Thompson, David Mimno

Clustering token-level contextualized word representations produces output that shares many similarities with topic models for English text collections. Unlike clusterings of vocabulary-level word embeddings, the resulti…

Clustering Topic Models Word Embeddings

Clustering-driven Memory Compression for On-device Large Language Models

2026-01-24 · Ondrej Bohdal, Pramit Saha, Umberto Michieli, Mete Ozay 외 arxiv

Large language models (LLMs) often rely on user-specific memories distilled from past interactions to enable personalized generation. A common practice is to concatenate these memories with the input prompt, but this app…

ZeroDL: Zero-shot Distribution Learning for Text Clustering via Large Language Models

2024-06-19 · Hwiyeol Jo, Hyunwoo Lee, Taiwoo Park

The recent advancements in large language models (LLMs) have brought significant progress in solving NLP tasks. Notably, in-context learning (ICL) is the key enabling mechanism for LLMs to understand specific tasks and g…

Clustering In-Context Learning Text Clustering