LoSoNA: A Benchmark for Local Social Norm Adaptation in Group Conversations
Online group chats are social spaces with local conversational norms that are rarely stated explicitly. The ability and willingness of LLM-based agents to recognize and adapt to these norms remains mostly unexplored. We introduce LoSoNA, a benchmark for local social norm adaptation in multi-party chat. Each scenario gives a subject model a curated group-chat transcript in which non-subject participants demonstrate a hidden local norm, followed by a final elicitor turn that forces a response revealing whether the subject has inferred that norm. We evaluate eight frontier and open-weight models under four prompting conditions that vary how explicitly the model is told to treat the prior conversation as evidence for how it should answer. Naive prompting remains limited for most models; explicit norm-aware prompting helps unevenly, with Gemini 3.1 Pro reaching $84.2\%$ and Claude Fable 5 reaching $81.6\%$, while several other models show small gains or regressions. LoSoNA contributes to recent calls for evaluating LLM social capabilities by testing whether models can infer local conversational norms from precedent and use them in a one-turn group-chat response.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
SCENE: Recognizing Social Norms and Sanctioning in Group Chats
Online group chats are social spaces with implicit behavior patterns that, when broken, are often met with social sanctioning from the group. The ability and willingness of LLM-based agents to recognize and adapt to thes…
CFA: Coupled-hypersphere-based Feature Adaptation for Target-Oriented Anomaly Localization
For a long time, anomaly localization has been widely used in industries. Previous studies focused on approximating the distribution of normal features without adaptation to a target dataset. However, since anomaly local…
Anomaly DetectionAnomaly LocalizationTransfer LearningUnsupervised Anomaly DetectionParser Adaptation for Social Media by Integrating Normalization
This work explores different approaches of using normalization for parser adaptation. Traditionally, normalization is used as separate pre-processing step. We show that integrating the normalization model into the parsin…
Domain AdaptationNamed Entity Recognition (NER)Sentiment AnalysisKoBBQ: Korean Bias Benchmark for Question Answering
The Bias Benchmark for Question Answering (BBQ) is designed to evaluate social biases of language models (LMs), but it is not simple to adapt this benchmark to cultural contexts other than the US because social biases de…
Question AnsweringLocal Media and the Shaping of Social Norms: Evidence from the Ebola outbreak
Media around the world is disseminated at the national level as well as at the local level. While the capacity of media to shape preferences and behavior has been widely recognized, less is known about the differential i…