Generalization Issues in Conjoint Experiment: Attention and Salience
Can the causal effects estimated in an experiment be generalized to real-world scenarios? This question lies at the heart of social science studies. External validity primarily assesses whether experimental effects persist across different settings, implicitly presuming the consistency of experimental effects with their real-life counterparts. However, we argue that this presumed consistency may not always hold, especially in experiments involving multi-dimensional decision processes, such as conjoint experiments. We introduce a formal model to elucidate how attention and salience effects lead to three types of inconsistencies between experimental findings and real-world phenomena: amplified effect magnitude, effect sign reversal, and effect importance reversal. We derive testable hypotheses from each theoretical outcome and test these hypotheses using data from various existing conjoint experiments and our own experiments. Drawing on our theoretical framework, we propose several recommendations for experimental design aimed at enhancing the generalizability of survey experiment findings.
Code (0)
등록된 구현이 없습니다.
Tasks
Experimental DesignSimilar Papers 제목 키워드 기반
Key predictors for climate policy support and political mobilization: The role of beliefs and preferences
Public support and political mobilization are two crucial factors for the adoption of ambitious climate policies in line with the international greenhouse gas reduction targets of the Paris Agreement. Despite their compo…
Learning Conjoint Attentions for Graph Neural Nets
In this paper, we present Conjoint Attentions (CAs), a class of novel learning-to-attend strategies for graph neural networks (GNNs). Besides considering the layer-wise node features propagated within the GNN, CAs can ad…
BenchmarkingGraph AttentionSalience DETR: Enhancing Detection Transformer with Hierarchical Salience Filtering Refinement
DETR-like methods have significantly increased detection performance in an end-to-end manner. The mainstream two-stage frameworks of them perform dense self-attention and select a fraction of queries for sparse cross-att…
2D Object DetectionComputational EfficiencyDense Object DetectionObject DetectionSalience Estimation with Multi-Attention Learning for Abstractive Text Summarization
Attention mechanism plays a dominant role in the sequence generation models and has been used to improve the performance of machine translation and abstractive text summarization. Different from neural machine translatio…
Abstractive Text SummarizationDecoderMachine TranslationText Summarization+1JCDNet: Joint of Common and Definite phases Network for Weakly Supervised Temporal Action Localization
Weakly-supervised temporal action localization aims to localize action instances in untrimmed videos with only video-level supervision. We witness that different actions record common phases, e.g., the run-up in the High…
Action LocalizationMultiple Instance LearningTemporal Action LocalizationWeakly-supervised Learning+1