Studying the Effects of Collaboration in Interactive Theme Discovery Systems
NLP-assisted solutions have gained considerable traction to support qualitative data analysis. However, there does not exist a unified evaluation framework that can account for the many different settings in which qualitative researchers may employ them. In this paper, we take a first step in this direction by proposing an evaluation framework to study the way in which different tools may result in different outcomes depending on the collaboration strategy employed. Specifically, we study the impact of synchronous vs. asynchronous collaboration using two different NLP-assisted qualitative research tools and present a comprehensive analysis of significant differences in the consistency, cohesiveness, and correctness of their outputs.
Code (1)
Similar Papers 제목 키워드 기반
Interactive Concept Learning for Uncovering Latent Themes in Large Text Collections
Experts across diverse disciplines are often interested in making sense of large text collections. Traditionally, this challenge is approached either by noisy unsupervised techniques such as topic models, or by following…
Topic ModelsTowards Effective Human-AI Collaboration in GUI-Based Interactive Task Learning Agents
We argue that a key challenge in enabling usable and useful interactive task learning for intelligent agents is to facilitate effective Human-AI collaboration. We reflect on our past 5 years of efforts on designing, deve…
Searching for Synergy in Shared Workspace Human-AI Collaboration
Automated AI agents are increasingly capable, yet many scientific and professional tasks require human judgment and contextual expertise. We use simulated shared-workspace human-AI teams as a controlled testbed for study…
Enhancing Reading Strategies by Exploring A Theme-based Approach to Literature Surveys
Searching large digital repositories can be extremely frustrating, as common list-based formats encourage users to adopt a convenience-sampling approach that favours chance discovery and random search, over meaningful ex…
ScholarMate: A Mixed-Initiative Tool for Qualitative Knowledge Work and Information Sensemaking
Synthesizing knowledge from large document collections is a critical yet increasingly complex aspect of qualitative research and knowledge work. While AI offers automation potential, effectively integrating it into human…