Towards More Accountable Search Engines: Online Evaluation of Representation Bias
Information availability affects people's behavior and perception of the world. Notably, people rely on search engines to satisfy their need for information. Search engines deliver results relevant to user requests usually without being or making themselves accountable for the information they deliver, which may harm people's lives and, in turn, society. This potential risk urges the development of evaluation mechanisms of bias in order to empower the user in judging the results of search engines. In this paper, we give a possible solution to measuring representation bias with respect to societal features for search engines and apply it to evaluating the gender representation bias for Google's Knowledge Graph Carousel for listing occupations.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Auditing Search Engines for Differential Satisfaction Across Demographics
Many online services, such as search engines, social media platforms, and digital marketplaces, are advertised as being available to any user, regardless of their age, gender, or other demographic factors. However, there…
Causal InferenceEvaluating Robustness of Generative Search Engine on Adversarial Factual Questions
Generative search engines have the potential to transform how people seek information online, but generated responses from existing large language models (LLMs)-backed generative search engines may not always be accurate…
RetrievalRetrieval-augmented GenerationOnline information of vaccines: information quality is an ethical responsibility of search engines
The fact that internet companies may record our personal data and track our online behavior for commercial or political purpose has emphasized aspects related to online privacy. This has also led to the development of se…
Measuring Five Accountable Talk Moves to Improve Instruction at Scale
Providing consistent, individualized feedback to teachers on their instruction can improve student learning outcomes. Such feedback can especially benefit novice instructors who teach on online platforms and have limited…
The Use of Generative Search Engines for Knowledge Work and Complex Tasks
Until recently, search engines were the predominant method for people to access online information. The recent emergence of large language models (LLMs) has given machines new capabilities such as the ability to generate…