Evaluating Privacy Questions From Stack Overflow: Can ChatGPT Compete?
Stack Overflow and other similar forums are used commonly by developers to seek answers for their software development as well as privacy-related concerns. Recently, ChatGPT has been used as an alternative to generate code or produce responses to developers' questions. In this paper, we aim to understand developers' privacy challenges by evaluating the types of privacy-related questions asked on Stack Overflow. We then conduct a comparative analysis between the accepted responses given by Stack Overflow users and the responses produced by ChatGPT for those extracted questions to identify if ChatGPT could serve as a viable alternative. Our results show that most privacy-related questions are related to choice/consent, aggregation, and identification. Furthermore, our findings illustrate that ChatGPT generates similarly correct responses for about 56% of questions, while for the rest of the responses, the answers from Stack Overflow are slightly more accurate than ChatGPT.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questions
Q&A platforms have been crucial for the online help-seeking behavior of programmers. However, the recent popularity of ChatGPT is altering this trend. Despite this popularity, no comprehensive study has been conducted to…
MisinformationLLMs and Stack Overflow Discussions: Reliability, Impact, and Challenges
Since its release in November 2022, ChatGPT has shaken up Stack Overflow, the premier platform for developers queries on programming and software development. Demonstrating an ability to generate instant, human-like resp…
Language ModellingLarge Language ModelFrom Mundane to Meaningful: AI's Influence on Work Dynamics -- evidence from ChatGPT and Stack Overflow
This paper illustrates how generative AI could give opportunities for big productivity gains but also opens up questions about the impact of these new powerful technologies on the way we work and share knowledge. More sp…
Are Large Language Models a Threat to Digital Public Goods? Evidence from Activity on Stack Overflow
Large language models like ChatGPT efficiently provide users with information about various topics, presenting a potential substitute for searching the web and asking people for help online. But since users interact priv…
Just another copy and paste? Comparing the security vulnerabilities of ChatGPT generated code and StackOverflow answers
Sonatype's 2023 report found that 97% of developers and security leads integrate generative Artificial Intelligence (AI), particularly Large Language Models (LLMs), into their development process. Concerns about the secu…