Which AI Technique Is Better to Classify Requirements? An Experiment with SVM, LSTM, and ChatGPT
Recently, Large Language Models like ChatGPT have demonstrated remarkable proficiency in various Natural Language Processing tasks. Their application in Requirements Engineering, especially in requirements classification, has gained increasing interest. This paper reports an extensive empirical evaluation of two ChatGPT models, specifically gpt-3.5-turbo, and gpt-4 in both zero-shot and few-shot settings for requirements classification. The question arises as to how these models compare to traditional classification methods, specifically Support Vector Machine and Long Short-Term Memory. Based on five different datasets, our results show that there is no single best technique for all types of requirement classes. Interestingly, the few-shot setting has been found to be beneficial primarily in scenarios where zero-shot results are significantly low.
Code (1)
Tasks
ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Classification of Natural Language Processing Techniques for Requirements Engineering
Research in applying natural language processing (NLP) techniques to requirements engineering (RE) tasks spans more than 40 years, from initial efforts carried out in the 1980s to more recent attempts with machine learni…
ClassificationResource-Efficient Computing in Wearable Systems
We propose two optimization techniques to minimize memory usage and computation while meeting system timing constraints for real-time classification in wearable systems. Our method derives a hierarchical classifier struc…
Activity RecognitionGeneral ClassificationIdentifying Performance-Sensitive Configurations in Software Systems through Code Analysis with LLM Agents
Configuration settings are essential for tailoring software behavior to meet specific performance requirements. However, incorrect configurations are widespread, and identifying those that impact system performance is ch…
RAGRetrieval-augmented GenerationClassifying Human Activities using Machine Learning and Deep Learning Techniques
Human Activity Recognition (HAR) describes the machines ability to recognize human actions. Nowadays, most people on earth are health conscious, so people are more interested in tracking their daily activities using Smar…
Activity RecognitionBIG-bench Machine LearningData VisualizationDeep Learning+3Content Type Profiling of Data-to-Text Generation Datasets
Data-to-Text Generation (D2T) problems can be considered as a stream of time-stamped events with a text summary being produced for each. The problem becomes more challenging when event summaries contain complex insights …
Data-to-Text GenerationText GenerationVocal Bursts Type Prediction