Automatic Assessment of the Design Quality of Python Programs with Personalized Feedback
The assessment of program functionality can generally be accomplished with straight-forward unit tests. However, assessing the design quality of a program is a much more difficult and nuanced problem. Design quality is an important consideration since it affects the readability and maintainability of programs. Assessing design quality and giving personalized feedback is very time consuming task for instructors and teaching assistants. This limits the scale of giving personalized feedback to small class settings. Further, design quality is nuanced and is difficult to concisely express as a set of rules. For these reasons, we propose a neural network model to both automatically assess the design of a program and provide personalized feedback to guide students on how to make corrections. The model's effectiveness is evaluated on a corpus of student programs written in Python. The model has an accuracy rate from 83.67% to 94.27%, depending on the dataset, when predicting design scores as compared to historical instructor assessment. Finally, we present a study where students tried to improve the design of their programs based on the personalized feedback produced by the model. Students who participated in the study improved their program design scores by 19.58%.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Automatic Generation of Python Programs Using Context-Free Grammars
In recent years, data has emerged as the new gold, serving as a powerful tool for creating intelligent systems. However, procuring high-quality data remains challenging, especially for code. To address this, we developed…
Code GenerationA machine-learning framework for daylight and visual comfort assessment in early design stages
This research is mainly focused on the assessment of machine learning algorithms in the prediction of daylight and visual comfort metrics in the early design stages. A dataset was primarily developed from 2880 simulation…
Learning Transformer Programs
Recent research in mechanistic interpretability has attempted to reverse-engineer Transformer models by carefully inspecting network weights and activations. However, these approaches require considerable manual effort a…
In-Context LearningInterpretable Machine Learningnamed-entity-recognitionNamed Entity Recognition+2DiSciPLE: Learning Interpretable Programs for Scientific Visual Discovery
Visual data is used in numerous different scientific workflows ranging from remote sensing to ecology. As the amount of observation data increases, the challenge is not just to make accurate predictions but also to u…
Common Sense ReasoningDensity EstimationAI Coders Are Among Us: Rethinking Programming Language Grammar Towards Efficient Code Generation
Artificial Intelligence (AI) models have emerged as another important audience for programming languages alongside humans and machines, as we enter the era of large language models (LLMs). LLMs can now perform well in co…
Code GenerationMath