Quality Evaluation of COBOL to Java Code Transformation
We present an automated evaluation system for assessing COBOL-to-Java code translation within IBM's watsonx Code Assistant for Z (WCA4Z). The system addresses key challenges in evaluating LLM-based translators, including model opacity and the complexity of translation quality assessment. Our approach combines analytic checkers with LLM-as-a-judge (LaaJ) techniques to deliver scalable, multi-faceted evaluations. The system supports continuous integration workflows, enables large-scale benchmarking, and reduces reliance on manual review. We describe the system architecture, evaluation strategies, and reporting mechanisms that provide actionable insights for developers and project managers, facilitating the evolution of high-quality, modernized codebases.
Code (0)
등록된 구현이 없습니다.
Tasks
Code TranslationSimilar Papers 제목 키워드 기반
Automated Testing of COBOL to Java Transformation
Recent advances in Large Language Model (LLM) based Generative AI techniques have made it feasible to translate enterprise-level code from legacy languages such as COBOL to modern languages such as Java or Python. While …
Language ModelingLanguage ModellingLarge Language ModelCode Reborn AI-Driven Legacy Systems Modernization from COBOL to Java
This study investigates AI-driven modernization of legacy COBOL code into Java, addressing a critical challenge in aging software systems. Leveraging the Legacy COBOL 2024 Corpus -- 50,000 COBOL files from public and ent…
Agentic Method for Deterministic Validation of Legacy Code Migration
Migration of legacy COBOL programs to Java requires extensive testing to ensure correct functionality. This effort is often complicated by the lack of test data and the difficulty of validating all corner cases. In this …
Evaluating perturbation robustness of generative systems that use COBOL code inputs
Systems incorporating large language models (LLMs) as a component are known to be sensitive (i.e., non-robust) to minor input variations that do not change the meaning of the input; such sensitivity may reduce the system…
Code GenerationRobustness and Reasoning Fidelity of Large Language Models in Long-Context Code Question Answering
Large language models (LLMs) increasingly assist software engineering tasks that require reasoning over long code contexts, yet their robustness under varying input conditions remains unclear. We conduct a systematic stu…
Question Answering