From Prompts to Scores: Generative AI Versus Human Grading in Writing for Standardized Language Testing

Rapid advancements in GenAI have revolutionized language learning, particularly in standard language test preparation, by providing learners with immediate, adaptive, and data-driven feedback on writing performance, enabling them to refine their linguistic accuracy, coherence, and argumentation through tailored guidance that closely mirrors the evaluation criteria of high-stakes assessments such as the IELTS. Therefore, this study compares language test scoring by human evaluators and GenAI for the IELTS Academic Writing Module. The research assessed four models—ChatGPT, Claude, Gemini, and DeepSeek—using 110 IELTS Task 1 and Task 2 essays and compared their performance to human markers through statistical analyses, including PCC, ICC, and MAE. The findings indicated varying degrees of alignment between GenAI and human scores, with DeepSeek exhibiting the highest correspondence with human assessments. A qualitative content analysis of the GenAI-generated comments on details, organization, and utility was performed. Notably, the ChatGPT provided comprehensive, well-organized, and actionable feedback. Claude offered a balanced approach, Gemini focused on strengths, and DeepSeek delivered more concise feedback, although less actionable. This study underscores the potential of GenAI as a supplementary tool for SDL in IELTS preparation and assessment while also highlighting limitations such as the depth of feedback and the risk of over-reliance on technology. Ethical considerations, including algorithmic bias, data privacy, and accessibility further underscore the need for human oversight in AI-driven assessments. Future research should investigate hybrid AI-human feedback models and conduct longitudinal studies on the impact of GenAI feedback on writing proficiency.