出版日期：2024-05-11 00:00:00

著者：Lin, Yi-Ti; Iain Kelsall Brown

著錄名稱、卷期、頁數：Education and Information Technologies 29,　p.22427-22445

摘要：To determine the impacts of using ChatGPT to assist English as a foreign language (EFL) English college majors in revising essays and the possibility of leading to higher scores and potentially causing unfairness. A prospective, double-blinded, paired-comparison study was conducted in Feb. 2023. A total of 44 students provided 44 original essays and 44 ChatGPT-assisted revised essays, which were rated by two independent graders in a randomized and crossover fashion to minimize grading bias. The original and revision scores were paired for before-after comparison. Eight control essays were also rated by both graders to ensure inter-rater reliability. This study used a rigorous experimental design to confirm that ChatGPT-assisted revised essays led to significantly higher scores for EFL college English majors. Significant improvements were observed in all four dimensions of writing quality assessment, with the largest effects observed in vocabulary, followed by grammar, organization, and content. ChatGPT-assisted revised essays shifted the score curve from a normal distribution to a skewed distribution towards higher grades, with the greatest increase in revision scores seen among students who had lower original scores. This disproportionate improvement raises concerns about fairness in evaluation. The findings suggest that ChatGPT is effective in providing timely feedback to EFL English majors in an affordable manner, but it also highlights the potential for unfairness in writing evaluation. We should note that ChatGPT-assisted revisions do not reveal learners’ writing competence. Therefore, new forms of writing performance assessment should be implemented in EFL composition classes in this AI era.

關鍵字：ChatGPT;Artificial intelligence;AI-assisted writing;EFL writing;NLP

語言：en_US

ISSN：1573-7608

期刊性質：國外

收錄於：SSCI

審稿制度：是

國別：USA

出版型式：電子版