Ryo KUROSE    Department   Kyoto University of Foreign Studies  ,    Position   Assistant Professor-lecturer
Date	2025/05/17
Presentation Theme	Comparing JLPT and ChatGPT-generated reading comprehension items
Conference	PanSIG 2025
Promoters	Japan Association for Language Teaching (JALT)
Conference Type	Domestic
Presentation Type	Speech (General)
Contribution Type	Collaborative
Venue	Kanda University of International Studies
Publisher and common publisher	Gilbert Dizon・Ryo Kurose
Details	The study examines the use of generative AI in language assessment by comparing JLPT reading comprehension items created by human experts with those generated by ChatGPT. Using sample JLPT materials as a basis, AI-generated passages and test items were developed and evaluated by Japanese speakers through a blind survey. The results showed no significant differences in the quality of the reading comprehension items. One possible explanation is that participants were not aware of the study’s exact purpose and may have assumed that all passages were created using generative AI. The findings suggest that ChatGPT can be used to create mock JLPT materials to save time and cost; however, it should not be relied upon as the sole resource.