The effect of ChatGPT on students’ learning performance: insights from a meta-analysis
A recent meta-analysis published in Humanities and Social Sciences Communications provides compelling evidence on the impact of ChatGPT in educational settings. Analysing 51 studies conducted between November 2022 and February 2025, researchers Jin Wang and Wenxiang Fan found that ChatGPT significantly enhances students’ academic performance, learning perception, and higher-order thinking skills.
Key Findings
Academic Performance
The analysis revealed a large positive effect of ChatGPT on students’ learning performance, with an effect size of g = 0.867. This suggests that integrating ChatGPT into learning environments can lead to substantial improvements in academic outcomes.
Learning Perception
Students reported a moderately positive perception of their learning experiences when using ChatGPT (g = 0.456). This indicates increased engagement and satisfaction, likely due to the tool’s ability to provide instant feedback and personalized support.
Higher-Order Thinking Skills
ChatGPT also fosters higher-order thinking skills, such as analysis and evaluation, with a moderate effect size (g = 0.457). Notably, the impact was more pronounced in STEM courses (g = 0.737) compared to language learning and skills development courses.
Influencing Factors
The effectiveness of ChatGPT varies depending on several factors:
- Course Type: STEM courses benefit more significantly from ChatGPT integration.
- Learning Model: Problem-based learning models show enhanced outcomes when combined with ChatGPT.
- Duration of Use: Optimal benefits are observed with continuous use over 4–8 weeks.
- Role of ChatGPT: Its effectiveness increases when used flexibly as a tutor, learning partner, or educational tool.
Recommendations for Educators
Based on these findings, the study suggests:
- Curriculum Integration: Incorporate ChatGPT across various subjects to support diverse learning needs.
- Educational Frameworks: Utilize frameworks like Bloom’s taxonomy to guide the development of higher-order thinking skills.
- Flexible Implementation: Employ ChatGPT in multiple roles to enhance its utility in the classroom.
- Sustained Usage: Maintain consistent use over several weeks to achieve stable and significant learning improvements.
Implications for Generative AI Education
This meta-analysis underscores the transformative potential of generative AI tools like ChatGPT in education. For initiatives focused on generative AI education, these insights highlight the importance of:
- Curriculum Development: Designing courses that incorporate AI tools to enhance learning outcomes.
- Teacher Training: Equipping educators with the skills to effectively integrate AI into their teaching strategies.
- Policy Formulation: Establishing guidelines that support the ethical and effective use of AI in education.
By embracing these strategies, educational institutions can harness the benefits of generative AI to foster improved student performance, engagement, and critical thinking.
For a detailed exploration of the study, refer to the full article: https://www.nature.com/articles/s41599-025-04787-y
