AI as a Thinking Partner: How Schools Can Foster Real Learning
A week-long AI pilot program with refugee students in Chicago revealed that AI tools designed to encourage critical thinking lead to better long-term retention. Students performed better on quizzes when AI engaged them in Socratic questioning rather than simply summarizing content. The findings highlight the need for AI literacy in schools to ensure equitable access and effective use of these tools.
A recent pilot program in Chicago demonstrated how artificial intelligence can transform learning for students who often face significant educational challenges. Over a one-week session at Forging Opportunities for Refugees in America, 22 students aged 11 to 17-many of whom had experienced interrupted schooling-engaged with AI in different ways to deepen their understanding of brain science. The results underscored the importance of using AI not as a shortcut but as a tool to foster critical thinking.
## AI as a Thinking Partner vs. a Shortcut
Initially, some students resisted the AI’s approach, expecting quick answers rather than questions. One teen, frustrated with Gemini’s questioning style, preferred ChatGPT, which she believed provided straightforward responses. However, when she discovered the intentional design behind Gemini’s prompts-meant to encourage reflection-her perspective shifted. By the end of the week, she scored perfectly on a quiz and took her time with lessons, a marked change from her usual approach.
The pilot tested three AI methods:
| Method | Same-Day Quiz Score | Next-Day Quiz Score | |-----------------------|---------------------|---------------------| | AI Summary | 72% | 67% | | Socratic Q&A | 74% | 83% | | Student-Driven Questions | 70% | 75% |
While same-day performance was similar across methods, students who engaged in Socratic questioning retained more knowledge the following day. Over two-thirds of participants reported that the questioning approach helped them learn more effectively than passive summaries.
## The Gap Between Confidence and Retention
Interestingly, students’ confidence in using AI grew significantly over the week, but this did not correlate with their actual retention of material. This disconnect highlights a critical issue: feeling like one has learned is not the same as truly learning. As schools invest heavily in AI education technology, success must be measured by long-term retention, not just engagement or satisfaction.
## The Need for AI Literacy in Schools
The students also revealed a pressing need for AI literacy education. Many struggled to get explanations at their grade and language level, finding it harder to use AI effectively than to study independently. This inequity risks leaving behind the very students who could benefit most from personalized learning. Without structured guidance, students from less tech-savvy backgrounds may fall further behind their peers.
The pilot underscores the potential of AI to enhance learning when used as a thinking partner rather than a crutch. Schools must prioritize teaching students how to leverage AI effectively, ensuring equitable access and meaningful educational outcomes.
