What Can 1,000 Maths Students Teach Us About Using AI Intentionally?

AI can help a student solve a maths problem in seconds.

But what happens when the AI is taken away?

A study involving nearly 1,000 high school maths students in Turkey provides an interesting answer. Students performed significantly better on practice problems when they had access to generative AI. But when that support was removed, the results depended on the type of AI they had used.

Students using AI without learning-focused safeguards subsequently performed worse than students who hadn’t used AI at all. Those using an AI tutor with safeguards, however, did not experience the same negative effect.

The study raises an important question for schools adopting AI:

Are we using technology to help students learn, or simply helping them perform while the technology is there?

That’s where intentional technology use matters.

What happened when students in Turkey were given AI?

In a 2025 study published in the Proceedings of the National Academy of Sciences, researchers investigated how generative AI affected maths learning among nearly 1,000 students in Years 9, 10 and 11 at a high school in Turkey.

Students were assigned to three different conditions.

One group completed maths activities without generative AI. A second used GPT Base, a GPT-4 interface resembling a standard AI chatbot. A third used GPT Tutor, which used the same underlying model but included safeguards designed with teacher input to support learning.

During practice, both AI groups performed substantially better.

Students using GPT Base scored 48% higher than the control group, while students using GPT Tutor scored 127% higher.

At first glance, the conclusion seems obvious: give students AI and their maths performance improves.

But the researchers then took the AI away.

What happened when the AI was taken away?

Students completed an exam without access to AI, their notes or their textbooks.

This time, students who had used GPT Base scored 17% lower than students who had learned without generative AI.

The same significant negative effect wasn’t found for GPT Tutor. Students who had used the safeguarded tutor performed similarly to the control group on the independent assessment.

Importantly, this doesn’t mean GPT Tutor made students perform better than the control group. Rather, its safeguards largely mitigated the negative learning effect associated with GPT Base.

The students’ performance while using technology didn’t necessarily tell us what they had learned.

Why did the type of AI matter?

Both GPT Base and GPT Tutor used GPT-4. The difference was how they were designed to support students.

Researchers found that students often used GPT Base as a “crutch”, asking for and copying solutions. GPT Tutor, however, was designed to provide hints rather than immediately revealing answers. It also incorporated teacher-developed guidance about common mistakes and how to address them.

The goal wasn’t to remove difficulty whenever students got stuck. It was to help them work through it.

Just as an effective teacher might identify a misconception or ask a guiding question, a thoughtfully designed AI tutor can provide support while keeping students actively involved in solving the problem.

The support is there, but the thinking stays with the student.

Why is personalised tutoring so valuable?

In 1984, educational psychologist Benjamin Bloom published research exploring the effectiveness of one-to-one tutoring.

His findings showed that students receiving individual tutoring performed approximately two standard deviations above students taught through conventional classroom instruction.

This became known as Bloom's 2 Sigma Problem.

The challenge was finding a way to provide similarly effective individualised support to every student without requiring a dedicated tutor for each learner.

Today, AI presents an opportunity to make personalised tutoring more accessible.

A thoughtfully designed AI tutor can provide guidance when students get stuck, respond to mistakes and offer support when teachers aren’t immediately available.

But as the Turkey study demonstrates, providing access to AI isn’t enough. How that support is designed matters.

What does this tell us about intentional technology use?

The Turkey study highlights an important distinction: students’ performance with technology doesn’t always reflect what they’ve learned.

For teachers and school leaders, that raises a question worth asking:

What will students be able to do when the technology isn’t there?

Intentional technology use means starting with the learning outcome, then deciding how technology can support students in achieving it.

Sometimes that means providing personalised guidance. Other times, it means giving students the opportunity to demonstrate their understanding independently.

The goal isn’t simply to make learning easier. It’s to provide the right support at the right time.

What does intentional AI support look like in Mathspace?

This same principle shapes how we approach AI at Mathspace.

Milo AI is designed as a tutor, not an answer engine.

Teachers can’t be beside every student at every moment. When students get stuck, Milo can provide guidance that helps them work through challenges while keeping the thinking in their hands.

But intentional technology use also means knowing when to step back.

Mathspace’s Test Mode removes the hints, worked examples and other learning support available during regular practice, allowing students to demonstrate their understanding independently. Teachers can then see not just whether students reached the correct answer, but where their reasoning may have gone wrong.

This reflects an important lesson from the Turkey study: effective AI support isn’t just about helping students succeed during practice, but ensuring they can apply what they’ve learned without it.

Want to explore how supported practice and independent learning work together?

In our webinar, former maths teacher Adrian Chan explores how to balance guided practice with independent problem-solving, and how Mathspace’s Test Mode can help teachers identify what students truly understand.

The goal isn’t more technology. It’s more intentional technology.

The Turkey study doesn’t tell us that AI is inherently good or bad for maths learning. It shows why the way technology is designed and used matters.

For schools, the question shouldn’t simply be what an AI tool can do, but what it encourages students to do.

Does it help them understand their mistakes? Keep thinking when they get stuck? Build the confidence to solve problems independently?

AI has the potential to make personalised learning support more accessible, but its value depends on how intentionally that support is designed.

Because the goal isn’t to create students who are better at maths when AI is beside them. It’s to use technology intentionally to help students become more capable, independent learners.