2022/23 Effectiveness Trial Results
This trial took place across the autumn and winter terms of 2022/2023. It was delivered by the Behavioural Intervention Team (BIT) with occasional input from Birbeck College, and was independently evaluated by the National Centre for Social Research (NATCEN).
A total of 14,718 children in Years 3 and 5 coming from 173 primary schools in England took part in this project. There was a mix of single form and multiform entry schools reflecting both rural and urban settings.
This study consisted of a Randomised Control Trial designed to test the effectiveness of the Stop and Think computerised learning activity at boosting science and maths achievement in primary school. Half of the participating children were randomly allocated by the independent evaluator to the “Stop and Think” science and maths condition and half were randomly allocated to a business as usual “control” condition. Effectiveness trials are large scale trials that assess whether an intervention works under conditions that are as normal as possible. Hence the developers (Birkbeck and the UCL-Institute of Education) were not involved in delivering or assessing the intervention.
The trial was designed to assess the impact of Stop and Think on children receiving Free School Meals (FSM) first, with a secondary outcome of assessing the impact of Stop and Think on children overall. The effectiveness trial largely replicated the findings of the initial 2018/2019 efficiency randomised control trial. Both trials showed a positive impact on attainment for science. The first efficacy trial found an impact of two additional months' progress in science for pupils who had received the Stop and Think programme, and one additional months progress for maths.
The current effectiveness trial showed the same positive impact for science - pupils who took part in Stop and Think made, on average, two months of extra progress compared to those who had standard lessons. For children eligible for Free School Meals (FSM), the impact was one additional month of progress. However, overall, there isn't evidence to suggest pupils are differently impacted by the programme.
For maths, the effectiveness trial found that Stop and Think did not lead to any extra progress for either group compared to regular teaching. The different results in maths and science may be because maths already has a strong emphasis in the curriculum and makes greater use of certain teaching techniques, such as using “non-examples” to explain concepts. Teachers also reported positive classroom impacts, particularly in science, including greater reflection on teaching practice and a more inclusive learning environment for SEND and EAL pupils.
This finding is extremely exciting, especially when considering that this was a relatively short intervention, over 10 weeks, and the low cost of the intervention (calculated as around £14 per pupil over 3 years). The result were awarded a three padlock security rating by the Education Endowment Foundation (EEF), meaning that they are confident that this difference was due to the intervention and not to other factors. The EEF has listed Stop and Think as one of its promising programmes because of its impact on science achievement.
To read the full report, please visit the EEF website:
https://educationendowmentfoundation.org.uk/projects-and-evaluation/projects/stop-and-think-learning-counterintuitive-concepts-regrant
Finally… we wish to say a gigantic THANK YOU to all the schools, teachers and pupils who took part in this study. We can have the best ideas in the world, but without your participation, research into children's learning and better educational practice cannot progress!