Harefield

Harefield
Harefield

Monday 4 February 2013

Small Bangs for Big Bucks: The long term efficacy of Reading Recovery


‘Best Evidence in Brief’ is a fortnightly email newsletter “brought to you by the Johns Hopkins School of Education's Center for Research and Reform in Education and the University of York's Institute for Effective Education”, both led by Bob Slavin. In the latest issue (January 30, 2013 http://tinyurl.com/a9bn66a), there is an interesting item entitled 'Lasting effects from Reading Recovery' citing a recent report (dated December, 2012) by Jane Hurry from the Institute of Education, University of London (the British home of Reading Recovery). Hurry’s report is entitled ‘The impact of Reading Recovery five years after intervention’ (http://tinyurl.com/avhubv9).
This is how ‘Best Evidence in Brief ‘ reports the study (with a link to the report):
“A recent report for the Every Child a Reader Trust looks at the impact of Reading Recovery five years after intervention. The program is known to have impressive effects in the short term, but less is known about its long-term effectiveness. This study suggests that the substantial gains which result from receiving Reading Recovery in Year 1 (the UK equivalent of kindergarten) continue to the end of primary school.
At the end of Year 6 (the UK equivalent of 5th grade), the study followed up 77 children who had received Reading Recovery five years earlier, 127 comparison children, and 50 children in Reading Recovery schools who had not received Reading Recovery. Findings showed that children who had received Reading Recovery made significantly greater progress in English than the comparison children by the end of Year 6. The 50 comparison children in the Reading Recovery schools were also significantly out-performing the comparison group in non-Reading Recovery schools on the reading test.”
It would be reasonable to conclude from this summary (published by two research centres apparently devoted to championing evidence-based practice) that Reading Recovery had already been shown, unequivocally, to be effective in the short term and that now there was convincing evidence for its efficacy in the longer term too, at the end of Year 6, in fact.
Many reading scientists are less convinced of the Efficacy of Reading Recovery in the short term; see, for example, Reynolds and Wheldall, 2007: http://tinyurl.com/bggyl46
But let us concentrate, for the present purposes, on the reporting of longer term effects as reported by Hurry and summarised by Best Evidence in Brief. This is how Hurry, herself, summarises her findings in her report:
“Reading Recovery is part of the Every Child a Reader strategy to enable children to make a good start in reading. Reading Recovery is well known to have impressive effects in the shorter term but less is known about its long term effectiveness. The present study followed up at the end of Year 6: 127 comparison children, 77 children who had received Reading Recovery five years earlier and 50 children in Reading Recovery schools who had not receive Reading Recovery. The children who had received Reading Recovery had made significantly greater progress in English than the comparison children by the end of Year 6, achieving on average a National Curriculum Level of 4b compared with a borderline between Level 3 and 4 in the comparison group. Comparison children in the Reading Recovery schools were also significantly outperforming the comparison in non Reading Recovery schools on the reading test. 78% of Reading Recovery children achieved Level 4 in English compared with 62% in the comparison group in non Reading Recovery schools and 64% for the comparison children in Reading Recovery schools. There was a tendency for Reading Recovery children to be receiving less SEN provision than children in the other two groups, but this only reached statistical significance for those on School Action Plus or a Statement. This suggests that the substantial gains which result from receiving Reading Recovery in Year 1 continue to deliver a significant advantage for those children at the end of the primary phase, providing a surer footing for transition to secondary school.” (P. 3)
If you read the details in the actual report, however, the facts for the end of Year 6 results are reported by the author (Hurry) as follows:
“The Reading Recovery children were still doing significantly better in reading (β=.191,  p<.005), effect size (Cohen’s d) = .39) and writing (β=.162, p<.013, effect size (Cohen’s d) = .33) than the comparison children in non Reading Recovery schools. They were also scoring significantly higher on their maths test (β=.154, p<.036, effect size (Cohen’s d) = .31). However, they were not significantly better than the comparison children from the Reading Recovery schools on any of the measures (reading, writing or maths). Indeed the comparison children from Reading Recovery schools, ie. those that were poor readers at six but did not receive the programme, were also doing significantly better in reading than the comparison children from non RR schools (β=.222, p<.002, effect size (Cohen’s d) = .24).” (p.12) (present author’s emphasis)
So, let’s be clear about this. The children who received Reading Recovery did not perform significantly better than the comparison students from the same school who had not received Reading Recovery. (Note: no details of statistical significance testing or effect sizes are reported for these comparisons.)
But the Conclusions section of the report states (and this is the only conclusion iterated):
“These findings indicate that effects of Reading Recovery are still apparent at the end of Year 6 and that even the children who attended Reading Recovery schools but were not offered the programme benefited somewhat from the ECaR programme.” (P. 22)
It is a source of some consternation to reflect on the fact that neither the report’s author (Hurry) nor the writers of ‘Best Evidence in Brief’ appear to have considered the (to me) obvious alternative conclusion: that there is no need to actually take part in Reading Recovery; merely attending a Reading Recovery school appears to be sufficient!
Another interpretation of these data is that they provide no evidence for the long term efficacy of Reading Recovery because those children in the school who did receive Reading Recovery performed no better than those who did not. And both of these groups performed better than the comparison children in the non-Reading Recovery schools. In other words, there is no discernable effect for the program per se, only for differences between schools. Moreover, even the significant differences between the two groups in the Reading Recovery Schools and the non-Reading Recovery school are accompanied by only small effect sizes, all of which are below Hattie’s hinge value of 0.4
Considering the huge expense involved in one to one Reading Recovery tutoring, these are very small bangs for very big bucks.