‘Best Evidence in Brief’ is a
fortnightly email newsletter “brought to you by the Johns Hopkins School of
Education's Center for Research and Reform in Education and the University of
York's Institute for Effective Education”, both led by Bob Slavin. In the
latest issue (January 30, 2013 http://tinyurl.com/a9bn66a), there is an interesting item entitled 'Lasting effects from Reading
Recovery' citing a recent report (dated December, 2012) by Jane Hurry from the
Institute of Education, University of London (the British home of Reading Recovery).
Hurry’s report is entitled ‘The impact of Reading Recovery five years after intervention’
(http://tinyurl.com/avhubv9).
This is how ‘Best Evidence in Brief ‘ reports the
study (with a link to the report):
“A recent report for the Every Child a Reader Trust looks at the impact of Reading
Recovery five years after intervention. The program is known to have impressive
effects in the short term, but less is known about its long-term effectiveness.
This study suggests that the substantial gains which result from receiving
Reading Recovery in Year 1 (the UK equivalent of kindergarten) continue to the
end of primary school.
At the end of Year 6 (the UK equivalent of 5th grade),
the study followed up 77 children who had received Reading Recovery five years
earlier, 127 comparison children, and 50 children in Reading Recovery schools
who had not received Reading Recovery. Findings showed that children who had
received Reading Recovery made significantly greater progress in English than
the comparison children by the end of Year 6. The 50 comparison children in the
Reading Recovery schools were also significantly out-performing the comparison
group in non-Reading Recovery schools on the reading test.”
It would be reasonable to conclude from this summary
(published by two research centres apparently devoted to championing
evidence-based practice) that Reading Recovery had already been shown,
unequivocally, to be effective in the short term and that now there was convincing
evidence for its efficacy in the longer term too, at the end of Year 6, in
fact.
Many reading scientists are less
convinced of the Efficacy of Reading Recovery in the short term; see, for
example, Reynolds and Wheldall, 2007: http://tinyurl.com/bggyl46
But let us
concentrate, for the present purposes, on the reporting of longer term effects
as reported by Hurry and summarised by Best Evidence in Brief. This is how
Hurry, herself, summarises her findings in her report:
“Reading Recovery is part of the Every Child a Reader strategy to enable
children to make a good start in reading. Reading Recovery is well known to
have impressive effects in the shorter term but less is known about its long
term effectiveness. The present study followed up at the end of Year 6: 127
comparison children, 77 children who had received Reading Recovery five years
earlier and 50 children in Reading Recovery schools who had not receive Reading
Recovery. The children who had received Reading Recovery had made significantly
greater progress in English than the comparison children by the end of Year 6,
achieving on average a National Curriculum Level of 4b compared with a
borderline between Level 3 and 4 in the comparison group. Comparison children
in the Reading Recovery schools were also significantly outperforming the
comparison in non Reading Recovery schools on the reading test. 78% of Reading
Recovery children achieved Level 4 in English compared with 62% in the comparison
group in non Reading Recovery schools and 64% for the comparison children in
Reading Recovery schools. There was a tendency for Reading Recovery children to
be receiving less SEN provision than children in the other two groups, but this
only reached statistical significance for those on School Action Plus or a
Statement. This suggests that the substantial gains which result from receiving
Reading Recovery in Year 1 continue to deliver a significant advantage for
those children at the end of the primary phase, providing a surer footing for
transition to secondary school.” (P. 3)
If you read the details
in the actual report, however, the facts for the end of Year 6 results are reported
by the author (Hurry) as follows:
“The Reading Recovery children were still doing
significantly better in reading (β=.191, p<.005), effect size (Cohen’s
d) = .39) and writing (β=.162, p<.013, effect size (Cohen’s d) = .33) than the comparison children in non Reading
Recovery schools. They were also scoring significantly higher on
their maths test (β=.154, p<.036, effect size (Cohen’s d) = .31). However, they were not significantly better than the comparison children from
the Reading Recovery schools on any of the measures (reading, writing or
maths). Indeed the comparison
children from Reading Recovery schools, ie. those that were poor readers
at six but did not receive the programme, were also doing significantly
better in reading than the comparison children from non RR schools (β=.222,
p<.002, effect size (Cohen’s d) = .24).” (p.12) (present author’s emphasis)
So, let’s be clear about this. The children who
received Reading Recovery did not perform significantly better than the
comparison students from the same school
who had not received Reading
Recovery. (Note: no details of statistical significance testing or effect sizes
are reported for these comparisons.)
But the Conclusions section of the report states (and
this is the only conclusion iterated):
“These findings indicate that effects of Reading
Recovery are still apparent at the end of Year 6 and that even the children who
attended Reading Recovery schools but were not offered the programme benefited
somewhat from the ECaR programme.” (P. 22)
It is a source of some consternation to reflect on the
fact that neither the report’s author (Hurry) nor the writers of ‘Best Evidence
in Brief’ appear to have considered the (to me) obvious alternative conclusion:
that there is no need to actually take part in Reading Recovery; merely
attending a Reading Recovery school appears to be sufficient!
Another interpretation of these data is that they
provide no evidence for the long term
efficacy of Reading Recovery because those children in the school who did
receive Reading Recovery performed no better than those who did not. And both of these groups performed better
than the comparison children in the non-Reading Recovery schools. In other
words, there is no discernable effect for the program per se, only for
differences between schools. Moreover, even the significant differences between
the two groups in the Reading Recovery Schools and the non-Reading Recovery
school are accompanied by only small effect sizes, all of which are below
Hattie’s hinge value of 0.4
Considering the huge expense involved in one to one
Reading Recovery tutoring, these are very small bangs for very big bucks.
