Small Bangs for Big Bucks: The long term efficacy of Reading Recovery

‘Best Evidence in Brief’ is a fortnightly email newsletter “brought to you by the Johns Hopkins School of Education's Center for Research and Reform in Education and the University of York's Institute for Effective Education”, both led by Bob Slavin. In the latest issue (January 30, 2013 http://tinyurl.com/a9bn66a), there is an interesting item entitled 'Lasting effects from Reading Recovery' citing a recent report (dated December, 2012) by Jane Hurry from the Institute of Education, University of London (the British home of Reading Recovery). Hurry’s report is entitled ‘The impact of Reading Recovery five years after intervention’ (http://tinyurl.com/avhubv9).

This is how ‘Best Evidence in Brief ‘ reports the study (with a link to the report):

“A recent report for the Every Child a Reader Trust looks at the impact of Reading Recovery five years after intervention. The program is known to have impressive effects in the short term, but less is known about its long-term effectiveness. This study suggests that the substantial gains which result from receiving Reading Recovery in Year 1 (the UK equivalent of kindergarten) continue to the end of primary school.

At the end of Year 6 (the UK equivalent of 5th grade), the study followed up 77 children who had received Reading Recovery five years earlier, 127 comparison children, and 50 children in Reading Recovery schools who had not received Reading Recovery. Findings showed that children who had received Reading Recovery made significantly greater progress in English than the comparison children by the end of Year 6. The 50 comparison children in the Reading Recovery schools were also significantly out-performing the comparison group in non-Reading Recovery schools on the reading test.”

It would be reasonable to conclude from this summary (published by two research centres apparently devoted to championing evidence-based practice) that Reading Recovery had already been shown, unequivocally, to be effective in the short term and that now there was convincing evidence for its efficacy in the longer term too, at the end of Year 6, in fact.

Many reading scientists are less convinced of the Efficacy of Reading Recovery in the short term; see, for example, Reynolds and Wheldall, 2007: http://tinyurl.com/bggyl46

But let us concentrate, for the present purposes, on the reporting of longer term effects as reported by Hurry and summarised by Best Evidence in Brief. This is how Hurry, herself, summarises her findings in her report:

“Reading Recovery is part of the Every Child a Reader strategy to enable children to make a good start in reading. Reading Recovery is well known to have impressive effects in the shorter term but less is known about its long term effectiveness. The present study followed up at the end of Year 6: 127 comparison children, 77 children who had received Reading Recovery five years earlier and 50 children in Reading Recovery schools who had not receive Reading Recovery. The children who had received Reading Recovery had made significantly greater progress in English than the comparison children by the end of Year 6, achieving on average a National Curriculum Level of 4b compared with a borderline between Level 3 and 4 in the comparison group. Comparison children in the Reading Recovery schools were also significantly outperforming the comparison in non Reading Recovery schools on the reading test. 78% of Reading Recovery children achieved Level 4 in English compared with 62% in the comparison group in non Reading Recovery schools and 64% for the comparison children in Reading Recovery schools. There was a tendency for Reading Recovery children to be receiving less SEN provision than children in the other two groups, but this only reached statistical significance for those on School Action Plus or a Statement. This suggests that the substantial gains which result from receiving Reading Recovery in Year 1 continue to deliver a significant advantage for those children at the end of the primary phase, providing a surer footing for transition to secondary school.” (P. 3)

If you read the details in the actual report, however, the facts for the end of Year 6 results are reported by the author (Hurry) as follows:

“The Reading Recovery children were still doing significantly better in reading (β=.191,  p<.005), effect size (Cohen’s d) = .39) and writing (β=.162, p<.013, effect size (Cohen’s d) = .33) than the comparison children in non Reading Recovery schools. They were also scoring significantly higher on their maths test (β=.154, p<.036, effect size (Cohen’s d) = .31). However, they were not significantly better than the comparison children from the Reading Recovery schools on any of the measures (reading, writing or maths). Indeed the comparison children from Reading Recovery schools, ie. those that were poor readers at six but did not receive the programme, were also doing significantly better in reading than the comparison children from non RR schools (β=.222, p<.002, effect size (Cohen’s d) = .24).” (p.12) (present author’s emphasis)

So, let’s be clear about this. The children who received Reading Recovery did not perform significantly better than the comparison students from the same school who had not received Reading Recovery. (Note: no details of statistical significance testing or effect sizes are reported for these comparisons.)

But the Conclusions section of the report states (and this is the only conclusion iterated):

“These findings indicate that effects of Reading Recovery are still apparent at the end of Year 6 and that even the children who attended Reading Recovery schools but were not offered the programme benefited somewhat from the ECaR programme.” (P. 22)

It is a source of some consternation to reflect on the fact that neither the report’s author (Hurry) nor the writers of ‘Best Evidence in Brief’ appear to have considered the (to me) obvious alternative conclusion: that there is no need to actually take part in Reading Recovery; merely attending a Reading Recovery school appears to be sufficient!

Another interpretation of these data is that they provide no evidence for the long term efficacy of Reading Recovery because those children in the school who did receive Reading Recovery performed no better than those who did not. And both of these groups performed better than the comparison children in the non-Reading Recovery schools. In other words, there is no discernable effect for the program per se, only for differences between schools. Moreover, even the significant differences between the two groups in the Reading Recovery Schools and the non-Reading Recovery school are accompanied by only small effect sizes, all of which are below Hattie’s hinge value of 0.4

Considering the huge expense involved in one to one Reading Recovery tutoring, these are very small bangs for very big bucks.

Preparing Pre-school Children for Learning to Read

If we are serious about ensuring that all children learn to read within their first few years of schooling, we should make sure that the basic building blocks of literacy are in place for all children when they begin formal schooling. The research shows clearly that children commencing school with both phonological awareness and well-developed general language skills are far more likely to learn to read easily and quickly. If all children were to receive a program of instruction in these essential pre-requisites in the year prior to commencing school, far fewer children would struggle to learn to read. It would also mean a levelling of the playing field so that all children, regardless of their family background, would be starting to learn to read from a more similar knowledge base. It is currently the case that many children from less advantaged home backgrounds beginning school are already way behind their more advantaged peers in these key pre-literacy skills.

The idea of teaching these skills to pre-school children may sound off-putting to some but there is no reason why these skills may not be taught effectively in an engaging and play-based way that is more appropriate for young children. An effective pre-literacy program for pre-school children should comprise instruction in the two key areas identified by research as the most important pre-requisite skills for learning to read. First, they should be engaged in games and play-based routines that teach systematically the skills of phonological awareness so that children come to school already able to break up words into their component sounds and to manipulate the sounds in words. The second key component is an emphasis on developing good oral language skills more generally, including explicit vocabulary instruction. The best means of achieving this is by structured storybook reading activities where children are encouraged to engage with the story being read, to answer questions about the story and to relate the events in the story to their own lives. A focus on these two prerequisite skill sets provides an excellent foundation for learning to read. This conceptualisation of what constitutes the best preparation for learning to read forms the basis for our new pre-school program, known as PreLit.

PreLit is an early literacy preparation program, designed to be delivered the year before children start formal schooling. It will also prove useful for teaching children who come to school without the necessary prerequisite skills in place. The purpose of the program is to lay the foundations for good phonological awareness and other language skills in young children, to facilitate literacy development in the early school years.  PreLit is particularly focused on improving the learning outcomes for those children considered at potential risk of long-term reading failure but will provide a good grounding in the key prerequisite skills for literacy for all children about to begin school. PreLit instruction is based on the findings of the accumulated research with this age group and will provide early childhood teachers with research-based teaching strategies and an effective model of delivery for the teaching of phonological awareness and oral language.  It is designed to complement a play-based learning environment through brief periods of daily instruction. 

Taking care of Matthew

Some 25 years ago now, Keith Stanovich popularised the concept of Matthew effects in reading. The Matthew Effect takes its name from a passage in the St Matthew Gospel:

“For unto every one that hath shall be given, and he shall have abundance: but from him that hath not shall be taken away even that which he hath” (Matthew 25:29)

Often translated as “the rich get richer and the poor get poorer”, this neat analogy clearly struck a chord with reading specialists. Because if you start off by learning to read well, you will indeed get richer and richer; certainly metaphorically, but probably quite literally too.

You will read far more than your less fortunate peers, you will have greater exposure to a wider variety of books, you will learn far more new vocabulary words, you will pass from learning to read to reading to learn much more quickly, and so on. If, on the other hand, you come to school ill-prepared for learning to read, with poor language skills as a result of social disadvantage for example, and you do not latch on to what reading is all about quickly, in that first year of schooling, you are likely to experience continuing failure throughout your educational trajectory. Because literacy underpins everything. In other words the gap between the reading haves and the reading have nots, the literacy rich and the literacy poor, widens over the years creating social division and perpetuating social disadvantage from one generation to the next. Teaching children to read, then, is not just an educational issue; it is a social justice issue.

And it is this concern with social justice that has motivated our work in MultiLit from the outset. Our aim has always been to help those who struggle to learn to read for whatever reason, but particularly those students from socially disadvantaged, culturally diverse and Indigenous backgrounds, as well as those with other special learning needs. Our non-categorical approach to teaching asserts that what all of these children need is effective scientific evidence-based instruction.

It is a source of continuing bewilderment to me, that so many professionals in education continue to resist the idea of direct, explicit and systematic instruction in phonics, while at the same time shedding crocodile tears about the plight of the disadvantaged children struggling in our schools. But they are often the cause of the problem. To use a medical term, the problem is iatrogenic; it is the ‘treatment’ the students are receiving, or rather the lack of effective instruction in schools, that is often the cause of the reading problem. Low-progress readers are frequently simply ‘instructional casualties’.

It is a concern with helping all children to learn to read, regardless of the apparent reason for their difficulties, that has been the driving force behind our work. This is why we launched the Making Up Lost Time In Literacy or MultiLit Initiative in 1995. We were initially concerned with trying to help older low-progress readers in the final years of primary school so that they could stand some chance of making a success of high school. The MultiLit Reading Tutor Program was our solution, a one to one program of direct, explicit and systematic instruction in the basics of reading comprising a focus on teaching word attack skills, teaching a bank of frequently occurring sightwords and providing a supportive context for book reading in which to generalise these newly acquired skills.

We worked with older low-progress readers because this is what we knew; this is what we had been researching for a number of years. But we were always aware that we were the ambulance picking up the instructional casualties while also lobbying for more effective initial reading instruction in schools. As we often said, our aim was to make ourselves redundant! We naively thought that the effort we put into pressing for the National Inquiry into the Teaching of Literacy might bring this about. But we all know what happened as a result of the Inquiry … very little indeed.

Whatever the case with initial reading instruction, and even if scientific evidence-based teaching of reading were to be introduced, there will always be a minority of children who will struggle to learn to read. We currently estimate that we should target the bottom 25% of students as being in need of additional remedial support. Consequently, it makes good sense to intervene early with children who are struggling to learn to read in their first year of schooling, before the Matthew effect becomes entrenched. Whatever reservations one might have about aspects of Reading Recovery (and we have certainly not been bashful about expressing our own reservations), the late Dame Professor Marie Clay deserves enduring credit for her pioneering work on the importance of intervening early with children who are seen still to be struggling in the key skills of learning to read after their first year of schooling.

While we can admire Marie Clay’s pioneering work, it has to be remembered that Reading Recovery has remained virtually unchanged in over thirty years now. An awful lot has happened in that time. We now know far more about how reading works and how it should best be taught. But we always thought that someone else would come up with an alternative program that would ideally be at least as effective but more cost effective than Reading Recovery, predicated as it is on daily one to one tutoring by a highly trained Reading Recovery Teacher. We waited but nothing happened and so, over five years ago now, we started to think about how MultiLit might be adapted to meet the needs of this much younger target group.

The MiniLit Early Literacy Intervention Program is the product of an ongoing program of research and development carried out by our specialist team of academic researchers and special educators. A continuing process of refinement by trial and revision was employed until the program met the stringent efficacy criteria to justify its release to the wider community. We do not just write and hope! This final published version of MiniLit entailed a major revision of previous experimental versions and represents what we believe to be scientific evidence-based best practice.

MiniLit is informed by the findings of scientific research, carried out over the past 40 years, into how reading works and how it may best be taught. It is also completely in accord with the recommendations of the various national reports into effective reading instruction that have emphasised the five key pillars of reading instruction (sometimes known as the ‘five big ideas’), namely: phonemic awareness, phonics, fluency, vocabulary and comprehension.

It consists of 80 carefully structured lessons (sufficient for at least two terms of instruction) in an easy to hard sequence and is divided into two levels, Levels 1 and 2, with 40 lessons at each level. There are three main components of each lesson: Sounds and Words Activities; Text Reading; and Story Book Reading. It is very important that all three of these components are taught daily, as specified in the manual.

But does it work? A recent analysis of the progress of 90 struggling young readers who had attended MiniLit programs for four days per week for 15 weeks showed that they had made substantial and statistically significant gains on all of the measures of reading and related skills with large effect sizes evident.  ‘Gold Standard‘ randomised control trials have also been completed supporting the efficacy of the new MiniLit program.

How do we ensure that all children learn to read?

            If all children are to learn to read to a good level of proficiency in the first few years of schooling, we need a clear plan to ensure that no child falls through the net. Such a plan must be both effective and cost-effective. It has become increasingly accepted in recent years that a three tier, phased model of reading instruction, known as Response to Intervention (or RtI), is the best means of achieving this.

            The three tier RtI model is predicated upon a first tier of exemplary initial instruction in reading for all students during their first year of schooling (Kindergarten in New South Wales). This first tier of instruction should essentially comprise the best scientific evidence-based instruction. To the layman, this sounds obvious but in many Australian schools a less effective implicit model of reading instruction has held sway for the last few decades. Much of this implicit approach to reading instruction is highly desirable as a bedrock upon which to build effective reading instruction, and it may be enough for a minority of children, but most will need direct, explicit and systematic instruction in the five pillars or ‘five big ideas’ of teaching reading : phonemic awareness, phonics, fluency, vocabulary and comprehension. What is often lacking in initial reading instruction, in particular, is effective, specific instruction in what is known as synthetic phonics; how to relate letters to sounds and to blend letter sounds into words.

            Even when afforded exemplary reading instruction, there will always be some children who take longer than others to catch on. It is important to identify these low-progress readers as early as possible so that they do not fall too far behind their peers as their difficulties compound. Children who do not learn to read in the first few years of schooling are typically destined to a school career of educational failure because reading underpins almost all subsequent learning. A safe strategy is to target students who fall into the bottom twenty five per cent of the population for remedial reading intervention, as soon as their difficulties become apparent. Students’ progress should be checked regularly, in order to provide remedial intervention for those who need it from the beginning of Year One, at the very latest.   

            The RtI model recommends that struggling readers, the low-progress readers who comprise the bottom quartile, should be offered more intensive Tier 2 intervention in small groups of three to four students. Again the instruction provided to these students should be based on what the scientific research evidence has shown to be most effective— essentially the same five big ideas of reading instruction but more intensive and more individualised. In small groups, teachers are able to be more responsive to the specific idiosyncratic needs of the students with whom they are working. Small group instruction can be just as effective as one-to-one instruction for children without severe reading difficulties.

            Even with a solid Tier 2 small group reading program in place, there will still be a very small number of students who ‘fail to thrive’, perhaps about 3-5% of all Year 1 students. These are the students for whom we should reserve Tier 3 one-to-one intensive reading instruction, preferably with a specialist reading teacher with a sound background in special education. The same five big ideas are still critical. What is different, of course, is the intensity of instruction. Having successfully taught the vast majority of Year 1 students the basics of learning to read by Tier 1 and, where necessary, Tier 2 small group teaching, it is a far more manageable proposition to provide these few remaining students with the individual reading support that they will need, for as long as they need it.

            With this three tier Response to Intervention model in place, predicated upon scientific evidence based reading instruction, almost all, if not all, children will become proficient readers. Of course, the RtI model does not stop at the end of Year 1, it is important to monitor reading progress closely for all students, especially for the first three years of schooling. But by employing these procedures rigorously and teaching scientifically, it is not too much to expect all of our children to learn to read.

Note: A longer version of this post is available at: http://tinyurl.com/7cr8ddo

About MiniLit

The MiniLit Early Literacy Intervention Program

What is MiniLit?

MiniLit stands for ‘Meeting Initial Needs In Literacy’ and is an early literacy intervention program. It is designed to be delivered daily, for one hour, to small groups of up to four Year 1 students who have struggled to make adequate progress in learning to read during their first year of schooling.

Who developed MiniLit?

MiniLit is the product of an ongoing program of research and development carried out by a specialist team of academic researchers and special educators, led by Professor Kevin Wheldall from Macquarie University. It has been under development for over five years. A continuing process of refinement by trial and revision was employed until the program met the stringent efficacy criteria to justify its release to the wider community. The published version of MiniLit entailed a major revision of previous experimental versions and represents scientific evidence-based best practice.

What is MiniLit based on?

MiniL it is informed by the findings of scientific research, carried out over the past 40 years, into how reading works and how it may best be taught. It is also in accord with the recommendations of national reports into effective reading instruction that have emphasised the five key pillars of reading instruction (sometimes known as the ‘five big ideas’), namely: phonemic awareness, phonics, fluency, vocabulary and comprehension. The relevant research and the findings of the national inquiries are reviewed in the following two refereed journal articles:

Reynolds, M., Wheldall, K., & Madelaine, A. (2010). Components of effective early reading interventions for young struggling readers. Australian Journal of Learning Difficulties, 19, 35-57.

Reynolds, M., Wheldall, K., & Madelaine, A. (2011).What recent reviews tell us about the efficacy of reading interventions for struggling readers in the early years of schooling. International Journal of Disability, Development and Education, 58, 257-286.

MiniLit is predicated on ten desiderata drawn from these sources:

1.     Intervention is timely and offered during the second year of formal schooling, as soon as it is identified that the student has ongoing difficulties that cannot be addressed by the regular classroom instruction.

2.     Instruction is delivered in small groups of up to four students.

3.     The program includes phonemic awareness, phonics, fluency, vocabulary and text comprehension.

4.     The main activities in phonemic awareness relate to learning to blend and segment and should ideally involve using letters once students are familiar with some letter-sound relationships.

5.     Phonics is taught through a synthetic approach.

6.     There are planned procedures for students to build automaticity in word recognition.

7.     Instruction is explicit and systematic.

8.     A well-trained teacher or a paraprofessional with teacher support delivers instruction.

9.     Sessions are frequent, preferably daily, and involve at least 20-30 minutes of intensive instruction.

10.  Assessment procedures and tools are available to identify struggling students and to monitor their progress.

What does MiniLit consist of?

MiniLit consists of 80 carefully structured lessons (sufficient for at least two terms of instruction) in an easy to hard sequence and is divided into two levels, Levels 1 and 2, with 40 lessons at each level. There are three main components of each lesson, all of which should be carried out on each occasion that a MiniLit lesson is taught. These three main components are:

·      Sounds and Words Activities

·      Text Reading, and

·      Story Book Reading

It is very important that all of these components are taught daily, as specified in the manual for the program:

MultiLit. (2011). MiniLit early literacy intervention program. Sydney: MultiLit Pty Ltd.

How are struggling students identified for participation in Minilit and how is their progress monitored?

Ideally, young struggling readers should be identified at the commencement of the second year of formal schooling. Preliminary benchmarks for early identification have been established on several literacy measures and are reported in:

 Reynolds, M., Wheldall, K., & Madelaine, A. (2011). Early identification of young struggling readers: Preliminary benchmarks for intervention for students in years one and two in schools in New South Wales. Australian Journal of Learning Difficulties, 20, (in press).

A weekly curriculum-based measure of early reading progress has also been developed to monitor the progress of struggling readers, reported in:

Reynolds, M., Wheldall, K. & Madelaine, A. (2009). Building the WARL: The development of the Wheldall Assessment of Reading Lists, a curriculum-based measure designed to identify young struggling readers and monitor their progress. Australian Journal of Learning Difficulties, 14, 89-111.

What is the evidence for the efficacy of MiniLit?

Successive iterations of the MiniLit program have been continually revised following efficacy trials. Reports of the early development trials of MiniLit are reported in the following journal articles:

Reynolds, M., Wheldall, K., & Madelaine, A. (2007). Developing a ramp to reading for at-risk year one students: A preliminary pilot study. Special Education Perspectives, 16, 39-69.

Reynolds, M., Wheldall, K, & Madelaine, A. (2007). ‘Meeting Initial Needs In Literacy’ (MINILIT): Why we need it, how it works and the results of pilot studies. Australian Journal of Special Education, 31, 147-158.

Reynolds, M., Wheldall, K, & Madelaine, A. (2007). ‘Meeting Initial Needs In Literacy’ (MINILIT): A ramp to MULTILIT for younger low-progress readers. Australian Journal of Learning Disabilities,12, 67-72.

A recent analysis of the progress of 90 struggling young readers who had attended MiniLit programs for four days per week for 15 weeks showed that they had made substantial and statistically significant gains (p<0.0005) on all of the measures of reading and related skills with large effect sizes evident (‘d’ ranged from 0.96 to 1.41, with a mean of 1.08). Randomised control trials have also been completed confirming the efficacy of the program.

Reynolds, M., Wheldall, K., & Madelaine, A. (2010). An experimental evaluation of the efficacy of an intervention for young struggling readers in year one. Special Education Perspectives, 19(2), 35-57.

Buckingham, J., Wheldall, K., & Beaman-Wheldall, R. (submitted for publication). A randomised control trial of a tier two small group intervention for young struggling readers. Sydney: Macquarie University Special Education Centre.