PIRLS before Swine: Or why Australia sucks at reading

John Lennon was renowned for his sharp, and oft times acidic, wit. When asked if Ringo was the best drummer in the world, he responded that Ringo was not even the best drummer in the Beatles! I was reminded of this when reviewing the latest (2011) results from the Progress in International Reading Literacy Study (or PIRLS). Not only were Australian students not the best readers in the world, they were not even the best readers among the English speaking nations surveyed. They were, in fact, the worst. We can take small comfort from the fact that New Zealand performed only marginally (and not significantly) better than Australia.

The PIRLS project essentially assesses reading comprehension by requiring students to read selected texts and then to answer questions about the material read. Year 4 students are assessed because this typically marks the point of transition from learning to read to reading to learn. (Note that 2011 was the first time that Australia had taken part in PIRLS.)

Overall, 45 countries were included in the study (excluding those countries who tested older or younger readers or who took part for their own internal benchmarking purposes, and whose results are not reported). Australia came 27^th in the league table of countries (with a mean score of 527), below all other English speaking countries and significantly lower than 21 other countries overall, including all other English speaking countries except New Zealand (mean score 531).

To put this in perspective let’s look at how some of these other English-speaking countries performed. Singapore (mean score 567), for example, came 4^th, one of the four top performing countries significantly above the others. Northern Ireland came fifth (mean score 558) and the United States came 6^th (mean score 556) (compared with 14th out of 40 in 2006). England came 11th (compared with 15th out of 40 in 2006) and Canada (mean score 548) came 12^th. (Note the improvements in performance from 2006 to 2011 by both England and the United States.)

As well as reporting mean scores by country, PIRLS also provides details of performance against four benchmarks: Advanced, High, Intermediate, and Low (and those who fail even to qualify for Low ie Below Low). In Australia 7% of students failed even to meet the Low benchmark and a further 17% met only the Low benchmark. Here are comparison percentage figures for other English speaking countries of interest: 

                        Below Low    Low     Combined              

Northern Ireland		3	10	13
Singapore		3	10	13
Canada		2	12	14
United States		2	12	14
England		5	12	17
New Zealand		8	17	25
(Australia)		7	17	24

In summary, Australia and New Zealand have over twice as many students failing to meet even the minimal Low standard as Northern Ireland, Singapore, Canada and the United States; and over one and three quarters times as many low-performing students overall (Below Low and Low combined). England falls in the middle of these two groups of countries.

These results may have come as a shock to many educationists who had been blithely arguing that there was no literacy crisis in Australia. But they provided simple confirmation for Australian reading scientists who had been warning of this problem for some time and had argued (remarkably accurately, as it turns out) that a quarter of Australia’s students could be regarded as low-progress readers. In 2004, a group of Australian reading scientists and clinicians wrote an open letter to the then Federal Minister of Education, Brendan Nelson, arguing the need for reform regarding the way reading is taught in Australia and the need for literacy teaching to be based on the available scientific evidence. The subsequent National Inquiry into the Teaching of Reading reported at the end of 2005, essentially reiterating these concerns and stating clearly what needed to be done to improve reading standards in Australia. In short, the Report was subsequently simply ignored.

Moreover, following the implementation of the National Assessments Program – Literacy and Numeracy (or NAPLAN) from 2008, we were subsequently assured (annually) that all was well on the reading front. As recently as in 2012, we were reassured that for the performance of Year 3 students in Reading only 4.4% were in Band 1, having failed to meet the National Minimum Standard, and only a further 9.4% were in Band 2 ie at the national Minimum Standard (a combined total of 13.8% of students). (Note that the NAPLAN measure of reading is similar to the reading comprehension measure employed by PIRLS.)

Clearly, we have been deluding ourselves by measuring student reading performance against unrealistically low benchmarks that do not withstand international scrutiny. NAPLAN, as a reading performance measure, needs to be recalibrated against international standards so that Bands 1 and 2 combined ‘capture’ the bottom performing 25% (not the current 14%) of students. (These low-progress readers should subsequently be earmarked for immediate additional instructional support.) Moreover, it should not be beyond the wit of the NAPLAN methodologists to tie the NAPLAN scale to the metric employed by PIRLS so that progress towards achieving the international standards could readily be monitored

Finally, it is interesting to note that two English-speaking countries that have begun to take reading instruction seriously in recent years, and who have urged that reading instruction be based on the best available scientific evidence, namely the US and England, have both improved their international standing substantially in the PIRLS league table from 2006 to 2011. Similarly, the two English-speaking countries that have performed so poorly, namely Australia and New Zealand, are those that have clung most tenaciously to the discredited ‘philosophy’ of whole language literacy instruction. Can this be simple coincidence? I think not.

Reference note:

A summary of Australia’s performance in PIRLS 2011 is provided in:

Thompson, S., Hillman, K., Wernert, N., Schmid, M., Buckley, S., & Munene, A. (2012). Highlights from TIMMS & PIRLS 2011from Australia's perspective. Melbourne: Australian Council for Educational Research.

http://www.acer.edu.au/documents/TIMSS-PIRLS_Australian-Highlights.pdf