Monday, 12 September 2011

This is what I call 'like for like'

In its recent media release, the Education Department criticised Frontier Economics' 'decline' graph as being a 'flawed' comparison and 'not like for like'.

Let me tell you a little story. In 2004, the States started publishing a 'Facts and Figures' booklet, showing trends in various key indicators which can help determine policy.

In 2005, Education included a graph in this booklet showing the proportion of Guernsey pupils gaining at least 5 GCSEs grade A*-C, compared with their UK counterparts. Here it is:

It shows the Education Department performing well, as their glowing analysis underneath explains.

On to 2006:

The graph in 2006 tells the same story, but Education decided to add an extra line. Aren't we doing well, overtaking Jersey! Although one can't help noticing that it's lucky these Jersey statistics weren't available last year, because without that most recent figure they would have suggested Guernsey performs consistently worse than the Crapauds...

Now it's 2007 and... oops! Guernsey's had a bit of a crash! But look, a new dotted line has swung in to rescue us. As the rubric says, the news isn't all that bad: It's because we have a school leaving age of 15 in Guernsey, and so the previous line is bound to look bad as it included all those early leavers. So, as the new dotted line shows, it's actually '68% [in Guernsey] compared to 58% in England'. Phew!

2008 - not much change since last year. Still a touch below England using the old 'all pupils' measure, but our new dotted line has risen this year to a magnificent 71%.

Swinging into 2009, and that Guernsey 'all pupils' line is still struggling to keep up with England. But look! The old 'all pupils' line is now a washed-out pastel shadow of its former self, and the dotted line has now become a big fat solid line, which means it's Official - we are doing better than England again, and pulling away from Jersey as well! Hurrah!

Into 2010, and... oh dear... the jiggery pokery isn't holding up... Our new solid Guernsey line, despite its patriotic green hue, is stagnant, and Jersey and the UK are powering along to draw level. The rubric notes we are holding on to our 71%, but no comparisons are offered. Even the masters of spin at Education can't paint a rosy picture of this one.

And so to the 2011 graph. Oh shit.

And then it gets worse again. The 2011 figures (not included in this last graph) show a further 5% fall in the Guernsey figure. Funnily, that part of the line doesn't make it into the graph Education included in their recent media release about GCSE statistics, even though it is included in the raw figures in the same document!

In light of all that, have another read of this paragraph from Education's media release:

'Education’s official statistics, which are included in the 2011 Facts and Figures booklet, accurately reflect the steady improvement in GCSE results from 2001 to 2010. This is contrary to figures published by Frontier Economics who have used the exam entrants only and not the full cohort which the England and Jersey figures seem to represent.'

But the fat solid line in the Facts and Figures booklet IS the same line published by Frontier Economics! And, as Commerce and Employment points out, its use in the Frontier report was signed off by Education. Now it is showing a decline, Education want to disown it.

They should never have started using it - they should have been up-front when we started struggling in 2007 and 2008. Instead we've wasted four years sweeping the problem under the carpet, and now Education have been caught with their pants down, using whatever statistics they think are going to paint the best picture for the purpose.

On its own that might be understandable. We are where we are, as Carol Steere is so fond of saying. But put it alongside her sanctimonious preaching about other States departments' 'flawed comparisons', and it looks like we're being hoodwinked again.

What is really needed is change at the top of Education to prevent us from wasting the next four years. But if it saves her having to have a few difficult conversations with her senior civil servants, it seems Deputy Steere would rather we have another four years of statistical acrobatics, moving deckchairs and deflecting blame.