Unjust deserts?

It is noticeable that today, even that scourge of right-wing governments, The Observer, accepts what I suggested in my earlier comments, that the parameters used in the exam-prediction algorithm made theoretical sense.

But it is also now clearer that something went desperately wrong with the way in which the algorithm worked or was applied. One would have thought that there had been adequate time for the Government to test this to destruction before the grades were announced – but it seems not. Such things are so far beyond my expertise that I cannot really comment further.

My comments were not intended to justify the outcome we have seen, which clearly needs further investigation and (probably) amendment. But I stand by my other observations, namely that a lot of the current outcry is not so much to do with the imperfection of the algorithm as the indignation that teachers were not “sufficiently listened to”. We need to rise above this: moral indignation – whether politically motivated or not – is neither the right nor the best approach.

The claims that teacher predicted grades should have been virtually the only criterion still do not hold water. As I pointed out, these grades are very often wrong; I have now seen a figure of 79% inaccuracy cited in The Observer. That does not surprise me: if the teaching profession is claiming that it is somehow immune to Optimism Bias, then it is certainly and miraculously the only part of the human species that is. That is not to suggest that there is deliberate distortion going on, but to deny that teachers are subject to such things is disingenuous, and in my view, this alone justified other factors being taken into consideration.

Even in these days of publicly-available marks schemes and exam hot-housing, school-reported results are only ever a less-than-impartial interpretation of what the exam board is looking for. I know from my own experience, that there were times when one used grades “elastically” for motivational or ‘stretch’ purposes, particularly near grade thresholds.

It is true that one does acquire a certain “gut feeling” for students’ abilities – but it is not reasonable to claim it is much more than that – nor that horse-trading does not go on between teachers over formal predictions. Nor it is untrue that knowledge of students’ target grades can distort the predictions teachers make. Being human, it can hardly be otherwise, no matter how hard we try – especially when we know how high the stakes can be. Equally, there are plenty of reasons for the grades that schools formally predict, for optimism bias to be a significant factor. Yet there is no acknowledgement of this fact.

The language that is used in education about such things also troubles me greatly. In particular, the use of the word “deserve”. From a moral perspective, Desert is a difficult matter. What one (supposedly) deserves at any given moment is deeply imponderable. I think it is dangerous for teachers to talk about what their pupils “deserve” – most of all when addressing the pupils themselves. And particularly so in the casual sense that is often used, implying that all young people deserve to succeed simply for being themselves, for being young, or for being students. What about young people who made no effort whatsoever? Do they “deserve” to be included in such blanket statements – because I’m pretty sure this is what such phrases imply? What about those who were disruptive in school or who even commit criminal acts? Do they deserve the same outcomes? Do people of unequal ability “deserve” to be given the same rewards, even though that would effectively devalue them for everyone? This is a word that teachers would wisely avoid.

Likewise, I have seen it said that these are “not the results we would want our young people to have received”. Well no, of course not. Superficially, we would all like prizes for all – but that is not to say it is a wise policy. And this again betrays a critical conflation of the desired outcomes with the actual ones, even accepting the deep flaws in what has happened. No exam ever delivered what all pupils and all teachers wanted. The answer is not to capitulate and just award teacher-predictions to all, as one Conservative M.P. has apparently suggested. The matter is more complex than that, even allowing for the difficulties caused during the last academic year.

As I said before, the aim should be to achieve maximum consistency with what has gone before – and what will come after. The “potential” that so many teachers seem to feel has not been recognised is not the same as actual exam performance, when many other factors have always come into play, that meant the results were not someone’s maximum potential. Like it or not, that has always been part of the exam “game” – and we have, as a nation, chosen to set disproportionate store by that game.

The trouble is, this is opening a Pandora’s Box: which limiting factors are admissible and which are not? What is an acceptable time threshold for mitigating circumstances? The fact that someone was feeling ill in the exam room? (Possibly). Issues like dyslexia? (often – but I am far from convinced this is never abused). Parental maltreatment as an infant? The fact that someone had broken up with their boy/girl friend the day before? Or that the weather was bad? Where does one stop? My own exam results were lower that what I think reasonably in hindsight reflected my “true potential” – but that deficit was largely due to poor choice of subjects and normal teenage turbulence and disaffection in the years before them. Why should that not be factored in too? Not everyone experiences them identically. Nor should we be swept away by claims that failure to compensate for these things inevitable “wrecks lives” That, given the blanket certainty with which it is cited, is just brinkmanship, more foot-stamping.

And this is before we consider those yet to come. If this year’s students are given an easy ride, how will it reflect on coming years’ students who do not receive such favours? They will eventually all be operating in the same higher education and jobs markets. Why should one cohort be given an extra-easy ride? They didn’t “deserve” the disruption from Covid (at least no more than the entire human race might) – but following cohorts don’t deserve to be put at a disadvantage as a result, either.

Some have observed that my earlier comments were unsympathetic. They were not meant as such – but I see no reason why the correct response to the current situation is indulgence. There are enough bona fide reasons not to trust teacher predictions 100% – without that implying acceptance of the shambles we have now.

This is why we must decouple teachers’ interests from those of their pupils. I don’t think it does the teaching profession any favours to be as invested as it is in a partisan stance: in the quest for fairness, I don’t believe that positive bias is any more acceptable than negative. This is perfectly compatible with wanting the best for your pupils because achieving it by misrepresentation is not the answer. I know for a fact that a few generations ago – when my parents taught – the witch-hunts that can follow from a teacher producing low exam results in a particular year did not happen. Again, I am not justifying repeated under-performance: this is where real individual professional responsibility needs to come in (though this, too, has been removed). But it needs to remove the reasons teachers have for partisan talking-up of specific student outcomes. Professionals need more detachment than that, for their students’ greater good, let alone their own.

The real problem is the simplistic, mechanical and consumerism-driven model of education that we now have. On the one hand, this has led people to believe that there is a simple causal relationship between what a teacher does with their students and the results those students achieve, as there is between a business and a customer. It is not so: there are many more factors involved than that, some which are neither predictable, nor in a teacher’s gift.

On the other, it has led people to cry foul when they don’t then receive what they feel they “deserve” as consumers of the education system – whether in the wider perspective it is justified or not. In education, the customer is not always right. Furthermore, the spectacle of teachers making such complaints is a significant factor in students and their parents following suit, whether well-considered or not.

I’m not suggesting there should not be reasonable grounds for appeal, as there have always been; indeed probably more so this year than usual. But it is also true that schools and teachers have increasingly used that procedure to “game” the system for furthering their own vested interests. I know of some schools where appealing grades has been pretty much an automatic annual policy in order to improve the statistics. This is not what I understand by professionalism.

This is not to imply that teachers are bent. The vast majority act in good faith – but whether it is wise or appropriate is a different matter. The recent spectacle has come close to an unedifying stamping of the feet. Professionalism should raise people above that, even though the partisanship has, I accept, been forced on teachers by the processes by which they are now appraised.

I repeat: this is why we desperately need to return teachers to a position where professional disinterest is possible. That is all I have been advocating in my posts.

Indulging in metaphorical foot-stamping, while repeatedly denying the validity of points such as those raised above, is disingenuous, and does the teaching profession no favours. Rather than reducing the matter to simplistic shouting, we should be providing a more rational, nuanced and disinterested narrative for the sake of all concerned. Where there is cause for concern, it should be advanced primarily through reasoned argument, rather than subjective matters of “fairness”. Teachers of course wish for the best for their students – but that is not the same as crying foul every time they don’t get what they want.

That this seems to be neither seen, nor perhaps possible, is said in sadness not anger.

Incidentally, I have seen no discussion, either, of the legal right of people to request decisions made by algorithm be reassessed by a human being. That ought to settle it.

3 thoughts on “Unjust deserts?

  1. Two very interesting blogs, Ian. I have very much enjoyed the read. I agree with much of what you say – I’ll come back to that in a moment – I think broadly, we have similar views of education and that the entire system is wrong.

    I do think that many students have been wronged by an algorithm and whilst algorithms are great for looking at big groups, they are terrible for dealing with individuals. For example, i know that about 70% of white, middle aged, wealthy men will vote Conservative. However, give me 10 such people in one room and my chances of picking the 7 misguided fools accurately are slim. It was the same to an extent when grading students. No system could accurately award grades to all students, but guessing which 7 out of 10 equally rated students would get the A grade, which 2 would just miss out and which 1 would flunk completely would be entirely unfair. The only fair thing to do was to accept all teacher grades, whilst randomly demanding some evidence to back up the awards. I had evidence to support all of my a History & Politics cohort, whilst also being happy to admit that where I hesitated between 2 grades, I always went for optimism. With 20 years experience, I am confident of getting the grading right within one grade in almost every case. The only times I have been surprised in recent years are by a few students who suddenly worked their socks off and pulled it out of the bag to exceed my expectations.

    I am happy that my 6 Politics students this year kept the awarded grades (A*, B, 3 Cs and a D) – I believe these are the best grades each could have achieved and though a couple may have missed out, they had to be given the benefit of the doubt under the circumstances. For one year, a year where young students (who are largely not even at risk from the virus) have missed an important year of their education – their final school year, I don’t think it would have been too much of a problem to accept inflated grades – better that than the chaos and anger we now have.

    In terms of the education system, I think that an overhaul is long overdue. I would scrap final exams completely – too much rests on them and good students can miss out, while bad students can get lucky. Why not portfolio based continuous assessment? Students submit their best work for moderation and this can be from coursework, projects, internal topic assessments and maybe one small exam as a course finale. I realise that this may be open to abuse, but factor in regular moderation, spot checks on assessments and stiff punishment/ disqualification for students (and staff) who try to cheat and I think it could be a better way forward.

    This crisis should have been an opportunity to revamp exams and think about what is actually taught, why and how. Instead, everyone buried their head in the sand and tried to carry on as normal. I fear that the aim of returning to normality is misguided and unachievable – both in education and the wider world.


    1. Thanks Richard – I’m interested in your comments. Between the first and second posts it became more apparent to me just how widespread and wide the errors were – which is why I felt it needed further comment. But my main issue is with the nature of teacher discussion I’ve seen. A lot on social media is really not very ‘professional’ and in my view simply risks stoking the fires (perhaps on purpose?) whereas what we really need are some cool heads to sort it out. Some of it likewise even from institutions, though there is certainly a more nuanced take coming out now as well. I also sense that quite a lot of it is actually politically driven – and while I tend to the left of centre myself, I do not like education being used for political purposes of *any* kind. My comments about the way in which predicted grades are arrived at come from first-hand experience, where at ‘A’ Level they were more often than not a compromise between the divergent feelings of two or more teachers – hardly scientific. I’m sure the vast majority of teachers do they best the can – but again it seems to me that predicted grades have been inflated out of all proportion by those who have capital of one sort or another to make. I never liked making them; I realised that they were at best an educated guess – and I certainly felt uncomfortable that they were then treated almost as guarantees. It’s the same with target grades – they came to be treated as cast-iron truth, and nothing but nothing would deflect those who relied on them from views to the contrary. Again it comes down to the tail wagging the dog.

      I don’t have an issue with exams as such, more the fact that they have become an end in their own right. I have witnessed lessons where almost the entire time was devoted to exam technique with minimal subject teaching at all – and it was not an exception. It was more or less what we were told to do. I think the reasons for this were originally forced on the profession by politicians but it is now so embedded that much of the profession has never known any other way. Exams are fine as *a*, perhaps *the* way of validating past learning – but they are meaningless if they are nothing more than a set of hoops to jump through. Even my more recent experience elsewhere suggested that this has become more rather than less prevalent in the time since I stopped teaching full time. Likewise, I agree with you about the value of diverse forms of assessment – but again am more concerned by the way the system as a whole has just become obsessed with the numbers that people achieve, rather than the individuals behind them. I would keep exams, but reduce them to their former status and place more equal value on wider aspects of education.

      p.s. without intending to push my own stuff unduly, quick update on recent ‘doings’…. , my main edu blog is https://wordpress.com/view/ijstock.wordpress.com which also feeds into https://wordpress.com/view/educationechochamber.wordpress.com where I was formerly on the editorial team. Might be of interest? And that in turn led to this ;-): https://www.amazon.co.uk/Great-Exception-teaching-profession-other/dp/1911382578


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s