Politicians are failing to convincingly justify their COVID-19 policies, and journalists are failing to hold them to account.
The Covid-19 pandemic has become a bit of a numbers game – charts of new cases, deaths and transmission rates dominate the daily press conferences from Downing Street. But as we see a mixture of silly questions being asked by what are supposedly the UK’s top journalists, and ministers consistently failing to answer any direct question with a direct answer, you may just want to stop and think about those numbers. I should note up front that I am not a medical expert – but I am trained as a statistician. Let’s start with 3 observations:
- There is seldom any context. At the time of writing, in a couple of months, Covid-19 has caused over 20 thousand deaths in the UK (probably nearer 30 thousand adding everything in). It sounds a lot and it is, in the sense that each death is an individual tragedy. But the UK population is getting on for 67 million. About ½ million people die in the UK every year from various causes – so about 10,000 a week on average, every week, every year.
- Influenza – flu – is always a killer: In England alone, according to Public Heath England, there are an average of 17 thousand flu deaths a year, but it varies widely – 28,330 in 2014/15 to a low of 1,692 in 2018/19. (One can probably add on around 10% to estimate a UK figure, based on England-only figures.) So we are not yet up to the peak annual number of flu deaths.
- Cause of death is an imprecise concept. Official causes of death depend on what the doctor records on the death certificate and people can have multiple life-shortening conditions when they die.
Now, let’s just jog back to the daily briefings. Journalists seemed to have been surprised to learn that the number of deaths being given out each day was only for those in hospital, not all deaths. But that was never a secret. The same awakening happened in France a couple of weeks before it became a hot UK issue. In fact, nearly every country only records hospital deaths in their daily figures.
This is perfectly reasonable. Most hospitals will have reasonably consistent diagnoses, a regular stream of deaths every day and a well-established reporting regime. In contrast, there are 10 times more care homes than there are hospitals. And care homes don’t normally have a death every month, let alone every day, so the reporting routine is absent. Reporting of deaths in other places, including the broader community, is likely to be even less timely. If you are a politician making a decision, consistency in the numbers can be more important – to track trends – than whether the numbers include absolutely everybody or not.
The fact is that none of the journalists bothered to look at the definition of the numbers right from the outset. So, what else might they be missing? If you read the small print you will realise that that ‘daily’ death number reflects when deaths were recorded – not when they happened. The timing is reflected more accurately in the weekly ONS figures.
The importance of this is that the daily numbers are lagging events and the trend will move upwards and then downwards faster than those figures suggest. Not an observation that I have seen reported by mainstream media.
Let’s look now at some of the ways in which numbers are presented. We see a lot of charts showing the number of cases, deaths and so on, and some of those comparisons are across countries. But many of these charts are either very difficult to read or downright misleading or both.
One of the worst was the first presentation of the weekly data in the daily briefing. None of the charts shown by the government or subsequently by the media, actually said that the y axis was showing weekly data – it just said ‘average deaths’ or something similar. Charts must be labelled properly if we are to understand anything from them.
For a disease like COVID-19, which replicates exponentially (i.e. it multiplies up), the absolute numbers will always give you a chart going from bottom left corner (where the axes usually cross) to the top right corner, usually accelerating in an upwards arc to start with. Some charts now present these numbers in logarithms which is at least one step forward. Such ‘log’ charts are designed so that, visually, the same movement in the chart reflects the same percentage change. For example, 10% of 40 is 4, 10% of 400 is 40. A chart of just pure levels shows the 40 as a ten times bigger movement – which it is. But a chart in log transformations would show these two as equal-sized movements (as both are 10%). So, if you are interested in whether the growth in the number of cases is increasing or decreasing, then log charts are an improvement.
I have not come across a single commentary by a journalist based on such a chart which draws out what it means. Perhaps most national journalists – who are ‘words’ people by and large – have forgotten whatever maths they did at school (who could blame them?).
The most useful data presentation would be to see a straightforward chart of the growth rate of the disease. Most people should be able to understand what a percentage growth rate is.
What about the cross-country comparisons? Be wary. Each country started off with different policies at different points in time. Choosing whether to compare timelines based on the first death, the first ten deaths or the first hundred – all have been used – will give somewhat different impressions. Some more favourable to the UK than others.
The most obvious problem for comparisons is differing definitions. Not that many countries have something like the NHS with consistent national standards of reporting. Germany appears to be doing very well – but supposedly, German doctors are much less likely to record COVID-19 as the principal cause of death. It’s unclear to what extent that is true or what numerical impact it could be having – but we do know that every country has a different reporting basis which means one can’t take cross-country differences at face value.
How to spot data issues
One of the curious things about COVID-19 is that, despite all the differences in data reporting, there is a fairly consistent underlying death rate being estimated by medical experts. It seems that, very approximately, 3% of all those who get infected die. But up to 80% may hardly notice. So, of the 20% who develop distressing symptoms, about 15% die. None of those numbers is reliable but they appear plausible enough to test the coherence of a nation’s statistics.
The UK is reporting nearly 150 thousand positive tests. But if we estimate a total of 25 to 30 thousand deaths (reflecting deaths in community, reporting lags etc) that would mean a fatality rate of around 20% – about the same as one would get if one was only measuring cases of those with severe symptoms. Since the UK testing strategy has been to do just that, it figures! So, suppose one instead assumed a 3% fatality rate and estimated the number of infections? That would imply say 800 thousand to one million infections, which sounds much more plausible than 150 thousand. Indeed, the medical experts said early on that the number of cases might be a factor of 5 to 10 higher than reported.
The US figures represent a puzzle. Nearly a million positive cases – but just 54 thousand deaths as I write. An implied fatality rate of 5% – which would mean that they must be finding and testing nearly everyone who has it. Given the haphazard approach across states in the US, that can’t be right. So, what might be going on?
The US could be drastically under-capturing the number of Covid-related deaths – which is quite plausible when they show only 120 thousand recoveries. What is the status of the other 800 thousand positives?
Or they could be generating false positives in their tests. That wouldn’t be altogether surprising given that we know they had ineffective tests initially, and that the US health system is run for profit. You can treat and charge a patient once they have tested to have something wrong with them. But actually, a million US cases with significant symptoms (and perhaps 5 million in total) is quite plausible given the 330 million US population, so COVID-19-linked deaths being under-recorded and/or badly lagging looks more likely.
But who cares about the numbers as long as we are doing our best to survive and help as many people as possible? I will give you two reasons why we should be concerned:
Populist politicians want to blame China so as not to accept responsibility for their own failures.
Chinese consumption preferences may have caused the crisis, the Chinese state covered it up, then they under-reported the severity. All plausibly true assertions.
But let’s just stop and consider for a minute. The UK and US took very little effective action even once it was clear what was happening in China. Even when the World Health Organisation (WHO) had declared a pandemic. Trump claimed it was a hoax, then made a big deal out of it when the US only had a few cases, then claimed that they would keep the virus out (through very partial and hence ineffective travel restrictions), even while the virus was taking firm hold and scientists – including the WHO – were ringing the alarm bells as loud as they could. And the UK and US data in their different ways are clearly dramatically under-reporting even now.
Some of this denial was a natural human response – no one wanted to believe it could be that bad. So, we shouldn’t be too critical as we might each have made the same mistakes.
But to blame China for mistakes that were subsequently made much more dramatically in the West – when the latter had the advantage of a clear warning and learning from Chinese experience, seems just a little like throwing stones whilst living in the most fragile of glass houses.
Politicians are getting ready to blame scientists.
Have you heard the UK government mantra? “We have always followed the scientific advice”. Challenges of competence are met with “so you think you know more than the medical scientists?” But when a journalist actually catches a scientist with their guard down for a second what they say is “I give my best interpretation to the government – they make the decisions”.
That’s because most of the decision making is not about medical science, it’s about politics. When to go into lockdown, whom to test, how to police it – all these issues are essentially political. We know there is not a unique approach based on science because every country has done different things and they all claim to be relying on scientific evidence. In fact, the medical experts were warning of a global pandemic at a time when the UK and US governments were still in denial.
The UK government is being blamed for being slow at every point. And they were, because that’s what the politics was driving. Rather than showing leadership by being ahead of the curve and making people do unpopular things, they wanted it to appear that it was the pandemic and the science driving them. That was a political choice.
And when it comes to the exit strategy, that will be a political choice too, both in form and timing even if informed by medical advice. But it’s no good journalists demanding to know what the exit strategy is now. If you listen carefully to the daily briefings it’s quite clear that the government won’t discuss it. Not because of the stated reason that they want to keep attention on the lockdown, but because they haven’t got a strategy yet. They don’t know. They will decide on the strategy when it seems the right time do so politically.
I am not suggesting that I, or anyone else would have been able to call it better. Hindsight is a wonderful thing. I am suggesting that we should not be preparing to blame the scientific advice for bad political decisions.
Non-political, technical experts have become an easy scapegoat for populist politicians. We should reject that and pin accountability on those who were elected to take responsibility.