(Small edit made 21 Apr to add a corollary at the bottom of the post.)
I’ve said bits of this in various places and at various times to various people, but I don’t think I have written it down in a complete and coherent form. Actually, bits and pieces of the story have come to me at different times and in different ways, so perhaps I didn’t have a coherent story before. Anyway, it seems important and I don’t want it to get whitewashed from history, so here goes.
The story possibly starts on the 12th March, when Vallance stated that we were 4 weeks behind Italy.
And also, quite specifically the peak was 3 months away:
For the UK, the peak is expected to fall in three months’ time, likely in the summer months, and tail off throughout the autumn, the government said. Vallance said that the UK is around four weeks behind Italy
It’s fair to say the “4 weeks” comment was met with a bit of scepticism by the general public, eg here. And here. When the Govt’s Chief Scientist is being openly mocked for his comments, it seems to me that something is seriously wrong. For context, on the 12th March we’d had about 500 cases and 8 deaths. 15 days earlier on the 26 Feb, Italy had had very similar numbers – in fact slightly fewer cases and more deaths. In both countries, the numbers of cases and deaths were doubling roughly every 3 days, meaning we would get to Italy’s then current values of 20,000 cases and 144 deaths in about a fortnight or so (5 doublings = 32x). 4 weeks was obviously risible.
Then a few days later the 16th March, Vallance talked specifically about a 5 day doubling time (on this youtube video, starting from 27 mins in). And people were puzzled. 5 day doubling would indeed put us about 4 weeks behind Italy (ie the required 5-and-a-bit doublings would take about 26-27 days), but Italy wasn’t doubling every 5 days, and neither were we. We were both doubling on a 3 day time scale instead, possibly quicker than that.
It was actually jules who cottoned on to this first. She had been looking at the numbers more than me, and working out the doubling rate. At this point I was more thinking about the govt’s strategy to fill the streets with bodies under their “herd immunity” plan. It seemed very clear that the weight of critically ill people was going to be a huge burden that the NHS would have no possibility of treating, and my first blog post (which didn’t even have a proper model in, just some curves) focussed on that particular detail.
Anyway, 5 day doubling. Where did this come from? Took me a little while to work it out. It wasn’t until I got hold of the SEIR model and started playing around with it that it started to come together. Ferguson had posted a paper on the 16th March that outlined his modelling. Although his model is of course far more detailed than the SEIR model I was using, it described the parameters in enough detail to emulated rather well by my simpler model. And….the doubling rate he had used was 5 days. You don’t need to do too much digging – or have a great deal of expert knowledge – to find it in the paper:
a 6.5-day mean generation time. Based on fits to the early growth-rate of the epidemic in Wuhan, we make a baseline assumption that R0=2.4 but examine values between 2.0 and 2.6.
What this means is, the number of cases grows by a factor of 2.4 in 6.5 days. Which is equivalent to doubling in 5.1 days. They just imposed that – admittedly, the parameters were estimated from the Wuhan outbreak, but this result came a very small data set very early on. It is also well known that the basic reproductive rate R0 depends on the social context and it’s far from certain that it would transfer directly from the Chinese denizens of a wet market to the population of the UK. To some extent, the effective duration of the period in which people pass on the infection could vary in practice vary too, depending on whether people go to bed or go to work etc. So there is simply no way that putting in the first estimate for Chinese parameters (with a modest uncertainty range) could be considered a robust and reliable modelling strategy, especially since there was already strong evidence that these values were not appropriate even for the later growth of the Wuhan outbreak let alone closer to home. There was ample evidence from other European countries that their doubling rates were far faster than 5 days, and growing evidence from the UK that we were following the same path.
I did a bit more playing around with my model, including parameter estimation, and quickly came to the realisation that R0 had to be rather larger than Ferguson had said.
I emailed Neil Ferguson about this on the 27th, and also CCed David Spiegelhalter, on the basis that as a public-facing stats expert with a strong interest in health he’d get what I was talking about and realise it was important..and, well, they did reply which was more than I was really expecting, but only to politely brush me off. Prof Ferguson did at least acknowledge that he now thought a higher value of R0 in the range of 2.8-3 was appropriate. And true enough, the very day I emailed them, Govey had talked of a 3-4 day doubling. But that requires a rather larger R0 in the range of of 3 to 4 (assuming the same 6.5 day time scale), and is still a bit slower than the data were consistently showing. Later research from IC with a simpler model pointed to a value of around 4.
As for why this matters, here are results from two model simulations. One of them – the uncalibrated one – is very close to what the IC group showed to the Government. The other one is what you get if you calibrate the model to the growth rate shown in the UK data that was available at that time.
For the red line, I used Ferguson’s model parameters and initialised the model as he described in his paper, timing the epidemic so that it had the correct number of total deaths (21) up to 14 Match. For the blue one, I first fitted the parameters to the time series of cases reported in the UK, which were probably reasonably reliable up to that time as they were still tracing contacts and testing etc. Similar parameters would have been obtained from fitting to Italy, Spain and the Wuhan outbreak. I then initialised the simulation as for the red curve (daily deaths on 14th are slightly different but the integral up to that date is the same).
Want to guess which one is closer to the future observations? Well, you don’t have to. The initialisation leaves the blue line about a day or two behind reality (only!) but tracking it at the same rate. The red line…just…well. No comment. The logarithmic axis really helps to hide how far away from reality it is.
And as for why this really mattered…the red curve below was how the Ferguson et al model predicted the epidemic was going to pan out. A couple of months to the peak in infections and deaths following almost a month after that. Terrible, but still a little way away, and and Vallance was saying we mustn’t suppress the epidemic too quickly.
However, in reality we were on the blue curve. A peak of over 3 million new cases per day was less than a month away. Well over 20k deaths per day at the start of May. And the govt was just shilly-shallying around.
The big puzzle for me in all this is, why on earth didn’t Ferguson calibrate his model to the 3-day doubling exponential growth rate that was clearly visible in the data? Ok, perhaps I’m a bit biased due to model calibration being basically what I have spent the last couple of decades on, but it’s a pretty fundamental component of forecasting in every field of science that you can think of. Apart from this one, it seems. Every weather forecast is generated by a model that’s tuned to look like reality, both in terms of parameters (as part of its design and commissioning) and also initialised to look like today’s weather. The epidemiologists did the latter ok – choosing a start date to fit their epidemic to happen about now – but never bothered to tune their parameters to match the growth rate.
It will, I suppose, forever remain a mystery as to why this happened.
A small corollary of the above, added on 21 Apr: It is very straightforward to calculate the effect of a delay to the lockdown. A week of unchecked growth at 3-day doubling corresponds to a factor of 5, meaning that 80% of the total size of the first wave we are currently in could be directly attributed to the Govt delaying by a week, if it was felt that the evidence could and should have supported action that much sooner (ie, when most of the rest of Europe was already taking action). That means 80% of the peak strain on the NHS, 80% of total cases and also 80% of all resulting deaths. What this calculation doesn’t account for, is what happens in the longer term. We may all get it in the longer term anyway (well 60%+ of us). But we might not, and even so, the huge peak was 5x bigger than it would have been if controlled just a week quicker.