Continuing from previously…I have done some upgrades to the method and now have some results that I think are quite acceptable.
Most importantly, after thinking about (and getting half-way to implementing) appropriate covariance matrices for the overlapping fortnightly observational summaries, I decided that the most sensible thing to do was just to use a single set of these analyses from the outset. The two sets are just summarising the same underlying data in two different ways, so double-counting it while accounting for the double-counting is a bit pointless compared to just using one analysis in the first place.
I’ve also done a few details more precisely than the approximations I’d used previously. Such as simulating the observational process more closely (such taking a fortnightly mean from the model rather than just the mid-point which I’d lazily used before). Here are the latest results, as an animated gif using initial subsets of the data, ie simulating forecasts from earlier in the summer. Note that there is no attempt to predict the effects of policy changes, so it’s not surprising the initial forecasts don’t see the autumn resurgence though it does seem to detect the late summer increase quite early. What I was mostly interested when starting this, was seeing if I got a set of consistent hindcasts as more data are added, which the ONS signally failed to do with their rather poor curve-fitting job. You can see that my system manages that pretty well. It certainly doesn’t flip around wildly like their approach.
I think it’s doing a pretty decent job with one significant caveat/issue. The spread in the plume is clearly too wide at the right hand end of the fit at the end of the sequence when all data are used. I’m not 100% certain why this is, and can’t rule out a bug or even an error in the algorithm, though I think it’s just the result of using an approach that assumes a perfect model in a linear Gaussian world, for a system where none of these things holds true. The plume is roughly twice the width it should be, so if I was in the business of issuing forecasts that I expected someone to pay attention to, I’d possibly just halve it. I could fix the linear/Gaussian thing with infinite computing power, or even perhaps just a faster laptop, so watch this space for future developments…
As mentioned, the model doesn’t attempt to predict the impact of policy changes, but allows for modest uncertainty in business-as-usual. The thing I’m most interested in is the modest rise from early July (when pubs opened and we were all told to go back to work), which seems very similar to what I was saying at the time, based on my own fit to the case and death data, but rather different to ONS’ interpretation of their own data both at that time and subsequently.
It is interesting to compare the R number that comes out of this analysis, with the value I obtain from my daily fit to cases and deaths. Here it what the ONS data generates:
And here is the above plot (in pink) superimposed on my other analysis:
Sorry for the scrappy plot. The agreement looks remarkably good to me, given that the ONS data are only available to me as fortnightly summaries. It is even better if I allow for two steps in R at the same time as for the daily analysis, to account for substantial policy changes:
There’s a clear disagreement in October, but other than that, the curves are pretty much superimposed and certainly consistent. The ONS analysis leads to a highly uncertain R number at present, basically because there are no sufficiently recent observations to constrain the late October jump (which I’m using as a catch-all for the higher tiers and the recent lockdown). The pub opening and back-to-work campaigning at the start of July appears to have had an appreciable impact, and it is interesting that SAGE continued to insist that R was less than 1 to the end of August when my analyses point to a much earlier rise. We might have been better prepared for the early autumn wave with some more competent analysis.
I’m going to have to do the same with the REACT data next.