Estimating the effectiveness of first dose of COVID-19 vaccine against mortality in England: a quasi-experimental study

AbstractBackgroundEstimating real-world vaccine effectiveness is vital to assess the impact of the vaccination programme on the pandemic and inform the ongoing policy response. However, estimating vaccine effectiveness using observational data is inherently challenging because of the non-randomised design and the potential for unmeasured confounding.MethodsWe used a Regression Discontinuity Design (RDD) to estimate vaccine effectiveness against COVID-19 mortality in England, exploiting the discontinuity in vaccination rates resulting from the UK’s age-based vaccination priority groups. We used the fact that people aged 80 or over were prioritised for the vaccine roll-out in the UK to compare the risk of COVID-19 and non-COVID-19 death in people aged 75–79 and 80–84.FindingsThe prioritisation of vaccination of people aged 80 or above led to a large discrepancy in vaccination rates in people 80–84 compared to those 75–79 at the beginning of the vaccination campaign. We found a corresponding difference in COVID-19 mortality, but not in non-COVID-19 mortality, suggesting that our approach appropriately addresses the issue of unmeasured confounding factors. Our results suggest that the first vaccine dose reduced the risk of COVID-19 death by 70.5% (95% CI 18.2–117.7) in those aged 80.InterpretationsOur results support existing evidence that a first dose of a COVID-19 vaccine has a strong protective effect against COVID-19 mortality in older adults. The RDD estimate of vaccine effectiveness is comparable to previously published studies using different methods, suggesting that unmeasured confounding factors are unlikely to substantially bias these studies.FundingOffice for National Statistics.Research in ContextEvidence before this studyWe searched PubMed for studies reporting on the ‘real-world’ effectiveness of the COVID-19 vaccination on risk of death using terms such as “COVID-19”, “vaccine effectiveness”, “mortality” and “death”. The relevant published studies on this topic report vaccine effectiveness estimates against risk of death ranging from 64.2% to 98.7%, for varying times post-vaccination. All of these are observational studies and therefore potentially subject to bias from unmeasured confounding.We found no studies that used a quasi-experimental method such as regression discontinuity design, which is not subject to bias from unmeasured confounding, to calculate the effectiveness of the COVID-19 vaccination on risk of COVID-19 death, or on other outcomes such as hospitalisation or infection.Added value of this studyThe estimates of vaccine effectiveness based on observational data may be biased by unmeasured confounding. This study uses a regression discontinuity design to estimate vaccine effectiveness, exploiting the fact that the vaccination campaign in the UK was rolled out following age-based priority groups. This enables the calculation of an unbiased estimate of the effectiveness of the COVID-19 vaccine against risk of death.The vaccine effectiveness estimate of 70.5% (95% CI 18.2–117.7) is similar to previously published estimates, therefore suggesting that these estimates are not substantially affected by unmeasured confounding factors and confirming the effectiveness of the COVID-19 vaccine against risk of COVID-19 death.Implications of all the available evidenceObtaining an unbiased estimate of COVID-19 vaccine effectiveness is of vital importance in informing policy for lifting COVID-19 related measures. The regression discontinuity design provides confidence that the existing estimates from observational studies are unlikely to be substantially biased by unmeasured confounding.