Assessing real-world vaccine effectiveness against severe forms of SARS-CoV-2 infection: an observational study from routine surveillance data in Switzerland

DOI: https://doi.org/10.4414/SMW.2022.w30163

Introduction

The continuous assessment of vaccine efficacy and effectiveness against SARS-CoV-2 is critically important for informing national vaccination campaigns and the public health response against the COVID-19 pandemic. Randomised controlled trials are the gold standard to estimate vaccine efficacy against symptomatic infection, hospitalization and death. Several randomised controlled trials have reported high levels of efficacy for several SARS-CoV-2 vaccines [1, 2]. For technical and ethical reasons, such trials have limitations when it comes to estimating vaccine effectiveness in real world conditions and over longer periods of time [3, 4]. Even though not ideal in terms of potential bias, observational data can be used to estimate vaccine effectiveness. When rich longitudinal data are available (e.g. insurance data or cohort studies), it becomes possible to directly estimate and compare the risk of symptomatic infection, hospitalisation or death in vaccinated and non-vaccinated individuals (adjusting for key characteristics) [5, 6]. When the vaccination status of SARS-CoV-2-negative controls is collected, a test-negative design can be used [7, 8].

Routine surveillance data often do not include any follow-up or control group. In Switzerland, routine surveillance data on COVID-19 contains detailed information only on reported confirmed cases, hospitalisations and deaths. The proportion of vaccinated individuals in surveillance reports can, however, be misleading as it is highly dependent on vaccine coverage [9]. Vaccine coverage can be heterogeneous and can vary by time, location and other characteristics, first of all age. In Switzerland, SARS-CoV-2 vaccination campaigns started in early 2021, first focusing on vulnerable groups (aged above 65 with comorbid conditions), then being gradually extended to younger age groups. Until May 2021, vaccination  with vaccines from Moderna and Pfizer-BioNtech was roughly equal, but from this point onward the Moderna vaccine was more commonly used. Vaccination with the Johnson-Johnson vaccine started only in October 2021.

In this study, we used a reformulation of the screening method [10, 11] to estimate vaccine effectiveness against severe forms of SARS-CoV-2 infection in real-world settings from routine surveillance data in Switzerland, accounting for the levels of vaccine coverage by week, age group and location. We also assessed the variation in vaccine effectiveness by age, vaccine type,  calendar time and – importantly – by time since vaccination.

Methods

Setting and data

We considered all routine surveillance data on laboratory-confirmed COVID-19-related hospitalisations and deaths received at the Federal Office of Public Health (FOPH) until 1 December 2021. These data included vaccination status at the individual level, which comprised the type of vaccine, the number of doses and the existence of a previous positive test. Individuals were considered fully vaccinated if they received either two doses of the Moderna or Pfizer-BioNtech vaccines, one dose of the Johnson-Johnson vaccine, or one dose of the Moderna or Pfizer-BioNtech vaccines with a previous positive test. Individuals were considered partially vaccinated if they received just one dose of Moderna or Pfizer-BioNtech without a previous positive test. Individuals were considered not vaccinated if they reported having not received any dose.

We included only COVID-19-related hospitalisations and deaths from 1 July 2021. Indeed, the proportion of missing vaccination status among hospitalisations and deaths was high (30–60%) during the early months of 2021, but rapidly decreased to around 10–15% from the month of July 2021 onward (supplementary fig. S1 in the appendix).

We treated missing information on the vaccination status as follows. For the baseline analysis, we defined as not fully vaccinated individuals who reported being either non-vaccinated or partially vaccinated, and excluded individuals with missing vaccination status. This assumes that vaccination status is missing at random (non-fully vaccinated individuals are equally likely to have missing vaccination status as fully vaccinated individuals). We then conducted several sensitivity analyses. First, we imputed missing vaccination status based on age group, canton, week and vaccine coverage using multiple imputation with chained equations [12]. Second, we assumed that all individuals with missing vaccination status were not fully vaccinated (“Best case scenario”). Third, we assumed that all individuals with missing vaccination status and all partially vaccinated were fully vaccinated (“Worst case scenario”).

Vaccine coverage data were obtained at the FOPH from the Vaccine Monitoring Data Lake database, which records every vaccination event. We considered individuals as fully vaccinated using the same criteria as for COVID-19-related hospitalisations and deaths. We aggregated this data by week, canton and age group.

Statistical model

We use a statistical model to assess the relative risk (RR) of hospitalisation among non-fully vaccinated individuals compared with fully vaccinated. The intuition is that, if vaccine had no influence on the risk of hospitalisation, the proportion of vaccinated among hospitalised people would be the same as the proportion of vaccinated in the population. The more vaccination reduces the risk of hospitalisation, the lower the proportion of vaccinated among hospitalised people will be.

More specifically, we defined the RR of hospitalization among non-fully vaccinated individuals (V_N) compared to fully vaccinated as:

where Pr(H|V_N) refers to the probability of hospitalisation given non-fully vaccinated status and Pr(H|V) to the probability of hospitalisation given fully vaccinated status. This model allows us to estimate RR while accounting for the dynamics of vaccine coverage over time, by age group and by canton. The approach, which is detailed in the appendix, is equivalent to the screening method [10]. Briefly, we considered the expected probability that a hospitalised individual is vaccinated [(Pr(V|H)] given the vaccine coverage at the time of hospitalisation in the same age group and canton [Pr(V)], and given the value of RR from the following equation:

We estimated the RR by comparing the expected probability Pr(V|H) to the actual vaccination status of every individual using a Bernoulli likelihood within a maximum likelihood framework. The RR can also be expressed as a relative risk reduction (1 – 1/RR). This last quantity is closely related to vaccine effectiveness against hospitalisation if weassume that all the other factors influencing the risk of COVID-19 hospitalisation (e.g., behaviour or exposure) are independent of the vaccination status.

This model also applies to vaccine effectiveness against death, and can be extended to assess variations in vaccine effectiveness across different stratification groups. We considered stratification by age group and calendar month. We also extended the approach to a situation where we compared more than the initial two categories (fully vaccinated or not) using a generalisation of the equations above (appendix). This allows assessment of variations in the RR associated with time since vaccination in three groups (vaccinated for up to 12 weeks, for 13 to 24 weeks, or for 25 weeks or more). We restricted this analysis to fully vaccinated individuals with non-missing time since vaccination, assuming that this variable is missing at random. We also stratified by age group and by vaccine type (restricted to Moderna or Pfizer-BioNtech).

We did this research using surveillance data collected by the Federal Office of Public Health according to the Swiss law on communicable diseases (EpG, SR 818.101). No ethics committee approval was required. The code is available at https://github.com/jriou/vaccine_effectiveness.

More details and a simulation study validating the statistical approach are available in the appendix. As data contain sensitive information at the individual level, it is only available on motivated request to the Swiss Federal Office of Public Health.

Results

From 1 July to 1 December 2021 we included a total of 5948 hospitalisations and 739 deaths (table 1). Among all hospitalisations and deaths the proportion of fully vaccinated patients increased over time to around 40% (fig. 1A). During the same period, vaccine coverage in the population increased to 65.7%, with important differences across age groups (fig. 1B). There were also geographical differences in vaccine coverage and the type of vaccine used (fig. 1C), highlighting the importance of accounting for vaccine coverage by time, age group and location.

Figure 1 (A) Proportion of hospitalised and deceased COVID-19 patients reported as fully vaccinated among all reported COVID-19-related hospitalisations and deaths. The dashed vertical line corresponds to the starting date of the analysis.

(B) Evolution of SARS-CoV-2 vaccine coverage in the population by age group.

(C) Cantonal differences regarding the overall SARS-CoV-2 vaccine coverage and the proportion of fully vaccinated individuals having received the Moderna vaccine in the population (as of one week before the end of the study period).

Of hospitalised individuals, 1245 (21%) were reported as being fully vaccinated (table 1). This number was 259 (35%) for deaths. Vaccination status was missing for 834 (14%) of hospitalisations and 98 (13%) of deaths. The age distribution of hospitalised patients was shifted towards older age groups, with 1221 (21%) being 80 and older. This is even more pronounced for deceased patients, with 464 (63%) being 80 and older. Almost all individuals received the mRNA vaccines of Moderna or Pfizer-BioNtech. Among the fully vaccinated, 373 (30%) of hospitalisations and 133 (51%) of deaths had been vaccinated for 25 weeks or more. Information about time since vaccination was missing for 584 (47%) of hospitalisations and 83 (32%) of deaths.

Accounting for vaccine coverage by week, age group and canton, we found that the RR of hospitalisation without full vaccination compared with full vaccination was 12.5 (95% confidence interval [CI] 11.7–13.4) (fig. 2A, supplementary table S1). This corresponded to a relative risk reduction against hospitalisation (or vaccine effectiveness) of 92.0% (95% CI 91.4–92.5%). Results from sensitivity analyses with alternative handling of missing vaccination status ranged between RRs of 6.6 (95% CI 6.2–7.0) and 15.7 (95% CI 14.7–16.8). The RR of hospitalisation decreased in older age groups. We also found a decrease of the RR of hospitalisation over time, which also constitutes indirect evidence of waning. After stratifying by both age group and month, we found evidence for a decrease in RR in October and November 2021 in age groups 70–79 and 80+, but inconclusive evidence for other age groups (supplementary fig. S3).

Figure 2 (A) Relative risk (RR) of COVID-19-related hospitalisation for individuals without full vaccination compared to individuals with full vaccination in the baseline analysis, in three sensitivity analyses, by age group, and by month.

(B) Relative risk of death for individuals without full vaccination compared to individuals with full vaccination in the baseline analysis, in three sensitivity analyses, by age group, and by month. Numbers correspond to group sizes (n).

The RR of death without full vaccination compared with full vaccination was 10.4 (95% CI 8.8–12.2) (fig. 2B, supplementary table S2), corresponding to a relative risk reduction against death of 90.3% (95% CI 88.6–91.8%). The RR of death decreased for older ages and over time, which again constitutes indirect evidence of waning. However, the small number of deaths leads to imprecise estimates with large confidence intervals.

To study direct evidence of waning, we restricted analyses only to fully vaccinated individuals with information about time since vaccination. Because of the large proportion of missing information on time since vaccination, this analysis should be seen as exploratory. The outcome of interest is now the relative change in the RR of hospitalisation compared with the reference group of individuals vaccinated for up to 12 weeks. Compared with the reference group, the RR of hospitalisation did not change when the time since vaccination was between 13 and 24 weeks, but increased by 1.5 (95% CI 1.1–2.1) when the time since vaccination was above 25 weeks (fig. 3A). With stratification by age group, this increase in the RR of hospitalisation 25 weeks after vaccination was significant only in the age groups 0–59 (4.0, 95% CI 2.0–7.8) and 60–69 (2.9, 95% CI 1.4–5.7), but not in age groups 70–79 (1.6, 95% CI 0.9–2.8) and 80+ (0.7, 95% CI 0.5–1.2; fig. 3B). With stratification by vaccine type, the increase in RR of hospitalisation appeared significant for both Moderna (1.8, 95% CI 1.1–3.0) and Pfizer-BioNtech (2.4,[95% CI 1.4–4.0; fig. 3C).

Figure 3 Change in the relative risk (RR9 of COVID-19-related hospitalisation depending on the time since vaccination in weeks (reference is 0–12 weeks), overall (panel A), by age group (panel B) and by vaccine type (panel C).

Numbers correspond to group sizes (n), analysis for vaccine type (panel C) is based on fewer  observations, as people with unknown vaccine type (n = 120, 19%) were excluded.

Discussion

Using surveillance data on COVID-19-related hospitalizations and deaths from Switzerland between 1 July 2021 to 1 December 2021 and accounting for the dynamics of vaccine coverage over time, by age and location, we estimated that non-fully vaccinated individuals have 12.5 times the risk of hospitalization and 10.4 times the risk of death compared with fully vaccinated individuals. This corresponds to a vaccine effectiveness of 92% against hospitalisation and 90.3% against death. This is in agreement with other studies about vaccine effectiveness of mRNA vaccines [13–16]. Note that delta was the dominant SARS-CoV-2 variant in Switzerland during this time period.

We also investigated the potential waning of vaccine effectiveness over time. We found evidence of a lower vaccine effectiveness in age groups above 70, which could be caused by a weaker immune response, but could also be interpreted as an indirect evidence of waning, as older age groups were vaccinated first. Similarly, we found a decrease in vaccine effectiveness in the months of October and November 2021, which also constitutes indirect evidence of waning. Directly investigating the variation in vaccine effectiveness by time since vaccination, we did not find evidence of a reduction of vaccine effectiveness in the group13 to 24 weeks after vaccination compared to the reference group 0 to 12 weeks. In the group 25+ weeks after vaccination, vaccine effectiveness appeared to be moderately reduced (increase in the RR of hospitalisation by 1.5). To give a sense of the effect size, this value of 1.5 would correspond to a reduction of vaccine effectiveness from 92% to 88%.

On a closer look, this reduction of vaccine effectiveness in the group of people vaccinated for 25 weeks and more did not appear consistently across age groups. It was visible only in the age groups below 70, to a lesser extent in the age group 70 to 79, but not in the age group 80+. This pattern could reflect an actual faster waning of immunity in younger individuals [17], although an opposite effect with a faster waning at older age (“immunosenescence”) has also been described [18]. It could also have been due to a confounding effect, whereby individuals younger than 70 with comorbidities were more likely to have been vaccinated for a longer time (as they were prioritised in the early stages of the vaccination campaign) and were also more likely to be hospitalised upon infection with SARS-CoV-2. Confounding by occupation could also play a role, as for instance healthcare workers were both more likely to have received vaccination early and more likely to be exposed to SARS-CoV-2. In any case, this apparent decrease of vaccine effectiveness in age groups below 70 has to be seen in relation to a higher baseline vaccine effectiveness.

The reduction in vaccine effectiveness was apparent for both the Moderna and the Pfizer-BioNtech vaccines. We thus found some direct but inconclusive evidence of a moderate waning of mRNA vaccine effectiveness against hospitalisation after 25 weeks, which is in agreement with data from Israel [19], Qatar [20] and New York state [21]. However, our findings are in contrast with other studies that showed considerably faster waning among older individuals after more than 6 months [22]. Of note, the direct comparison of vaccine types suggested that Pfizer-BioNtech is associated with a slightly higher RR of hospitalisation than Moderna (relative change of 1.8, 95% CI 1.5–2.1; supplementary fig. S4), as was shown in previous studies [14].

This study has some strengths and limitations. There was a substantial proportion of hospitalisations and deaths with missing data on the vaccination status. We proposed several sensitivity analyses to circumvent this issue. Applicable without control group or long follow-up times, our approach used individual data and reverse conditionality to estimate the RR of hospitalisation or death for not fully vaccinated compared with fully vaccinated persons, taking into account the dynamics of vaccine coverage by age group and location. This quantity can be estimated using different types of stratification, and is closely related to vaccine effectiveness. Our approach relied on several assumptions. In order to interpret the RR in terms of vaccine effectiveness, we assumed that, within a population of the same age group, in the same location and during the same time frame, fully vaccinated and non-fully vaccinated individuals (1) are as likely to be exposed to the disease; (2) are as likely to be reported to surveillance authorities if they are hospitalised or deceased; and (3) are as likely to disclose their vaccination status if they are reported. Since our study looked at hospitalisations, we believe that assumptions (2) and (3) are likely to hold, but assumption (1) might be violated. If, for example, vaccinated individuals were feeling more protected owing to the vaccination, they might have been less careful and thus more exposed to the disease. This in turn would mean that the estimated RR in our study would lead to an underestimate of vaccine effectiveness. We grouped the small number of partially vaccinated individuals with the non-vaccinated, which may have led to a slight underestimation of vaccine effectiveness. However, due to the very small number of partially vaccinated (as people only remain in this state for a few weeks), the influence of partially vaccinated individuals on the overall results are likely to have been negligible. We also ignored the healthy vaccinee effect, whereby newly vaccinated people are likely to be healthier than the general population, reducing the risk of severe outcomes upon infection. We did not account for other potential confounding factors associated with both vaccination and the risk of hospitalisation or death besides age, time and location. Next to comorbid conditions and occupation, potential confounding factors include socioeconomic status [23] and differential behaviour between vaccinated and non-vaccinated people. We also did not account for increasing levels of natural immunity among non-vaccinated people as time passes, which would lead to an underestimation of effectiveness of vaccines. Investigating this last point would require precise knowledge of the history of infection in hospitalised patients and in the general population.

Conclusions

We assessed real-world vaccine effectiveness against severe forms of SARS-CoV-2 infection from routine surveillance data in Switzerland, confirming the high effectiveness of mRNA vaccines from Moderna and Pfizer-BioNtech against hospitalisation and death in all age groups. Effectiveness appeared comparatively lower in age groups over 70, suggesting the importance of booster vaccinations. We found some evidence that the effectiveness is moderately waning over time. However, confounding by comorbid conditions and the increasing levels of natural immunity among non-vaccinated in time was not accounted for. Repeated analyses will be able to better assess waning and the effect of boosters. This approach could be implemented in most routine surveillance settings to monitor vaccine effectiveness in real time.

Acknowledgements

We warmly thank Prof. Pierre-Yves Boëlle for helpful comments and discussions.

References

1. Polack FP , Thomas SJ , Kitchin N , Absalon J , Gurtman A , Lockhart S , et al.; C4591001 Clinical Trial Group . Safety and efficacy of the bnt162b2 mRNA covid-19 vaccine. N Engl J Med. 2020 Dec;383(27):2603–15. https://doi.org/10.1056/NEJMoa2034577  

2. Baden LR , El Sahly HM , Essink B , Kotloff K , Frey S , Novak R , et al.; COVE Study Group . Efficacy and safety of the mRNA-1273 sars-cov-2 vaccine. N Engl J Med. 2021 Feb;384(5):403–16. https://doi.org/10.1056/NEJMoa2035389  

3. Evans SJ , Jewell NP . Vaccine effectiveness studies in the field. N Engl J Med. 2021 Aug;385(7):650–1. https://doi.org/10.1056/NEJMe2110605  

4. Patel MK , Bergeri I , Bresee JS , Cowling BJ , Crowcroft NS , Fahmy K , et al.  Evaluation of post-introduction COVID-19 vaccine effectiveness: Summary of interim guidance of the World Health Organization. Vaccine. 2021 Jul;39(30):4013–24. https://doi.org/10.1016/j.vaccine.2021.05.099  

5. Dagan N , Barda N , Kepten E , Miron O , Perchik S , Katz MA , et al.  BNT162b2 mRNA covid-19 vaccine in a nationwide mass vaccination setting. N Engl J Med. 2021 Apr;384(15):1412–23. https://doi.org/10.1056/NEJMoa2101765  

6. Nunes B , Rodrigues AP , Kislaya I , Cruz C , Peralta-Santos A , Lima J , et al.  mRNA vaccine effectiveness against COVID-19-related hospitalisations and deaths in older adults: a cohort study based on data linkage of national health registries in Portugal, February to August 2021. Euro Surveill. 2021 Sep;26(38):2100833. https://doi.org/10.2807/1560-7917.ES.2021.26.38.2100833  

7. Bernal JL , et al. Effectiveness of the pfizer-biontech and oxford-astrazeneca vaccines on covid-19 related symptoms, hospital admissions, and mortality in older adults in england: Test negative case-control study. BMJ. 2021. 

8. Kissling E , Hooiveld M , Sandonis Martín V , Martínez-Baz I , William N , Vilcu AM , et al.; I-MOVE-COVID-19 primary care study team; . Vaccine effectiveness against symptomatic SARS-CoV-2 infection in adults aged 65 years and older in primary care: I-MOVE-COVID-19 project, Europe, December 2020 to May 2021. Euro Surveill. 2021 Jul;26(29):2100670. https://doi.org/10.2807/1560-7917.ES.2021.26.29.2100670  

9. World Health Organization   . “Evaluation of covid-19 vaccine effectiveness,” INTERIM GUIDANCE, 2021. 

10. Farrington CP . Estimation of vaccine effectiveness using the screening method. Int J Epidemiol. 1993 Aug;22(4):742–6. https://doi.org/10.1093/ije/22.4.742  

11. Mazagatos C , Monge S , Olmedo C , Vega L , Gallego P , Martín-Merino E , et al.; Working group for the surveillance and control of COVID-19 in Spain . Effectiveness of mRNA COVID-19 vaccines in preventing SARS-CoV-2 infections and COVID-19 hospitalisations and deaths in elderly long-term care facility residents, Spain, weeks 53 2020 to 13 2021. Euro Surveill. 2021 Jun;26(24):2100452. https://doi.org/10.2807/1560-7917.ES.2021.26.24.2100452  

12. van Buuren S , Groothuis-Oudshoorn K . mice: multivariate imputation by chained equations in r. J Stat Softw. 2011;45(3):1–67. Available from: https://www.jstatsoft.org/v45/i03/ https://doi.org/10.18637/jss.v045.i03  

13. Tartof SY , Slezak JM , Fischer H , Hong V , Ackerson BK , Ranasinghe ON , et al.  Effectiveness of mRNA BNT162b2 COVID-19 vaccine up to 6 months in a large integrated health system in the USA: a retrospective cohort study. Lancet. 2021 Oct;398(10309):1407–16. https://doi.org/10.1016/S0140-6736(21)02183-8  

14. Self WH , Tenforde MW , Rhoads JP , Gaglani M , Ginde AA , Douin DJ , et al.; IVY Network . Comparative effectiveness of moderna, pfizer-biontech, and janssen (johnson & johnson) vaccines in preventing covid-19 hospitalizations among adults without immunocompromising conditions-united states, march–august 2021. MMWR Morb Mortal Wkly Rep. 2021 Sep;70(38):1337–43. https://doi.org/10.15585/mmwr.mm7038e1  

15. Lopez Bernal J , Andrews N , Gower C , Gallagher E , Simmons R , Thelwall S , et al.  Effectiveness of covid-19 vaccines against the b. 1.617. 2 (delta) variant. N Engl J Med. 2021 Aug;385(7):585–94. https://doi.org/10.1056/NEJMoa2108891  

16. Bruxvoort K , et al. Effectiveness of mRNA-1273 against delta, mu, and other emerging variants,” medRxiv, 2021. https://doi.org/10.1101/2021.09.29.21264199  

17. Pollard AJ , Bijker EM . A guide to vaccinology: from basic principles to new developments. Nat Rev Immunol. 2021 Feb;21(2):83–100. https://doi.org/10.1038/s41577-020-00479-7  

18. Crooke SN , Ovsyannikova IG , Poland GA , Kennedy RB . Immunosenescence and human vaccine immune responses. Immun Ageing. 2019 Sep;16(1):25. https://doi.org/10.1186/s12979-019-0164-9  

19. Goldberg Y , Mandel M , Bar-On YM , Bodenheimer O , Freedman L , Haas EJ , et al.  Waning immunity after the bnt162b2 vaccine in israel. N Engl J Med. 2021 Dec;385(24):e85. https://doi.org/10.1056/NEJMoa2114228  

20. Chemaitelly H , Tang P , Hasan MR , AlMukdad S , Yassine HM , Benslimane FM , et al.  Waning of bnt162b2 vaccine protection against sars-cov-2 infection in qatar. N Engl J Med. 2021 Dec;385(24):e83. https://doi.org/10.1056/NEJMoa2114114  

[21] ES Rosenberg  et al. , Covid-19 vaccine effectiveness in new york state. New Engl J Med. 2022. doi: https://doi.org/10.1056/NEJMoa2116063. 

22. Nordström P , Ballin M , Nordström A . Effectiveness of covid-19 vaccination against risk of symptomatic infection, hospitalization, and death up to 9 months: A Swedish total-population cohort study. 2021. 

23. Riou J , Panczak R , Althaus CL , Junker C , Perisa D , Schneider K , et al.  Socioeconomic position and the COVID-19 care cascade from testing to mortality in Switzerland: a population-based analysis. Lancet Public Health. 2021 Sep;6(9):e683–91. https://doi.org/10.1016/S2468-2667(21)00160-2  

Appendix: Supplementary information

The appendix is available in the pdf version of the article.