الملخص
Background: On 18 May 2021, Saudi Arabia announced that, from 1 August 2021, proof of COVID-19 vaccination in the Tawakkalna application would be required to enter government and private establishments, educational institutions, events and public transport. We estimated how vaccine uptake changed around the announcement and enforcement of this requirement.
Methods: Quasi-experimental study of national weekly data (3 January to 30 October 2021; 43 weeks) from Our World in Data, with policy timing from the Oxford COVID-19 Government Response Tracker. Segmented regression with autocorrelation-robust errors modelled weekly doses per 100 population with interruptions at announcement and enforcement. A synthetic control built from 47 countries without mandatory-vaccination codes in the policy tracker estimated the gap in cumulative primary-series doses per 100 population (primary estimand), with placebo-in-space, in-time placebo, negative-control and timing-shift analyses.
Results: Mean weekly doses per 100 population rose from 1.64 before the announcement to 3.84 between announcement and enforcement, peaking at 7.51 in the first enforcement week; second doses accounted for most of the July increase. The weekly trend steepened after the announcement (slope 0.41 versus 0.21 doses per 100 per week) and reversed after enforcement (slope change −0.87 (95% CI −1.18 to −0.56)). By 30 October 2021, Saudi Arabia had administered 14.0 more cumulative doses per 100 population than its synthetic control (125.4 versus 111.4), but this gap was not unusual relative to placebo countries (permutation p = 0.60), was −4.2 at enforcement, and varied in sign across sensitivity analyses.
Conclusions: Vaccination activity in Saudi Arabia accelerated between the announcement and the enforcement of the vaccination requirement, mainly through completion of second doses. Comparison with a synthetic counterfactual did not provide robust evidence of additional cumulative uptake attributable to the policy, and the result depends on assumptions that the falsification tests only partly supported.
الكلمات المفتاحية
- COVID-19 vaccines
- vaccination mandate
- interrupted time series
- synthetic control
- health policy
- Saudi Arabia
النص الكامل
لا تتوفر نسخة عربية من النص الكامل لهذا الإصدار. المعروض أدناه هو النص الأصلي.
COVID-19 Vaccination Uptake Around Saudi Arabia's 2021 Vaccination Requirement for Access to Public and Private Establishments: An Interrupted Time-Series and Synthetic Control Analysis
ABSTRACT
Background: On 18 May 2021, Saudi Arabia announced that, from 1 August 2021, proof of COVID-19 vaccination in the Tawakkalna application would be required to enter government and private establishments, educational institutions, events and public transport. We estimated how vaccine uptake changed around the announcement and enforcement of this requirement.
Methods: Quasi-experimental study of national weekly data (3 January to 30 October 2021; 43 weeks) from Our World in Data, with policy timing from the Oxford COVID-19 Government Response Tracker. Segmented regression with autocorrelation-robust errors modelled weekly doses per 100 population with interruptions at announcement and enforcement. A synthetic control built from 47 countries without mandatory-vaccination codes in the policy tracker estimated the gap in cumulative primary-series doses per 100 population (primary estimand), with placebo-in-space, in-time placebo, negative-control and timing-shift analyses.
Results: Mean weekly doses per 100 population rose from 1.64 before the announcement to 3.84 between announcement and enforcement, peaking at 7.51 in the first enforcement week; second doses accounted for most of the July increase. The weekly trend steepened after the announcement (slope 0.41 versus 0.21 doses per 100 per week) and reversed after enforcement (slope change −0.87 (95% CI −1.18 to −0.56)). By 30 October 2021, Saudi Arabia had administered 14.0 more cumulative doses per 100 population than its synthetic control (125.4 versus 111.4), but this gap was not unusual relative to placebo countries (permutation p = 0.60), was −4.2 at enforcement, and varied in sign across sensitivity analyses.
Conclusions: Vaccination activity in Saudi Arabia accelerated between the announcement and the enforcement of the vaccination requirement, mainly through completion of second doses. Comparison with a synthetic counterfactual did not provide robust evidence of additional cumulative uptake attributable to the policy, and the result depends on assumptions that the falsification tests only partly supported.
Cite as: Alawad F, Elhassan KE, Almas K, El-Setouhy M, Fattah MA. COVID-19 Vaccination Uptake Around Saudi Arabia's 2021 Vaccination Requirement for Access to Public and Private Establishments: An Interrupted Time-Series and Synthetic Control Analysis. Multidisciplinary Healthcare Journal. 2025;1(5):MHJ-2026-000367. © The authors. Published open access under the Creative Commons Attribution 4.0 International licence (CC BY 4.0), https://creativecommons.org/licenses/by/4.0/.
1. Introduction
COVID-19 vaccination prevented a large share of the deaths that the pandemic would otherwise have caused during 2021 [1], yet high coverage depended on the willingness of eligible people to come forward, and willingness varied widely between and within countries [2,3]. Governments therefore combined supply-side measures with demand-side policies, ranging from information campaigns and financial incentives to lotteries and requirements to show proof of vaccination for access to workplaces, venues and services [4–6]. Requirements of the last type, often called vaccine certificates, passes or mandates, were among the most consequential and contested measures of the pandemic because they made participation in ordinary social and economic life conditional on vaccination status [7–9].
The evidence on whether such requirements increase uptake comes mostly from Europe and North America. A synthetic control study of six European countries found that certificate requirements were followed by increased daily vaccinations, beginning about 20 days before implementation, with larger effects where baseline uptake was below average [10]. Difference-in-differences and time-series analyses of Canadian provinces, France, Italy and Germany associated mandate announcements with a surge of more than 60% in weekly first doses [11], and modelling for France, Italy and Germany linked certificates to additional vaccinations and to health and economic benefits [12]. Subsequent interrupted time-series studies in Canada, France, Italy, California and New Orleans broadly reported increases following announcements, while noting that the size of the response depended on pre-existing coverage, hesitancy, timing and the epidemic context [13–18]. Behavioural research also suggests that mandatory approaches can provoke reactance or crowd out voluntary motivation in some groups [19,20].
Saudi Arabia provides an instructive and under-studied case. The national vaccination campaign began in December 2020 and was organised around a single digital platform: vaccination appointments, certificates and health status were integrated into the Sehhaty and Tawakkalna applications, which were already used to manage movement permissions during earlier phases of the pandemic [21,22]. On 18 May 2021 the Ministry of Interior announced that, from 1 August 2021, entry to government and private establishments, educational institutions, entertainment and sports events and the use of public transport would require an "immune" status in Tawakkalna. The policy was part of a wider package of more than thirty pandemic measures [23] and was applied at national scale with digital verification, which distinguishes it from the regionally or sectorally implemented certificates evaluated elsewhere. Before the campaign, surveys reported vaccine acceptance among Saudi residents ranging from about half to two thirds of respondents, with hesitancy concentrated in specific groups [24–26]; later surveys examined motivations for vaccination and perceptions after the requirement [27–29]. To our knowledge, the population-level change in vaccine uptake around the announcement and enforcement of the requirement has not been evaluated with a quasi-experimental design.
We therefore asked whether the announcement and enforcement of the Saudi vaccination requirement were followed by changes in the level and trend of vaccine doses administered, relative to Saudi Arabia's own pre-announcement trajectory and to a synthetic counterfactual constructed from countries that did not adopt mandatory vaccination policies during 2021. Because a single national policy cannot be randomised, we combined two complementary designs, interrupted time series and synthetic control, and subjected both to pre-specified falsification tests [30,31].
2. Materials and Methods
2.1. Study Design
This was a quasi-experimental evaluation of a national policy using publicly available aggregate data. The single-group interrupted time series (ITS) used Saudi Arabia's own pre-announcement trend as the counterfactual [32,33]. The synthetic control method (SCM) built the counterfactual from a weighted combination of comparison countries chosen to reproduce Saudi Arabia's pre-announcement trajectory [34,35]. The statistical analysis plan, including the primary estimand, donor-pool rules, sensitivity analyses and a rule for reporting artefacts, was written and frozen before effects were estimated; two later clarifications (donor-series completeness and the cross-validation scheme of one sensitivity analysis) were logged with their reasons before interpretation.
2.2. Intervention
The intervention is described in Table 1 following the Template for Intervention Description and Replication (TIDieR) logic. The requirement was announced by the Ministry of Interior on 18 May 2021 and took effect on 1 August 2021; the Oxford COVID-19 Government Response Tracker (OxCGRT) codes Saudi Arabia's mandatory-vaccination indicator as active from 1 August 2021 [36]. Because behaviour may respond when a policy is announced rather than when it binds [10,11], we treated the 11 weeks between announcement and enforcement as a distinct anticipation phase.
Table 1. Description of the intervention.
| Item | Description |
|---|---|
| Name | National requirement of COVID-19 vaccination ("immune" status) for access to establishments, events and public transport |
| Issuing authority | Ministry of Interior, Kingdom of Saudi Arabia (announced through the Saudi Press Agency) |
| Announcement | 18 May 2021 |
| Enforcement | 1 August 2021 (OxCGRT mandatory-vaccination indicator active from this date) |
| Settings | Government and private establishments, educational institutions, entertainment, sports and other events, public transport |
| Mechanism | Digital verification of vaccination status in the Tawakkalna application at points of entry |
| Target population | Residents of Saudi Arabia eligible for vaccination |
| Concurrent context | Free vaccination for citizens and residents; expanding supply and second-dose scheduling during mid-2021 |
OxCGRT, Oxford COVID-19 Government Response Tracker. Exemption rules defined by the Ministry of Health are not described in the data sources used and were not modelled.
2.3. Data Sources
Vaccination data were obtained from the Our World in Data (OWID) COVID-19 vaccination database, which compiles official country-reported cumulative counts of doses administered, people with at least one dose, people with a completed primary series and booster doses [37]. Weekly confirmed COVID-19 deaths reported to the World Health Organization were obtained from the OWID cases and deaths file, and population denominators (United Nations World Population Prospects 2022) from the OWID input file. Policy timing for all countries was taken from the OxCGRT compact national dataset [36]. All files were downloaded on 6 November 2026 from the public GitHub repositories of the two projects, frozen and checksummed; they are distributed under the Creative Commons Attribution 4.0 licence.
2.4. Study Period, Units and Outcomes
The unit of analysis was the country-week, with weeks running from Sunday to Saturday to follow the Saudi working week. The study period comprised 43 weeks from 3 January to 30 October 2021: a pre-announcement phase of 19 weeks (to 15 May), an anticipation phase of 11 weeks (16 May to 31 July) and an enforcement phase of 13 weeks (1 August to 30 October). Follow-up ended before booster doses were reported for Saudi Arabia. Cumulative counts reported on irregular dates were linearly interpolated in calendar time and read at each week end; interpolation was never extended beyond the observed range. Weekly doses were the first differences of the interpolated cumulative series.
The primary outcome was cumulative primary-series doses per 100 population, defined as total doses administered minus booster doses (boosters set to zero before a country's first booster report). This definition prevents booster campaigns that began in some comparison countries in late 2021 from inflating the counterfactual. The ITS outcome was weekly doses per 100 population. Because Saudi Arabia reported first and completed-series counts separately only from late June 2021, the split between first and second doses was described for July to October without inferential modelling. Weekly COVID-19 deaths per million served as a negative-control outcome for which an immediate change at the announcement would be implausible.
2.5. Comparison Countries
Donor countries for the SCM were selected by rules fixed in advance: population of at least one million; a reported cumulative dose value within 10 days of every week end and a valid interpolated value at every week end; no OxCGRT mandatory-vaccination coding at any time between 1 January and 31 October 2021; and exclusion of the six countries whose certificate requirements were evaluated by Mills and Rüttenauer (Denmark, France, Germany, Israel, Italy and Switzerland) [10]. Countries meeting these rules were retained without inspection of their outcomes. 47 countries were eligible; Gulf states that adopted requirements in 2021 were excluded by the OxCGRT rule.
2.6. Statistical Analysis
The primary estimand was the difference between observed and counterfactual cumulative primary-series doses per 100 population in Saudi Arabia at the end of follow-up (week ending 30 October 2021), with exposure beginning in the announcement week. The identifying assumption was that a weighted combination of donor countries that reproduced Saudi Arabia's pre-announcement trajectory would have continued to approximate it in the absence of the policy, with no spillover between countries and no concurrent Saudi-specific shock.
SCM weights were constrained to be non-negative and to sum to one and were chosen to minimise the mean squared prediction error over the 19 pre-announcement weeks, using every pre-period value of the outcome as a predictor [34,35]. The quality of pre-period fit was summarised by the root mean squared prediction error (RMSPE). Inference used placebo-in-space permutation: each donor in turn was treated as if it had adopted the policy and its own synthetic control was built from the remaining donors. We report two permutation p-values: the rank of Saudi Arabia's post-to-pre RMSPE ratio among all units, and the rank of the absolute end-of-follow-up gap among placebos whose pre-period RMSPE was no more than five times that of Saudi Arabia [34,38]. We also report the gap at 31 July 2021, the end of the anticipation phase.
For the single-group ITS, weekly doses per 100 population were modelled by segmented linear regression with terms for the baseline level and trend, a level change and a trend change at the announcement week, and a further level and trend change at the enforcement week [32,33]. Standard errors were heteroskedasticity- and autocorrelation-consistent (Newey–West, Bartlett kernel, lag of three weeks chosen by the rule floor(4(n/100)2/9)) [39]. Residual autocorrelation was summarised by the Durbin–Watson statistic and the Ljung–Box test at lag 4 [40]. Phase-specific slopes and their confidence intervals were obtained as linear combinations of coefficients.
Pre-specified sensitivity analyses for the SCM were: onset at enforcement; onset moved two and four weeks earlier than the announcement; exclusion of Middle Eastern and North African donors; restriction to donors whose cumulative level at 15 May 2021 was within ±50% of Saudi Arabia's; total doses including boosters as the outcome; leave-one-out removal of each donor with a weight of at least 0.01; and a ridge-augmented synthetic control, a penalised-regression (machine-learning) correction for imperfect pre-period fit, with the penalty chosen by hold-out validation on the last six pre-period weeks [41,42]. For the ITS, sensitivity analyses used regression with first-order autoregressive errors, a baseline starting in March 2021 after vaccine eligibility had been extended to all adults, and a logarithmic outcome [43,44]. Falsification tests comprised an in-time placebo with a pseudo-onset on 21 March 2021 using only pre-announcement data, the negative-control outcome, and the permutation distributions. A rule defined in advance flagged reporting artefacts (a weekly increment that was negative or more than three times the median of the four preceding and four following increments). The primary SCM gap and the ITS coefficients were recomputed by an independent implementation using a different optimisation algorithm (non-negative least squares) applied to series re-derived from the raw files and hand-coded Newey–West estimation.
One primary estimand was defined; secondary estimates are reported with 95% confidence intervals as supporting evidence without multiplicity adjustment. Analyses used Python 3 with pandas, NumPy, SciPy and statsmodels.
2.7. Reporting Guideline
The study is reported in accordance with the Transparent Reporting of Evaluations with Nonrandomized Designs (TREND) statement [45], with elements of guidance for interrupted time-series and synthetic control studies in public health [30,32,46].
2.8. Ethical Considerations
The study used only publicly available, aggregate, de-identified national statistics. No individual-level data were accessed, and ethical approval and informed consent were not required.
3. Results
3.1. Descriptive Trends
Saudi Arabia had administered 31.6 doses per 100 population by 15 May 2021, compared with a median of 16.4 among donor countries. Cumulative doses reached 73.8 per 100 population by 31 July and 125.4 by 30 October 2021 (donor median 95.4). Mean weekly doses per 100 population were 1.64 in the pre-announcement phase, 3.84 in the anticipation phase and 3.97 in the enforcement phase; the highest weekly value, 7.51, occurred in the first enforcement week, ending 7 August 2021 (Table 2, Figure 1). No week met the pre-specified artefact rule.
Between 3 and 31 July 2021, the share of the population with at least one dose rose from 45.6% to 51.6%, whereas the share with a completed primary series rose from 4.7% to 22.2%. Over the four weeks to 31 July, about 6.0 first doses and 17.5 second doses per 100 population were administered. By 30 October, coverage reached 66.4% for at least one dose and 58.9% for the completed series (Figure 4A).
Table 2. Vaccine doses administered in Saudi Arabia by policy phase, January–October 2021.
| Phase | Weeks | Dates | Mean weekly doses per 100 | Cumulative doses per 100 at end of phase |
|---|---|---|---|---|
| Pre-announcement | 19 | 3 Jan–15 May 2021 | 1.64 | 31.6 |
| Anticipation | 11 | 16 May–31 Jul 2021 | 3.84 | 73.8 |
| Enforcement | 13 | 1 Aug–30 Oct 2021 | 3.97 | 125.4 |
Doses are all doses administered (no booster doses were reported for Saudi Arabia during the period). Population denominator: United Nations World Population Prospects 2022.
Figure 1. Weekly COVID-19 vaccine doses per 100 population in Saudi Arabia with segmented regression fit, January–October 2021. Dashed lines mark announcement and enforcement; the dotted line projects the pre-announcement trend. Doses rose in July and declined after enforcement.
3.2. Interrupted Time Series
Before the announcement, weekly doses increased by 0.21 (95% CI 0.16 to 0.27) doses per 100 population per week (Table 3). The announcement week was not followed by a clear level change (−1.99 (95% CI −4.11 to 0.13)), whereas the trend became steeper: the slope in the anticipation phase was 0.41 (95% CI 0.13 to 0.69) doses per 100 per week, a change of 0.19 (95% CI −0.08 to 0.46) relative to baseline. At enforcement there was no detectable level change (0.45 (95% CI −1.40 to 2.30)), but the trend reversed (slope change −0.87 (95% CI −1.18 to −0.56); slope during enforcement −0.46 (95% CI −0.54 to −0.39)). Residual autocorrelation was moderate (Durbin–Watson 1.40; Ljung–Box lag 4 p = 0.022). Estimates were similar with autoregressive errors (estimated first-order coefficient 0.37) and with a March baseline, in which the anticipation slope change was 0.28 (95% CI 0.02 to 0.55). On the logarithmic scale, however, the announcement was associated with a lower level and a small negative slope change, indicating that the apparent acceleration reflects the absolute volume of doses rather than a proportional change in growth.
Table 3. Segmented regression of weekly vaccine doses per 100 population in Saudi Arabia.
| Parameter | Primary model, estimate (95% CI) | p | AR(1) errors (95% CI) | Baseline from March 2021 (95% CI) |
|---|---|---|---|---|
| Baseline weekly trend | 0.21 (95% CI 0.16 to 0.27) | p < 0.001 | 0.19 (95% CI 0.00 to 0.37) | 0.12 (95% CI 0.04 to 0.20) |
| Level change at announcement | −1.99 (95% CI −4.11 to 0.13) | p = 0.066 | −1.16 (95% CI −2.57 to 0.25) | −1.62 (95% CI −3.75 to 0.51) |
| Trend change after announcement | 0.19 (95% CI −0.08 to 0.46) | p = 0.159 | 0.17 (95% CI −0.11 to 0.44) | 0.28 (95% CI 0.02 to 0.55) |
| Level change at enforcement | 0.45 (95% CI −1.40 to 2.30) | p = 0.635 | 0.53 (95% CI −1.62 to 2.68) | 0.45 (95% CI −1.44 to 2.33) |
| Trend change after enforcement | −0.87 (95% CI −1.18 to −0.56) | p < 0.001 | −0.80 (95% CI −1.08 to −0.53) | −0.87 (95% CI −1.18 to −0.56) |
Primary model: ordinary least squares with Newey–West standard errors (lag 3), 43 weeks. AR(1), first-order autoregressive error model. CI, confidence interval. Trend parameters are in doses per 100 population per week; level parameters in doses per 100 population.
3.3. Synthetic Control
The synthetic control for Saudi Arabia combined India (0.478), Singapore (0.317), Dominican Republic (0.164), Mongolia (0.041), and reproduced the pre-announcement trajectory closely (pre-period RMSPE 0.58 doses per 100 population; Figure 2). During the anticipation phase, observed cumulative doses tracked slightly below the synthetic control; at 31 July 2021 the gap was −4.2 doses per 100 population (73.8 observed versus 78.0 synthetic). During enforcement, Saudi Arabia overtook its synthetic control, and by 30 October 2021 the gap was 14.0 doses per 100 population (125.4 versus 111.4).
This gap was not distinguishable from the distribution of placebo gaps (Figure 3). Among 44 placebo countries with acceptable pre-period fit, the permutation p-value for the end-of-follow-up gap was 0.60; for the post-to-pre RMSPE ratio across all 47 placebos, it was 0.88. The large ratio for Saudi Arabia (16.3) reflects its very small pre-period error rather than an unusually large post-period divergence.
Figure 2. Cumulative primary-series doses per 100 population in Saudi Arabia and its synthetic control (India, Singapore, the Dominican Republic and Mongolia), January–October 2021. Saudi Arabia tracked slightly below its synthetic control until enforcement and above it thereafter.
Figure 3. Gap between observed and synthetic cumulative doses per 100 population for Saudi Arabia and 44 placebo countries with acceptable pre-period fit (grey). The Saudi gap lies within the range of placebo gaps throughout follow-up.
3.4. Sensitivity and Falsification Analyses
The direction and size of the end-of-follow-up gap depended on specification (Table 4, Figure 5). Moving the onset to enforcement gave a gap of 20.3 doses per 100 population (permutation p = 0.22), and moving it two or four weeks earlier gave 8.9 and 8.0. With earlier onsets the gap at 31 July was clearly negative (−16.6 and −26.8), meaning that the synthetic control predicted faster accumulation during June and July than was observed. Restricting the donor pool to countries with a similar level of coverage in mid-May reversed the sign of the end-of-follow-up gap (−5.5). Excluding Middle Eastern and North African donors did not change the estimate because none received positive weight. Using total doses including boosters reduced the gap to 7.4. Leave-one-out estimates ranged from 7.5 to 16.8. Hold-out validation for the ridge-augmented synthetic control selected the largest penalty on the grid, so the augmentation added essentially no correction and the estimate coincided with the primary analysis (14.0 at 30 October); an earlier run with a one-step validation scheme had selected the smallest penalty and produced an implausible, extrapolated counterfactual, which motivated the logged change of scheme.
The in-time placebo did not pass: a pseudo-onset on 21 March 2021, estimated with pre-announcement data only, produced a gap of −20.9 doses per 100 population by 15 May 2021, larger in absolute value than the primary estimate. This shows that, even without any policy, the synthetic control could diverge from Saudi Arabia by more than ten doses per 100 within eight weeks. For the negative-control outcome, weekly deaths showed no clear level change at the announcement (0.75 (95% CI −0.10 to 1.60) deaths per million), consistent with the absence of an immediate artefact at that date; deaths declined from August as the epidemic wave subsided (Figure 4B). Table 5 summarises the falsification results.
Table 4. Synthetic control estimates: primary analysis and sensitivity analyses.
| Specification | Gap at 31 July 2021 | Gap at 30 October 2021 | Pre-period RMSPE | Permutation p (end gap) |
|---|---|---|---|---|
| Primary (onset at announcement) | −4.2 | 14.0 | 0.58 | 0.60 |
| Onset at enforcement | 1.4 | 20.3 | 1.21 | 0.22 |
| Onset 2 weeks earlier | −16.6 | 8.9 | 0.44 | 0.73 |
| Onset 4 weeks earlier | −26.8 | 8.0 | 0.39 | 0.72 |
| Donors with similar coverage level | −22.8 | −5.5 | 1.08 | 0.63 |
| Total doses including boosters | −4.8 | 7.4 | 0.58 | 0.71 |
| Ridge-augmented synthetic control | −4.2 | 14.0 | 0.58 | NA |
| Leave-one-donor-out (range) | NA | 7.5 to 16.8 | NA | NA |
Gaps are observed minus synthetic cumulative primary-series doses per 100 population. RMSPE, root mean squared prediction error. Permutation p: rank of the absolute end-of-follow-up gap among placebo countries with pre-period RMSPE no more than five times that of Saudi Arabia. Excluding Middle Eastern and North African donors gave results identical to the primary analysis. NA, not applicable.
Figure 4. (A) Coverage with at least one dose and with a completed primary series, July–October 2021. (B) Weekly COVID-19 deaths per million (negative-control outcome). The July acceleration came mainly from completed series; deaths showed no change at the announcement.
Figure 5. Gap in cumulative primary-series doses per 100 population at 30 October 2021 across synthetic control specifications; the bar shows the leave-one-donor-out range. Estimates vary in size and sign, indicating sensitivity to modelling choices.
Table 5. Falsification tests and their interpretation.
| Test | Result | Interpretation |
|---|---|---|
| Placebo in space (end gap) | p = 0.60 (44 placebos) | Saudi gap within placebo distribution |
| Placebo in space (RMSPE ratio) | p = 0.88 (47 placebos) | No unusual post-period divergence |
| In-time placebo (21 March 2021) | Gap −20.9 by 15 May | Failed: counterfactual unstable over 8 weeks |
| Negative-control outcome (deaths) | Level change 0.75 (−0.10 to 1.60) | Passed at announcement |
| Reporting-artefact rule | No weeks flagged | No artefact-driven estimates |
| Timing shift (−2, −4 weeks, enforcement) | End gaps 8.9, 8.0, 20.3 | Same sign, varying size |
Gaps in cumulative primary-series doses per 100 population; death level change in deaths per million per week (95% confidence interval). RMSPE, root mean squared prediction error.
4. Discussion
4.1. Principal Findings
Vaccination activity in Saudi Arabia increased markedly in the weeks between the announcement and the enforcement of the national vaccination requirement and peaked in the first week of enforcement, after which weekly doses declined steadily. The July surge consisted mainly of second doses, as people who had received a first dose earlier in the year completed their primary series. The single-group time series is compatible with an anticipatory response, but when the counterfactual was constructed from countries without requirements, Saudi Arabia's cumulative uptake was not clearly different from what would be expected: it was slightly below its synthetic control at enforcement and above it by the end of October, and neither gap stood out against the placebo distribution. The in-time placebo and the dependence of the estimate on the donor pool indicate that the comparative design cannot separate a modest policy effect from the variability of vaccination trajectories across countries in 2021.
4.2. Comparison with Other Studies
The timing of the Saudi response resembles that reported for European certificates, where vaccinations rose about three weeks before implementation [10], and the Canadian experience of an announcement-driven surge [11,13]. Our findings differ in two respects. First, the Saudi increase was dominated by second doses rather than first doses. This pattern is consistent with the design of the requirement, which linked the "immune" status to vaccination history, and with the scheduling of second doses that had been deferred while supply was constrained, as in other countries that prioritised first doses [6]. Because first and second doses cannot be separated before late June 2021 in the public data, the relative contributions of the policy and of supply-driven second-dose scheduling cannot be disentangled. Second, the comparative analysis did not detect a robust additional effect on cumulative uptake. Studies that found larger effects often evaluated settings with lower or stagnating uptake at the time of the announcement [10,14,16]; studies in settings with high coverage reported small effects, such as increases of less than one percentage point in first-dose coverage after passport announcements in Quebec and Ontario [17]. Saudi Arabia was above the donor median at the time of the announcement, which may have limited the scope for additional uptake. The findings are also consistent with reviews concluding that the behavioural effects of health certification vary with context and may bring harms as well as benefits [7,9].
The two designs answer different questions, and their apparent disagreement is informative. The single-group time series compares Saudi Arabia with its own early-2021 trajectory, which was shaped by limited supply and staged eligibility; any later expansion of supply would appear as a departure from that trend whether or not the policy contributed. The synthetic control compares Saudi Arabia with countries that faced the same global supply environment in mid-2021 and also accelerated their campaigns during the northern summer, which is why much of the Saudi increase was matched by the counterfactual. Under the assumptions of the synthetic control, the policy is therefore more plausibly associated with a shift in the timing of doses, from the weeks immediately before enforcement into the enforcement period, than with a large net gain. The reversal of the weekly trend after enforcement is also expected when an increasing share of the eligible population has completed its primary series, and this saturation cannot be separated from the end of the anticipation incentive. These interpretations remain conditional on the absence of other Saudi-specific changes in mid-2021, such as adjustments to dose intervals or appointment capacity, that the public data do not record.
4.3. Relevance for Saudi Arabia and the Region
Saudi Arabia's requirement was distinctive in being national, digitally verified through a single application and applied across nearly all public settings [21–23]. Such infrastructure allows rapid and uniform implementation, and the steep rise in completed primary series before enforcement suggests that people responded to the announced deadline. For future health emergencies, and for recurring mass-gathering requirements such as those for Hajj and Umrah [47,48], the results suggest that announcements with a clear lead time may concentrate demand into a short window, which requires matching supply and appointment capacity. They also suggest that requirements may affect the timing of vaccination more than its eventual level when coverage is already high. Evaluations of subsequent booster requirements [29] and of individual-level determinants of uptake would add to this evidence.
4.4. Strengths and Limitations
Strengths include the use of openly available, version-controlled data with reproducible processing; a statistical analysis plan fixed before estimation; the combination of two designs with different identifying assumptions; separate modelling of announcement and enforcement; a primary outcome that excludes booster doses; and a pre-specified battery of falsification tests whose failures are reported. The primary estimate was reproduced by an independent implementation.
Several limitations apply. National aggregate data cannot show who responded to the requirement, and the analysis cannot separate effects on residents of different nationality, age or region. Saudi first-dose counts are unavailable before late June 2021, so the main ITS outcome mixes first and second doses whose timing was also shaped by supply and scheduling decisions. Other national measures and the summer epidemic wave occurred concurrently and may have influenced uptake. The synthetic control relies on a small number of heavily weighted donors whose vaccination campaigns were shaped by their own supply constraints, policies and epidemics, and the failed in-time placebo shows that the counterfactual can drift substantially over a few weeks. The OxCGRT indicator may not capture all certificate-type policies in donor countries, which would bias the comparison towards the null. Finally, the requirement was a bundle of setting-specific rules, and we could not evaluate the components separately.
4.5. Implications for Policy and Research
Policy-makers considering vaccination requirements should expect responses to begin at announcement, plan supply and appointment capacity for the period before enforcement, and monitor whether the response reflects new vaccinations or the completion of courses already begun. Research using individual-level vaccination records linked to residence, age and nationality, together with first-dose series for the full period, would allow stronger designs, such as comparisons of groups more and less affected by the requirement, and could estimate effects on coverage gaps rather than on national totals.
5. Conclusions
Weekly COVID-19 vaccinations in Saudi Arabia accelerated after the national vaccination requirement was announced and peaked when it took effect, mainly through completion of second doses. A synthetic counterfactual built from countries without requirements did not show a robust additional gain in cumulative doses, and falsification tests indicate that the comparative estimate is sensitive to modelling assumptions. The requirement appears to have concentrated vaccination in time; whether it increased the eventual level of coverage cannot be established from national aggregate data.
Declarations
Author Contributions: NA
Funding: NA
Conflicts of Interest: NA
Institutional Review Board Statement: Not applicable: secondary analysis of publicly available, de-identified aggregate data.
Informed Consent Statement: Not applicable.
Data Availability Statement: All data are publicly available: Our World in Data COVID-19 vaccinations (https://github.com/owid/covid-19-data/tree/master/public/data/vaccinations), Our World in Data cases and deaths (https://github.com/owid/covid-19-data/tree/master/public/data/cases_deaths), and the Oxford COVID-19 Government Response Tracker (https://github.com/OxCGRT/covid-policy-dataset). The analysis code is available from the authors.
References
[1] Watson OJ, Barnsley G, Toor J, Hogan AB, Winskill P, Ghani AC. Global impact of the first year of COVID-19 vaccination: a mathematical modelling study. The Lancet Infectious Diseases. 2022;22(9):1293-1302. doi:10.1016/s1473-3099(22)00320-6
[2] Sallam M. COVID-19 Vaccine Hesitancy Worldwide: A Concise Systematic Review of Vaccine Acceptance Rates. Vaccines. 2021;9(2):160. doi:10.3390/vaccines9020160
[3] Lazarus JV, Wyka K, White TM, Picchio CA, Gostin LO, Larson HJ, et al.. A survey of COVID-19 vaccine acceptance across 23 countries in 2022. Nat Med. 2023;29(2):366-375. doi:10.1038/s41591-022-02185-4
[4] Walkey AJ, Law A, Bosch NA. Lottery-Based Incentive in Ohio and COVID-19 Vaccination Rates. JAMA. 2021;326(8):766. doi:10.1001/jama.2021.11048
[5] Thirumurthy H, Milkman KL, Volpp KG, Buttenheim AM, Pope DG. Association between statewide financial incentive programs and COVID-19 vaccination rates. PLoS ONE. 2022;17(3):e0263425. doi:10.1371/journal.pone.0263425
[6] Kuznetsova L, Diago-Navarro E, Mathu R, Trilla A. Effectiveness of COVID-19 Vaccination Mandates and Incentives in Europe. Vaccines. 2022;10(10):1714. doi:10.3390/vaccines10101714
[7] Bardosh K, de Figueiredo A, Gur-Arie R, Jamrozik E, Doidge J, Lemmens T, et al.. The unintended consequences of COVID-19 vaccine policy: why mandates, passports and restrictions may cause more harm than good. BMJ Glob Health. 2022;7(5):e008684. doi:10.1136/bmjgh-2022-008684
[8] Ward JK, Gauna F, Gagneux-Brunon A, Botelho-Nevers E, Cracowski JL, Khouri C, et al.. The French health pass holds lessons for mandatory COVID-19 vaccination. Nat Med. 2022;28(2):232-235. doi:10.1038/s41591-021-01661-7
[9] Drury J, Mao G, John A, Kamal A, Rubin GJ, Stott C, et al.. Behavioural responses to Covid-19 health certification: a rapid review. BMC Public Health. 2021;21(1):1205. doi:10.1186/s12889-021-11166-0
[10] Mills MC, Rüttenauer T. The effect of mandatory COVID-19 certificates on vaccine uptake: synthetic-control modelling of six countries. The Lancet Public Health. 2022;7(1):e15-e22. doi:10.1016/s2468-2667(21)00273-5
[11] Karaivanov A, Kim D, Lu SE, Shigeoka H. COVID-19 vaccination mandates and vaccine uptake. Nat Hum Behav. 2022;6(12):1615-1624. doi:10.1038/s41562-022-01363-1
[12] Oliu-Barton M, Pradelski BSR, Woloszko N, Guetta-Jeanrenaud L, Aghion P, Artus P, et al.. The effect of COVID certificates on vaccine uptake, health outcomes, and the economy. Nat Commun. 2022;13(1):3942. doi:10.1038/s41467-022-31394-1
[13] Maquiling A, Jeevakanthan A, Ho Mi Fane B. The effect of vaccine mandate announcements on vaccine uptake in Canada: An interrupted time series analysis. Vaccine. 2023;41(18):2932-2940. doi:10.1016/j.vaccine.2023.03.040
[14] Santolini R. The COVID-19 green certificate’s effect on vaccine uptake in French and Italian regions. Journal of Policy Modeling. 2023;45(5):1036-1057. doi:10.1016/j.jpolmod.2023.06.005
[15] Ratnayake A, Stoecker C, Kissinger PJ. Effect of a Vaccine Mandate in Nonessential Public Spaces in New Orleans, Louisiana, on COVID-19 Vaccine Uptake, 2021-2022. Public Health Rep. 2025:00333549251315073. doi:10.1177/00333549251315073
[16] Gebremariam AG, Genie M, Le H, Regan A, Beard F, Macartney K, et al.. Vaccine uptake in the context of mandate announcement and removal: Evidence from Europe and North America. Vaccine. 2026;88:128825. doi:10.1016/j.vaccine.2026.128825
[17] Anato JLF, Ma H, Hamilton MA, Xia Y, Harper S, Buckeridge D, et al.. Impact of a vaccine passport on first-dose SARS-CoV-2 vaccine coverage by age and area-level social determinants of health in the Canadian provinces of Quebec and Ontario: an interrupted time series analysis. cmajo. 2023;11(5):E995-E1005. doi:10.9778/cmajo.20220242
[18] Campanozzi LL, Tambone V, Ciccozzi M. A Lesson from the Green Pass Experience in Italy: A Narrative Review. Vaccines. 2022;10(9):1483. doi:10.3390/vaccines10091483
[19] Sprengholz P, Betsch C, Böhm R. Reactance revisited: Consequences of mandatory and scarce vaccination in the case of COVID‐19. Applied Psych Health & Well. 2021;13(4):986-995. doi:10.1111/aphw.12285
[20] Schmelz K, Bowles S. Overcoming COVID-19 vaccination resistance when alternative policies affect the dynamics of conformism, social norms, and crowding out. Proc. Natl. Acad. Sci. U.S.A.. 2021;118(25):e2104912118. doi:10.1073/pnas.2104912118
[21] Khan A, Alahmari A, Almuzaini Y, Alturki N, Aburas A, Alamri FA, et al.. The Role of Digital Technology in Responding to COVID-19 Pandemic: Saudi Arabia’s Experience. RMHP. 2021;Volume 14:3923-3934. doi:10.2147/rmhp.s317511
[22] Samha AK, Alrashdi AM, Alshammri GH. The influencing factors of digital health passport adoption and acceptance during COVID-19 in Saudi Arabia. DIGITAL HEALTH. 2022;8:205520762211426. doi:10.1177/20552076221142668
[23] Amamou M, Ben-Ahmed K. Managing the COVID-19 pandemic in thirty-two policy measures in Saudi Arabia: A mixed-methods analysis. Journal of Infection and Public Health. 2023;16(10):1650-1658. doi:10.1016/j.jiph.2023.08.008
[24] Al-Mohaithef M, Padhi BK. <p>Determinants of COVID-19 Vaccine Acceptance in Saudi Arabia: A Web-Based National Survey</p>. JMDH. 2020;Volume 13:1657-1663. doi:10.2147/jmdh.s276771
[25] Alfageeh EI, Alshareef N, Angawi K, Alhazmi F, Chirwa GC. Acceptability of a COVID-19 Vaccine among the Saudi Population. Vaccines. 2021;9(3):226. doi:10.3390/vaccines9030226
[26] Alrajeh AM, Daghash H, Buanz SF, Altharman HA, Belal S. COVID-19 Vaccine Hesitancy Among the Adult Population in Saudi Arabia. Cureus. 2021. doi:10.7759/cureus.20197
[27] Gray J, AlAnazi AA, AlSumait F, Abu-Shaheen A, Bashir MS, Al Sheef M. What motivated residents of Saudi Arabia to receive the COVID-19 vaccine?. Front. Public Health. 2023;11:1065157. doi:10.3389/fpubh.2023.1065157
[28] Alsuhebany N, Alowais SA, Aldairem A, Almohareb SN, Bin Saleh K, Kahtani KM, et al.. Identifying gaps in vaccination perception after mandating the COVID-19 vaccine in Saudi Arabia. Vaccine. 2023;41(24):3611-3616. doi:10.1016/j.vaccine.2023.04.057
[29] Alshahrani NZ, Alsabaani AA, Ridda I, Rashid H, Alzahrani F, Almutairi TH, et al.. Uptake of COVID-19 Booster Dose among Saudi Arabian Population. Medicina. 2022;58(7):972. doi:10.3390/medicina58070972
[30] Craig P, Katikireddi SV, Leyland A, Popham F. Natural Experiments: An Overview of Methods, Approaches, and Contributions to Public Health Intervention Research. Annu. Rev. Public Health. 2017;38(1):39-56. doi:10.1146/annurev-publhealth-031816-044327
[31] Lopez Bernal J, Cummins S, Gasparrini A. The use of controls in interrupted time series studies of public health interventions. International Journal of Epidemiology. 2018;47(6):2082-2093. doi:10.1093/ije/dyy135
[32] Lopez Bernal J, Cummins S, Gasparrini A. Interrupted time series regression for the evaluation of public health interventions: a tutorial. Int. J. Epidemiol.. 2016:dyw098. doi:10.1093/ije/dyw098
[33] Wagner AK, Soumerai SB, Zhang F, Ross-Degnan D. Segmented regression analysis of interrupted time series studies in medication use research. J Clin Pharm Ther. 2002;27(4):299-309. doi:10.1046/j.1365-2710.2002.00430.x
[34] Abadie A, Diamond A, Hainmueller J. Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program. Journal of the American Statistical Association. 2010;105(490):493-505. doi:10.1198/jasa.2009.ap08746
[35] Abadie A. Using Synthetic Controls: Feasibility, Data Requirements, and Methodological Aspects. Journal of Economic Literature. 2021;59(2):391-425. doi:10.1257/jel.20191450
[36] Hale T, Angrist N, Goldszmidt R, Kira B, Petherick A, Phillips T, et al.. A global panel database of pandemic policies (Oxford COVID-19 Government Response Tracker). Nat Hum Behav. 2021;5(4):529-538. doi:10.1038/s41562-021-01079-8
[37] Mathieu E, Ritchie H, Ortiz-Ospina E, Roser M, Hasell J, Appel C, et al.. A global database of COVID-19 vaccinations. Nat Hum Behav. 2021;5(7):947-953. doi:10.1038/s41562-021-01122-8
[38] Firpo S, Possebom V. Synthetic Control Method: Inference, Sensitivity Analysis and Confidence Sets. Journal of Causal Inference. 2018;6(2):20160026. doi:10.1515/jci-2016-0026
[39] Newey WK, West KD. A Simple, Positive Semi-Definite, Heteroskedasticity and Autocorrelation Consistent Covariance Matrix. Econometrica. 1987;55(3):703. doi:10.2307/1913610
[40] LJUNG GM, BOX GEP. On a measure of lack of fit in time series models. Biometrika. 1978;65(2):297-303. doi:10.1093/biomet/65.2.297
[41] Ben-Michael E, Feller A, Rothstein J. The Augmented Synthetic Control Method. Journal of the American Statistical Association. 2021;116(536):1789-1803. doi:10.1080/01621459.2021.1929245
[42] Ferman B, Pinto C. Synthetic controls with imperfect pretreatment fit. QE. 2021;12(4):1197-1221. doi:10.3982/qe1596
[43] Schaffer AL, Dobbins TA, Pearson SA. Interrupted time series analysis using autoregressive integrated moving average (ARIMA) models: a guide for evaluating large-scale health interventions. BMC Med Res Methodol. 2021;21(1):58. doi:10.1186/s12874-021-01235-8
[44] Turner SL, Karahalios A, Forbes AB, Taljaard M, Grimshaw JM, McKenzie JE. Comparison of six statistical methods for interrupted time series studies: empirical evaluation of 190 published series. BMC Med Res Methodol. 2021;21(1):134. doi:10.1186/s12874-021-01306-w
[45] Des Jarlais DC, Lyles C, Crepaz N, the TREND Group. Improving the Reporting Quality of Nonrandomized Evaluations of Behavioral and Public Health Interventions: The TREND Statement. Am J Public Health. 2004;94(3):361-366. doi:10.2105/ajph.94.3.361
[46] Bouttell J, Craig P, Lewsey J, Robinson M, Popham F. Synthetic control methodology as a tool for evaluating population-level health interventions. J Epidemiol Community Health. 2018;72(8):673-678. doi:10.1136/jech-2017-210106
[47] Alshamrani M, Farahat F, Alzunitan M, Hasan MA, Alsherbini N, Albarrak A, et al.. Hajj vaccination strategies: Preparedness for risk mitigation. Journal of Infection and Public Health. 2024;17(11):102547. doi:10.1016/j.jiph.2024.102547
[48] Al-Tawfiq JA, Memish ZA. COVID-19 in the Eastern Mediterranean Region and Saudi Arabia: prevention and therapeutic strategies. International Journal of Antimicrobial Agents. 2020;55(5):105968. doi:10.1016/j.ijantimicag.2020.105968

التعليقات
التعليق يتطلب حساباً وتسجيل الدخول.