A study of the impact of policy interventions on daily COVID scenario in India using interrupted time series analysis

Introduction

Interrupted Time Series (ITS), also known as quasi-experimental time series analysis is a kind of statistical analysis used to capture the change in trend in time series data before and after a point of intervention. The influence of the intervention is examined based on the statistical significance of the intervention parameters. This is a counterfactual model that works when (a) the intervention begins at a specific known point in time, and (b) we have successive data on the metrics we are interested in, before and during the intervention. ITS design has some similarities with the regression discontinuity (RD) design. The usage of this kind of analysis for measuring intervention effects is seen across different fields ranging from epidemiology to medical research, public health, social sciences, and econometrics. Using this technique, the influence of a certain welfare policy, a drug, a campaign, or any other such intervention can be studied by collecting and assessing data before and after the introduction of the intervention. These techniques are used widely in instances where randomized control trials (some classical examples of these would be clinical drug trials, A/B testing) are either unethical or not feasible. Kontopantelis et al. (2015) conducted an ITS analysis to assess the introduction of a voluntary annual reward and incentive program in UK primary care and laid an example of an instance where full randomization was not possible by clearly outlining the manner in which ITS was modelled using different regression models and the interpretation of the coefficients of the three time varying covariates. Hoffman et al. (2019) conducted a research to quasi-experimentally gauge the impact of the WHO Framework Convention on Tobacco Control (FCTC) on global cigarette consumption using ITS analysis and in-sample forecast event model and came up with some interesting conclusions.

The outbreak of the COVID-19 pandemic generated enormous quantity of data from different sectors and in different domains across the world and opened the floodgates of research in this direction to study its impact and possible measures to curb its spread. Silva, Filho, and Fernandes (2020) studied the effect of lockdown on the COVID-19 pandemic in four city capitals in Brazil through ITS design. Hamadani et al. (2020) used ITS design to evaluate the socio-economic health, food insecurity, mental health and domestic violence scenario in rural Bangladesh post the imposition of lockdown to curb COVID-19 in 2020. ITS analysis was also used to see the effect of the pandemic induced lockdown in ambulatory clinic visits (see Siedner et al. 2020a) which found a significant but temporary plummet in child healthcare visitation. Mulholland et al. (2020) employed ITS to understand the impact of COVID-19 on healthcare services in Scotland and was able to depict through its analysis the major impact it had on emergency hospital admissions and the planned ones and the overall burden on the healthcare services. Vokó and Pitter (2020) detected the change-points in the flow of the COVID-19 cases in twenty eight European countries and measured the association of the level of social distancing with the observed decline in the national pandemics by threshold regression setup. Siedner et al. (2020b) conducted ITS analysis using mixed effects linear regression models to study the log differences in daily cases and COVID-19 attributed mortality cases before and after the implementation of social distancing norms in the US. Figueiredo et al. (2020) and Thayer et al. (2021) presented studies with similar motivation as us to employ ITS analysis to assess the effects of the lockdown policy in curtailing the spread of the virus. Schaffer, Dobbins, and Pearson (2021) performed ITS analysis using autoregressive integrated moving average (ARIMA) models to evaluate the impact of large-scale interventions.

The ghastly impact of the COVID-19 pandemic was felt by countries across the world leaving the governments, think tanks and healthcare organisations overwhelmed. An immediate action taken by most governments was the imposition of a complete lockdown to curtail the spread of the virus. In India, the second wave (started around March, 2021) of the virus specifically proved to be more deadly since the daily number of confirmed cases began to soar which also led to a rise in the death tolls. Delhi and Maharashtra topped the charts of the union territories and states respectively where the rise in the daily confirmed cases began to overburden the healthcare facilities compelling the governments to enforce a complete lockdown along with speeding up the vaccination process which was introduced from the start of 2021.

Our objective is to study the impact of the lockdown coupled with the inoculation drive (that began around January, 2021 in India and increased rapidly from around April) on the daily number of confirmed cases and deaths in both Maharashtra and Delhi using ITS analysis. The structure of this article is depicted as follows. We give a brief description about the ITS method in Section 2. In Section 3.1, we discuss the motivational data examples of Delhi and Maharashtra. We provide the results of the segmented regression model in Section 3.3. We extensively discuss about that the behaviour of post-intervention slopes for the daily new cases as well as the daily deaths in both Delhi and Maharashtra in Section 3.3.1. We also include the counterfactuals and delayed time effects in our study in Section 3.3.2 to investigate the validity of our model. In Section 3.3.3, we perform the residual analysis obtained from the main ITS regression setup. Lastly, we provide a discussion about the importance of the interventions in tackling the COVID-19 second wave and how we can learn from it to be ready for any future wave of the virus (recent emersion of highly contagious Omicron variant) in Section 4.

Methods

An Interrupted Time Series (ITS) analysis (see Bernal, Cummins, and Gasparrini (2017) and Biglan, Ary, and Wagenaar (2004)) can be represented using the following segmented regression model:

where Y _t represents the number of daily new cases or deaths at time t, X _1t (time) denotes the discrete variable indicating the time elapsed since the start of the study, X _2t (treatment) is the dummy variable signalling the immediate level change in post-intervention period (0 for pre-intervention period and 1 for post-intervention period), and X _3t (time since treatment, which is a multiplicative form of X _1t ’s and X _2t ’s) represents the variable indicating the slope change following the intervention. More precisely, the above model in our context can be written as

where (T + 1)th point denotes the start of the intervention, and I(t>T) denotes the indicator function which takes value 1 when t>T, and 0 otherwise.

Here, the regression coefficients of three time based covariates account for the baseline level at the time point zero (β ₀), pre-intervention slope (β ₁), the level change at the intervention point (β ₂), and the slope change from the pre-intervention period to the post-intervention period (β ₃). In fact, (β ₃) takes into account the interaction between time and the intervention.

Here, our interest is to see the change in trend in the outcome variable Y after the introduction of the intervention assuming that the average level and the trend of the series would remain constant without the interventions (see Silva, Filho, and Fernandes (2020)). ITS analysis found widespread usage across different sectors to mathematically measure the effect of an intervention on a certain characteristic of a population under study. The segmented regression model of ITS analysis enables a researcher or a practitioner to evaluate the change in slope from the pre-intervention to the post-intervention period through interesting graphical representations (see Biglan, Ary, and Wagenaar (2004) and Silva, Filho, and Fernandes (2020)). Besides this, one can assess the change in level in the data as a result of the intervention. The coefficients corresponding to the level change and slope change help to identify if there is any immediate effect and a sustained effect as a result of the intervention respectively. We use these ideas broadly in the subsequent sections.

Data analysis

Data

We record the data for Delhi (one of the most affected union territories with highest population among all the union territories during the second wave which started around March, 2021 in India) and Maharashtra (one of the most affected states with second highest population among all the states) (see report Worst affected states in India (2021)). We later find out the time frame for the intervention policy, i.e., the duration of the strict “lockdowns” during the second wave in both places (see Delhi Lockdown (2021) and Maharashtra Lockdown (2021)). Another policy which may also be described as an intervention policy along with “lockdowns” is the “vaccination process” (began from the start of 2021 and speeded up from around April, 2021) which can have indirect effects on declining COVID cases. However, in our analysis, the pre-intervention and the post-intervention periods mainly indicate the pre-lockdown and the post-lockdown periods, respectively. We consider a two-week observation window (see Silva, Filho, and Fernandes (2020)) after the end of intervention policies for studying the effects of the interruptions keeping in mind that the average incubation period for COVID is 14 days, confirmed by the World Health Organization (see World Health Organization report on COVID-19 (2020)). The data of daily new cases and daily deaths for Delhi and Maharashtra are recorded from March 15 to June 14 (intervention window is from April 19 to May 31, see Relaxation for Delhi (2021) and post-intervention window is from June 1 to June 14) and from March 15 to June 18 (intervention window is from April 14 to June 4, see Relaxation for Maharashtra (2021) and post-intervention window is from June 5 to June 18), respectively (see Table 1).

Table 1:

Information based on intervention in Delhi and Maharashtra.

State	Projected population (approx.)	Int. period	Duration	Post int. window
Delhi	31.2 million	April 19 to May 31	43	Until June 14
Maharashtra	126.2 million	April 14 to June 4	52	Until June 18

1. Int.: Intervention.

In Figures 1 and 2, we provide the daily COVID scenario in both Delhi and Maharashtra during the second wave period in India. From the plots, we can have some ideas about the lowest and highest numbers of daily scenario in both Delhi and Maharashtra. In the given period, the lowest number of cases in Delhi was 131, whereas the highest number reached almost 28,500, and the lowest number of deaths reported in Delhi was 1, but the peak occurred at around 450-mark. For Maharashtra, the lowest and the highest numbers of cases reported during that period were around 7,700 and 68,700, respectively, and regarding the number of deaths, the lowest and the highest were around 50 and 2,800, respectively.

Figure 1:

Figure showing the trend in daily cases and death cases in Delhi.

Figure 2:

Figure showing the trend in daily cases and death cases in Maharashtra.

Results

The post-intervention slope

In Figures 3 and 4, we provide the graphical representations of the segmented regression models regarding the daily cases and the daily deaths of both Delhi and Maharashtra with the red line in the graphs indicating the start of the intervention period. The results of the ITS models are shown in Tables 2 and 3. Before the implementation of the interventions, the trend of daily new cases is positive and statistically significant. These results are given in Table 2. But we observe negative slopes (statistically significant) for the post-intervention period indicating a decrease in the count of daily new cases for both the states: and β ₃=−0.224 with p-value < 0.001 for Delhi, and β ₃=−0.077 with p-value < 0.001 for Maharashtra. For the daily number of deaths, we also notice positive trends (statistically significant) before the implementation of the government policies. The results regarding the daily deaths are shown in Table 3. For the post-intervention period, we find a statistically significant decrease in the slopes for daily death cases in both the states: β ₃=−0.172 with p-value < 0.001 for Delhi, and β ₃=−0.052 with p-value < 0.001 for Maharashtra.

Figure 3:

ITS analyses on COVID scenario in Delhi – (i) analysis on daily cases (log), and (ii) analysis on daily deaths (log).

Figure 4:

ITS analyses on COVID scenario in Maharashtra – (i) analysis on daily cases (log), and (ii) analysis on daily deaths (log).

Table 2:

ITS analysis results for daily cases (14 day post-intervention period).

	Delhi	Maharashtra
	(n=92)	(n=96)
	β (SE)	β (SE)
Intercept	5.815a(0.118)	9.896a(0.064)
Time	0.121a(0.006)	0.041a(0.004)
Level	1.140a(0.146)	0.366a(0.075)
Trend	−0.224a(0.006)	−0.077a(0.004)
R 2	0.95	0.93

1. SE: Standard error; ^ap-value < 0.001.

Table 3:

ITS analysis results for daily deaths (14 day post-intervention period).

	Delhi	Maharashtra
	(n=92)	(n=96)
	β (SE)	β (SE)
Intercept	0.508a(0.146)	4.063a(0.136)
Time	0.123a(0.007)	0.064a(0.008)
Level	1.736a(0.180)	0.269b(0.157)
Trend	−0.172a(0.008)	−0.052a(0.008)
R 2	0.93	0.84

1. SE: Standard error; ^ap-value < 0.001; ^bp-value < 0.1.

The Figures 5 and 6 present the dot and whisker plots of the regression coefficients obtained by varying the post-intervention window between 15 and 21 days for all the regression models for both Delhi and Maharashtra (see Silva, Filho, and Fernandes (2020)). Here, we observe that the slope changes in the daily number of confirmed cases and the daily new deaths continue to remain negative and statistically significant following the policies implemented by respective authorities in both the states of Delhi and Maharashtra.

Figure 5:

Dot and whisker plots showing the changes in the regression coefficients due to varying the post intervention period in Delhi.

Figure 6:

Dot and whisker plots showing the changes in the regression coefficients due to varying the post intervention period in Maharashtra.

The counterfactual and delayed effects

In ITS analysis, it is crucial to understand the counterfactuals meaning what would have happened to the outcomes had the interventions not been implemented. Then, to examine the significance of the difference between the predicted outcome and its counterfactual, we need to look whether there is an immediate change after the intervention and whether the slope changes after the intervention or not. There is no direct statistical test to look at that and so comes the importance of delayed time effects into our study. Since the incubation period of this virus is 14 days, it would be difficult to observe any change in the outcomes right after the interventions and hence the analysis is executed again to observe any immediate effects two and three weeks after the intervention.

In Figures 7 and 8, we present the graphs by plotting all predicted outcomes and their counterfactuals regarding the daily new cases and the daily new deaths for both Delhi and Maharashtra where the dashed line represents the counterfactual. From the Figures, we can clearly see that there would have been significant increasing slopes after the intervention points if they were not in place, as expected.

Figure 7:

ITS analysis on daily COVID scenario in Delhi and their counterfactuals.

Figure 8:

ITS analysis on daily COVID scenario in Maharashtra and their counterfactuals.

To check the significance of our study, we run our design again to observe some delayed time effects since it would be difficult to find any immediate change just after the beginning of intervention policies. We can see that β ₂>0 (statistically significant) for each study in Tables 2 and 3, i.e., there is no immediate change in trend after the start of intervention. So at first, we take a delay of 14 days to execute all regression models regarding the daily new cases and notice significant immediate effect for Delhi but not for Maharashtra. Then, we again consider a delay of 21 days where we see significant immediate effects for both Delhi and Maharashtra for daily new cases. And for a delay of 14 days regarding the daily new deaths, we do not observe any immediate effect for both Delhi and Maharashtra, but for a delay of 21 days, we notice significant immediate changes for both of the states (see Tables 4–7).

Table 4:

ITS analysis results for daily cases (14 day post-intervention period) with 14 days delay.

	Delhi	Maharashtra
	(n=92)	(n=96)
	β (SE)	β (SE)
Intercept	6.179a(0.088)	10.058a(0.044)
Time	0.097a(0.003)	0.029a(0.002)
Level	−0.560a(0.127)	−0.075(0.059)
Trend	−0.222a(0.005)	−0.072a(0.002)
R 2	0.96	0.95

1. SE: Standard error; ^ap-value < 0.001.

Table 5:

ITS analysis results for daily cases (14 day post-intervention period) with 21 days delay.

	Delhi	Maharashtra
	(n=92)	(n=96)
	β (SE)	β (SE)
Intercept	6.498a(0.121)	10.164a(0.049)
Time	0.079a(0.004)	0.022a(0.002)
Level	−1.391a(0.192)	−0.294a(0.071)
Trend	−0.206a(0.008)	−0.067a(0.003)
R 2	0.92	0.93

1. SE: Standard error; ^ap-value < 0.001.

Table 6:

ITS analysis results for daily deaths (14 day post-intervention period) with 14 days delay.

	Delhi	Maharashtra
	(n=92)	(n=96)
	β (SE)	β (SE)
Intercept	0.516a(0.100)	4.134a(0.106)
Time	0.124a(0.003)	0.058a(0.004)
Level	−0.120(0.144)	−0.085(0.141)
Trend	−0.195a(0.005)	−0.051a(0.005)
R 2	0.95	0.86

1. SE: Standard error; ^ap-value < 0.001.

Table 7:

ITS analysis results for daily deaths (14 day post-intervention period) with 21 days delay.

	Delhi	Maharashtra
	(n=92)	(n=96)
	β (SE)	β (SE)
Intercept	0.786a(0.115)	4.198a(0.100)
Time	0.109a(0.003)	0.054a(0.003)
Level	−0.607a(0.182)	−0.340a(0.144)
Trend	−0.194a(0.008)	−0.046a(0.005)
R 2	0.93	0.85

1. SE: Standard error; ^ap-value < 0.001.

Residual analysis from the ITS model

In this section, we performed the residual analysis obtained from the above ITS regression analysis, as suggested by Bernal, Cummins, and Gasparrini 2017. After running the Durbin-Watson test on the residuals from the above ITS analysis, we observed the p-value for the daily number of cases in Delhi and Maharashtra as less than 0.001, and for the daily number of death cases in Delhi and Maharashtra as less than 0.001. As we can see, in all four cases, the p-value came out to be significant at the 5% level, which is suggestive of the presence of auto-correlation. To remove the auto-correlation, we fit suitable ARIMA models in all four cases by running the auto.arima function in R. After running it, we performed Ljung–Box test (with lag=10) for testing auto-correlation of the residuals of residual. The p-value for all the models except the model for the daily cases in Maharashtra comes out to be greater than 0.05, suggesting the absence of any auto-correlation for those models except for the model for the daily cases in Maharashtra, which still suffers from auto-correlation. The graphical representation of the results is given in the following figures (Figures 9 –12).

Figure 9:

Residual analysis from the ITS model for the daily cases in Delhi.

Figure 10:

Residual analysis from the ITS model for the daily death cases in Delhi.

Figure 11:

Residual analysis from the ITS model for the daily cases in Maharashtra.

Figure 12:

Residual analysis from the ITS model for the daily death cases in Maharashtra.

Discussion

The popularity of ITS design massively increased in various fields of research due to its flexibility towards incorporating both continuous and discrete outcomes of regularly spaced time intervals which enables this design to measure the impact of interventions on time series data. This method is also attractive because of its visual representations by which one can easily observe the nature of an outcome over time during the pre- and post-intervention periods. The ongoing COVID-19 pandemic and the implementation of policies like “lockdowns”, “social distancing”, and “vaccination drive” to curb the waves of COVID-19 opened the door all over the world to investigate the importance of such interventions in tackling this kind of infectious disease through ITS analysis.

Though we mention some works regarding ITS design on the effects that intervention policies can have on this type of pandemic in Section 1 already, the gists of some popular studies among them are provided in this section too, just to summarize what kind of ITS studies were performed in different places, similar to our study. Silva, Filho, and Fernandes (2020) used a segmented linear regression model to study the effect of lockdowns in four city capitals of Brazil and showed how the interventions helped reduce the epidemic in Brazil significantly through ITS model. Thayer et al. (2021) studied the role of lockdown policies in India during the first wave to slow down daily rate of increase in new cases through ITS analysis. Figueiredo et al. (2020) estimated the impact of the interventions on COVID-19 scenario in China. Vokó and Pitter (2020) and Siedner et al. (2020b) evaluated the significant contribution of intervention policies to prevent the spread of COVID-19 by ITS studies in Europe and the United States, respectively. Many other interesting studies were executed in other parts of the world, not only to analyse the influence of interventions to prohibit the surge of this disease but also study the gradual growth of the virus (see Ghosh, Ghosh, and Chakraborty (2020)).

Using the real data of Delhi and Maharashtra in our ITS design, we find the post-intervention trends to be statistically significant and negative for both the daily cases and the daily deaths. We also re-execute the regression models using a 15–21 days window other than the 14-day post-intervention period and observe that the post-intervention trends remain negative and statistically significant (idea from Silva, Filho, and Fernandes (2020)). This adds a more robust view regarding the post-intervention trends to our analysis.

We also add the counterfactuals to our analysis for both the studies of the daily cases and the daily deaths of Delhi and Maharashtra, and this provides an intuitive idea of the situation if the intervention had not occurred. From the counterfactuals, we can clearly observe that there would have been a strict upsurge in COVID-19 scenario in both the states during second wave without the intervention policies. Now to check the validity of this insight, we also study the delayed time effects since it would be difficult to find any immediate change regarding the levels of daily cases and deaths just after the start of intervention due to the incubation period of two weeks (see World Health Organization report on COVID-19 (2020)). In our regression analysis, we indeed do not observe any immediate changes and so at first take a delay of 14 days to re-run the analysis. We notice statistically significant immediate effect for Delhi regarding daily cases but not for Maharashtra, and for daily deaths, we do not find any statistically significant immediate effects for both data sets. For a delay of 21 days, we notice statistically significant immediate effects on both of daily cases and deaths for both of Delhi and Maharashtra. This shows that reduction in level in Maharashtra’s daily cases was delayed with respect to that of Delhi’s, and the changes in levels for daily deaths of both places started significantly around third week after the start of the intervention.

One limitation regarding our analysis is having the auto-correlated residuals from different regression setups as this problem is a tenacious issue to overcome in the field of Statistics. To examine this phenomena, one should check the Durbin Watson (DW) test statistic whose value varies from 0 to 4. Any value near to 0 or 4 indicates high serial correlation among residuals, whereas DW statistic near to 2 implies no serial correlation. In our study, the DW test statistic for most of the regression models is around 1 that indicates moderate positive serial correlation between the residuals. This is in general a common phenomena in time series regression. But as mentioned earlier, getting rid of this problem involves some further advanced analyses that can be pursued in future. However, the residual analysis obtained from the main ITS regression design suggests the absence of any auto-correlation for all the cases except for the daily cases in Maharashtra.

Although the intervention policies mainly talk about the effects of lockdown in our analysis, it is important to note that the process of vaccination began from January, 2021 in India and started to accelerate from around April (see COVID-19 INDIA Data (2021)). Therefore, it can have an indirect effect on daily COVID-19 scenario especially during post-April period. In a nutshell, we have clear evidences through our analysis that the proper implementation of intervention policies can significantly diminish the rise of COVID-19. Although the limitations like the tendency of having underestimated standard errors, the underestimation of impact of policies due to intensified tests during the post-intervention period, and the delay between testing and actual report exist, we try to consider all possible scenarios through incorporating all ingredients of ITS design to see whether any significant alterations are required or not. In this way, we attempt to provide a viable analysis for this type of study to present the importance of having proper intervention policies in place for curbing this type of highly infectious disease.

Many studies across the world show that the lockdowns can have significant negative effects on the economy of a country, mental and physical health of a community, but proper implementation of the interventions can slow down this type of pandemic rapidly, i.e., in a very short period of time, and hence our recent study can work as a reminder to ourselves especially in the recent wake of Omicron variant of COVID-19 and also for any other variant in future (if any). Although this Omicron variant is less severe than the Delta ones, the higher transmissibility of the Omicron variant caused significant damages to the health services (see Omicron variant (2021)). Delhi and Maharashtra reported huge number of the Omicron cases, as it was the scenario during the second wave. So some intervention policies (not necessarily strict) were needed to be imposed to curtail the spread of this variant and may be needed again to prohibit the surge of any other variant in future (if any). Our study corroborates the requirement of the interventions to be made effective properly to break the chain to soften up the infectivity of any future variant of this highly contagious virus if required.