A comparative study on predicting influenza outbreaks using different feature spaces: application of influenza-like illness data from Early Warning Alert and Response System in Syria

Abstract

Objective

An accurate forecasting of outbreaks of influenza-like illness (ILI) could support public health officials to suggest public health actions earlier. We investigated the performance of three different feature spaces in different models to forecast the weekly ILI rate in Syria using EWARS data from World Health Organization (WHO). Time series feature space was first used and we applied the seven models which are Naïve, Average, Seasonal naïve, drift, dynamic harmonic regression (Dhr), seasonal and trend decomposition using loess (STL) and TBATS. The Second feature space is like some state-of-the-art, which we named $53-weeks-before\_52-first-order-difference$ feature space. The third one, we proposed and named $n-years-before\_m-weeks-around$ (YnWm) feature space. Machine learning (ML) and deep learning (DL) model were applied to the second and third feature spaces (generalized linear model (GLM), support vector regression (SVR), gradient boosting (GB), random forest (RF) and long short term memory (LSTM)).

Results

It was indicated that the LSTM model of four layers with $1-year-before\_4-weeks-around$ feature space gave more accurate results than other models and reached the lowest MAPE of $3.52\%$ and the lowest RMSE of 0.01662. I hope that this modelling methodology can be applied in other countries and therefore help prevent and control influenza worldwide.

Introduction

Influenza epidemic results in three to five million cases of severe illnesses and approximately 290,000 to 650,000 deaths worldwide each year [1]. WHO’s Early Warning, Alert and Response System (EWARS) is designed to improve disease outbreak detection in emergency settings. The system has been built in SYRIA since 2012 to collect and detect near real-time information on several outbreaks including influenza [2]. The Existing researches on modeling influenza epidemic falls into two categories: Mechanistic and Statistical models. They are summarized in literature reviews [3,4,5,6] and in the CDC comparisons [7, 8]. Researches under statistical category vary according to the different Features and methods used. Some researchers used the number of patients in the past as features [9,10,11,12,13], while others integrated other data sources to predict the number of patients in the future. Examples of these sources are climatological data [14, 15], search engine queries [16,17,18,19], public comments on social media like Twitter [20, 21], online information-seeking behavior on websites like Wikipedia [22, 23] and a combination of multiple data streams [15, 24,25,26]. Different methods on these features were applied. Some researches treated the problem as an instance of more general time series forecasting using time series methods (ARIMA, ARIMA-STL, GARMA) [9, 10, 17, 27], while others used ML methods including Stacked linear regression [24, 26], AdaBoost regression with decision trees [26], GB [12], SVR [26, 28], elastic net [28] RF [11, 12, 28], Artificial Neural Network (ANN) [12, 20]. Recently, a DL method Called LSTM has attracted much interest in ILI prediction and gave excellent results, which are more accurate than those of other methods [12, 13, 15, 29]. In addition to investigating the performance of the three different feature spaces with multiple time-series, ML and DL based methods to predict the weekly ILI rate in Syria; we proposed novel future spaces $n-years-before\_m-weeks-around$ that integrate into state-of-the-art ML and DL methods. There are two important contributions of this paper. First, the use of $n-years-before\_m-weeks-around$ future spaces to predict ILI rate. Second, analyzing multiple models performance over the EWARS data from WHO in Syria.

Main text

Materials and methods

Data

We collected SYRIA flu data from the EWARS reports published by WHO on the website [30]. We only used the Flu Data from the first week of 2014 to the 42nd week of 2018. To avoid any possible population variations, we adopted the ILI rates as predictors (x) and responses (y) of our models.

$${\text{ILI\, rate}} = {\text{ILI\,number}} / {\text{Total\,number\,of \, illnesses}}$$

Fig. 1

a The ILI rate in SYRIA from the first week of 2014 to the 42nd week of 2018; The Y-axis represents the weekly ILI rate, and the X-axis represents the time series. b The ILI rate in SYRIA from the first week of 2014 to the 42nd week of 2018 (seasonal plot); The Y-axis represents the weekly ILI rate, and the X-axis represents the weeks

Fig. 2

a $n-years-before\_m-weeks-around (YnWm)$ feature spaces. b The real and prediction of weekly ILI rate of the testing set for TBATS with fs1: time series feature space, LSTM of 3 layers with fs2: $53- weeks-before\_52-first-order-differences$ feature space, LSTM of 4 layers with fs3: $n-years-before\_m-weeks-around (y1w4)$ feature space

Figure 1a illustrates the raw data. The Y-axis represents the weekly ILI rate, and the X-axis represents the time series. The seasonality is obvious, as shown in Fig 1b. We split the data into two parts: the first 90% was the training set and the last 10% was the testing set.

Feature space

In this study, we reviewed three types of feature space. We named them as (time series feature space, $53-weeks-before\_52-first-order-difference$ feature space and $n-years-before\_m-weeks-around$ feature space). Tables S1 and S2 illustrate the data set, response, predictors and the pretreatment of the source data. An additional pdf file shows this in more detail (see Additional file 1).

Time series feature space (fs1) We treated the ILI weekly data as time series with seasonal value 52.

$53- weeks-before\_52-first-order-differences$feature space (fs2) Some previous studies found that using the ILI rate of the past 53 weeks and the 52 first-order differences helped improve the results of the prediction models for influenza data [11,12,13].

$n-years-before\_m-weeks-around$(YnWm) feature spaces (fs3) We reviewed a maximum of 3 years before and 5 weeks around. To predict the ILI rate of week WX of year YX, We used the ILI rate of the past m weeks before WX in YX, the ILI rate of past n seasonal weeks and the ILI rate of the m weeks around seasonal weeks. In the case of $1-year-before\_4-weeks-around$, the response y is the ILI rate of week 30 of year 2017 then the predictors are the ILI rates of weeks 29, 28, 27, 26 from year 2017 and the ILI rates of weeks 26, 27, 28, 29, 30, 31, 32, 33, 34 from year 2016. The number of predictors and rows changed depending on the value of n and m and we had to drop some data rows so that we get the same training data length for any value of n and m. Figure 2a illustrates the feature spaces with examples of $3-years-before\_5-weeks-around$ and $1-year-before\_4-weeks-around$ feature spaces.

Models

The models, programming languages, and libraries, which were used in this study were illustrated in Additional file 1. We trained all models in R Programming Language (version 3.4.4). For time series models, we used the “forecast” package (version 8.4). For ML models, we applied the caret package (Version 6.0-8). For DL models, we used the Keras package (Version 2.2.4) based on Tensorflow (Version 1.10). A personal computer with Intel I7-8550U processor, 8 GB of RAM and an NVIDIA 130 MX GPU was used for the experiments. Each experiment takes approximately 1 to 30 min to train the model. The prediction takes less than 5 s on the same hardware.

Metrics

We compared different models with different feature spaces using the mean absolute percentage error (MAPE) and root mean squared error (RMSE) as key performance indicators (KPIs).

$$\begin{aligned} {\text{MAPE}}& = \frac{1}{n} \sum _{t=1}^n 100\, \left| \frac{F_t-A_t}{A_t}\right| \% \\ {\text{RMSE}} &= \sqrt{\frac{1}{n} \sum _{t=1}^n (F_t-A_t)^2} \end{aligned}$$

where At is the actual value and Ft is the forecasted value.

Results

fs1 with time series models

Seven methods (Average, Naïve, Seasonal Naïve, Drift, STL, DHR and TBATS) were applied in a recursive way. We trained the model on the training data set then predicted the ILI rate of week number x then we repeated the process by combining the actual ILI rate of week number x with the training data set. The metrics of all models are presented in Table  1(a), (b). it was indicated that the TBATS model outperformed other six models in predicting weekly ILI rate (MAPE = 4.66%, RMSE = 0.03096). Our explanation of the success of TBATS model is that the seasonality is allowed to change slowly over time in a TBATS model, while DHR, STL, Seasonal Naïve models force the seasonal patterns to repeat periodically without changing and Average, Naïve, Drift models do not depend on the seasonal patterns. We considered TBATS model a baseline in comparing the results of the following models.

Table 1 The MAPEs and RMSEs of the testing set for all methods applied on fs1: time series and fs2: $53- weeks-before\_52-first-order-differences$ and fs3: $n-years-before\_m-weeks-around$ feature space

fs2 with ML and DL models

We applied four ML methods (GLM, SVR, GB and RF) and two DL methods (LSTM with 3 layers and LSTM with 4 layers). Table 1(a), (b) shows that none of the models achieved better results than the baseline model in MAPE metric but in RMSE metric, the LSTM model of 3 layers (RMSE = 0.02294) outperformed the baseline model and all otehr models with fs1 and fs2 .

fs3 with ML AND DL models

According to $0<=n<=3\ and\ 0<=m<=5$ we trained 23 feature spaces in four ML methods (GLM, SVR, GB and RF) and two DL methods (LSTM with 3 layers and LSTM with 4 layers). Totally, we made 138 experiments with results shown in Table  1(a), (b). In accordance with MAPE metric, We found that 22 experiments show a decrease in metrics less than that of the baseline model (1 experiment: by using GB model with Y2W2 feature space, 10 experiments: by using LSTM model of 3 layers with “Y1Wm, Y2Wm: excluding m=0 and m=4” feature spaces, 11 experiments: by using LSTM model of 4 layers with “Y1Wm: excluding m=1,Y2Wm: excluding m=5,Y3w1” feature spaces) but In accordance with RMSE metric, We found that 44 experiments show a decrease in metrics less than that of the baseline model (1 experiment: by using GB model with Y2W2 feature space, 20 experiments: by using LSTM model of 3 layers with “YnWm: excluding n=3 with m=3,4,5” feature spaces, 23 experiments: by using LSTM model of 4 layers with “YnWm” feature spaces). we achieved the best result (MAPE = 3.52%, RMSE = 0.01662) by using LSTM model of 4 layers with Y1W4 feature space. in comparison to models with fs2, the results show that for any model used, there are at least one value to n and m in fs3 that achieved better result than the same model with fs3. Our explanation of the success of this feature space is that models can learn the seasonality by n years before and can recognize the trend of the data by m weeks around.

The temporal variation of the real weekly ILI rate and the predicted values obtained from the three models (the best result in each feature space) for the test period were plotted in Fig 2b. As can be seen, the predicted values of weekly ILI rate were in a good agreement with their related observed values and the used models could be used to model the weekly ILI rate. Moreover, LSTM model of 4 layers with fs3 resulted in better predicted values and trend between them for the observed values of ILI rate than the other models especially when there is a change in trend between observations.

Discussion

The accurate forecast of an outbreak of ILI could support public health officials in taking public health actions, such as allocating or temporarily readjusting medical resources for hospitals and medical centers. The ILI rate varies from year to year depending on multiple factors. Despite this difference, the ILI rate plot in each year takes a semi-constant form with an increase or decrease compared to previous years and could be predicted by using specific observations of past years with statistical models. Performance of statistical models is data dependent and there is no model that performs well in all situations. Therefore, evaluating the performance of different models is of great importance as they provide useful and important information regarding strengths and weaknesses of the models and gives an insight to use better models for forecasting purposes. Some sate of the arts [11, 12] utilize 53 weeks before with 52 first order difference with different statistical models and found that both the recent observations and the later observations with the difference were interesting and had significant influence on the predication. In this study, we proposed novel future spaces, namely $n-years-before\_m-weeks-around$, and compared to some existing future spaces that utilize historical observations in different ways by integrating it into state-of-the-art ML and DL models. Our results revealed the success of our future space for some values of n and m and its failure for other values to outperform other future spaces in prediction ILI rate in Syria. This fact suggests that combining carefully selected number of Nearby Historical Observations with Carefully selected number of seasonal Historical Observations is advantageous over simply choosing all Historical Observations. While the results presented here are for ILI within Syria, our novel future spaces shows promise to be easily extended to accurately track not only influenza in other countries but also other infectious diseases Through careful tuning of the values of n and m. We believe that it is up to policymakers in organizations concerned with EWARS to decide whether these forecasts are ready for use in decision support at the current level of accuracy.

In conclusion, we performed Naïve, Average, Seasonal naïve, Drift, DHR, STL, TBATS, GLM, SVR, GB, RF and LSTM methods with different feature spaces to predict the weekly ILI rate using EWARS data from WHO in Syria. We found that the TBATS method with time series feature space gave better results than those resulted from all methods in $53-weeks-before\_52-first-order-differences$ feature space. We also found that the GLM, SVR, GB, RF and LSTM methods with a good choice to n and m in $n-years-before\_m-weeks-around$ feature space gave better results than those resulted in $53-weeks-before\_52-first-order-differences$ feature space. In all the models, the LSTM model of 4 layers reached the lowest MAPE (3.52%) and the lowest RMSE (0.01662).

Limitations

Climatological data and Pharmaceutical Sales could be used to achieve better performance of the used models. We would like to investigate the impact of these parameters in future work.