preface

ARMA: Automatic regressive Moving Average model (ARMA). This model is an important method to study time series. The greatest use of time series is to predict future events, and all kinds of means are also used to improve the accuracy of prediction. The model consists of two parts: Regressive (AR) and moving Average (MA) time series were used to transform unstable time series into stable time series, that is, the parameters of I in ARIMA, and then of course the parameters p and Q of AR and MA.Copy the code

AR autoregressive model

In regressive AR models, the regressive AR model was simple in that the results of the task's current time xT correlated with the results of previous times. In simple terms, the current result is equal to the weighted average of the historical data, and then plus a disturbance of the current time t εt, we just need to calculate the weight parameter ϕ at each time, the formula is as follows: Xt =ϕ0+ϕ1xt−1+ϕ2xt−2+...... ϕ PXT −p+εt formula XT-1, xT-2,.... Xt minus p is the value of the previous time, and the p parameter is the value that we need to determine.Copy the code

MA moving average model

MA model thought is also very simple, think that the result of the current xt associated with the disturbance of before, simple is that xt is mainly affected by the past q phase error term, namely the current result is equal to the weighted average of the disturbance of history, and then combined with historical results mean mu, we only need to calculate the parameters ϕ is also ok. And that makes sense, if our data fluctuates around the mean, what makes it fluctuate? It's the perturbation term, and the perturbation causes the data to change over and over again, eventually becoming the current data. Xt = u + t - theta epsilon 1 epsilon t - 1 - theta - 2-2 epsilon t... The q parameter θpεt−q is the value we need to determine.Copy the code

The ARMA model

ARMA model is the most commonly used 'stationary series fitting model', xT is not only related to the sequence values of the previous P period, but also related to the random perturbation of the previous Q period. In other words, the formula of ARMA model is the combination of the formula of AR model and the formula of MA model. In this formula, we need to calculate that xt=ϕ0+ϕ1xt−1+ϕ2xt−2+...... ϕ PXT - p + t - epsilon theta epsilon t - 1-2-1 - theta epsilon 2 t... Theta epsilon t p - qCopy the code

ARIMA model grading

We need to calculate the above parameters, which make the smaller the prediction error of our model, the better our model.Copy the code

The index to measure the advantages and disadvantages of time series forecasting model

Generally speaking, Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC) are two indicators for evaluating excellent models. These two evaluation indexes are not only applicable to the event sequence model, but also widely used in other mathematical models. AIC=−2ln(L)+2k BIC=−2ln(L)+ln(n)∗k Let us understand the meaning of AIC, AIC is composed of two parts, one is logarithmic maximum likelihood function, the other is the number of parameters. Maximum likelihood function is an index to evaluate the fit of the model. The larger the value is, the better the fit effect is. However, if too many parameters are used, it can be fitted well, but it will be over-fitted. Such model has poor generalization ability, so adding the number of parameters is actually "punishing" the maximum likelihood function. Selecting the model with the lowest AIC value as the optimal model essentially balances the underfitting and overfitting.Copy the code

Create training data
train_start = datetime(2019.10.1)
train_end = datetime(2020.10.1)
train_data = df[train_start:train_end]

# set the order
warnings.filterwarnings("ignore") # specify to ignore warning messages

pmax,qmax,dmax,smax = 8 ,8 ,2 ,7
# p is order AR, q is order MA, d is number of difference, and s is period
pdq = itertools.product(range(1,pmax+1),range(1,dmax+1),range(1,qmax+1),[smax])
aic_matrix = {} # aic matrix

for p,d,q,s in pdq:
    try:
        model = SARIMAX(endog=train_data['cnt'],order=(p,d,q,s))
        results = model.fit(disp=True)
        aic_matrix['{}, {}, {}'.format(p,d,q)] = results.aic
        print('ARIMA p:{} d:{} q:{} - AIC:{}'.format(p, d, q, results.aic))
    except:
        continue
p,d,q = sorted(aic_matrix.items(), key=lambda x: x[1[])0] [0].split(', ')
print('AIC minimum p: {}, d: {}, q: {}'.format(p,d,q),end='\r')

#ARIMA P :8 d:1 q: 8-aic :9138.284284512645
#ARIMA P :8 d:2 q: 1-aic :9165.776270275972
#ARIMA P :8 d:2 Q: 2-aic :9166.238801899044
#ARIMA P :8 d:2 Q: 3-aic :9166.731346671577
#ARIMA P :8 d:2 q: 4-aic :9169.846039297874
#ARIMA P :8 d:2 q: 5-aic :9167.21095209448
#ARIMA P :8 d:2 q: 6-aic :9145.26033882199
#ARIMA P :8 d:2 q: 7-aic :9129.24423753544
#ARIMA P :8 d:2 q: 8-aic :9132.197927380188
#AIC: p: 6, D: 2, Q: 7

Copy the code

Difference between ARMA and ARIMA

<1> Different application objects

AR, MA and ARMA are all applied to time series where the original data is stationary. After ARIMA is applied to the difference of original data, it is a stationary time series.Copy the code

<2> Different time series

AR(autoregressive model), AR(p), p order autoregressive model. MA(moving average model), MA(q), moving average model of order Q. ARIMA(Differential autoregressive moving Average model).Copy the code

<3> differences in stationarity

The stationarity of ARMA model requires that the mean, variance and autocovariance of Y are finite constants independent of time. It can be proved that the stationarity condition of ARMA(p, q) model is that the moduli of the solutions of equation ()0Lφ= are all greater than 1, and the reversibility condition is that the moduli of the solutions of equation ()0Lθ= are all greater than 1. ARMA model can only deal with stationary series, so AR, MA or ARMA models can be directly established for stationary series. However, common time series are generally non-stationary. ARMA model can be used only after it is transformed into stationary series after difference. ARIMA model (Autoregressive Integrated Moving Average) If the stationary series Zt = △kyt of the non-stationary time series YT after k difference obeys ARMA(P, q) model, then the original series YT obeys ARIMA(P, k, Q) model. In other words, the original sequence is I(k) sequence, and the stationary sequence is I(0) after k difference. Stationary sequence I(0) obeys ARMA model, while non-stationary sequence I(k) obeys ARIMA model.Copy the code

Stationary time series and non-stationary time series

A time series is said to be stationary when each period has the same 'probability distribution'. If the probability distribution of a time series changes with time, then the time series is not stationary. In AR model, as long as the absolute value of autoregressive coefficient is less than 1, the sequence is stable. So if a sequence has an autoregressive coefficient of one, it's an unstable sequence.Copy the code

What is difference

Difference is a method to transform nonstationary sequence into stationary sequence. A nonstationary sequence usually becomes stationary after one or two differences. What is difference? It's the value of the later time minus the value of the current time, which is YT - YT -1.Copy the code

How do you determine if a sequence is stationary