5 – Autoregressive Integrated Moving Average (ARIMA) Models Box & Jenkins Methodology Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 1 ARIMA Box-Jenkins Methodology Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 2 Example 1/4 Time Series Plot of Index 290 280 270 The series show an upward trend. 250 240 230 220 210 1 6 12 18 24 30 36 Index 42 48 54 60 Autocorrelation Function for Index (with 5% significance limits for the autocorrelations) 1,0 The first several autocorrelations are persistently large and trailed off to zero rather slowly a trend exists and this time series is nonstationary (it does not vary about a fixed level) 0,8 0,6 Autocorrelation Index 260 0,4 0,2 0,0 -0,2 -0,4 -0,6 -0,8 -1,0 Idea: to difference the data to see if we could eliminate the trend and create a stationary series. 1 2 3 4 5 6 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 7 8 9 Lag 10 11 12 13 14 15 16 3 Example 2/4 Time Series Plot of Diff1 First order differences. 5 4 A plot of the differenced data appears to vary about a fixed level. 3 Diff1 2 -2 -3 -4 1 0,8 0,8 0,6 0,6 Partial Autocorrelation Autocorrelation 1,0 0,4 0,2 0,0 -0,2 -0,4 -0,6 10 11 12 48 54 60 0,0 -0,4 -0,6 -1,0 8 9 Lag 42 -0,2 -1,0 7 30 36 Index 0,2 -0,8 6 24 0,4 -0,8 5 18 (with 5% significance limits for the partial autocorrelations) 1,0 4 12 Partial Autocorrelation Function for Diff1 Autocorrelation Function for Diff1 3 6 A constant term in each model will be included to allow for the fact that the series of differences appears to vary about a level greater than zero. (with 5% significance limits for the autocorrelations) 2 0 -1 Comparing the autocorrelations with their error limits, the only significant autocorrelation is at lag 1. Similarly, only the lag 1 partial autocorrelation is significant. The PACF appears to cut off after lag 1, indicating AR(1) behavior. The ACF appears to cut off after lag 1, indicating MA(1) behavior we will try: ARIMA(1,1,0) and ARIMA(0,1,1) 1 1 13 14 15 16 1 2 3 4 5 6 7 8 9 Lag Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 10 11 12 13 14 15 16 4 Example 3/4 ARIMA(1,1,0) ARIMA(0,1,1) The LBQ statistics are not significant as indicated by the large pvalues for either model. Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 5 Example 4/4 Autocorrelation Function for RESI1 (with 5% significance limits for the autocorrelations) 1,0 0,8 0,4 0,2 0,0 -0,2 -0,4 -0,6 -0,8 -1,0 1 2 3 4 5 6 7 8 9 Lag 10 11 12 13 14 15 16 Autocorrelation Function for RESI2 (with 5% significance limits for the autocorrelations) 1,0 0,8 Finally, there is no significant residual autocorrelation for the ARIMA(1,1,0) model. The results for the ARIMA(0,1,1) are similar. 0,6 Autocorrelation Autocorrelation 0,6 0,4 0,2 0,0 -0,2 -0,4 -0,6 -0,8 -1,0 Therefore, either model is adequate and provide nearly the same one-step-ahead forecasts. 1 2 3 4 5 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 6 7 8 9 Lag 10 11 12 13 14 15 16 6 Examples Makridakis – – – – – – – ARIMA 7.1 ARIMA PIGS ARIMA DJ ARIMA Electricity ARIMA Computers ARIMA Sales Industry ARIMA Pollution Minitab – Employ (Food) Montgomery – – – – EXEMPLO PAG 267 EXEMPLO PAG 271 EXEMPLO PAG 278 EXEMPLO PAG 283 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 7 ARIMA Basic Model Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 8 Basic Models ARIMA (0, 0, 0) ― WHITE NOISE ARIMA (0, 1, 0) ― RANDOM WALK ARIMA (1, 0, 0) ― AUTOREGRESSIVE MODEL (order 1) ARIMA (0, 0, 1) ― MOVING AVERAGE MODEL (order 1) ARIMA (1, 0, 1) ― SIMPLE MIXED MODEL Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 9 AR MA Example Models Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 10 Autocorrelation - ACF Lag 1 2 ACF 0,0441176 -0,0916955 T 0,15 -0,32 LBQ 0,03 0,17 Diferenças são devido a pequenas modificações nas fórmulas de Regressão e Time Series Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 11 Partial Correlation • Suppose X, Y and Z are random variables. We define the notion of partial correlation between X and Y adjusting for Z. • First consider simple linear regression of X on Z CovZ , X ˆ X a1 b1Z where b1 VarZ • Also the linear regression of Y on Z CovZ , Y ˆ Y a2 b2 Z where b2 Var Z Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 12 Partial Correlation • Now consider the errors X * X Xˆ X a1 b1Z Y * Y Yˆ Y a b Z 2 2 • Then the partial correlation between X and Y, adjusting for Z, is corr X , Y corr X Xˆ , Y Yˆ * * Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 13 Partial Autocorrelation - PACF Yt Yt-1 Yt-2 Yt=f(Yt-2) Yt-1=f(Yt-2) Yt* Yt-1* Corr(Yt*, Yt-1*) Corr(X*, Y*) Correlations: X*; Y* Pearson correlation of X* and Y* =0,770 P-Value = 0,000 Partial Autocorrelation Function: X Lag 1 2 3 PACF 0,900575 -0,151346 0,082229 T 6,98 -1,17 0,64 Diferenças são devido a pequenas modificações nas fórmulas de Regressão e Time Series e no número de termos da Regressão Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 14 Theorectical Behavior for AR(1) ACF 0 PACF = 0 for lag > 1 Yt 0 1Yt 1 2Yt 2 ... pYt p t Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 15 Theorectical Behavior for AR(2) ACF 0 PACF = 0 for lag > 2 Yt 0 1Yt 1 2Yt 2 ... pYt p t Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 16 Theorectical Behavior for MA (1) PACF 0 ACF = 0 for lag > 1 Yt t 1 t 1 2 t 2 ... q t q Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 17 Theorectical Behavior for MA(2) PACF 0 PACF = 0 for lag > 2 Yt t 1 t 1 2 t 2 ... q t q Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 18 Theorectical Behavior ACF PACF AR(p) Die out Cut off after the order p of the process MA(q) Cut off after the order q of the process Die out Die out Die out ARMA(p,q) In practice, the values of p and q each rarely exceed 2. Note that: • ARMA(p,0) = AR(p) • ARMA(0,q) = MA(q) In this context… • “Die out” means “tend to zero gradually” • “Cut off” means “disappear” or “is zero” Yt 0 1Yt 1 2Yt 2 ... pYt p t 1 t 1 2 t 2 ... q t q Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 19 Review of Main Characteristics of ACF and PACF ACF MAq AR p ARMA p, q cutsof afterlag q exp.decay and/or dampedsinusoid exp.decay and/or dampedsinusoid PACF exp.decay and/or dampedsinusoid cutsof afterlag p exp.decay and/or dampedsinusoid Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 20 Example 5.1 • Weekly total number of loan applications The weekly data tend to have short runs and that the data seem to be indeed autocorrelated. Next, we visually inspect the stationarity. Although there might be a slight drop in the mean for the second year (weeks 53-104 ), in general it seems to be safe to assume stationarity. 90 Applications 80 70 60 50 1 10 EXEMPLO PAG 267.MPJ 20 30 40 60 50 Index 70 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 80 90 100 21 Example 5.1 Autocorrelation Function for Applications (with 5% significance limits for the autocorrelations) 1,0 1. It cuts off after lag 2 (or maybe even 3), suggesting a MA(2) (or MA(3)) model. 0,8 Autocorrelation 0,6 0,4 0,2 0,0 -0,2 -0,4 -0,6 2. It has an (or a mixture ot) exponential decay(s) pattern suggesting an AR(p) model. -0,8 -1,0 2 4 6 8 10 12 14 Lag 16 18 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 20 22 24 26 22 Example 5.1 Partial Autocorrelation Function for Applications (with 5% significance limits for the partial autocorrelations) 1,0 Partial Autocorrelation 0,8 It cuts off after lag 2. Hence we use the second interpretation of the sample ACF plot and assume that the appropriate model to fit is the AR(2) model. 0,6 0,4 0,2 0,0 -0,2 -0,4 -0,6 -0,8 -1,0 2 4 6 8 10 12 14 Lag 16 18 20 22 24 26 23 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 23 Autocorrelation Function for Applications Partial Autocorrelation Function for Applications (w ith 5% significance limits for the autocorrelations) (w ith 5% significance limits for the partial autocorrelations) 1,0 1,0 0,8 0,8 0,6 0,6 0,4 0,4 P ar tial A utocor r elation A utocor r elation Example 5.1 0,2 0,0 -0,2 -0,4 -0,6 -0,8 AR(p) 0,2 0,0 -0,2 -0,4 ACF -0,6 PACF Die out -0,8 Cut off after the order p of the process -1,0 -1,0 2 4 6 8 10 12 14 16 18 20 22 24 26 Lag 2 4 6 8 10 12 14 16 18 20 22 24 26 Lag Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 24 Example 5.1 The modified Box-Pierce test suggests that there is no autocorrelation left in the residuals. Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 25 Autocorrelation Function for RESI1 Partial Autocorrelation Function for RESI1 (w ith 5% significance limits for the autocorrelations) (w ith 5% significance limits for the partial autocorrelations) 1,0 1,0 0,8 0,8 0,6 0,6 0,4 0,4 P ar tial A utocor r elation A utocor r elation Example 5.1 0,2 0,0 -0,2 -0,4 0,2 0,0 -0,2 -0,4 -0,6 -0,6 -0,8 -0,8 -1,0 -1,0 2 4 6 8 10 12 14 16 18 20 22 24 26 Lag 2 4 6 8 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 10 12 14 16 18 20 22 24 26 Lag 26 Example 5.1 Normal Probability Plot Versus Fits 99,9 20 99 Residual Percent 90 50 10 0 -10 1 0,1 10 -20 -10 0 Residual 10 20 60 Histogram 80 20 15 Residual Frequency 75 Versus Order 20 10 5 0 65 70 Fitted Value 10 0 -10 -10 -5 0 5 Residual 10 15 1 10 20 30 40 50 60 70 80 Observation Order Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 90 100 27 Example 5.1 90 Variable Applications FITS1 Data 80 70 60 50 1 10 20 30 40 50 60 Index 70 80 90 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 100 28 Example 5.2 • Dow Jones Index Exemplo: Página 271 12000 Dow Jones 11000 10000 The process shows signs of nonstationarity with changing mean and possibly variance. 9000 8000 1 8 16 24 32 40 48 Index 56 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 64 72 80 29 Example 5.2 Autocorrelation Function for Dow Jones (with 5% significance limits for the autocorrelations) 1,0 0,8 Autocorrelation 0,6 0,4 0,2 0,0 -0,2 -0,4 -0,6 The slowly decreasing sample ACF and sample PACF with significant value at lag 1, which is close to 1 confirm that indeed the process can be deemed nonstationary. -0,8 -1,0 2 4 6 8 10 12 14 16 18 20 Lag Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 30 Example 5.2 Partial Autocorrelation Function for Dow Jones (with 5% significance limits for the partial autocorrelations) 1,0 Partial Autocorrelation 0,8 0,6 0,4 0,2 0,0 -0,2 -0,4 -0,6 -0,8 -1,0 One might argue that the significant sample PACF value at lag I suggests that the AR( I) model might also fit the data well. We will consider this interpretation first and fit an AR( I) model to the Dow Jones Index data. 2 4 6 8 10 12 14 16 18 20 Lag Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 31 Example 5.2 The modified Box-Pierce test suggests that there is no autocorrelation left in the residuals. This is also confirmed by the sample ACF and PACF plots of the residuals Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 32 Autocorrelation Function for RESI1 Partial Autocorrelation Function for RESI1 (w ith 5% significance limits for the autocorrelations) (w ith 5% significance limits for the partial autocorrelations) 1,0 1,0 0,8 0,8 0,6 0,6 0,4 0,4 P ar tial A utocor r elation A utocor r elation Example 5.2 0,2 0,0 -0,2 -0,4 0,2 0,0 -0,2 -0,4 -0,6 -0,6 -0,8 -0,8 -1,0 -1,0 2 4 6 8 10 12 Lag 14 16 18 20 2 4 6 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 8 10 12 Lag 14 16 18 20 33 Example 5.2 Residual Plots for Dow Jones Normal Probability Plot Versus Fits 99,9 1000 500 90 Residual Percent 99 50 10 -1000 -500 0 Residual 500 1000 8000 15 Residual Frequency The only concern in the residual plots in Histogram is20 in the changing variance observed in the time series plot of the residuals. 10 5 0 -500 -1000 1 0,1 0 9000 10000 Fitted Value 11000 Versus Order 1000 500 0 -500 -1000 -1200 -800 -400 0 Residual 400 800 1 10 20 30 40 50 60 Observation Order Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 70 80 34 Example 5.2 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 35 Example 5.2 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 36 Example 5.2 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 37 Example 5.2 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 38 Example 5.3 90 Prediction with AR(2) Applications 80 70 60 50 1 10 20 30 40 50 60 Week 70 80 90 100 Exemplo pag 278 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 39 Example 5.3 Time Series Plot for Applications (with forecasts and their 95% confidence limits) 90 Applications 80 70 60 50 1 10 20 30 40 50 60 Time 70 80 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 90 100 110 40 Example 5.3 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 41 Example 5.5 • U.S. Clothing Sales Data 17500 Exemplo: Página 283 The data obviously exhibit some seasonality and upward linear trend. c 15000 Sales 12500 10000 7500 5000 jan-92 fev-93 abr-94 jun-95 ago-96 out-97 dez-98 fev-00 Date abr-01 jun-02 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br ago-03 42 Example 5.5 Autocorrelation Function for Sales (with 5% significance limits for the autocorrelations) 1,0 0,8 Autocorrelation 0,6 0,4 0,2 0,0 -0,2 -0,4 -0,6 The sample ACF and PACF indicate a monthly seasonality, s = 12, as ACF values at lags 12, 24, 36 are significant and slowly decreasing -0,8 -1,0 1 5 10 15 20 25 30 35 Lag Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 43 Example 5.5 Autocorrelation Function for Sales (with 5% significance limits for the autocorrelations) 1,0 0,8 Autocorrelation 0,6 0,4 0,2 0,0 -0,2 -0,4 -0,6 The sample ACF and PACF indicate a monthly seasonality, s = 12, as ACF values at lags 12, 24, 36 are significant and slowly decreasing -0,8 -1,0 1 5 10 15 20 25 30 35 Lag Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 44 Example 5.5 Partial Autocorrelation Function for Sales (with 5% significance limits for the partial autocorrelations) 1,0 Partial Autocorrelation 0,8 0,6 0,4 0,2 0,0 -0,2 -0,4 There is a significant PACF value at lag 12 that is close to 1. Moreover, -0,6 the slowly decreasing ACF in general also indicates a nonstationarity that can be remedied by taking the first difference. Hence -1,0 we would now consider -0,8 1 5 10 15 20 25 30 35 Lag Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 45 Example 5.5 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 46 Example 5.5 Partial Autocorrelation Function for Sales (with 5% significance limits for the partial autocorrelations) 1,0 Partial Autocorrelation 0,8 0,6 0,4 0,2 0,0 -0,2 -0,4 There is a significant PACF value at lag 12 that is close to 1. Moreover, -0,6 the slowly decreasing ACF in general also indicates a nonstationarity that can be remedied by taking the first difference. Hence -1,0 we would now consider -0,8 1 5 10 15 20 25 30 35 Lag Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 47 Example 5.5 Figure shows that first difference together with seasonal differencing helps in terms of stationarity and eliminating the seasonality 1500 Comp Dif 1000 500 0 -500 -1000 jan-92 fev-93 abr-94 jun-95 ago-96 out-97 dez-98 fev-00 abr-01 Date Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br jun-02 ago-03 48 Example 5.5 Autocorrelation Function for Comp Dif (with 5% significance limits for the autocorrelations) 1,0 0,8 Autocorrelation 0,6 0,4 0,2 0,0 -0,2 -0,4 -0,6 -0,8 -1,0 1 5 10 15 20 25 30 Lag Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 49 Example 5.5 Partial Autocorrelation Function for Comp Dif (with 5% significance limits for the partial autocorrelations) 1,0 Partial Autocorrelation 0,8 0,6 0,4 0,2 0,0 -0,2 -0,4 The sample ACF with a significant value at lag 1 and the sample PACF with exponentially decaying values at the first 8 lags suggest that a nonseasonal MA( I) model should be used. -0,6 -0,8 -1,0 1 5 10 15 20 25 30 Lag Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 50 Example 5.5 The interpretation of the remaining seasonality is a bit more difficult. For that we should focus on the sample ACF and PACF values at lags 12. 24, 36, and so on. The sample ACF at lag 12 seems to be significant and the sample PACF at lags 12, 24, 36 (albeit not significant) seems to be alternating in sign. That suggests that a seasonal MA(1) model can be used as well. Hence an ARIMA (0, 1, 1) x (0, 1, 1) 12 model is used to model the data, yt Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 51 Example 5.5 Both MA( 1) and seasonal MA( 1) coefficient estimates are significant. As we can see from the sample ACF and PACF plots, while there are still some small significant values, as indicated by the modified Box-Pierce statistic, most of the autocorrelation is now modeled out. Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 52 Autocorrelation Function for RESI1 Partial Autocorrelation Function for RESI1 (w ith 5% significance limits for the autocorrelations) (w ith 5% significance limits for the partial autocorrelations) 1,0 1,0 0,8 0,8 0,6 0,6 0,4 0,4 P ar tial A utocor r elation A utocor r elation Example 5.5 0,2 0,0 -0,2 -0,4 -0,6 0,2 0,0 -0,2 -0,4 -0,6 and PACF plots, while there As we can see from the sample ACF are still some small significant values, -0,8 as indicated by the modified Box-Pierce statistic, most of the autocorrelation is now modeled out. -0,8 -1,0 -1,0 1 5 10 15 20 Lag 25 30 1 5 10 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 15 20 Lag 25 30 53 Example 5.5 Normal Probability Plot Versus Fits 99,9 99 500 Residual Percent 90 50 10 0 -500 1 0,1 -1000 -500 0 Residual 500 -1000 1000 6000 9000 12000 Fitted Value Histogram 500 15 Residual Frequency 18000 Versus Order 20 10 5 0 15000 -900 -600 -300 0 Residual 300 600 0 -500 -1000 1 10 20 30 40 50 60 70 80 90 100 110 120 130 140 Observation Order Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 54 Example 5.5 Time Series Plot for Sales (with forecasts and their 95% confidence limits) 20000 17500 Sales 15000 12500 10000 7500 5000 1 12 24 36 48 60 72 84 Time 96 108 120 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 132 144 156 55 Introduction Exponential smoothing. The general assumption for these models was that any time series data can be represented as the sum of two distinct components: deterministic and stochastic (random). The former (deterministic) is modeled as a function of time whereas for the latter (stochastic) we assumed that some random noise that is added on the deterministic signal generates the stochastic behavior of the time series. One very important assumption is that the random noise is generated through independent shocks to the process. In practice, however, this assumption is often violated. That is, usually successive observations show serial dependence. Under these circumstances, forecasting methods based on exponential smoothing may be inefficient and sometimes inappropriate because they do not take advantage of the serial dependence in the observations in the most effective way. To formally incorporate this dependent structure, we will explore a general class of models called autoregressive integrated moving average models or ARIMA models (also known as Box-Jenkins models). Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 56 Linear Models for Stationary Time Series • A linear filter is defined as Um conceito de Processamento de Sinais is said to be Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 57 Stationarity Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 58 Some Examples Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 59 Stationary Time Series • Many time series do not exhibit a stationary behavior • The stationarity is in fact a rarity in real life • However it provides a foundation to build upon since (as we will see later on) if the time series in not stationary, its first difference (yt-yt-1) will often be Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 60 Linear Filter Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 61 If Input is White Noise Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 62 Using the Backshift Operator Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 63 Wold’s Decomposition Theorem • Any nondeterministic weakly stationary time series can be written as an infinite sum of weighted random shocks (disturbances) yt i t i i 0 where 2 i i 0 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 64 How useful is this? Well, not so much!!! How can we come up with “infinitely” many terms? Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 65 Maybe we should consider some special cases: Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 66 Finite Order Moving Average Processes (Ma(q)) Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 67 Some Properties • Expected Value • Variance Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 68 Some Properties • Autocovariance Function • Autocorrelation Function (ACF) Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 69 Autocorrelation Function of MA(q) • ACF of Ma(q) ”cuts off” after lag q • This is very useful in the identification of an MA(q) process Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 70 Example Employ.mtw Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 71 Diferences Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 72 Autocorrelation The graphs for the autocorrelation function (ACF) of the ARIMA residuals include lines representing two standard errors to either side of zero. Values that extend beyond two standard errors are statistically significant at approximately a = 0.05, and show evidence that the model does not explain thel autocorrelation in the data. Because you did not specify the lag length, autocorrelation uses the default length of n / 4 for a series with less than or equal to 240 observations. Minitab generates an autocorrelation function (ACF) with approximate a = 0.05 critical bands for the hypothesis that the correlations are equal to zero. Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 73 Autocorrelation The ACF for these data shows large positive, significant spikes at lags 1 and 2 with subsequent positive autocorrelations that do not die off quickly. This pattern is typical of an autoregressive process. Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 74 Ljung-Box q statistic Use to test whether a series of observations over time are random and independent. If observations are not independent, one observation may be correlated with another observation k time units later, a relationship called autocorrelation. Autocorrelation can impair the accuracy of a time-based predictive model, such as time series plot, and lead to misinterpretation of the data. For example, an electronics company tracks monthly sales of batteries for five years. They want to use the data to develop a time series model to help forecast future sales. However, monthly sales may be affected by seasonal trends. For example, every year a rise in sales occurs when people buy batteries for Christmas toys. Thus a monthly sales observation in one year could be correlated with a monthly sales observations 12 months later (a lag of 12). Before choosing their time series model, they can evaluate autocorrelation for the monthly differences in sales. The Ljung-Box Q (LBQ) statistic tests the null hypothesis that autocorrelations up to lag k equal zero (i.e., the data values are random and independent up to a certain number of lags--in this case 12). If the LBQ is greater than a specified critical value, autocorrelations for one or more lags may be significantly different from zero, suggesting the values are not random and independent over time. LBQ is also used to evaluate assumptions after fitting a time series model, such as ARIMA, to ensure that the residuals are independent. The Ljung-Box is a Portmanteau test and is a modified version of the Box-Pierce chi-square statistic. Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 75 You can use the Ljung-Box Q (LBQ) statistic to test the null hypothesis that the autocorrelations for all lags up to lag k equal zero. Let's test that all autocorrelations up to a lag of 6 are zero. The LBQ statistic is 56.03. Ho: Autocorrelation (lag<6) = 0 Variable CumProb is created Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 76 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 77 In this example, the p-value is 0.000000, which means the pvalue is less than 0.0000005. The very small p-value implies that one or more of the autocorrelations up to lag 6 can be judged as significantly different from zero at any reasonable a level. Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 78 Partial autocorrelation computes and plots the partial autocorrelations of a time series. Partial autocorrelations, like autocorrelations, are correlations between sets of ordered data pairs of a time series. As with partial correlations in the regression case, partial autocorrelations measure the strength of relationship with other terms being accounted for. The partial autocorrelation at a lag of k is the correlation between residuals at time t from an autoregressive model and observations at lag k with terms for all intervening lags present in the autoregressive model. The plot of partial autocorrelations is called the partial autocorrelation function or PACF. View the PACF to guide your choice of terms to include in an ARIMA model. Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 79 You obtain a partial autocorrelation function (PACF) of the food industry employment data, after taking a difference of lag 12, in order to help determine a likely ARIMA model. Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 80 Minitab generates a partial autocorrelation function with critical bands at approximately a = 0.05 for the hypothesis that the correlations are equal to zero. In the food data example, there is a single large spike of 0.7 at lag 1, which is typical of an autoregressive process of order one. There is also a significant spike at lag 9, but you have no evidence of a nonrandom process occurring there. Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 81 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 82 Sample ACF • Will not be equal to zero after lag q for an MA(q) • But it will be small • For the same size of N, this can be tested using the limits: 2 N Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 83 First-Order Moving Average Process MA(1) for which autocovariance and autocorrelation functions are given as Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 84 Some Examples Note, the behavior of sample ACF Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 85 Second-Order Moving Average Process MA(2) for which autocovariance and autocorrelation functions are given as Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 86 An Example Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 87 Finite Order Autoregressive Processes (AR(p)) • MA(q) processes take into account disturbances up to q lags in the past • What if all past disturbances have some lingering effects? Back to square one? • We may be able to come up with some special cases though Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 88 A very special case • What if we let j j for 1 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 89 Decomposition yt t i t t i i i 0 i i 1 and i 0 i 1 yt 1 i t 1i t i t i 1 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 90 Combining the two equations i i yt yt 1 t i t i 1 i 0 i 0 t yt yt 1 at This is an AR(1) model Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 91 First-Order Autoregressive Process (AR(1)) yt yt 1 at Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 92 Properties • Expected Value • Autocovariance Function • Autocorrelation Function Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 93 Some Examples Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 94 Second-Order Autoregressive Process (AR(2)) Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 95 Conditions for Stationarity yt 1 yt 1 2 yt 2 t 1 B B y 2 1 2 t t B yt t yt B B t 1 B t Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 1 B 96 AR(2) is stationary if … we can writeas B i B i Since B B 1 i 0 1 1B 2 B 0 1B 2 B 1 2 0 1 1 0 B 2 1 1 2 0 B 2 j 1 j 1 2 j 2 B 1 j Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 97 AR(2) is stationary if … 0 1 1 1 0 0 2 1 1 2 0 0 j 1 j 1 2 j 2 0 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br j2 98 AR(2) • Hence {j} satisfy the 2nd order linear difference equation. So the i can be expressed as the solution to this equation in terms of the 2 roots m1 and m2 of the associated polynomial m 1m 2 0 2 • If the roots m1 and m2 satisfy m1 , m2 1 then e.g. m , m 1 2 j 0 j are real then j c1m1j c2m2j Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 99 AR(2) is stationary if the roots m1 and m2 of m 1m 2 0 are both less than 2 one in absolute value Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 100 ACF of a stationary AR(2) k Cov yt , yt k Cov1 yt 1 2 yt 2 t , yt k 1Cov yt 1 , yt k 2Cov yt 2 , yt k Cov t , yt k 2 if k 0 1 k 1 2 k 2 0 if k 0 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 101 ACF of a stationary AR(2) 0 1 1 2 2 2 k 1 k 1 2 k 2 k 1,2, Yule-Walker Equations k 1 k 1 2 k 2 k 1,2, Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 102 ACF of a stationary AR(2) • Hence ACF satisfies the 2nd order linear difference equation. So the (k) can be expressed as the solution to this equation in terms of the 2 roots m1 and m2 of the associated polynomial m 1m 2 0 2 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 103 ACF of a stationary AR(2) Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 104 Some Examples Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 105 AR(p) p yt i yt i t i 1 B yt t where B 1 1 B 2 B p B 2 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br p 106 AR(p) is Stationary • If the roots of m 1m p p 1 2m p 2 p 0 are less than one in absolute value. Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 107 Infinite MA representation Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 108 Expected Value of an AR(p) Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 109 Autocovariance Function of an AR(p) Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 110 Autocorrelation Function of an AR(p) Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 111 ACF of AR(p) p k i k i k 1,2, i 1 In general ACF of AR(p) can be a mixture of exponential decay and damped sinusoidal behavior depending on the solution to the corresponding Yule-Walker equations. Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 112 ACF of AR(p) Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 113 ACF for AR(p) and MA(q) • ACF of MA(q) “cuts off” after q k 1 k 1 q k q y k 1 12 22 q2 0 k 0,, q k q • ACF of AR(p) can be a mixture of exponential decay and damped sinusoidal Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 114 So how are we going to determine p in the AR(p) model? Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 115 Partial Correlation • Suppose X, Y and Z are random variables. We define the notion of partial correlation between X and Y adjusting for Z. • First consider simple linear regression of X on Z CovZ , X ˆ X a1 b1Z where b1 VarZ • Also the linear regression of Y on Z CovZ , Y ˆ Y a2 b2 Z where b2 Var Z Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 116 Partial Correlation • Now consider the errors X * X Xˆ X a1 b1Z Y * Y Yˆ Y a b Z 2 2 • Then the partial correlation between X and Y, adjusting for Z, is corr X , Y corr X Xˆ , Y Yˆ * * Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 117 Partial Autocorrelation Function (PACF) Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 118 Partial Autocorrelation Function (PACF) Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 119 Partial Autocorrelation Function (PACF) Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 120 Partial Autocorrelation Function (PACF) Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 121 Sample Partial Autocorrelation Function Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 122 Some Examples Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 123 PACF • For an AR(p) process, PACF cuts off after lag p. • For an MA(q) process, PACF has an exponential decay and/or a damped sinusoid form Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 124 Invertibility of a MA Process Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 125 Invertibility of a MA Process Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 126 Invertibility of a MA Process We have Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 127 The ACF and PACF do have very distinct and indicative properties for MA and AR models. Therefore in model identification it is strongly recommended to use both the sample ACF and PACF simultaneously Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 128 Mixed Autoregressive-Moving Average (ARMA(p,q)) Process Byt B t Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 129 Stationarity of ARMA(p,q) Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 130 Invertibility of ARMA(p,q) Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 131 ACF and PACF of an ARMA(p,q) • Both ACF and PACF of an ARMA(p,q) can be a mixture of exponential decay and damped sinusoids depending on the roots of the AR operator. Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 132 ARMA Models • For ARMA models, except for possible special cases, neither ACF nor PACF has distinctive features that would allow “easy identification” • For this reason, there have been many additional sample functions considered to help with identification problem: – – – – Extended sample ACF (ESACF) Generalized sample PACF (GPACF) Inverse ACF Use of “canonical correlations” Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 133 Some Examples Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 134 Review of Main Characteristics of ACF and PACF ACF MAq AR p ARMA p, q cutsof afterlag q exp.decay and/or dampedsinusoid exp.decay and/or dampedsinusoid PACF exp.decay and/or dampedsinusoid cutsof afterlag p exp.decay and/or dampedsinusoid Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 135 Review of Main Characteristics of Sample ACF and PACF Sample ACF E ˆ 0 for k q MAq 1 ˆ Var for k q T AR p exp.decay and/or ARMA p, q exp.decay and/or dampedsinusoid dampedsinusoid Sample PACF exp.decay and/or dampedsinusoid E ˆ 0 for k p Var ˆ 1 for k p T exp.decay and/or dampedsinusoid Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 136 Some Examples Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 137 Some Examples Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 138 Some Examples Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 139 Some Examples Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 140 Some Examples Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 141 ARIMA Models • Process {yt} is ARIMA(p,d,q), if the dth order differences, wt=(1-B)dyt, form a stationary ARMA(p,q) process: Bwt B t • Thus {yt} satisfies B1 B yt B t d Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 142 Some Examples Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 143 Some Examples Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 144 Model Building • Given T observations from a process, want to obtain a model that adequately represents the main features of the time series data. Model can be used for purposes of forecasting, control, … Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 145 3-Stage Procedure • STAGE 1: Model Specification or Identification – Consider issue of nonstationarity vs. stationarity of series. Use procedures such as differencing to obtain a stationary series; say wt=(1-B)dyt • Examine sample ACF and PACF of wt and use features of these functions to identify an appropriate ARMA model. The specification is “tentative” Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 146 Review of Main Characteristics of ACF and PACF ACF MAq cutsof afterlag q AR p ARMA p, q exp.decay and/or dampedsinusoid exp.decay and/or dampedsinusoid PACF exp.decay and/or dampedsinusoid cutsof afterlag p exp.decay and/or dampedsinusoid Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 147 Review of Main Characteristics of Sample ACF and PACF Sample ACF E ˆ 0 for k q MAq 1 Var ˆ for k q T AR p exp.decay and/or ARMA p, q exp.decay and/or dampedsinusoid dampedsinusoid Sample PACF exp.decay and/or dampedsinusoid E ˆ 0 for k p 1 Var ˆ for k p T exp.decay and/or dampedsinusoid Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 148 ARMA Models • For ARMA models, except for possible special cases, neither ACF nor PACF has distinctive features that would allow “easy identification” • For this reason, there have been many additional sample functions considered to help with identification problem: – – – – Extended sample ACF (ESACF) Generalized sample PACF (GPACF) Inverse ACF Use of “canonical correlations” Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 149 3-Stage Procedure • STAGE 2: Estimation of Parameters in Tentatively Specified Model – Method of moments – Least Squares – Maximum Likelihood Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 150 3-Stage Procedure • STAGE 3: Model Checking – Based on examining features of residuals p q i 1 i 1 ˆt yt ˆi yt i ˆ ˆiˆt i Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 151 3-Stage Procedure • STAGE 3: If the specified model is appropriate order p, q; then we expect the residuals behave similar to the “true” white noise t. Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 152 Example 5.1 • Weekly total number of loan applications Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 153 Example 5.1 154 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 154 Example 5.1 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 155 Example 5.1 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 156 Example 5.1 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 157 Example 5.1 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 158 Example 5.1 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 159 Example 5.2 • Dow Jones Index Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 160 Example 5.2 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 161 Example 5.2 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 162 Example 5.2 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 163 Example 5.2 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 164 Example 5.2 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 165 Example 5.2 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 166 Example 5.2 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 167 Example 5.2 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 168 Example 5.2 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 169 Forecasting ARIMA Processes Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 170 Forecasting ARIMA Processes Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 171 The “best” forecast Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 172 Forecast Error Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 173 Prediction Intervals Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 174 Two Issues Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 175 Illustration Using ARIMA(1,1,1) • ARIMA(1,1,1) process is given as 1 B1 ByT 1 BT • Two commonly used approaches Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 176 Approach 1 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 177 Approach 2 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 178 Example 5.3 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 179 Seasonal Processes Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 180 Seasonal Processes Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 181 Seasonal Processes Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 182 Seasonal Processes Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 183 Example 5.4 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 184 Example 5.5 • U.S. Clothing Sales Data Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 185 Example 5.5 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 186 Example 5.5 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 187 Example 5.5 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 188 Example 5.5 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 189 Example 5.5 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 190 Example 5.5 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 191 Example 5.5 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 192 Example 5.5 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 193 Example 5.5 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 194 Use ARIMA to model time series behavior and to generate forecasts. ARIMA fits a Box-Jenkins ARIMA model to a time series. ARIMA stands for Autoregressive Integrated Moving Average with each term representing steps taken in the model construction until only random noise remains. ARIMA modeling differs from the other time series methods in the fact that ARIMA modeling uses correlational techniques. ARIMA can be used to model patterns that may not be visible in plotted data. Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 195 The ACF and PACF of the food employment data suggest an autoregressive model of order 1, or AR(1), after taking a difference of order 12. You fit that model here, examine diagnostic plots, and examine the goodness of fit. To take a seasonal difference of order 12, you specify the seasonal period to be 12, and the order of the difference to be 1. Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 196 1 Model is specified by the usual notation (pdq) x (PDQ) S: (pdq) is for a nonseasonal model; (PDQ) for a seasonal, and S is the seasonality. 2 At least one of the p, P, q, or Q parameters must be non-zero, and none may exceed five. 3 The maximum number of parameters you can estimate is ten. 4 At least three data points must remain after differencing. That is, S * D + d + 2 must be less than the number of points, where S is the length of a season. 5 The maximum "back order" for the model is 100. In practice, this condition is always satisfied if S * D + d + p + P + q + Q is at most 100. 6 The ARIMA model normally includes a constant term only if there is no differencing (that is, d = D = 0). 7 Missing observations are only allowed at the beginning or the end of a series, not in the middle. 8 The seasonal component of this model is multiplicative, and thus is appropriate when the amount of cyclical variation is proportional to the mean. Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 197 The ARIMA model converged after nine iterations. The AR(1) parameter had a t-value of 7.42. As a rule of thumb, you can consider values over two as indicating that the associated parameter can be judged as significantly different from zero. The MSE (1.1095) can be used to compare fits of different ARIMA models. The Ljung-Box statistics give nonsignificant p-values , indicating that the residuals appeared to uncorrelated. The ACF and PACF of the residuals corroborate this. You assume that the spikes in the ACF and PACF at lag 9 are the result of random events The coefficients are estimated using an iterative algorithm that calculates least squares estimates. At each iteration, the back forecasts are computed and SSE is calculated. Back forecasts are calculated using the specified model and the current iteration's parameter estimates Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 198 Box and Jenkins [2] present an interactive approach for fitting ARIMA models to time series. This iterative approach involves identifying the model, estimating the parameters, checking model adequacy, and forecasting, if desired. The model identification step generally requires judgment from the analyst. 1 First, decide if the data are stationary. That is, do the data possess constant mean and variance . · Examine a time series plot to see if a transformation is required to give constant variance. · Examine the ACF to see if large autocorrelations do not die out, indicating that differencing may be required to give a constant mean. A seasonal pattern that repeats every kth time interval suggests taking the kth difference to remove a portion of the pattern. Most series should not require more than two difference operations or orders. Be careful not to overdifference. If spikes in the ACF die out rapidly, there is no need for further differencing. A sign of an overdifferenced series is the first autocorrelation close to -0.5 and small values elsewhere. Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 199 2 Next, examine the ACF and PACF of your stationary data in order to identify what autoregressive or moving average models terms are suggested. · An ACF with large spikes at initial lags that decay to zero or a PACF with a large spike at the first and possibly at the second lag indicates an autoregressive process. · An ACF with a large spike at the first and possibly at the second lag and a PACF with large spikes at initial lags that decay to zero indicates a moving average process. · The ACF and the PACF both exhibiting large spikes that gradually die out indicates that both autoregressive and moving averages processes are present. For most data, no more than two autoregressive parameters or two moving average parameters are required in ARIMA models. Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 200 3 Once you have identified one or more likely models, you are ready to use the ARIMA procedure. · Fit the likely models and examine the significance of parameters and select one model that gives the best fit. · Check that the ACF and PACF of residuals indicate a random process, signified when there are no large spikes. You can easily obtain an ACF and a PACF of residual using ARIMA's Graphs subdialog box. If large spikes remain, consider changing the model. · You may perform several iterations in finding the best model. When you are satisfied with the fit, go ahead and make forecasts. The ARIMA algorithm will perform up to 25 iterations to fit a given model. If the solution does not converge, store the estimated parameters and use them as starting values for a second fit. You can store the estimated parameters and use them as starting values for a subsequent fit as often as necessary. Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 201 The graphs for the ACF and PACF of the ARIMA residuals include lines representing two standard errors to either side of zero. Values that extend beyond two standard errors are statistically significant at approximately a = 0.05, and show evidence that the model has not explained all autocorrelation in the data. Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 202 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 203 The AR(1) model appears to fit well so you use it to forecast employment. Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 204 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 205 Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 206 The ARIMA algorithm is based on the fitting routine in the TSERIES package written by Professor William Q. Meeker, Jr., of Iowa State University. • W.Q. Meeker, Jr. (1977). "TSERIES-A Useroriented Computer Program for Identifying, Fitting and Forecasting ARIMA Time Series Models," ASA 1977 Proceedings of the Statistical Computing Section. • W.Q. Meeker, Jr. (1977). TSERIES User's Manual, Statistical Laboratory, Iowa State University. Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br 207

Download
# arima - Unifei