5 – Autoregressive
Integrated Moving
Average (ARIMA) Models
Box & Jenkins
Methodology
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
1
ARIMA
Box-Jenkins
Methodology
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
2
Example 1/4
Time Series Plot of Index
290
280
270
The series
show an
upward
trend.
250
240
230
220
210
1
6
12
18
24
30
36
Index
42
48
54
60
Autocorrelation Function for Index
(with 5% significance limits for the autocorrelations)
1,0
The first several autocorrelations are
persistently large and trailed off to zero
rather slowly  a trend exists and this
time series is nonstationary (it does
not vary about a fixed level)
0,8
0,6
Autocorrelation
Index
260
0,4
0,2
0,0
-0,2
-0,4
-0,6
-0,8
-1,0
Idea: to difference the data to see if we could
eliminate the trend and create a stationary series.
1
2
3
4
5
6
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
7
8
9
Lag
10
11
12
13
14
15
16
3
Example 2/4
Time Series Plot of Diff1
First order differences.
5
4
A plot of the
differenced data
appears to vary about
a fixed level.
3
Diff1
2
-2
-3
-4
1
0,8
0,8
0,6
0,6
Partial Autocorrelation
Autocorrelation
1,0
0,4
0,2
0,0
-0,2
-0,4
-0,6
10
11
12
48
54
60
0,0
-0,4
-0,6
-1,0
8
9
Lag
42
-0,2
-1,0
7
30
36
Index
0,2
-0,8
6
24
0,4
-0,8
5
18
(with 5% significance limits for the partial autocorrelations)
1,0
4
12
Partial Autocorrelation Function for Diff1
Autocorrelation Function for Diff1
3
6
A constant term in each model will be included to
allow for the fact that the series of differences
appears to vary about a level greater than zero.
(with 5% significance limits for the autocorrelations)
2
0
-1
Comparing the autocorrelations with their error limits, the
only significant autocorrelation is at lag 1. Similarly, only
the lag 1 partial autocorrelation is significant. The PACF
appears to cut off after lag 1, indicating AR(1) behavior.
The ACF appears to cut off after lag 1, indicating MA(1)
behavior  we will try: ARIMA(1,1,0) and ARIMA(0,1,1)
1
1
13
14
15
16
1
2
3
4
5
6
7
8
9
Lag
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
10
11
12
13
14
15
16
4
Example 3/4
ARIMA(1,1,0)
ARIMA(0,1,1)
The LBQ statistics are not significant as indicated by the large pvalues for either model.
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
5
Example 4/4
Autocorrelation Function for RESI1
(with 5% significance limits for the autocorrelations)
1,0
0,8
0,4
0,2
0,0
-0,2
-0,4
-0,6
-0,8
-1,0
1
2
3
4
5
6
7
8
9
Lag
10
11
12
13
14
15
16
Autocorrelation Function for RESI2
(with 5% significance limits for the autocorrelations)
1,0
0,8
Finally, there is no significant residual
autocorrelation for the ARIMA(1,1,0)
model. The results for the ARIMA(0,1,1)
are similar.
0,6
Autocorrelation
Autocorrelation
0,6
0,4
0,2
0,0
-0,2
-0,4
-0,6
-0,8
-1,0
Therefore, either model is adequate and provide
nearly the same one-step-ahead forecasts.
1
2
3
4
5
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
6
7
8
9
Lag
10
11
12
13
14
15
16
6
Examples
Makridakis
–
–
–
–
–
–
–
ARIMA 7.1
ARIMA PIGS
ARIMA DJ
ARIMA Electricity
ARIMA Computers
ARIMA Sales Industry
ARIMA Pollution
Minitab
– Employ (Food)
Montgomery
–
–
–
–
EXEMPLO PAG 267
EXEMPLO PAG 271
EXEMPLO PAG 278
EXEMPLO PAG 283
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
7
ARIMA Basic Model
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
8
Basic Models
ARIMA (0, 0, 0)
―
WHITE NOISE
ARIMA (0, 1, 0)
―
RANDOM WALK
ARIMA (1, 0, 0)
―
AUTOREGRESSIVE
MODEL (order 1)
ARIMA (0, 0, 1)
―
MOVING AVERAGE MODEL
(order 1)
ARIMA (1, 0, 1)
―
SIMPLE MIXED MODEL
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
9
AR MA Example Models
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
10
Autocorrelation - ACF
Lag
1
2
ACF
0,0441176
-0,0916955
T
0,15
-0,32
LBQ
0,03
0,17
Diferenças são devido a pequenas modificações nas fórmulas
de Regressão e Time Series
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
11
Partial Correlation
• Suppose X, Y and Z are random variables. We
define the notion of partial correlation between
X and Y adjusting for Z.
• First consider simple linear regression of X on Z
CovZ , X 
ˆ
X  a1  b1Z where b1 
VarZ 
• Also the linear regression of Y on Z
CovZ , Y 
ˆ
Y  a2  b2 Z where b2 
Var Z 
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
12
Partial Correlation
• Now consider the errors
X *  X  Xˆ  X  a1  b1Z 
Y *  Y  Yˆ  Y  a  b Z 
2
2
• Then the partial correlation between X and Y,
adjusting for Z, is



corr X , Y  corr X  Xˆ , Y  Yˆ
*
*
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br

13
Partial Autocorrelation - PACF
Yt
Yt-1
Yt-2
Yt=f(Yt-2) Yt-1=f(Yt-2)
Yt*
Yt-1*
Corr(Yt*, Yt-1*)
Corr(X*, Y*)
Correlations: X*; Y*
Pearson correlation of X* and Y* =0,770
P-Value = 0,000
Partial Autocorrelation Function: X
Lag
1
2
3
PACF
0,900575
-0,151346
0,082229
T
6,98
-1,17
0,64
Diferenças são devido a pequenas
modificações nas fórmulas de
Regressão e Time Series e no
número de termos da Regressão
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
14
Theorectical Behavior for AR(1)
ACF  0
PACF = 0 for lag > 1
Yt  0  1Yt 1  2Yt 2  ...  pYt  p   t
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
15
Theorectical Behavior for AR(2)
ACF  0
PACF = 0 for lag > 2
Yt  0  1Yt 1  2Yt 2  ...  pYt  p   t
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
16
Theorectical Behavior for MA (1)
PACF  0
ACF = 0 for lag > 1
Yt     t  1 t 1  2 t 2  ...  q t q
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
17
Theorectical Behavior for MA(2)
PACF  0
PACF = 0 for lag > 2
Yt     t  1 t 1  2 t 2  ...  q t q
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
18
Theorectical Behavior
ACF
PACF
AR(p)
Die out
Cut off after the order
p of the process
MA(q)
Cut off after the order
q of the process
Die out
Die out
Die out
ARMA(p,q)
In practice, the values
of p and q each rarely
exceed 2.
Note that:
• ARMA(p,0)
= AR(p)
• ARMA(0,q)
= MA(q)
In this context…
• “Die out” means “tend to zero gradually”
• “Cut off” means “disappear” or “is zero”
Yt  0  1Yt 1  2Yt 2  ...  pYt  p   t  1 t 1  2 t 2  ...  q t q
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
19
Review of Main Characteristics of ACF
and PACF
ACF
MAq 
AR p 
ARMA p, q 
cutsof afterlag q
exp.decay and/or
dampedsinusoid
exp.decay and/or
dampedsinusoid
PACF
exp.decay and/or
dampedsinusoid
cutsof afterlag p
exp.decay and/or
dampedsinusoid
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
20
Example 5.1
• Weekly total number of loan applications
The weekly data tend to have short runs and that
the data seem to be indeed autocorrelated. Next,
we visually inspect the stationarity. Although there
might be a slight drop in the mean for the
second year (weeks 53-104 ), in general it
seems to be safe to assume stationarity.
90
Applications
80
70
60
50
1
10
EXEMPLO PAG 267.MPJ
20
30
40
60
50
Index
70
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
80
90
100
21
Example 5.1
Autocorrelation Function for Applications
(with 5% significance limits for the autocorrelations)
1,0
1. It cuts off after lag 2 (or maybe even 3),
suggesting a MA(2) (or MA(3)) model.
0,8
Autocorrelation
0,6
0,4
0,2
0,0
-0,2
-0,4
-0,6
2. It has an (or a mixture ot) exponential decay(s)
pattern suggesting an AR(p) model.
-0,8
-1,0
2
4
6
8
10
12
14
Lag
16
18
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
20
22
24
26
22
Example 5.1
Partial Autocorrelation Function for Applications
(with 5% significance limits for the partial autocorrelations)
1,0
Partial Autocorrelation
0,8
It cuts off after lag 2. Hence we use the
second interpretation of the sample ACF
plot and assume that the appropriate
model to fit is the AR(2) model.
0,6
0,4
0,2
0,0
-0,2
-0,4
-0,6
-0,8
-1,0
2
4
6
8
10
12
14
Lag
16
18
20
22
24
26
23
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
23
Autocorrelation Function for Applications
Partial Autocorrelation Function for Applications
(w ith 5% significance limits for the autocorrelations)
(w ith 5% significance limits for the partial autocorrelations)
1,0
1,0
0,8
0,8
0,6
0,6
0,4
0,4
P ar tial A utocor r elation
A utocor r elation
Example 5.1
0,2
0,0
-0,2
-0,4
-0,6
-0,8
AR(p)
0,2
0,0
-0,2
-0,4
ACF
-0,6
PACF
Die out
-0,8
Cut off after the order
p of the process
-1,0
-1,0
2
4
6
8
10 12 14 16 18 20 22 24 26
Lag
2
4
6
8
10 12 14 16 18 20 22 24 26
Lag
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
24
Example 5.1
The modified Box-Pierce test
suggests that there is no
autocorrelation left in the
residuals.
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
25
Autocorrelation Function for RESI1
Partial Autocorrelation Function for RESI1
(w ith 5% significance limits for the autocorrelations)
(w ith 5% significance limits for the partial autocorrelations)
1,0
1,0
0,8
0,8
0,6
0,6
0,4
0,4
P ar tial A utocor r elation
A utocor r elation
Example 5.1
0,2
0,0
-0,2
-0,4
0,2
0,0
-0,2
-0,4
-0,6
-0,6
-0,8
-0,8
-1,0
-1,0
2
4
6
8
10 12 14 16 18 20 22 24 26
Lag
2
4
6
8
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
10 12 14 16 18 20 22 24 26
Lag
26
Example 5.1
Normal Probability Plot
Versus Fits
99,9
20
99
Residual
Percent
90
50
10
0
-10
1
0,1
10
-20
-10
0
Residual
10
20
60
Histogram
80
20
15
Residual
Frequency
75
Versus Order
20
10
5
0
65
70
Fitted Value
10
0
-10
-10
-5
0
5
Residual
10
15
1
10
20
30 40 50 60 70 80
Observation Order
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
90 100
27
Example 5.1
90
Variable
Applications
FITS1
Data
80
70
60
50
1
10
20
30
40
50
60
Index
70
80
90
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
100
28
Example 5.2
• Dow Jones Index
Exemplo: Página 271
12000
Dow Jones
11000
10000
The process shows signs
of nonstationarity with
changing mean
and possibly variance.
9000
8000
1
8
16
24
32
40
48
Index
56
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
64
72
80
29
Example 5.2
Autocorrelation Function for Dow Jones
(with 5% significance limits for the autocorrelations)
1,0
0,8
Autocorrelation
0,6
0,4
0,2
0,0
-0,2
-0,4
-0,6
The slowly decreasing sample ACF and sample PACF with
significant value at lag 1, which is close to 1 confirm that
indeed the process can be deemed nonstationary.
-0,8
-1,0
2
4
6
8
10
12
14
16
18
20
Lag
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
30
Example 5.2
Partial Autocorrelation Function for Dow Jones
(with 5% significance limits for the partial autocorrelations)
1,0
Partial Autocorrelation
0,8
0,6
0,4
0,2
0,0
-0,2
-0,4
-0,6
-0,8
-1,0
One might argue that the significant sample PACF value at lag I
suggests that the AR( I) model might also fit the data well.
We will consider this interpretation first and fit an AR( I) model
to the Dow Jones Index data.
2
4
6
8
10
12
14
16
18
20
Lag
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
31
Example 5.2
The modified Box-Pierce test
suggests that there is no
autocorrelation left in the
residuals. This is also
confirmed by the sample ACF
and PACF plots of the
residuals
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
32
Autocorrelation Function for RESI1
Partial Autocorrelation Function for RESI1
(w ith 5% significance limits for the autocorrelations)
(w ith 5% significance limits for the partial autocorrelations)
1,0
1,0
0,8
0,8
0,6
0,6
0,4
0,4
P ar tial A utocor r elation
A utocor r elation
Example 5.2
0,2
0,0
-0,2
-0,4
0,2
0,0
-0,2
-0,4
-0,6
-0,6
-0,8
-0,8
-1,0
-1,0
2
4
6
8
10 12
Lag
14
16
18
20
2
4
6
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
8
10 12
Lag
14
16
18
20
33
Example 5.2
Residual Plots for Dow Jones
Normal Probability Plot
Versus Fits
99,9
1000
500
90
Residual
Percent
99
50
10
-1000
-500
0
Residual
500
1000
8000
15
Residual
Frequency
The only concern
in the residual plots in
Histogram
is20 in the changing variance observed in
the time series plot of the residuals.
10
5
0
-500
-1000
1
0,1
0
9000
10000
Fitted Value
11000
Versus Order
1000
500
0
-500
-1000
-1200
-800
-400
0
Residual
400
800
1
10
20
30 40 50 60
Observation Order
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
70
80
34
Example 5.2
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
35
Example 5.2
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
36
Example 5.2
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
37
Example 5.2
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
38
Example 5.3
90
Prediction
with AR(2)
Applications
80
70
60
50
1
10
20
30
40
50
60
Week
70
80
90
100
Exemplo pag 278
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
39
Example 5.3
Time Series Plot for Applications
(with forecasts and their 95% confidence limits)
90
Applications
80
70
60
50
1
10
20
30
40
50
60
Time
70
80
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
90
100
110
40
Example 5.3
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
41
Example 5.5
• U.S. Clothing Sales Data
17500
Exemplo: Página 283
The data obviously exhibit some
seasonality and upward linear trend. c
15000
Sales
12500
10000
7500
5000
jan-92 fev-93
abr-94
jun-95 ago-96 out-97 dez-98 fev-00
Date
abr-01 jun-02
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
ago-03
42
Example 5.5
Autocorrelation Function for Sales
(with 5% significance limits for the autocorrelations)
1,0
0,8
Autocorrelation
0,6
0,4
0,2
0,0
-0,2
-0,4
-0,6
The sample ACF and PACF indicate a monthly seasonality,
s = 12, as ACF values at lags 12, 24, 36 are significant and slowly
decreasing
-0,8
-1,0
1
5
10
15
20
25
30
35
Lag
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
43
Example 5.5
Autocorrelation Function for Sales
(with 5% significance limits for the autocorrelations)
1,0
0,8
Autocorrelation
0,6
0,4
0,2
0,0
-0,2
-0,4
-0,6
The sample ACF and PACF indicate a monthly seasonality,
s = 12, as ACF values at lags 12, 24, 36 are significant and slowly
decreasing
-0,8
-1,0
1
5
10
15
20
25
30
35
Lag
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
44
Example 5.5
Partial Autocorrelation Function for Sales
(with 5% significance limits for the partial autocorrelations)
1,0
Partial Autocorrelation
0,8
0,6
0,4
0,2
0,0
-0,2
-0,4
There is a significant PACF value at lag 12 that is close to 1. Moreover,
-0,6
the slowly decreasing ACF in general also indicates a
nonstationarity that can be remedied by taking the first difference. Hence
-1,0
we would now consider
-0,8
1
5
10
15
20
25
30
35
Lag
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
45
Example 5.5
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
46
Example 5.5
Partial Autocorrelation Function for Sales
(with 5% significance limits for the partial autocorrelations)
1,0
Partial Autocorrelation
0,8
0,6
0,4
0,2
0,0
-0,2
-0,4
There is a significant PACF value at lag 12 that is close to 1. Moreover,
-0,6
the slowly decreasing ACF in general also indicates a
nonstationarity that can be remedied by taking the first difference. Hence
-1,0
we would now consider
-0,8
1
5
10
15
20
25
30
35
Lag
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
47
Example 5.5
Figure shows that first difference
together with seasonal
differencing helps in terms of
stationarity and eliminating the
seasonality
1500
Comp Dif
1000
500
0
-500
-1000
jan-92 fev-93
abr-94
jun-95 ago-96 out-97 dez-98 fev-00 abr-01
Date
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
jun-02
ago-03
48
Example 5.5
Autocorrelation Function for Comp Dif
(with 5% significance limits for the autocorrelations)
1,0
0,8
Autocorrelation
0,6
0,4
0,2
0,0
-0,2
-0,4
-0,6
-0,8
-1,0
1
5
10
15
20
25
30
Lag
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
49
Example 5.5
Partial Autocorrelation Function for Comp Dif
(with 5% significance limits for the partial autocorrelations)
1,0
Partial Autocorrelation
0,8
0,6
0,4
0,2
0,0
-0,2
-0,4
The sample ACF with a significant value at lag 1 and the
sample PACF with exponentially decaying values at
the first 8 lags suggest that a nonseasonal MA( I)
model should be used.
-0,6
-0,8
-1,0
1
5
10
15
20
25
30
Lag
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
50
Example 5.5
The interpretation of the remaining seasonality is a
bit more difficult. For that we should focus on the
sample ACF and PACF values at lags 12. 24,
36, and so on. The sample ACF at lag 12 seems
to be significant and the sample PACF at lags
12, 24, 36 (albeit not significant) seems to be
alternating in sign. That suggests that a
seasonal MA(1) model can be used as well. Hence
an ARIMA (0, 1, 1) x (0, 1, 1) 12 model is used
to model the data, yt
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
51
Example 5.5
Both MA( 1) and seasonal MA( 1)
coefficient estimates are significant.
As we can see from the sample ACF
and PACF plots, while there are still some
small significant values, as indicated by the
modified Box-Pierce statistic, most of the
autocorrelation is now modeled out.
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
52
Autocorrelation Function for RESI1
Partial Autocorrelation Function for RESI1
(w ith 5% significance limits for the autocorrelations)
(w ith 5% significance limits for the partial autocorrelations)
1,0
1,0
0,8
0,8
0,6
0,6
0,4
0,4
P ar tial A utocor r elation
A utocor r elation
Example 5.5
0,2
0,0
-0,2
-0,4
-0,6
0,2
0,0
-0,2
-0,4
-0,6 and PACF plots, while there
As we can see from the sample ACF
are still some small significant values,
-0,8 as indicated by the modified
Box-Pierce statistic, most of the autocorrelation is now modeled out.
-0,8
-1,0
-1,0
1
5
10
15
20
Lag
25
30
1
5
10
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
15
20
Lag
25
30
53
Example 5.5
Normal Probability Plot
Versus Fits
99,9
99
500
Residual
Percent
90
50
10
0
-500
1
0,1
-1000
-500
0
Residual
500
-1000
1000
6000
9000
12000
Fitted Value
Histogram
500
15
Residual
Frequency
18000
Versus Order
20
10
5
0
15000
-900
-600
-300
0
Residual
300
600
0
-500
-1000
1 10 20
30
40
50 60 70 80 90 100 110 120 130 140
Observation Order
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
54
Example 5.5
Time Series Plot for Sales
(with forecasts and their 95% confidence limits)
20000
17500
Sales
15000
12500
10000
7500
5000
1
12
24
36
48
60
72
84
Time
96
108 120
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
132
144 156
55
Introduction
Exponential smoothing. The general assumption for these models was that
any time series data can be represented as the sum of two distinct
components: deterministic and stochastic (random). The former
(deterministic) is modeled as a function of time whereas for the latter
(stochastic) we assumed that some random noise that is added on the
deterministic signal generates the stochastic behavior of the time series.
One very important assumption is that the random noise is generated through
independent shocks to the process.
In practice, however, this assumption is often violated. That is, usually
successive observations show serial dependence. Under these
circumstances, forecasting methods based on exponential smoothing may be
inefficient and sometimes inappropriate because they do not take advantage
of the serial dependence in the observations in the most effective way.
To formally incorporate this dependent structure, we will explore a general class
of models called autoregressive integrated moving average models or ARIMA
models (also known as Box-Jenkins models).
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
56
Linear Models for Stationary Time Series
• A linear filter is defined as
Um conceito de
Processamento
de Sinais
is said to be
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
57
Stationarity
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
58
Some Examples
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
59
Stationary Time Series
• Many time series do not exhibit a
stationary behavior
• The stationarity is in fact a rarity in real life
• However it provides a foundation to build
upon since (as we will see later on) if the
time series in not stationary, its first
difference (yt-yt-1) will often be
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
60
Linear Filter
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
61
If Input is White Noise
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
62
Using the Backshift Operator
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
63
Wold’s Decomposition Theorem
• Any nondeterministic weakly stationary
time series can be written as an infinite
sum of weighted random shocks
(disturbances)

yt     i t i
i 0
where

2

 i 
i 0
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
64
How useful is this?
Well, not so much!!!
How can we come up with
“infinitely” many terms?
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
65
Maybe we should consider some
special cases:
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
66
Finite Order Moving Average Processes
(Ma(q))
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
67
Some Properties
• Expected Value
• Variance
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
68
Some Properties
• Autocovariance Function
• Autocorrelation Function (ACF)
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
69
Autocorrelation Function of MA(q)
• ACF of Ma(q) ”cuts off” after lag q
• This is very useful in the identification of
an MA(q) process
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
70
Example
Employ.mtw
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
71
Diferences
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
72
Autocorrelation
The graphs for the autocorrelation function (ACF) of
the ARIMA residuals include lines representing two
standard errors to either side of zero. Values that
extend beyond two standard errors are statistically
significant at approximately a = 0.05, and show
evidence that the model does not explain thel
autocorrelation in the data.
Because you did not specify
the lag length,
autocorrelation uses the
default length of n / 4 for a
series with less than or
equal to 240 observations.
Minitab generates an
autocorrelation function
(ACF) with approximate a =
0.05 critical bands for the
hypothesis that the
correlations are equal to
zero.
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
73
Autocorrelation
The ACF for these
data shows large
positive, significant
spikes at lags 1 and
2 with subsequent
positive
autocorrelations that
do not die off
quickly. This pattern
is typical of an
autoregressive
process.
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
74
Ljung-Box q statistic
Use to test whether a series of observations over time are random and independent. If
observations are not independent, one observation may be correlated with another
observation k time units later, a relationship called autocorrelation. Autocorrelation can
impair the accuracy of a time-based predictive model, such as time series plot, and lead to
misinterpretation of the data.
For example, an electronics company tracks monthly sales of batteries for five years. They
want to use the data to develop a time series model to help forecast future sales. However,
monthly sales may be affected by seasonal trends. For example, every year a rise in sales
occurs when people buy batteries for Christmas toys. Thus a monthly sales observation in
one year could be correlated with a monthly sales observations 12 months later (a lag of
12).
Before choosing their time series model, they can evaluate autocorrelation for the monthly
differences in sales. The Ljung-Box Q (LBQ) statistic tests the null hypothesis that
autocorrelations up to lag k equal zero (i.e., the data values are random and independent
up to a certain number of lags--in this case 12). If the LBQ is greater than a specified critical
value, autocorrelations for one or more lags may be significantly different from zero,
suggesting the values are not random and independent over time.
LBQ is also used to evaluate assumptions after fitting a time series model, such as ARIMA,
to ensure that the residuals are independent.
The Ljung-Box is a Portmanteau test and is a modified version of the Box-Pierce chi-square
statistic.
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
75
You can use the Ljung-Box Q
(LBQ) statistic to test the null
hypothesis that the autocorrelations
for all lags up to lag k equal zero.
Let's test that all autocorrelations
up to a lag of 6 are zero. The LBQ
statistic is 56.03.
Ho: Autocorrelation (lag<6) = 0
Variable
CumProb is
created
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
76
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
77
In this example, the p-value is
0.000000, which means the pvalue is less than 0.0000005. The
very small p-value implies that
one or more of the
autocorrelations up to lag 6 can
be judged as significantly
different from zero at any
reasonable a level.
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
78
Partial autocorrelation computes and plots the
partial autocorrelations of a time series. Partial
autocorrelations, like autocorrelations, are
correlations between sets of ordered data
pairs of a time series. As with partial
correlations in the regression case, partial
autocorrelations measure the strength of
relationship with other terms being accounted for.
The partial autocorrelation at a lag of k is the
correlation between residuals at time t from an
autoregressive model and observations at lag k
with terms for all intervening lags present in the
autoregressive model. The plot of partial
autocorrelations is called the partial
autocorrelation function or PACF. View the PACF
to guide your choice of terms to include in an
ARIMA model.
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
79
You obtain a partial
autocorrelation function
(PACF) of the food industry
employment data, after
taking a difference of lag 12,
in order to help determine a
likely ARIMA model.
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
80
Minitab generates a partial
autocorrelation function with
critical bands at
approximately a = 0.05 for
the hypothesis that the
correlations are equal to
zero.
In the food data example,
there is a single large spike
of 0.7 at lag 1, which is
typical of an autoregressive
process of order one. There
is also a significant spike at
lag 9, but you have no
evidence of a nonrandom
process occurring there.
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
81
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
82
Sample ACF
• Will not be equal to zero after lag q for
an MA(q)
• But it will be small
• For the same size of N, this can be
tested using the limits:
2 N
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
83
First-Order Moving Average Process
MA(1)
for which autocovariance and autocorrelation
functions are given as
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
84
Some Examples
Note, the behavior
of sample ACF
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
85
Second-Order Moving Average Process
MA(2)
for which autocovariance and autocorrelation
functions are given as
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
86
An Example
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
87
Finite Order Autoregressive Processes
(AR(p))
• MA(q) processes take into account
disturbances up to q lags in the past
• What if all past disturbances have some
lingering effects? Back to square one?
• We may be able to come up with some
special cases though
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
88
A very special case
• What if we let
 j 
j
for   1
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
89
Decomposition


yt      t i     t    t i
i
i 0
i
i 1
and


i 0
i 1
yt 1     i t 1i     t   i t i 1
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
90
Combining the two equations






i
i
yt  yt 1        t i          t i 1 
i 0
i 0

 

       t




yt    yt 1  at
This is an AR(1) model
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
91
First-Order Autoregressive Process
(AR(1))
yt    yt 1  at
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
92
Properties
• Expected Value
• Autocovariance Function
• Autocorrelation Function
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
93
Some Examples
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
94
Second-Order Autoregressive Process
(AR(2))
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
95
Conditions for Stationarity
yt  1 yt 1  2 yt  2     t
1   B   B y
2
1
2
t
   t
  B  yt     t
yt    B      B   t


 
1

    B  t
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
1
 B 
96
AR(2) is stationary if …

we can writeas  B    i B i
Since  B  B   1
i 0
1  1B  2 B  0  1B  2 B    1
2
 0   1  1 0 B   2  1 1  2 0 B  
2
   j  1 j 1  2 j  2 B    1
j
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
97
AR(2) is stationary if …
0 1
 1  1 0   0
 2  1 1  2 0   0


j
 1 j 1  2 j  2   0
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
j2
98
AR(2)
• Hence {j} satisfy the 2nd order linear difference
equation. So the i can be expressed as the
solution to this equation in terms of the 2 roots
m1 and m2 of the associated polynomial
m  1m  2  0
2
• If the roots m1 and m2 satisfy
m1 , m2  1 then
e.g. m , m
1
2


j 0
j

are real then  j  c1m1j  c2m2j
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br

99
AR(2) is stationary if the roots m1 and m2
of
m  1m  2  0 are both less than
2
one in absolute value
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
100
ACF of a stationary AR(2)
 k   Cov yt , yt k 
 Cov1 yt 1  2 yt  2     t , yt k 
 1Cov yt 1 , yt k   2Cov yt  2 , yt  k   Cov t , yt  k 
 2 if k  0
 1 k  1  2 k  2  
 0 if k  0
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
101
ACF of a stationary AR(2)
 0  1 1  2 2   2
 k   1 k  1  2 k  2
k  1,2, 
Yule-Walker
Equations
 k   1  k  1  2  k  2 k  1,2,
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
102
ACF of a stationary AR(2)
• Hence ACF satisfies the 2nd order linear
difference equation. So the (k) can be
expressed as the solution to this equation
in terms of the 2 roots m1 and m2 of the
associated polynomial
m  1m  2  0
2
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
103
ACF of a stationary AR(2)
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
104
Some Examples
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
105
AR(p)
p
yt   i yt i     t
i 1
  B  yt     t
where  B   1  1 B  2 B     p B
2
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
p
106
AR(p) is Stationary
• If the roots of
m  1m
p
p 1
 2m
p 2
   p  0
are less than one in absolute
value.
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
107
Infinite MA representation
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
108
Expected Value of an AR(p)
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
109
Autocovariance Function of an AR(p)
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
110
Autocorrelation Function of an AR(p)
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
111
ACF of AR(p)
p
 k   i  k  i 
k  1,2,
i 1
In general ACF of AR(p) can be a mixture
of exponential decay and damped
sinusoidal behavior depending on the solution
to the corresponding Yule-Walker equations.
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
112
ACF of AR(p)
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
113
ACF for AR(p) and MA(q)
• ACF of MA(q) “cuts off” after q
   k  1 k 1     q k q

 y k    1  12   22     q2
 0

k  0,, q
k q
• ACF of AR(p) can be a mixture of
exponential decay and damped
sinusoidal
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
114
So how are we going to
determine p in the AR(p)
model?
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
115
Partial Correlation
• Suppose X, Y and Z are random variables. We
define the notion of partial correlation between
X and Y adjusting for Z.
• First consider simple linear regression of X on Z
CovZ , X 
ˆ
X  a1  b1Z where b1 
VarZ 
• Also the linear regression of Y on Z
CovZ , Y 
ˆ
Y  a2  b2 Z where b2 
Var Z 
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
116
Partial Correlation
• Now consider the errors
X *  X  Xˆ  X  a1  b1Z 
Y *  Y  Yˆ  Y  a  b Z 
2
2
• Then the partial correlation between X and Y,
adjusting for Z, is



corr X , Y  corr X  Xˆ , Y  Yˆ
*
*
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br

117
Partial Autocorrelation Function (PACF)
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
118
Partial Autocorrelation Function (PACF)
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
119
Partial Autocorrelation Function (PACF)
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
120
Partial Autocorrelation Function (PACF)
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
121
Sample Partial Autocorrelation Function
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
122
Some Examples
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
123
PACF
• For an AR(p) process, PACF cuts
off after lag p.
• For an MA(q) process, PACF has
an exponential decay and/or a
damped sinusoid form
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
124
Invertibility of a MA Process
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
125
Invertibility of a MA Process
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
126
Invertibility of a MA Process
We have
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
127
The ACF and PACF do have very distinct
and indicative properties for MA and AR
models. Therefore in model identification it
is strongly recommended to use both the
sample ACF and PACF simultaneously
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
128
Mixed Autoregressive-Moving
Average (ARMA(p,q)) Process
Byt    B t
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
129
Stationarity of ARMA(p,q)
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
130
Invertibility of ARMA(p,q)
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
131
ACF and PACF of an ARMA(p,q)
• Both ACF and PACF of an ARMA(p,q) can
be a mixture of exponential decay and
damped sinusoids depending on the roots
of the AR operator.
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
132
ARMA Models
• For ARMA models, except for possible special
cases, neither ACF nor PACF has distinctive
features that would allow “easy identification”
• For this reason, there have been many
additional sample functions considered to help
with identification problem:
–
–
–
–
Extended sample ACF (ESACF)
Generalized sample PACF (GPACF)
Inverse ACF
Use of “canonical correlations”
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
133
Some Examples
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
134
Review of Main Characteristics of ACF
and PACF
ACF
MAq 
AR p 
ARMA p, q 
cutsof afterlag q
exp.decay and/or
dampedsinusoid
exp.decay and/or
dampedsinusoid
PACF
exp.decay and/or
dampedsinusoid
cutsof afterlag p
exp.decay and/or
dampedsinusoid
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
135
Review of Main Characteristics of Sample
ACF and PACF
Sample ACF
E ˆ   0 for k  q
MAq 
1
ˆ


Var   for k  q
T
AR p 
exp.decay and/or
ARMA p, q 
exp.decay and/or
dampedsinusoid
dampedsinusoid
Sample PACF
exp.decay and/or
dampedsinusoid
E ˆ   0 for k  p
Var ˆ  
1
for k  p
T
exp.decay and/or
dampedsinusoid
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
136
Some Examples
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
137
Some Examples
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
138
Some Examples
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
139
Some Examples
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
140
Some Examples
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
141
ARIMA Models
• Process {yt} is ARIMA(p,d,q), if the dth
order differences, wt=(1-B)dyt, form a
stationary ARMA(p,q) process:
Bwt    B t
• Thus {yt} satisfies
B1  B yt    B t
d
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
142
Some Examples
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
143
Some Examples
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
144
Model Building
• Given T observations from a process,
want to obtain a model that
adequately represents the main
features of the time series data.
Model can be used for purposes of
forecasting, control, …
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
145
3-Stage Procedure
• STAGE 1: Model Specification or
Identification
– Consider issue of nonstationarity vs.
stationarity of series. Use procedures such
as differencing to obtain a stationary series;
say wt=(1-B)dyt
• Examine sample ACF and PACF of wt and use
features of these functions to identify an
appropriate ARMA model. The specification is
“tentative”
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
146
Review of Main Characteristics of ACF
and PACF
ACF
MAq 
cutsof afterlag q
AR p 
ARMA p, q 
exp.decay and/or
dampedsinusoid
exp.decay and/or
dampedsinusoid
PACF
exp.decay and/or
dampedsinusoid
cutsof afterlag p
exp.decay and/or
dampedsinusoid
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
147
Review of Main Characteristics of Sample
ACF and PACF
Sample ACF
E ˆ   0 for k  q
MAq 
1
Var ˆ   for k  q
T
AR p 
exp.decay and/or
ARMA p, q 
exp.decay and/or
dampedsinusoid
dampedsinusoid
Sample PACF
exp.decay and/or
dampedsinusoid
E ˆ   0 for k  p
1
Var ˆ   for k  p
T
exp.decay and/or
dampedsinusoid
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
148
ARMA Models
• For ARMA models, except for possible special
cases, neither ACF nor PACF has distinctive
features that would allow “easy identification”
• For this reason, there have been many
additional sample functions considered to help
with identification problem:
–
–
–
–
Extended sample ACF (ESACF)
Generalized sample PACF (GPACF)
Inverse ACF
Use of “canonical correlations”
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
149
3-Stage Procedure
• STAGE 2: Estimation of Parameters in
Tentatively Specified Model
– Method of moments
– Least Squares
– Maximum Likelihood
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
150
3-Stage Procedure
• STAGE 3: Model Checking
– Based on examining features of residuals
p
q
i 1
i 1
ˆt  yt  ˆi yt i  ˆ  ˆiˆt i
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
151
3-Stage Procedure
• STAGE 3: If the specified model is
appropriate order p, q; then we
expect the residuals behave similar
to the “true” white noise t.
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
152
Example 5.1
• Weekly total number of loan applications
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
153
Example 5.1
154
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
154
Example 5.1
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
155
Example 5.1
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
156
Example 5.1
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
157
Example 5.1
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
158
Example 5.1
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
159
Example 5.2
• Dow Jones Index
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
160
Example 5.2
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
161
Example 5.2
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
162
Example 5.2
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
163
Example 5.2
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
164
Example 5.2
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
165
Example 5.2
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
166
Example 5.2
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
167
Example 5.2
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
168
Example 5.2
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
169
Forecasting ARIMA Processes
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
170
Forecasting ARIMA Processes
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
171
The “best” forecast
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
172
Forecast Error
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
173
Prediction Intervals
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
174
Two Issues
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
175
Illustration Using ARIMA(1,1,1)
• ARIMA(1,1,1) process is given as
1  B1  ByT   1 BT 
• Two commonly used approaches
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
176
Approach 1
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
177
Approach 2
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
178
Example 5.3
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
179
Seasonal Processes
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
180
Seasonal Processes
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
181
Seasonal Processes
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
182
Seasonal Processes
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
183
Example 5.4
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
184
Example 5.5
• U.S. Clothing Sales Data
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
185
Example 5.5
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
186
Example 5.5
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
187
Example 5.5
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
188
Example 5.5
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
189
Example 5.5
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
190
Example 5.5
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
191
Example 5.5
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
192
Example 5.5
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
193
Example 5.5
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
194
Use ARIMA to model time series behavior
and to generate forecasts. ARIMA fits a
Box-Jenkins ARIMA model to a time series.
ARIMA stands for Autoregressive
Integrated Moving Average with each term
representing steps taken in the model
construction until only random noise
remains. ARIMA modeling differs from the
other time series methods in the fact that
ARIMA modeling uses correlational
techniques. ARIMA can be used to model
patterns that may not be visible in plotted
data.
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
195
The ACF and PACF of the food employment data suggest an
autoregressive model of order 1, or AR(1), after taking a
difference of order 12. You fit that model here, examine
diagnostic plots, and examine the goodness of fit. To take a
seasonal difference of order 12, you specify the seasonal
period to be 12, and the order of the difference to be 1.
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
196
1
Model is specified by the usual notation (pdq) x (PDQ) S:
(pdq) is for a nonseasonal model; (PDQ) for a seasonal, and S is the seasonality.
2 At least one of the p, P, q, or Q parameters must be non-zero, and none may exceed five.
3 The maximum number of parameters you can estimate is ten.
4 At least three data points must remain after differencing. That is, S * D + d + 2 must be less than the number of
points, where S is the length of a season.
5 The maximum "back order" for the model is 100. In practice, this condition is always satisfied if S * D + d + p +
P + q + Q is at most 100.
6 The ARIMA model normally includes a constant term only if there is no differencing (that is, d = D = 0).
7 Missing observations are only allowed at the beginning or the end of a series, not in the middle.
8 The seasonal component of this model is multiplicative, and thus is appropriate when the amount of cyclical
variation is proportional to the mean.
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
197
The ARIMA model converged after nine iterations. The
AR(1) parameter had a t-value of 7.42. As a rule of thumb,
you can consider values over two as indicating that the
associated parameter can be judged as significantly
different from zero. The MSE (1.1095) can be used to
compare fits of different ARIMA models.
The Ljung-Box statistics give nonsignificant p-values ,
indicating that the residuals appeared to uncorrelated. The
ACF and PACF of the residuals corroborate this. You
assume that the spikes in the ACF and PACF at lag 9 are
the result of random events
The coefficients are estimated using an iterative algorithm that
calculates least squares estimates. At each iteration, the back
forecasts are computed and SSE is calculated.
Back forecasts are calculated using the specified model and
the current iteration's parameter estimates
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
198
Box and Jenkins [2] present an interactive approach for fitting
ARIMA models to time series. This iterative approach involves
identifying the model, estimating the parameters, checking model
adequacy, and forecasting, if desired. The model identification step
generally requires judgment from the analyst.
1 First, decide if the data are stationary. That is, do the data
possess constant mean and variance .
· Examine a time series plot to see if a transformation is required
to give constant variance.
· Examine the ACF to see if large autocorrelations do not die out,
indicating that differencing may be required to give a constant
mean.
A seasonal pattern that repeats every kth time interval suggests
taking the kth difference to remove a portion of the pattern. Most
series should not require more than two difference operations or
orders. Be careful not to overdifference. If spikes in the ACF die out
rapidly, there is no need for further differencing. A sign of an
overdifferenced series is the first autocorrelation close to -0.5 and
small values elsewhere.
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
199
2 Next, examine the ACF and PACF of your stationary
data in order to identify what autoregressive or moving
average models terms are suggested.
· An ACF with large spikes at initial lags that decay to
zero or a PACF with a large spike at the first and
possibly at the second lag indicates an autoregressive
process.
· An ACF with a large spike at the first and possibly at
the second lag and a PACF with large spikes at initial
lags that decay to zero indicates a moving average
process.
· The ACF and the PACF both exhibiting large spikes
that gradually die out indicates that both autoregressive
and moving averages processes are present.
For most data, no more than two autoregressive
parameters or two moving average parameters are
required in ARIMA models.
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
200
3 Once you have identified one or more likely models, you
are ready to use the ARIMA procedure.
· Fit the likely models and examine the significance of
parameters and select one model that gives the best fit.
· Check that the ACF and PACF of residuals indicate a
random process, signified when there are no large spikes.
You can easily obtain an ACF and a PACF of residual using
ARIMA's Graphs subdialog box. If large spikes remain,
consider changing the model.
· You may perform several iterations in finding the best
model. When you are satisfied with the fit, go ahead and
make forecasts.
The ARIMA algorithm will perform up to 25 iterations to fit a
given model. If the solution does not converge, store the
estimated parameters and use them as starting values for a
second fit. You can store the estimated parameters and use
them as starting values for a subsequent fit as often as
necessary.
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
201
The graphs for the ACF and PACF of the ARIMA residuals
include lines representing two standard errors to either
side of zero. Values that extend beyond two standard
errors are statistically significant at approximately a = 0.05,
and show evidence that the model has not explained all
autocorrelation in the data.
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
202
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
203
The AR(1) model appears to fit
well so you use it to forecast
employment.
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
204
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
205
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
206
The ARIMA algorithm is based on the fitting routine
in the TSERIES package written by Professor
William Q. Meeker, Jr., of Iowa State University.
• W.Q. Meeker, Jr. (1977). "TSERIES-A Useroriented Computer Program for Identifying, Fitting
and Forecasting ARIMA Time Series Models," ASA
1977 Proceedings of the Statistical Computing
Section.
• W.Q. Meeker, Jr. (1977). TSERIES User's
Manual, Statistical Laboratory, Iowa State
University.
Previsão | Pedro Paulo Balestrassi | www.pedro.unifei.edu.br
207