6.2. ACF AND PACF OF ARMA(P,Q)

109

6.2 ACF and PACF of ARMA(p,q)
6.2.1 ACF of ARMA(p,q)
In Section 4.6 we have derived the ACF for ARMA(1,1) process. We have used the linear process representation and the fact that
∞

γ(τ ) = σ

2

ψj ψj+τ . j=0 We have calculated the coefficients ψj from the relation ψ(B) =

θ(B)
,
φ(B)

which (in case of ARMA(1,1)) gives the values ψj = φj−1(θ1 + φ1 ).
1
This allows us to calculate the ACF of the process ρ(τ ) =

γ(τ )
.
γ(0)

Another way of finding the coefficients ψ is using the homogeneous difference equations. However, we may obtain such equation directly in terms of γ(τ ) or ρ(τ ).
For ARMA(1,1)
Xt − φXt−1 = Zt + θZt−1 we can write γ(τ ) = cov(Xt+τ , Xt )
= E(Xt+τ Xt )
= E[(φXt+τ −1 + Zt+τ + θZt+τ −1 )Xt ]
= E[φXt+τ −1 Xt + Zt+τ Xt + θZt+τ −1 Xt ]
= φ E[Xt+τ −1 Xt ] + E[Zt+τ Xt ] + θ E[Zt+τ −1 Xt ]
Here we consider a causal ARMA(1,1) process, hence
∞

Xt =

ψj Zt−j . j=0 CHAPTER 6. ARMA MODELS

110
This gives
∞

E[Zt+τ Xt ] = E[Zt+τ

ψj Zt−j ] j=0 ∞

=

ψj E[Zt+τ Zt−j ] j=0 =

ψ0 σ 2 for τ = 0,
0
for τ ≥ 1.

Also,
∞

E[Zt+τ −1 Xt ] = E[Zt+τ −1

ψj Zt−j ] j=0 ∞

=

ψj E[Zt+τ −1 Zt−j ] j=0 
 ψ1 σ 2 for τ = 0, ψ0 σ 2 for τ = 1
=

0
for τ ≥ 2.

Furthermore,

ψ0 = 1 ψ1 = φ + θ.
Putting all these together we obtain

γ(τ ) = φ E[Xt+τ −1 Xt ] + E[Zt+τ Xt ] + θ E[Zt+τ −1 Xt ]

 φγ(1) + σ 2 (1 + φθ + θ2 ) for τ = 0, φγ(0) + σ 2 θ for τ = 1,
=
 φγ(τ − 1) for τ ≥ 2.

The ACVF is in fact given here in the form of a homogeneous difference equation of order 1 with initial conditions specifying γ(0) and γ(1). Namely, we have γ(τ ) − φγ(τ − 1) = 0

(6.11)

and the initial conditions are γ(0) = φγ(1) + σ 2 (1 + φθ + θ2 ) γ(1) = φγ(0) + σ 2 θ
Note that the equation (6.11) γ(τ ) = φγ(τ − 1)

(6.12)

6.2. ACF AND PACF OF ARMA(P,Q)

111

has an iterative form and we can write γ(2) = φγ(1) γ(3) = φγ(2) = φ2 γ(1) γ(4) = φγ(3) = φ3 γ(1)
...
γ(τ ) = φτ −1 γ(1)
The polynomial associated with the equation (6.11) is
1 − φz = 0 with root z0 =

1
.
φ

So we can write
−1
γ(τ ) = (z0 )τ −1 γ(1).

This depends only on the root of the associated polynomial and on the initial conditions. Solving (6.12) for γ(0) and γ(1) we obtain γ(0) = σ 2 and γ(1) = σ 2

1 + 2θφ + θ2
1 − φ2

(1 + θφ)(φ + θ)
1 − φ2

This gives us γ(τ ) = σ 2

(1 + θφ)(φ + θ) τ −1 φ , for τ ≥ 1.
1 − φ2

Finally dividing by γ(0) we get the ACF, which is the same as the one derived in
Section 4.6, that is ρ(τ ) =

(1 + θφ)(φ + θ) τ −1 φ , for τ ≥ 1.
1 + 2θφ + θ2

ACF for ARMA(p,q)
Assume that the model φ(B)Xt = θ(B)Zt

(6.13)

CHAPTER 6. ARMA MODELS

112

ACF

x

1.0

10

0.8

0.6

5
0.4

0
0.2

0.0

-5

30

80

130

180

t

0

10

20

30

40

Figure 6.1: ARMA(1,1) simulated process xt − 0.9xt−1 = zt + 0.5zt−1 , sample
ACF and the theoretical ACF of this process. is causal, that is the roots of φ(B) are outside the unit circle. Then we can write
Xt = ψ(B)Zt , where ∞

ψj B j

ψ(B) = j=0 and it follows immediately that E(Xt ) = 0.
As in the example for ARMA(1,1), we can obtain a homogeneous differential equation in terms of γ(τ ) with some initial conditions. Namely γ(τ ) = cov(Xt+τ , Xt ) p =E

q

φj Xt+τ −j + j=1 θj Zt+τ −j

p

θj E[Zt+τ −j Xt ]

φj E[Xt+τ −j Xt ] +

= j=1 p

j=0 q φj γ(τ − j) + σ 2

= j=1 Xt

j=0 q θj ψj−τ j=τ Here, as before, we used the linear representation of Xt , the fact that Zt+i and Xt are uncorrelated for i > 0, ψi = 0 for i < 0 and that θ0 = 1.
This gives the general homogeneous difference equation for γ(τ ), γ(τ ) − φ1 γ(τ − 1) − . . . − φp γ(τ − p) = 0

for τ ≥ max(p, q + 1),

(6.14)

50

τ

6.2. ACF AND PACF OF ARMA(P,Q)

113

with initial conditions γ(τ )−φ1 γ(τ −1)−. . .−φp γ(τ −p) = σ 2 (θτ ψ0 +θτ +1 ψ1 +. . .+θq ψq−τ ) (6.15) for 0 ≤ τ < max(p, q + 1).
Example 6.4. ACF of an AR(2) process
Let
Xt − φ1 Xt−1 − φ2 Xt−2 = Zt be a causal AR(2) process. From (6.14) we have γ(τ ) − φ1 γ(τ − 1) − φ2 γ(τ − 2) = 0 for τ ≥ 2 with initial conditions γ(0) − φ1 γ(−1) − φ2 γ(−2) = σ 2 γ(1) − φ1 γ(0) − φ2 γ(−1) = 0
It is convenient to write these equations in terms of the autocorrelation function ρ(τ ). Dividing them by γ(0) we obtain

 ρ(τ ) − φ1 ρ(τ − 1) − φ2 ρ(τ − 2) = 0, for τ ≥ 2



ρ(0) = 1
(6.16)
 φ1 
 ρ(1) =

1 − φ2
We know that a general solution to a second order difference equation is
−τ
−τ ρ(τ ) = c1 z1 + c2 z2

where z1 and z2 are the roots of the associated polynomial φ(z) = 1 − φ1 z − φ2 z 2 , and c1 and c2 can be found from the initial conditions.
Take φ1 = 0.7 and φ2 = −0.1, that is the AR(2) process is
Xt − 0.7Xt−1 + 0.1Xt−2 = Zt .
It is a causal process as the coefficients lie in the admissible parameter space.
Also, the roots of the associated polynomial φ(z) = 1 − 0.7z + 0.1z 2

CHAPTER 6. ARMA MODELS

114

ACF

x
5

0.8

3

1
0.4

0
-1

0.0

-3

30

80

130

180

t

0

10

20

30

40

Figure 6.2: AR(2) simulated process xt − 0.7xt−1 + 0.1xt−2 = zt , sample ACF and the theoretical ACF of this process. are z1 = 2 and z2 = 5, i.e., they are outside the unit circle. The initial conditions are 
 ρ(0) = 1
7
0.7
 ρ(1) =
=
1 + 0.1
11
They give the set of equations for c1 and c2 , namely

 c1 + c2 = 1
 1 c1 + 1 c2 = 7
2
5
11
These give

16
5
, c2 = −
11
11 and finally we obtain the ACF for this AR(2) process c1 =

ρ(τ ) =

16 −τ
5
24−τ − 51−τ
2 − 5−τ =
.
11
11
11

Simulated AR(2) process, its sample ACF and the theoretical ACF are shown in
Figure 6.2. As we can see, the theoretical ACF decreases quickly towards zero, but it never attains zero, we say it tails off.

50

τ