Econometrics I
Professor William Greene
Stern School of Business
Department of Economics
4-1/22
Part 4: Partial Regression and Correlation
Econometrics I
Part 4 – Partial Regression
and Correlation
4-2/22
Part 4: Partial Regression and Correlation
Frisch-Waugh (1933) Theorem
Context: Model contains two sets of variables:
X = [ (1,time) : (other variables)]
= [X1 X2]
Regression model:
y = X11 + X22 +  (population)
= X1b1 + X2b2 + e (sample)
Problem: Algebraic expression for the second set
of least squares coefficients, b2
4-3/22
Part 4: Partial Regression and Correlation
Partitioned Solution
Direct manipulation of normal equations produces
( XX )b = X y
 X1 X1 X1 X 2 
 X1 y 
X = [X1 , X 2 ] so XX = 
 and Xy =  X y 


X
X
X
X
2 2
 2 1
 2 
 X X X1 X 2   b1   X 1 y 
(XX)b =  1 1
=




 X2 X1 X2 X 2   b2   X2 y 
X1 X1b1  X1 X 2b2  X 1 y
X2 X1b1  X 2 X 2b2  X 2 y ==> X 2 X 2b2  X 2 y - X 2 X 1b1
= X2 (y - X1b1 )
4-4/22
Part 4: Partial Regression and Correlation
Partitioned Solution
Direct manipulation of normal equations produces
b2 = (X2X2)-1X2(y - X1b1)
What is this? Regression of (y - X1b1) on X2
If we knew b1, this is the solution for b2.
Important result (perhaps not fundamental). Note
the result if X2X1 = 0.
4-5/22
Part 4: Partial Regression and Correlation
Partitioned Inverse
Use of the partitioned inverse result
produces a fundamental result: What is
the southeast element in the inverse of
the moment matrix?
-1
 X1'X1
 X 'X
 2 1
4-6/22
X1'X2 

X2'X2 
Part 4: Partial Regression and Correlation
Partitioned Inverse
The algebraic result is:
[ ]-1(2,2) = {[X2’X2] - X2’X1(X1’X1)-1X1’X2}-1
= [X2’(I - X1(X1’X1)-1X1’)X2]-1
= [X2’M1X2]-1

Note the appearance of an “M” matrix.
How do we interpret this result?
4-7/22
Part 4: Partial Regression and Correlation
Frisch-Waugh Result
Continuing the algebraic manipulation:
b2 = [X2’M1X2]-1[X2’M1y].
This is Frisch and Waugh’s famous result - the “double residual
regression.”
How do we interpret this? A regression of residuals on residuals.
Regrido Y em X1 – pego o resíduo que sobra e regrido esta
parte no que sobra de X2 regredido em X1 (resíduo).
Tudo de Y que não foi explicado por X1 mas pode ser
explicado por X2.
Tudo de X2 que não está sendo explicado por X1
4-8/22
Part 4: Partial Regression and Correlation
4-9/22
Part 4: Partial Regression and Correlation
Important Implications

M1 is idempotent. (This is a very useful result.)
(i.e., X2M1’M1y = X2M1y)

(Orthogonal regression) Suppose X1 and X2 are
orthogonal; X1X2 = 0. What is M1X2?
4-10/22
Part 4: Partial Regression and Correlation
Applying Frisch-Waugh
Using gasoline data from Notes 3.
X = [1, year, PG, Y], y = G as before.
Full least squares regression of y on X.
4-11/22
Part 4: Partial Regression and Correlation
Detrending the Variables - Pg
4-12/22
Part 4: Partial Regression and Correlation
Regression of Detrended G on Detrended
Pg and Detrended Y
4-13/22
Part 4: Partial Regression and Correlation
Partial Regression
Important terms in this context:
Partialing out the effect of X1.
Netting out the effect …
“Partial regression coefficients.”
To continue belaboring the point: Note the interpretation of partial
regression as “net of the effect of …”
Now, follow this through for the case in which X1 is just a constant term,
column of ones.
What are the residuals in a regression on a constant. What is M1?
Note that this produces the result that we can do linear regression on
data in mean deviation form.
As inclinações numa regressão múltipla que contém o termo
constante são obtidas por transformar os dados como desvios em
relação à média e regredir a variável y na forma de desvio em
todas variáveis explicativas também como desvios em relação às
suas médias.
'Partial regression coefficients' are the same as 'multiple regression
coefficients.' It follows from the Frisch-Waugh theorem.
4-14/22
Part 4: Partial Regression and Correlation
Partial Correlation
Working definition. Correlation between sets of residuals.
Some results on computation: Based on the M matrices.
Some important considerations:
Partial correlations and coefficients can have signs and
magnitudes that differ greatly from gross correlations and
simple regression coefficients.
Compare the simple (gross) correlation of G and PG with the partial
correlation, net of the time effect.
CALC;list;Cor(g,pg)$
Result = .7696572
4. 50
4. 00
3. 50
CALC;list;cor(gstar,pgstar)$
Result = -.6589938
PG
3. 00
2. 50
2. 00
1. 50
1. 00
. 50
70
80
90
100
110
120
G
4-15/22
Part 4: Partial Regression and Correlation
Partial Correlation
4-16/22
Part 4: Partial Regression and Correlation
THE Application of Frisch-Waugh
The Fixed Effects Model
A regression model with a dummy variable for
each individual in the sample, each observed Ti times.
yi = Xi + diαi + εi, for each individual
N columns
 y1 
 
 y2  
 
 
 yN 
 X1
X
 2


 X N
d1
0
0
0
d2
0
0
0
0
0
0   β 
ε
  α 

dN 
N may be thousands. I.e., the
regression has thousands of
variables (coefficients).
β
= [X, D]    ε
 α
= Zδ  ε
4-17/22
Part 4: Partial Regression and Correlation
Estimating the Fixed Effects Model
The FEM is a linear regression model but
with many independent variables
1
 b   XX XD   Xy 
  





 a  D X D D  D y 
Using the Frisch-Waugh theorem
b
4-18/22
=[XMD X]1 XMD y 
Part 4: Partial Regression and Correlation
Fixed Effects Estimator (cont.)
M1D 0

2
0
M
D
MD  


0
 0
0 

0 
(The dummy variables are orthogonal)

N
MD 
MDi  I Ti  di (didi ) 1 d = I Ti  (1/Ti )did
1 - T1i - T1i
 1
1
 - Ti 1 - Ti
=
...
 ...
1
 - T1
T
 i
i
4-19/22
...
- T1i 

1
... - Ti 

... ... 
... 1 - T1i 
Part 4: Partial Regression and Correlation
‘Within’ Transformations
XMD X = Ni=1 XiMDi X i ,
XMD y = Ni=1 XiMDi y i ,


XM y 
X iMDi X i
i
i
D
i k
k,l
Ti
 t=1 (x it,k -x i.,k )(x it,l -x i.,l )
T
i
 t=1
(x it,k -x i.,k )(y it -y i. )
Least Squares Dummy Variable Estimator

b is obtained by ‘within’ groups least squares
(group mean deviations)
4-20/22
Part 4: Partial Regression and Correlation