Calibração de Câmera
Como determinar o modelo e os
parâmetros que transformam a
radiância da cena 3D numa
imagem digital
É o processo inverso da
camera de rendering
Real e virtual com uma mesma camera
Novos pontos de vista (novos elementos)
Extração de feições
min  | pi  f K ,R,T ( Pi ) |
2
K , R,T
Extração de feições
Calibração de camera
min | pi  f ( Pi ) |2
Matrizes de posicionamento
Matrizes de projeção
“CAMERAS” DE VISÃO E
DO OPENGL
Mudança de base (revisão)
p  uu  vv  ww
p  xi  yj  zk
u  ui i  u j j  uk k
v  vi i  v j j  vk k
w  wi i  wj j  wk k
p  uui i  u j j  uk k  vvi i  v j j  vk k  wwi i  wj j  wk k
p  uui  vvi  wwi i  uuj  vv j  wwj j  uuk  vvk  wwk k
y
x
 x   ui
  
 y   u j
 z  u
   k
vi
vj
vk
wi  u 
 

wj  v 

wk  w 
z
 u   ui
  
 v    vi
 w  w
   i
uj
vj
wj
uk  x 
 
vk  y 
wk  z 
Visão: matriz de parâmetros extrínsicos
Pc  Pw  T
Pw c
 X c   i w  i c 
 
  Yc    i w  jc 

 Z  i  k 
 c  w c
 X c    i w  i c 
  
Pc   Yc     i w  jc 
 Z   i  k 
 c   w c
 X c   m11
 
Pc   Yc   m21

 Z  m
 c   31
 X c  ro
 
Pc   Yc    r3

 Z  r
 c 6
 X 
m14   w 
Y
m24   w 
  Z 
m32 m33 m34   w 
 1 
 X 
r1 r2 t x   w 
Y
r4 r5 t y   w 
  Z 
r7 r8 t z   w 
 1 
m12
m22
yc
jw  i c  k w  i c   X w 


jw  jc  k w  jc   Yw 
jw  k c  k w  k c  Z w 
jw  i c  k w  i c  
jw  jc  k w  jc  
jw  k c  k w  k c 
zc
xc
 X 
 t x   w 
   Yw 

 t y  

 t   Z w 
 z   1 


pc t
zw
m13
m23
Pc  R tP
xw
pw
yw
yim y’
 x   s x ( xim  ox ) 

p     

s
(
y

o
)

y
   y im y 
sx
p'
sy
3
oy
2
o
1
0
0
1 2
ox
x’
3
4 5
xim
Projeção cônica simples
X
 
P  Y 
Z
 
y’
p
z'
x’
 x' 
X
  n  
p   y'  
Y 
 z'   Z  Z 
 
 
n
X
x'
n
=
X
-Z
Y
y'
x'
x' = n X
-Z
y'
n
=
Y
-Z
-Z
n
n
z'
x'
y'
z'
y' = n Y
-Z
X


s
(
x

o
)

x


 
x im
x
Z
  f 
p     



Y
s
(
y

o
)

y
y
im
y
  

 Z
f X
xim  
 ox
sx Z
f Y
yim  
 oy
sy Z
 f
sx
 wxim  
 wy    0
 im  
 w   0


0
f
sy
0
K 
ox 
X
 
oy   Y 
Z

1  


Parâmetros intrínsecos com
cisalhamento
X 
 xh   f x sh ox   
 y    0  f o  Y 
y
y
 h 
Z 
 w   0 0 1   
1
fx  f
fy  f
sx
sy
p  KI3 0P
[ fx
f y sh ox oy
]
Sem correção
Com correção radiométrica
Tipos de distorção radial
k1  0
xd  x(1  k1rd2  k2 rd4 )
x
xd
1  k1rd2  k2 rd4
y
yd
1  k1rd2  k2 rd4
yd  y (1  k1rd2  k2 rd4 )
rd  x  y
2
d
2
d
k1  0
k 2  k1
k2  0
Exemplo de distorção radial
Imagem original
Corrigida k1=0.11, k2=0.019
Com correção radiométrica e radial
CALIBRAÇÃO
Calibração de camera
min | pi  f ( Pi ) |2
Compondo as transformações
[ p]  K
R
T
P
parâmetros extrínsecos
parâmetros intrínsecos
 wxim   f x
 wy    0
 im  
 w   0
sh
 fy
0
ox    i w  i c 

o y    i w  jc 
 
1   i w  k c 
jw  i c  k w  i c  
jw  jc  k w  jc  
jw  k c  k w  k c 
 X w  
t
 x     
   Yw 
 t y     
 t    Z w  
 z   1 


Parâmetros intrínsecos com distorção radial
x'  xd (1  k r  k r )
2
1 d
4
2 d
y '  yd (1  k r  k r )
2
1 d
rd  x  y
2
d
4
2 d
2
d
k 2  k1
k2  0
[ fx
f y sh ox oy
k1 ]
Parâmetros extrínsecos
• Translação –Tx, Ty, Tz (3 g.l.)
• Rotação (3 g.l.)
 r11
r
 21
 r31
r12
r22
r32
r13 
r33 
r33 
 iw  
 j  
w


 k w 
i w  jw  i w  k w  k w  jw  0
i w  i w  jw  jw  k w  k w  1
Calibração de câmera
• Problema:
obter os parâmetros extrínsecos (R, T) e
intrínsecos (K) da transformação projetiva de
câmera.
• Dados:
n pares de pontos correspondentes (Pi, pi) na
cena e na imagem.
Calibração de câmeras
• Calibração  estimação de parâmetros  otimização
min  | pi  f K ,R,T ( Pi ) |
2
K , R,T
pontos da cena
projeção (função não linear)
pontos da imagem
Proceedings of SIBGRAPI'98, Rio de Janeiro, Brazil, 1998, pp. 388-399.
Equações de projeção
Xc
xim   f x c  ox
Z
Yc
yim   f y c  o y
Z
 X   r11
 c 
 Y   r21
 Z c  r31
 
c
r12
r22
r32
0 ox   X c 
 wxim   f x
wy    0  f o   Y c 

y
y 
 im  
 w   0
0
1   Z c 
r13   X  Tx 
 
r33   Y w   Ty 
r33   Z w  Tz 
w
 wxim   f x
wy    0
 im  
 w   0
0
 fy
0
 X   r11
 c 
 Y   r21
 Z c  r31
 
c
ox   r11 r12
o y  r21 r22

1   r31 r32
r13
r23
r33
r12
r13
r22
r23
r32
r33
X w
Tx   w 
Y
Ty   w 
 Z 
Tz   
 1 
X w
Tx   w 
Y 


Ty  w
Z 
Tz   
 1 
Equações de projeção
 wxim   f x
wy    0
 im  
 w   0
0
 fy
0
ox   r11 r12
o y  r21 r22

1   r31 r32
 wxim    f x r11  ox r31  f x r12  ox r32
wy    f r  o r  f r  o r
y 22
x 32
 im   y 21 x 31
 w  
r31
r32
 wxim   m11 m12
wy   m
m22
 im   21
 w  m31 m32
m13
m23
m33
r13
r23
r33
X w
Tx   w 
Y
Ty   w 
 Z 
Tz   
 1 
 f x r13  ox r33
 f y r23  ox r33
r33
X w
m14   w 
Y 


m24
 Z w 
m34   
 1 
X w
 f xTx  oxTz   w 
Y 


 f yTy  oxTz
 Z w 
  
Tz
 1 
Calibração a partir da matriz
Para cada ponto:
 ui   m11
  
 vi   m21
 w  m
 i   31
xi 
yi 
m12
m13
m22
m23
m32
m33
 X iw 
m14   w 
Y
m24   i w 
 Zi 
m34   
 1 
m11 X iw  m12Yi w  m13 Z iw  m14
m31 X iw  m32Yi w  m33 Z iw  m34
m21 X iw  m22Yi w  m23 Z iw  m24
m31 X iw  m32Yi w  m33 Z iw  m34

 y m

 m  0
m11 X iw  m12Yi w  m13 Z iw  m14  xi m31 X iw  m32Yi w  m33 Z iw  m34  0
m21 X iw  m22Yi w  m23 Z iw  m24
i
w
w
w
X

m
Y

m
Z
i
i
i
31
32
33
34
Calibração a partir da matriz
Para cada ponto:

 y m

 m  0
m11 X iw  m12Yi w  m13 Z iw  m14  xi m31 X iw  m32Yi w  m33 Z iw  m34  0
m21 X iw  m22Yi w  m23 Z iw  m24
i
w
w
w
X

m
Y

m
Z
i
i
i
31
32
33
34
X iwm11  Yiwm12  Z iwm13  m14  xi X iwm31  xiYiwm32  xi m33 Z iw  xi m34  0
X iwm21  Yiwm22  Z iwm23  m24  yi X iwm31  yiYiwm32  yi m33 Z iw  y3i m34  0
 m11 
m 
 12 
m  
 
 m33 
 m34 
Am  0
 m  0
Reconstruindo as matrizes
q4
q1
q2
q3
  f x r11  ox r31
  f y r21  ox r31

r31
 m11
m
 21
 m31
m12
m13
m22
m32
m23
m33
 f x r12  ox r32
 f x r13  ox r33
 f y r22  ox r32
r32
 f y r23  ox r33
r33
2
2
2
q3  m31
 m32
 m33

Tz   m34
m14 
m24 
m34 
 f xTx  oxTz 
 f yTy  oxTz 

Tz
r312  r322  r332  
escolha   1 tal queTz  0
r31  m31, r32  m32 , r33  m33
Reconstruindo as matrizes
q4
 m11
m
 21
 m31
q1
q2
q3
  f x r11  ox r31
 f r  o r
 y 21 x 31

r31
m12
m13
m22
m32
m23
m33
m14 
m24 
m34 
 f x r12  ox r32
 f x r13  ox r33
 f y r22  ox r32
r32
 f y r23  ox r33
r33
q3  k c
q1   f x i c  oxk c
q 2   f y jc  o y k c
 f xTx  oxTz 
 f yTy  oxTz 

Tz
ox  q1  q3
f x  q1  q1  ox
o y  q 2  q3
f y  q 2  q 2  oy
Reconstruindo as matrizes
q4
q1
q2
q3
  f x r11  ox r31
 f r  o r
 y 21 x 31

r31
 m11
m
 21
 m31
m12
m13
m22
m32
m23
m33
m14 
m24 
m34 
 f x r12  ox r32
 f x r13  ox r33
 f y r22  ox r32
r32
 f y r23  ox r33
r33
r1i  (ox m3i  m1i ) / f x
i  1,2,3
r2i  (o y m3i  m2i ) / f y
i  1,2,3
Tx  (ox m33  m14 ) / f x
Ty  (o y m33  m24 ) / f y
 f xTx  oxTz 
 f yTy  oxTz 

Tz
R  UDVT  R  UIVT
MÉTODOS DE TSAI
Com notação do OpenGL
Y
y'
c
p’
Zc
Xc
x'
Xc


c
c
P Y 
 Zc 
 
 x' 
p   
 y' 
Xc c


Z
f c

Y

c 
Z 

Câmera para imagem
 x   s x ( xim  ox ) 

p     

s
(
y

o
)

y
   y im y 
Xc c


Z
f c

Y

c 
Z 

f Xc
xim  ox  
c
sx Z
c
f Y
yim  o y  
sy Z c
Concatenando
f Xc
xim  ox  
sx Z c
c
f Y
yim  o y  
sy Z c
 X c   r11
 c 
 Y   r21
 Z c   r31
 
r12
r22
r32
r13   X w  Tx 
 
r33   Y w   Ty 
r33   Z w  Tz  camera
r11 X w  r12Y w  r13 Z w  Tx
xim  ox   f x
r31 X w  r32Y w  r33 Z w  Tz
r21 X  r22Y  r23 Z  Ty
w
yim  o y   f y
w
w
r31 X  r32Y  r33 Z  Tz
w
w
w
Os passos do Método de Tsai
Passo 1:
Assuma:
(ox , ox )
conhecidos
Distorção radial insignificante
r11 X i  r12Yi  r13 Z i  Tx
w
xi   f x
yi   f y
w
w
r31 X iw  r32Yi w  r33 Z iw  Tz
r21 X iw  r22Yi w  r23 Z iw  Ty
r31 X iw  r32Yi w  r33 Z iw  Tz
xi  xim  ox
yi  yim  oy
Método de Tsai
xi   f x
yi   f y
r11 X iw  r12Yi w  r13 Z iw  Tx
r31 X iw  r32Yi w  r33 Z iw  Tz
r21 X iw  r22Yi w  r23 Z iw  Ty
r31 X i  r32Yi  r33 Z i  Tz
w
w
w

fx
r31 X i  r32Yi  r33 Z i  Tz  
r11 X iw  r12Yi w  r13 Z iw  Tx
xi
w
w
w
r31 X i  r32Yi  r33 Z i  Tz  
w
w
w
fy
yi
r

w
w
w
X

r
Y

r
Z
21 i
22 i
23 i  Ty

Método de Tsai
Passo 1:

fx
r31 X i  r32Yi  r33 Z i  Tz  
r11 X iw  r12Yi w  r13 Z iw  Tx
xi
w
w
w
r31 X i  r32Yi  r33 Z i  Tz  
w

w
w

fy
yi
r

w
w
w
X

r
Y

r
Z
21 i
22 i
23 i  Ty


xi f y r21 X iw  r22Yi w  r23Z iw  Ty  yi f x r11 X iw  r12Yi w  r13Z iw  Tx

xi X iwr21  xiYi wr22  xi Z iwr23  xiTy  yi X iwr11  yiYi wr12  yi Z iwr13  yiTx
fx
f sy sy
 

f y sx f sx
Método de Tsai
Passo 1:
xi X iwr21  xiYi wr22  xi Z iwr23  xiTy  yi X iw r11  yiYi w r12  yi Z iw r13  yi Tx
v1
v2
v3
v4
 r21
 r22
 r23
 Ty
v5
v6
v7
v8
 r11
 r12
 r13
 Tx
xi X iwv1  xiYi wv2  xi Z iwv3  xi v4  yi X iwv5  yiYi wv6  yi Z iwv7  yi v8
xi X iwv1  xiYi wv2  xi Z iwv3  xi v4  yi X iwv5  yiYi wv6  yi Z iwv7  yi v8  0
Correspondência
x1 , y1 
 x2 , y 2 
x3 , y3 
 xN , y N 
X ,Y , Z 
X ,Y , Z 
X ,Y , Z 
w
w
1
w
1
w
1
w
w
2
2
2
w
w
w
3
X
3
w
3
w
w
,
Y
,
Z
N
N
N

Sistema Ax=0
 x1 X 1w

w
X
x
2
2

 

w
 xN X N
x1Y1w
x1Z1w
x1
 y1 X 1w
 y1Y1w
 y1Z iw
x2Y2w
x2 Z 2w
x2
 y2 X 2w
 y2Y2w
 y2 Z 2w






xN YNw
xN Z Nw
xN
 y N X Nw
 y N YNw
 y N Z Nw
Av  0
 v1  0
v  0 
 2  
 y1  v3  0
   
 y 2   v4   0 



0 

v5

   
 y N  v6  0
v  0 
 7  
v8  0
rank( A)  7
Compute v by SVD decomposition of A=UDVT (The solution
vector is the column of V corresponding to null (or smallest)
singular value) in D.
Estimativa dos parâmetros da
câmera
Fator de escala
Seja v um a soluçãonão  trivial,
v  γr21
r22
r23 Ty r11 r12 r13 Tx 
T
Como r212  r222  r232  1,


v12  v22  v32   2 r212  r222  r232  
Sinal do fator de escala
Sinal de

yi   f y
Como:
r21 X iw  r22Yi w  r23 Z iw  Ty
r31 X iw  r32Yi w  r33 Z iw  Tz
Zic  r31 X iw  r32Yi w  r33Z iw  Tz  0


w
w
w
Temos yi r21 X i  r22Yi  r23Z i  Ty  0
Caso isto não seja verdade troque o sinal de v
Estimativa do fator 
Since r212  r222  r232  1, we have


v52  v62  v72   2 2 r212  r222  r232   

1

v52  v62  v72
Última linha da matriz de rotação
 r31   r11   r21 
i
r   r   r   det  r
 32   12   22 
 11
r33   r13   r23 
r21
Reortogonalize:
 r11
r
 21
 r31
r12
r22
r32
j
r12
r22
k
r13 
r33 
r13 
r33 
r33 
R  UDVT  R  UIVT
Cáculo de fx fy e Tz

For each corresponding pairxi , yi  and X iw ,Yiw , Z iw




xi r31 X iw  r32Yiw  r33Z iw  Tz   f x r11 X iw  r12Yiw  r13Z iw  Tx




xiTz  r11 X iw  r12Yiw  r13Z iw  Tx f x  xi r31 X iw  r32Yiw  r33Z iw
TZ 
Solve A   b
 fx 

Cáculo de fx fy e Tz
 x1

x
A 2


 xN
r
r










  x1 r31 X 1w  r32Y1w  r33 Z1w

w
w
w

x
r
X

r
Y

r
Z
2
31
2
32
2
33
2
b



w
w
w
 xN r31 X N  r32YN  r33 Z N
w
w
w

X

r
Y

r
Z
11 1
12 1
13 1  Tx

w
w
w
X

r
Y

r
Z

T
11 2
12 2
13 2
x 



w
w
w
r11 X N  r12YN  r13 Z N  Tx 



Ax  b
UDV x  b
x  VD U b
Ax  b
A Ax  A b
x  A A AT b
T
T
T
1

T

1
T
Ponto de fuga
Cálculo do centro ótico pelos
pontos de fuga
Passo 2 do Tsai

Computing Image Center ox , o y

v3
v1
v2
Pontos de fuga do padrão 3D
Tsai 2D
Ziw  0
Coordenada do mundo para a câmera
 X c    i w  i c 
  
Pc   Yc     i w  jc 
 Z   i  k 
 c   w c
 X c   r11
 c 
 Y   r21
 Z c   r31
 
jw  i c  k w  i c  
jw  jc  k w  jc  
jw  k c  k w  k c 
r12
r22
r32
zc
xc
 X 
 t x   w 
   Yw 

 t y  

 t   Z w 
 z   1 


yc
r13   X w  Tx 
 
r33   Y w   Ty 
r33   Z w  Tz  camera
Pc
T
zw
xw
Pw
yw
Câmera para imagem
 x   s x ( xim  ox ) 

p     

s
(
y

o
)

y
   y im y 
Xc c


Z
f c

Y

c 
Z 

f Xc
xim  ox  
sx Z c
c
f Y
yim  o y  
c
sy Z
Concatenando
f Xc
xim  ox  
sx Z c
c
f Y
yim  o y  
sy Z c
 X c   r11
 c 
 Y   r21
 Z c   r31
 
r12
r22
r32
r13   X w  Tx 
 
r33   Y w   Ty 
r33   Z w  Tz  camera
r11 X w  r12Y w  r13 Z w  Tx
xim  ox   f x
r31 X w  r32Y w  r33 Z w  Tz
r21 X  r22Y  r23 Z  Ty
w
yim  o y   f y
w
w
r31 X  r32Y  r33 Z  Tz
w
w
w
Método de Tsai plano
Passo 1:
xi   f x
yi   f y
r11 X iw  r12Yi w  r13 Z iw  Tx
=0
r31 X iw  r32Yi w  r33 Z iw  Tz
r21 X iw  r22Yi w  r23 Z iw  Ty
r31 X iw  r32Yi w  r33 Z iw  Tz
r11 X i  r12Yi  Tx
w
xi   f x
yi   f y
w
r31 X i  r32Yi  Tz
w
w
r21 X iw  r22Yi w  Ty
r31 X iw  r32Yi w  Tz
Método de Tsai plano

fx
r31 X i  r32Yi  Tz  
r11 X iw  r12Yi w  Tx
xi
w
w
r31 X i  r32Yi  Tz  
w

w
fy
yi

r

w
w
X

r
Y
21 i
22 i  Ty


xi f y r21 X iw  r22Yi w  Ty  yi f x r11 X iw  r12Yi w  Tx

xi X iwr21  xiYi wr22  xiTy  yi X iwr11  yiYi wr12  yiTx
 1
xi X iwr21  xiYi wr22  xiTy  yi X iwr11  yiYi wr12  yiTx
Método de Tsai
Passo 1:
xi X iwv1  xiYi wv2  xi v3  yi X iwv4  yiYi wv5  yi v6  0
v1  r21
v2  r22
v3  Ty
v4  r11
v5  r12
v6  Tx
r11   v4
r12   v5
r21   v1
Av  0    v  0
r22   v2
Tx   v 6
T y   v3
r13  ?
r23  ?
Método de Tsai
ˆi  r r
c
11
12
r11   v4
r13 
T
r12   v5
r21   v1
ˆj  r r
c
21
22
r22   v2
r23 
T
v42  v52   2 r13   2
Tx   v 6
v12  v22   2 r23   2
T y   v3
v4 v1  v5v2   2 r13 r23  0
 4  v1  v2  v4  v5  2  v2v4  v1v5   0
 2   v1  v2  v4  v5  
1
2
v1  v2  v4  v5 2  4v2v4  v1v5  

Sinal de 
  2
escolha um sinal
r11   v4
r12   v5
xi   f x
r21   v1
r22   v2
r11 X iw  r12Yi w  Tx
r31 X iw  r32Yi w  Tz
Zic  0
Tx   v 6
T y   v3
Logo


xi r11 X iw  r12Yi w  Tx  0
Caso isto não seja verdade troque o sinal de 
Fator de escala
r11   v4
r12   v5
r21   v1
r22   v2
Tx   v 6
r13  ?
r23  ?
f ?
Tz  ?
T y   v3
escolha um sinal
r13   1  r112  r122
escolha um sinal
r23   1  r212  r222
r13r23  r11r21  r12 r22 
corrija a escolha
 r31   r11   r21 
r   r   r 
 32   12   22 
r33   r13   r23 
Cáculo de fx fy e Tz

For each corresponding pairxi , yi  and X iw ,Yiw , Z iw




xi r31 X iw  r32Yiw  Tz   f x r11 X iw  r12Yiw  Tx




xiTz  r11 X iw  r12Yiw  Tx f x  xi r31 X iw  r32Yiw
TZ 
Solve A   b
 fx 

Implementation of “A Flexible
New Technique for Camera
Calibration”
based on the report of:
Zhengyou Zhang
December 2, 1998
Notação do artigo de Zhang:
y
Y
x
X
 sx    u0 
 sy    0  v  r r r
0 1
2
3
  
 1   0 0 1 
X 
Y 
t 
0
 
1
 sx    u0 
 sy    0  v  r r
0 1
2
  
 1   0 0 1 
X 
Y 
t 
 
 
1
~
~
sm  AR tM
Determinação de uma Homografia
~
~  AR tM
sm
ui  h0
v   h
 i  3
 si  h6
h1
h4
h7
~
~
sm  HM
h2   X i 
h5   Yi 
h8   1 
h X  hY  h
xi  0 i 1 i 2
h6 X i  h7Yi  h8
H  AR t 
ui  h0 X i  h1Yi  h2
vi  h3 X i  h4Yi  h5
si  h6 X i  h7Yi  h8
h3 X i  h4Yi  h5
yi 
h6 X i  h7Yi  h8
h6 X i  h7Yi  h8 xi  h0 X i  h1Yi  h2
h6 X i  h7Yi  h8 yi  h3 X i  h4Yi  h5
y
Y
x
X
h6 X i  h7Yi  h8 xi  h0 X i  h1Yi  h2
h6 X i  h7Yi  h8 yi  h3 X i  h4Yi  h5
X i h0  Yi h1  1h2  0h3  0h4  0h5  xi X i h6  xiYi h7  xi h8  0
0h0  h1  0h2  X i h3  Yi h4  1h5  xi X i h6  xiYi h7  xi h8  0
2 equações por ponto. 9 (8) incógnitas
int homography(int nPoints, double* modelPoints, double* imagePoints, double* H)
{
int k;
double* L=(double*)malloc(2*nPoints*9*sizeof(double));
/* L is a 2nx9 matrix where Lij is in L[9*i+j] */
/* Assembles coeficiente matrix L */
for(k=0; k<nPoints; k++) {
double X=modelPoints[3*k+0]; /* X coord of model
double Y=modelPoints[3*k+1]; /* Y coord of model
double W=modelPoints[3*k+2]; /* W coord of model
double x=imagePoints[2*k+0]; /* x coord of image
double y=imagePoints[2*k+1]; /* y coord of image
point
point
point
point
point
k
k
k
k
k
int
i=2*k;
/* line number in matrix L
L[9*i+0] =
X; L[9*i+1] =
Y; L[9*i+2] =
W;
L[9*i+3] =
0; L[9*i+4] =
0; L[9*i+5] =
0;
L[9*i+6] = -x*X; L[9*i+7] = -x*Y; L[9*i+8] = -x*W;
i=2*k+1;
L[9*i+0] =
0; L[9*i+1] =
0; L[9*i+2] =
0;
L[9*i+3] =
X; L[9*i+4] =
Y; L[9*i+5] =
W;
L[9*i+6] = -y*X; L[9*i+7] = -y*Y; L[9*i+8] = -y*W;
}
solveAx0(2*nPoints,9,L,H); /* solves the system [L]{h}={0} */
free(L);
return 0;
}
*/
*/
*/
*/
*/
*/
Minimização (Levenberg-Marquardt)
Tome H como solução inicial
2
2





u
v
 x  i    y  i  

 i s   i s  

i 0 
i 
i 



N
Minimize:
onde:
ui  h0
v   h
 i  3
 si  h6
h1
h4
h7
h2   X i 
h5   Yi 
h8   1 
int homography(int nPoints, double* modelPoints, double* imagePoints, double* H)
{
int k;
double* L=(double*)malloc(2*nPoints*9*sizeof(double));
/* L is a 2nx9 matrix where Lij is in L[9*i+j] */
/* Assembles coeficiente matrix L */
for(k=0; k<nPoints; k++) {
double X=modelPoints[3*k+0]; /* X coord of model point k
double Y=modelPoints[3*k+1]; /* Y coord of model point k
double W=modelPoints[3*k+2]; /* W coord of model point k
double x=imagePoints[2*k+0]; /* x coord of image point k
double y=imagePoints[2*k+1]; /* y coord of image point k
int
i=2*k;
/* line number in matrix L
L[9*i+0]
L[9*i+3]
L[9*i+6]
i=2*k+1;
L[9*i+0]
L[9*i+3]
L[9*i+6]
=
X; L[9*i+1] =
Y; L[9*i+2] =
W;
=
0; L[9*i+4] =
0; L[9*i+5] =
0;
= -x*X; L[9*i+7] = -x*Y; L[9*i+8] = -x*W;
=
0; L[9*i+1] =
0; L[9*i+2] =
0;
=
X; L[9*i+4] =
Y; L[9*i+5] =
W;
= -y*X; L[9*i+7] = -y*Y; L[9*i+8] = -y*W;
}
solveAx0(2*nPoints,9,L,H); /* solves the system [L]{h}={0} */
if (OPTIMIZE) {
double lmH[9];
lmHomography(imagePoints,modelPoints,H,nPoints,lmH);
mtxMatCopy(lmH,3,3,H);
}
free(L);
return 0;
}
*/
*/
*/
*/
*/
*/
Restrições na matriz de parâmetros intrínsicos
~
~  AR tM
sm
h1
r1 
r2 
~
~
sm  HM
h2 h3   Ar1 r2
1

1

A 1h1
A 1h 2
H  AR t 
t
r1  r2  0
r1 r2  0
r1  r1  r2  r2  1
r1 r1  r2 r2  0
T
T
h1 A T A 1h 2  0
T
h1 A T A 1h1  h 2 A T A 1h 2  0
T
T
T
Das homografias para parâmetros intrínsecos
h1 A T A 1h 2  0
T
h1 A T A 1h1  h 2 A T A 1h 2  0
T
T
b0
onde: B  A T A 1   b1

b3
h B h2  0
T
1
h B h1  h 2 B h 2  0
T
1
T
b1
b2
b4
b3 
h0 h1 h2 
b4  e H   h3 h4 h5 


h6 h7 h8 
b5 
h0h1 b0  (h0h4  h3h1 )b1  h3h4 b2  (h0h7  h6h1 )b3  (h3h7  h6h4 )b4  h6h7 b5  0
(h0  h1 )b0  2(h0h3  h1h4 )b1  (h3  h4 )b2  2(h0h6  h1h7 )b3  3(h3h6  h4h7 )b4  (h6  h7 )b5  0
2
2
2
2
2 equações por homografia. 6 incógnitas
2
2
void calcB(int nH, double* H, double* B)
{
int m = 2*nH;
int n = 6;
double* V =(double*) calloc(sizeof(double),m*n);
double* b =(double*) malloc(sizeof(double)*n);
int i;
for(i=0;i<nH;i++){
double* h = &H[9*i];
int line1 = 2*n*i;
int line2 = line1+6;
V[line1+0]=h[0]*h[1];
V[line1+2]=h[3]*h[4];
V[line1+4]=h[3]*h[7]+h[6]*h[4];
}
V[line1+1]=h[0]*h[4] + h[3]*h[1];
V[line1+3]=h[0]*h[7]+h[6]*h[1];
V[line1+5]=h[6]*h[7];
V[line2+0]=h[0]*h[0] - h[1]*h[1];
V[line2+1]=2*(h[0]*h[3] - h[1]*h[4]);
V[line2+2]=h[3]*h[3] - h[4]*h[4];
V[line2+3]=2*(h[0]*h[6] - h[1]*h[7]);
V[line2+4]=2*(h[3]*h[6] - h[4]*h[7]); V[line2+5]=h[6]*h[6] - h[7]*h[7];
if (NORMALIZE) {
mtxNormalizeVector(6,&V[line1]);
mtxNormalizeVector(6,&V[line2]);
}
solveAx0(m,n,V,b); /* solves the system [V]{x}={0} */
i=0;
B[i++]=b[0]; B[i++]=b[1]; B[i++]=b[3];
B[i++]=b[1]; B[i++]=b[2]; B[i++]=b[4];
B[i++]=b[3]; B[i++]=b[4]; B[i++]=b[5];
}
free(b);
free(V);
Cálculo da matrix A
/* Get intrinsic parameters from matrix B*/
int calcA(double* B, double* A)
{
double alpha,betha,gamma,u0,v0,lambda;
double den=B[0]*B[4]-B[1]*B[1];
if (fabs(den)< TOL ) return 0;
v0 = (B[1]*B[2]-B[0]*B[5])/den;
if (fabs(B[0])<TOL) return 0;
lambda = B[8]-(B[2]*B[2]+v0*(B[1]*B[2]-B[0]*B[5]))/B[0];
if (lambda/B[0]<0) return 0;
alpha=sqrt(lambda/B[0]);
if ((lambda*B[0]/den)<0) return 0;
betha = sqrt(lambda*B[0]/den);
gamma = - B[1]*alpha*alpha*betha/lambda;
u0=gamma*v0/betha-B[2]*alpha*alpha/lambda;
A[0]=alpha; A[1]=gamma; A[2]=u0;
A[3]=0;
A[4]=betha; A[5]=v0;
A[6]=0;
A[7]=0;
A[8]=1;
}
return 1;
Cálculo de R e t
Calibração de Zhang para uma câmera “boa”
yim
yc
y0
oc
x'
eixo óptico
x0
xc
w
ox 
2
h
oy 
2
xim
f
yim
y'
vista lateral
yc
zc
X
 
P Y 
Z 
 
fovy
oc
f
h pixels
zc
y'
oy
x'
ox
w pixels
xim
Notação do artigo de Zhang:
y
Y
x
X
X 
u


u
  
0
Y 




s v    0  v0  r1 r2 r3 t   
0
1  0 0 1 
 
1
u
s
v
y
s
x
     f p   f x (em pixel)
x
  u0  v0  0
X 
u   f x 0 0
Y 




s v    0  f x 0 r1 r2 r3 t  
0
1  0 0 1
 
1
~
~
sm  AR tM
u   f x 0 0
s v    0  f x 0 r1 r2
1  0
0 1
X 
Y 
t  
 
 
1
Determinação de uma Homografia
~
~  AR tM
sm
ui  h0
v   h
 i  3
 si  h6
h1
h4
h7
~
~
sm  HM
h2   X i 
h5   Yi 
h8   1 
h X  hY  h
xi  0 i 1 i 2
h6 X i  h7Yi  h8
H  AR t 
ui  h0 X i  h1Yi  h2
vi  h3 X i  h4Yi  h5
si  h6 X i  h7Yi  h8
h3 X i  h4Yi  h5
yi 
h6 X i  h7Yi  h8
h6 X i  h7Yi  h8 xi  h0 X i  h1Yi  h2
h6 X i  h7Yi  h8 yi  h3 X i  h4Yi  h5
y
Y
x
X
h6 X i  h7Yi  h8 xi  h0 X i  h1Yi  h2
h6 X i  h7Yi  h8 yi  h3 X i  h4Yi  h5
X i h0  Yi h1  1h2  0h3  0h4  0h5  xi X i h6  xiYi h7  xi h8  0
0h0  h1  0h2  X i h3  Yi h4  1h5  xi X i h6  xiYi h7  xi h8  0
2 equações por ponto. 9 (8) incógnitas
int homography(int nPoints, double* modelPoints, double* imagePoints, double* H)
{
int k;
double* L=(double*)malloc(2*nPoints*9*sizeof(double));
/* L is a 2nx9 matrix where Lij is in L[9*i+j] */
/* Assembles coeficiente matrix L */
for(k=0; k<nPoints; k++) {
double X=modelPoints[3*k+0]; /* X coord of model
double Y=modelPoints[3*k+1]; /* Y coord of model
double W=modelPoints[3*k+2]; /* W coord of model
double x=imagePoints[2*k+0]; /* x coord of image
double y=imagePoints[2*k+1]; /* y coord of image
point
point
point
point
point
k
k
k
k
k
int
i=2*k;
/* line number in matrix L
L[9*i+0] =
X; L[9*i+1] =
Y; L[9*i+2] =
W;
L[9*i+3] =
0; L[9*i+4] =
0; L[9*i+5] =
0;
L[9*i+6] = -x*X; L[9*i+7] = -x*Y; L[9*i+8] = -x*W;
i=2*k+1;
L[9*i+0] =
0; L[9*i+1] =
0; L[9*i+2] =
0;
L[9*i+3] =
X; L[9*i+4] =
Y; L[9*i+5] =
W;
L[9*i+6] = -y*X; L[9*i+7] = -y*Y; L[9*i+8] = -y*W;
}
solveAx0(2*nPoints,9,L,H); /* solves the system [L]{h}={0} */
free(L);
return 0;
}
*/
*/
*/
*/
*/
*/
Minimização (Levenberg-Marquardt)
Tome H como solução inicial
2
2





u
v
 x  i    y  i  

 i s   i s  

i 0 
i 
i 



N
Minimize:
onde:
ui  h0
v   h
 i  3
 si  h6
h1
h4
h7
h2   X i 
h5   Yi 
h8   1 
int homography(int nPoints, double* modelPoints, double* imagePoints, double* H)
{
int k;
double* L=(double*)malloc(2*nPoints*9*sizeof(double));
/* L is a 2nx9 matrix where Lij is in L[9*i+j] */
/* Assembles coeficiente matrix L */
for(k=0; k<nPoints; k++) {
double X=modelPoints[3*k+0]; /* X coord of model point k
double Y=modelPoints[3*k+1]; /* Y coord of model point k
double W=modelPoints[3*k+2]; /* W coord of model point k
double x=imagePoints[2*k+0]; /* x coord of image point k
double y=imagePoints[2*k+1]; /* y coord of image point k
int
i=2*k;
/* line number in matrix L
L[9*i+0]
L[9*i+3]
L[9*i+6]
i=2*k+1;
L[9*i+0]
L[9*i+3]
L[9*i+6]
=
X; L[9*i+1] =
Y; L[9*i+2] =
W;
=
0; L[9*i+4] =
0; L[9*i+5] =
0;
= -x*X; L[9*i+7] = -x*Y; L[9*i+8] = -x*W;
=
0; L[9*i+1] =
0; L[9*i+2] =
0;
=
X; L[9*i+4] =
Y; L[9*i+5] =
W;
= -y*X; L[9*i+7] = -y*Y; L[9*i+8] = -y*W;
}
solveAx0(2*nPoints,9,L,H); /* solves the system [L]{h}={0} */
if (OPTIMIZE) {
double lmH[9];
lmHomography(imagePoints,modelPoints,H,nPoints,lmH);
mtxMatCopy(lmH,3,3,H);
}
free(L);
return 0;
}
*/
*/
*/
*/
*/
*/
Restrições na matriz de parâmetros intrínsicos
~
~  AR tM
sm
h1
r1 
r2 
~
~
sm  HM
h2 h3   Ar1 r2
1

1

A 1h1
A 1h 2
H  AR t 
t
r1  r2  0
r1 r2  0
r1  r1  r2  r2  1
r1 r1  r2 r2  0
T
T
h1 A T A 1h 2  0
T
h1 A T A 1h1  h 2 A T A 1h 2  0
T
T
T
Das homografias para parâmetros intrínsecos
h1 A T A 1h 2  0
T
h1 A T A 1h1  h 2 A T A 1h 2  0
T
h1 B h 2  0
T
T
onde:
h B h1  h 2 B h 2  0
T
1
T
b0 b1
B  A T A 1   b1 b2
b3 b4
1
 2
b3   f x
b4    0

b5   0


0
1
f x2
0

0

0

1


h0
e H   h3

h6
h1
h4
h7
h2 
h5 
h8 
h0h1 b0  (h0h4  h3h1 )b1  h3h4 b2  (h0h7  h6h1 )b3  (h3h7  h6h4 )b4  h6h7 b5  0
(h0  h1 )b0  2(h0h3  h1h4 )b1  (h3  h4 )b2  2(h0h6  h1h7 )b3  3(h3h6  h4h7 )b4  (h6  h7 )b5  0
2
2
2
2
2
h0 h1  h3h4
h6 h7
h0h1 h3h4
 2  h6h7  0
2
fx
fx
fx  
h0  h1 h3  h4
2
2

 (h6  h7 )  0
2
2
fx
fx
(h  h1 )  (h3  h4 )
fx   0
2
2
h6  h7
2
2
2
2
2
2
2
2
2
Cálculo de R e t
FIM