Motion Estimation by Affine Transforms Based on

Motion Estimation by Affine Transforms Based on Codirectionality of Movements Mohrekesh, S. Samavi, N. Karimi, S. Shirani, P. Behnamfar Department of Electrical and Computer Engineering Isfahan University of Technology, Isfahan, Iran McMaster University, Hamilton, Canada University of British Columbia, Canada

Outline • Introduction • Motion vectors • Block based • Mesh based

• ABC • Results • Conclusion

Importance of Video Compression • Improvements in Video Processing • Video Applications • Machine vision

• Medical imaging • Video conferencing • Remote learning

• Information storage/transmission in limited memory/bandwidth  Impossible storage/transmission of raw data  Definite need for compression

Introduction: Video Compression • Video Compression Basis • Motion estimation/compensation • Time redundancy elimination

• Differences Between Methods • Complexity • Accuracy • Time for calculation

Block Based Motion Estimation • Best mach of current block in the reference frame • Current block displacement from reference block f ( x , y )  x  ui ui  v   Motion Vector g( x , y )  y  vi  i Reference Frame

Current Frame

Error Criteria 1 MSE  2 N

1 MAE  2 N

2

N 1 N 1

 C i 0 j 0

ij

N 1 N 1

 C i 0 j 0

N 1 N 1

SAD   Cij  Rij i 0 j 0

ij

 Rij 

 Rij

Block Based Shortcomings • Just Translation • Unable to identify motions such as: • Rotation • Shearing • Zoom in/out

• Disability in Codirectionality • Equal Motions for Pixels of a Block • Reconstructed Frame Discontinuities

Mesh Based Motion Estimation • Various Motions Modeling • Using Transforms • Current Frame  Mesh • Mesh • Regular: less accurate • Irregular: more complex

Transforms • Affine

f ( x , y )  ai 1 x  ai 2 y  ai 3 g( x , y )  ai 4 x  ai 5 y  ai 6

•Ability to Model Different Motions • Disability in Codirectionality • More Complex

Proposed Method: ABC • Affine transform Based on Codirectionality • Current Frame Partitioning • Assuming Block for Nodes • Block Matching

Reference Frame

Current Frame

ABC • Finding Triangles Motion Vectors •Triangle Partitioning (if needed)

ABC • Transforms • Affine • Bilinear vector interpolation • Translation

ABC: Transform Selection Criteria • Affine Domain • Rotation • Zoom in • Zoom out

Almost equilateral triangles produced  Vector difference lengths almost equal  Selection based on closeness of vector differences

ABC: Finding Different Vector

a c

b

a b c (a+b)/4

(a+b)/2

One of differences smaller than half of average of the others

for each ΔABC

α,β  A,B,C 



d α,β   xMV α   xMV  β     yMV α   yMV  β   2

N1,N 2   arg min  d  γ  α,β  if d N1,N 2   1  d γ  10

2

α  β

γ

F  A,B,C   N1,N 2  F  N1 F  N2  , N2  2 2 translate  ΔFN1 N 2 ,MV F  interpolat e ΔFN1 N 2  ΔFN1 N 2  N1 

else

affine  ΔABC,MV  A,MV B ,MV C 

end if end for



1

ABC 2

Results Paris ABC

Results Mobile

ABC

Results Foreman ABC ABC

Results Hall Monitor ABC ABC

Results Original frame

MFMB result

ABC result

Conclusion • Video Compression Importance • Time redundancy

• Motion Estimation • Block based method • Mesh based method

•Proposed ABC Method • Better performance in codirectionality • Higher PSNR

• Results