Department of Electrical and Computer Engineering. Isfahan. Isfahan University of Technology, Isfahan, Iran. University of Technology, Isfahan, Iran. McMaster ...
Motion Estimation by Affine Transforms Based on Codirectionality of Movements Mohrekesh, S. Samavi, N. Karimi, S. Shirani, P. Behnamfar Department of Electrical and Computer Engineering Isfahan University of Technology, Isfahan, Iran Isfahan University of Technology, Isfahan, Iran McMaster University, Hamilton, Canada McMaster University, Hamilton, McMaster University, Hamilton, Canada University of British Colombia, Canada
Outline • Introduction • Motion vectors • Block based • Mesh based
• ABC • Results • Conclusion
Importance of Video Compression • Improvements in Video Processing • Video Applications pp • Machine vision
• Medical imaging • Video Vid conferencing f i • Remote learning
• Information storage/transmission in limited memory/bandwidth Î Impossible storage/transmission of raw data Î Definite need for compression
Introduction: Video Compression • Video Compression Basis • Motion estimation/compensation • Time redundancy elimination
•D Differences fferences Between Methods • Complexity • Accuracy • Time for calculation
Block Based Motion Estimation • Best mach of current block in the reference f f frame • Current block displacement from reference block f ( x , y ) = x − ui ⎡ui ⎤ ⎢v ⎥ Î Motion Vector g( x , y ) = y − vi ⎣ i⎦ Reference Frame
Current Frame
Error Criteria 2
N −1 N −1
1 MSE = 2 N
∑ ∑ (C
1 MAE = 2 N
N −1 N −1
i =0 j =0
ijj
∑∑ C i =0 j =0
N −1 N −1
SAD = ∑∑ Cij − Rij i =0 j = 0
ij
− Rijj )
− Rij
Block Based Shortcomings • Just Translation • Unable to identify motions such as: • Rotation • Shearing • Zoom in/out
• Disability in Codirectionality • Equal Motions for Pixels of a Block • Reconstructed Frame Discontinuities
Mesh Based Motion Estimation
• • • •
Various Motions Modeling Using Transforms Current Frame F Î Mesh h Mesh • Regular: less accurate g more complex p • Irregular:
Transforms • Affine
f ( x , y ) = ai 1 x + ai 2 y + ai 3 g( x , y ) = ai 4 x + ai 5 y + ai 6
•Ability y to Model Different Motions • Disability in Codirectionality • More Complex
Proposed Method: ABC • • • •
Affine transform Based on Codirectionality Current Frame Partitioning Assuming g Block for Nodes Block Matching
Reference Frame
Current Frame
ABC • Finding Triangles Motion Vectors •Triangle Triangle Partitioning (if needed)
ABC • Transforms • Affine • Bilinear vector interpolation • Translation
ABC: Transform Selection Criteria • Affine Domain • Rotation • Zoom in • Zoom out
Almost equilateral triangles produced Î Vector difference lengths almost equal Î Selection based on closeness of vector differences
ABC: Finding Different Vector
a b c
One of differences smaller than half of average of the others smaller than half of average of the others
for each ΔABC
∀α,β ∈ {A,B,C}
(
d (α,β ) = (xMV (α ) − xMV ( β ) ) + ( yMV (α ) − yMV ( β ) ) 2
(N1,N 2 ) = arg minγ (d ) ∀γ ∈ {(α,β ) if d ( N1,N 2 ) < 1 ∑ d (γ ) 10
2
α ≠ β}
γ
F = {A,B,C} − {N1,N 2 } F + N1 F + N2 ′ , N2 = 2 2 translate( ΔFN1′ N 2′ ,MV (F )) interpolate( ΔFN1 N 2 − ΔFN1′ N 2′ ) N1′ =
else affine ff ( ΔABC,MV ( A),MV (B ),MV (C )) end if end for
)
1
2
ABC
Results Paris ABC
Results Mobile
ABC
Results Foreman ABC ABC
Results Hall Monitor ABC ABC
Results Original frame
MFMB result
ABC result
Conclusion • Video Vid C Compression i I Importance t • Time redundancy
• Motion Estimation • Block based method • Mesh based method
•Proposed Proposed ABC Method • Better performance in codirectionality • Higher PSNR
• Results
