Implementation of GLCM Features in Thermal ... - ScienceDirect.com

6 downloads 0 Views 569KB Size Report
In this experiment, a totally polar opposite emotion has been ... Throughout the session, we recorded the thermal video using FLIR T420 at 30 frames ..... Emotion Recognition using Hidden marNov models from Facial temperature Sequences ...
Available online at www.sciencedirect.com

ScienceDirect Procedia Computer Science 76 (2015) 308 – 315

,(((,QWHUQDWLRQDO6\PSRVLXPRQ5RERWLFVDQG,QWHOOLJHQW6HQVRUV ,5,6 

,PSOHPHQWDWLRQRI*/&0)HDWXUHVLQ7KHUPDO,PDJLQJIRU+XPDQ $IIHFWLYH6WDWH'HWHFWLRQ $EG/DWLI0+D0G   @  )URPWKHNHUQHOWKH*/&0ZLOOKDYHIRXUGLVSODFHPHQWGLVWDQFHVG DQGIRXURULHQWDWLRQVDQJOHVș ƒ ƒƒƒ6RIRUHDFK*/&0VWDWLVWLFDOIHDWXUHVWKHUHDUHIHDWXUHV IHDWXUHV[ IHDWXUHVIRUVLQJOH 52, +HQFHIRUHDFK52,WKHUHDUH IHDWXUHV[LPDJHV IHDWXUHV 7KH+DSS\6DGDQG)HDUGDWDVHW KDYH  IHDWXUHV UHVSHFWLYHO\ ZKLFK OHDGV WR D WRWDO RI  IHDWXUHV  VXEMHFWV DOO PDOH  RYHUDOO IRU HDFK 52, 7KH GDWDVHW (PRWLRQ7DEOH  WKHQ UHSUHVHQWHG LQ WKH IRUP RI  IHDWXUHV [  52,V 6XSUDRUELWDO 3HULRUELWDO1DVDODQG0RXWK  4.4. Classification )RUWKHFODVVLILFDWLRQSDUWVHYHUDO&ODVVLILHUVKDYHEHHQWHVWHGLQRUGHUWRDFKLHYHWKHEHVWFODVVLILFDWLRQUHVXOW $PRQJ WKRVH ZHUH 6XSSRUW 9HFWRU 0DFKLQH 690  .1HDUHVW 1HLJKERU .11  DQG 7UHH 7KH VL]H RI GDWDVHW (PRWLRQ7DEOH IHGLQWRWKHFODVVLILHUZDV[IHDWXUHV0RVWRIWKHFODVVLILFDWLRQSURFHVVHVZDVGRQHXVLQJ &ODVVLILFDWLRQ/HDUQHULQ0DWODED7KH52,VVHUYHGDVWKHSUHGLFWRUVZKLOHWKH(PRWLRQODEHOVDVWKHUHVSRQVH 7KH)ROG&URVV9DOLGDWLRQZDVFKRVHQIRUPRGHOYDOLGDWLRQ 5HVXOW 'LVFXVVLRQ )URPDOOWKHFODVVLILHUVWHVWHGLQFODVVLILFDWLRQVWDJH:HLJKWHG.11KDVVKRZQWKHEHVWFODVVLILFDWLRQDFFXUDF\ ZLWK  RYHUDOO DFFXUDF\ 7KH VHFRQG KLJKHVW DFFXUDF\ VFRUH EHORQJ WR )LQH .11 ZLWK RYHUDOO DFFXUDF\ RI ZKLFKORZHUVWKDQ:HLJKWHG.11%HVLGHV)LQH*DXVVLDQDQG0HGLXP*DXVVLDQ690GLVSOD\VTXLWH LPSUHVVLYHFODVVLILFDWLRQUHVXOWZLWKDQGRYHUDOODFFXUDF\UHVSHFWLYHO\7KHFRPSDULVRQVRIWKHFODVVLILHU SHUIRUPDQFHDUHWDEXODWHGLQ7DEOHEHORZLQWHUPRIRYHUDOODFFXUDF\7KHGHWDLOHGDFFXUDF\RIWKHFODVVLILHULV WDEXODWHGLQ7DEOH7KH$8&ZHUHREWDLQHGIURPWKH5HFHLYHU2SHUDWLQJ&KDUDFWHULVWLF 52& FXUYH7KHKLJKHU WKH$8&YDOXHWKHEHWWHUWKHDFFXUDF\RIWKHFODVVLILHUDVWKH52&FXUYHUHSUHVHQWVWKHVHQVLWLYLW\DVDIXQFWLRQRI IDOORXW  7DEOH&ODVVLILHU3HUIRUPDQFH&RPSDULVRQ &ODVVLILFDWLRQ0HWKRG

)HDWXUH

$FFXUDF\  

690)LQH*DXVVLDQ

*/&0



6900HGLXP*DXVVLDQ

*/&0



7UHH&RPSOH[

*/&0



7UHH0HGLXP

*/&0



.11)LQH.11

*/&0



.11:HLJKWHG.11

*/&0



  7DEOH'HWDLOHG$FFXUDF\E\&ODVVIRU:HLJKWHG.11&ODVVLILHU

 

&ODVV

735DWH

)35DWH

3UHFLVLRQ

5HFDOO

)0HDVXUH

52&$UHD$8&

+DSS\













6DG













)HDU













:HLJKWHG$YJ













314

M.H. Abd Latif et al. / Procedia Computer Science 76 (2015) 308 – 315

7DEOHWDEXODWHGWKHGHWDLOHGDFFXUDF\RI:HLJKWHG.11&ODVVLILHUE\FODVVHVRIHPRWLRQV7KHHPRWLRQFODVVHV DUH+DSS\6DGDQG)HDU7KH7UXH3RVLWLYH 73 5DWHLVDOVRNQRZQDVVHQVLWLYLW\ZKHUHLWVPHDVXUHVWKHSURSRUWLRQ RISRVLWLYHVWKDWDUHFRUUHFWO\FODVVLILHGDVSRVLWLYH7KHKLJKHVW735DWHIURPWKHFODVVHVRIHPRWLRQEHORQJWR)HDU &ODVVZLWKFRUUHFWO\FODVVLILHGLQVWDQFHV7KH)DOVH3RVLWLYH )3 5DWHLVDPHDVXUHVRILQFRUUHFWO\LGHQWLILHG RUIDOVHDODUPZKLFKFRQWULEXWHWRW\SH,HUURULQVWDWLVWLF7KHVPDOOHVW)35DWHFRXOGEHREVHUYHGIURP)HDU&ODVV ZLWK  7KH 3UHFLVLRQ SDUDPHWHU LV DOVR NQRZQ DV 3RVLWLYH 3UHGLFWLYH 9DOXH 339  ZKHUH LWV PHDVXUHV WKH SURSRUWLRQRIWKHWUXHSRVLWLYHVDJDLQVWDOOWKHSRVLWLYHUHVXOW 75DQG)3 /DVWO\WKH)0HDVXUHLVDSDUDPHWHUWKDW PHDVXUHVWKHKDUPRQLFPHDQRISUHFLVLRQDQGUHFDOO,WFDQEHXVHGDVDVLQJOHSHUIRUPDQFHLQGH[IRUWKHSRVLWLYH FODVVFODVVLILFDWLRQ )LJLOOXVWUDWHVWKHFRQIXVLRQPDWUL[IRU:HLJKWHG.11)URPWKHFRQIXVLRQPDWUL[VHYHUDOLPSRUWDQWYDULDEOHV FRXOGEHH[WUDFWHG7UXH3RVLWLYH 73 7UXH1HJDWLYH 71 )DOVH3RVLWLYH )3 )DOVH1HJDWLYH )1 7UXH3RVLWLYH 5DWH 739 )DOVH1HJDWLYH5DWH )15 3RVLWLYH3UHGLFWLYH5DWH 335 )DOVH'LVFRYHU\5DWH )'5 HWF                          )LJXUH&RQIXVLRQ0DWUL[ :HLJKWHG.11    ,QDGGLWLRQWRWKDWWKHIROGFURVVYDOLGDWLRQSHUIRUPHGRQWKH:HLJKWHG.11&ODVVLILHUKDVUHVXOWHGLQN IROG ORVV ZKLFK GHPRQVWUDWHV WKH VPDOO RXW RI VDPSOH HUURU 7KH VPDOO RXW RI VDPSOH ORVV VKRZV WKDW WKH PRGHO SHUIRUPHG DFFXUDWHO\ DV WKH ILWWLQJ IRU HDFK ILW VHVVLRQ IROG  DUH ZHOOILWWHG ZLWK PLQLPXP DYHUDJH HUURU 7KH REMHFWLYHRINIROGFURVVYDOLGDWLRQLVWRHVWLPDWHKRZDFFXUDWHO\WKHSUHGLFWLYHPRGHOZRXOGSHUIRUPLQSUDFWLFH :LWKWKHVPDOOUHVXOWHGNIROGORVVZHDUHSRVLWLYHDQGRSWLPLVWLFWKDWWKHSUHGLFWLYHPRGHOZRXOGSHUIRUPZHOOLQ UHDO OLIH VFHQDULR  %\ FRQVLGHULQJ VHYHUDO IHDWXUHV YHFWRU EHLQJ FRQFDWHQDWHG WRJHWKHU WKH RYHUDOO SHUIRUPDQFH PLJKWEHLPSURYHGGXHWRDYDLODELOLW\RIVHYHUDOGLVWLQFWIHDWXUHV%HVLGHVWKHIHDWXUHVYHFWRUVGDWDVHWFRXOGEHSUH SURFHVV EHIRUH IHHG LQ FODVVLILFDWLRQ SKDVH XVLQJ 3ULQFLSOH &RPSRQHQW $QDO\VLV 3&$  RU /LQHDU 'LVFULPLQDQW $QDO\VLV /'$ WRUHGXFHWKHGLPHQVLRQRIWKHGDWDVHW)XUWKHUPRUHIRUUHDOOLIHDSSOLFDWLRQRIWKHUPDOLPDJLQJ VHYHUDO NQRZQ OLPLWDWLRQV H[LVW 7KH HQYLURQPHQWDO FRQGLWLRQ SOD\V WKH FUXFLDOUROH VLQFH WKH KRPHRVWDVLV DQG WKH FXWDQHRXV WHPSHUDWXUH DUH FRQWLQXRXVO\ UHJXODWHG ,Q RUGHU WR DYRLG DWWULEXWLQJ DQ\ SV\FKRORJLFDO YDOHQFH WR XQPLWLJDWHGWKHUPRUHJXODWRU\SURSHUFRXQWHUPHDVXUHVDQGFDXWLRQVVKRXOGEHDGRSWHG 0HUOD %HVLGHVWKH RFFOXVLRQ RI 52,V E\ VSHFWDFOH RU KDLUEDQJ FRXOG UHQGHU WKH YLVLELOLW\ RI WKH WKHUPDO LPSULQW LQ WKH WDUJHWHG

M.H. Abd Latif et al. / Procedia Computer Science 76 (2015) 308 – 315

315

ORFDWLRQ7KHPHWDEROLFHIIHFWRIGLJHVWLRQDOVRDIIHFWHGWKHUHJXODWRU\RIFXWDQHRXVWHPSHUDWXUHRIWKHIDFLDOVNLQDV UHSRUWHGLQVHYHUDOVWXGLHV7KHPRGHOFRXOGEHLPSURYHGE\FRQVLGHULQJWKH52,VORFDWLRQWUDFNHUWRWUDFNWKH52,V ZKLFK PLJKW KHOS UHGXFLQJ WKH RXW RI IRFXV LPDJHV 2XU UHVHDUFK LV IDU IURP EHLQJ FRQFOXVLYH +RZHYHU WKHVH ILQGLQJV DUH XWPRVW SURPLVLQJ )RU IXUWKHU LPSURYHPHQW ZH ZRXOG QHHG PRUH WUDLQLQJ GDWD WKDQ ZKDW KDV EHHQ WHVWHG LQ WKLV SDSHU 7KLV GHYHORSPHQW RI DIIHFWLYH VWDWH GHWHFWLRQ IURP WKHUPDO LPDJLQJ FRXOG EH XVHIXO LQ +5, HVSHFLDOO\ZKHQGHDOLQJZLWKHPRWLRQDOO\FKDOOHQJHGLQGLYLGXDO HJ$XWLVWLF$VSHUJHU$'+'HWF :HEHOLHYH WKDWWKLVDIIHFWLYHVWDWHPRGHOFRXOGSURYLGHFRQWDFWOHVVQRQLQYDVLYHHPRWLRQPHDVXUHPHQWDQGKHOSVSV\FKLDWULVW ZLWKHPRWLRQDOVWDWHVHYDOXDWLRQDQGHQJDJHPHQWZLWKWKHVXEMHFW $FNQRZOHGJHPHQWV :HZLVKWRH[SUHVVRXUJUDWLWXGHWRWKH0LQLVWU\RI+LJKHU(GXFDWLRQ 02+( IRUIXQGLQJWKHSURMHFWXQGHUWKH )XQGDPHQWDO 5HVHDUFK *UDQW 6FKHPH )5*6  *UDQW QR )5*6 2XU GHHSHVW DSSUHFLDWLRQ DOVR JRHVWRSRVWJUDGXDWHVWXGHQWDQGOHFWXUHUVLQYROYHGLQPDNLQJWKLVSURMHFWSRVVLEOH 5HIHUHQFHV $'L *LDFLQWR 0%UXQHWWL*6HSHGH$)HUUHWWLDQG$0HUOD  7KHUPDOVLJQDWXUHRIIHDUFRQGLWLRQLQJLQ PLOGSRVWWUDXPDWLFVWUHVV GLVRUGHUNeuroscience ± $0HUODDQG*/5RPDQL  ³7KHUPDOVLJQDWXUHRIHPRWLRQDODURXVDODIXQFWLRQDOLQIUDUHGLPDJLQJVWXG\´LQProceedings of the 29th Annual International Conference of the IEEE EMBS Cite Internationale/\RQ)UDQFH$XJXVW $ 0HUOD  ´5HYHDOLQJ SV\FKRORJ\ DQG HPRWLRQ WKURXJK WKHUPDO LPDJLQJ ³LQ Proceeding of International Conference on Physiological Computing System $0HUOD   ³$GYDQFHV LQ WKHUPDO LPDJLQJ IRU PRQLWRULQJ SK\VLRORJ\ DQG VRFLDO LQWHUDFWLRQ XQREWUXVLYHO\ ³LQ  3roceedings of the International Conference on Psychophysiology6 $6KDVWUL$0HUOD37VLDP\UW]LVDQG,3DYOLGLV  ´,PDJLQJIDFLDOVLJQVRIQHXURSK\VLRORJLFDOUHVSRQVH³LQIEEE Trans. Biomed. Eng  %+HUQDQGH]*2ODJXH5+DPPRXG/7UXMLOOR(5RPHUR  ´9LVXDOOHDUQLQJRIWH[WXUHGHVFULSWRUVIRUIDFLDOH[SUHVVLRQUHFRJQLWLRQLQ WKHUPDOLPDJHU\´LQComputer Vision and Image Understanding  ± %51KDQDQG7&KDX  ³&ODVVLI\LQJ$IIHFWLYH6WDWHV8VLQJ7KHUPDO,QIUDUHG,PDJLQJRIWKH+XPDQ)DFH³LQIEEE Trans. Biomedical Eng.YROQR '$&ODXVL  ´$QDQDO\VLVRIFRRFFXUUHQFHWH[WXUHVWDWLVWLFVDVDIXQFWLRQRIJUH\OHYHOTXDQWL]DWLRQ´&DQ-5HPRWH6HQVLQJ9RO 1RSS '*%DUEHU()/H'UHZ  ³6$5VHDLFHGLVFULPLQDWLRQXVLQJWH[WXUHVWDWLVWLFVDPXOWLYDULDWHDSSURDFK´3KRWRJUDPPHWULF(QJLQHHULQJ DQG5HPRWH6HQVLQJ± ,3DYOLGLV-'RZGDOO16XQ&3XUL-)HLDQG0*DUEH\  ³,QWHUDFWLQJZLWKKXPDQSK\VLRORJ\´Comput. Vis. Image Understanding YROQR±± ,3DYOLGLV37VLDP\UW]L'6KDVWUL$:HVOH\=KRX3/LQGQHUHWDO  )DVWE\QDWXUH±KRZVWUHVVSDWWHUQVGHILQHKXPDQH[SHULHQFHDQG SHUIRUPDQFHLQGH[WHURXVWDVNVSci.Rep. ± /=KLOHLDQG:6KDQJIHL  ´(PRWLRQ5HFRJQLWLRQXVLQJ+LGGHQ0DUNRY0RGHOVIURP)DFLDO7HPSHUDWXUH6HTXHQFHV´6SULQJHU9HUODJ %HUOLQ+HLGHOEHUJ 0 &ROL / )RQWDQHOOD /,SSROLWL DQG $ 0HUOD   ³0XOWL UHVROXWLRQ ./( RI SV\FKRSK\VLRORJLFDO VLJQDOV´ LQ Proceedings of S.Co.2007,Book of Short Papers,6%19HQLFH3DGRYD± 0*DUEH\16XQ$0HUODDQG,3DYOLGLV  ³&RQWDFWIUHHPHDVXUHPHQWRIFDUGLDFSXOVHEDVHGRQWKHDQDO\VLVRIWKHUPDOLPDJHU\,” IEEE Trans. Biomed. Eng 00.KDQ5':DUGDQG0,QJOHE\  ³&ODVVLI\LQJSUHWHQGHGDQGHYRNHGIDFLDOH[SUHVVLRQVRISRVLWLYHDQGQHJDWLYHDIIHFWLYHVWDWHV XVLQJLQIUDUHGPHDVXUHPHQWRIVNLQWHPSHUDWXUH´Trans. Appl. Percept 50XUWK\DQG,3DYOLGLV  ´1RQFRQWDFW0HDVXUHPHQWRIEUHDWKLQJIXQFWLRQ´IEEE Eng. Med. Biol.Mag 50 +DUDOLFN . 6KDQPXJDP , 'LQVWHLQ   ³7H[WXUDO )HDWXUHV IRU ,PDJH &ODVVLILFDWLRQ´ IEEE Transactions on Systems, Man, and Cybernetics9RO1R± 7%RXUODL$5RVV&&KHQDQG/+RUQDN  ´$6WXG\RQ8VLQJ0LG:DYH,QIUDUHG,PDJHVIRU)DFH5HFRJQLWLRQ´LQ6HQVLQJ7HFKQRORJLHV IRU *OREDO +HDOWK 0LOLWDU\ 0HGLFLQH 'LVDVWHU 5HVSRQVH DQG (QYLURQPHQWDO 0RQLWRULQJ ,, DQG %LRPHWULF 7HFKQRORJ\ IRU +XPDQ ,GHQWLILFDWLRQ,;Proc. of SPIE9RO 76KHULGDQ  7HOHURERWLFV$XWRPDWLRQDQG+XPDQ6XSHUYLVRU\&RQWURO&DPEULGJH0$0,73UHVV 9(QJHUW $ 0HUOD -$ *UDQW ' &DUGRQH $ 7XVFKH DQG 76LQJHU   ([SORULQJ WKH XVH RI WKHUPDO LQIUDUHG LPDJLQJ LQ KXPDQ VWUHVV UHVHDUFKPLos ONE