In this experiment, a totally polar opposite emotion has been ... Throughout the session, we recorded the thermal video using FLIR T420 at 30 frames ..... Emotion Recognition using Hidden marNov models from Facial temperature Sequences ...
Available online at www.sciencedirect.com
ScienceDirect Procedia Computer Science 76 (2015) 308 – 315
,(((,QWHUQDWLRQDO6\PSRVLXPRQ5RERWLFVDQG,QWHOOLJHQW6HQVRUV,5,6
,PSOHPHQWDWLRQRI*/&0)HDWXUHVLQ7KHUPDO,PDJLQJIRU+XPDQ $IIHFWLYH6WDWH'HWHFWLRQ $EG/DWLI0+D0G @ )URPWKHNHUQHOWKH*/&0ZLOOKDYHIRXUGLVSODFHPHQWGLVWDQFHVG DQGIRXURULHQWDWLRQVDQJOHVș 6RIRUHDFK*/&0VWDWLVWLFDOIHDWXUHVWKHUHDUHIHDWXUHVIHDWXUHV[ IHDWXUHVIRUVLQJOH 52, +HQFHIRUHDFK52,WKHUHDUHIHDWXUHV[LPDJHV IHDWXUHV 7KH+DSS\6DGDQG)HDUGDWDVHW KDYH IHDWXUHV UHVSHFWLYHO\ ZKLFK OHDGV WR D WRWDO RI IHDWXUHV VXEMHFWV DOO PDOH RYHUDOO IRU HDFK 52, 7KH GDWDVHW (PRWLRQ7DEOH WKHQ UHSUHVHQWHG LQ WKH IRUP RI IHDWXUHV [ 52,V 6XSUDRUELWDO 3HULRUELWDO1DVDODQG0RXWK 4.4. Classification )RUWKHFODVVLILFDWLRQSDUWVHYHUDO&ODVVLILHUVKDYHEHHQWHVWHGLQRUGHUWRDFKLHYHWKHEHVWFODVVLILFDWLRQUHVXOW $PRQJ WKRVH ZHUH 6XSSRUW 9HFWRU 0DFKLQH 690 .1HDUHVW 1HLJKERU .11 DQG 7UHH 7KH VL]H RI GDWDVHW (PRWLRQ7DEOH IHGLQWRWKHFODVVLILHUZDV[IHDWXUHV0RVWRIWKHFODVVLILFDWLRQSURFHVVHVZDVGRQHXVLQJ &ODVVLILFDWLRQ/HDUQHULQ0DWODED7KH52,VVHUYHGDVWKHSUHGLFWRUVZKLOHWKH(PRWLRQODEHOVDVWKHUHVSRQVH 7KH)ROG&URVV9DOLGDWLRQZDVFKRVHQIRUPRGHOYDOLGDWLRQ 5HVXOW 'LVFXVVLRQ )URPDOOWKHFODVVLILHUVWHVWHGLQFODVVLILFDWLRQVWDJH:HLJKWHG.11KDVVKRZQWKHEHVWFODVVLILFDWLRQDFFXUDF\ ZLWK RYHUDOO DFFXUDF\ 7KH VHFRQG KLJKHVW DFFXUDF\ VFRUH EHORQJ WR )LQH .11 ZLWK RYHUDOO DFFXUDF\ RI ZKLFKORZHUVWKDQ:HLJKWHG.11%HVLGHV)LQH*DXVVLDQDQG0HGLXP*DXVVLDQ690GLVSOD\VTXLWH LPSUHVVLYHFODVVLILFDWLRQUHVXOWZLWKDQGRYHUDOODFFXUDF\UHVSHFWLYHO\7KHFRPSDULVRQVRIWKHFODVVLILHU SHUIRUPDQFHDUHWDEXODWHGLQ7DEOHEHORZLQWHUPRIRYHUDOODFFXUDF\7KHGHWDLOHGDFFXUDF\RIWKHFODVVLILHULV WDEXODWHGLQ7DEOH7KH$8&ZHUHREWDLQHGIURPWKH5HFHLYHU2SHUDWLQJ&KDUDFWHULVWLF52& FXUYH7KHKLJKHU WKH$8&YDOXHWKHEHWWHUWKHDFFXUDF\RIWKHFODVVLILHUDVWKH52&FXUYHUHSUHVHQWVWKHVHQVLWLYLW\DVDIXQFWLRQRI IDOORXW 7DEOH&ODVVLILHU3HUIRUPDQFH&RPSDULVRQ &ODVVLILFDWLRQ0HWKRG
)HDWXUH
$FFXUDF\
690)LQH*DXVVLDQ
*/&0
6900HGLXP*DXVVLDQ
*/&0
7UHH&RPSOH[
*/&0
7UHH0HGLXP
*/&0
.11)LQH.11
*/&0
.11:HLJKWHG.11
*/&0
7DEOH'HWDLOHG$FFXUDF\E\&ODVVIRU:HLJKWHG.11&ODVVLILHU
&ODVV
735DWH
)35DWH
3UHFLVLRQ
5HFDOO
)0HDVXUH
52&$UHD$8&
+DSS\
6DG
)HDU
:HLJKWHG$YJ
314
M.H. Abd Latif et al. / Procedia Computer Science 76 (2015) 308 – 315
7DEOHWDEXODWHGWKHGHWDLOHGDFFXUDF\RI:HLJKWHG.11&ODVVLILHUE\FODVVHVRIHPRWLRQV7KHHPRWLRQFODVVHV DUH+DSS\6DGDQG)HDU7KH7UXH3RVLWLYH73 5DWHLVDOVRNQRZQDVVHQVLWLYLW\ZKHUHLWVPHDVXUHVWKHSURSRUWLRQ RISRVLWLYHVWKDWDUHFRUUHFWO\FODVVLILHGDVSRVLWLYH7KHKLJKHVW735DWHIURPWKHFODVVHVRIHPRWLRQEHORQJWR)HDU &ODVVZLWKFRUUHFWO\FODVVLILHGLQVWDQFHV7KH)DOVH3RVLWLYH)3 5DWHLVDPHDVXUHVRILQFRUUHFWO\LGHQWLILHG RUIDOVHDODUPZKLFKFRQWULEXWHWRW\SH,HUURULQVWDWLVWLF7KHVPDOOHVW)35DWHFRXOGEHREVHUYHGIURP)HDU&ODVV ZLWK 7KH 3UHFLVLRQ SDUDPHWHU LV DOVR NQRZQ DV 3RVLWLYH 3UHGLFWLYH 9DOXH 339 ZKHUH LWV PHDVXUHV WKH SURSRUWLRQRIWKHWUXHSRVLWLYHVDJDLQVWDOOWKHSRVLWLYHUHVXOW75DQG)3 /DVWO\WKH)0HDVXUHLVDSDUDPHWHUWKDW PHDVXUHVWKHKDUPRQLFPHDQRISUHFLVLRQDQGUHFDOO,WFDQEHXVHGDVDVLQJOHSHUIRUPDQFHLQGH[IRUWKHSRVLWLYH FODVVFODVVLILFDWLRQ )LJLOOXVWUDWHVWKHFRQIXVLRQPDWUL[IRU:HLJKWHG.11)URPWKHFRQIXVLRQPDWUL[VHYHUDOLPSRUWDQWYDULDEOHV FRXOGEHH[WUDFWHG7UXH3RVLWLYH73 7UXH1HJDWLYH71 )DOVH3RVLWLYH)3 )DOVH1HJDWLYH)1 7UXH3RVLWLYH 5DWH739 )DOVH1HJDWLYH5DWH)15 3RVLWLYH3UHGLFWLYH5DWH335 )DOVH'LVFRYHU\5DWH)'5 HWF )LJXUH&RQIXVLRQ0DWUL[:HLJKWHG.11 ,QDGGLWLRQWRWKDWWKHIROGFURVVYDOLGDWLRQSHUIRUPHGRQWKH:HLJKWHG.11&ODVVLILHUKDVUHVXOWHGLQN IROG ORVV ZKLFK GHPRQVWUDWHV WKH VPDOO RXW RI VDPSOH HUURU 7KH VPDOO RXW RI VDPSOH ORVV VKRZV WKDW WKH PRGHO SHUIRUPHG DFFXUDWHO\ DV WKH ILWWLQJ IRU HDFK ILW VHVVLRQ IROG DUH ZHOOILWWHG ZLWK PLQLPXP DYHUDJH HUURU 7KH REMHFWLYHRINIROGFURVVYDOLGDWLRQLVWRHVWLPDWHKRZDFFXUDWHO\WKHSUHGLFWLYHPRGHOZRXOGSHUIRUPLQSUDFWLFH :LWKWKHVPDOOUHVXOWHGNIROGORVVZHDUHSRVLWLYHDQGRSWLPLVWLFWKDWWKHSUHGLFWLYHPRGHOZRXOGSHUIRUPZHOOLQ UHDO OLIH VFHQDULR %\ FRQVLGHULQJ VHYHUDO IHDWXUHV YHFWRU EHLQJ FRQFDWHQDWHG WRJHWKHU WKH RYHUDOO SHUIRUPDQFH PLJKWEHLPSURYHGGXHWRDYDLODELOLW\RIVHYHUDOGLVWLQFWIHDWXUHV%HVLGHVWKHIHDWXUHVYHFWRUVGDWDVHWFRXOGEHSUH SURFHVV EHIRUH IHHG LQ FODVVLILFDWLRQ SKDVH XVLQJ 3ULQFLSOH &RPSRQHQW $QDO\VLV 3&$ RU /LQHDU 'LVFULPLQDQW $QDO\VLV/'$ WRUHGXFHWKHGLPHQVLRQRIWKHGDWDVHW)XUWKHUPRUHIRUUHDOOLIHDSSOLFDWLRQRIWKHUPDOLPDJLQJ VHYHUDO NQRZQ OLPLWDWLRQV H[LVW 7KH HQYLURQPHQWDO FRQGLWLRQ SOD\V WKH FUXFLDOUROH VLQFH WKH KRPHRVWDVLV DQG WKH FXWDQHRXV WHPSHUDWXUH DUH FRQWLQXRXVO\ UHJXODWHG ,Q RUGHU WR DYRLG DWWULEXWLQJ DQ\ SV\FKRORJLFDO YDOHQFH WR XQPLWLJDWHGWKHUPRUHJXODWRU\SURSHUFRXQWHUPHDVXUHVDQGFDXWLRQVVKRXOGEHDGRSWHG0HUOD %HVLGHVWKH RFFOXVLRQ RI 52,V E\ VSHFWDFOH RU KDLUEDQJ FRXOG UHQGHU WKH YLVLELOLW\ RI WKH WKHUPDO LPSULQW LQ WKH WDUJHWHG
M.H. Abd Latif et al. / Procedia Computer Science 76 (2015) 308 – 315
315
ORFDWLRQ7KHPHWDEROLFHIIHFWRIGLJHVWLRQDOVRDIIHFWHGWKHUHJXODWRU\RIFXWDQHRXVWHPSHUDWXUHRIWKHIDFLDOVNLQDV UHSRUWHGLQVHYHUDOVWXGLHV7KHPRGHOFRXOGEHLPSURYHGE\FRQVLGHULQJWKH52,VORFDWLRQWUDFNHUWRWUDFNWKH52,V ZKLFK PLJKW KHOS UHGXFLQJ WKH RXW RI IRFXV LPDJHV 2XU UHVHDUFK LV IDU IURP EHLQJ FRQFOXVLYH +RZHYHU WKHVH ILQGLQJV DUH XWPRVW SURPLVLQJ )RU IXUWKHU LPSURYHPHQW ZH ZRXOG QHHG PRUH WUDLQLQJ GDWD WKDQ ZKDW KDV EHHQ WHVWHG LQ WKLV SDSHU 7KLV GHYHORSPHQW RI DIIHFWLYH VWDWH GHWHFWLRQ IURP WKHUPDO LPDJLQJ FRXOG EH XVHIXO LQ +5, HVSHFLDOO\ZKHQGHDOLQJZLWKHPRWLRQDOO\FKDOOHQJHGLQGLYLGXDOHJ$XWLVWLF$VSHUJHU$'+'HWF :HEHOLHYH WKDWWKLVDIIHFWLYHVWDWHPRGHOFRXOGSURYLGHFRQWDFWOHVVQRQLQYDVLYHHPRWLRQPHDVXUHPHQWDQGKHOSVSV\FKLDWULVW ZLWKHPRWLRQDOVWDWHVHYDOXDWLRQDQGHQJDJHPHQWZLWKWKHVXEMHFW $FNQRZOHGJHPHQWV :HZLVKWRH[SUHVVRXUJUDWLWXGHWRWKH0LQLVWU\RI+LJKHU(GXFDWLRQ02+( IRUIXQGLQJWKHSURMHFWXQGHUWKH )XQGDPHQWDO 5HVHDUFK *UDQW 6FKHPH )5*6 *UDQW QR )5*6 2XU GHHSHVW DSSUHFLDWLRQ DOVR JRHVWRSRVWJUDGXDWHVWXGHQWDQGOHFWXUHUVLQYROYHGLQPDNLQJWKLVSURMHFWSRVVLEOH 5HIHUHQFHV $'L *LDFLQWR 0%UXQHWWL*6HSHGH$)HUUHWWLDQG$0HUOD 7KHUPDOVLJQDWXUHRIIHDUFRQGLWLRQLQJLQ PLOGSRVWWUDXPDWLFVWUHVV GLVRUGHUNeuroscience ± $0HUODDQG*/5RPDQL ³7KHUPDOVLJQDWXUHRIHPRWLRQDODURXVDODIXQFWLRQDOLQIUDUHGLPDJLQJVWXG\´LQProceedings of the 29th Annual International Conference of the IEEE EMBS Cite Internationale/\RQ)UDQFH$XJXVW $ 0HUOD ´5HYHDOLQJ SV\FKRORJ\ DQG HPRWLRQ WKURXJK WKHUPDO LPDJLQJ ³LQ Proceeding of International Conference on Physiological Computing System $0HUOD ³$GYDQFHV LQ WKHUPDO LPDJLQJ IRU PRQLWRULQJ SK\VLRORJ\ DQG VRFLDO LQWHUDFWLRQ XQREWUXVLYHO\ ³LQ 3roceedings of the International Conference on Psychophysiology6 $6KDVWUL$0HUOD37VLDP\UW]LVDQG,3DYOLGLV ´,PDJLQJIDFLDOVLJQVRIQHXURSK\VLRORJLFDOUHVSRQVH³LQIEEE Trans. Biomed. Eng %+HUQDQGH]*2ODJXH5+DPPRXG/7UXMLOOR(5RPHUR ´9LVXDOOHDUQLQJRIWH[WXUHGHVFULSWRUVIRUIDFLDOH[SUHVVLRQUHFRJQLWLRQLQ WKHUPDOLPDJHU\´LQComputer Vision and Image Understanding ± %51KDQDQG7&KDX ³&ODVVLI\LQJ$IIHFWLYH6WDWHV8VLQJ7KHUPDO,QIUDUHG,PDJLQJRIWKH+XPDQ)DFH³LQIEEE Trans. Biomedical Eng.YROQR '$&ODXVL ´$QDQDO\VLVRIFRRFFXUUHQFHWH[WXUHVWDWLVWLFVDVDIXQFWLRQRIJUH\OHYHOTXDQWL]DWLRQ´&DQ-5HPRWH6HQVLQJ9RO 1RSS '*%DUEHU()/H'UHZ ³6$5VHDLFHGLVFULPLQDWLRQXVLQJWH[WXUHVWDWLVWLFVDPXOWLYDULDWHDSSURDFK´3KRWRJUDPPHWULF(QJLQHHULQJ DQG5HPRWH6HQVLQJ± ,3DYOLGLV-'RZGDOO16XQ&3XUL-)HLDQG0*DUEH\ ³,QWHUDFWLQJZLWKKXPDQSK\VLRORJ\´Comput. Vis. Image Understanding YROQR±± ,3DYOLGLV37VLDP\UW]L'6KDVWUL$:HVOH\=KRX3/LQGQHUHWDO )DVWE\QDWXUH±KRZVWUHVVSDWWHUQVGHILQHKXPDQH[SHULHQFHDQG SHUIRUPDQFHLQGH[WHURXVWDVNVSci.Rep. ± /=KLOHLDQG:6KDQJIHL ´(PRWLRQ5HFRJQLWLRQXVLQJ+LGGHQ0DUNRY0RGHOVIURP)DFLDO7HPSHUDWXUH6HTXHQFHV´6SULQJHU9HUODJ %HUOLQ+HLGHOEHUJ 0 &ROL / )RQWDQHOOD /,SSROLWL DQG $ 0HUOD ³0XOWL UHVROXWLRQ ./( RI SV\FKRSK\VLRORJLFDO VLJQDOV´ LQ Proceedings of S.Co.2007,Book of Short Papers,6%19HQLFH3DGRYD± 0*DUEH\16XQ$0HUODDQG,3DYOLGLV ³&RQWDFWIUHHPHDVXUHPHQWRIFDUGLDFSXOVHEDVHGRQWKHDQDO\VLVRIWKHUPDOLPDJHU\,” IEEE Trans. Biomed. Eng 00.KDQ5':DUGDQG0,QJOHE\ ³&ODVVLI\LQJSUHWHQGHGDQGHYRNHGIDFLDOH[SUHVVLRQVRISRVLWLYHDQGQHJDWLYHDIIHFWLYHVWDWHV XVLQJLQIUDUHGPHDVXUHPHQWRIVNLQWHPSHUDWXUH´Trans. Appl. Percept 50XUWK\DQG,3DYOLGLV ´1RQFRQWDFW0HDVXUHPHQWRIEUHDWKLQJIXQFWLRQ´IEEE Eng. Med. Biol.Mag 50 +DUDOLFN . 6KDQPXJDP , 'LQVWHLQ ³7H[WXUDO )HDWXUHV IRU ,PDJH &ODVVLILFDWLRQ´ IEEE Transactions on Systems, Man, and Cybernetics9RO1R± 7%RXUODL$5RVV&&KHQDQG/+RUQDN ´$6WXG\RQ8VLQJ0LG:DYH,QIUDUHG,PDJHVIRU)DFH5HFRJQLWLRQ´LQ6HQVLQJ7HFKQRORJLHV IRU *OREDO +HDOWK 0LOLWDU\ 0HGLFLQH 'LVDVWHU 5HVSRQVH DQG (QYLURQPHQWDO 0RQLWRULQJ ,, DQG %LRPHWULF 7HFKQRORJ\ IRU +XPDQ ,GHQWLILFDWLRQ,;Proc. of SPIE9RO 76KHULGDQ 7HOHURERWLFV$XWRPDWLRQDQG+XPDQ6XSHUYLVRU\&RQWURO&DPEULGJH0$0,73UHVV 9(QJHUW $ 0HUOD -$ *UDQW ' &DUGRQH $ 7XVFKH DQG 76LQJHU ([SORULQJ WKH XVH RI WKHUPDO LQIUDUHG LPDJLQJ LQ KXPDQ VWUHVV UHVHDUFKPLos ONE