UBICOMP/ISWC ’17 ADJUNCT, SEPTEMBER 11-15, 2017, MAUI, HAWAII, USA
6FDODEOH0HWKRGVDQG7RROVWR6XSSRUW7KHUPRJUDSKLF 'DWD&ROOHFWLRQDQG$QDO\VLVIRU(QHUJ\$XGLWV 0DWWKHZ/RXLV0DXULHOOR 0DNHDELOLW\/DE_+XPDQ&RPSXWHU,QWHUDFWLRQ/DE 'HSDUWPHQWRI&RPSXWHU6FLHQFH 8QLYHUVLW\RI0DU\ODQG&ROOHJH3DUN PDWWP#FVXPGHGX (a)
(b)
(c)
(d)
(e)
Figure 1: (a) Professional energy auditor inspects an attic storage space [15], (b) UAV collected thermographic data [11], (c) Ground-based robot for indoor and outdoor thermographic data collection [4], (d) Thermographic data collection using cars [12], (e) Novice auditor retrospectively analyzes thermal imagery [16].
$XWKRU.H\ZRUGV
(QHUJ\ DXGLWLQJ KHOSV LGHQWLI\ EXLOGLQJ LQHIILFLHQFLHV WKURXJK ZDONWKURXJK LQVSHFWLRQV RQVLWH PHDVXUHPHQWV DQG FRPSXWHU VLPXODWLRQV >@ :LWK LPSURYHPHQWV WR KDQGKHOG LQIUDUHG VHQVRUV DQG WKHLU IDOOLQJ FRVWV HQHUJ\ DXGLWRUVDUHLQFUHDVLQJO\XVLQJWKHUPRJUDSK\GXULQJDXGLWV >@7KHUPRJUDSK\LVDGDWDFROOHFWLRQDQGYLVXDODQDO\WLFV WHFKQLTXH WKDW XVHV WKHUPDO FDPHUDV WR KHOS GHWHFW GLDJQRVH DQG GRFXPHQW SUREOHPV WKDW DUH YLVLEOH WR D WKHUPDO FDPHUD DV DQRPDORXV SDWWHUQV RI VXUIDFH WHPSHUDWXUHV >@ :KLOH WKH '2( UHFRPPHQGV WKHUPRJUDSKLFEDVHG HQHUJ\ DXGLWV FULWLFLVPV LQFOXGH WKDW LW UHPDLQV D TXDOLWDWLYH PHWKRG VXEMHFW WR WKH H[SHUWLVH RI WKH DXGLWRU DQG ODFNV VSHFLDOL]HG VRIWZDUH VXSSRUW WRROV DOJRULWKPV DQG DXGLW JXLGHOLQHV >@ 0RUHRYHU WKHUPRJUDSKLFEDVHGHQHUJ\DXGLWVDUHODERULRXV5HVHDUFK LQ WKLV DUHD KDV ODUJHO\ IRFXVHG RQ KRZ WR VFDOH WKHUPRJUDSKLFDVVHVVPHQWVWKURXJKWHFKQLFDODSSURDFKHVWR DXWRPDWLQJ GDWD FROOHFWLRQ e.g., XVLQJ URERWLFV >±@ )LJXUH EG DV D PHDQV RI UHGXFLQJ FRVWV DQG VFDOLQJ WKHUPRJUDSKLF DVVHVVPHQWV +RZHYHU WKHUH KDV EHHQ D VXUSULVLQJ ODFN RI DWWHQWLRQ JLYHQ WR GHYHORSLQJ PHWKRGV DQG WRROV WKDW VXSSRUW KXPDQRULHQWHG WKHUPRJUDSKLF GDWD FROOHFWLRQDQGDQDO\VLVDFWLYLWLHV )LJXUHDH
)RUPDWLYH,QTXLU\ 6XSSRUW7RROV 0RELOH'HYLFHV6HQVRU .LWV6XVWDLQDEOH+&, 7KHUPRJUDSK\ (QHUJ\(IILFLHQF\ $&0&ODVVLILFDWLRQ.H\ZRUGV
+P,QIRUPDWLRQLQWHUIDFHVDQGSUHVHQWDWLRQ HJ+&, 0LVFHOODQHRXV ,1752'8&7,21
%XLOGLQJVFRQVXPHRIHQHUJ\SURGXFHGLQWKH86DQG FRQWULEXWH DQ LQFUHDVLQJ SRUWLRQ RI WRWDO FDUERQ GLR[LGH HPLVVLRQV² LQ FRPSDUHG WR LQ > @ 2QH IDFWRU FRQWULEXWLQJ WR WKHVH LVVXHV LV EXLOGLQJ DJH 5HVLGHQWLDO EXLOGLQJV IRU H[DPSOH FRQVWLWXWH RI DOO EXLOGLQJV LQ WKH 86 DQG DUH RQ DYHUDJH RYHU \HDUV ROG >@ PRVWRIWKHVHEXLOGLQJVZHUHFRQVWUXFWHGXVLQJHQHUJ\ LQHIILFLHQW GHVLJQV DQG WKHLU PDWHULDOV KDYH GHJUDGHG RYHU WLPH 0RUHRYHU WKH 86 'HSDUWPHQW RI (QHUJ\ KDV VHW D JRDORIUHGXFLQJKRXVLQJHQHUJ\XVHE\XSWR>@7R PHHWWKLVJRDOUHQRYDWLRQVDQGUHWURILWVRIH[LVWLQJEXLOGLQJ VWRFN KDV EHFRPH D SUHVVLQJ QHHG 2QH HIIHFWLYH SUDFWLFH IRU PRWLYDWLQJ WKHVH LPSURYHPHQWV LV HQHUJ\ DXGLWLQJ ZKLFKKDVVHHQDUHVXUJHQFHRILQWHUHVWLQUHFHQW\HDUV>@ Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from
[email protected]. UbiComp/ISWC'17 Adjunct, September 11–15, 2017, Maui, HI, USA © 2017 Association for Computing Machinery. ACM ISBN 978-1-4503-5190-4/17/09...$15.00 https://doi.org/10.1145/3123024.3123193
,Q P\ GLVVHUWDWLRQ,SODQ WRH[DPLQH FXUUHQWO\ XQH[SORUHG GLPHQVLRQV RI VFDOLQJ WKHUPRJUDSKLF DVVHVVPHQW GLPHQVLRQVRIwho DQG time.7R GR WKLV ,SODQ WRGHYHORS QRYHOWRROVDQGPHWKRGVWKDWFDQ L LQFUHDVHWKHQXPEHURI SHRSOHZKRDUHDEOHWRSHUIRUPWKHUPRJUDSKLFDVVHVVPHQWV DQG LL H[SORUH ZKDW EHQHILWV LI DQ\ WHPSRUDO DQDO\VLV PLJKW RIIHU HQHUJ\ DXGLWLQJ 0RUH VSHFLILFDOO\ , ZLOO GHPRQVWUDWHWKDW by integrating image processing, machine learning, and information visualization techniques we can develop mixed-initiative support tools for both in-situ and temporal thermographic data collection and analysis that
360
UBICOMP/ISWC ’17 ADJUNCT, SEPTEMBER 11-15, 2017, MAUI, HAWAII, USA
LLL DWHQGHQF\WRH[DFHUEDWHWKHODERULRXVQHVVRIDQDO\]LQJ FROOHFWHG LPDJHU\ WR FUHDWH HQGXVHU UHSRUWV >@ )XUWKHUPRUH WKHUH KDV EHHQ OLWWOH UHVHDUFK LQWR WKH QHZ DQDO\VLV SRVVLELOLWLHV HQDEOHG E\ IUHTXHQW WHPSRUDO VFDQQLQJ 0\ ZRUN QRW RQO\ FRQWULEXWHV WR WKH DUHD RI DXWRPDWHG WKHUPRJUDSKLFGDWD FROOHFWLRQXVLQJ VHQVRU NLWV EXWDOVRH[SORUHVWHPSRUDODQDO\VHVZKLFKPD\EHHQDEOHG E\VXFKV\VWHPV
will allow end-users of varying skill to gain insights that lead to improvements in building energy efficiency. :KLOH WKH VSDFH RI WKHUPRJUDSK\ UHVHDUFK LV DQ DFWLYH RQH P\ ZRUN XQLTXHO\ IRFXVHV RQ SURYLGLQJ VXSSRUW IRU KXPDQ RULHQWHG GDWD FROOHFWLRQ DQG DQDO\VLV DFWLYLWLHV ZKLOH DWWHPSWLQJWRDQVZHUPXOWLSOHUHVHDUFKTXHVWLRQVLQFOXGLQJ How might off-the-shelf machine learning and image processing techniques aid interpreting thermal imagery? How might temporal thermography be incorporated into energy auditing activities? And, how might mixed-initiative support tools contribute to improved recommendations? 0RUHRYHU P\ ZRUN LV GLVWLQFW ZLWKLQ WKHUPRJUDSK\ UHVHDUFK DV P\ HYDOXDWLRQV LQYROYH WKH HQGXVHUV RI WKLV WHFKQRORJ\ )LQDOO\ VRPH RI WKH H[SHFWHG FRQWULEXWLRQV RI P\ ZRUN LQFOXGH DQ LPSURYHG XQGHUVWDQGLQJ RI FXUUHQW WKHUPRJUDSKLF HQHUJ\ DXGLWLQJ SUDFWLFHV QRYHO PHWKRGV DQG WRROV WR VXSSRUW HQGXVHUV DQG D W\SRORJ\ RI WKH LQVLJKWVJDLQHGIURPWHPSRUDOWKHUPRJUDSK\GDWD
7KHUPRJUDSKLF6XSSRUW7RROV
'HYHORSLQJ VXSSRUW WRROV IRU KXPDQ DXGLWRUV LV D JURZLQJ DUHD RI WKHUPRJUDSKLF UHVHDUFK 6HYHUDO V\VWHPV H[SORUH DXJPHQWLQJ KXPDQRULHQWHG GDWD FROOHFWLRQ DQG DQDO\VLV DFWLYLWLHV ZLWK ZHDUDEOHV > @ DXJPHQWHG UHDOLW\ GHYLFHV >@ DQG QHZ DQDO\VLV VRIWZDUH >@ 0\ LQWHUHVWV DUH LQ WDNLQJ DGYDQWDJH RI WKH XELTXLWRXV QDWXUH RI VPDUWSKRQHV ZKLFKDUHSDUWLFXODUO\DWWUDFWLYHWRROVIRUHQHUJ\DXGLWLQJDV WKH\ DUH HTXLSSHG RU FDQ EH HTXLSSHG ZLWK VHQVRUV LQFOXGLQJWKHUPDOFDPHUDVSURYLGHOLJKWZHLJKWFRPSXWLQJ FDSDELOLWLHVDQGJHQHUDWHHDVLO\VKDUHDEOHGDWD:KLOHWKHUH DUH VHYHUDO FRPPHUFLDO DSSOLFDWLRQV IRU WKHUPDO LPDJLQJ e.g., )/,5 21( WKH\ DUH OLPLWHG LQ KRZ WKH\ DVVLVW HQGXVHUVRIYDU\LQJVNLOODQGLQWKHW\SHVRIDQDO\VHVWKDW WKH\VXSSRUW0\UHVHDUFKLQQRYDWHVLQERWKDUHDV
5(/$7(':25.
+HUH , SURYLGH EDFNJURXQG RQ EXLOGLQJ WKHUPRJUDSK\ DV ZHOO DV EULHI GLVFXVVLRQV RQ VFDOLQJ WKHUPRJUDSKLF GDWD FROOHFWLRQDQGWKHUPRJUDSKLFVXSSRUWWRROV %XLOGLQJ7KHUPRJUDSK\
$V QRWHG LQ WKH LQWURGXFWLRQ WKHUPRJUDSKLFEDVHG HQHUJ\ DXGLWV DUH EHFRPLQJ PRUH FRPPRQ GXH WR WKH LQFUHDVHG DYDLODELOLW\ RI WKHUPDO FDPHUDV DQG WKHLU GHFUHDVHG FRVW 7KHUPDO FDPHUDV ZRUN E\ GHWHFWLQJ WKH HOHFWURPDJQHWLF UDGLDWLRQHPLWWHGE\DOOREMHFWVDERYHDEVROXWH]HUR>@7KH WKHUPDOGDWDLVDXWRPDWLFDOO\FRPELQHGZLWKLPDJHVIURPD FRQYHQWLRQDO FDPHUD WR SURGXFH D FRQWH[WXDOL]HG WKHUPDO LPDJHRUthermogram(QHUJ\DXGLWRUVXVHWKHUPDOFDPHUDV WR PHDVXUH VXUIDFH WHPSHUDWXUHV LQ ZDOOV URRIV FHLOLQJV DQG RWKHU SDUWV RI D EXLOGLQJ¶V HQYHORSH ZKLOH ORRNLQJ IRU LQFRQVLVWHQW SDWWHUQV GLVFRQWLQXLWLHV DQG RWKHU DQRPDORXV KHDW VLJQDWXUHV WKDW PD\ LQGLFDWH WKH SUHVHQFH RI DQ HIILFLHQF\LVVXH>@+RZHYHUEHIRUHVXUYH\LQJDEXLOGLQJ WKHUPRJUDSKHUVPXVWDVVHVVHQYLURQPHQWDOFRQGLWLRQVVXFK DVZHDWKHUZLQG+9$&RSHUDWLRQVDQGGLUHFWLRQLQWHQVLW\ RIWKHVXQ²DOORIZKLFKFDQSUHYHQWSURSHUVFDQV e.g.,WKH ,62 VWDQGDUG UHTXLUHV D PLQLPXP WHPSHUDWXUH GLIIHUHQWLDO RI&EHWZHHQWKHLQWHULRUDQGH[WHULRUWRSURSHUO\GHWHFW WKHUPDOLUUHJXODULWLHV>@
0(7+2'2/2*,&$/$3352$&+
0\ WKHVLV LV FRPSULVHG RI WKUHH WKUHDGV RI UHVHDUFK L IRUPDWLYH LQTXLULHV DLPHG DW EHWWHU XQGHUVWDQGLQJ SURIHVVLRQDO DQG QRYLFH WKHUPRJUDSKLF HQHUJ\ DXGLWLQJ SUDFWLFHV LL WKHGHYHORSPHQWRIDQDO\VLVPHWKRGVWKDWFDQ EH XVHG LQ PL[HGLQLWLDWLYH EXLOGLQJ WKHUPRJUDSK\ WRROV DQG LLL H[SORUDWLRQV LQWR WHPSRUDO WKHUPRJUDSLF GDWD FROOHFWLRQ DQG DQDO\VLV ,Q WKH ILUVW WKUHDG ZKLFK LV FRPSOHWHG , IRFXVHG RQ FRQGXFWLQJ VHPLVWUXFWXUHG LQWHUYLHZVREVHUYDWLRQV DQG H[SORUDWRU\ILHOG VWXGLHV)RU WKUHDGV WZR DQG WKUHH , DP FXUUHQWO\ GHYHORSLQJ PHWKRGV DQG WRROV IRU LQVLWX DQG WHPSRUDO GDWD FROOHFWLRQ DQG DQDO\VLV , ZLOO HYDOXDWH WKHVH WRROV WKURXJK ODEEDVHG XVDELOLW\ VWXGLHV DQG VPDOO GHSOR\PHQWV )ROORZLQJ WKHVH VWXGLHV , SODQ WR UHILQH WKH WRROV , KDYH GHVLJQHG DQG SHUIRUP ILQDO HYDOXDWLRQV )RU WKH LQVLWX WRROV , ZLOO HYDOXDWHWKHPE\FRQGXFWLQJDFRQWUROOHGEHWZHHQVXEMHFWV ILHOGVWXG\)RUWKHWHPSRUDOWRROV,ZLOOSHUIRUPPXOWLSOH FDVHVWXG\GHSOR\PHQWV%HORZ,VXPPDUL]HP\FRPSOHWHG DQGSURSRVHGZRUNFRUUHVSRQGLQJWRWKHVHWKUHHWKUHDGV
6FDOLQJ7KHUPRJUDSLF'DWD&ROOHFWLRQ
:KLOH SULRU ZRUN KDV GHPRQVWUDWHG WKH YDOXH RI WKHUPRJUDSKLFEDVHG HQHUJ\ DXGLWV e.g., >@ FROOHFWLQJ DQG DQDO\]LQJ WKHUPDO LPDJHU\ LV ODERULRXV DQG WLPH FRQVXPLQJ $V D UHVXOW PXFK RI WKH UHFHQW ZRUN LQ WKHUPRJUDSK\ UHVHDUFK KDV IRFXVHG RQ VFDOLQJXS GDWD FROOHFWLRQXVLQJDXWRPDWLF>@DQGVHPLDXWRPDWLF>@ URERWLFV V\VWHPV 7KHVH V\VWHPV SURPLVH WR PDNH WKHUPRJUDSKLF DVVHVVPHQWV RI XUEDQ HQYLURQPHQWV HDV\ SURYLGH GDWD DERXW LQDFFHVVLEOH DUHDV e.g., URRIWRSV DQG PDNH IUHTXHQW WHPSRUDO VFDQQLQJ SRVVLEOH +RZHYHU OLPLWDWLRQV WR WKHVH VROXWLRQV LQFOXGH L DQ LQDELOLW\ WR FRQILJXUHWKHHQYLURQPHQWIRUDQDFFXUDWHLQVSHFWLRQ LL D IRFXV RQ FDSWXULQJ H[WHULRU YLHZV RI EXLOGLQJ IDFDGHV DQG
&203(/7(':25.
6WXGLHV RI HQHUJ\ DXGLWLQJ IRFXV ODUJHO\ RQ SRWHQWLDO HQYLURQPHQWDO EHQHILWV >@ DQG RQ WKH EXLOGLQJ RZQHU¶V SHUVSHFWLYHV RI HQHUJ\ DXGLWV >@ +RZHYHU VWXGLHV RI SURIHVVLRQDOHQHUJ\DXGLWRUVWKHPVHOYHVDUHUDUHDQGVWXGLHV RI QRYLFH WKHUPRJUDSKHUV DUH YLUWXDOO\ QRQH[LVWHQW GHVSLWH WKH LQFUHDVLQJ DFFHVVLELOLW\ RI FRQVXPHURULHQWHG WKHUPRJUDSKLF WRROV 7R XQGHUVWDQG WKH SRWHQWLDO EHQHILWV DQG FKDOOHQJHV DVVRFLDWHG ZLWK WKH XVH RI WKHUPRJUDSK\ GXULQJ HQHUJ\ DXGLWV DFTXLUH IHHGEDFN RQ FXUUHQW DSSURDFKHV WR VFDOLQJ WKHUPRJUDSK\ ZLWK DXWRPDWLRQ DQG JDLQ LQVLJKWV LQWR WKH GHVLJQ RI IXWXUH WHFKQRORJLHV , FRPSOHWHGWKUHHVWXGLHVZLWKWKHUPRJUDSKHUV7KHILUVWWZR 361
SESSION: DOCTORAL SCHOOL
VWXGLHV SXEOLVKHG DW &+, >@ ZHUH IRFXVHG RQ WKH FXUUHQW UROH RI WKHUPRJUDSK\ LQ SURIHVVLRQDO HQHUJ\ DXGLWLQJ WKH\ LQYROYHG VHPLVWUXFWXUHG LQWHUYLHZV ZLWK SURIHVVLRQDO WKHUPRJUDSKHUV UHFUXLWHG IURP DFURVV WKH 8QLWHG 6WDWHV DQG DQ REVHUYDWLRQDO FDVH VWXG\ RI D UHVLGHQWLDO DXGLW 7KH WKLUG VWXG\ SXEOLVKHG DW &+, >@ ZDV DQ H[SORUDWRU\ ILHOG VWXG\ IRFXVHG RQ XQGHUVWDQGLQJQRYLFHDSSURDFKHVWREXLOGLQJWKHUPRJUDSK\ LW LQYROYHG ZHHNO\ WDVNV FRUUHVSRQGLQJ VXUYH\V DQG D GHEULHILQWHUYLHZV
1H[W , ZLOO GHYHORS D PL[HGLQLWLDWLYH FODVVLILFDWLRQ ZRUNIORZXVLQJP\FRUSXVRIWKHUPRJUDSKLFLPDJHVZKLFK ,EHOLHYHZLOOEHKHOSIXOWRXVHUVRIYDU\LQJVNLOO:KHQWKLV ZRUNIORZ GHWHFWV DQ DQRPDO\ LW ZLOO FDOFXODWH PHWULFV DERXW WKH GHWHFWLRQV LQFOXGLQJ ERWK WKHUPDO DQG PDWHULDO SURSHUWLHVWRKHOSGHWHUPLQHZKHWKHUWKHGHWHFWLRQLVZRUWK IXUWKHULQYHVWLJDWLRQ8VLQJWKHVHPHWKRGV,ZLOOGHYHORSD SURWRW\SHVPDUWSKRQHEDVHGWRROWREHHYDOXDWHGWKURXJKD EHWZHHQVXEMHFWV ILHOG VWXG\ 0\ ZRUN LQ WKLV WKUHDG ZLOO HYDOXDWH KRZ PL[HGLQLWLDWLYH WRROV PLJKW FRPSOHPHQW KXPDQDXGLWRUVDQGZKDWLPSDFWLIDQ\VXFKWRROVKDYHRQ UHQRYDWLRQDQGUHWURILWGHFLVLRQV
&RQWULEXWLRQV
8QOLNH RWKHU ZRUN LQ WKLV DUHD P\ SXEOLVKHG ZRUN LQ WKLV FRPSOHWHG UHVHDUFK WKUHDG RIIHUV D TXDOLWDWLYH SHUVSHFWLYH RQ HQHUJ\ DXGLWLQJ ZLWK D VSHFLILF HPSKDVLV RQ WKHUPRJUDSK\ $GGLWLRQDOO\LW SUHVHQWV WKHSHUVSHFWLYHVRI ERWKSURIHVVLRQDODQGQRYLFHWKHUPRJUDSKHUV7KHUHVHDUFK KDV FRQWULEXWHG L LPSURYHPHQWV WR RXU XQGHUVWDQGLQJ RI FXUUHQW KXPDQRULHQWHG WKHUPRJUDSKLF HQHUJ\ DXGLWLQJ SUDFWLFHV DQG WHFKQLTXHV LL D V\QWKHVLV RI IHHGEDFN RQ SURSRVHG DXWRPDWHG WKHUPRJUDSKLF V\VWHPV DQG LLL GHVLJQUHFRPPHQGDWLRQVIRUIXWXUHWKHUPRJUDSKLFWRROV
Progress. 8VLQJ WKH WKHUPDO LPDJHU\ FROOHFWHG E\ P\ UHVHDUFK WHDP DQG SDUWLFLSDQWV LQ WKH IRUPDWLYH VWXGLHV , KDYH EHJXQ GHYHORSLQJ D FODVVLILFDWLRQ ZRUNIORZ IRU DXWRPDWLFDOO\ GHWHFWLQJ UHJLRQVRILQWHUHVW EDVHG RQ FRPPRQ REMHFWV ZLWKLQ WKHUPRJUDSKLF LQVSHFWLRQ WDVNV e.g., DLU OHDNDJH DURXQG D ZLQGRZ RU GRRU 0\ FXUUHQW SLSHOLQH XVHV D +DDU &DVFDGHV ZLWK /RFDO %LQDU\ 3DWWHUQV DV SUHYLRXV ZRUN VXJJHVWHG WKH XWLOLW\ RI WKLV WUDLQLQJ DSSURDFK DQG WKH UHODWLYH HDVH RI LPSOHPHQWDWLRQ XVLQJ 2SHQ&97KLVLPSOHPHQWDWLRQKDVDOORZHGPHWRSURWRW\SH D EDVLF SURFHVV IRU XSORDGLQJ DQ LPDJH UXQQLQJ WKH FODVVLILHU DQG JHQHUDWLQJ GHVFULSWLYH VWDWLVWLFV DERXW WKH WKHUPDO GDWD FRQWDLQHG ZLWKLQ WKH UHJLRQVRILQWHUHVW , DP FXUUHQWO\ H[SORULQJ WKH XVH RI FRQYROXWLRQDO QHXUDO QHWZRUNV VSHFLILFDOO\ &DIIH DQG &DIIH ZKLFK RIIHU SUH WUDLQHG PRGHOV IRU UHFRJQL]LQJ QRW RQO\ REMHFWV EXW DOVR PDWHULDO SURSHUWLHV WKDW FRXOG RIIHU LQVLWX VXSSRUW GXULQJ WKHUPRJUDSKLF DVVHVVPHQW DFWLYLWLHV 1H[W VWHSV LQFOXGH EXLOGLQJDPL[HGLQLWLDWLYHVPDUWSKRQHDSSOLFDWLRQWKDWFDQ LQWHUIDFH ZLWK WKLV ZRUNIORZ SURYLGH XVHU LQSXW DQG PHWDGDWD DQG GLVSOD\ WKH SURFHVVHG LPDJH UHVXOWV WR HQG XVHUV )LJXUHD
352326(':25.
7KH UHPDLQGHU RI P\ GLVVHUWDWLRQ ZRUN DQG ZKDW , DP LQWHUHVWHG LQ GLVFXVVLQJ GXULQJ WKH 'RFWRUDO &ROORTXLXP IRFXVHV RQ LQYHVWLJDWLQJ L LQVLWX PHWKRGV DQG WRROV WR VXSSRUWHQHUJ\DXGLWVDQG LL PHWKRGVDQGWRROVWRVXSSRUW WHPSRUDOWKHUPRJUDSKLFGDWDFROOHFWLRQDQGDQDO\VLV+HUH , GLVFXVV P\ SURJUHVV HYDOXDWLRQ SODQV DQG H[SHFWHG FRQWULEXWLRQVIRUERWKUHVHDUFKWKUHDGV 0HWKRGVDQG7RROVIRU,Q6LWX7KHUPRJUDSK\
7KLV WKUHDG RI P\ UHVHDUFK H[SORUHV ZD\V WKDW PDFKLQH OHDUQLQJ DQG LPDJH SURFHVVLQJ FDQ VXSSRUW WKHUPRJUDSKLF GDWD FROOHFWLRQ DQG DQDO\VLV 0\ IRUPDWLYH UHVHDUFK VXJJHVWHG WKDW WKHUH DUH VHYHUDO DUHDV RI KXPDQRULHQWHG GDWD FROOHFWLRQ DQG DQDO\VLV WKDW FRXOG EHQHILW IURP WKHVH DSSURDFKHVLQFOXGLQJ L DXWRPDWLFDQRPDO\GHWHFWLRQ LL WKHUPRJUDSKLF LPDJH FRUUHFWLRQ e.g., DXWRPDWLFDOO\ FDOFXODWLQJ WKH HPLVVLYLW\ RI REVHUYHG PDWHULDOV LLL HQFRGLQJV RI HQYLURQPHQWDO GDWD DQG LY LQVLWX RQERDUGLQJIRUWKHUPRJUDSKLFWDVNV,ZLOOILUVWFROOHFWDQG DQDO\]HWKHUPDOLPDJHU\XVLQJFRQYHQWLRQDOWRROVIROORZHG E\H[SORULQJWKHXVHRIRIIWKHVKHOILPDJHSURFHVVLQJDQG PDFKLQHOHDUQLQJWHFKQLTXHVWRFODVVLI\UHJLRQVRILQWHUHVW
Evaluation. $IWHU IXUWKHU GHYHORSPHQW , ZLOO UHFUXLW QRYLFHSDUWLFLSDQWVWRSDUWLFLSDWHLQDZHHNILHOGVWXG\RI VPDUWSKRQHEDVHG WKHUPRJUDSKLF HQHUJ\ DXGLWLQJ WRROV VLPLODUWR>@3DUWLFLSDQWVZLOOFRPSOHWHDVKRUWHOLJLELOLW\ TXHVWLRQQDLUHDQGWKHQ,ZLOOUDQGRPO\GLYLGHWKHUHFUXLWHG SRSXODWLRQ LQWR FRQWURO DQG LQWHUYHQWLRQ JURXSV EDODQFLQJ GHPRJUDSKLFIDFWRUV e.g., DJHJHQGHU 3DUWLFLSDQWVLQWKLV EHWZHHQVXEMHFWV VWXG\ ZLOO EH SURYLGHG ZLWK D )/,5 21( WKHUPDO FDPHUD DWWDFKPHQW IRU WKHLU SHUVRQDO
Figure 2: (a) User investigates an area of their home and takes a picture, which is automatically analyzed; results are shown by tapping on the detected problems. (b) A temporal thermographic sensor kit collects data over several days and displays the data through an interactive information visualization web application.
362
UBICOMP/ISWC ’17 ADJUNCT, SEPTEMBER 11-15, 2017, MAUI, HAWAII, USA
VPDUWSKRQHV DQG DVNHG WR XVH HLWKHU WKH )/,5 21( DSSOLFDWLRQ RU P\ LQWHUYHQWLRQ DSSOLFDWLRQ 7KH\ ZLOO FRPSOHWHZHHNO\DXGLWLQJWDVNVZHHNO\VXUYH\VDERXWWKHVH DFWLYLWLHV DQG SDUWLFLSDWH LQ D GHEULHI VHPLVWUXFWXUHG LQWHUYLHZ VHVVLRQ $ EULHI IROORZXS VXUYH\ ZLOO EH DGPLQLVWHUHG RQH PRQWK ODWHU WR H[SORUH DQ\ ORQJWHUP LQIOXHQFHV IURP SDUWLFLSDWLRQ LQ WKH VWXG\ WKLV VXUYH\ ZLOO QRW RQO\ DVN LI WKH\ WULHG WR DGGUHVV VSHFLILF LVVXHV WKH\ QRWLFHGGXULQJWKHVWXG\EXWDOVRZKHWKHUWKHLUSHUVSHFWLYHV RQHQHUJ\XVHRUWKHEXLOWHQYLURQPHQWPD\KDYHFKDQJHG, ZLOOIROORZWKHVDPHDQDO\VLVSURWRFROXVHGLQP\SUHYLRXV VWXG\IRUDOOVLPLODUGDWD KRZHYHUZLWKWKHDGGLWLRQRIDQ LQWHUYHQWLRQ WHFKQRORJ\ , ZLOO VSHFLILFDOO\ ORRN IRU GLIIHUHQFHV EHWZHHQ WKH WZR JURXSV )RU WKH QHZ LQVWUXPHQWV i.e., IROORZXS VXUYH\ , ZLOO DQDO\]H RSHQ HQGHG TXHVWLRQV XVLQJ WKHPDWLF DQDO\VLV DQG D VHW RI TXDOLWDWLYHFRGHVLQDGGLWLRQWREDVLFFRXQWVDQGGHVFULSWLYH VWDWLVWLFVRIWKHFORVHGHQGTXHVWLRQV
HIIHFWV RI VRODU ORDGLQJ DQG FRQVLVWHQWO\ GHVLUHG QRUPDOL]LQJ WKH FRORU VFDOHV DSSOLHG WR WKHUPDO LPDJHV DFURVVWKHWHPSRUDOGDWDVHWWKH\ZHUHDQDO\]LQJ1H[WVWHSV LQFOXGH LPSURYLQJ WKH HDVH LQ ZKLFK WKH VHQVRU NLW FDQ EH GHSOR\HG DQG LQFRUSRUDWLQJ SDUWLFLSDQW IHHGEDFN LQWR WKH YLVXDOL]DWLRQWRROEHIRUHFRQGXFWLQJIXUWKHUHYDOXDWLRQV Evaluation. $IWHU IXUWKHU LWHUDWLRQ , ZLOO UHFUXLW SDUWLFLSDQWV DQG SURYLGH WKHP ZLWK WKH VHQVRU NLW ZKLFK WKH\ZLOOXVHWRDVVHVVHQHUJ\HIILFLHQF\LVVXHVLQORFDWLRQV RIWKHLUFKRRVLQJ:KLOH,ZLOOQRWLQVWUXFWWKHPVSHFLILFDOO\ RQKRZWRXVHWKHVHQVRUNLW,ZLOOUHFRPPHQGWKDWWKH\VHW XS WKH VHQVRU NLW WR UHFRUG GDWD RYHU WKH FRXUVH RI D IHZ GD\V RU D ZHHN $IWHU SHUIRUPLQJ GDWD FROOHFWLRQ SDUWLFLSDQWV ZLOO SDUWLFLSDWH LQ D VHPLJXLGHG DQDO\VLV VHVVLRQWRUHYLHZWKHLUGDWDXVLQJDQXSGDWHGYHUVLRQRIWKH YLVXDOL]DWLRQ WRRO DQG D ³think aloud´ SURWRFRO 'XULQJ WKHVH VHVVLRQV , ZLOO WUDFN PHWULFV VXFK DV WLPHWRILUVW LQVLJKW WKH SHUFHLYHG YDOXH RI WKHLU LQVLJKWV OLNHV DQG GLVOLNHV ZLWK UHVSHFW WR V\VWHP IHDWXUHV WKH W\SHV RI LQVLJKWVWKDWWKH\DUHDEOHWRJDLQDQGDQ\LPSHGLPHQWVWR WKHLU VXFFHVV $IWHU H[SORULQJ WKH GDWD SDUWLFLSDQWV ZLOO FRPSOHWHDVKRUWGHEULHIDFWLYLW\
Contributions. 7KHH[SHFWHGFRQWULEXWLRQVIURPWKLVWKUHDG RIUHVHDUFKLQFOXGH L DQRYHOLQVLWXWRRODQGPHWKRGVIRU FROOHFWLQJ DQG DQDO\]LQJ WKHUPRJUDSKLF LPDJHU\ LL D GDWDVHW RI ODEHOHG WKHUPDO LPDJHU\ IRU XVH LQ IXWXUH WKHUPRJUDSK\ UHVHDUFK LLL D SUDFWLFDO HYDOXDWLRQ RI RII WKHVKHOILPDJHSURFHVVLQJDQGPDFKLQHOHDUQLQJDOJRULWKPV WKDW VXSSRUW WKHUPRJUDSKLF DQDO\VLV DQG LY UHFRPPHQGDWLRQVIRUIXWXUHWKHUPRJUDSKLFV\VWHPV
Contributions.7KHH[SHFWHGFRQWULEXWLRQVIURPWKLVWKUHDG RI UHVHDUFK LQFOXGH L QRYHO WRROV DQG PHWKRGV IRU H[SORULQJ WHPSRUDO WKHUPRJUDSK\ GDWD LL D W\SRORJ\ RI LQVLJKWVJDLQHGIURPWHPSRUDOWKHUPRJUDSKLFDQDO\VLV DQG LLL UHFRPPHQGDWLRQVIRUIXWXUHWKHUPRJUDSKLFV\VWHPV
0HWKRGVDQG7RROVIRU7HPSRUDO7KHUPRJUDSK\
5HFHQW UHVHDUFK KDV VXJJHVWHG WKDW WHPSRUDO YLVXDOL]DWLRQV FDQ SURYLGH HQHUJ\ DXGLWRUV ZLWK LQVLJKWV DERXW HQHUJ\ HIILFLHQF\LVVXHV>@+RZHYHUVWXGLHVLQWKLVDUHDODUJHO\ IRFXVRQSURRIRIFRQFHSWYLVXDOL]DWLRQVZLWKLQWKHVSDFHRI KLVWRULFDOEXLOGLQJSUHVHUYDWLRQ>@7KHVHVWXGLHVJHQHUDOO\ GRQRWLQYROYHHQGXVHUVDVSDUWLFLSDQWVEXWUDWKHUIRFXVRQ WKH LQVLJKWV JDLQHG E\ WKH UHVHDUFKHUV WKHPVHOYHV 7KXV LW LV FXUUHQWO\ XQFOHDU LI WKHUH LV DFWXDO YDOXH IRU HQHUJ\ DXGLWRUV WR FROOHFW DQG DQDO\]H WHPSRUDO GDWD ZLWKLQ WKH FRQWH[WRIHQHUJ\DXGLWLQJ,QWKLVWKUHDGRIUHVHDUFK,ZLOO L GHVLJQ DQG EXLOG DQ HDV\WRGHSOR\ ORZ FRVW WHPSRUDO WKHUPRJUDSKLF VHQVRU NLW DQG LL GHYHORS D QRYHO LQWHUDFWLYHLQIRUPDWLRQYLVXDOL]DWLRQV\VWHP,ZLOOIRFXVRQ XVLQJ WKHVH WRROV LQ GHSOR\PHQWV ZLWK HQHUJ\ DXGLWRUV WR H[SORUH WKH SRWHQWLDO EHQHILWV RI WHPSRUDO WKHUPRJUDSKLF GDWD FROOHFWLRQ DQG DQDO\VLV ZLWKLQ WKH VSDFH RI HQHUJ\ DXGLWLQJ
5(6($5&+67$7( (;3(&7('%(1(),76
, DP D VL[WK \HDU 3K' FDQGLGDWH DW WKH 8QLYHUVLW\ RI 0DU\ODQG , H[SHFW WKRXJKWIXO IHHGEDFN RQ P\ UHVHDUFK PHWKRGRORJ\ WRROV DQG HYDOXDWLRQV , SODQ WR ILQLVK P\ SURSRVHGZRUNDQGGHIHQGP\GLVVHUWDWLRQLQ $&.12:/('*0(176
,WKDQNP\DGYLVRU'U-RQ()URHKOLFKIRUKLVJXLGDQFH 7KRXJKWKLVSDSHUFDUULHVRQHQDPHLWUHSUHVHQWVWKHZRUN DQG HIIRUWV RI P\ FROODERUDWRUV FRPSOHWHG ZRUN SDUWLDOO\ IXQGHGE\80'¶V2IILFHRI6XVWDLQDELOLW\ 5()(5(1&(6
$VKRN%DUGKDQ'ZLJKW-DIIHH&\QWKLD.UROODQG 1DQF\:DOODFH(QHUJ\HIILFLHQF\UHWURILWVIRU 86KRXVLQJ 5HPRYLQJWKHERWWOHQHFNVRegional Science and Urban Economics ±5HWULHYHG $XJXVWIURP KWWS ZZZVFLHQFHGLUHFWFRPVFLHQFHDUWLFOHSLL6
Progress. , KDYH FRQVWUXFWHG D VHQVRU NLW IRU FROOHFWLQJ WHPSRUDO WKHUPRJUDSKLF DQG HQYLURQPHQWDO GDWD , KDYH DOVR SURWRW\SHG D ZHE DSSOLFDWLRQ IRU YLVXDOL]LQJ WKLV GDWD WKDWFHQWHUVRQD3DUDOOHO&RRUGLQDWH3ORW )LJXUHE 8VLQJ WKHVH WRROV , KDYH FRQGXFWHG D XVDELOLW\ SLORW VWXG\ DQG D FDVHVWXG\GHSOR\PHQWLQVXSSRUWRIDQHQHUJ\DXGLWEHLQJ FRQGXFWHG RQ D XQLYHUVLW\ EXLOGLQJ >@ %DVHG RQ WKHVH VWXGLHV , OHDUQHG WKDW SDUWLFLSDQWV SUHIHUUHG EHLQJ DEOH WR UDSLGO\ DFFHVV WHPSRUDO LPDJHV E\ ORDGLQJ WKHP LQWR D WLPHOLQHYLHZ YHUVXVPDQXDOO\DFFHVVLQJLPDJHVLQDIROGHU YLHZ FRXOG TXLFNO\ LGHQWLI\ WUDQVLHQW FRQGLWLRQV e.g.,
0DULD'DQHVH8UVND'HPVDU1LFROD0DVLQLDQG 0DUWLQ&KDUOWRQ,QYHVWLJDWLQJ0DWHULDO'HFD\RI +LVWRULF%XLOGLQJVXVLQJ9LVXDO$QDO\VLVZLWK0XOWL 7HPSRUDO,QIUDUHG7KHUPRJUDSKLF'DWDArchaeometry ±
363
SESSION: DOCTORAL SCHOOL
6DYHULR'HEHUQDUGLV0LFKHOH)LRUHQWLQR$QWRQLR( 8YDDQG*LXVHSSH0RQQR$6\VWHPWR([SORLW 7KHUPRJUDSKLF'DWD8VLQJ3URMHFWHG$XJPHQWHG 5HDOLW\,QInternational Conference on Augmented Reality, Virtual Reality and Computer Graphics ±
0DWWKHZ/RXLV0DXULHOORDQG-RQ()URHKOLFK 7RZDUGV$XWRPDWHG7KHUPDO3URILOLQJRI%XLOGLQJVDW 6FDOH8VLQJ8QPDQQHG$HULDO9HKLFOHVDQG' UHFRQVWUXFWLRQ,QProceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication 8EL&RPS ¶$GMXQFW ±
*LUXP*'HPLVVH'RULW%RUUPDQQDQG$QGUHDV 1XFKWHU,QWHUSUHWLQJWKHUPDO'PRGHOVRI LQGRRUHQYLURQPHQWVIRUHQHUJ\HIILFLHQF\,Q Advanced Robotics (ICAR), 2013 16th International Conference on±
0DWWKHZ/RXLV0DXULHOOR/H\OD1RURR]DQG-RQ( )URHKOLFK8QGHUVWDQGLQJWKH5ROHRI 7KHUPRJUDSK\LQ(QHUJ\$XGLWLQJ &XUUHQW3UDFWLFHV DQGWKH3RWHQWLDOIRU$XWRPDWHG6ROXWLRQV,Q Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems KWWSV GRLRUJ
5LNNH*DGHDQG7KRPDV%0RHVOXQG7KHUPDO FDPHUDVDQGDSSOLFDWLRQV DVXUYH\Machine Vision and Applications ±
0DWWKHZ/RXLV0DXULHOOR0DQDVZL6DKD(ULFD %URZQDQG-RQ()URHKOLFK([SORULQJ1RYLFH $SSURDFKHVWR6PDUWSKRQHEDVHG7KHUPRJUDSKLF (QHUJ\$XGLWLQJ $)LHOG6WXG\,QProceedings of the 35th Annual ACM Conference on Human Factors in Computing Systems KWWSV GRLRUJ
