Jul 11, 2017 - Paul Ekman's six basic classes of emotions are most popular and remain the focus of researchers. 7/11/2017. Anger. Sadness. Fear. Surprise.
Exploiting IoT Services by Integrating Emotion Recognition in Web of Objects Muhammad Aslam Jarwar and Ilyoung Chong Dept. of Computer and Information Communications Engineering Hankuk University of Foreign Studies, Rep. of Korea E-mail: {aslam.jarwar, iychong}@hufs.ac.kr 7/11/2017
1
Overview Introduction Emotion Sources WoO Concepts WoO Functional Architecture WoO Layering Mechanism Emotions Aware IoT Services Knowledge Creation For Emotions Feature Extraction Tools Use Case Conclusion
7/11/2017
2
INTRODUCTION (1) In our daily life during performing various activities, we create & generate different types of data in large quantity. This data contains an important information regarding Activities Behavior in different situations Emotions Emotions plays an important role in our daily life, our actions depends on our emotions With correctly detection of emotions with context, we can better provide IoT services to user
7/11/2017
3
INTRODUCTION (2)
Anger
Sadness
Fear
Surprise
Love
Joy
The Scientist have categorized different types of emotions Paul Ekman's six basic classes of emotions are most popular and remain the focus of researchers
7/11/2017
4
USER CENTRIC IoT SERVICES Emotion
Activities
Activities
Emotions
IoT services provided on the basis of user’s activities and emotions These services includes isolation avoidance, health assistance, social care, emergency response services etc.
7/11/2017
5
DATA SOURCES FOR EMOTIONS
In IoT environment Emotion data generated from various sources Activities at Social media networks Physical activities Multimedia data sources (facial expressions, voice) 7/11/2017
6
WEB OF OBJECTS (WoO) CONCEPTS
WoO =
Web’s easy application creation
IoT’s abundance of connected smart objects
Web of Objects (WoO) An application level common platform Facilitates easy creation of cross-domain applications Simplifies object and application development, deployment, maintenance and operation for IoT infrastructures Provides user-centric IoT services by enabling object virtualization
7/11/2017
7
WoO FUNCTIONAL ARCHITECTURE
7/11/2017
8
ARCHITECTURE WoO PLATFORM
WoO is a very good platform for IoT services and application 7/11/2017
9
VIRTUALS OBJECTS
RWOs are virtualized to cope with heterogeneous data Harmonization among real world objects and VOs is performed for provision of emotion aware IoT service 7/11/2017
10
WoO LAYER MECHANSIM FOR SUPPORT OF EMOTION AWARE IoT SERVICES WoO is an IoT platform, which provides abstraction at Data Layer Abstraction at emotion data source by introducing Virtual Objects (Vos) Processing layer Abstraction at data processing for emotion recognition handled with Composite virtual objects (CVOs) Here CVO is kind of microservices which recognize the each emotion type individually Service layer This part helps in creation of knowledge for detecting user’s context The emotion aggregation is also performed at level 7/11/2017
11
EMOTION BASED IoT SERVICE PROVISIONING IN WoO RWK Emotion Detection Service Activity Recognition
Voice Detection
Motion Detection
CVO
CVO CVO
VO
7/11/2017
Service Request Analysis Machine learning
CVO
VO
CVO CVO
VO
VO
12
WoO ENABLED IoT PLATFORM FOR EMOTION BASED IoT SERVICES
7/11/2017
13
WoO ENABLED IoT PLATFORM FOR EMOTIONS AWARE IoT SERVICES
7/11/2017
14
KNOWLEDGE CREATION FOR EMOTION AWARE IoT SERVICES
The most important part is user contextual information for finegrained emotion analysis, which identified by recognizing user context 7/11/2017
15
FEATURE EXTRACTION OPEN SOURCE TOLLS FOR EMOTION Physiological Data e-Health Sensor Platform V2.0 for Arduino and Raspberry Pi Wrist mounted devices Textual Data IBM Waston Synesketch Stanford Sentiment Analysis Image Data OpenFace Java Emotion Recognition
7/11/2017
Voice Data Emovoice OpenEar Image Data
16
USE CASE: A PERSON IS MISSING TO HIS FAMILY
The user subscribed for emotion based IoT services User template is created WoO based model track his activities and provides activities and emotion based IoT services 7/11/2017
17
CONCLUSION In this paper we categorized the emotion data sources & input objects The input data objects was connected with WoO platform by support of virtualization without any proprietary protocols The emotion recognition of each data sources was performed at CVO level by microservices paradigm.