Exploiting IoT Services by Integrating Emotion ...

15 downloads 0 Views 2MB Size Report
Jul 11, 2017 - Paul Ekman's six basic classes of emotions are most popular and remain the focus of researchers. 7/11/2017. Anger. Sadness. Fear. Surprise.
Exploiting IoT Services by Integrating Emotion Recognition in Web of Objects Muhammad Aslam Jarwar and Ilyoung Chong Dept. of Computer and Information Communications Engineering Hankuk University of Foreign Studies, Rep. of Korea E-mail: {aslam.jarwar, iychong}@hufs.ac.kr 7/11/2017

1

Overview Introduction Emotion Sources WoO Concepts WoO Functional Architecture WoO Layering Mechanism Emotions Aware IoT Services Knowledge Creation For Emotions Feature Extraction Tools Use Case Conclusion

7/11/2017

2

INTRODUCTION (1) In our daily life during performing various activities, we create & generate different types of data in large quantity. This data contains an important information regarding Activities Behavior in different situations Emotions Emotions plays an important role in our daily life, our actions depends on our emotions With correctly detection of emotions with context, we can better provide IoT services to user

7/11/2017

3

INTRODUCTION (2)

Anger

Sadness

Fear

Surprise

Love

Joy

The Scientist have categorized different types of emotions Paul Ekman's six basic classes of emotions are most popular and remain the focus of researchers

7/11/2017

4

USER CENTRIC IoT SERVICES Emotion

Activities

Activities

Emotions

IoT services provided on the basis of user’s activities and emotions These services includes isolation avoidance, health assistance, social care, emergency response services etc.

7/11/2017

5

DATA SOURCES FOR EMOTIONS

In IoT environment Emotion data generated from various sources Activities at Social media networks Physical activities Multimedia data sources (facial expressions, voice) 7/11/2017

6

WEB OF OBJECTS (WoO) CONCEPTS

WoO =

Web’s easy application creation

IoT’s abundance of connected smart objects

Web of Objects (WoO) An application level common platform Facilitates easy creation of cross-domain applications Simplifies object and application development, deployment, maintenance and operation for IoT infrastructures Provides user-centric IoT services by enabling object virtualization

7/11/2017

7

WoO FUNCTIONAL ARCHITECTURE

7/11/2017

8

ARCHITECTURE WoO PLATFORM

WoO is a very good platform for IoT services and application 7/11/2017

9

VIRTUALS OBJECTS

RWOs are virtualized to cope with heterogeneous data Harmonization among real world objects and VOs is performed for provision of emotion aware IoT service 7/11/2017

10

WoO LAYER MECHANSIM FOR SUPPORT OF EMOTION AWARE IoT SERVICES WoO is an IoT platform, which provides abstraction at Data Layer Abstraction at emotion data source by introducing Virtual Objects (Vos) Processing layer Abstraction at data processing for emotion recognition handled with Composite virtual objects (CVOs) Here CVO is kind of microservices which recognize the each emotion type individually Service layer This part helps in creation of knowledge for detecting user’s context The emotion aggregation is also performed at level 7/11/2017

11

EMOTION BASED IoT SERVICE PROVISIONING IN WoO RWK Emotion Detection Service Activity Recognition

Voice Detection

Motion Detection

CVO

CVO CVO

VO

7/11/2017

Service Request Analysis Machine learning

CVO

VO

CVO CVO

VO

VO

12

WoO ENABLED IoT PLATFORM FOR EMOTION BASED IoT SERVICES

7/11/2017

13

WoO ENABLED IoT PLATFORM FOR EMOTIONS AWARE IoT SERVICES

7/11/2017

14

KNOWLEDGE CREATION FOR EMOTION AWARE IoT SERVICES

The most important part is user contextual information for finegrained emotion analysis, which identified by recognizing user context 7/11/2017

15

FEATURE EXTRACTION OPEN SOURCE TOLLS FOR EMOTION Physiological Data e-Health Sensor Platform V2.0 for Arduino and Raspberry Pi Wrist mounted devices Textual Data IBM Waston Synesketch Stanford Sentiment Analysis Image Data OpenFace Java Emotion Recognition

7/11/2017

Voice Data Emovoice OpenEar Image Data

16

USE CASE: A PERSON IS MISSING TO HIS FAMILY

The user subscribed for emotion based IoT services User template is created WoO based model track his activities and provides activities and emotion based IoT services 7/11/2017

17

CONCLUSION In this paper we categorized the emotion data sources & input objects The input data objects was connected with WoO platform by support of virtualization without any proprietary protocols The emotion recognition of each data sources was performed at CVO level by microservices paradigm.

7/11/2017

18

Thank You

7/11/2017

19