Context-aware pervasive service composition and ... - MediaTeam Oulu

Pers Ubiquit Comput (2011) 15:291–303 DOI 10.1007/s00779-010-0333-5

ORIGINAL ARTICLE

Context-aware pervasive service composition and its implementation Jiehan Zhou • Ekaterina Gilman • Juha Palola Jukka Riekki • Mika Ylianttila • Junzhao Sun

•

Received: 7 December 2009 / Accepted: 24 May 2010 / Published online: 10 August 2010 Ó Springer-Verlag London Limited 2010

Abstract Incorporating service composition and pervasive computing into managing users’ complex everyday activities calls for the Pervasive Service Composition paradigm for everyday life. In this paper, we propose the concept of Context-Aware Pervasive Service Composition (CAPSC), which aims at enabling a pervasive system to provide user service compositions that are relevant to the situation at hand. We investigate CAPSC requirements and design a CAPSC architecture by taking into account context-aware peer coordination, context-aware process service adaptation, and context-aware utility service adaptation. We present a proof of concept application prototype as well. Keywords Service composition Service-oriented computing Context-awareness Pervasive computing

1 Introduction Pervasive Service Composition (PSC) is regarded as a Web services-based solution that realizes the pervasive computing paradigm. Context-awareness has been recognized as an essential characteristic of pervasive service composition. Context-Aware Pervasive Service Composition enables a pervasive system to provide the user service compositions that are relevant to contextual information on networks, devices, and users. These applications are flexible and adaptive to changes [1]. Although work on contextaware Web services has been reported, for example, [2–4],

J. Zhou (&) E. Gilman J. Palola J. Riekki M. Ylianttila J. Sun Department of Electrical and Information Engineering and Infotech Oulu, University of Oulu, Oulu, Finland e-mail: [email protected]

publications that address the following two questions are rare: What is context-aware pervasive service composition? How to accommodate context-awareness in pervasive service composition? Motivated by answering these two questions, we broaden the definitions of context and context-awareness in terms of pervasive service composition. Context encompasses not only information that can be used to characterize the situation of a user (user context) and a peer (peer context) but also information that can be used to characterize the situation of a service (service context) and an activity (process context). With peer, we refer to a service provider; when the services are offered by several service providers, their contexts are important as well. Consequently, an application reacts not only to user and service contexts, e.g. by switching content to a different display when a user moves, but also to peer and process contexts, e.g. by switching to the most suitable multimedia service component among the ones offered by different peers. We start by investigating the nature of Context-Aware Pervasive Service Composition (CAPSC). Then, we propose an architecture for context-aware service composition. Applications are classified into three types; applications performing context-aware peer coordination, applications performing process service adaptation, and applications performing utility service adaptation. Each application is dynamically assembled from services based on contextual information categorized into user context, peer context, process context, physical context, and service context. This paper is an extended version of a conference article [5], especially extending the initial CAPSC prototype design and implementation. The remainder of this paper is organized as follows: Sect. 2 defines terminology related to Context-Aware Pervasive Service Composition. Section 3 investigates a design process for building CAPSC

123

292

applications and categorizes these applications. Section 4 examines two scenarios of context-aware pervasive service composition, presents a requirement analysis for CAPSC architecture, and designs the architecture. A prototype is designed and implemented as a proof of the concept in Sect. 5. Related studies are overviewed in Sect. 6 and conclusions are drawn in Sect. 7.

Pers Ubiquit Comput (2011) 15:291–303

requesting, receiving, processing, and exchanging information between service world and a user for fulfilling a task. The user’s everyday activity differs from business activity in the sense that exchanging information, peer coordination, and service collaboration are considered not from financial, commercial, and industrial aspects. 2.2 Definitions for context-aware pervasive service composition

2 Definitions 2.1 Definitions for pervasive service composition Pervasive Service Computing is regarded as a Web servicecentric solution to facilitate modern human everyday activities with the emphasis on service composition [6]. Service composition, in its turn, is a process of building applications from services and controlling the execution of these applications. Pervasive Service Composition takes full advantage of pervasive computing and service computing to ubiquitously provide users with mobile services for managing their everyday activities. In the user generic activity model [1], we determine that each user task is part of an interaction session (i.e. task session) between the user and the service world (i.e. his/her surroundings). From the service world’s perspective, a task session includes two major phases: peer coordination and service collaboration. Peer coordination seeks appropriate parties (i.e. service providers) to cooperate in the task session and assembles the co-operators’ functions and competences to implement the task. When peer coordination is set down and services have been identified, service collaboration comprises of the identified services that run on the coordinated service peers. A peer is considered as an organization or party owning various computing resources. A peer that provides services is a service peer. Peers can form groups, join groups, and resign from groups. Web service is an autonomous, standard-based component whose public interfaces are defined and described using XML (Extensible Markup Language) and that supports interoperable machine-to-machine interaction over a network using mainly Web-based standards [7, 8]. In this paper, we use the terms ‘‘Web service’’ and ‘‘service’’ interchangeably. Mobile services refer to Web services developed and deployed for mobile phones and other hand-held devices. Mobile services are easily deployed and accessible across occasionally connected mobile devices. Mobile services provide similar computing and communication experiences across the desktop, the Internet, phones, and other devices. The user’s everyday behaviour (activity) refers to an interaction session of communicating with peers,

123

Many definitions have been given to context and contextawareness. Dey defines context as any information that can be used to characterize the situation of an entity [9]. An entity, in its turn, is a person, place, or object that is considered relevant to the interaction between a user and an application. Schilit et al. [10] refer context-awareness to the capability of mobile systems to sense their physical environment, i.e. their context of use, and adapt their behaviour accordingly. Schilit and Theimer [11] exemplify context as location, identities of nearby people and objects, and changes to those objects. Pascoe defines the context as the subset of physical and conceptual states of interest to a particular entity: context is all about the whole situation relevant to an application and its set of users [12]. A large amount of research for context-aware computing has been reported since the early 1990s. Early examples of such applications include Badge Location System [13], Personal Shopping Assistant [14], Context Sensitive Tourist GUIDE [15], Context Sensitive Nomadic Exhibition Guide Hippie [16, 17], and the HP Exploratorium Project [18]. Recent context-aware applications in smart environments include MediaCup [19], Chameleon Tables [20], ICrafter [21], Gaia [22, 23], and smart kitchen [24, 25]. We define context as the explicit and implicit information characterizing an interaction between a user and his/her service world. However, context is more than information characterizing a participated entity. It is also information utilized for initiating, controlling, and maintaining a task session. As a result, we define context as follows: Context is any information characterizing the situation of a task session or interaction between a user and his/her service world. Context is categorized into user context, peer context, process context, physical context, and service context. Schmidt et al. [26] list three categories of user context: information of the user (e.g. knowledge of habits and emotional state), the user’s social environment (e.g. colocation of others, social interaction, and group dynamics), and the user’s tasks (e.g. spontaneous activity, engaged tasks, and general goals). Peer context (i.e. service provider context) includes the peer’s service role and social role. Process context includes information about initiating,


controlling, and maintaining task sessions. Physical context includes information on physical surrounding environments (e.g. absolute position, relative position, and co-location) and physical conditions (e.g. noise, light, and pressure). Service context includes managerial and technical information about the service, such as service functions, network connectivity, communication bandwidth, and service cost. Context-aware computing is the ability of mobile user’s applications to discover and react to changes in the environment in which they are situated [11]. Pascoe identifies four generic contextual capabilities: sensing, adaptation, resource discovery, and augmentation [12]. We define context-aware computing as the process of collecting, using, and managing the various types of context mentioned above during a task session. Contextawareness enables computer systems to adaptively initiate and manage an interaction session with the user by correctly modelling and reasoning with the various types of context. Next, we discuss the four generic stages of context-aware pervasive computing: At the context sensing stage, contextual information is collected. Information about a specific context is usually gathered by different sensors, for example with global positioning system (GPS) receivers, acceleration sensors, and radio-frequency identification (RFID) readers. The accuracy and fidelity of context are important for context modelling and context-aware adaptation. At the context modelling stage, context information is interpreted and structured in a holistic, coherent, and systematic way. Approaches for context modelling are exemplified in [27]: Key-Value Models, Markup Scheme Models, Object-Oriented Models, Logic-Based Models, and Ontology-Based Models. At the context reasoning stage, context units are composed, and higher-level context data are deduced. Context reasoning can be categorized on two levels [28]: context inference/recognition and context reasoning. The reasoning technology is rooted in AI. Commonly used reasoning techniques include rule-base inference, logic-based inference, and machine learning methods. At the service composition stage, context-aware peer coordination responds to changes in the situation of service peers. Based on service context, context-aware service collaboration composes identified services that run on service peers.

3 CAPSC scenarios examination 3.1 Case 1. PSC weekly meeting Scenario: The PSC weekly meeting will start in 10 min. The project manager, Emilia, receives a message from

293

Sophia telling that, after all, she can attend the weekly meeting due to cancelling of a lecture and hence she can present her slides. Emilia changes the meeting agenda and reserves 15 min for Sophie’s presentation. Emilia goes to the meeting room and lays her terminal device on the table. The terminal device recognizes the situation, sets itself into silent mode and Emilia’s state as ‘‘in a meeting’’, and creates a connection with the desktop screen of Emilia’s seat. The desktop displays the documents related to this meeting. During Sophia’s slide presentation, her laptop warns that it will shut down in 30 s due to low battery. Emilia’s terminal smoothly switches Sophia’s slides to the meeting room’s desktop computer. Sophia ends her presentation with a video clip. The sound is too weak. The sound service adjusts itself and increases the volume. All participants enjoy the PSC weekly meeting. Examination: This scenario presents three categories of CAPSC applications. The three categories are explained in Sect. 4. Each of them more or less responds to changes and facilitates complex meeting activities. Table 1 lists a summary of the scenario with respect to a user activity, context, and CAPSC application type. For example, in the session of including Sophia’s presentation in the meeting agenda, peer coordination captures the message about Sophia’s attendance on time, knows that Sophia is a member of the project and rearranges the meeting agenda. In the session of transferring the presentation from a laptop to a desktop computer, a process service senses an exception taking place in the laptop and maintains Sophia’s presentation successfully. In the session of silencing Emilia’s terminal and increasing the speaker’s volume, two utility services adapt themselves to respond to Emilia’s situation and to the video’s sound level. 3.2 Case 2. Sharing user experience Scenario: After a week-long holiday in a Mediterranean country, Emilia makes an appointment with Sofia to share her experience and drink a cup of coffee. Emilia wants to show Sofia pictures taken during the holiday, to present her new boyfriend, Tom. Emilia requests her terminal to search for a quiet and full-facility coffee bar. When the terminal presents the alternatives, they decide to visit one coffee bar about 500 metres away, which is located near the city lake. Emilia uses her terminal device to pay their coffees and to rent wall display and printers facilities. She is going to use the display to show Sofia some holiday photographs while they drink coffee. When they arrive at the table, Emilia turns on the terminal’s photo album application and transfers it to the wall display. The wall display resolution is automatically adjusted. Emilia requests photographs about Tom and herself. Sofia comments the photographs. Emilia picks some of the photographs and prints them with the colour printer she

123

294


Table 1 The scenario of PSC weekly meeting in context of CAPSC User activity

Context

CAPSC application type

Sophia is a project member and is free at the meeting time

Peer coordination adaptation

PSC weekly meeting Join Sophia in the meeting Make Emilia’s terminal silent

Emilia is in a meeting situation

Utility service adaptation

Transfer a presentation from a laptop to a desktop computer

An exception occurs in a laptop and there is an alternative available

Process service adaptation

Increase a speaker’s volume

The sound is weak


Table 2 The scenario of sharing user experience in context of CAPSC User activity

Context

CAPSC application type

Locating a coffee bar

Quiet, full-facility and not far

Peer coordination adaptation

Requesting photographs about Tom and Emilia

Photographs about Tom and Emilia


Playing music while enjoying the view Printing colourful photographs

Enjoying the view instead of displaying photographs Printing colourful photographs

Process service adaptation Utility service adaptation

Sharing user experience

rented. When they end discussion and just enjoy the view, Emilia’s favourite music is automatically played. Examination: This scenario presents three categories of typical CAPSC applications as well. Each of them senses peer context, process context, and service context and responds to changes in order to facilitate its respective user activities. Table 2 lists the summary of the scenario. For example, in the session of locating coffee bar, peer coordination captures peer context and seeks a quiet and fullfacility coffee bar. In the session of requesting photographs about Tom and Emilia, a utility service is executed for searching photographs. In the session of playing background music, a process service recognizes that Emilia and Sophia stop discussing. The process service automatically plays background music. In the session of printing photographs, the utility service reacts to colour pictures by requesting a colour printer.

4 CAPSC composition application categories and architecture 4.1 Composition application categories In CAPSC, applications are composed by a process service from elementary services (i.e. utility services) to achieve a specific task. A utility service provides reusable functions related to processing data within new or legacy application environments and expresses technology-specific functionality. Erl identifies the characteristics of utility services, such as functionality exposure within a specific processing context [29]. The utility services can be provided and

123

maintained by different service peers. Process service links process logic to service interaction and realizes workflow management. All process services are controller services and required to compose other services to execute logic. Process services also have the potential to become the utility services. In CAPSC application development, service peers, process services, and utility services are the three core agents involved in service composition. Context-aware Pervasive Service Composition aims at supporting applications reacting to changing context. Applications can be classified based on the context changes as follows: First, peer coordination is required when the group of participating service peers needs to be changed. To accomplish a complex task in a pervasive service composition, seeking and grouping peers are required before composing and executing services. Many changes can occur, such as peer join, peer disconnection, and peer leave. Context-aware peer coordination is expected to manage those changes in the peer coordination level. Second, process service adaptation is required when the process services need to be changed. A process service introduces a parent level of abstraction to manage interaction to ensure that service operations are executed in a specific sequence. Many changes could take place in process logic, such as time constrains, exception handling, and structured activities. Third, utility service adaptation is required when the utility services need to be changed. Adaptation and reaction takes place in the utility service layer. Utility service reacts to context changes, such as location, temperature, weather, proximity, and bandwidth.

Pers Ubiquit Comput (2011) 15:291–303 Fig. 1 A typical life cycle of a CAPSC application

295

Context sensing

The CAPSC prototype implemented in Sect. 5 presents how process service reacts to the user’s context changes. 4.2 Architecture analysis CAPSC applications should adapt themselves to the changing needs of users and the dynamic service communication, computing and controlling environments that users may encounter while consuming services. To enable such applications, the CAPS architecture has to offer the following functionality. A typical life cycle of CAPSC applications is illustrated in Fig. 1. 4.2.1 Context sensing Context sensing is the process for collecting sensor data and recognizing from this data the context of a task session. Data can be collected from physical sensors measuring the environment or virtual sensors that monitor users’ calendars, for example. 4.2.2 Context modelling Context modelling is the process for the abstract description of services, processes, users, and peers participating in Context-aware Pervasive Service Composition and their relations. Context model provides data and activity constructors for perceiving and capturing the context of identifiable services, users, processes, and peers participated in one pervasive interaction session. This context will be processed and exchanged during context reasoning, peer coordination, and service collaboration. 4.2.3 Context reasoning Context reasoning is a computing process of determining the situation of a human–computer system relevant to an interaction session by examining the contextual information against the predefined context model. Many approaches have been developed and employed for context reasoning such as formal logic-driven reasoning [30] and ontology-driven reasoning [31]. 4.2.4 Context-aware service composition In this step, software applications are automatically composed from the pervasive environment’s services based on

Context modeling

Context reasoning

Context -aware service composition

the contextual information that context reasoning delivers. According to our given composition application categories, there are three optional application compositions. (1) Context-aware peer coordination. Peer coordination is a global activity responsible for seeking appropriate parties to cooperate in a task session. Peer coordination offers a means by which the rules for collaborating peers can be clearly defined and agreed. Context-aware peer coordination emphasizes processing and exchanging of peer context to compose an efficient application. (2) Context-aware process management. In an executable service composition (i.e. service collaboration), process service describes and controls the process logic of service interaction. Contextaware process management considers the changes in the process logic, such as conditional situations and device exceptions. The contextual information of a process is utilized by process service adaptation in order to maintain an interaction session. (3) Context-aware utility service computing. Utility service provides executable logic and realizes functions related to processing service context. Context-aware utility service computing pays attention to utility service adaptation into an interaction situation by examining service context. 4.3 Architecture design CAPSC architecture intends to accommodate the above functional requirements in Context-aware Pervasive Service Composition. The proposed architecture consists of two parts which are relevant for context-aware computing and service composition, respectively (Fig. 2). The major components in CAPSC architecture are specified as follows: Context sensing perceives contextual information and transmits it to the rest of the system. From a user’s perspective, the context information can be passively acquired from sensors, e.g. the user’s location information by GPS receiver, or actively sent by the user, e.g. a process control instruction. Context modelling acquires and works with the context to convert it to a usable form through interpretation. Context modelling analyses the context and represents it as an independent variable or combines it with other context information at a conceptual level. The context model is comprised of user context, object context, peer context, service context, and process context. These contexts consist of a name, type, and set of contextual states. Most

123

296


Fig. 2 CAPSC architecture design User activity

The user Composition request

Context sensing

Context modeling

Peer coordination

Context storage

Composition adaptor

Context reasoning

context models use standard methodologies for describing context, such as key-value pairs and ontologies [17, 27, 32]. Context reasoning makes decisions by examining contextual information or user requests against the predefined context model. Context storage stores context. Historical context information can be used to establish trends and predict future context values. Composition adaptor receives commands from context reasoning, chooses and performs context-aware service composition with respect to peer coordination, process adaptation or utility service adaptation. Peer coordination takes care about peer management within the composition process. It emphasizes processing and exchanging peer context in an efficient composition application. Process service adaptation considers the changes in the process logic and adapts process services for maintaining a service composition session. Utility service adaptation pays attention to utility service adaptation into an interaction situation by examining service context.

5 CAPSC Prototype design and implementation To prove the concept, we implemented a context-aware service composition prototype. In the prototype, we focused on the second CAPSC application category, i.e. process service adaptation, to emphasize how process service reacts to a user’s context changes. The prototype’s user interface is merely provided for entering the required configuration and context data. In a real pervasive application, part of the data entered here through the user interface would be provided by sensors and part would be specified in user profiles.

123

Process service adaptation

Service repository


5.1 Prototype description The prototype is an application that enhances images with digital content based on user profile and context. The application might be used, for example, to display a user’s situation on a display: the mood the user is in and the event. For a given image, the composed application detects peoples’ faces. It renders the images with smiley faces according to the user’s mood context or the relevant event. The images might be taken in real time from the user’s local environment or fetched from the Internet based on recognized context. If a service detecting mood from an image of a face would be available, the smileys could reflect the moods of the people in the image. The application can also, depending on the user’s choice, play background music according to mood or event context. 5.2 Prototype architecture The architecture of the prototype is presented in Fig. 3. The prototype consists of a client Web application, a Business Process Execution Language (BPEL) process service, a reasoning engine service, people detection service, and three utility services. All of them, except the client Web application, are independently running Web services. Client Web application allows the user to interact with the system. People Detection Service is a multimedia service developed by Zhou et al. [33] which is utilized to detect images with human faces. Utility Web services are used for service composition. Currently, we have three utility services as follows: Smiley service processes mood context and renders images with smiley faces. Event service processes user event context and renders images with extra information. Music service processes music preferences and

Pers Ubiquit Comput (2011) 15:291–303 Fig. 3 Context-aware multimedia service composition architecture

297

Client Web application

The user

5

1

People detection service

2

BPEL process service

3

Reasoning engine

Utility Web services 4 A loop for contacting utility services

mood or event context and presents background music for displaying images. Reasoning engine implements the logic of the application. In this prototype, the reasoning engine is aware of the semantics and dependencies between utility Web services. It utilizes the BPEL request information and decides which utility Web services need to be included in the composition, then it forwards the inferences to the BPEL process. BPEL service composition is the main control part of the application prototype that manages all interactions between services. It receives user inputs from the client application and queries people detection service and the reasoning engine to build a composition from utility Web services. Figure 4 shows the interactions between services and a UML sequence diagram. 5.3 Implementation In the prototype, all utility Web services are implemented with Java and deployed on the Glassfish 2.1 Application Server. Glassfish is chosen as a server because of its support of Enterprise JavaBeans (EJB) technology and BPEL Service Engine. Servers are running on 3-GHz PCs with 2 GB of memory. All utility Web services implement identical Web Service Description Language (WSDL) interface so they can be easily accessed as a part of a service composition. Information on user context, such as mood, event, and music, is mainly defined and processed for triggering adaptive composition. XML Schema is used to represent those context information. Client Web application is implemented with Hypertext Preprocessor (PHP) and deployed on Apache Web Server 2.2.12. PHP allows fast Web applications prototyping and together with Apache Web Server provides a stable execution environment.

Smiley service

Music service Event service

The reasoning engine is the Web service, which implements the logic of our prototype. It is implemented with WIN-Prolog [34], a Prolog compiler system for Windows-based computers, and wrapped with Java to the Web service. In our case, the reasoning engine is aware of semantics of application, utility Web services’ addresses and content. It has a knowledge base, containing all information related to utility Web services, such as their addresses, rendering pictures, or music files. The knowledge base contains facts and a set of rules (in Prolog terminology, compound terms, and clauses). Figure 5 shows the reasoning engine architecture. The reasoning engine core controls the reasoning engine. It gets input from the BPEL process, formats it with the message processor and launches the Win-Prolog inference execution. Then, it processes the reasoning result, transforms the result into the right format with the message processor, and sends the response back to the BPEL process. The context model defines the main concepts and their relations; for our reasoning engine, the context model is defined with terms and clauses. Table 3 gives the main concepts of our system we have defined with Win-Prolog. Clauses control the inference process of the reasoning engine. Basically, the reasoning engine gets a request from the BPEL process containing the user input and all detected faces’ coordinates in the original image. The engine then inferences the proper list of services together with their content as the response (corresponding rendering images for faces, music file to play, etc.) in order to be executed. Figure 6 presents the clause providing the BPEL process the corresponding services’ information with pictures and music files, based on a given mood and detected faces. So, by parsing the reasoning engine’s response, the BPEL

123

298


Fig. 4 The sequence diagram for the context-aware multimedia service composition

Reasoning engine Web service Input from BPEL Output to BPEL

Reasoning engine core

Message processor

Win-Prolog compiler Terms and clauses

Fig. 5 Reasoning engine architecture

process knows which services should be utilized in the composition and all information they require in order to be executed. For people detection service, we focus on cascadestructured algorithms offered by Viola-Jones embedded in OpenCV [24] and Wu’s [22]. Cascade classifiers are used for rare event detections, like face detection. A cascade face detector uses a sequence of node classifiers to distinguish faces from non-faces. The main advantage is its processing speed. A cascade detector can detect faces in real time. BPEL process service takes care of all interactions between services. BPEL composite application implements a BPEL module that consists of an XML Schema, WSDL description, and a BPEL process description. The composite application is developed with NetBeans IDE 6.5.1 software and is implemented with Java and deployed on a Glassfish 2.1 Application Server. It uses SOAP messages to

123

communicate with other service components’ WSDL interfaces. Key parts of BPEL process service definition are presented below as code fragments (Fig. 7). The essential actions of the BPEL process sequence are the following: 1. 2.

3. 4. 5. 6.

receive a service request from the client application, invoke People detection service and receive information about locations of people’s faces in a given image, invoke the Reasoning engine and receive a list of utility services’ endpoint references (listOfServices), create a loop for contacting each utility service referred in listOfServices, inside the loop, invoke the utility services and finally, reply to the client application.

The service composition is built based on the list of utility services, i.e. a list of endpoint references, given by the reasoning engine. The endpoint references consist of network address URIs. Each of the utility services implements an identical interface compared to each other. This means that the BPEL process does not have to be aware of which kind of service (e.g. event service or music service) it is invoking. The utility services are invoked inside a loop and on each round one service is invoked. Once the loop has finished, a return message is created and sent back to the client application. Figure 8a and b present the screenshots of the application prototype.


299

Table 3 Main concepts of context model Term

Description

Win-Prolog example

Service (N,A), N = name, A = address

Defines the service

service(‘image substitutor‘, ‘http://localhost:8080/SmileySubService/ SmileySubstitution‘).

Event (D,N), D = date, N = name

Defines the event

event(date(24,12), ‘Christmas‘).

Substitute (N,E,I,M), N = name of service, E = event or mood, I = list of images, M = list of music files

Defines which pictures and music are appropriate for a certain event or mood

substitute(‘image substitutor‘,‘happy‘, [face(‘happy.png‘), face(‘happy2.png‘)], [‘happy1.mp3‘,‘happy2.mp3‘]).

user_input(Mood,Music,Coordinates):service(ìmage substitutor`,Address), length(Coordinates,Faces_amount), (compare(=,Faces_amount,0)->(substitute(ìmage substitutor`,Mood,Z,M), first_face(Z,Result)); (substitute(ìmage substitutor`,Mood,Images,M), build_substitutions(ìmage substitutor`,Mood,Coordinates,Images,Result)) ), (compare(=,Music,'yes')->(length(M,L),compare(>,L,0)->R is ip(1+rand(L)), member(File,M,R), service(`music service`,MusicAddress), write_result(ok,Address,Result,MusicAddress,File,null)); compare(=,Music,'no')->write_result(ok,Address,Result,null) ).

Fig. 6 Context reasoning example with WIN-Prolog clauses

. 1 $nroOfServices

Fig. 7 Code fragments for key elements of the BPEL process service

123

300


Fig. 8 Screen shots of user interface for CAPSC prototype a Mood-based rendering in CAPSC prototype—rendering type selection (left) and rendering result (right). b Event-based rendering in CAPSC prototype—rendering type selection (left) and rendering result (right)

6 Related studies Context-awareness is a major characteristic of pervasive computing by which applications automatically adapt their operations to dynamic changes of context data. Apart from earlier research on context definitions [9, 10] and contextaware applications [13–17, 19] given in Sect. 2, recent studies are rapidly evolving in terms of context definition [35], context accuracy [36], security-relevant context [37], context management platform [38], extended contextaware applications such as battery management [39] and safety critical systems [40], and context verification [41]. To our knowledge, recent works have paid less attention to context-aware pervasive service composition and its application into the multimedia annotation domain, which is emphasized in our paper. For extending use of context, in terms of similarity judgment, Janowicz et al. [42] defined context as any kind of additional information which has an impact on similarity judgments at execution time. Six kinds of contexts are numerated, such as user context, undesired context and

123

desired context, application context, discourse context, representation context, and interpretation context. Each kind of contexts has its own impact on the resulting similarity values and their interpretation. To address emerging contexts in mobile privacy, Mancini et al. [35] focused on a notion of context in relation to privacy, which is subjectively defined by emerging socio-cultural knowledge, functions, relations, and rules and called this place, as opposed to a notion of context that is objectively defined by physical and factual elements, which is called space. For determining reliable contextual information in smart spaces, Lee et al. [36] proposed a pragmatic context classification and a generalized context modelling scheme based on sensor fusion techniques. For achieving effective security in a mobile computing environment, Johnson et al. [37] defined the term of security-relevant context and proposed the notion of shrink-wrapped security which couples a user’s situation with security. Li et al. [43] also introduced trust computing technology into pervasive computing and proposed terminal equipment architecture. For providing high interoperability for pervasive


computing, Schmidt et al. [38] introduced a generic context service as a standard Web service interface which allows adding context components and their sensors at run-time. Moreover, low-level context such as the process context and high-level context are paid attention in this paper. Hu et al. [44] introduced context management and proposed an autonomic context management system (ACoMS), which supports dynamic configuration and re-configuration of context information required by applications, and achieves fault tolerance without relying on redundancy. In [45], Gu et al. applied P2P approach to context reasoning for deriving and obtaining additional context data from lowlevel context data that may be spread over multiple domains in pervasive computing environments. To provide a uniform view of multiple distributed context sources, Xue et al. [46] studied schema matching for context-aware computing and proposed schema matching middleware, in which local schemas of context data from individual sources can be matched into a set of global schemas. In the extension of context-aware applications, Ravi et al. [39] proposed a system for context-aware battery management. The system extends the use of context information for battery management such as location to predict the next charging opportunity, discharge speedup factor, and crucial applications such as telephony. Similarly, to reduce power consumption in wearable computing systems, Murao et al. [47] proposed a context-aware system that changes the combination of accelerometers considering energy consumption. The proposed system can manage sensors to achieve a high accuracy of activity recognition with low energy consumption. To enable enterprises more flexible and adaptive, and survive within a more and more dynamic market, Wieland et al. [48] introduced workflow technology into pervasive production environment with the notion of smart workflow. Smart workflow is context-aware process which can adapt to changes in their physical context, e.g. machines, tools, and workers in a production environment. Bardram et al. [40] who extended contextawareness technologies in building safety critical systems present a context-aware patient safety system CAPSIS for the operating room. The system has a general awareness of the working context inside the operating room, such as the staff, the patient, equipment, and medical material. The system is able to provide the surgical team with important clinical data at the appropriate moment, as well as to detect potential safety critical situations. To provide locationspecific information for mobile visually impaired users, Stewart et al. [49] presented an urban orientation system Talking Points, based on the idea that an individual’s walking journey can be enhanced by providing contextual information about points of interest (POIs) along their route. The research revealed that the system could be enhanced by

301

accessing a combination of contextual information such as orientational (left, right, ahead, behind) information; annotations for barriers and obstruction markers; porting the client to a mobile phone platform (such as Google Android); and support for on-the-go user contributed content directly from the mobile device. For checking consistency of ontology-based context models, Schmidtke et al. [41] applied mereotopological theories [50] to formally specify context models. The hierarchical method shows that mereotopology is suitable to describe lattices, structures that are not trees, and continuous domains, such as ranges of sensor values and uncertainty regions around GPS coordinates. All of the related works studied only individual topics discussed in this paper, not CAPSC-related topics to the same extent that our paper does. In this paper, we extend the definitions of context and context-awareness in exploring pervasive service composition. Moreover, we develop the context-awareness architecture for pervasive service composition and apply it to a multimedia annotation application.

7 Conclusion and future work Context-Aware Pervasive Service Composition enables applications that are flexible and adaptive to changes. Many efforts have been made on context definition, quality of contextual information, context management platforms, and extended context-aware applications. This paper focuses on addressing Context-aware Pervasive Service Composition and accommodating context-awareness in Pervasive Service Composition. First, we define context and context-awareness in terms of pervasive service composition. We investigate a design process for building CAPSC applications and categorize these applications into three groups of peer coordination adaptation, process service adaptation, and utility service adaptation. After examining requirements, we propose CAPSC architecture for Context-aware Pervasive Service Composition. An initial prototype is designed and implemented as the proof of the concept, which emphasizes the implementation of process service adaptation. We identify peer-to-peer-based peer coordination as one interesting direction for our future work. This effort will focus on implementing pervasive service composition via a distributed service indexing environment. Acknowledgments This work was carried out in the SOPSCC project (Pervasive Service Computing: A Solution Based on Web Services), funded in the Ubiquitous Computing and Diversity of Communication (MOTIVE) program by the Academy of Finland, and ITEA2-CAM4Home project funded by the Finnish Funding Agency

123

302 for Technology and Innovation (Tekes). Many thanks to our colleagues Mika Rautiainen, Arto Heikkinen and Jouni Sarvanko for the prototype implementation.

References 1. Zhou J, Sun J, Rautiainen M, Davidyuk O, Liu M, Gilman E, Su X, Ylianttila M, Riekki J (2009) PSC-RM: reference model for pervasive service composition. In: Proceedings of the 4th international conference on frontier of computer science and technology, FCST 2009, Shanghai, 17–19 December, IEEE Computer Society, pp 705–709 2. Truong H-L, Dustdar S (2009) A survey on context-aware web service systems. Int J Web Inf Syst 5:5–31 3. Prezerakos GN, Tselikas ND, Cortese G (2007) Model-driven composition of context-aware Web services using ContextUML and aspects, pp 320–329 4. Mostefaoui SK, Hirsbrunner B (2003) ‘‘Towards a context-based service composition framework,’’ in Las Vegas, Nevada, 2003, pp 42–45 5. Zhou J, Riekki J (2010) Context-aware pervasive service composition. In: Proceedings of 1st international conference on intelligent systems, modelling and simulation, ISMS2010, 27–29 Jan, Liverpool, UK, IEEE Computer Society, pp 437–442 6. Zhou J, Riekki J, Ylianttila M (2009) Modeling service composition and exploring its characteristics. In: Proceedings of 3rd international workshop on web service composition and adaptation (WSCA-2009) in conjunction with IEEE ICWS2009, 6–10 July, LA, USA, IEEE Computer Society, pp 446–451 7. Lee TB, Hendler J, Lassila O (2001) The semantic web. Sci Am, May 8. Booth D, Haas H, McCabe F, Newcomer E, Champion M, Ferris C, Orchard D (2004) Web services architecture. W3C Working Group Note. February 9. Dey AK (2001) Understanding and using context. Personal Ubiquitous Comput 5(1):4–7 10. Schilit B, Adams N, Want R (1994) Context-aware computing applications. In: Proceedings of IEEE workshop on mobile computing systems and applications (WMCSA’94), Santa Cruz, CA, US, pp 85–90 11. Schilit BN, Theimer MM (1994) Disseminating active map information to mobile hosts. Netw IEEE 8(8):22–32 12. Pascoe J (1998) Adding generic contextual capabilities to wearable computers. In: Proceedings of second international symposium on wearable computers, pp 92–99 13. Want R, Hopper A, Veronica F, Gibbons J (1992) The active badge location system. ACM Trans Inf Syst 10(1):91–102 14. Asthana A, Crauatts M, Krzyzanowski P (1994) An indoor wireless system for personalized shopping assistance. In: Proceedings of workshop on mobile computing systems and applications (WMCSA 1994), Washington, DC, USA, IEEE Computer Society, pp 69–74 15. Cheverst K, Mitchell K, Davies N (1999) Design of an object model for a context sensitive tourist GUIDE. Comput Graph 23(6):883–891 16. Oppermann R, Specht M (2000) A context-sensitive nomadic information system as an exhibition guide. In: Proceedings of the second international symposium of handheld and ubiquitous computing, HUC 2000, Bristol, 25–27 Sept, Springer-Verlag, London, UK, pp 127–142 17. Henricksen K, Indulska J, Rakotonirainy A (2002) Modeling context information in pervasive computing systems. In: Goos G, Hartmanis J, Leeuwen JV (eds) Pervasive computing: lecture notes in computer science 2414. Springer, Berlin, pp 79–117

123

Pers Ubiquit Comput (2011) 15:291–303 18. Fleck M, Frid M, Kindberg T, O’Brien-Strain E, Rajani R, Spasojevic M (2002) From informing to remembering: ubiquitous systems in interactive museums. IEEE Pervasive Comput 1(1):13–21 19. Gellersen H, Beigl M, Krull H (1999) The MediaCup: awareness technology embedded in an everyday object. In: Proceedings of 1st international symposium on handheld and ubiquitous computing, HUC99, 27–29 Sept, Karlsruhe, Germany, SpringerVerlag, pp 308–310 20. Selker T, Arroyo E, Burleson W (2002) Chameleon tables: using context information in everyday objects. In: Proceedings of CHI ’02: extended abstracts on human factors in computing systems, Minneapolis, Minnesota, USA 2002, ACM New York, NY, USA, pp 580–581 21. Ponnekanti S, Lee B, Fox A, Hanrahan P, Winograd T (2001) ICrafter: a service framework for ubiquitous computing environments. In: Proceedings of the 3rd international conference on ubiquitous computing, UbiComp ’01, September 30–October 2, Atlanta, Georgia, Springer-Verlag, pp 56–75 22. Chetan S, Al-Muhtadi J, Campbell R, Mickunas MD (2005) Mobile gaia: a middleware for ad-hoc pervasive computing. In: Proceedings of second IEEE consumer communications and networking conference, CCNC 2005, 3–6 Jan, Las vegas, Nevada, USA, pp 223–228 23. Roman M, Hess C, Cerqueira R, Ranganathan A, Campbell RH, Nahrstedt K (2002) Gaia: a middleware platform for active spaces. SIGMOBILE Mob Comput Commun Rev 6(4):65–67 24. Kotsovinos E, Vukovic M (2005) Su-chef: adaptive coordination of intelligent home environments. In: Proceedings of joint international conference on autonomic and autonomous systems and international conference on networking and services, ICAS-ICNS 2005, 23–28 Oct, Papeete, Tahiti, pp 74–74 25. Vukovic M (2007) Context aware service composition. Dissertation/Thesis, Computer Laboratory, University of Cambridge 26. Schmidt A, Beigl M, Gellersen HW (1999) There is more to context than location. Comput Graph J 23(6):893–902 27. Strang T, Linnhoff-Popien C (2004) A context modeling survey. In: Proceedings of workshop on advanced context modelling, reasoning and management, UbiComp 2004—the Sixth International Conference on Ubiquitous Computing, Nottingham/ England 28. Sun J, Wu Z (2006) Context reasoning technologies in ubiquitous computing environment. In: Sha E, Han SK, Xu CZ, Kim MH, Yang LT, Xiao B (eds) Embedded and ubiquitous computing LNCS 4096. Springer, Berlin, pp 1027–1036 29. Erl T (2005) Service-oriented architecture (SOA): concepts, technology, and design. Prentice Hall, Upper Saddle River 30. Hunter A (1999) A default logic based framework for contextdependent reasoning with lexical knowledge. J Intell Inf Syst 16(1):65–87 31. Wang XH, Zhang DQ, Gu T, Pung HK (2004) Ontology based context modeling and reasoning using OWL. In: Proceedings of second IEEE annual conference on pervasive computing and communications workshops, March 14–March 17, Orlando, Florida, IEEE Computer Society, pp 18–22 32. Loke SW (2006) On representing situations for context-aware pervasive computing: six ways to tell if you are in a meeting. In: Proceedings of fourth annual IEEE international conference on pervasive computing and communications workshops, IEEE Computer Society, 13–17 March, Pisa, pp 5–39 33. Zhou J, Rautiainen M, Ylianttila M (2008) P2P SCCM: serviceoriented Community Coordinated Multimedia modeling multimedia applications as web services and experience. In: Proceedings of IEEE asia-pacific services computing conference (APSCC 2008), 9–12 Dec, Yilan, Taiwan, pp 145–149

Pers Ubiquit Comput (2011) 15:291–303 34. WIN-Prolog, http://www.lpa.co.uk/win.htm. Retrieved on 30.11. 2009 35. Mancini C, Thomas K, Rogers Y, Price BA, Jedrzejczyk L, Bandara AK, Joinson AN, Nuseibeh B (2009) From spaces to places: emerging contexts in mobile privacy. In: Proceedings of the 11th International Conference on Ubiquitous Computing, Ubicomp ’09, pp 1–10 36. Lee H, Choi JS, Elmasri R (2009) A classification and modeling of the quality of contextual information in smart spaces. In Proceedings of IEEE international conference on pervasive computing and communications, PerCom 2009, pp 1–5 37. Johnson GM (2009) Towards shrink-wrapped security: a taxonomy of security-relevant context. In: Proceedings of IEEE international conference on pervasive computing and communications, 2009. PerCom 2009, pp 1–2 38. Schmidt H, Flerlage F, Hauck FJ (2009) A generic context service for ubiquitous environments. In: Proceedings of IEEE international conference on pervasive computing and communications, PerCom 2009, pp 1–6 39. Ravi N, Scott J, Han L, Iftode L (2008) Context-aware battery management for mobile phones. In: Proceedings of sixth annual IEEE international conference on pervasive computing and communications, PerCom 2008, pp 224–233 40. Bardram JE, Norskov N (2008) A context-aware patient safety system for the operating room. In: Proceedings of UbiComp ’08: proceedings of the 10th international conference on ubiquitous computing, pp 272–281 41. Schmidtke HR, Woo W (2009) Towards ontology-based formal verification methods for context aware systems. In: Proceedings of the 7th international conference on pervasive computing, Pervasive ’09, pp 309–326 42. Janowicz K (2008) Kinds of contexts and their impact on semantic similarity measurement. In: Proceedings of sixth annual IEEE international conference on pervasive computing and communications, PerCom 2008, pp 441–446

303 43. Li C, Zhang Y, Duan L (2008) Establishing a trusted architecture on pervasive terminals for securing context processing. In: Proceedings of sixth annual IEEE international conference on pervasive computing and communications, PerCom 2008, pp 639–644 44. Hu P, Indulska J, Robinson R (2008) An autonomic context management system for pervasive computing. In: Proceedings of sixth annual IEEE international conference on pervasive computing and communications, PerCom 2008, pp 213–223 45. Gu T, Pung HK, Zhang DQ (2008) Peer-to-peer context reasoning in pervasive computing environments. In: Proceedings of sixth annual IEEE international conference on pervasive computing and communications, PerCom 2008, pp 406–411 46. Xue W, Pung H, Palmes PP, Gu T (2008) Schema matching for context-aware computing. In: Proceedings of the 10th international conference on ubiquitous computing, UbiComp ’08. pp 292–301 47. Murao K, Terada T, Takegawa Y, Nishio S (2008) A contextaware system that changes sensor combinations considering energy consumption. In: Proceedings of the 6th international conference on pervasive computing, pp 197–212 48. Wieland M, Kaczmarczyk P, Nicklas D (2008) Context integration for smart workflows. In: Proceedings of sixth annual IEEE international conference on pervasive computing and communications, PerCom 2008, pp 239–242 49. Stewart J, Bauman S, Escobar M, Hilden J, Bihani K, Newman MW (2008) Accessible contextual information for urban orientation. In: Proceedings of the 10th international conference on ubiquitous computing, pp 332–335 50. Varzi AC (2007) Spatial reasoning and ontology: parts, wholes, and locations. In: Aiello M, Pratt-Hartmann I, van Benthem J (eds) Handbook of spatial logics. Springer, Heidelberg, pp 945–1038

123