A Communication Protocol for Haptic Interaction

0 downloads 0 Views 417KB Size Report
different haptic applications into one generic protocol. First, the generic transport layer protocols (namely TCP and UDP) have been shown as not suitable to use ...
Evaluating ALPHAN: A Communication Protocol for Haptic Interaction Hussein Al Osman, Mohamad Eid, and Abdulmotaleb El Saddik Multimedia Communications Research Laboratory - MCRLab School of Information Technology and Engineering - University of Ottawa Ottawa, Ontario, K1N 6N5, Canada ABSTRACT In our previous work we introduced a novel application layer protocol, named ALPHAN, for haptic data communication. The protocol is characterized by three distinguished features: first, it is designed at the application layer to enable the application to define and control the networking parameters. Second, it is made highly customizable using XML-based descriptions. Finally, the protocol supports multi-buffering mechanisms to prioritize the communicated information. In this paper, we present a thorough evaluation of the protocol using a collaborative haptic game; the balance ball game that we developed is for this purpose. The performance metrics and the test bed of the protocol evaluation are also discussed. Finally, we comment on our findings and provide directions for prospective research. KEYWORDS: Tele-Haptics, networking descriptions, collaborative haptics.

protocol,

XML

INDEX TERMS: H.5.2 [Information Interfaces and Presentation]: User Interfaces – Haptic I/O 1

INTRODUCTION AND BACKGROUND

Haptics, a term derived from the Greek verb “haptesthai” (to touch), has come, in recent years, to refer to the science of sensing and manipulation of given environments through touch [1]. With the growing interest in haptics, many researchers have opted to introduce this technology into the field of networked virtual environments [2-4]. In this paper, such a field of research will be referred to as Collaborative Haptic Audio Visual Environments (C-HAVE). Recently, there has been a substantial research effort to benefit from the C-HAVE technologies in many fields including distributed training or tele-mentoring, surgical simulations [5], entertainment and gaming [6], computer-mediated social interaction [7], visual art and museums [8], assembly planning [9], among others. C-HAVE application developers are challenged by a number of issues at both the application level and the communication (networking) level. Firstly, at the application level, consistency assurance, access control, transparency, and stability are all common issues that have to be dealt with. On the communication level, Quality of Service (QoS) parameters such as network latency, jitter, packet loss, scalability, and compression are key aspects that have been investigated and researched. The haptic interaction requires simultaneous and interactive input and output with the haptic device at a significantly high update rate (1 KHz). Ideally, these updates should be transmitted at the same rate at which they are generated. The focus of the current research is on {HALOsman, eid, abed} @ mcrlab.uottawa.ca

Symposium on Haptic Interfaces for Virtual Environments and Teleoperator Systems 2008 13-14 March, Reno, Nevada, USA 978-1-4244-2005-6/08/$25.00 ©2008 IEEE

the design and evaluation of the networking protocol for telehaptic applications. Very few haptic communication protocols have been designed since it is difficult to capture the widely varying requirements of different haptic applications into one generic protocol. First, the generic transport layer protocols (namely TCP and UDP) have been shown as not suitable to use for haptic data communication [11]. Other transport layer protocols have been developed to make up for the shortcomings of TCP and UDP. The Synchronous Collaboration Transport Protocol (SCTP), proposed in [12], sends “normal messages” unreliably whereas key messages are sent reliably using sequence numbers. Smoothed SCTP [13] is heavily based on SCTP, yet it provides a mechanism for jitter smoothing. The Light TCP [11], as the name suggests, is inspired from TCP yet it supports the notion of key and non-key updates. Unlike our previous work [17] where we used simulated haptic data to measure the protocol performance, in this paper, we present a thorough evaluation of the protocol using a real collaborative haptic application. In general, a communication protocol serves two main purposes. Firstly, it permits the standardization of the communication between participating network end points. Secondly, it allows application developers to avoid the repetitive implementation of the same functionality in similar applications. The ALPHAN protocol (Application Layer Protocol for HAptic Networking) [17] is designed with the two aforementioned objectives in mind by providing a general framework through which C-HAVE applications can communicate. On the other hand, the protocol is strategically placed on the application layer, which allows it to be easily customizable in order to meet the different application requirements. The QoS parameters associated with a specific haptic application are listed and defined using the XML-based HAML format [10] (such as, the maximum allowable response time and jitter and packet loss, shared object priorities, security, trustworthiness and availability). HAML is an XML-based, technology-neutral, description language for haptic models and interactions. The protocol parses the HAML file and sets up the communication session accordingly. The HAML descriptions are assumed to be provided by the application itself that may include other aspects of the application such as the graphic and haptic models and rendering, haptic interface descriptions, among others. The ALPHAN protocol operates on top of the UDP since the latter does not impose any reliability or flow control schemes that do not fit the C-HAVE application requirements. Additionally, ALPHAN supports the notion of key updates that are widely supported by most of the haptic communication transport layer protocols, by implementing an application layer reliability mechanism that is only applied to such updates, while “normal updates” remain unaffected. ALPHAN also makes use of the Multiple Buffering (MB) scheme. In this scheme, every object in the C-HAVE environment is attributed to a sending buffer. Allocating a buffer for each object permits the decoupling of update transmission for different objects, especially if they are independent from each other or they need to be prioritized based on user and/or application preferences.

361 Authorized licensed use limited to: University of Ottawa. Downloaded on November 11, 2008 at 16:06 from IEEE Xplore. Restrictions apply.

Figure 1. ALPHAN header format

This paper presents the procedure we used to evaluate the protocol through the implementation of a C-HAVE gaming application that runs on top of the ALPHAN, namely the balance ball game. The remainder of the paper is organized as follows: in section 2, we introduce the ALPHAN protocol and discuss its comprising features. Section 3 presents the implementation aspects of the balance ball C-HAVE game application in addition to the evaluation metrics and test bed. In section 4, we discuss and analyze the gathered results. Finally, we conclude in section 5 with a contribution summary and our immediate future work. 2

ALPHAN PROTOCOL

The ALPHAN protocol operates on top of UDP. UDP is chosen for its simplicity and speed. It does not impose any reliability schemes that are not usually needed in real time applications. Furthermore, a summary of the frame fields and their descriptions are listed in Table 1. Table 1: Frame fields description

Field

Bits

Description

V

2

The version of the protocol; set to 0 for the current version

H

1

This bit is set if the data is haptic, and is reset if the data is graphic

C

1

Reporting collision state: the payload has position and force information if 1, position only if 0

E

1

Not relevant for haptics. Indicate the last fragment of a packet for graphics data

Type

3

Payload Type

8

Sequence Number

16

Identify each packet and see if any packets were lost or delivered out of sequence

Participant ID

16

Identifies each participant in the C-HAVE environment.

Object ID

16

Identifies each object in the C-HAVE environment.

Fragment Count

16

Not relevant for haptics. Identify the number of fragments for graphic data

Length

16

Timestamp

32

The type of haptic data: I, P, B frame and state query types for late comers Indicates single/multiple points of interaction for haptics and object types for graphics (rigid, deformable, etc.)

Indicates the length of the payload Time at which the event in the payload should be executed. Important for the implementation of a local lag mechanism.

2.1 Application Layer Protocol The ALPHAN protocol operates on the application layer. To further improve the customizability of the protocol, the generic quality of service (QoS) parameters associated with a specific

haptic application are listed and defined in HAML format. ALPHAN operates on the application layer and thus application layer framing is performed. Figure 1 presents the haptic frame header format as defined in the current version. 2.2 Multiple Buffer Scheme ALPHAN utilizes a multiple buffering scheme where each object in the environment, whether it is haptic or graphic, is allocated a sending buffer. Each sending buffer consists of two queue data structures, a sending queue and a retransmission queue. Since ALPHAN is an application layer protocol, these data structures operate on the application layer, especially that UDP does not provide any queuing schemes. The sending queue holds all the updates to be sent over the network. The retransmission queue holds copies of the key updates from the time they are sent, until they are acknowledged. This enables resending the key update message in case the transmission of the packet has failed. Attributing a buffer for each object has two key advantages. Firstly, it is possible to prioritize the buffers. The priority of the buffer corresponds to the importance of the object in the environment. Objects with higher priority will get precedence during the transmission of updates. This provides the protocol with a greater degree of customizability. Secondly, attributing a retransmission queue for each object makes the protocol more flexible. In the single buffer approach, copies of all the key updates that are transmitted are queued in the same retransmission queue. If the sending data rate is high enough, the queue will be full of obsolete key updates in the retransmission queue since the application has probably produced newer ones. For graphic objects, the size of the retransmission queue can be set to one whereas for haptic objects the size can be varied according to the object’s nature and application needs. 2.3 Application Layer Reliability The application layer reliability is implemented only for key updates. When a key update is sent, a timer is started; if a timeout occurs before the receipt of the acknowledgement, it is resent again. 2.3.1 Messages Classification The communication update messages that are exchanged between haptic entities are classified based on their importance. The principle of I, P and B packets is partially borrowed from the Motion Picture Experts Group (MPEG) standard [19]. The I packets are sent reliably and are considered to be key packets. P packets are encoded differentially based on previous I or P packets while B packets are encoded based on previous and future I and/or P packets. P and B frames are considered to be “normal updates” and therefore are sent unreliably. The rate at which I and P packets are sent can be optimized to suit the network condition. Such a form of encoding can save significant network resources. Furthermore the way the I, P, and B packets are defined is left to the application developer to decide upon. This makes the protocol more generic and abstract to accommodate different message classification criteria.

362 Authorized licensed use limited to: University of Ottawa. Downloaded on November 11, 2008 at 16:06 from IEEE Xplore. Restrictions apply.

2.3.2 Local Lag The local lag scheme is proposed in this protocol for consistency maintenance. In order to maintain the generality of the protocol, this feature is kept optional. Other consistency schemes, such as dead reckoning, could be easily implemented on top of ALPHAN. Nonetheless, the use of local lag not only facilitates consistency maintenance but also helps to smooth jitter. Choosing the most efficient local lag value is very important when using this scheme. As we have previously discussed, this choice is dependent on the delay in the network and on the maximum allowable response time. The latter can be read from the HAML description file of the haptic application and thus the protocol can be dynamically customized to fit the needs of the application. Simulation Timer

Graphic Loop Network Sensing Module

NTP Module

Protocol Manager Haptic Loop Simulation Module

Control Module

Sender Module

Receiver Module Update Decoder

Sending Buffers

Update Relayer

Key Update Timing Module Network Communication Module

Figure 2. High level view of the library architecture

2.3.3 Library Architecture A code library has been implemented based on the ALPHAN specifications. Figure 2 shows a high level view of the library architecture. The Protocol Manager Module is at the basis of the architecture and most other modules communicate with it. The C-HAVE application interacts only with the Protocol Manager module for all its communication needs.

amount of water is dropped from the bucket. In our application, a virtual adaptation of the game is developed where the bucket is replaced with a ball. We named this game the “Balance Ball” game. A snapshot of the game is shown in Figure 3. The virtual board is held from each side by a haptic device. The two haptic devices reside on geographically distant workstations. The performance of the players is graded according to two variables: the time it takes the players to complete a particular task and the variations of the ball position around the center of the board. In order to grab an end of the board, the player must point the haptic proxy on that end and press a button on the haptic device. This will attach the haptic proxy to the board’s end. The simulation game sends the position of the haptic device to the other party and the haptic rendering is performed locally at each side of the network. To ensure consistency, the graphic information of both scenes are also exchanged yet at a lower rate. As per haptic rendering, the application is developed in C++ using the OpenHaptics API [18]. Figure 4 presents an overview of the forces at stake. F1 and F2 are the forces felt by player 1 and player 2 respectively. A player holding one end of the board would experience many forces that mimic the real world feeling. The gravity force generated by the weight of the board is constant during the simulation since at any point in time, the player is always carrying half of the board weight. The gravity force generated by the weight of the ball is calculated according to the position of the ball on the board. It is natural that the player closer to the ball would carry more of its weight. The mutual force applied by one player on the board is instantly felt by the other. A constraint force is also applied to restrict the movement of the player by adhering to the physical limitations of the board. Other forces are also felt by the user when a collision occurs. The friction coefficient that dictates the friction forces between the ball and the board is an important factor since it affects the velocity of the ball movement and thus the difficulty level of the game.

Figure 4. Forces overview

Figure 3. The “Balance Ball” game snapshot

3

BENCHMARKING APPLICATION

The application implemented in this study is inspired by the children’s game bucket on a board. A bucket is filled with water and placed on a long wooden board that is held by two players from each side. The two participating players must pass over obstacles while carefully balancing the board so that a minimal

3.1 Implementation Details As we have previously mentioned, only position information is exchanged between the two ends of the communication. It is assumed that the ball is owned by workstation A and thus it continuously sends the position of the ball to workstation B at the graphic loop rate. Both ends of the communication send the position of their proxies respectively to each other. Before attaching a haptic proxy to a board edge, its position information is sent at the graphic loop rate, after attaching it, it is sent at the haptic loop rate. These position updates will now be used to calculate forces on the other side and thus must be frequently sent. The incoming proxy position is used to calculate the forces that the sender player has exerted on the receiver player by moving her/his proxy from point x to point y. These are called the mutual forces. Equation 1 is used to calculate the mutual force by relying on previously produced forces in order to avoid sudden jumps caused by either network loss or jitter. Fm represents the mutual force and λ is a heuristic value between 0 and 1.

363 Authorized licensed use limited to: University of Ottawa. Downloaded on November 11, 2008 at 16:06 from IEEE Xplore. Restrictions apply.

Figure 5. The experimental setup

FinalForce = λ × Fm(t ) + (1 − λ ) × Fm(t − 1)

(1)

In order to conserve bandwidth, several “normal updates” are buffered and sent in one packet. This method also reduces the overhead created by the addition of a header for each update. On the other hand, buffering a large amount of updates before transmitting them can result in hindering the responsiveness of the application. Collaborative haptic applications in particular cannot afford major increases in response time since this would destabilize the application. Also, it is highly discouraged to produce packets larger than the Maximum Transmission Unit (MTU) since this would result in fragmentation on the Network Layer. In the case of fragmentation, if one fragment is lost, the whole packet is discarded. Conversely, a packet can only hold one key update. Key updates are often chosen using heuristic methods that depend on the nature of the application. In this application, the key updates are chosen at points on the trajectory curve of the haptic proxy, where the slope evaluates to either zero or infinity, since this would indicate a major change in movement and thus must be carefully conveyed to the other side. The haptic proxy trajectory is the collection of points the proxy passes through during the course of its movement. Figure 6 shows an example of a possible trajectory for a haptic device attached to one end of the board.

Figure 6. Possible haptic device trajectory

In order to achieve a more stable haptic experience, a jitter smoothing algorithm is employed. Whenever an update is produced on the sender side, its timestamp is calculated according to equation 2, where the AverageNetworkDelay is the average delay between the two workstations (this value is periodically measured by the Network Sensing module of each end point (see Figure 2)) and α is a value between 1 and 2 that dictates the smoothing level.

The timestamp represents the time when the received update must be consumed on the receiver side. Choosing a large value for α would ensure that an update is consumed almost every millisecond in a timely fashion on the receiver side. But it also results in reducing the responsiveness of the application felt on the receiver side. A trade-off must be performed to ensure the best results for specific network conditions.

Timestamp = CurrentTime + (α × AverageNetwokDelay)

4

(2)

PERFORMANCE EVALUATION

This section presents the experimental setup and the performance evaluation of the proposed application. 4.1 Task Objectives and Description The objective of the designed task is to measure the level of collaboration and co-presence between the two players when the communication is handled by ALPHAN. Two users have to carry the board and move it across a barrier as shown in Figure 3. The task is simulated in the presence of gravity and friction forces using two haptic devices. The collaboration is imposed by the gravity: if both users do not tightly synchronize their actions, the board deviates from the horizontal direction and eventually the ball moves away from the center of the board towards the lower side of the board. In order to evaluate the suitability of ALPHAN in compensating for short delays, we perform the same task in two cases: first with no delay imposed and secondly in the presence of a constant delay. 4.2 Experimental Setup For the experiment, we used two Pentium 4 PCs with 2 Gb RAM and 100Mbs Ethernet cards. The two haptic devices were the Phantom Omni and the Phantom Desktop, developed and marketed by SensAble Devices, Inc. A snapshot of the experimental setup is shown in Figure 5. Notice that the ALPHAN protocol is implemented on the application layer of the network protocol stack. The application resides on top of ALPHAN. The experiment was conducted on an Ethernet Local Area Network. Network disturbances such as delay and jitter were

364 Authorized licensed use limited to: University of Ottawa. Downloaded on November 11, 2008 at 16:06 from IEEE Xplore. Restrictions apply.

Task Completion Time With Delays

120 100

Table 2: Teams scores without and with 60 ms delays

Without delay Teams

With delay

Variance

Score

Variance

Score

(cm)

(%)

(cm)

(%)

Team 1

198.04

80.19

252.16

74.78

Team 2

435.23

56.47

533.37

46.66

Team 3

414.22

58.57

540.47

45.95

Team 4

876.52

12.34

896.22

10.37

Team 5

673.03

32.69

698.50

30.14

Team 6

254.90

74.50

297.88

70.21

Figure 7 corresponds to the task completion times obtained for each group when a delay of 60 ms is introduced and when no delays were added. The diagram shows that the task completion time with or without delay is almost the same, which suggests that the performance of the users has not been significantly degraded. Figure 8 shows the variations of the variance for a particular group with and without the delay. The variations in the case with delays did not differ significantly from those in the case of no delays.

80 60 40 20 0 Team 1

Team 2

Team 3

Team 4

Team 5

Team 6

Figure 7. The task completion time without and with 60 ms delay

4.3 Performance Variables In order to determine the effectiveness of the collaboration between the two users, two main parameters were measured: the task completion time and the variance of the ball position from the mid point of the board. Based on the variance, we computed a score for the users out of 100. The score was computed based on the assumption that any variance larger than or equal to 1000 cm would be considered as a zero score, and for smaller values, the score was calculated based on the relative difference between the actual variance and the 1000 threshold. 4.4 Results The two main quality parameters (the task completion time and the score) were collected during the experiment for two cases. Case 1 is by performing the task when no delay is imposed and case 2 is where a delay of 60 ms is incorporated. Table 2 shows the variance and corresponding scores for the six teams for both cases. It is clear from the table that the scores decrease when the delays are introduced. However, this decrease is not significant in most of the cases. The users were not able to recognize a significant difference between the two trials. The performance of ALPHAN has compensated for the introduced delay.

Without Delay

140

Time (seconds)

simulated using a software tool we developed for this experiment. In order to make use of the jitter smoothing algorithm, the clocks of both workstations were synchronized. For this purpose, a Network Time Protocol (NTP) server was used. Both workstations maintained a connection with the NTP server in order to synchronize their clocks. The clock synchronization precision obtained for this experiment was within one millisecond. Twelve subjects (forming six teams of two members each) took part in the experiment; all of them were graduate students from the School of Information Technology and Engineering. No particular reward was given to them for their collaboration. Prior to the start of recording the experiment sessions, we asked the teams to perform “rehearsal” sessions just to eliminate any inconsistencies in the performance due to a lack of experience in using haptic devices.

Variance Variations Over Time With 60 ms delay

Without delay

30 20

variance (cm)

10 0 1

164 327 490 653 816 979 1142 1305 1468 1631 1794 1957 2120 2283 2446 2609

-10 -20 -30 -40 Time (mseconds)

Figure 8. The variations of the variance over time

5

CONCLUSION

This paper presents the performance evaluation of the ALPHAN protocol. A collaborative haptic game, named the balance game, has been developed to test the performance of the protocol qualitatively as well as quantitatively. The application is tested in two cases: when a 60 ms delay is introduced and when no delays are used. The performance of the subjects showed that the difference between the two was not significant. This suggests that the protocol is performing well and the 60 ms are almost compensated for in the simulation. Many aspects of the proposed protocol need further testing and evaluation. For instance, we are planning to evaluate the performance of the protocol in the case of three or more collaborators in the environment. Additionally, we will evaluate the protocol in medical applications such as collaborative Telesurgery or Tele-mentoring. REFERENCES [1]

[2] [3]

[4]

M. Eid, M. Orozco and A. El Saddik, “A Guided Tour in Haptic Audio Visual Environment and Applications”, J. of Advanced Media and Comm., v1 (3), pp: 265 – 297, 2007. X. Shen, J. Zhou, N.D. Georganas, "Evaluation Patterns of TeleHaptics", Proc. CCECE2006, Ottawa, Canada, May 2006. J. P. Hespanha, M. McLaughlin, G. S. Sukhatme, M. Akbarian, R. Garg, and W. Zhu. Haptic collaboration over the internet. In Touch in Virtual Environments: Haptics and the Design of Interactive Systems. Prentice Hall, 2002. X. Shen, J. Zhou, A. El Saddik, N.D. Georganas, "Architecture and Evaluation of Tele-Haptic Environments", Proc. of DS-RT 2004, Budapest, Hungary, 2004.

365 Authorized licensed use limited to: University of Ottawa. Downloaded on November 11, 2008 at 16:06 from IEEE Xplore. Restrictions apply.

[5]

[6]

[7] [8] [9]

[10] [11] [12]

[13]

[14]

[15]

[16]

[17]

[18] [19]

B. Chebbi, D. Lazaroff, F. Bogsany, P. X. Liu, N. Liya, M. Rossi, “Design and implementation of a collaborative virtual haptic surgical training system”, IEEE Int. Conf. on Mechatronics and Automation. V. 1, Page(s): 315 – 320, 2005. C. Basdogan, C. Ho, M. A. Srinivasan, M. Slater, “An experimental study on the role of touch in shared virtual environments”, ACM Trans. Comput.-Hum. Interact. 7, 4, 2000. S. Brave, A. Daley, “inTouch: A Medium for Haptic Interpersonal Communication”, CHI ’97, 363-364. C. Gunn, “Collaborative virtual sculpting with haptic feedback”, Virtual Reality Journal, Vol 10, No 2, Oct 2006. R. Iglesias, A. Carrillo, S. Casado, T. Gutiérrez, J.I. Barbero, “Virtual Assembly Simulation in a Distributed Haptic Virtual Environment”, in Int. con. on Adv. Design and Manufacture, 2006. F. R. El-Far, M. Eid, M. Orozco, A. El Saddik, “Haptic Application Meta-Language”, DS-RT, Malaga, Spain, 2006. S. Dodeller, “Transport Layer Protocols for Haptic Virtual Environments”, M.S. thesis, University of Ottawa, 2004. S. Shirmohammadi, N.D. Georganas, “An End-to-End Communication Architecture for Collaborative Virtual Environments”, Computer Networks, Vol.35, No.2-3, 2001. S. Dodeller, “Transport Layer Protocols for Telehaptics Update Messages”, Proceedings of the 22nd Biennial Symposium on Communications, June 2004. M. Mauve, V. Hilt, C. Kuhmünch, W. Effelsberg, “RTP/I Toward a Common Application-Level Protocol for Distributed Interactive Media”, IEEE Transactions on Multimedia, 3(1):152– 161, March 2001. P. Karn, C. Partridge, “Improving round-trip time estimates in reliable transport protocols”, ACM Trans. Comput. Syst. 9, 4 Nov. 1991. P. Hinterseer, E. Steibach, S. Chaudhuri, “Model based data compression for 3D virtual haptic teleinteraction”, ICCE '06, Jan. 2006. H. Al Osman, M. Eid, R. Iglesias, and A. El Saddik, “ALPHAN: Application Layer Protocol for HAptic Networking”, IEEE International Workshop on Haptic Audio Visual Environments and their Applications (HAVE 2007), October 2007. Open Haptics Toolkit: http://www.sensable.com/productsopenhaptics-toolkit.htm. Last viewed on June 1, 2007. Moving Picture Experts Group: http://www.mpeg.org. Last viewed on October 1, 2007.

366 Authorized licensed use limited to: University of Ottawa. Downloaded on November 11, 2008 at 16:06 from IEEE Xplore. Restrictions apply.