Paper Title (use style: paper title)

0 downloads 0 Views 503KB Size Report
LORIA UMR 7503, University of Lorraine-INRIA-CNRS. F-54506 Nancy ... Donetsk National Technical University. 58 Artyom Street, 83000 Donetsk, Ukraine ..... Medical engineering & physics, vol. 30, no. 10, pp. 1364–1386, Dec. 2008.
Sensor network architecture to measure characteristics of a handshake between humans Artem Melnyk 1, 3 ETIS UMR 8051, UCP-ENSEA-CNRS Cergy-Pontoise University F-95000 Cergy Pontoise, France [email protected] 2

Patrick Henaff LORIA UMR 7503, University of Lorraine-INRIA-CNRS F-54506 Nancy [email protected]

Abstract—Handshaking is an important component of social interaction between people in many cultures. Thus, for further applications in human/humanoid-robot interaction it is important to understand and measure the characteristics of a handshake during interaction between humans. In this paper, a new embedded system to measure a handshake is described. It consists of a set of several sensors (accelerometers, gyroscopes and force sensors) attached to the arm, and of a microcontroller for signal acquisition and conditioning. The paper focuses on the applicability and qualitative analysis of the proposed architecture of sensors through several experiments of handshaking between two human subjects. The results show that the proposed system allows reproducible experiments to quantify handshake characteristics such as duration and strength of the grip, vigor and rhythmicity of a handshake Keywords— handshaking, physical interaction, rhythmicity, inertial sensor, force sensor

I.

INTRODUCTION

The origin of nonverbal human communication like handshaking goes back to extreme antiquity. Nowadays, the development of the science investigating human psychology is giving rise to comprehensive study of this social phenomenon. An argument that handshaking serves the necessary and important social function of regulating and maintaining human interactions is developed [1]. The research [2] indicated that handshaking can consciously modify the accuracy of impressions, especially between men, which can explain the importance of handshaking during face-to-face business negotiations. Handshaking is a nearly universally accepted pattern of behavior in societies that initiates and sometimes constitutes social interaction [3]. According to [4] a handshake provides the information about character. Several studies showed that specific handshake manners may depend on personality traits [5]. In [6] it is shown that handshake characteristics like completeness of grip, strength, duration, vigor and others are related to particular orthogonal factors of personality. Authors defined a set of qualitative characteristics related to impressions of participants of the experiment.

Viacheslav Khomenko3 Donetsk National Technical University 58 Artyom Street, 83000 Donetsk, Ukraine [email protected] Volodymyr Borysenko 3 Donetsk National Technical University 58 Artyom Street, 83000 Donetsk, Ukraine [email protected]

The further progress in humanoid robotics depends on solution of fundamental problems like cognitive mechanisms in a human being: learning, adaptation, memory, developmental capabilities. These properties will be required in the nearest future because robots will interact with humans physically and socially more and more often. Indeed, the humanoid robot will became a new agent in the circle of human communications. This interaction phenomenon is reasoned by the rapid progress in many different fields of computer and engineering sciences during last two decades. The humanoid robot already appears in a human environment as a servant or a partner to live or work with human beings. This raises the relevant question of interactions between human and robot in a human friendly-way. These problems still remain unsolved. The social robot is a focus of attention of researchers in sociology, psychology, neuroscience and robotics. There are discussions about security and ethical problems and the real role of a robot ([7], [8]). A great number of researches was carried out in the field of relevant HumanRobot Interaction (HRI) [9], the physical HRI (pHRI) is now an essential branch of robotics research. Handshaking has an important role in pHRI. In order to better understand the nature of handshaking phenomenon it is important to add quantitative measurements to qualitative characteristics defined in [4][5][6]. These quantitative measurements will be taken in this work. These quantitative measurements are directly related to motion properties during interactions: forces, positions, velocities, accelerations, frequencies and so on. In the last decade, a number of authors proposed different methods based on inertial, optoelectronic or electromagnetic sensors to measure the parameters of physical interpersonal interaction [17], [18], [19], [20]. In [10] and [11] authors used a magnetic sensor (FASTRAK) to measure positions and angles of a human hand, and a force sensor to measure human force. Human handshake motions are measured using a 3dimentional motion capture system (VICON). Five reflection markers were fixed on the subjects and their positions were

measured using ten cameras. An approach based on surface electromyography is employed to estimate body motions [12]. A haptic device was used to simulate human handshakes via a metal rod in [14]. In [15], temperature tactile sensors and cameras were used to interpret human feelings towards a social interaction, including a handshaking ritual with a robot. Some authors used devices sensitive to force in robotics and entertainment as in [22], communication appliances for handicaps [23] or biomedical research and rehabilitation [24], and other devises based on the data glove prototypes [25]. In this paper a new architecture of wearable sensors network to measure handshake characteristics during interaction between humans is described. This measurement system enabling quantitative analysis of limbs movements and interactions consists of a set of inertial sensors (accelerometers, gyroscopes) and force sensors. After this introduction, the second part of this paper describes the proposed handshaking measurement system. It also presents the principle of measurement and calibration techniques of sensors network. In the fourth part, the experimental results of several handshake measurements are shown in qualitative and quantitative perspective. Finally, in the fourth part a conclusion is given. II.

a)

Accelerometer CS

b)

Gyroscope CS c)

Fig. 1. Data gloves (subjects I and II) with 6 force sensors and local coordinate system (CS) for accelerometers b) and gyroscopes c).

THE PROPOSED HANDSHAKING MEASUREMENT SYSTEM

Analog signals

A. Principle of measurement. The human arm is equipped with inertial sensors and force sensitive resistors attached to the hand, as shown in Fig. 1. The complete list of the sensors is given in Table 1. The curent values of accelerations are processed in real time with a sampling time of 20 ms by a microcontroller and sent asynchronously via an USB bus to the registration software after filtering and transformations of coordinate systems. The architecture of a proposed handshaking measurement system is based on a 16 MHz microcontroller ATmega2560 (Fig. 2). A glove equipped with a 6-DOF inertial sensor and 6 force sensitive sensors transduces hand movement accelerations and interaction force values. The force sensors are attached using hot melt adhesive on the palm region of the glove (Ossa metacarpalia) to maximize the information about the fullness of the grip of two interacting hands (Fig.1a). The inertial sensor 6-DOF IMU is attached to each glove on the back of palm. Fig. 1b and Fig. 1c presents the sensing axes of accelerometer and gyroscope. TABLE I. Measured value

Sensor

APPLIED UNITS Range

Analog input

Acceleration, rH

ADXL335

±3g

Velocities, ωH

LY530

300 s-1

Velocities, ωH

PR530

-1

300 s

A4, A5 and A9, A10

Force, fHi

FSR

10 kg

A11…A14 and A15.1 …A15.8

A0..A2 and A6..A7

MUX Digital control

Sensor Network

Power supply

Amplifier

Analog data

ATmega 2560

USB

PC

Fig. 2. Architecture of the proposed handshaking measurement system supporting analog sensors.

B. Calibration of Sensors Network To obtain adequate measurements in the proposed architecture of the sensors, we use a two-stage calibration. The first stage is characterized by motionless of the subject (person) in their natural standing position (arm lowered down). For this fixed position, the calibration subroutine of analog angular rate sensors and acceleration is launched. This set of data is averaged per each axis in the interval of 3-5 seconds. These mean values are used as the offset voltage to determine the angular velocity i for each of the three axes i of the analog gyroscope following eq. (1).

i  k U ADC  U offset 

A3 and A8

(1)

U ADC is the voltage obtained after analog-to-digital conversion; U offset is the offset voltage as defined in the where

calibration phase;

k is the sensitivity parameter of the sensor.

 a  a x2  a y2  a z2 where

(2)

a x , a y a z - measured accelerations according to the

axis in Fig.1. In a quiescence position of the arm, the geometric sum of accelerations (2) is equal to one. The second stage consists firstly of the task for the subject to produce, with his hand in front of him, the elementary movements: pronation/supination, flexion/extension, ulnar/radial deviations (Fig. 3).

Acceleration (g) Acceleration (g) Acceleration (g)

The calibration subroutine adjusts the sensitivity of the accelerometer using eq. (2).

Pronation 0.5 /Supination 0 -0.5 0 0.5 0 -0.5 -1 -1.5 0

Flexion Radial/Ulnar Deviation /Extension

5

aII x

time (s) 15

10

aIy

5

aII y

time (s) 15

10

1

aIz

0 -1 0

aIx

5

aII z

time (s) 15

10

Fig. 4. Acceleration as measured in the coordinate frame of the 6-DOF IMU during calibration movements from two persons. Supination

Pronation o

Radial Deviation

Velocity ( /s)

Ulnar Deviation

Extension

Flexion

o o

Velocity ( /s)

A typical set of data from the inertial sensors is plotted in Fig.4 and Fig.5 respectively and force sensor data is shown in Fig.6. One can see acceleration of the hand during the second calibration stage for the test movements of two different persons. Each of the subjects performed movements in their own rhythm and their own dynamics. The hand performs pronation and supination movements with acceleration not exceeding ±0.4 g. Flexion /extension occurs with faster dynamics ±0.5 g. Finally, ulnar/radial deviation occurs with the fastest dynamics of ±1.0 g in a sagittal plane. Fig. 4 shows the angular velocities of the hand. The first subfigure shows velocities of two hands which move in a vertical plane during a radial/ulnar hand deviation. The second subfigure shows a pair of velocities in horizontal plane during flexion/extension movement. The third subfigure depicts parameters during pronation/supination. There are sufficiently high velocities in the third phase of a physical contact, reaching the limit of a gyroscope sensitivity range. The curves in Fig. 5 show the forces during the independent test of six force sensors. The test presses were made according to Fig. 1. The press force sensor is arbitrary and average value is 7 N.

I x  II x



0 -500 0

time (s) 5

500

10 Flexion/Extension

15 I

y  II y

0 -500 0 5 Pronation/Supination 500

time (s) 10

15  Iz

0 -500 0

 II z

time (s) 5

10

15

Fig. 5. Angular velocity as measured in the coordinate frame of the 6-DOF IMU during calibration movements from two persons. 10

fI1 fI2

8

fI3

Force (N)

In the second part of the stage the subject puts his hand on his chest and touches three times, one by one, the six force sensors with an average force. At this time, operation of the system can be observed in real time through the signals displayed on a computer screen.

Velocity ( /s)

Fig. 3. Six motions of the wrist (extracted from [24])

Radial/Ulnar Deviation 500

6

fI4 fI5

4

fI6 2

time (s) 0 0

5

10

15

Fig. 6. Force values from setup of sensors during calibration procedure from one person.

III.

HUMAN-HUMAN PHYSICAL INTERACTION

Phase 1

Phase 3

13.5

14

14.5

2 0 -2 12.5

15

aII x

13.5

14

14.5

time (s) 15 15.5

0

aII z

time (s) 13.5

14

14.5

15

o

x

time (s)

-500 12.5

13

13.5

14

14.5

15

15.5

500

I

y

0



time (s)

-500 12.5

13

13.5

14

14.5

15

15.5

Fig. 8. Acceleration measured in the coordinate frame of the 6-DOF IMU during handshaking. Four phases of handshaking are found.

II y

15.5

500

 Iz

0

 II z

time (s)

-500 12.5

13

13.5

14

14.5

15

15.5

Phase 2

Phase 3

Phase 4 fI1 fI2

4

fI3 2

0 12.5

aII y

aIz

13

 II x

time (s)

2

-2 12.5

0

15.5 aIy

13

I

6

aIx

13

Phase 4

500

Phase 1

Phase 4

time (s)

Phase 3

The group of force sensors gives the information about the moment of grip and drop of interacting hands. Moreover, we have access to the topology of the grip and the conclusion about its completeness can be made. Fig. 10 provides gives us the information about strength via force values of the six sensors during a handshake. The first subfigure depicts forces on the upper side of the hand received from sensors f1,f2,f3, and the second depicts state of force on the upper side of the hand from sensors f4,f5,f6 (Fig. 1a).

Force (N)

3 2 1 0 -1 12.5

Phase 2

Phase 2

Fig. 9. Angular velocity as measured in the coordinate frame of the 6-DOF IMU during handshaking.

13

13.5

14

14.5

15

15.5

6

Force (N)

Acceleration (g) Acceleration (g) Acceleration (g)

Phase 1

o

The study and observation of experimental handshaking between two persons showed that we can decompose the interaction into four phases. In phase 1, a visual contact (VC) is established and the persons bring their hands to shake. In phase 2, a physical contact (PC) is established and the participants are in the first phase of the interaction, unconsciously synchronizing their movements. In phase 3, movements are mutually synchronized (MS) and there is a handshake. In phase 4 there is successful completion of the handshake and physical connection is broken and hands freely move back to subjects’ bodies (EoH). The results are shown in Fig. 8 in three subplots of data sets. The first subplot shows the values of hand acceleration of two persons in the sagittal plane, the second - in the frontal plane and the third subplot - in the horizontal plane. The participants move their hands to shake from [12.5 s to –13.3 s] until a physical contact is confirmed by a force sensor (Phase 1). During the next time range [13.3–13.6 s], humans interact physically together and unconsciously synchronize their movements (phase 2). At time [13.6–15.9 s], there is a handshake with a stable and synchronous rhythm (phase 3). After t=14.8s, rhythmic movement is interrupted by the end of a handshake act (phase 4).

Velocity ( /s)

Fig. 7. Two interacting subjects carrying the prototype system.

Velocity ( /s)

o

Velocity ( /s)

Two subjects are wearing the system of sensors and after having described all tests we propose to start a casual handshake. Persons are allowed looking at and speaking to each other for more natural experimental settings. They are situated at arm's length, not to make additional movements in space, Fig. 7.

This particular handshake duration value is 3.5 s with maximum acceleration values reaching 3g (3 gravity acceleration). The angular speeds of hands during the interaction are shown in fig. 9 axis per axis for two persons. In the penultimate phase of the hand the maximum speed is ±200 degrees per second.

fI4 fI5

4

fI6 fmean

2

time (s) 0 12.5

13

13.5

14

14.5

15

15.5

Fig. 10. Force values from setup of sensors during handshaking

IV.

CONCLUSION

In this work we proposed a handshaking measurement system based on the microcontroller and a set of several inertial sensors and force sensors. The proposed system is able to capture the upper limbs movement parameters for quantitative analysis. The handshake act was investigated and its characteristics like completeness of grip, strength, duration, vigor were analyzed for a particular interaction case. The four phases of handshaking were identified, and it was shown that physical handshaking is composed of two phases, the first (transient) without synchrony between the two persons and the second (permanent) with synchrony. The paper also shows that our measurement system is able to define the fundamental qualitative and quantitative characteristics of handshaking. This system is actually used to investigate large number of handshaking experiments with 30 persons in order to carry out statistic studies. The results will provide behavior models that we could implement in a compliant robotic arm in order to reproduce handshaking between human and robots. ACKNOWLEDGMENT Authors would like to thank the head of Neurocybernetics team of ETIS laboratory, Philippe Gaussier for his advice. Special thanks to associated professor Rozkariaka Pavel for his help in gyroscope calibration. This work is supported in part by the French Embassy in Ukraine and by the INTERACT French project referenced ANR-09-CORD-014.

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

REFERENCES [1] [2]

[3] [4]

[5]

[6]

[7]

[8]

[9]

D. Schiffrin, “Handwork as Ceremony: The Case of the Handshake,” Semiotica. 12(3): 185-280. doi:10.1515/semi.1974.12.3.189 F. J. Bernieri and K. N. Petty, “The influence of handshakes on first impression accuracy,” Social Influence, vol. 6, no. 2, pp. 78–87, Apr. 2011. http://dx.doi.org/10.1080/15534510.2011.566706 P. M. Hall and D. A. S. Hall, “The handshake as interaction,” Semiotica. 45(3-4): 191-379. doi:10.1515/semi.1983.45.3-4.249 J. Astr m and . . Thorell, “ reeting ehaviour and psychogenic need interviews on experiences of therapists, clergymen, and car salesmen.” Perceptual and motor skills, vol. 83, no. 3 Pt 1, pp. 939–956, Dec. 1996. http://view.ncbi.nlm.nih.gov/pubmed/8961332 J. Astrom, “Introductory greeting behaviour: a laboratory investigation of approaching and closing salutation phases, Perceptual and Motor Skills, Vol 79(2), Oct 1994, 863-897. doi: 10.2466/pms.1994.79.2.863 W. F. Chaplin, J. B. Phillips, J. D. Brown, N. R. Clanton, and J. L. Stein, “ andshaking, gender, personality, and first impressions.” Journal of personality and social psychology, vol. 79, no. 1, pp. 110–117, Jul. 2000. http://view.ncbi.nlm.nih.gov/pubmed/10909881 M. L. Walters, K. Dautenhahn, S. N. Woods, K. L. Koay, R. Te Boekhorst, and D. ee, “Exploratory studies on social spaces etween humans and a mechanical-looking ro ot,” Connection Science, vol. 18, no. 4, pp. 429–439, Dec. 2006. http://dx.doi.org/10.1080/09540090600879513 K. Dautenhahn, S. Woods, C. Kaouri, M. L. Walters, K. L. Koay, and I. Werry, “What is a ro ot companion - friend, assistant or utler?” in Intelligent Robots and Systems, (IROS 2005). 2005 IEEE/RSJ Int. Conf. on. IEEE, pp. 1192–1197. http://dx.doi.org/10.1109/iros.2005.1545189 R. Alami, A. Albu-Schaeffer, A. Bicchi, R. Bischoff, R. Chatila, A. D. Luca, A. D. Santis, G. Giralt, J. Guiochet, G. Hirzinger, F. Ingrand, V. Lippiello, R. Mattone, D. Powell, S. Sen, B. Siciliano, G. Tonietti, and L. Villani, "Safe and dependable physical Human-Robot interaction

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

in anthropic domains: State of the art and challenges," in Procceedings IROS Workshop on pHRI - Physical Human-Robot Interaction in Anthropic Domains, A. Bicchi and A. D. Luca, Eds., Beijing, China, Oct. 2006. M. Jindai and T. Watana e, “Development of a handshake ro ot system ased on a handshake approaching motion model,” in Advanced intelligent mechatronics, 2007 IEEE/ASME international conference on. IEEE, 2007, pp. 1–6. http://dx.doi.org/10.1109/aim.2007.4412423 Y. Yamato, M. Jindai, and T. Watana e, “Development of a shakemotion leading model for human-ro ot handshaking,” in SICE Annual Conference, 2008. IEEE, Aug. 2008, pp. 502–507 http://dx.doi.org/10.1109/sice.2008.4654707 S. Kwon and J. Kim, “Real-Time upper limb motion estimation from surface electromyography and joint angular velocities using an artificial neural network for Human–Machine cooperation,” Information Technology in Biomedicine, IEEE Transactions on, vol. 15, no. 4, pp. 522–530, Jul. 2011. [Online]. Available: http://dx.doi.org/10.1109/titb.2011.2151869 A. Karniel, I. Nisky, G. Avraham, and B.-c. Peles, “A turing-like handshake test for motor intelligence.” http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.167.1586 E. Giannopoulos, Z. Wang, A. Peer, M. Buss, and M. Slater, “Comparison of people’s responses to real and virtual handshakes within a virtual environment.” Brain research ulletin, vol. 85, no. 5, pp. 276– 282, Jun. 2011. http://dx.doi.org/10.1016/j.brainresbull.2010.11.012 W. Bain ridge, S. Nozawa, R. Ueda, K. Okada, and M. Ina a, “Ro ot sensor data as a means to measure human reactions to an interaction,” in Humanoid Robots (Humanoids), 2011 11th IEEE-RAS International Conference on. IEEE, Oct. 2011, pp. 452–457. http://dx.doi.org/10.1109/humanoids.2011.6100823 J. R. Morris, “Accelerometry–a technique for the measurement of human ody movements.” Journal of iomechanics, vol. 6, no. 6, pp. 729–736, Nov. 1973. http://view.ncbi.nlm.nih.gov/pubmed/4757490 A. Godfrey, R. Conway, D. Meagher, and . O aighin, “Direct measurement of human movement y accelerometry.” Medical engineering & physics, vol. 30, no. 10, pp. 1364–1386, Dec. 2008. http://dx.doi.org/10.1016/j.medengphy.2008.09.005 K. Aminian and B. Najafi, “Capturing human motion using body-fixed sensors: Outdoor measurement and clinical applications: Research articles,” Comput. Animat. Virtual Worlds, vol. 15, no. 2, pp. 79–94, May 2004. [Online]. Available: http://dx.doi.org/10.1002/cav.v15:2 K. Saber-Sheikh, E. C. Bryant, C. Glazzard, A. Hamel, and R. Y. W. ee, “Feasi ility of using inertial sensors to assess human movement,” Manual Therapy, vol. 15, no. 1, pp. 122–125, Feb. 2010. http://dx.doi.org/10.1016/j.math.2009.05.009 A. G. G. Cutti, A. Giovanardi, L. Rocchi, A. Davalli, and R. Sacchetti, “Am ulatory measurement of shoulder and el ow kinematics through inertial and magnetic sensors.” Medical & iological engineering & computing, vol. 46, no. 2, pp. 169–178, Feb. 2008. [Online]. Available: http://dx.doi.org/10.1007/s11517-007-0296-5 D. Roetenberg, H. Luinge, and P. Slycke, "Xsens MVN: Full 6DOF human motion tracking using miniature inertial sensors," xsens, Tech. Rep., 2009. K. N. Tarchanidis and J. N. ygouras, “Data glove with a force sensor,” Instrumentation and Measurement, IEEE Transactions on, vol. 52, no. 3, pp. 984–989, Jun. 2003. http://dx.doi.org/10.1109/tim.2003.809484 S. S. Fels and . E. inton, “ love-Talk: a neural network interface between a data-glove and a speech synthesizer.” IEEE transactions on neural networks / a publication of the IEEE Neural Networks Council, vol. 4, no. 1, pp. 2–8, 1993. http://dx.doi.org/10.1109/72.182690 A. Karime, H. Al-Osman, W. ueaie , and A. El Saddik, “E-Glove: An electronic glove with vibro-tactile feedback for wrist rehabilitation of post-stroke patients,” in Multimedia and Expo (ICME), 2011 IEEE International Conference on. IEEE, Jul. 2011, pp. 1–6. http://dx.doi.org/10.1109/icme.2011.6012252 M. Bianchi, P. Salaris, and A. Bicchi, “Synergy-based hand pose sensing: Reconstruction enhancement,” CoRR, vol. a s/1206.0555, 2012. http://dblp.uni-trier.de/rec/bibtex/journals/corr/abs-1206-0555