Paper Title (use style: paper title)

0 downloads 0 Views 789KB Size Report
INTRODUCTION ... character animation or even sign language interpretation [1]. Impairment ..... Figure 4 – Blender 3D hand model used for evaluating the glove.
Real-time hand tracking for rehabilitation and character animation António H.J. Moreira1,2, Sandro Queirós 2, José Fonseca1, Pedro L. Rodrigues 2, Nuno F. Rodrigues1, João L. Vilaça1,2 1

2

DIGARC, Polytechnic Institute of Cávado and Ave. 4750-810 Barcelos, Portugal. Life and Health Sciences Research Institute, University of Minho. 4710-057 Braga, Portugal.

Abstract — Hand and finger tracking has a major importance in healthcare, for rehabilitation of hand function required due to a neurological disorder, and in virtual environment applications, like characters animation for on-line games or movies. Current solutions consist mostly of motion tracking gloves with embedded resistive bend sensors that most often suffer from signal drift, sensor saturation, sensor displacement and complex calibration procedures. More advanced solutions provide better tracking stability, but at the expense of a higher cost. The proposed solution aims to provide the required precision, stability and feasibility through the combination of eleven inertial measurements units (IMUs). Each unit captures the spatial orientation of the attached body. To fully capture the hand movement, each finger encompasses two units (at the proximal and distal phalanges), plus one unit at the back of the hand. The proposed glove was validated in two distinct steps: a) evaluation of the sensors’ accuracy and stability over time; b) evaluation of the bending trajectories during usual finger flexion tasks based on the intra-class correlation coefficient (ICC). Results revealed that the glove was sensitive mainly to magnetic field distortions and sensors tuning. The inclusion of a hard and soft iron correction algorithm and accelerometer and gyro drift and temperature compensation methods provided increased stability and precision. Finger trajectories evaluation yielded high ICC values with an overall reliability within application’s satisfying limits. The developed low cost system provides a straightforward calibration and usability, qualifying the device for hand and finger tracking in healthcare and animation industries. Keywords – motion tracking glove; finger flexion; inertial measurement unit; hand rehabilitation.

I.

INTRODUCTION

Recent advances in micro-electromechanical systems (MEMS) technology have reduced the size of accelerometer, magnetometer and gyroscope devices to the point where it is practical to construct an inertial measurement unit (IMU) with 9 degrees of freedom (9DOF) of the size of a fingernail. In this way, a new set of applications emerge were finger tracking has major importance, namely hand rehabilitation, character animation or even sign language interpretation [1]. Impairment of hand function caused by a neurological disorder, such as stroke or cervical spinal injury, has a high impact on the independence and quality of life of the affected person. Recent studies have suggested that for upper extremity functional recovery, repetitive and long duration training using robots and virtual reality is helpful [2], [3].

With more than 7 million people only in US dealing with disabilities related to stroke and nearly 795,000 new strokes occurring each year, leaving more than 80% of all stroke patients with impairment of the upper limb motor function and only 30% to 40% regaining some dexterity after six months, rehabilitation has a huge impact in our society [4]. Although it is been proven that long term rehabilitation provides significant functional benefits, its cost currently prevents a longer usage. Rehabilitation is commonly classified into two categories: physical rehabilitation (PR) and functional rehabilitation (FR). PR exercises imply the use of force to improve the patient’s motor skills (exercise muscles and joints). FR is performed to regain lost skills (such as those needed in the activities of daily living (ADL) or job-related skills). The essential feature of these exercises is the patient’s interaction with the objects he grasps [5]. Therefore, the ability to monitor hand motion for extended periods of time in a more natural environment can help assess ADL–based hand function and ultimately improve the rehabilitation outcome. Beside the importance of a motion glove in rehabilitation, character animation can also benefit from the several degrees of freedom (DOFs) within this type of device. Automatic motion generation for digital characters under real-time user control is a challenging problem for computer graphic research and virtual environment applications (such as on-line games) [6]. Currently, the process of motion generation involves recording body movements of a real actor (by means of a motion capture suit), which is then used to control an animated character to move exactly like the original real actor did. However, such system lacks finer movements, like finger flexion, mapped in the movement of the animated character's hand. Animating a character’s hand can require several artists illustrating the complicated hand movements, increasing the time and cost involved in the process [6], [7]. As already described for the healthcare industry, motion-capture gloves can help to avoid this cumbersome work and still provide an accurate solution. In order to provide a satisfying solution, a motion-capture glove must be capable of tracking mainly three types of movements: abduction between the fingers, flexure for each finger and wrist flexion. Among the state-of-the-art, Dipietro et al. [8] identified different sensing technologies used to design these gloves, with resistive bend sensors within the most used due to its relative low cost per unit. However, to deal with all possible movements, some commercial gloves, such as the 5DT Data Glove Series (Fifth Dimension Technologies, Inc; Irvine, California), the CyberGlove (Immersion Corp; San Jose,

California), Humanglove (Humanware S.R.L., Pisa, Italy) and the ShapeHand (Measurand Inc; Fredericton, Canada), have to apply up to 22 sensors (with three sensors for each finger, four for fingers’ abduction and the remaining for wrist tracking), which complicates its combined use. Moreover, motion gloves with this technology most often suffer from signal drift, sensor saturation, sensor displacement, the inability to monitor the three joints of one finger and the need for complex calibration procedures [9]. The present paper presents a new motion-capture glove with potential applications in the healthcare and virtual animation industries. The proposed glove aims to provide the required DOFs for hand and fingers tracking, while minimizing measurement inaccuracies (e.g., drift and/or offset errors) and avoiding the need for cumbersome calibration procedures. Simultaneously, a low-cost solution is targeted to reduce the economic burden associated with the end-user application. The proposed solution uses a set of 9DOF inertial measurement units to capture both hand and fingers spatial orientation. In this paper, the glove’s features are described, including sensors’ characteristics, fusion algorithm used, calibration steps and processor capabilities. Moreover, two types of evaluation are here presented, namely a technical evaluation of the measurements’ stability and accuracy, as well as a reliability evaluation in a possible application scenario. II.

METHODS

A. Requirements A complete hand model has up to 30 DOFs, resulting in a complex motion system [10]. The hand bones are grouped as carpals, metacarpals, and phalanges. Finger joints are named as metacarpophalangeal (MCP) joints, which are connected to the palm, or interphalangeal (IP) ones, between proximal, intermediate and distal phalanges. Each joint type has different DOFs, according to the anatomical movements available (Fig. 1). The nine IP joints have only one DOF, specifically, flexion/extension. MCPs are described as saddle joints with two DOFs: one for abduction/adduction and one for flexion/extension. The thumb is the shortest and most mobile finger and is composed by two phalanges only. Lastly, the wrist has 3 DOFs, although with rotational anatomical limits. In order to be useful for the abovementioned applications, a motion-capture glove should capture the most of this complex system. In this sense, the proposed device implements a model approximated as following:

Figure 1 – Degrees of freedom of the wrist and fingers’ joints.

a) Real-time acquisition of flexion/extension abduction/ adduction of the 5 fingers; b) Automatic magnetometer, gyroscope accelerometer calibration; c) Portable and low cost tracking device; d) Blender script for hand animation.

and

B. Hardware Solution The acquisition hardware specifically developed for the hand tracking device was selected to ensure the capture of all hand and fingers inertial movements in real time and with “Plug and Play” characteristics. To this end, the hardware requisites of this system are: a) absolute hand and fingers orientation; b) real-time data transmission; and c) miniaturization. To accomplish these requirements, the chosen hardware is divided in two main modules: a) eleven 9DOFs inertial measurement units, and b) an embedded STM32F4 ARM processor (Fig. 2).

 The palm is a rigid body;  The hand is a skeleton and each finger is represented as a kinematic chain. The base is attached to the palm and the fingertip is an end-effector. In order to increase the effectiveness of hand rehabilitation, one must have the ability to record patient evolution. Benefiting from this ability, character and hand animation can also be performed in a faster and less expensive way. Taking into account the application requirements, the following features were implemented in the proposed glove tracking device:

and

Figure 2 – Glove tracking system hardware schematic.

1) 9DoF Inertial measurement unit The developed inertial measurement unit (IMU) packs a 3axis gyroscope (L3GD20) and a combined 3-axis accelerometer and 3-axis magnetometer (LSM303DLHC). The goal is to combine the nine independent rotation, acceleration, and magnetic readings (9DOFs) and create an attitude and heading reference system (AHRS – Section II.C.2) to track the orientation of the board. Overall, the idea is to combine the ability to accurately track rotation on a short timescale of the gyroscope with the absolute frame of reference provided by the accelerometer and magnetometer, which compensates for gyroscope drifts over time. Since all 9 readings should be combined later, the respective axes of the two chips (L3GD20 and LSM303DLHC) were aligned on the board to facilitate sensor fusion calculations. Regarding sensors’ features, both LSM303DLHC and L3GD20 have many configurable options, including dynamically selectable sensitivities, as well as a choice of output data rates (ODR) for each sensor. In the current glove, the sensors were configured as follow:  Gyroscope: one 16-bit word per axis, with a sensitivity of ±2000 degrees per second and ODR of 400 Hz.  Accelerometer: one 16-bit word per axis, with a sensitivity of ±2 g and ODR of 200 Hz.  Magnetometer: one 12-bit word per axis, with a sensitivity of ± 1.3 Ga and ODR of 75 Hz. Finally, the communication between each unit and the embedded processor is accomplished through a shared I2C protocol, with all three chips at clock frequencies up to 800 kHz. 2) Embedded processor and communication Achieving a real-time and smooth tracking for all hand fingers is a complex and time-consuming operation for an embedded processor. Some newer embedded processors, such as the Cortex-M4 family from ARM, ease this problem by providing high clock speeds (up to 180MHz), embedded DSP with single-cycle arithmetic operations (which help boost the AHRS algorithm throughput) and low current consumption, which delivers a complete solution for the most demanding applications. The proposed solution incorporates a STM32F407VG microcontroller, with enough computational power for the sensor fusion algorithm, while allowing the communication between the glove and an external device (e.g. computer) through a Universal Asynchronous Receiver/Transmitter (USART) protocol at 512kbps. A future wireless solution is also planned with a Wi-Fly RN-131C wireless 802.11b/g networking module to ease the glove usage. 3) Glove design The prototype was implemented in a right-hand glove made of polyamide stretchable fabric (Fig. 3). This fabric was chosen due to its broad stitches, which allow more contact between the skin and the grasped object compared to other types of materials.

Figure 3 – Sensor placement in the proposed motion-capture glove.

The glove is equipped with 11 IMUs in the back of the hand and fingers. Each of these was stitched to the fabric with four fixation points. To capture the required DOFs (Fig. 1), each finger encompasses two IMUs, one at the distal phalanx (DP) and another at the proximal phalanx (PP). Note that in order to simplify the current solution (by decreasing the required number of sensors and thus minimizing costs and data transmission time), the orientation of the intermediate phalanx is not tracked directly, but instead interpolated based on the DP and PP orientation (as discussed in Section II.C.4). The eleventh IMU is placed at the back of the palm for wrist flexion tracking (Fig. 3). All sensors are connected to the main board through an I2Cbus, as previously indicated. C. Software Solution 1) Data acquisition All 11 IMU sensors data acquisition is achieved through a multiplexed I2C-bus; this is accomplished using 2 Texas TCA9548A multiplexers. Each device has eight bidirectional translating switches that can be controlled via the I2C bus. Acquiring the data of the 1 to 6 IMUs requires the selection of multiplexer 1 (with address 0x70) and the selection of channel 0 to 5, respectively. From IMU 7 to 11, multiplexer 2 (with address 0x71) channels (0 to 4) are selected. Sensor data can be retrieved, after multiplexer channel selection, from the accelerometer through the address 0x19, from the magnetometer through the address 0x1E and from the gyroscope through the address 0x6A. In overall, reading each IMU (the combination of the 3 sensors) takes only 180µs, approximately. 2) Attitude and Heading Reference System Attitude and heading for each IMU is computed using the quaternion-based gradient descent algorithm proposed by Madgwick et al. [11]. In essence, and being an orientation filter, the main goal of Madgwick’s algorithm is to fuse the 3 sensors (gyroscope,

accelerometer and magnetometer) data in order to find the orientation of the attached body relative to the Earth’s magnetic field. The main principle is to integrate the estimated orientation rate using the rate of change given by the gyroscope (which has a low-frequency bias but is responsive at high frequencies), later corrected by the readings of the accelerometer and magnetometer. To this end, the magnitude of the gyroscope measurement error is removed in the direction of the estimated error, which is computed from the low-frequency attitude references (e.g. magnetic heading and IMU-based gravity vector estimate). So, the high frequency components of the integrated gyroscope measurements are passed, along with the low-frequency attitudes derived from the references; the low frequency gyroscope biases are rejected.

Each data transmission occurs in an interrupt event (nonblocking transmission) function. This allows for I2C to communicate with the sensors while transmitting data through USART.

The Madgwick’s algorithm was chosen mainly due to its computational efficiency, but also due to its performance even at low sampling rates, as well as the embedded inclusion of online magnetic distortion and gyroscope bias drifts compensation algorithms [11].

An important issue is the range of motion of the hand and finger joints and the relationship between adjacent joint angles. In fact, most of the possible combinations of joints’ position yield unrealistic hand configurations. To avoid these configurations, and to simplify the model by reducing the freedom of the whole system, we introduced some constraints in the represented virtual orientation:

At each cycle, the readings from the 3sensors are combined and the estimate of the rotation for the particular time lapse is added to the current global orientation estimation. At the end, the AHRS quaternion for the new spatial positioning is obtained. Note that the resulting AHRS quaternions are converted to Euler angles (yaw, pitch and roll angles) before transmission, as further explained in Section II.C.3. Overall, the sensor fusion algorithm implementation needs approximately 280µs per IMU, representing a total of 460µs per cycle (including sensor reading). Thus, the eleven IMUs require 5.05ms, approximately, to update, meaning a near 200Hz sampling and computational rate. 3) Data transmission The transmitted data format between the glove tracking system and an external device are summarized in Table I. TABLE I. FRAME SYNTAX BETWEEN GLOVE AND EXTERNAL DEVICE. Glove Data Frame IMU 1

...

IMU N

Begin

#

Yaw

Pitch

Roll

...

Yaw

Pitch

Roll

32b

32b

32b

...

32b

32b

32b

CRC

End

0x00

@

4) Virtual hand animation The output data from the motion-tracking glove was connected to Blender 3D software through a script written in python and using a 3D hand model with 17 bones (Fig. 4). In Blender 3D software, each bone is rotated locally using the values received from the glove. Therefore, in order to properly animate the virtual hand model, the device outputs joint angles as a kinematic chain from the palm to the tip (see Section II.D.5).

a) The intermediate phalanges angles are defined as half of the corresponding DP angle (properly computed based on the Euler angles mathematics); b) The pitch amplitude in all IP joints is limited from -5º to 90º; c) The roll angle is not applied to any fingers’ joint, only to the wrist; d) The yaw angle is limited to an amplitude of -10º to 10º in the fingers’ joints (except thumb). D. Calibration and output setup MEMS chip sensors used in the measurement of the acceleration, rotation and magnetic field have all sorts of errors. Some of the errors are insignificant but others are important and need to be taken into account, correcting them if possible to get an accurate AHRS result. Most common sources of error are due to zero-offsets, nonlinearities, noise, scaling factors or thermal drift. In order to minimize these sources of errors and

Although the internal control loop computes each IMU quaternion orientation at a rate of approximately 200Hz, the external data transmission only occurs at 25Hz. Such transmission rate is enough for visualization, avoiding an excessive amount of frames to be sent. Moreover, and to further optimize the data transmission, each time the output interruption occurs, the quaternion angles (4 floats) are converted to Euler angles (3 floats) reducing the number of bytes to transmit. The frame transmission starts with a start identifier (#) followed by the yaw, pitch and roll angles of the 11 IMUs. To ensure correct frame transmission, a cyclic redundancy check (CRC) is performed using the built-in CRC peripheral in the STM32F4 and also included within the frame. Finally, an end identifier (\n) is transmitted.

Figure 4 – Blender 3D hand model used for evaluating the glove tracking system.

1) Accelerometer and gyroscope offset In order to compensate the accelerometer readings for offset errors, each sensor was moved slowly around each axis (X, Y and Z) and the minimum and maximum registered raw acceleration values were measured in each. The zero offset for each axis is computed as half the sum of the extreme values and the scale factor is half their difference.

Compensating these errors was achieved using the ellipsoid least-square fit algorithm proposed in [12]. Overall, the idea is to collect a cloud of points from the magnetometer by rotating it around multiple spatial orientations (a larger amount of points usually result in a better fitting) and then fit an ellipsoid to such data set (Fig. 5). Later, the coefficients of the fitted ellipsoid can be used to create a matrix than can be applied to stretch and rotate the measured magnetometer raw values, trying to approximate the resulting cloud to a perfect sphere. Additionally, the ellipsoid center can also be used to correct for possible magnetometer offsets.

Regarding the gyroscopes calibration, removing their readings offset is of critical importance to reduce the device drift over time. This is the only calibration performed prior to a given tracking session. To this end, when the glove is initialized (in a stationary position), the average gyroscope value per axis is repeatedly computed (using each time 1000 samples) until the associated standard deviation is below a given threshold. Such condition allows avoiding computing gyroscope offsets during initial sensors warm up. When this condition is satisfied, the gyroscope offset values are stored and later used during sensors reading correction.

4) Quaternion reference Since the IMU boards are manually stitched to the glove, their initial orientation is dependent on their fixation. In order to calibrate each IMU with one another and to the virtual hand model, a reference should be set. Such reference will be used to orient all IMUs equally in the hand (defining a new board’s reference system) and in accordance to the virtual model used. In this sense, 10 seconds after the gyroscope offset calibration (see Section II.D.1) the current orientation of each IMU is stored. Using the quaternions mathematics, the rotated orientation is given by:

thus improve the current solution accuracy, stability and reliability, a set of calibration steps were performed previous to any glove usage and one prior to a given tracking session.

2) Temperature offset Although the gyroscopes’ zero offset is canceled during initialization, the temperature during the usage of the glove can change. Since gyroscopes present a very strong relation between their zero offsets and temperature, it is paramount to reduce the effect of temperature variation during runtime. Thus, to find the correct temperature-offset coefficient, all sensors were placed in a freeze for 45 minutes at 4ºC and subsequently their raw values were acquired until the measured temperature (using the on-chip temperature sensor) matched the room temperature. Later, and since the variation with the temperature can be considered nearly linear, a linear least-square fit was used to find the best coefficient to be applied in each axis. Finally, to compensate for temperature-based gyroscope drift during runtime, the difference between the current temperature and the temperature measured during calibration is used to apply the corresponding zero-offset correction.

𝑞𝑐𝑜𝑟𝑟 = 𝑞𝑟𝑒𝑓 −1 × 𝑞𝑒𝑠𝑡

(1)

Where qref −1 corresponds to the inverse of the initial reference quaternion and qest is the current orientation estimation given by the AHRS algorithm. 5) Output preparation In order to facilitate the usage of the outputted data in the animation of a virtual hand, the resulting orientation angles were altered to represent localized joint angles. To this end, the tracked hand movements were described as a kinematic chain assembled from the palm to the fingers’ tips. Such assumption implies that the IMUs at the DP are influenced by the orientation of the corresponding PP ones, while these are described relative to the palm orientation (given by the eleventh IMU). Again, using the quaternions mathematics, the localized joint orientation quaternions are given by:

In opposition, the accelerometer values change very little over temperature and also very little over time. Besides, their influence in the algorithm output is also less critical. In this sense, temperature calibration was not performed for these sensors. 3) Magnetometer distortion Magnetic measurements are continuously subjected to distortion, which are usually classified as hard or soft iron distortions. Hard iron distortions are created by objects that produce a magnetic field, like a piece of magnetized iron. If the piece of the magnetic material is physically attached to the same reference frame as the sensor, then a distortion occurs and is seen as a permanent bias in the sensor output. Soft iron distortions, on the other hand, are considered alterations in the existing magnetic field. These distortions will alter the magnetic field depending upon which direction the field acts relative to the sensor. This type of distortion is commonly caused by metals, such as nickel and iron.

Figure 5 – Magnetometer calibration example (red – calibrated; blue – non-calibrated).

𝑞𝑝𝑎𝑙𝑚,𝑐𝑜𝑟𝑟 = 𝑞𝑝𝑎𝑙𝑚

(2)

𝑞𝑃𝑃,𝑐𝑜𝑟𝑟 = 𝑞𝑝𝑎𝑙𝑚,𝑐𝑜𝑟𝑟 −1 × 𝑞𝑃𝑃

(3)

𝑞𝐷𝑃,𝑐𝑜𝑟𝑟 = 𝑞𝑝𝑎𝑙𝑚,𝑐𝑜𝑟𝑟 −1 × 𝑞𝑃𝑃,𝑐𝑜𝑟𝑟 −1 × 𝑞𝐷𝑃

(4)

Where qpalm , qPP and qDP are the orientation quaternions of the palm, proximal phalanx and distal phalanx, respectively. Note that the input quaternions were already rotated using the reference rotation quaternion presented in Section D.4. The final corrected quaternion can then be converted to Euler angles (yaw, pitch and roll) and sent through the USART protocol for the external device. III.

EXPERIMENTS

The validation of the developed system was twofold: first, a technical test to check the performance of the hand tracking system by analyzing the stability, accuracy and consistency of the angles measured with the sensors embedded in the glove and received in the external device; and secondly, a user evaluation to assess the repeatability of the finger bending trajectories recorded from a healthy subject during the performance of four daily tasks. 1) Technical Evaluation A technical evaluation was carried out to analyze the following performance aspects of the proposed glove: a) Sensor stability over time (i.e., drift); b) AHRS accuracy and repeatibility; c) Successful output data rate (i.e., frames per second); d) Hand movement visual inspection. In the first test, with all IMUs fixed on a flat surface, one acquired the computed Euler angles for 1 hour and evaluated the variability of the outputted data. To this end, the standard deviation of the samples was computed individually for each IMU and for each Euler angle. Using this data, the range of standard deviations per angle (yaw, pitch and roll) was computed. Moreover, and to have a global measurement of the variability, for each angle and IMU, the average value was subtracted to the outputted samples and the global standard deviation (combining the information from all 11 IMUs) was computed for each angle individually and combined. In the second test, each IMU was rotated and fixed at -90º, 0º and +90º for yaw, pitch and roll angles. The outputted angles were recorded and analyzed, with and without the abovementioned calibration procedures, to assess the device accuracy and the influence of the applied calibration. Note that for each rotation, three measurements were performed and the average result is presented. In the third test, the output data was analyzed for inconsistent received frames (according to the frame syntax presented in Section II.B.3). To this end, the output data was recorded for 1 hour and later analyzed for frame error counting. Finally, in the last test, the visual output of the 3D virtual hand model in Blender was compared to the real hand positioning by performing a qualitatively visual assessment.

2) Reliability Evaluation The reliability of the glove was assessed by determining the repeatability of the finger bending trajectories recorded from one healthy subject during the performance of four daily tasks. In each task, a specific grasp form had to be performed:  Task 1 (“Closed fist”) - Comprehends an initial extension of all fingers followed by the full flexion and again extension of all fingers.  Task 2 (“Counting”) - Comprehends an initial flexion of all fingers followed by the extension of index, middle, ring and pinky in sequence.  Task 3 (“Glass”) - Comprehends an initial extension of all fingers followed by picking up a glass and then releasing it with full finger extension.  Task 4 (“Mouse”) – The initial position comprehends the extension of index and middle fingers and the flexion of the ring and pinky fingers. Then, the index finger performs a ‘click’ followed by the middle finger. For each task, 5 repetitions were performed and the output data recorded. Then, the intra-class correlation coefficient (ICC) was calculated between acquisitions. For each measure, the bending trajectories of the fingers’ joints were added and time-normalized. IV. RESULTS AND DISCUSSION The main goal of this work was the implementation of a motion-capture glove, based on inertial measurement units, able to fulfill the different requisites associated with hand and fingers tracking, which can be used for hand function assessment in functional rehabilitation, as well as for motion generation in digital characters animation. In the current development stage of this project, some preliminary results concerning laboratorial testing of the developed prototype were already achieved. In regard to the technical evaluation, the results of the variability test of the measured orientations are summarized in Table II. The developed IMUs presented a low drift over time, with a small standard deviation range for each angle. Overall, a standard deviation of 0.128° is reported, which is an interesting result that shows the stability of the sensors output. Although the different outputted angles have similar results, note that the pitch measurement is the most stable one (0.112°), while the yaw measurement has a slightly larger variability (0.134°). Such result may be related to the increased instability of the magnetometer (due to variations in the magnetic distortions when in a different location from the magnetic calibration), which directly influences the yaw angle in the ARHS algorithm. TABLE II. IMUS STABILITY OVER TIME.

Range [σmin - σmax] σ per angle Global σ

Yaw (°)

Pitch (°)

Roll (°)

[0.084 - 0.254]

[0.082 – 0.181]

[0.095 – 0.185]

0.134

0.112

0.131

0.128

Figure 6 – Several example poses used for visual qualitative assessment in Blender software.

Regarding the accuracy and repeatability test, Table III summarizes the main achieved results. Through the analysis of the table, one immediately comprehends the importance of the calibration procedures. The application of the accelerometer, gyroscope and magnetometer corrections resulted in a significant decrease of the measured errors. Moreover, note that the error associated with a pitch rotation is substantially larger when compared to the yaw and roll errors (for both calibrated and non-calibrated cases). Such result might be associated with the Euler angle representation, but might also be related to errors associated with the quantization of the gyroscope raw values and other rounding errors during algorithm computation. Nevertheless, the accuracy of the system is within the limits of the end-user application, as demonstrated in the reliability evaluation. TABLE III. AHRS ACCURACY AND REPEATIBILITY TEST RESULTS.

Yaw (°)

Pitch (°)

Roll (°)

+89.34 ± 0.17

+84.14 ± 0.14

+89.83 ± 0.13

-91.45 ± 0.14

-87.31 ± 0.14

-89.25 ± 0.16

+84.83 ± 0.14

+87.90 ± 0.13

+89.1 ± 0.21

-92.51 ± 0.15

-84.30 ± 0.14

-88.3 ± 0.17

Calibrated + 90° - 90° Non-calibrated + 90° - 90°

The third evaluation comprehends the analysis of the number of frames received using the abovementioned connection to Blender. To this extent, each transmitted and received frame was associated with a time stamp and the number of received frames in each second (during 1 hour) was evaluated. At the same time, the CRC feature was used to assess the number of frame errors. This revealed a frame loss of 0.331% (298 in 90053 frames). Nevertheless, such percentage of frame loss is insignificant compared to the total number of frames received, leading to enough data for an efficient movement of the virtual hand model in Blender (25 frames per second) and for finger trajectory analysis.

In the last technical evaluation test, the output from the hand tracking system was visually assessed by comparing the 3D virtual model pose in Blender and the real hand. For each hand position in Fig. 6, the corresponding model pose is achieved without any major perceptive mismatch. The only exception is the thumb, which presented in some cases a small misplacement (mostly in the yaw angle) when compared to the real orientation. This can be expected due to its higher degrees of freedom. In fact, taking into account such observation, it is possible to conjecture that a third IMU in the thumb (in the metacarpal bone) would possibly benefit the device by more accurately track the thumb’s spatial positioning. Still regarding the blender virtual model, one is capable to conclude that the anatomical simplifications assumed for the intermediate phalanges (through the angles interpolation), as well as the DOFs restricted amplitudes in the 3D skeleton, presented a visual satisfactory motion for the 3D virtual hand model. In regard to the reliability evaluation, four tasks were performed by a healthy subject. Each task was repeated five times (Fig. 7) and the fingers extension/flexion (analyzed as the sum of the absolute measured angles) compared using ICC (Table IV). TABLE IV. AVERAGE INTRA-CLASS CORRELATION ASSOCIATED WITH THE FINGER TRACJECTORIES EVALUATION.

ICC

Task 1

Task 2

Task 3

Task 4

Mean

0.985

0.995

0.950

0.880

0.952

Individually, the lowest results occurred for the task 4, mainly due to the increased variability in execution of this task. Such variability is related with differences in the amount of finger flexion for the “mouse click” simulation among repetitions, which intrinsically resulted in different angle amplitudes and consequently decreasing the ICC value. Nevertheless, since a reliability coefficient of 0.70 or higher is commonly a criterion of acceptability for traditional upper limb function assessment instruments [9], it is reasonable to conclude that, in overall, the proposed hand tracking system

offset correction, temperature-based offset compensation and magnetic distortions correction). Moreover, the system was considered to be reliable, obtaining high ICC values (0.88 0.99) in four different tasks. Finally, its interest for hand/characters animation was also assessed in a Blender simulation, resulting in a visually acceptable result. In sum, the proposed glove presented interesting results, providing a precise finger tracking alternative to traditional clinical hand function assessment. As future work, it should be considered the introduction of an additional IMU for the thumb metacarpal bone in order to improve its motion capture. Furthermore, a more extensive user evaluation should be performed both for hand rehabilitation (healthy subjects and patients) and character animation. REFERENCES [1]

Figure 7 – Pattern comparison for finger bending trajectories in the four studied daily tasks: a) “closed fist”, b) “counting”, c) “glass” and “mouse”.

performed very well in all tasks, with a high degree of similarity in the finger flexion/extension. Note, however, that these results only express the consistency for one person. A more extensive evaluation (with a higher number of healthy subjects and preferably with patients too) should be performed for a better understanding of the real performance of the device. V.

CONCLUSION

This work presented the design and evaluation of a low-cost motion-capture glove prototype, based on inertial measurement units, for the assessment of hand function in rehabilitation and real-time automatic motion generation for digital characters animation. The proposed glove presented high stability and an overall satisfying accuracy, notably after the presented calibration procedures (accelerometer and gyroscope zero-

A. Zobaa, B. Relijn, A. Prieto, J. Cabestany, F. Sandoval, C. Oz, and M. C. Leu, “Linguistic properties based on American Sign Language isolated word recognition with artificial neural networks using a sensory glove and motion tracker,” Neurocomputing, vol. 70, no. 16, pp. 2891–2901, 2007. [2] A. S. Merians, E. Tunik, and S. V Adamovich, “Virtual reality to maximize function for hand and arm rehabilitation: exploration of neural mechanisms.,” Stud. Health Technol. Inform., vol. 145, pp. 109–25, Jan. 2009. [3] M. A. W. Z. R. D. J. R. P. D. Leonard E. Kahn, “Comparison of Robot-Assisted Reaching to Free Reaching in Promoting Recovery from Chronic Stroke.” [4] F. E. Buma, E. Lindeman, N. F. Ramsey, and G. Kwakkel, “Functional neuroimaging studies of early upper limb recovery after stroke: a systematic review of the literature.,” Neurorehabil. Neural Repair, vol. 24, no. 7, pp. 589–608, Sep. 2010. [5] G. Placidi, “A smart virtual glove for the hand telerehabilitation,” Comput. Biol. Med., vol. 37, no. 8, pp. 1100–1107, 2007. [6] I. C. S. H. Y. C. L. T. L. Zhiqiang Luo, “Building Hand Motion-Based Character Animation: The Case of Puppetry.” [7] L. Torrao, S. F. Queiros, P. M. Teixeira, J. L. Vilaca, and N. F. Rodrigues, “Palco: A multisensor realtime 3D cartoon production system,” in 2013 IEEE 2nd International Conference on Serious Games and Applications for Health (SeGAH), 2013, pp. 1–6. [8] L. Dipietro, A. M. Sabatini, and P. Dario, “A Survey of GloveBased Systems and Their Applications,” IEEE Trans. Syst. Man, Cybern. Part C (Applications Rev., vol. 38, no. 4, pp. 461–482, Jul. 2008. [9] N. P. Oess, J. Wanek, and A. Curt, “Design and evaluation of a low-cost instrumented glove for hand function assessment.,” J. Neuroeng. Rehabil., vol. 9, p. 2, Jan. 2012. [10] G. Placidi, D. Avola, D. Iacoviello, and L. Cinque, “Overall design and implementation of the virtual glove.,” Comput. Biol. Med., vol. 43, no. 11, pp. 1927–40, Nov. 2013. [11] S. O. H. Madgwick, A. J. L. Harrison, and A. Vaidyanathan, “Estimation of IMU and MARG orientation using a gradient descent algorithm.,” IEEE Int. Conf. Rehabil. Robot., vol. 2011, p. 5975346, Jan. 2011. [12] J. F. Vasconcelos, G. Elkaim, C. Silvestre, P. Oliveira, and B. Cardeira, “Geometric Approach to Strapdown Magnetometer Calibration in Sensor Frame,” IEEE Trans. Aerosp. Electron. Syst., vol. 47, no. 2, pp. 1293–1306, Apr. 2011.