A Joint Localizer for Finger Length Measurements Wei-Wen Wang, Kai-Wen Lee, Sheng-Yen Lin, Chia-Hsun Lin, and Li-Chen Fu
Jin-Shin Lai, Jer-Junn Luh, Wen-Shiang Chen, and Tyng-Guey Wang
Department of Electrical Engineering National Taiwan University Taipei, Taiwan
[email protected]
Department of Physical Medicine and Rehabilitation National Taiwan University Hospital Taipei, Taiwan
Abstract—Hands play a very important role in human daily activities. Hand robotic exoskeletons for rehabilitation have been and continuously are developed by many research institutions. Generally, the information on finger length is required while using such device and assessing part of hand function. In this paper, we propose a hand joint localizer and a related hand model to measure the individual finger length by marking every key joint location on the plane. One can further translate these points into the line segments connecting them, and all these data will be stored into the computer database afterwards. The hereby developed device actually provides a simple and intuitive solution to finger length acquisition.
rehabilitation of hand sensory functions [8]. Because of different finger lengths, end-effect manipulation will cause the joint angle variations.
Keywords-finger length; hand joint; joint location; hand model; acquirement.
I.
INTRODUCTION
Disability of upper limb, especially of hands due to neurologic and orthopedic disorder or injuries is commonly observed clinically. There is higher frequency of patients getting accidental injuries in surgery and orthopedic cases. Stroke is the most common cause of severe disabilities. There are approximately 6.4 million stroke survivors in the United States and many of them live with disabled sequelae [1]. Impairment of fingers, hands and arm functions is a typical outcome following stroke or peripheral nerve injury. As we all know, hands play a very important role in activities of daily lives [2]. For recovering the normal functions, rehabilitation for injured hands is needed. Hand robotic exoskeletons for rehabilitation have been developed by many research institutions [3-6]. The finger length information is the basic requirement when using such device and part of hand function assessment and it’s useful in extended medical applications. For physical therapy, we may try to increase the range of motion and recover the muscle strength after hand injury. Many hand rehabilitation robots, and continuous passive motion devices used simple structures to pull the fingertips [79]. Such kind of devices can provide safe operation environment due to its end-effect driven property [7]. Oliver et al. set up an adjustable finger carriage connected through a linear guide allows to the device to adapt to different finger lengths, and such device is available for assessment and
This work was supported in part by National Science Council, Taiwan, under the grant: NSC 100-2321-B-002-076, and in part by National Taiwan University Hospital via contracts: NTUH.99-P08.
978-1-4673-5197-3/13/$31.00 ©2013 IEEE
Other prototypes of complex hand rehabilitation gloves or exoskeletons were developed to provide more advanced tasks [10-15]. Such devices not only need to consider the weight and range of motion when worn on human hands but also need to be adjusted to fit individual finger length. Except the above device needs to be considered the effect from different finger length, other interactive environments such as virtual reality employ the hand position, and orientation also needs this length information to enhance the precision of localization and application to capture or grasp an object with fixed width or radius [16]. For biological psychology, the finger length ratio (2D:4D) is used to determine and analyze the sexually trait [17]. The finger length information is used in several areas. Traditionally, for easy measurement reasons, people use the ruler or goniometer. Using such tools manually may take a lot of time, and it is very difficult to make the measurement process a repeatable one so that the precision and repeatability can hardly be guaranteed. On the other hand, X-ray may provide the highest precision image, but the cost is high and the image needs some calibration. Some researches use camera instead to obtain the features based on finger knuckle surface and contour [18-21]. However, the knuckle surface feature is available for PIP and DIP finger joints and can measure the length in a short time, and the measurement results can be stored in the computer. But unfortunately, for the MCP joint of each finger and wrist joint, the knuckle surfaces are not easy to identify. Different skin colors also cause some problems in threshold adjustment and calibration. Under these circumstances, a hand joint localizer, together with a hand model, is proposed to measure the finger lengths by finding every key joint’s location on a plane. These points can be used to calculate the associated line segment between every two adjacent points, and then all these information will be stored into the computer database afterwards. Such device can provide simple and intuitive finger length examiner. The rest of this paper is organized as follows. The overall system architecture and the mechanism of our hand joint
localizer are introduced in section II. After each joint angle and finger lengths are obtained, the kinematic validation of a hand model is presented in section III. Several experimental results containing the scenario of game therapy are shown in section IV. Finally, conclusions are drawn and future directions are provided in section V.
II.
SYSTEM ARCHITECTURE
A. Platform A hand joint localizer is used to measure the individual's finger lengths by marking every key joint location on the plane. We use these points to calculate the line segment between them and then store this personal information in the computer database shortly after. Such device provides a simple and intuitive means to acquire the finger lengths. The physical picture is shown in Fig. 1. The camera catches the whole hand on the horizontal plane from the beginning, and then we mark the hand joint position and fingertip points (21 points in total) manually. These mark positions are selected from the finger’s range of motion joints [22]. Finally, based on the geometrical relationship, the length of the line segment connecting two joints is calculated automatically (totally 20 line segments).
(a)
(b)
Figure 1. The platform of hand joint localizer. (a) skeleton; (b) reality.
B. Hand Model Modeling the hand may need the information such as joint location, joint angle, etc. Based on the joint position information measured by the joint localizer, the hand model can be constructed as depicted in Fig. 2. Notice that the length information is from the 2D space, and we can only get the length between two joints without each joint’s angle unless other methods are used. For most applications, we need to combine the joint angle information to realize the functional tasks. To facilitate the acquirement of 3D hand joint positions, an IMU system [23] is developed to support the computation such that the system can be integrated with interactive and virtual reality environments.
Figure 2. The hand model constructed from joint localizer.
C. System Architecture
Figure 3.
Overall system
The overall system architecture is shown in Fig. 3. For the angle calculation, we simplify the complex wrist posture in order to verify the effect of end point for different finger lengths. Three accelerometers are used to detect the inclination of each finger’s segment (distal phalanx, middle phalanx, and proximal phalanx) and one accelerometer is installed to extract the inclination of the palm. After the phalanx’s inclination is obtained, we use the difference of inclination between the neighboring phalanxes to get the joint angle. Finally, when the necessary information about length and joint for hand is collected, it will be stored into a personal database and then a hand model will be displayed in 3D graphics. III.
KINEMATICS VALIDATION
A. Forward Kinematics of Fingers In this pilot study, we focus on fingers excluding the thumb and develop the relevant applications. The position coordinates of the finger joints are shown in Fig. 4, and its associated D-H parameters is shown in Table I. Based on eq. (1), the forward kinematics from wrist to fingertip can be obtained as shown in eq. (2). y0
z0
θ2
z1
l1
θ1
y3
y2
x0 z2
θ3
y4
l2
y5
l3
x1 , x2
z3
θ4
l4
x3 z4
θ5
x4
x5 z5
Figure 4. Reference coordinates frames of hand joint.
TABLE I.
THE D-H PARAMETER TABLE FOR THE FINGER JOINT
di
θi
0
θ1
0
θ2
0
0
θ3
l3
0
0
θ4
l4
0
0
θ5
αi
Frame No.
ai
1
l1
−90
2
0
90
D
3
l2
4 5
D
⎡cosθi −sinθi cosαi sinθi sinαi ai cosθi ⎤ ⎢sinθ cosθ cosα −cosθ sinα a sinθ ⎥ i i i i i i i⎥ i−1 Ti = ⎢ ⎢ 0 sinαi cosαi di ⎥ ⎢ ⎥ 0 0 1 ⎦ ⎣ 0 0
T5 = 0T1 1T2 2T3 3T4 4T5
(1)
Figure 6. The X-ray image of the right hand.
(2)
B. Verification of Direct Kinematics of Hand Model To verify the correctness of direct kinematics, we use the software LabVIEW to simulate the posture of fingers, given the positions of the fingertip, MCP, PIP, and DIP joints, with four DOF as shown in Fig. 5. The palm curve is not considered in the model of this pilot study since the effect of the palm is not very obvious and needs some extra sensors to measure its variations.
Figure 5. The 3D simulation of finger’s posture.
IV.
ruler and hand joint localizer is about 75 seconds and 18 seconds, respectively.
EXPERIMENT RESULTS
A. Finger Length Measurement In order to compare the finger lengths measured with different instruments, we invite a male with 168cm height to be our experiment subject and the X-ray image of his right hand is used as the ground-truth and the enlarged scale is also used to calculate the length five times, as shown in Fig. 6. Table II shows the index finger’s data obtained from a ruler and our hand joint localizer after five times measurement, respectively. The measurement results are almost identical, but the time for measuring the total finger length from the beginning to the end, including data processing time, using
TABLE II.
COMPARE THE FINGER LENGTHS USING DIFFERENT INSTRUMENTS (FOR INDEX FINGER) Unit: mm
Joint segment
X-ray
Ruler
Joints’ localizer
Wrist-MCP
101.18 ± 0.34
95.8 ± 2.6
98.18 ± 2.9
MCP-PIP
41.35 ± 0.15
44.4 ± 3.5
46.14 ± 2.6
PIP-DIP
23.48 ± 0.57
23.8 ± 1.3
22.8 ± 0.8
DIP-Fingertip
20.31 ± 0.59
23.7 ± 1.2
21.1 ± 0.6
B. ROM Analysis For different subjects who use the endpoint type finger trainers as shown in Fig. 7, or grasp an object with certain fixed width size geometry, the joint angles will be different. Fig. 8 shows the joint location for index finger and the IMU installation for each finger segment. Table III shows the index finger length for two different subjects with different heights. We set the same distance (120mm) from wrist to fingertip. Table IV shows the joint angle obtained from the IMU measurement system.
Figure 7. Endpoint type finger trainer: Amadeo Robot. [7]
V.
θ4 θ5
θ3
θ2
θ1
Figure 8. The joint location for index finger and the IMU location for each segment.
TABLE III.
THE SEGMENTS LENGTH OF INDEX FINGER FOR TWO DIFFERENT SUBJECTS WITH DIFFERENT HEIGHTS Unit: mm
Joint segment
TABLE IV.
Subject A
Subject B
Wrist-MCP
98.18 ± 2.9
108.94 ± 1.6
MCP-PIP
46.14 ± 2.6
49.43 ± 1.2
PIP-DIP
22.8 ± 0.8
23.43 ± 1.2
DIP-Fingertip
21.1 ± 0.6
22.56 ± 1.1
COMPARE THE JOINT ANGLE OF FINGER BETWEEN DIFFERENT SUBJECTS (FOR INDEX FINGER) Joint
θ1 θ2 θ3 θ4 θ5
Subject A
Subject B
10° ± 1.2°
8° ± 1.9°
141° ± 1.4°
150° ± 1.1°
140° ± 1.7°
152° ± 1.3°
163° ± 0.9°
170° ± 1.1°
60° ± 0.8°
55° ± 0.5°
C. Game Exercise with Piano Game A virtual reality game which is a mission of piano playing has been developed and used to maintain patient’s interest and to enhance the effect of rehabilitation as shown in Fig. 9. The patient can press the individual fingertip. When the force sensor detects the force exerted by the patient, the corresponding key and sound of the virtual keyboard will become active.
In this research, a new method to get the finger length information without using rulers is developed. We built a hand joint localizer, together with a hand model, to measure the finger lengths, and its system architecture is detailed presented in this paper. Under this setup, the finger length information can be acquired more easily and intuitively so that the joint angle for different subjects can be extracted while the subjects are using the endpoint type finger trainers. For most novel exoskeleton type of hand device, this finger length information can assist the device better fit the user's fingers more agreeably. Besides, when performing some tasks, such as grasping an object with certain fixed width or radius, the appropriate joint angle adjustment for different users can be done. From the point of view of interaction between the haptic device and the user in the virtual reality environment, the position of each fingertip also can be more precisely localized and utilized. The database system recording all the joint positions can also provide convenient data access so as to make this processing flow more quickly. In our future work, we plan not only to improve its exterior appearance but also to enlarge the hand joint range to cope with disparities in age and sex. A novel exoskeleton hand rehabilitation robot will also be combined to verify the above-mentioned advantage of such device and besides more training tasks and applications will be performed as well. Finally, data from the healthy subjects will be collected and compared with those from the patients, so that the training outcome can be more precisely evaluated. ACKNOWLEDGMENT We would like to thank all rehabilitation therapists of National Taiwan University Hospital for their assistance and valuable suggestions. REFERENCES [1]
[2]
[3]
[4]
[5]
[6]
[7] Figure 9. The game of rehabilitation for patient: mission of play piano.
CONCLUSION
D. Lloyd-Jones, R. J. Adams, T. M. Brown, M. Carnethon, S. Dai, G. De Simone et al., "Executive summary: heart disease and stroke statistics-2010 update: a report from the American Heart Association," in Circulation, vol. 121, 2010, pp. 948-954. N. Kamakura et al. , "Patterns of static prehension in normal hands," the American Journal of Occupational Therapy, Vol. 34, No. 7, pp. 437-445, 1980. I. Sarakoglou, S. Kousidou, N. G. Tsagarakis, and D. G. Caldwell, "Exoskeleton-based exercisers for the disabilities of the upper arm and hand," in Rehabilitaiton Robotics, S. S. Kommu, Ed., ed: I-Tech Education and Publishing, 2007, pp. 499-522. E. G. Cruz and D. G. Kamper, "Use of a novel robotic interface to study finger motor control," Annals of Biomedical Engineering, vol. 38, pp. 259-268, Feb 2010. S. Ueki, H. Kawasaki, S. Ito, Y. Nishimoto, M. Abe, T. Aoki et al., "Development of a hand-assist robot with multi-degrees-of-freedom for rehabilitation therapy," IEEE/ASME Trans. Mechatronics, vol. 17, pp. 136-146, 2012. E. T. Wolbrecht, D. J. Reinkensmeyer, and A. Perez-Gracia, "Single degree-of-freedom exoskeleton mechanism design for finger rehabilitation," in IEEE Int. Conf. Rehabilitation Robotics, 2011, pp. 1-6. J. Stein, L. Bishop, G. Gillen, and R. Helbok, "A pilot study of roboticassisted exercise for hand weakness after stroke," in IEEE Int. Conf. Rehabilitation Robotics, 2011, pp. 1-4.
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
O. Lambercy, A. J. Robles, Y. Kim, and R. Gassert, "Design of a robotic device for assessment and rehabilitation of hand sensory function," in IEEE Int. Conf. Rehabilitation Robotics, 2011, pp. 1-6. L. Dovat, O. Lambercy, V. Johnson, B. Salman, S. Wong, R. Gassert et al., "A cable driven robotic system to train finger function after stroke," in IEEE Int. Conf. Rehabilitation Robotics, 2007, pp. 222-227. M. Bouzit, G. Burdea, G. Popescu, and R. Boian, "The Rutgers Master II-new design force-feedback glove," IEEE/ASME Trans. Mechatronics, vol. 7, pp. 256-263, 2002. A. Chiri, N. Vitiello, F. Giovacchini, S. Roccella, F. Vecchi, and M. C. Carrozza, "Mechatronic design and characterization of the index finger module of a hand exoskeleton for post-stroke rehabilitation," IEEE/ASME Trans. Mechatronics,, vol. 17, pp. 884-894, 2011. N. S. K. Ho, K. Y. Tong, X. L. Hu, K. L. Fung, X. J. Wei, W. Rong, and E. A. Susanto, "An EMG-driven exoskeleton hand robotic training device on chronic stroke subjects: Task training system for stroke rehabilitation," in IEEE Int. Conf. Rehabilitation Robotics, 2011, pp. 1-5. T. M. W. Burton, R. Vaidyanathan, S. C. Burgess, A. J. Turton, and C. Melhuish, "Development of a parametric kinematic model of the human hand and a novel robotic exoskeleton," in IEEE Int. Conf. Rehabilitation Robotics, 2011, pp. 1-7. A. Wege, K. Kondak, and G. Hommel, "Development and control of a hand exoskeleton for rehabilitation," in Human Interaction with Machines, G. Hommel and S. Huanye, Eds., ed: Springer Netherlands, 2006, pp. 149-157. Y. Hasegawa, Y. Mikami, K. Watanabe, and Y. Sankai, "Five-fingered assistive hand with mechanical compliance of human finger," in IEEE Int. Conf. Robotics and Automation, 2008, pp. 718-724.
[16] Salvador Cobos, Manuel Ferre, M.A. Sanchez Uran, Javier Ortego, and Cesar Pena, "Efficient human hand kinematics for manipulation tasks," in IEEE/RSJ Int. Conf. Intelligent Robots and Systems, 2008, pp. 22462251. [17] A. A. Bailey and P. L. Hurd, "Finger length ratio (2D:4D) correlates with physical aggression in men but not in women," In Biological Psychology, vol. 68, no. 3, 2005. [18] A. Kumar and C. Ravikanth, "Personal authentication using finger knuckle surface,", IEEE Trans. Information Forensics and Security, vol. 4, pp. 98-110, 2009. [19] R. Sanchez-Reillo, C. Sanchez-Avila, and A. Gonzalez-Marcos, "Biometric identification through hand geometry measurements," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, pp. 11681171, 2000. [20] V. Kanhangad, A. Kumar, and D. Zhang, "Contactless and Pose Invariant Biometric Identification Using Hand Surface,", IEEE Trans. Image Processing, vol. 20, pp. 1415-1424, 2011. [21] A. El-Sawah, N. D. Georganas, and E. M. Petriu, "Calibration and error model analysis of 3D monocular vision model based hand posture estimation," in Proc. IEEE Instrumentation and Measurement Technology Conf., 2007, pp. 1-6. [22] Cynthia C. Norkin, Joyce D. White, Measurement of Joint Motion: A Guide to Goniometry, 3rd ed., 2003, pp. 137-180. [23] W.-W. Wang, and L.-C. Fu, "Mirror therapy with an exoskeleton upperlimb robot based on IMU measurement system," in IEEE International Symposium on Medical Measurements and Applications, 2011, pp. 370375.
