Concurrent validity of the Microsoft Kinect for ...

Journal of Biomechanics ∎ (∎∎∎∎) ∎∎∎–∎∎∎

Contents lists available at ScienceDirect

Journal of Biomechanics journal homepage: www.elsevier.com/locate/jbiomech www.JBiomech.com

Short communication

Concurrent validity of the Microsoft Kinect for assessment of spatiotemporal gait variables Ross A. Clark a,n, Kelly J. Bower b, Benjamin F. Mentiplay a, Kade Paterson a, Yong-Hao Pua c a

School of Exercise Science, Australian Catholic University, Melbourne, Australia Department of Physiotherapy, The University of Melbourne, Melbourne, Australia c Department of Physiotherapy, Singapore General Hospital, Singapore b

art ic l e i nf o

a b s t r a c t

Article history: Accepted 20 August 2013

Spatiotemporal characteristics of gait such as step time and length are often associated with overall physical function in clinical populations, but can be difficult, time consuming and obtrusive to measure. This study assessed the concurrent validity of overground walking spatiotemporal data recorded using a criterion reference – a marker-based three-dimensional motion analysis (3DMA) system – and a lowcost, markerless alternative, the automated skeleton tracking output from the Microsoft Kinect™ (Kinect). Twenty-one healthy adults performed normal walking trials while being monitored using both systems. The outcome measures of gait speed, step length and time, stride length and time and peak foot swing velocity were derived using supervised automated analysis. To assess the agreement between the Kinect and 3DMA devices, Bland–Altman 95% bias and limits of agreement, percentage error, relative agreement (Pearson's correlation coefficients: r) overall agreement (concordance correlation coefficients: rc) and landmark location linearity as a function of distance from the sensor were determined. Gait speed, step length and stride length from the two devices possessed excellent agreement (r and rc values 40.90). Foot swing velocity possessed excellent relative (r ¼0.93) but only modest overall (rc ¼ 0.54) agreement. Step time (r ¼0.82 and rc ¼ 0.23) and stride time (r ¼0.69 and rc ¼0.14) possessed excellent and modest relative agreement respectively but poor overall agreement. Landmark location linearity was excellent (R2 ¼0.991). This widely available, low-cost and portable system could provide clinicians with significant advantages for assessing some spatiotemporal gait parameters. However, caution must be taken when choosing outcome variables as some commonly reported variables cannot be accurately measured. & 2013 Elsevier Ltd. All rights reserved.

Keywords: Walk Gaming Biomechanics Physical function GAITRite Accelerometer

1. Introduction The spatiotemporal characteristics of gait have been associated with adverse outcomes such as falls risk in a range of populations, including the elderly (Barak et al., 2006) and those with neurological conditions (Grimbergen et al., 2008). Furthermore, the accurate analysis of gait in clinical settings is critical for the identification of dysfunction and to monitor change over time (Baker, 2006). Variables such as step time and length, gait speed and foot swing velocity have been assessed previously using systems including three-dimensional motion analysis (3DMA) cameras (Nieuwboer et al., 2007), foot switches (Hill et al., 1994), instrumented walkway systems (e.g. GAITRite) (Bilney et al., 2003) and accelerometers (Kavanagh and Menz, 2008). While these systems provide rich, quantitative data on a variety of gait parameters, the cost, space, time requirements and

technical expertise required typically preclude their use in clinical environments (Baker, 2006; Wall and Kirtley, 2001). Recent evidence indicates that the Microsoft Kinect™ (Kinect), which uses depth and image sensor data combined with artificial intelligence algorithms to identify anatomical landmarks in real-time without requiring markers/sensors attached to the body, can be used to validly assess lateral trunk lean during postural control tests (Clark et al., 2012). Additionally, research shows that the Kinect can be used to validly assess stride dynamics during walking (Stone and Skubic, 2011), however this study did not use the freely available automated algorithm designed by Microsoft and therefore its applicability on a large scale is limited. Consequently, the aim of this study was to assess the validity of the anatomical landmark data derived from the Kinect's skeleton tracking algorithm for examining the spatiotemporal characteristics of gait in young, healthy individuals.

2. Methods n

Correspondence to: School of Exercise Science, Faculty of Health Sciences, Australian Catholic University, Fitzroy, Victoria 3065, Australia. Tel.: þ 61 431737609. E-mail address: [email protected] (R.A. Clark).

Twenty-one young, injury free individuals (age: 26.974.5 years, height: 173.7710.5 cm, mass: 72.0711.2 kg, male¼ 10) volunteered to participate. Participants were required to wear tight-fitting shorts and an upper body garment that allowed

0021-9290/$ - see front matter & 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.jbiomech.2013.08.011

Please cite this article as: Clark, R.A., et al., Concurrent validity of the Microsoft Kinect for assessment of spatiotemporal gait variables. Journal of Biomechanics (2013), http://dx.doi.org/10.1016/j.jbiomech.2013.08.011i

R.A. Clark et al. / Journal of Biomechanics ∎ (∎∎∎∎) ∎∎∎–∎∎∎

2

placement of a reflective marker on the sternum. This study was approved by the institution's Human Research Ethics Committee and all participants provided informed consent. This study consisted of a single session, concurrent validity assessment comparing spatiotemporal gait data derived from the Kinect with a 3DMA system. Participants performed a total of nine trials in which they walked at a self-selected pace along a 2.5 m walkway, starting at 3.8 m and beginning deceleration at 1.3 m from the Kinect, alternating the starting leg each trial. This distance allowed for a minimum of one full gait cycle (i.e. a complete stride) to be recorded for each limb per walking trial that did not include the first initiation step or the final step at the end of the walkway. This resulted in all gait cycles used in analysis occurring between approximately 1.8 m and 3.5 m from the Kinect, which is within the limits of the skeleton tracking algorithm. Data from the Kinect were obtained using the official Microsoft Software Development Kit (v1.0.0.0) and customized software created in LabVIEW 2009 (National Instruments, U.S.A.). The 3D skeleton position data for the ankle and shoulder center (ie. the position in the middle of the sternum) were recorded throughout the trial, expressed relative to the Kinect camera located in front of the participant and calibrated to the global reference frame of the 3DMA system (Clark et al., 2012). These data were acquired at their native sampling frequency, which is irregular and fluctuates at around 30 Hz. The data for the 3DMA system were acquired at 120 Hz using Vicon Nexus V1.5.2 with 12 Vicon MX cameras (Vicon, U.K.). To overcome the sampling irregularity issues of the Kinect, and allow for synchronization of the data between the two devices, spline interpolation was used to resample the Kinect data to 120 Hz. Both data sets were loaded into a custom program, filtered using a Daubechies 4 undecimated wavelet 3.75 Hz lowpass filter, then crosscorrelated to temporally align the data traces before analysis. The gait event time points of toe-off and ground contact were used to identify phases of the gait cycle, and the outcome measures derived for this study were gait speed, step length and time, stride length and time and foot swing velocity. These variables were obtained using the supervised automated analysis criteria outlined in Table 1. Examples of the cross-correlated 3DMA and Kinect traces for three trials are provided in Fig. 1. Agreement between the Kinect and 3DMA devices were assessed using Bland– Altman 95% bias and limits of agreement (LoA), percentage error, and Pearson's and

concordance correlation coefficients. Because both left and right lower limb gait data were obtained using each method, the Bland–Altman analysis was adjusted for the effects of repeated measurements in a random-effects model (Carstensen et al., 2008; Myles and Cui, 2007). The percentage error was computed to express the standard deviation (SD) of the between-method difference scores (bias) as a percentage of the mean measure of the two methods (100 (2SD of bias)/[(MeanKinect þ MeanVicon)/2]) (Critchley and Critchley, 1999). Pearson's (r) and concordance correlation coefficients (rc) (both adjusted for repeated measures within each subject (Bland and Altman, 1995; Carrasco et al., 2013)) were computed to explore the relative and overall agreement, respectively, between the two methods. Specifically, Pearson's correlation coefficient assesses precision whereas the concordance coefficient assesses both precision and deviations from the line of identity (y¼ x). Correlation thresholds were set as poor ( o 0.40), modest (0.40–0.74) or excellent ( 40.75) (Fleiss, 1986). All statistical analyses were done with R version 2.15.2 (http://www.r-project.org) using the MethComp (Carstensen et al., 2013, 2008) and cccrm (Carrasco et al., 2013) packages. The relationship between distance from the Kinect and anatomical landmark accuracy error was assessed by examining the linearity of all recorded foot position values from both devices. This was determined using linear regression (R2), calculating the slope of the fit and a Bland– Altman plot of the difference between the two devices against the mean value.

3. Results The mean ( þSD) values for each outcome measure are outlined in Table 2, and percentage error and agreement values are provided in Table 3. Gait speed, step length and stride length possessed excellent relative and overall agreement between devices, with low percentage errors (r and rc values 40.90 and percentage error o8%). Foot swing velocity possessed excellent relative (r ¼ 0.93) but only modest overall (rc ¼0.54) agreement

Table 1 Gait events and variables derived from the anatomical landmark data provided by the Microsoft Kinect skeleton tracking algorithm and the 3D motion analysis (3DMA) system, along with the methods of identification used for each system. Variable

Kinect

3DMA

Toe-off

The foot center local velocity minimum immediately preceding the foot center velocity first exceeding a threshold of þ 0.1 m/s in the AP plane The ankle center local velocity minimum immediately after the ankle center velocity first dropping below a threshold of þ0.1 m/s in the AP plane Mean velocity of the shoulder center in the AP plane during the ground contact phase of the gait cycle Distance between inter-limb toe-off positions Time between inter-limb toe-off positions Distance between intra-limb toe-off positions Time between intra-limb toe-off positions Peak velocity of the foot center in the AP plane in the phase between intra-limb toe-off and ground contact

The toe marker local velocity minimum immediately preceding the toe marker velocity first exceeding a threshold of þ 0.1 m/s in the AP plane The heel marker local velocity minimum immediately after the heel marker velocity first dropping below a threshold of þ 0.1 m/s in the AP plane Mean velocity of the sternum marker in the AP plane during the ground contact phase of the gait cycle Distance between inter-limb toe-off positions Time between inter-limb toe-off positions Distance between intra-limb toe-off positions Time between intra-limb toe-off positions Peak velocity of the toe marker in the AP plane in the phase between intra-limb toe-off and ground contact

Ground contact

Gait speed (m/s) Step length (m) Step time (s) Stride length (m) Stride time (s) Foot swing velocity (m/s) AP: anterior–posterior.

Fig. 1. Cross-correlated 3DMA (solid) and Kinect (dashed) traces for three trials. These traces represent the 3DMA/Kinect derived anterior-posterior (AP) axis traces for: (A) toe/foot position (left limb example), (B) sternum/shoulder center position, and (C) toe/foot swing velocity (left limb example). Note the inter-device systematic bias for swing velocity, which influenced the poor concordance correlation coefficient value for this measure, and the differences in the initial and terminal gradients of the swing velocity traces, which influenced the step/stride time results.



and percentage error (13%). Although the Kinect and 3DMA showed excellent to modest relative agreement for the step and stride time parameters (r values were 0.82 and 0.69 respectively), overall agreement between the two devices was poor (rc values were 0.23 and 0.14 respectively). This is possibly due to the high percentage errors (16% and 19%), and the consistently higher values of the 3DMA (mean differences were 0.17 s and 0.20 s). The linearity assessment for the accuracy of landmark detection with respect to distance from the Kinect is provided in Fig. 2, with an excellent linear fit (R2 4 0.99), a slope equal to one, and the points on the Bland–Altman plot were uniformly and tightly scattered around the horizontal axis.

3

moderate percentage errors and poor overall agreement scores. Landmark identification issues may be overcome by manually identifying gait events using the raw depth and video image data, as per Stone and Skubic (2011), however this would significantly reduce the clinical feasibility of the Kinect as a gait assessment tool due to the required post-processing analysis time. With respect to hardware, the current output resolution of both the video and depth cameras is 640 480 pixels (Menna et al., 2011), with precision of both sensors reducing as distance increases (Menna et al., 2011). However, this should be overcome to some extent in the Skeleton tracking algorithm by its calculation of the global 3D centers of probability mass for each body part

4. Discussion The present study found that the Microsoft Kinect possesses concurrent validity with a 3DMA system for some spatiotemporal components of gait, however it does possess limitations. The measures which were not heavily reliant on accurate identification of specific event timing, namely the length of steps and strides and average gait velocity, were the most valid. In contrast, the assessment of step and stride time, which requires accurate identification of discrete time points, showed only modest or poor validity. The poor findings for some variables may be due to a number of software and hardware factors, including: (1) the irregular sampling rate of the skeleton tracking algorithm, which typically fluctuates between 25 and 30 frames/s, (2) poor detection of anatomical landmarks during the toe-off and ground contact phases of movement, which is likely due to the inability of the machine learning algorithm to accurately identify landmarks in this position when compared to normal standing positions, and (3) the inability to identify multiple important landmarks on the foot such as the metatarsophalangeals and calcaneus, which would allow for more precise detection of gait events. The latter two factors are likely to result in reduced ground contact time, as observed in this study, and may be most responsible for the Table 2 Mean ( þ SD) values for each outcome measure using the Microsoft Kinect and 3D motion analysis (3DMA) systems. Gait parameters

Kinect

3DMA

Step time (s) Step length (mm) Gait speed (m/s) Stride time (s) Stride length (mm) Foot swing velocity (m/s)

0.667 0.09 6437 76 0.94 7 0.09 0.737 0.06 12777 133 3.917 0.47

0.83 7 0.08 654 7 84 0.95 7 0.10 0.93 7 0.09 1282 7 138 3.497 0.35

Fig. 2. Assessment of the relationship between distance from the Kinect sensor and the accuracy of the automated foot position tracking algorithm in the anterior– posterior axis of movement. Both axes are reported in distance (millimeter) from the Kinect sensor, with the values plotted in (A) being the identified ground contact positions of the foot (Kinect) and toe (3DMA) during each trial for the left and right limbs, with a linear regression trend line and the slope of the curve calculated. (B) Presents the Bland–Altman plot with limits of agreement. The difference between the two devices is plotted on the Y-axis, and the mean score on the X-axis.

Table 3 Mean difference in Kinect and 3D motion analysis system derived gait parameters, along with 95% limits of agreement (LoA), percentage error (PE), Pearson's correlation coefficients (r), and concordance correlation coefficients (rc). Gait parameters

Mean diff

95% LoAa

PE (%)b

r

P-value

rc (95% CI)c

Step time (s) Step length (mm) Gait speed (m/s) Stride time (s) Stride length (mm) Foot swing velocity (m/s)

0.17 11.5 0.01 0.20 4.2 0.43

0.03 to 0.31 42.8 to 19.7 0.08 to 0.06 0.34 to 0.06 37.2 to 28.9 0.07 to 0.92

19 5 8 16 3 13

0.82 0.99 0.95 0.69 0.99 0.93

o 0.001 o 0.001 o 0.001 o 0.001 o 0.001 o 0.001

0.23 0.97 0.93 0.14 0.99 0.54

(0.14 to 0.33) (0.96 to 0.98) (0.87 to 0.96) (0.06 to 0.22) (0.987 to 0.995) (0.37 to 0.68)

a The 95% limits of agreement estimates were obtained from a Bland–Altman analysis which accounted for repeated (right and left limbs) measurements within each participant. b Percentage error was calculated as 100 (2 SD of bias)/[(MeanKinect þMeanVicon)/2]. c Concordance correlation coefficients take into account of both precision and deviation from the line of identity (y¼x).



4

(Shotton et al., 2011), and our excellent linearity assessment results supported this. Future Kinect iterations are likely to improve the resolution and precision of both sensors and reduce sampling rate variability, resulting in enhanced accuracy of the currently used randomized decision forest algorithm (Shotton et al., 2011), or the creation of entirely new algorithms and processing methods for identifying landmarks. This study had limitations. Firstly, our subjects were young and healthy, and therefore the ability of the Kinect to assess abnormal clinical gait patterns was not elucidated. However, as an initial proof-of-concept study this population was appropriate for a number of reasons, including not wanting to burden patients with functional deficits by testing them on a device with limited established validity. Additionally, due to its relative homogeneity because of a lack of impairments, this population would be the most difficult to obtain excellent relative agreement results on (i.e., the restriction-of-range effect). This is important because if this system is to be used on a larger scale normative data in a healthy population must be obtained. Secondly, analysis was limited to a range within 3.5 m of the Kinect, which is towards the outer limit of the Skeleton tracking algorithm. This range included gait acceleration, and required participants to stop abruptly in front of the Kinect so that deceleration was not performed during analysis cycles. This restricted capture volume is a limitation compared to other systems such as body-worn sensors, which have much greater/potentially unlimited ranges, however it is comparable to standard camera-based 3DMA which typically has a field of view restricted to a small area surrounding a force platform and is often used to assess and validate these aspects of gait (Stokic et al., 2009; Webster et al., 2005). Assessment of constant velocity gait could be achieved by beginning the test at a distance further away from the Kinect (for example, 10 m). The participant would then be detected at approximately 3.5 m from the Kinect, a point at which they had already walked 6.5 m and achieved constant velocity. Finally, our study assessed only the concurrent validity of the Kinect with a 3DMA system, with no determination of reliability performed. Despite its limitations, the Kinect provides many advantages over other sensor-based devices for examining gait. It is low-cost, widely available and does not require any markers or sensors to be attached to the body, which significantly improves clinical feasibility and patient testing time. By simply mounting a Kinect to a set position in a clinic, and performing a one-off calibration to align its field of view with the axis of walking, spatiotemporal gait characteristics can be assessed quickly and easily using automated analysis algorithms. At present there are no widely available programs to achieve this in a clinical setting; however these are likely to become available in the near future given the number of programmers developing for this system. Conflict of interest statement There is no conflict of interest.

Acknowledgments We would like to thank Karine Fortin and Stephanie Vernon for their assistance with data collection/analysis.

References Baker, R., 2006. Gait analysis methods in rehabilitation. Journal of NeuroEngineering and Rehabilitation 3, 4. Barak, Y., Wagenaar, R.C., Holt, K.G., 2006. Gait characteristics of elderly people with a history of falls: a dynamic approach. Physical Therapy 86, 1501–1510. Bilney, B., Morris, M., Webster, K., 2003. Concurrent related validity of the GAITRites walkway system for quantification of the spatial and temporal parameters of gait. Gait and Posture 17, 68–74. Bland, J.M., Altman, D.G., 1995. Calculating correlation coefficients with repeated observations: Part 2 – correlation between subjects. British Medical Journal 310, 633. Carrasco, J.L., Phillips, B.R., Puig-Martinez, J., King, T.S., Chinchilli, V.M., 2013. Estimation of the concordance correlation coefficient for repeated measures using SAS and R. Computer Methods and Programs in Biomedicine 109, 293–304. Carstensen, B., Gurrin, L.C., Ekstrom, C., 2013. MethComp: functions for analysis of agreement in method comparison studies. R Package Version 1, 21. Carstensen, B., Simpson, J., Gurrin, L.C., 2008. Statistical models for assessing agreement in method comparison studies with replicate measurements. International Journal of Biostatistics 4, 1–26. Clark, R.A., Pua, Y.H., Fortin, K., Ritchie, C., Webster, K.E., Denehy, L., Bryant, A.L., 2012. Validity of the Microsoft Kinect for assessment of postural control. Gait and Posture 36, 372–377. Critchley, L.A., Critchley, J.A., 1999. A meta-analysis of studies using bias and precision statistics to compare cardiac output measurement techniques. Journal of Clinical Monitoring and Computing 15, 85–91. Fleiss, J., 1986. The Design and Analysis of Clinical Experiments. John Wiley & Sons, New York, USA. Grimbergen, Y.M.A., Knol, M.J., Bloem, B.R., Kremer, B.P.H., Roos, R.A.C., Munneke, M., 2008. Falls and gait disturbances in Huntington's disease. Movement Disorders 23, 970–976. Hill, K.D., Goldie, P.A., Baker, P.A., Greenwood, K.M., 1994. Retest reliability of the temporal and distance characteristics of hemiplegic gait using a footswitch system. Archives of Physical Medicine and Rehabilitation 75, 577–583. Kavanagh, J.J., Menz, H.B., 2008. Accelerometry: a technique for quantifying movement patterns during walking. Gait and Posture 28, 1–15. Menna, F., Remondino, F., Battisti, R., Nocerino, E., 2011. Geometric investigation of a gaming active device. In: Proceedings of SPIE – The International Society for Optical Engineering, vol. 8085. Myles, P.S., Cui, J., 2007. Using the Bland–Altman method to measure agreement with repeated measures. British Journal of Anaesthesia 99, 309–311. Nieuwboer, A., Chavret, F., Willems, A.M., Desloovere, K., 2007. Does freezing in Parkinson's disease change limb coordination? A kinematic analysis. Journal of Neurology 254, 1268–1277. Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., Blake, A., 2011. Real-time human pose recognition in parts from single depth images. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1297–1304. Stokic, D.S., Horn, T.S., Ramshur, J.M., Chow, J.W., 2009. Agreement between temporospatial gait parameters of an electronic walkway and a motion capture system in healthy and chronic stroke populations. American Journal of Physical Medicine and Rehabilitation 88, 437–444. Stone, E.E., Skubic, M., 2011. Passive in-home measurement of stride-to-stride gait variability comparing vision and Kinect sensing. In: Proceedings of IEEE Engineering in Medicine and Biology Society, pp. 6491–6494. Wall, J.C., Kirtley, C., 2001. Strategies for clinical gait assessment. Orthopaedic Physical Therapy Clinics of North America 10, 35–54. Webster, K.E., Wittwer, J.E., Feller, J.A., 2005. Validity of the GAITRites walkway system for the measurement of averaged and individual step parameters of gait. Gait and Posture 22, 317–321.
