computer methods biomechanics & biomedical engineering

3 downloads 191 Views 222KB Size Report
motion recovery methods using no a priori shape models; and model based ... human motion capture from the last two decades. ... and activity understanding. .... authors present a framework for the modelling and animation of human ...
HUMAN MOTION ANALYSIS: METHODOLOGIES AND APPLICATIONS

Maria João M. Vasconcelos 1 and João Manuel R.S. Tavares 2

1. ABSTRACT The study of motion is one of the most interesting and active areas in Computational Vision, particularly considering the human motion. Human motion analysis usually follows a general framework: feature extraction, where the identification of the objects characteristics to be analyzed in the image frames is made; feature correspondence, where the problem of matching features between consecutive frames is approached; and finally high level processing can be considered, for instance, in the recognition of human movements, activities or poses. In this paper, we present a review about the leading computational techniques used in human motion analysis and some of their main applications.

2. INTRODUCTION During the last decades several surveys have been made regarding the subject of human motion analysis. The first significant review about human motion analysis was probably due to Aggarwal et al [1]. In this paper, the authors reported the developments on nonrigid motion analysis regarding the articulated and the elastic motion, and discuss both: motion recovery methods using no a priori shape models; and model based approaches. Aggarwal and Cai presented another overview of the tasks involved in human motion analysis [2] which covered the work prior to 1998. The paper focuses on three major areas related to interpreting human motion: motion analysis involving human body parts; tracking of human motion using single or multiple cameras; and recognizing human activities from image sequences. In the same year, Gavrila [3] published a survey about the analysis of human movement. The work was limited on whole-body or hand motion and does not include the work on human faces. Here the author grouped the methodologies in 2D approaches, with or without explicit shape models, and 3D approaches. Later, Moeslund and Granum [4] presented a survey about computer vision based human motion capture from the last two decades. The focus was on a general overview 1

PhD student, Laboratory of Optics and Experimental Mechanics, Institute of Mechanical Engineering and Industrial Management, Faculty of Engineering, University of Porto, Rua Dr. Roberto Frias, s/n, 4200-465 Porto, Portugal 2 Professor, Laboratory of Optics and Experimental Mechanics, Institute of Mechanical Engineering and Industrial Management, Faculty of Engineering, University of Porto, Rua Dr. Roberto Frias, s/n, 4200465 Porto, Portugal

based on the taxonomy of system functionalities: initialization, tracking, pose estimation and recognition. Throughout the paper a number of general assumptions used in this field were identified and suggestions for future research directions were presented. Recently, a survey of the various studies related to the human tracking and body parts was presented by Wang and Singh [5]. Approaches related with modeling behavior using motion analysis are also presented. In the same year, Wang et al. [6] presented a review of research on computer-vision-based human motion analysis, giving special emphasis on three major issues involved in this area, namely human detection, tracking and activity understanding. The authors also discussed some research challenges and future directions. In this paper we intend to present a review about the key computational methodologies used in human motion and some of their main applications. Usually, the study of human motion in image sequences starts with feature extraction, where the identification of the objects characteristics to be analyzed in the image frames is made. The second step regards to feature correspondence, where the problem of matching features between two consecutive frames is addressed. And, finally, after the features are extracted and correctly matched over an image sequence, high level processing is taken, for instance, in the recognition of human movements, activities or poses. 3. HUMAN MOTION METHODOLOGIES Most methods developed for human motion analysis use models to fit human body parts to the given images. The geometric structure of human body can be represented as stick figures, 2D contours or volumetric models. In [7], the authors used a stick figure model which learns the 3D variability of human posture using a set of training sequences. They developed a matching algorithm based on Dynamic Programming to establish a mapping between postures from different motion cycles. Then, the model is trained, a mean walking performance is automatically learnt and the statistics about the observed variability of the postures and motion direction are also computed. 2D contours are often used to detect humans in image sequences; for example, in [8] an algorithm was presented that consists in three main steps: detecting human candidates, validating the model of a human and tracking of the model in consequent frames. The model adopted is a six-link model with an articulated head that can cope with a frontal view of a person. It starts using simple means to find a human candidate within a region of interest and afterward validates it using an extended biped human model. In [9] the authors presented an integrated system for automatic acquisition of the human body model and motion tracking using input data acquired from multiple synchronized video streams. The system performs the tracking on the 3D voxel reconstructions computed from the 2D foreground silhouettes, the human body model used consists of ellipsoids and cylinders and is described using the twists framework resulting. Other type of methodology consists in using the appearance to construct the human model. In [10] moving people are modeled with the assumption that, while configuration can vary substantially from frame to frame, appearance does not. Thus, the authors present an algorithm that first builds a model of the appearance of the body of each individual by clustering candidate body segments and then uses this model to find all individual in each frame. A different possibility is to use a motion model to accomplish human tracking. For example, in [11] a motion model was built from the semi-automatically acquired training data and motion constraints were explored by analyzing the dependency of

joints. Both of them were then integrated into a dynamic model in order to reduce the size of the sample set. In [12] the authors combine the last two methodologies by integrating information from appearance with motion information. They use a detection style algorithm and train it to take advantage of both motion and appearance information to detect a walking person. In [13] the authors presented a robust feature-based tracking method of human motion. The approach presented enables to track motions of different body parts without articulated body models and their initialization by using a standard point-wise tracker modified for robustness and grouping image points undergoing the same rigid motions. An approach that combines the prior knowledge regarding a person’s motion with human body kinematics constraints was presented in [14]. The former technique uses an efficient feature point selection and tracking approach to compute feature points’ trajectories and then 3D motion models associated with each joint are locally tined by using the key frames, meaning frames where both legs are in contact in the floor. In [15], the authors propose a marker less system for analyzing and classifying human gait by computer vision techniques. The gait figure is extracted from the body contour by determining the body points using linear regression and motion tracking with topological analysis. Then, the detection of the gait cycle was done by symmetry analysis and the extraction of the gait figure was made using 2D stick figures; finally, kinematic analysis and feature extraction was done to classify the gait patterns. In [16] another approach for model-free marker less model and motion capture is presented. In their approach, a kinematic model and joint angle motion are extracted from volume sequences of subjects with arbitrary tree-structured kinematics. The motion capture method uses a skeleton curve, found in each frame of a volume sequence, to automatically determine kinematic postures and latter these postures will be aligned to determine a common kinematic model for the volume sequence. The motion sequence suited to this model in found through the reapplication of the kinematic model to each frame. In [17] the authors described a method for automatic person recognition from body silhouette and gait. It combines a background subtraction procedure with a simple correspondence method to segment and track spatial silhouettes of a walking figure. In order to reduce the computational cost during training and recognition, simple feature selection and parametric eigenspace representation are used. 4. APPLICATIONS The improvement of the interaction between men and machines is essential for the growth of human motion analysis. A wide variety of disciplines has been interested in human motion. For instance, in surveillance systems the human motion analysis can be used to identify suspicious movements of persons in a parking lot or to monitor the actions of individuals and classifying its nature in a commercial space. These types of activity can require a considerable effort from the human operators, since it is common to have several cameras in a parking lot or a shopping area that should be analysed simultaneously. In [18] the authors proposed an algorithm to model, segment and classify human activities in a constrained environment by using switched dynamical models. In [19] the authors analyse human behaviours by classifying the posture of the monitored person and consequently detecting corresponding events and alarm situations, like a fall. The former approach can be applied to monitor people at home, especially elders with limited autonomy, and define potential alarm situations.

In sports, the biomechanical analysis of the movements of athletes can help them to understand and improve their performances or even facilitate the recovery process after injuries. In [20] the authors present a model-based image matching technique to extract kinematic characteristics of three typical anterior cruciate ligament (ACL) injury situations, which can provide valuable information on the mechanisms for ACL injuries in sports. Other example, is the Football Interaction and Process Model system (FIPM) that can acquire action models, infer action-selection criteria and determine player and team strengths and weaknesses, [21]. Another application area where human motion analysis plays an important role is Gait Analysis. In [7] the authors propose an action specific model which automatically learns the variability of 3D human postures observed in a set of training sequences. Dynamic Programming techniques are used to synchronize the training sequences and, as a result, they obtain an action model with a representative manifold for the action; namely, the mean performance, the standard deviation from the mean performance and the mean observed direction vectors from each motion subsequence of a given length. The resulted model can be used for gait recognition applications such as in the identification of a subject when performing an action by observing only a very reduced motion portion of it. In [22] the authors show results that support vector machines are able to automatically recognize gait patterns of old and young people. Both histogram and Poincaré plot diagrams features are effective in discriminating the two age groups, which can indicate that such plots might be useful in detecting movement abnormalities or for monitoring improvements in walking performances because of treatment or intervention in a clinical/rehabilitation procedure. In medicine, the study of human motion can be also extremely valuable. In [23] the authors described a clinical gait analysis system used at the Newington Children’s Hospital, the clinical testing protocol and the algorithms used are also presented. Over ten years later, in [24], motion analysis was used in the study of spondylolisthesis and in [25] is also presented a motion study in patients with Parkinson’s Disease. In [26] the authors present tests of an extensive range of dimensionality reduction and robust classification techniques for linking pathological plantar hyperkeratosis and functional biomechanical foot data. Other area of application of human motion analysis is Computer Graphics. In [27] the authors present a framework for the modelling and animation of human characters from monocular videos. In [28] is described a real-time system for capturing humans in 3D and placing them into a mixed reality environment, where the images of the subject are constructed using a robust and fast shape-from-silhouette algorithm. 5. CONCLUSIONS The analysis of the human motion has been a subject of large research in the last decades. The detection, tracking and identification of humans have attracted great interests from computer vision researchers due to its promising and important applications in many key areas. The majority of the methodologies used for human motion analysis are based on models, for example, shape models like stick figures, 2D contours or volumetric models. Other researchers use models based on appearance to build the human model. The motion information can also be used in human motion analysis and some researchers combine both the appearance with motion information. Other examples of methodologies are feature-based or take into account the human body kinematic constraints.

So, this paper aims to provide a comprehensive survey of the most recent developments in human motion analysis, particularly in the tracking issue and its main applications, covering the latest research ranging mainly from 2003 to 2007. 6. ACKNOWLEDGEMENTS The first author would like to thank the support of the PhD grant SFRH/BD/28817/2006 from FCT – Fundação para a Ciência e Tecnologia from Portugal. This work was partially done in the scope of the project “Segmentation, Tracking and Motion Analysis of Deformable (2D/3D) Objects using Physical Principles”, reference POSC/EEASRI/55386/2004, financially supported by FCT. 7. REFERENCES 1. 2. 3. 4. 5. 6. 7. 8.

9. 10. 11. 12. 13.

Aggarwal, J. K., et al., Articulated and elastic non-rigid motion: a review, in Proceedings of the IEEE Workshop on Motion of Non-Rigid and Articulated Objects, 1994 Austin, Texas, USA, 2-14. Aggarwal, J. K. and Cai Q., Human Motion Analysis: A Review, Computer Vision and Image Understanding, 1999, Vol. 73, No. 3, 428-440. Gavrila, D. M., The Visual Analysis of Human Movement: A Survey, Computer Vision and Image Understanding, 1999, Vol. 73, No. 1, 82-98. Moeslund, T. B. and Granum E., A Survey of Computer Vision-Based Human Motion Capture, Computer Vision and Image Understanding, 2001, Vol. 81, No. 3, 231-268. Wang, J. W. and Singh S., Video Analysis of Human Dynamics - A Survey, Real-time Imaging Journal, 2003, Vol. 9, No. 5, 320-345. Wang, L., Hu W., and Tan T., Recent developments in human motion analysis, Pattern Recognition, 2003, Vol. 36, 585-601. Rius, I., et al., Automatic Learning of 3D Pose Variability in Walking Performances for Gait Analysis, International Journal for Computational Vision and Biomechanics, 2007, Vol. 1, No. 1, 33-43. Korc, F. and Hlavac V., Detection and Tracking of Humans in Single View Sequences Using 2D Articulated Models, in Human Motion: Understanding, Modelling, Capture and Animation, B. Rosenhaln, R. Klette, and D. Metaxas, Editors, 2007, Springer. Mikic, I., et al., Human Body Model Acquisition and Tracking Using Voxel Data, International Journal for Computater Vision, 2003, Vol. 53, No. 3, 199223. Ramanan, D. and Forsyth D. A., Finding and Tracking People from the Bottom Up, in IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003, Wisconsin, USA. Ning, H., et al., People tracking based on motion model and motion constraints with automatic initialization, Pattern Recognition, 2004, Vol. 37, 1423-1440. Viola, P., Jones M. J., and Snow D., Detecting Pedestrians Using Patterns of Motion and Appearance, International Journal for Computer Vision, 2005, Vol. 63, No. 2, 153-161. Gonzalez, J. J., et al., Robust tracking and segmentation of human motion in an image sequence, in International Conference on Acoustics, Speech and Signal Processing 2003 - ICASSP03, 2003, Hong Kong, China.

14. 15. 16. 17. 18. 19. 20.

21. 22. 23. 24. 25. 26. 27.

28.

Sappa, A. D., et al., Prior Knowledge Based Motion Model Representation, Electronic Letters on Computer Vision and Image Analysis, 2005, Vol. 5, No. 3, 55-67. Yoo, J. H. and Nixon M., Markerless Human Gait Analysis via Image Sequences, in International Society of Biomechanics XIXth Congress, 2003, Dunedin NZ. Chu, C. W., Jenkins O. C., and Mataric M. J., Markerless Kinematic Model and Motion Capture from Volume Sequences, in Proceedings of IEEE Computer Vision and Pattern Recognition, 2003, Wisconsin, USA. Wang, L., et al., Silhouette Analysis-Based Gair Recognition for Human Identification, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, Vol. 25, No. 12, 1505-1516. Nascimento, J. C., Figueiredo M. A. T., and Marques J. S., Segmentation and Classification of Human Activities, in HAREM 2005 - International Workshop on Human Activity Recognition and Modelling, 2005 Oxford, UK. Cucchiara, R., et al., Probabilistic Posture Classification for Human-Behavior Analysis, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 2005, Vol. 35, No. 1, 42-54. Krosshaug, T., et al., Biomechanical analysis of anterior cruciate ligament injury mechanisms: three-dimensional motion reconstruction from video sequences, Scandinavian Journal of Medicine and Science in Sports, 2007, Vol. 17, 508519. Beetz, M., Kirchlechner B., and Lames M., Computerized Real-Time Analysis of Football Games, PERVASIVE computing, 2005, Vol., 33-39. Begg, R. K., Palaniswami M., and Owen B., Support Vector Machines for Automated Gait Classification, IEEE Transactions on Biomedical Engineering, 2005, Vol. 52, No. 5, 828-838. Davis, R. B., Õunpuu S., and Tybursky D., A gait analysis data collection and reduction technique, Human Movement Science, 1991, Vol. 10, 575-587. Simsik, D., et al., Study of spondylolisthesis using videomotion analysis, Computer Methods in Biomechanics and Biomedical Engineering, 2005, Vol. Supplement 1, 293-294. Schubert, M., et al., Visual Kinesthesia and Locomotion in Parkison's Disease, Movement Disorders, 2005, Vol. 20, No. 2, 141-150. Goulermas, J. Y., et al., Automated Design of Robust Discriminant Analysi Classifier for Foot Pressure Lesions Using Kinematic Data, IEEE Transactions on Biomedical Engineering, 2005, Vol. 52, No. 9, 1549-1562. Remondino, F. and Roditakis A., Human motion reconstruction and animation from video sequences, in 17th International Conference on Computer Animation and Social Agents, 2004, Computer Graphics Society (CGS) Geneva, Switzerland. Nguyen, T. H. D., et al., Real-Time 3D Human Capture System for MixedReality Art and Entertainment, IEEE Transactions on Visualization and Computer Graphics, 2005, Vol. 11, No. 6, 706-721.