A Virtual Upper Limb Prosthesis as a Training ... - Semantic Scholar

2010 7th International Conference on Electrical Engineering, Computing Science and Automatic Control (CCE 2010) Tuxtla Gutiérrez, Chiapas, México. September 8-10, 2010.

A Virtual Upper Limb Prosthesis as a Training System Barraza-Madrigal José Antonio1, Ramírez-García Alfredo1, Muñoz-Guerrero Roberto1

1

Department of Electrical Engineering, Bioelectronics Section, CINVESTAV-IPN, Mexico D.F., Mexico Phone (55) 5747-3800 E-mail: [email protected]

Abstract — A virtual reality system that improves the functional adjustment between an amputee and an active prosthesis is described. It includes the development of a virtual prosthesis and a myoelectric interface integration. The main purpose of this work is to provide a training system as a previous stage, to subjects who need to use an upper limb myoelectric prosthesis, which will allow that the users control easily a real prosthesis, optimizing the adaptation process through virtual training.

This work presents the development of a trainer based on a VP for people with an upper limb amputation. The system will help the user to familiarize it with a real prosthesis. This VP design is based on a prototype that is being developed in the CINVESTAV-IPN [2], [3], which has 4 active degrees of freedom, opposite to commercial prosthesis, these are flexion-extension elbow, pronationsupination, humeral rotation and prehension; these movements are possible due to the parallel mechanism of the prosthesis.

Keywords –– Virtual reality system, symbiosis, myoelectric prosthesis.

Besides these “simple” movements, it is possible to perform natural movement trajectories [4]. The trainer based on the virtual upper limb prosthesis enables the adaptation process through training, because it attempts that the myoelectric signals interpretation is done not only with geometric figures [5] trying to identify the movement, in addition, the interpretation of these signals will be related with a movement or a series of predefined movements which may be observed with a virtual system of the prosthesis.

I. INTRODUCTION One of the aims in rehabilitation process of an amputee subject is to provide him with a prosthesis.Nowadays, there exist different options of functional prosthesis which can be controlled by mean of a myoelectric signal (MES) [1]. Normally the MES sources are the residual muscles in the stump. Due to the amputation process the subject loss the volunteer control of these muscles, so, it is necessary to develop systems that allow a muscular rehabilitation and moreover it can be used in the adaptation of the prosthesis with the amputee subject.

II. METHODOLOGY A. Virtual Reality with MATLAB® (Virtual Reality Toolbox)

It is clear that the use of a myoelectric prosthesis requires a training phase which attempt that the patient relates the muscular contraction with the functions that it can perform. In a previous work [5], the information of the muscular activity was showed by mean of a graphical interface. The patient had to fill the area of a geometrical figure according to the contraction level. During the training, the protocol did not allow that the user gains experience about the relation between a muscular contraction and a prosthesis function. Although, two levels of contraction were controlled.

¨Virtual Reality Toolbox¨ is a solution for the visualization and interaction with dynamic systems in a tridimensional environment of virtual reality. The virtual worlds of tridimensional stages use the VRML technology (Virtual Reality Modeling Language), which is a language used to show tridimensional objects through the VRML viewer. ¨Virtual Reality Toolbox analyzes the structure of a virtual world and determines which signals are available with MATLAB®, where the objects are created and connected in a virtual world to be able, through the use of command lines, to control, and perform changes with Simulink®, which creates a physical model of the system, that is controlled through flow diagrams that are connected to a virtual world allowing to perform changes to the model and to be able to view it in a tridimensional environment.

A. Problem definition Due to the nature and characteristics that the design must cover, it is necessary to define the kind of platform to be used because at first glance, the requirements include, not only the virtual prosthesis (VP) design, moreover, that it provides movements and performs a series of trajectories, which is feasible with almost any software design such as AutoCAD®, 3D Studio Max®, Maya®, Flash®, etc. Besides, it is required that the trajectories will be performed in an autonomous way and it functions with the signals received by a PC port.

IEEE Catalog Number: CFP10827-ART ISBN: 978-1-4244-7314-4 978-1-4244-7314-4/10/$26.00 ©2010 IEEE

210

2010 7th International Conference on Electrical Engineering, Computing Science and Automatic Control (CCE 2010) Tuxtla Gutiérrez, Chiapas, México. September 8-10, 2010. There is more than one way to create a virtual world that is described with VRML code, for example, it is possible use a text editor for writing a VMRL code directly, or use a VMRL editor for make a virtual world of similar way in which a page HTML is created. These 3D tools do not use VRML as native format but exports their formats to VRML for its later use, also there exist VRML editors which use this format like their native language which guarantees that all the characteristics of the editor are compatible with this one and some of these tools even own characteristics that are unique for this format, like interpolators and sensors.

EMG (Amplifying, Preamplifiers, Filters)

User

EMG Acquisition System PC and MATLAB

Pre processing of the signal

Processing in software

PC Comunication

MATLAB and VRealm Builder

Acquisition Card

Virtual Surrounding. Computer.

Unfortunately exist very few editors advances who use this language like their native language and they are even more difficult to use than the 3D editors, except V-Realm Builder by Ligos corporation which is at the moment one of editing tools more powerful VRLM format for PC, apart one of the advantage is that Virtual Reality Toolbox includes VRealm Builder like 3D editor [6].

Virtual Prosthesis

Virtual Prosthesis Development system

Fig. 1. Stages of development.

D. Isolation -amplifier. The isolation amplifier is an amplifier where ohmic continuity between the input and the output does not exist. The input terminal reference is floating with respect to the output terminal reference and it can be that both are floating with respect to the feeding reference [8].

B. V-Realm Builder It is a software with a flexible Graphical surrounding oriented for 3D editing and it is an advisable interface for VRML Edition, since it is main format is this one. Apart from which it is a graphical interface that not only offers representation of 3D elements and tools for the interactive creation of graphical elements, but it also presents a hierarchical tree of all the present elements in the virtual World [7].

E. Isolation Stage. The acquisition system of MES is isolated, however the interface of communication with the virtual atmosphere is not because the processing signals is carried out through a not grounded PC, which puts in risk the security of the user, thus an isolation system was processed by using two isolation amplifiers ISO124p from Texas Instruments, which besides isolating it of the PC, assures that the user is never connected to physical earth and avoids electrical shocks, see Fig. 2.

V-Realm Builder is part of MATLAB® and all designs based on it are composed of 4 predefined figures: cube, pyramid, cylinder and sphere.

F. Signal preprocessing.

C. Stages of the Virtual Prosthesis.

The recorded signals were received through an acquisition card developed with a PIC18F4550 microcontroller (Microchip®) at 10 bits resolution with a sampling frequency of 1 kHz, it uses the AN0 and AN1 channels corresponding to the ADC entrances for recording data. For the processing of the signal the received samples were used to generate vectors, to allow the classification in levels of force of the muscular contraction in relation to the rank of a certain voltage.

Firstly the MES acquisition is required, for this, surface electrodes are placed on the stump of the amputee on biceps and triceps muscles, and both are received through an electromyography signal system, made up preamplifiers, amplifiers and filters to acquire and to adapt MES in order to be able to be used. At this point it is created an interface between the virtual system and the acquisition system of MES, which consists of the following stages (Fig. 1): isolation, preprocessing of the signal in hardware, computer (PC) communication, communication between the PC and MATLAB®, processing in software, communication between MATLAB® and V-Realm Builder.


Isolation

The communication with the PC is through the USB port, taking advantage of the PIC18F4550 characteristics, since it allows this type of communication. The USB port was configured to use a vector of 64 bytes as shipping data, considering a 10 bits resolution, obtaining in this way 32 samples by cycle, which 16 correspond to Channel 1 obtained from biceps and 16 more than correspond to the Channel 2 whose values come from triceps.

211


Fig. 2. Isolation Stage.

The obtained values and their position are alternated in order to obtain the readings between each channel as precise as possible and in this way to avoid the variation due to the sampling time. For the data reception on the PC, the libraries provided by MICROCHIP® (Microchip Application Libraries v2009-11-18) [9] were used, since these contain drivers necessary for the recognition of the microcontroller in the PC.

After calibration process, input and output channels were enabled on a continuous basis allowing the acquisition only of the average values, which serve as a command, it contains the instructions related to the movement of the virtual environment through V-Realm Builder thereby enabling control of the VP, which has 2 types of control: through “simple movements” or “tasks”. The “simple movements” meet previously defined vectors, which are themselves chosen according to the average value, which indicate the start and end position of the elements of the virtual world, thus allowing independent control of the actions corresponding to the VP: flexion, extension, pronation, supination, opening and closing of pliers.

G. Signal Processing Once the interface between the PC and the acquisition card was configurated, it was made the communication between the USB port and software. For this procedure MATLAB® was used, it offers the advantage of its processing tools, which allowed the development of the VP. For the transfer data from the PIC to MATLAB®, the internal ADC module was used. An optimal communication was carried out by the libraries _mpusbapi and mpusbapi.dll from MICROCHIP®, i.e., the libraries were MATLAB® drivers used for communication between the PC and the acquisition card.

The movement through tasks are not allocated for this by mean of predefined vectors, but, its answer is through vectors whose values are obtained from data tables which take the values of a motion analysis that was previously carried out [3], this analysis reflects the pre-defined task such as pouring with a jar, to take a glass of water, answering the phone, greeting or opening a door, the advantage of this movement is that not only it combines the action of “the simple” movements, but allows to introduce new motion analysis tables allowing VP generates movements that are not pre programmed. Moreover, it not only represents the signals in the virtual environment but adds graph, which show the trajectory of the values taken from the same (in degrees) and the followed trajectory of the arm (in radians).

As far the processing of the received signals, a data vector of 32 samples was taken, where 16 samples correspond to Channel 1 and the other 16 samples correspond to Channel 2, it represents a cycle of data acquisition. This process is repeated until obtain a matrix of 512 samples which 256 correspond to Channel 1 and the others 256 correspond to Channel 2, and later the average value of each one was computed. This is used as a command to perform functions. In first instance this value is used to create a system calibration stage, in which the user is asked to try to generate the highest possible level of force in order that this value is used as the maximum value for establishing the ranges of values assigning to each one of the movements that must be controlled, allowing to the system provide flexibility so that it can be used under different conditions, among which is the level of user training, fatigue due to training time, the maximum level that can be achieved among others.


Fig. 3 shows the system developed. It consists of several stages among which data collection though the acquisition card, refers the stages of communication between the acquisition card and the PC, communication between the port of the PC and processing software, a processing stage MATLAB® and the visualization of the movements in the virtual prosthesis through V-Realm Builder.

212


Vector 64 Elements 16samples Channel1 16samples Channel2

AN0 AN1

PIC 18F4550

Sent and Reception Tunnels

PC Port

Elements Concentration Matrix 512 Elements 256samples Channel 1 256samples Channel 2

USB _mpusbapi Mpusbapi.dl

Adquisition card

Avergae Value

Behavior Commands Virtual Surrounding

Tables of Data

V-Realm Buider

Behavior Commands

Flexion. Extension pronation Supination Opening of pliers Closing of pliers Pouring with a jar. To take water. To answer the telephone. Greeting. To open door. Others.

Calibration. Simple Movements

Tasks

MATLAB Computer (PC)

Fig. 3. Functional diagram of the system

III. RESULTS

was positive because it could be realized all the movements, although mainly the extension movements Fig. 5(a), flexion Fig. 5(b), closed Fig. 5(c) and opening of pliers Fig 5(d) it was easier to control than the movements of pronation Fig 5( b, c) and supination Fig. 5(d), This is due the latter are at intermediate force levels at which their control was not simple.

Regarding the VP, two models were realized, the first was developed as a wire structure, it consisted of a pair of cylinders and spheres see Fig. 4(a). Just for testing purposes, mind later undertook the design of the prosthesis more like a human upper limb; see Fig. 4(b). The second model was presented more visually appealing in some way which improvements in the evidence presented, this is because the movement effects are clearer, since in the hardwire some commands such as pronation and supination can be difficult to identify due to the shape of element used (cylinder), and in the model of the human upper limb geometric elements used (irregular Figures) facilitate the movement visualization.

Fig. 5(a). Extension and opening of pliers, (b) flexion and closed of pliers, (c) Pronation and closed of pliers (d) Supination and closed of pliers.

It was observed that the graphs of response Fig. 6(a) are very useful for both the user and the amputee user, as these clearly show the trajectory of the task carried out, see Fig. 6(b) which allows identify the movement and understand their functioning, as well as for identifying if the path made by the VP corresponds to the values obtained from the data tables generated during the motion analysis and hence infer whether the movement is taking place correctly (Fig. 6).

Fig. .(a) Wire structure and (b) Virtual Human upper limb.

As for the movements, there was a difference between the “simple movements” and the movements by “tasks” appeared, since in the “simple” movements it is easier to maintain a certain force level, more than the movements for tasks, this is due to the duration associated with the trajectories of the tasks. In both cases the response of the VP


213

2010 7th International Conference on Electrical Engineering, Computing Science and Automatic Control (CCE 2010) Tuxtla Gutiérrez, Chiapas, México. September 8-10, 2010. •

•

The model does not reduce its functionality by variations due to the form of the received signal (amplitude and phase), this is because the commands are given the average response of these signals which considerably reduces the signal levels. The control of movement carried out by tasks represents major complexity as far as its handling, which allows us to use it to generate new levels of difficulty which is helpful for the training.

The results show that the system is functional and that presents improvements as far as other systems in which the feedback between user and trainer system of acquisition for this type of signals is through charts [5].

Fig. 6.(a) The left section shows the graph generated by the values of the data tables of movement analysis, in the right the graph generated by the trajectory of the virtual upper limb can be observed, (b) Task carried out by the virtual prosthesis.

Through average of the matrix generated by the obtained data vectors, it is possible to discern between different levels from the signal and in this way to be able to use this like commands for our system.

IV. DISCUSSION A visual feedback system allows the user to control voluntary muscle contractions, towards them to be able to control the VP and thus to become familiar with a real prosthesis. The advancement of learning depends on both user and feedback system, so it is necessary that the system allows the user to exercise its muscles to learn new tasks. The proposed system, allows to the user to exercise triceps and biceps muscles, this allows to reduce the complexity of manipulation of muscle contractions, unlike other systems that require a greater number of muscles to control [10].

Finally we observed that the use of a VP (user/ prosthesis) like answer to the commands generated by the average of the received myoelectric signals, it shows a great potential like a training tool of the patient, and it even can be used like a simulation atmosphere to study new designs and methods of control. ACKNOWLEDGMENT

The system has some flexibility to adapt to any user, since at the beginning of the training it is possible to be calibrated to know the maximum voluntary contraction and thereby adapt it to the movements, similarly, after some movement it can be recalibrated to allow its use in any condition, e.g., when the user is tired.

Authors thank to the Consejo Nacional de Ciencia y Tecnología (CONACYT, México), for the scholarship granted to J.A Barraza-Madrigal; and the Instituto de Ciencia y Tecnología del Distrito Federal (ICyTDF) by the support for this research.

This system, unlike others, has the option to choose to realize “simple” movements or defined tasks, allowing the user to have different levels of difficulty, and may even change the tasks to be performed.

REFERENCES [1]

V. CONCLUSION

[3]

[2]

In this work, the authors developed a system of visual feedback (virtual upper limb prosthesis), whose purpose is to serve during a training phase for the use of a active prosthesis that responds to MES, this system was simulated with the aid of MATLAB® and V-realm Builder®, for the simulated processing and simulation of the signals we received by means of virtual surroundings. It was observed that: • The signal processing is fast, simple and it does not require many computing resources.


[4] [5] [6] [7] [8]

214

D. H. Plettenburg. Upper extremity prosthetics: current status & evaluation. 2006. 1st ed. The Netherlands: VSSD. A.Z. Escudero. Doctoral Tesis Dept of Electrical Engineering, Bioelectronics section, CINVESTAV- IPN, Distrito federal México. 2002. A.Z. Escudero, J. Alvarez, L. Leija. Development and characterisation of electromechanical muscles for driving trans- humeral myoelectric prostesis. Prosthetics and Orthotics International. 2002 Vol. 26 ; 226234. A. Ramirez-García, L. Leija and R. Muñoz. Active upper limb prothesis based on a natural movment trajectories. Prosthetics and Orthotics International. 2010. Vol. 34(1): 58-72. A. Ramírez, R. Muñoz, L. Leija and A. Vera. Muscular training system with visual feedback. Conference PAHCE 2006. Virtual Reality Toolbox User's Guide, For use with MATLAB® and Simulink, THE MathWorks, Version. V-Realm Builder User´s Guide and reference. AC. Metting van Rjin, A. Peper, C.A. Grimbergen.”The Isolation Mode Rejection Ratio in Bioelectric Amplifiers”. IEEE Transactions on Biomedical Engineering, Vol 38,11 Pág.1154-1157. (1991).

2010 7th International Conference on Electrical Engineering, Computing Science and Automatic Control (CCE 2010) Tuxtla Gutiérrez, Chiapas, México. September 8-10, 2010. http://www.microchip.com/ (Aplication Æ Wired Conectivity Æ USB ÆSoftware/ToolsÆ MHPFSUSB framework v2.4.) [10] A. Soares, A. Andrade, E. Lamounier and R. Carrijo. The development of a virtual myoelectric prosthesis controlled by an EMG pattern recognition system based on neural networks. Journal of Intelligent Information Systems. 2003. Vol. 21(2) : 127-141. [9]


215