Silver Nanowire Strain Sensors for Wearable Body Motion Tracking Shanshan Yao*, Jeong Seok Lee§, K’Ehleyr James*, Jace Miller**, Venkataramana Narasimhan**, Andrew Joseph Dickerson**, Xu Zhu†, Yong Zhu* *
Department of Mechanical and Aerospace Engineering, North Carolina State University, Raleigh, NC 27695, USA Device Innovation Lab., DMC R&D Center, Samsung Electronics, 129, Samsung-ro, Yeongtong-gu, Suwon-si, Gyenggi-do 443742, South Korea ** MSCA, Samsung Electronics America, 1301 E. lookout Drive, Richardson, TX 75082, USA † Samsung Research America – Dallas, 1301 E. lookout Drive, Richardson, TX 75082, USA Correspondence should be addressed to
[email protected] or
[email protected]
§
Abstract—This paper demonstrates a wearable body motion tracking technology in the form of data glove to measure the instantaneous bending positions of individual finger knuckles. Attached to the glove is a highly stretchable and flexible silver nanowire (AgNW) based capacitive strain sensor which can adapt to curvilinear surfaces. The sensor shows a linear response to large tensile strain up to 60% with less than 5 msec response time. Such kind of merits enable many applications, e.g. Virtue Reality, gaming, and robot control, which desire natural humanmachine interactions associated with typical human motions such as finger movements, walking, running and jumping, etc. Keywords—silver nanowires; wearable devices; capacitive sensing; strain sensors; motion tracking
I.
interpret the users’ intended motion. All these technologies face challenges to distinguish subtle and complicated human motion, such as finger’s position and movement for gesture recognition . In this paper, a highly stretchable silver nanowire (AgNW) strain sensor system is developed that can catch the finger’s position and instantaneous motion in details. The working principle of the AgNW sensor is illustrated first, followed by the sensor fabrication. Applications of the strain sensors in human motion tracking by directly mounting to skin are demonstrated. Then the user interface prototype system, in the form of a glove, is briefly described. Finally the measured performance for mobile device user interface application is reported.
INTRODUCTION
Over past decades, keyboard and mouse are two dominate interfaces for human-machine interaction. The touch screen or stylus pen can be considered as an extension of these two fundamental methods. Much research and commercialization effort has been devoted to innovative user interface technologies. The Ninetendo Wii MotionPlus, a very successful product, uses inertial sensors to implement gaming control functions. The startup companies, e.g. HelloNod, are trying to extend this inertial measurement unit (IMU) technology as a generic mobile device input method. Many reported or commercialized new methods require sensors to operate at line-of-sight condition and to “watch” the object movement from a short distance away, and then rely on complicated algorithm to interpret the meaning behinds these motions. Examples are: Microsoft’s Kinetic, using optical image detection; Sony’s Play Station Move, combining optics and IMU; Google’s Soli, exploring millimeter wave radar for motion tracking; Chrip, developing acoustics motion tracking; LeapMotion, investing on infra imaging sensing. Researchers have also explored technologies in which sensors are attached to human body, which makes the sensing mechanism more natural. Popular methods being studied over decades are BCI (brain-computer interface) and EMG (Electromyography) with a representative product Myo from Thalmic Lab. These interface technologies require delicate signal acquisition hardware and complicated data processing algorithm to
978-1-4799-8203-5/15/$31.00 ©2015 IEEE
II.
STRAIN SENSOR FABRICATION AND CHARACTERIZATION
A. Strain Sensing Mechanism Strain sensors convert mechanical deformation into electrical signal. Among various sensing mechanisms including piezoelectricity, triboelectricity and optical methods, resistive and capacitive sensing are most commonly used [1]. Gage factor (GF), defined by the relative change in resistance (for resistive strain sensors) or capacitance (for capacitive strain sensors) divided by the mechanical strain, is widely used to evaluate the strain sensors. The critical components of the highly stretchable strain sensors are the highly stretchable conductors/electrodes, which should be conductive and reliable over large strain ranges. Here novel highly conductive and stretchable conductors made of percolation networks of AgNWs embedded in polydimethylsiloxane (PDMS) are used as the stretchable conductors [2]. The AgNW based stretchable strain sensors are based on the capacitive sensing, as developed in our previous publication [3]. The capacitor is composed of two AgNW/PDMS electrodes and a thin layer of Ecoflex sandwiched in between. Under a tensile strain of ε , the length of the capacitor increases to (1 + ε )l0 . Owing to the Poisson effect, the width of and the spacing between the two
electrodes decrease to (1 −ν electrode ε )w0 and (1 −ν dielectricε )d 0 , respectively, where ν electrode and ν dielectric represent the Poisson’s ratios of the electrode and the dielectrics, respectively. Considering both the electrodes and dielectrics are mainly composed of polymers, whose Poisson’s ratios can be assumed to be ν electrode = ν dielectric = ν = 0.5 . The capacitance under tensile strain ε is therefore estimated to be (1 + ε )l0 (1 −νε ) w0 C = ε 0ε r = (1 + ε )C 0 (1) (1 −νε )d 0 where
ε0
and
(a)
ε r are the dielectric constants for the vacuum
and dielectrics, respectively. C0 represents the initial capacitance. The theoretical gage factor of a capacitive sensor is thus calculated to be GF = ΔC / C0ε = 1 . B. Sensor Fabrication The fabrication process of the strain sensors is schematically illustrated in Fig. 1. Briefly, a thin layer of liquid PDMS (Sylgard 184, Dow Corning) with the base to curing agent ratio of 10:1 is coated on a silicon substrate, degassed in a vacuum chamber and then cured at 100 oC for 1 hr to solidify the liquid. Rectangular patterns are then created using the prepared PDMS to serve as the mask for AgNW depositon. AgNWs in isopropyl alcohol (SLV-NW-90, Blue Nano) are drop cast into the area defined by the mask. The concentration of the AgNW solution is 10 mg/mL, the average length and diameter of AgNWs are 10 µm and 90 nm, respectively. After drying the AgNWs on a hotplate with a temperature of 50 oC, the mask is removed. Following that, another thin layer of PDMS with the base to curing agent ratio of 10:1 is spread onto the patterned AgNW strips, followed by degassing and curing at 80 oC for 2 hr. After the AgNW/PDMS composites are peeled off from the silicon substrate, stretchable and highly conductive conductors are formed, where AgNWs are embedded just below the surface of PDMS. Liquid metal (EGaIn, Sigma-Aldrich, ≥ 99.99%) is applied on one end of the AgNW/PDMS conductor to serve as a conformal connecting pad. After copper wires are inserted into the liquid metal, the liquid metal is encapsulated by Ecoflex 00-10 (Smooth-on, Inc.). Two of such AgNW/PDMS strips are sandwiched with a thin layer of liquid Ecoflex 00-10 as the dielectrics followed by curing the liquid at room temperature for 4 hr. Finally, three layer structures of AgNW/PDMSEcoflex-AgNW/PDMS form a capacitor which can be used for strain sensing. Besides PDMS, more compliant materials such as Dragon Skin, Ecoflex can also be used to embed AgNWs and form stretchable conductors. The resulting conductors are less conductive, but still acceptable as capacitive electrodes. C. Sensor Characterization The as-fabricated strain sensors are first characterized in the lab to obtain the calibration curve and then attached directly to the skin to test the feasibility for human motion detection. The capacitance values of the strain sensors are measured by an AD7152 evaluation board (Analog Devices).
(b)
(c)
Stretched Fig. 1. (a) Schematic illustration of strain sensor fabrication process. (b) Cross-sectional view of the sensor structure. (c) Schematic illustration of strain sensing mechanism. The schematics are not drawn to scale.
The calibration curve (Fig. 2) for the strain sensor during strain increase and decrease is obtained by stretching the sensors using a tensile stage and measuring the capacitance change at the same time. As shown in Fig. 2, the strain sensor shows a linear increase upon stretching up to a large tensile strain of 60%. The excellent stretchability is far beyond the strain range for the traditional strain gage (~5%), making the AgNW based strain sensors good candidates for wearable applications, where the strain associated with human motions can reach as high as 50% [4, 5]. The fitting result suggests a GF of 1, which is in perfect agreement with the calculation, as indicated in the previous sensing mechanism section. Besides the good linearity, the stretchable capacitive strain sensors exhibit low hysteresis. In contrast, the strain sensors based on resistive sensing are often susceptible to non-linearity and large hysteresis [6]. It is worth noting that for strain sensors with Dragon Skin and Ecoflex, higher stretchability of 100% with the same gage factor can be achieved.
the aforementioned sensors are mounted on a typical glove with associated electronics. The prototype hardware system implementation is straight forward, as shown in Fig. 5. Multichannel analog front end (by AD1752, a 12-bit capacitance-todigital converter) is capable to simultaneously acquire multisensor strain/flexing responses. The front-end is set up with 5 msec conversion time, in single-end mode, and with 4 pF dynamic range capable of 1 fF resolution. The collected data are passed down to TI CC2650, a lower power MCU chip with integrated Bluetooth Low Energy wireless connections. The data is then sent to a mobile device or a computer in a postprocess or raw format. This system is capable to transmit 200 kpbs data, in the range of 10~20 meter being test so far. The system is powered up from a 3.7 V Li-ion battery, and the total power consumption is about 10 mA. Fig. 2. Strain sensor calibration: relative capacitance change as a function of applied tensile strain during strain increase and strain decrease. The red line shows the fitting to the experimental data.
Prior to utilize the sensor for the “glove” user interface, the strain sensors are attached to the skin to demonstrate their applications in large deformation measurements: finger flexion and knee motion. The strain sensors are mounted onto the thumb of two persons with different thumb genes, as indicated in Fig. 3. The left one has the “Curved thumb” while the right one has “Straight thumb (Hitchhiker's thumb)” [7].The plots below present the relative capacitance change and the resulting strain associated with thumb flexion. With thumb-up gesture, the straight thumb cannot bend backwards as the curved thumb does, instead, the thumb bends a little forward for this particular person. As a result, the strain sensor is under a small tension when the person holds thumb up and thus a slight capacitance increase is observed. It is demonstrated that, the strain sensor is able to capture the detailed strain changes involved with finger flexion. To some extent, the strain change associated in the thumb-up sign reflects the gene differences between the two persons. In the second demonstration, the sensor is placed onto the knee when the subject is standing up. The capacitance from the sensor is monitored when the subject is walking and running. The sensor experiences a strain increase across the joint area when the knee is flexed during motion, as presented in Fig. 4. Both the amplitude and frequency of the motion can be deduced from the results. Such information can be beneficial for evaluating athletes’ performances, providing feedback to the prosthetics/robots and motion monitoring during rehabilitation.
B. Prototype Hardware – Glove for Finger Motion Tracking A wearable glove with the AgNW strain sensors for human-machine interface is shown in Fig. 6. The glove can capture the entire range of finger knuckle movement, from around -30 to 90 degree. The sensor’s response speed was measured up to 5 msec and angle resolution was up to 5 degree in the current dynamic measurement setup, both are limited by system front end analog to digital converter. The data display here only show 16 msec time stamp because it is aligned with the 60 frame/sec video signal. This result shows similar response as the sensor is directly attached to the human skin.
Fig. 3. Demonstration of using strain sensors for finger flexion detection: Relative capacitance change and the resulting strain associated with thumb flexion for two person with curved (left) and straight (right) thumbs.
The capacitive sensors exhibit good linearity, low hysteresis, fast response, and are less susceptible to overshoot and stress relaxation. In addition, the sensors are mechanically compliant and simple in fabrication. The reported strain sensors show great potentials in wearable devices, prosthetics, health monitoring and human-machine interfaces. III.
“GLOVE” USER INTERFACE PROTOTYPE
A. System Architecture For many applications, the reusable, skin-mountable, flexible and stretchable form factor is preferred. In this work,
Fig. 4. Demonstration of using strain sensors for knee motion detection: Relative capacitance change and the resulting strain associated with walking and running for five continuous cycles.
AD7152
sensor
sensor
TI CC2650
AD7152
sensor
sensor Fig. 5.
(b)
(a)
Application system architecture.
C. Prototype Hardware – Glove for Finger Motion Tracking A wearable glove with the AgNW strain sensors for human-machine interface is shown in Fig. 6. The glove can capture the entire range of finger knuckle movement, from around -30 to 90 degree. The sensor’s response speed was measured up to 5 msec and angle resolution was up to 5 degree in the current dynamic measurement setup, both are limited by system front end analog to digital converter. The data display here only show 16 msec time stamp because it is aligned with the 60 frame/sec video signal. This result shows similar response as the sensor is directly attached to the human skin. The purlicue area motion of the hand can also be tracked by this sensor technology (Fig. 7). This motion can be used to interpret more complicated index finger motion, therefore a lot of substitute hand gestures. The good stretchability enables the sensor to stay in heavily bending position for long time without fracture. Opening up the index finger induces the stretch of the sensor, therefore leading to an increase of the capacitance value. IV.
Fig. 6. Strain sensors for wearable glove application. (a) Glove with three sensors (marked by red dashed line) on the 2nd knuckle position of thumb, index and middle finger with a data acquisition system. (b) Demonstration of real time tracking of the middle figure knuckle bending by the AgNW stretch sensor glove. The particular case showing here tracks the knuckle 90 degree bending completed in about 150 msec.
sensor
Fig. 7. Strain sensor mounted at purlicue area to track subtle index figure and palm motion.
REFERENCES [1]
CONCLUSIONS
A versatile motion tracking technology based on AgNW capacitive strain sensor is demonstrated in this paper. The flexibility, stretchability, fast and sensitive response of the AgNW sensor enables the capability to obtain motion details which are not easily implemented by other technologies. The device has gone through initial cycling test and shows no device failure under month long and thousands cycles usage. The requirement on data processing to interpret complicated gesture is relative simple. Extremely low power consumption makes it very attractive for consumer applications and missions having constrains on power budget. ACKNOWLEDGMENT The authors would like to acknowledge the support from the National Science Foundation through ASSIST Engineering Research Center (EEC-1160483).
[2] [3] [4] [5] [6]
[7]
J. Park, I. You, S. Shin, and U. Jeong, "Material approaches to stretchable strain sensors," ChemPhysChem, vol. 16, pp. 1155-1163, 2015. F. Xu and Y. Zhu, "Highly conductive and stretchable silver nanowire conductors," Adv. Mater., vol. 24, pp. 5117-5122, 2012. S. Yao and Y. Zhu, "Wearable multifunctional sensors using printed stretchable conductors made of silver nanowires," Nanoscale, vol. 6, pp. 2345-2352, 2014. S. Yao and Y. Zhu, "Nanomaterial‐Enabled stretchable conductors: strategies, materials and devices," Adv. Mater., vol. 27, pp. 1480-1511, 2015. A. M. Wessendorf and D. J. Newman, "Dynamic understanding of human-skin movement and strain-field analysis," IEEE Trans. Biomed. Eng., vol. 59, pp. 3432-3438, 2012. T. Yamada, Y. Hayamizu, Y. Yamamoto, Y. Yomogida, A. IzadiNajafabadi, D. N. Futaba, et al., "A stretchable carbon nanotube strain sensor for human-motion detection," Nat. Nanotechnol., vol. 6, pp. 296301, 2011. B. Glass and J. Kistler, "Distal hyperextensibility of the thumbs," Hum. Hered., vol. 4, pp. 192-206, 1953.
