Pretouch Sensing for Manipulation - Sensor Systems Laboratory

0 downloads 0 Views 2MB Size Report
Pretouch Sensing for Manipulation. Liang-Ting Jiang. Department of Mechanical Engineering. University of Washington. Seattle, WA 98105. Email: jianglt@uw.
Pretouch Sensing for Manipulation Liang-Ting Jiang

Joshua R. Smith

Department of Mechanical Engineering University of Washington Seattle, WA 98105 Email: [email protected]

Departments of Computer Science and Engineering Department of Electrical Engineering University of Washington Seattle, WA 98105 Email: [email protected]

Abstract—“Pretouch” refers to sensing modalities that are intermediate in range between touch and vision. This paper describes two pretouch senses we have created (electric field and seashell effect), and discusses the use of pretouch in manipulation.

Mid-Range

Long-Range

I. INTRODUCTION Our work on “pretouch” sensing is exploring the hypothesis that short range, non-contact sensors mounted in robot hands can benefit manipulation. In this paper we review two pretouch sensing mechanisms we have created, discuss their application in manipulation tasks, and outline future work using pretouch for manipulation. 1 A. Why Pretouch for Manipulation? The range at which vision operates is essentially unbounded. Thus vision is a natural sense to rely on for detecting and localizing objects of interest at long range. At short range, however, when it comes time to execute a grasp, vision has shortcomings. The robot’s hand and arm often occudes a headmounted camera’s view of the object to be manipulated. A head-mounted camera is not in the same coordinate frame as the hand, and thus even if the camera has an unrestricted view of the object, uncertainties associated with actuation can lead to manipulation errors. Touch sensing occurs in the frame of the hand, and is not subject to occlusion, but because it relies on contact between the manipulator and the object, touch sensing tends to displace objects whose positions are not precisely known in the hand’s coordinate frame. The aim of pretouch sensing is to combine the desirable features of these two sensing modalities: like touch, it provides information about the relative geometry of the hand and the object, in the hand’s coordinate frame; like vision, it is noncontact and does not disturb the object’s position. II. ELECTRIC FIELD PRETOUCH 1) Physics of Electric Field Sensing: In Electric Field Sensing, an AC signal is applied to a transmit electrode. This induces an AC current in the receive electrode, which is amplified and processed by the analog front end (a current amplifier, which measures current induced at the receiver) and subsequent signal processing (in our case, an analog to digital 1 This

paper is an overview extracted from our previous papers [1][2].

Fig. 1. Iso-signal surfaces illustrating response of the electric field pretouch sensor.

converter and signal processing software in a microcontroller). The sensed object modifies the current induced in the reader by interacting with the transmit and receive antennas. In shunt mode (the most commonly used mode), the object is well-grounded. Bringing a sensed object closer to transmitreceive pair shunts displacement current that would have otherwise reached the receiver, decreasing the measured sensor value as the object gets closer to the electrodes. Figure 1 shows the response of the sensors to a small grounded object; the object placed anywhere on the pictured iso-signal surfaces will yield the same sensor reading. The sensor range is determined by the transmit-receive electrode spacing. Larger spacing results in a sensor that operates on a longer length scale. Figure 1 compares mid-range and long-range sensors. Note that at the frequencies at which we are operating, the human is typically well-coupled to ground, often through the shoes. (We are in the regime of AC coupling, so although there is typically no DC electrical path from your body to ground, there is usually a relatively good high-frequency AC path to ground.) This in turn means that conductive objects that a person holds or touches are also relatively well-grounded. The sensors detect both conductive objects, and nonconductive objects whose dielectric constants differ from that of the air. Only the surface of a conductive object affects the sensors. For dielectrics, however, the entire bulk of the material affects the sensors. For this reason, the net dielectric constant of an object is proportional to density. Fig. 3 compares the response of various dielectric objects to a conductor. Some of the dielectric objects work quite well. The ones that do not are very low density.

Left Receive

Right Receive

Short Range Transmit

Mid-Range Transmit

Fig. 2. Electric field pretouch sensor hardware, designed for mounting in the fingers of the Barrett Hand. E−field Sensor Response to Non−conductive Objects

Normalized Sensor Response

2

Long Range Sensor Mid Range Sensor

1.5

1

0.5

0

−0.5

Copper Sphere

Ceramic Mug

Shot Glass

Acrylic Block

Wood Block

EPS Foam Block

1

2

3

4

5

6

Fig. 3. Response of electric field sensors to non-conductive materials (dielectrics). For dielectric objects, sensor response is typically proportional to object density, which could be another useful pre-touch cue.

III. SEASHELL EFFECT PRETOUCH SENSOR Optical and electric field sensors both rely on electromagnetics (though at greatly different wavelengths). It would be desirable to have a pretouch modality that worked using completely different physical mechanisms, to increase the chances that a second technique would detect the material if the first failed. Acoustic impedance is a convenient measurement of how much the particles move given the sound pressure at particular frequencies. At the open end of a pipe, the impedance is supposed to be zero (infinite velocity), but the sound field around the opening results in additional radiation impedance. The imaginary part of the radiation impedance introduces an end correction to the original pipe length, therefore the resonance frequency of the pipe is changed. When an object approaches to the pipe opening, the sound field between the surface of the object and the pipe opening will further introduce more length correction to the pipe. Dalmont [3] found an empircal formula for the end correction in this kind of pipe-surface configuration by fitting a function to values produced by a finite element model.

A. Sensor Design In order to detect changes in pipe resonance, a microphone collects the sound filtered by the acoustic cavity (i.e. the pipe). The pipe’s resonant frequency is found by looking for a peak in the spectrum. To avoid being confounded by features in the raw (unfiltered) ambient audio itself (including loud ambient sounds), a reference microphone is used to collect environmental sounds not filtered by the pipe; this background spectrum is subtracted from the actual pretouch sensor channel. This noise cancellation approach substantially improves sensing accuracy. Unlike the previous sensor [1], the new sensor described in this paper is completely integrated into the Willow Garage PR2 robot; all external cables and electronics have been eliminated. A customized printed circuit board (PCB) and fingertip structure was designed to hold all the electronic and mechanical components, including the microphone and the pipe. Figure 2(b) shows the PCB and fingertip fixture with all the components attached, and Figure 2(a) shows the completed pretouch sensing fingertip installed on the Willow Garage PR2 gripper. In the current implementation, we have two fingers on one of the grippers as the actual sensing channels, and a reference channel on another gripper for spectrum subtraction. The only difference between the design of the actual sensing channels and the reference channel is that the microphone on the reference channel is not attched to the pipe (acoustic cavity), so it simply collects the ambient (unfiltered) sound. The sound signal path implementation is decribed briefly here. The pipe cavity filters the ambient noise, and the sound signal collected by an electret microphone and amplified by a 40dB-Gain low-noise amplifier. The power spectral density of the sound signal from both channels are estimated using Welch spectrum estimation (Ns = 1024; overlap ratio = 70%; Hanning data taper). The spectrum of the reference channel is subtracted from the spectrum of the sensor signal before peak finding, which avoids the effect of loud sounds, outside of the sensor’s frequency range, misleading the peak tracking. The peak finding and estimation algorithm we used here is the same as described in [1], which is adapted from [4]. The new embedded sensor design eliminates the constraints on robot arm motion caused by the external wires and electronics, and thus broadens the applicability of the sensors. An object contour tracking application enabled by the new sensor design will be described later. The design presented here could be adpated to integrate the sensor into other robotic platforms. B. Evaluation The performance of the sensor on the fingertip is evaluated by collecting 1000 sensor readings (filtered spectral peak frequency) at various distance from 1mm to 10mm. A boxand-whisker plot is plotted to present the performance of the sensor (Figure 5). A contrast to noise ratio (CNR) for evaluating the sensor performance is defined as mean(f10 ) − mean(f1 ) , i = 1..10 CN R = mean(std(fi ))

11000

(a) Resonance Frequency (Hz)

10500

10000

9500

Pretouch Thresold

9000

8500

8000 Contrast to Noise Ratio (CNR) = 22.04 7500

(b)

SPI / Programmer

Microcontroller

Microphone Amplifier

1

2

Fig. 5.

3

4

5 6 Distance (mm)

7

8

9

10

Performance of the seashell effect sensor

Microphone & Sensor Cavity

Fig. 4. The seashell effect pretouch sensor (a) mounted in the PR2 robot; (b) sensor hardware

where i is the distance in millimeter. The box plot shows the filtered resonance frequency starts to decrease from 6 mm. Based on that, we select the threshold to 9500Hz (the lower quartile at 3 mm) as the first distance being able to be measured with confidence. In this way, the upper quartile at 3 mm is smaller than the lower quartile at 6 mm, so the sensor is not getting confused. All the software in this work is implmented in ROS (Robot Operation System). The signal processing and frequency estimation is implemented in Python as a ROS node, which continuously publishes the estimated resonance frequency, total signal energy, and distance in a ROS message at the rate of 20 Hz. C. Material sensitivity The seashell effect pretouch sensor does not depend on optical or electrical material properties. Instead, it depends on mechanical / acoustic properties. This characteristic makes it a good complement to long range optical depth sensors. For example, seashell effect pretouch can sense highly transparent, reflective, or light-absorbing materials, which are difficult for optical sensors. In our previous work, we reported that the only materials we have found on which the sensor fails are open foams and certain rough fabrics. In this section, we compare the sensor output for several different materials at the same distance. 1000 readings are measured for each object at

Cloth

Fig. 6.

Foam

Glass Cup

Iron Plate

Paper Box

PVC Plastic

Tissue Paper

The material sensitivity of the seashell effect pretouch sensor.

a distance of 2mm from the object’s surface. The collected data is plotted in 6. The results show that the readings from materials with more porosity, such as cloth and foam, are more noisy than others. We hypothesize that these materials are the acoustic analog of optically transparent and absorbtive materials, respectively. IV. A PPLICATIONS OF PRETOUCH A. Finger alignment and grasp pre-shaping The Barrett Hand is a multi-fingered gripper in which each finger can be actuated independently. We used electric field pretouch to independently servo each finger to be close to, but not in contact with, the object. Then by decreasing the servoing setpoint, the 3 fingers approach the object in a coordinated fashion. The fingers make contact with the object at nearly the same time, which typically allows all 3 fingers to make contact without displacing the object.

V. R ELATED W ORK Hsiao et al. [5] described an optical pretouch system, with optical emitters and detectors built into the fingers of a Barrett Hand. Capacitive sensing has been explored in robotics in various contexts. In addition to our prior work, recently Solberg, Lynch, and MacIver presented fish-inspired underwater robot capable of localizing object using electric field sensing. [6] For several earlier instances of above-water capacitive sensing for robotics, please see [7], [8], [9] and [10]. None of these schemes were targeted specifically at sensing for manipulation. We are not aware of any prior work on seashell effect sensing. VI. FUTURE WORK

Fig. 7. (a) PR2 robot probing the unseen area of the bottle and adding pretouch pointcloud before grasp planning. (b) The 3-D visualization in rviz. The red points are generated from the narrow stereo camera , and the yellow points are generated by the pretouch sensor during the pretouch motion. (c) The view from the left-narrow stereo camera overlayed with the visualization markers.

One promising future direction is to combine multiple distinct pretouch modalities. We plan to integrate pretouch sensing more deeply with grasping. In [1], we demonstrated that the pretouch sensor combined with information from the arm encoders can be used to generate additional 3D points, augmenting a parial pointcloud collected by a Kinect operating on a partially transparent object. One next step is to choose pretouch probe points in a sophisticated fashion, perhaps by probing regions of maximum uncertainty. Beyond that, we aim to truly integrate grasping and sensing. A grasp plan would be generated using partial information from a depth sensor. As the grasp plan is executed, additional information collected by the pretouch sensors would be used to refine and replan the grasp. This effort will require breaking abstraction barriers between normally separate processes of sensing, perception, and manipulation. R EFERENCES

B. Arm alignment and servoing In another application of electric field pretouch, we transmitted from the palm of the Barrett hand to receivers in each finger tip. By moving the endpoint of the arm within a preselectred plane until the 3 sensor values balanced, we were able to servo the arm to align with an object presented by a human. C. Reactive grasping of compliant objects In one of our first demonstrations of seashell effect pretouch, we showed that the seashell effect sensor can detect highly compliant and insubtantial objects, such as thin paper boxes. This object cannot be reliably detected by e-field sensors, and is too insubstantial for the PR2’s grip-mounted pressure sensors to detect, leading to crushing errors. D. Pretouch assisted grasp planning In this application, the pretouch sensor is combined with information from arm kinematics to augment a partial pointcloud created by a depth sensor.

[1] L. Jiang and J. Smmith, “Seashell effect pretouch sensing for robotic grasping,” in IEEE International Conference on Robotics and Automation (ICRA ’12), 2012. [2] B. Mayton, L. LeGrand, and J. Smith, “An electric field pretouch system for grasping and co-manipulation,” in IEEE International Conference on Robotics and Automation (ICRA ’10), 2010. [3] J. Dalmont, C. Nederveen, and N. Joly, “Radiation impedance of tubes with different flanges: numerical and experimental investigations,” Journal of Sound and Vibration, vol. 244(3), pp. 505–534, 2001. [4] E. Jacobsen and P. Kootsookos, “Fast, accurate frequency estimators,” IEEE Signal Processing Magazine, vol. 24(3), pp. 123–125, 2007. [5] K. Hsiao, P. Nangeroni, M. Huber, A. Saxena, and A. Y. Ng, “Reactive grasping using optical proximity sensors,” in To appear in International Conference on Robotics and Automation (ICRA), Kobe, Japan, 2009. [6] J. R. Solberg, K. M. Lynch, and M. A. MacIver, “Robotic electrolocation: Active underwater object localization with electric fields,” in IEEE International Conference on Robotics and Automation, Rome, Italy, 2007. [7] G. Mauer, “An end-effecter based imaging proximity sensor,” Journal of Robotic Systems, pp. 301–316, 1989. [8] J. Novak and I. Feddema, “A capacitance-based proximity sensor for whole arm obstacle avoidance,” in Proceedings of the 1992 IEEE International Conference on Robotics and Automation, Nice, France, 1992, pp. 1307–1314. [9] N. Karlsson, “Theory and application of a capacitive sensor for safeguarding in industry,” in Proceedings of IEEE Instr. and Msmt. Technology Conf.–IMTC 94, Hammamatsu, Japan, 1994. [10] D. Schmitt, J. Novak, G. Starr, and J. Maslakowski, “Real-time seam tracking for rocket thrust chamber manufacturing,” in IEEE Robotics and Automation Proceedings, 1994.