Feb 15, 2006 - of electronic noses into an intelligent system non-trivial. Challenges ... mobile robot equipped with an electronic nose, vision and tactile modal-.
Odour Recognition using Electronic Noses in Robotic and Intelligent Systems Amy Loutfi February 15, 2006
2
Abstract This thesis is about integrating the sense of smell into artificial intelligent systems. In order to endow such systems with olfaction, we use a device called an electronic nose or e-nose. An e-nose consists of a number of gas sensors with partial selectivity and a pattern recognition component trained to recognize both complex and simple types of odours. Discussed in this thesis are a number of challenges which makes the integration of electronic noses into an intelligent system non-trivial. Challenges unique to the current technological state of odour identification include the characteristics of the gas sensing technologies such as sensitivity and drift, and the limitations of the pattern recognition algorithms to cope with these characteristics. Another challenge general to olfaction is the inherent difficulty in conveying or communicating a perception of an odour to a human user. If we are to consider e-noses into today’s and tomorrow’s intelligent systems, it is important to examine the ingredients currently present in such systems. Sensing modalities such as vision, sonar and infrared are often used to collect a number of different properties about the external world. Planning and reasoning components are used to determine the appropriate responses to execute. The system can also perform actions which can then change the state of the environment. Using an electronic nose in a system containing these properties requires much more than simply mounting a physical sensor. In order for the system to really embody electronic olfaction, each level of the system architecture needs to be considered. We investigate the use of current techniques from AI and robotics that are available to confront these integration problems. Specifically this thesis investigates (1) the abstraction e-nose data from low-level sensor readings to a high-level representation of odour concepts, (2) the combination of perceptual information from the e-nose with other sensing modalities (3) coordination of sensing actions to determine where and when the e-nose should be activated in this context. We view these three constituents as important contributions for the advancement of electronic olfaction toward real application domains ranging from safety and security to the food industry. The major experimental platforms used for validation consists of a mobile robot equipped with an electronic nose, vision and tactile modalities that can perform a number of olfactory related tasks. Other experimental platforms include e-noses as part of a distributed sensing network and also as part of a visual interface interacting with humans.
ii
Acknowledgements My interest in electronic olfaction began in my fourth year of studies as an electrical engineering student at the University of New Brunswick. ¨ During that year, I had done an exchange in Sweden at Orebro University and it was then I met my current supervisor, Silvia Coradeschi. She took me under her wing and since then I had the privilege of working closely with her. I would like to express my deepest thanks to Silvia for all the time she has given me and all the supervision and guidance I have received. I am proud to have her as a supervisor, colleague and friend. In the past five years, there have been many other contributors to this work. I would like to express thanks to my co-supervisor Prof. Alessandro Saffiotti for providing useful comments and collaborating together ¨ with the writing of papers. At the beginning of my stay here in Orebro, I was part of the Biologically Inspired Systems lab and would like to express my thanks to Prof. Peter Wide for his contributions during the first half of my studies. I am particularly grateful to Prof. Dimiter Driankov for making my stay here in Sweden possible. The work that has been presented in this book is based on a number of experiments that has been a joint effort between the members of the mobile robotics lab. I would like to particularly express my gratitude to Mathias Broxvall and Lars Karlsson. Both of them have offered many insights and in some cases endured many late nights in the lab trying to complete experiments and write papers. I am also grateful to Achim Lilienthal for providing good insights to the topic of artificial olfaction and being one of the very few having also endured the trials and tribulations of working with gas sensors. Coming from Canada to Sweden and having to start from scratch with a new culture and a new language, meant also having to establish new contacts and friendships. So many people have made an impact and although it is impossible to single out any one person, there is one group of people in particular to whom I owe my gratitude. Thank you to all the PhD students (past and present) at AASS for being such excellent colleagues, providing good friendships and in some cases a family far from home. iii
iv
Publications The work presented in this thesis has also been presented in a number of journals, conference proceedings and workshops. Where appropriate, results which correspond to specific chapters in this book are indicated with the chapter number in superscript at the end of the reference.
Journals and Book Chapters 1. A. Loutfi and S. Coradeschi. Odour Recognition for Intelligent Systems. IEEE Intelligent Systems Magazine. Under Second Revision, 2006. 2. A. Loutfi and S. Coradeschi. Smell, Think and Act: A Cognitive Robot Discriminating Odours. Autonomous Robots. Accepted for Publication, 2006. (6) 3. A. Loutfi, M. Broxvall, L. Karlsson and S. Coradeschi. Object Recognition: A new application for smelling robots. Robotics and Autonomous Systems, 52(4):272-289, 2005. (6) 4. A. Loutfi, M. Lindquist and P. Wide. Artificial Perceptual Systems. Handbook of Sensor Networks: Compact Wireless and Wired Sensing Systems. Chapter 7. 2004. (3)
Conference Proceedings and Workshops 1. M. Broxvall, S. Coradeschi, A. Loutfi and A. Saffiotti. An Ecological Approach to Odour Recognition in Intelligent Environments. To appear in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA 2006). Orlando, USA, 2006. (7) 2. A. Loutfi, S. Coradeschi and A. Saffiotti. Maintaining Coherent Perceptual Information Using Anchoring. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI 2005). Pages 1477-1482. Edinburgh, Scotland, 2005. (4) v
vi 3. A. Loutfi, S. Coradeschi, L. Karlsson and M. Broxvall. Putting Olfaction into Action: Anchoring Symbols to Sensor Data Using Olfaction and Planning. Workshop on Planning and Learning in A Priori Unknown or Dynamic Domains. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI 2005). Pages 35-41. Edinburgh, Scotland, 2005.(5) 4. A. Loutfi and S. Coradeschi. Improving Odour Analysis Through Human Robot Cooperation. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA 2005), pages 4454-4460. Barcelona, Spain, 2005. (6) 5. L. Robertsson, M. Lindquist, A. Loutfi, B. Iliev and P. Wide Human Based Sensor Systems for Safety Assessment. In Proceedings of the IEEE International Conference on Computational Intelligence for Homeland Security and Personal Safety. Pages 137-142. Orlando, USA, 2005. 6. A. Loutfi, S. Coradeschi, L. Karlsson, and M. Broxvall. Putting Olfaction into Action: Using an Electronic Nose on an Multi-Sensing Mobile Robot. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2004). Pages 347-342. Sendai, Japan. 2004. (6) 7. A. Loutfi and S. Coradeschi. Forming Odour Categories Using an Electronic Nose. In Proceedings of the European Conference on Artificial Intelligence (ECAI 2004). Pages 119 - 124 Valencia, Spain. 2004.(3) 8. A. Loutfi, J. Widmark, P. Wide, E. Wikstrom. Social Agent: Expressions Driven by an Electronic Nose. In Proceedings of the International Symposium on Virtual Environments, Human Computer Interfaces and Measurement Systems (VECIMS 2003) . Lugano Switzerland. 2003. (3) 9. A. Loutfi, S.Coradeschi. Symbolically describing the sensory perceptions from an artificial nose. In Proceedings of the STarting Artificial Intelligence Researchers Symposium (STAIRS 2002). Lyon France. 2002. (3) 10. A. Loutfi and P. Wide. Symbolic estimation of food odors using fuzzy techniques. In Proceedings Of Information Processing and Management of Uncertainty (IPMU 2002). Pages 919-926. Annecy, France, 2002. (3)
vii 11. A. Loutfi and S. Coradeschi. Relying on an Electronic Nose for Odor Localization. In Proceedings of the IEEE Virtual Instrumentation and Measurement Systems. Volume 1. Pages 46-50. Anchorage, Alaska, USA, 2002. 12. A. Loutfi and S. Coradeschi, P. Wide. An Introductory Concept to Human Like Odor Descriptions by Grounding Linguistic Symbols. In Proc. IEEE Instrumentation and Measurement Technology Conference (IMTC 2002). Volume 1 . Pages 55-60. Anchorage, Alaska, USA, 2002. 13. A. Loutfi, S. Coradeschi, T. Duckett, P. Wide. Odor Source Identification by Grounding Linguistic descriptions in an artificial nose. In Proc. SPIE Conference on Sensor Fusion: Architectures, Algorithms and Applications V. Volume 4385 pages 272-281. Orlando, Florida, 2001.
A view of the mobile robot with electronic nose . . . . . . 82 Overview of the Pippi’s system architecture . . . . . . . . 84 Sampling setup for the electronic nose . . . . . . . . . . . 86 Raw sensor data . . . . . . . . . . . . . . . . . . . . . . . 87 The olfactory interface used in the robotics system . . . . 91 The local perceptual space of the robot . . . . . . . . . . 92 Effect of classification depending on distance from the source 92 Clustering performance shown in the compactness and separation measure . . . . . . . . . . . . . . . . . . . . . . 94 Finding odours by sample . . . . . . . . . . . . . . . . . . 98 Online acquisition of an odour . . . . . . . . . . . . . . . 98 A priori map of the corridor . . . . . . . . . . . . . . . . . 100 Validity analysis and clustering results . . . . . . . . . . . 102 Closer inspection of the drift in the sensor data . . . . . . 104
7.1 7.2 7.3 7.4 7.5
An example of olfaction in a Peis-Ecology . . . Snapshots of the Peis-Home . . . . . . . . . . . The refrigerator Peis . . . . . . . . . . . . . . . Response from the Fridge Alarm . . . . . . . . The connections between the Peis components
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
112 115 116 117 120
List of Tables 2.1 2.2 2.3
Example of a sevenfold classification scheme . . . . . . . . 12 Examples of commercially available SnO2 sensors . . . . . 18 Application areas for e-noses . . . . . . . . . . . . . . . . 32
3.1 3.2 3.3
Validity Measures for different cluster patitions . . . . . . Trained odours and respective symbolic descriptions . . . The identification of new odours with respect to the trained concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . Contents of the odour containers and the respective names assigned by the human user . . . . . . . . . . . . . . . . . Correct classification results on the testing data on 20 new samples of each odour . . . . . . . . . . . . . . . . . . . .
48 49
Experimental results from Ambiguous Cases . . . . . . . . Average euclidean distances of the total samples for an odour depending on training methods . . . . . . . . . . . Cluster Number with respective symbolic description . . . Classification Performance after Corridor Inspection . . . Comparison of performance results from single tasks vs. tasks in a long term scenario . . . . . . . . . . . . . . . .
95
3.4 3.5
6.1 6.2 6.3 6.4 6.5
7.1 7.2 7.3
51 52 52
97 103 103 106
List of substances in each repository. . . . . . . . . . . . . 123 Cross-validation results from context dependent training . 123 Identification of unknown samples of milk quality. . . . . 124
xv
xvi
LIST OF TABLES
Chapter 1
Introduction Smell. It helps remind us when it is time to take out the garbage, or time for the baby to change diapers. It is responsible for the perception of 7580 % of everything we taste [12]. We use it to detect danger, for example, a gas leak in our car or if the food we are cooking is burning. Industries that develop perfumes and fragrances to either help us to smell good, or in some cases prevent us from smelling bad only testify to the aesthetical importance of odours, let alone the economical value. In 2005, the sales of fragrances in Europe alone have been estimated at approximately û6,9 billion [108]. Yet, the significance of human olfaction is unparalleled in the animal kingdom where the survival of many species would be either short-term, or non-existent, without the ability for them to detect and recognize odours. The ability to artificially replicate the biological sense of smell has been a topic of interest to the sensor community for several decades. Devices called electronic noses or e-noses made their debut in the 1980’s and using an array of gas sensors together with pattern recognition techniques, e-noses have been used to distinguish a variety of odours [119, 48, 117]. In more recent years, advances in electronics, sensors and computing have made the manufacturing of compact electronic nose devices possible, and particularly suitable for integration onto platforms such as mobile robots or intelligent appliances [27, 7, 5].
1.1
Objectives of this Thesis
Research in electronic olfaction can be seen as having progressed on two fronts. One deals with the development and improvement of gas sensing technologies. This development is not within the scope of the work 1
2
Chapter 1. Introduction
presented here. Instead, our contributions pertain to the second research direction which deals with the integration of electronic olfaction to a number of existing domains. Such domains include medical diagnosis for detection of illness, environmental monitoring, quality evaluation for the foodstuff industry, and mobile robotics. In this thesis we are particularly concerned with the latter domain concerning the integration of e-noses on robotic platforms. To date, most applications for olfaction on robotic platforms has focused on the use of gas sensors in mobile robotics for the investigation of odour based navigation strategies. Examples of these platforms include (but are not limited to): trail following, plume tracking, and odour source localization [69, 124, 55]. In these works, often single sources whose chemical composition is known a priori are used. The actual electronic nose per se, that is the ability to discriminate and recognize the chemical components have been considered in only a few cases [122, 105]. Yet, the integration of a complex odour recognition component with today’s and tomorrow’s robotics and intelligent systems offer a growing number of potential applications. A robot with the ability to discriminate odours could be used in a production line testing the quality of products. A space exploration robot could collect odour samples from distant worlds and describe its perceptions back to the humans on earth. A home care robot for the elderly could identify when the milk has gone bad, throw it away and take out the garbage when it is time. In a future context, with advanced technologies working side by side with humans at home and in the industry, the possibilities for artificial olfaction are endless. Although we are still several years away from realizing these scenarios, the groundwork is now in place to start investigating the fusion of e-noses into robotic and intelligent systems. Our objective is to advocate the use of e-noses for this domain and highlight a number of novel challenges as well as benefits that arise as consequence. This is, of course, a long-term objective that needs to consider a diversity of systems and platforms. The specific objectives of this thesis however, still constitute an important and tractable intermediate step toward this goal. We consider a special type of system which we presuppose of having a number of specific properties and investigate the integration of e-noses onto that platform. In this work we consider systems that already contain other sensing modalities, such as vision and sonar, working in tandem. These systems are autonomous and need to make decisions and reason about uncertainties to choose the appropriate response to be executed. These systems are embodied and moving about in the environment affecting
1.2. Our Approach
3
Abstraction y anchoring odour categories
Electronic noses future
x
Time
spatial tactile
z
Sensing Modalities Figure 1.1: Integration of electronic olfaction into intelligent systems.
its state. Finally, these systems need to interact with humans. The specific objectives of this thesis are: Establish an effective means to collect odour samples by reasoning about sensing actions. Investigate the management of artificial olfactory information in the system in the presence of multiple modalities that collect perceptual information either in tandem or at different moments in time. Establish an effective interface to present olfactory data to a human user. Provide an experimental validations on real platforms using real sensors that show the feasibility to odour recognition in the context of an intelligent system.
1.2
Our Approach
The approach taken in this work is best defined using Figure 1.1. The Cartesian space is a representation of the system and at its origin we find the olfactory module with odour recognition processes. The y-axis represents the extension of olfactory information from a low-level representation to a high-level one, in other words from sensor data to meaningful odour categories. On this axis we also include the notion of anchoring
4
Chapter 1. Introduction
which creates and maintains the link between the symbols and the sensor data it refers to. Our discussions will begin along this axis in Chapter 3 and 4. Intelligent systems that operate in real and dynamic environments often need to determine which future actions are necessary and when they should be taken. AI techniques such as planning, reasoning and scheduling use the notion of time to generate a response or sequence of actions which permits the system to reach its objective. The x-axis, integrates olfaction into this important dimension of processing for intelligent systems, and considers how a system can use concepts such as active sensing and planning for sensing actions to acknowledge important characteristics in the drift in the e-nose data or facilitate the proper collection of odour samples. The x-y plane in particular then becomes important since high-level planners often rely on symbolic information. Chapter 5 will address these challenges. The last dimension which we add in Chapter 6 is shown on the zaxis. It considers the fusion of electronic nose information with other sensing modalities such as vision. We discuss the problem of coordinating sensing actions among different modalities, and of management of the sensor when it comes from different modalities at different instances in time. We show how the z-y plane, that is the combination of sensing modalities with anchoring can be used as a theoretical framework for the sensor fusion and that the z-x plane can be used to coordinate sensing actions among the different modalities. In this thesis, we apply the concepts we discuss in three experimental domains. An picture of each of these domains is given in Figure 1.2. The first experiments consider the intelligent system working together with humans to communicate and convey the electronic perceptions of odours. Focus is particularly placed on the x-axis where an algorithm for generating odour categories is presented and a means for presenting the categories is introduced. The second domain considers a mobile robot where the relationship between sensing and acting is examined. An electronic nose is mounted on the robot and many challenges specifically regarding odour recognition on mobile platforms are examined. This platforms forms the major experimental contribution of the work. A final domain is examined where the intelligent system is distributed in an environment. We consider the use of electronic olfaction in a intelligent room whose sensor modalities are physically distributed. We examine the possibility of the system to provide a context to the odour classification modules to improve classification performance.
1.3. Contributions of this Thesis
5
Abstraction y anchoring odour categories
Electronic noses future
x
Time
spatial tactile
z
Sensing Modalities
Figure 1.2: E-noses for different intelligent systems.
1.3
Contributions of this Thesis
This thesis is primarily concerned with the integration of odour recognition using electronic noses into intelligent systems. However, in the exploration of our objective, a number of contributions relevant to the field of electronic olfaction, mobile robotics and artificial perceptual systems were made. These specific contributions include: 1. A new application of fuzzy-logic based classification techniques on electronic nose data which coordinates symbolic concepts from a human user to the electronic perceptions of odours. 2. The consideration of the symbol to sensor data correspondence using an anchoring framework that considers olfactory data as well as performs top-down and bottom-up anchoring in the same system. This is a significant addition to the original framework proposed by [24]. 3. The use of a symbolic planner to coordinate sensing actions for multiple and heterogeneous modalities. The particular investigation of active sensing for a modality which provides other than the spatial attributes of the environment.
6
Chapter 1. Introduction 4. A first implementation of odour recognition in an intelligent environment with distributed sensing.
1.4
Thesis Outline
Chapter 2 is a background to electronic olfaction with a review of relevant algorithms for processing electronic nose data. A literature survey is interwoven in the text. This chapter provides a comprehensive but by no means exhaustive review of the field. Rather, its objective is to touch on the most common research topics relevant to electronic noses and odour discrimination. Chapter 3 details the difficulties to communicate olfactory information to an end user. An approach based on symbolic representation is presented which extends from the traditional pattern recognition techniques. A further extension is presented that uses an animated social agent to interpret the artificial perceptions of odours. Chapter 4 tackles the problem of managing symbolic information in the context of a system, particularly managing the link between high-level representation and the low-level sensor data. Chapter 5 discusses the need to coordinate sensing actions when using electronic noses. We describe a symbolic planner that determines which sensing actions should occur, when odour detection processes should be activated, and how to best collect samples. Chapter 6 presents an implementation and validation onto a mobile robot with multiple sensing modalities. In this chapter we describe the components of the robot and how they are integrated with the e-nose. A number of experiments are performed where this robot is tested in a variety of scenarios which requires the use of olfaction, vision, planning, and human interaction to accomplish its goals. Chapter 7 presents a further integration of electronic noses in another type of intelligent system. This system consists of an ecology of sensors distributed in the environment. We show in particular that information from the ecology can be used to provide a context for odour classification thus improving performance quality.
1.5. Publications
1.5
7
Publications
The results presented in this thesis have been presented in a number of publications at workshops, international conferences and journals. The publications are available online at: http://www.aass.oru.se/
8
Chapter 1. Introduction
Chapter 2
A Review of Electronic Olfaction The purpose of this chapter is to provide an overview of electronic olfaction and electronic nose technology. The chapter summarizes the relevant aspects of research and development of gas sensors, pattern recognition techniques applied to e-nose data and the applications that have contributed to the current state of machine olfaction. A number of short papers also provide a similar revision [58, 9, 23] in concise form and a handful of textbooks on electronic noses are also available [117, 48, 124]. This chapter is presented as a literature survey, interwoven with some basic theoretic information.
2.1
Biological Olfaction
When discussing artificial olfaction, a comparison of biological olfaction can be of some benefit, in particular the human olfactory system and the respective cognitive processes. Although much of the olfactory system is still largely unknown, many steps have been made in the fields of molecular biology and neurology to uncover some of the mysteries of human olfaction [12]. Odour detection begins high up in the nasal cavity in an area containing a thin sheet of cells called the olfactory epithelium. Odours are exposed by bringing a percentage of the air molecules in contact with the epithelium. As a layer of protection, mucous surrounds the epithelium to act as a barrier against harmful viruses and bacteria. The mucous in the nose is constantly flowing and is regenerated in 10 minute intervals. The epithelium contains three basic type of cells; olfactory receptor cells, the 9
10
Chapter 2. A Review of Electronic Olfaction
supporting cells and the basal cells. The receptor cells are neurons that connect to the central nervous system. The supporting cells are mainly used to produce mucous and the basal cells are used in the regeneration of the receptor cells in 4 − 8 week cycles [33, 53, 26]. See Figure 2.1 for view of the olfactory epithelium. There are an estimated 10 million olfactory receptor cells in the human nose and perhaps more than 20 times as many in the dog, which likely accounts for the dog’s superior ability to track scents [115, 147]. At the lower end of the olfactory receptor cells small hair-like projections project down into the fluid covering the mucous membrane of the epithelium plate. It has been hypothesized that the mucosa contains molecules of special protein (olfactory binding protein) that binds to odorant molecules. The olfactory binding protein with the attached odorant are then transported to the olfactory receptor cells. Extending from upper end of the olfactory receptor cells are nerve filaments which connect to the olfactory bulb at a synaptic region called the glomerulus. The glomerulus contains a cluster of synaptic regions called glomeruli. Connections are made from the glomeruli to the brain by means of the olfactory tracts. Olfactory receptors cells serve as both receptors and conductors. As receptors, they transduce the chemical stimulus from an odour into neural impulses. As conductors, they transmit these impulses to the olfactory bulb and further into the brain. This kind of dual functionality for cells is also common in primitive animals and reflects the antiquity of the olfactory system. Recognition of odours The chemical functions that trigger the recognition of odours in humans are still largely unknown. It has been hypothesised that an individual glomeruli may respond to more than one odour, however, the entire pattern of activity over all glomeruli is different for each odorant [52]. There are many structures of the brain that are linked to the olfactory pathways. These include the discrimination of conscious perception, motivational and emotional behaviours, reproduction and feeding, imprinting and memory. It was initially believed that the range of odours detected by humans could be reduced to approximately 6 or 7 primary smells [65, 6]. Different classification schemes were presented such as the one given in Table 2.1 which is based both on the perception of odours and their chemical properties. However, such schemes have failed to become widely accepted, and for the most part are not adopted as standards [34]. Instead, odours are described with elaborate descriptions, that relate not only to their characteristics but also to the experience it may
2.1. Biological Olfaction
Olfactory bulb
11
Olfactory nerve Olfactory tract
Cribriform plate
Epithelial plate Air molecules
Figure 2.1: Location and structure of the olfactory epithelium. Axons on the olfactory cells penetrate the bony cribriform plate, on their way to the central nervous system [12].
12
Chapter 2. A Review of Electronic Olfaction
evoke [36]. This lack of fixed standards in odour identification is most likely attributed to a biological property. Unlike all other sensor systems, the olfactory bulb sends tracts to several regions of the brain: one set goes to the thalamus, which projects to the region of the frontal cortex [33]; some tracts extend to the limbic system (which is closely involved with our experience of emotion and memories). This is perhaps the reason why many perceptions of odours takes place both on subconscious level and are closely linked to emotional evocation. Primary Odour Camphoraceous Musky Floral Minty Etheral Pungent Putrid
Chemical Example Hexachloroethane Butylbenzene Ethyl carbinol Menthol Diethyl ether Formic acid Butyl mercaptan
Table 2.1: Example of a sevenfold classification scheme proposed by Amoore, [6].
2.2
Emergence of Artificial Olfaction
We have already stated that the sense of smell is one of the most important senses in most biological beings. It aids in the localization of food sources, detects hazards and in some species smell is the key sense used to find a mate [12]. In humans the sense of smell has also an emotional and aesthetical aspect that not only affects the perceptions of the environment but also the social interactions with others. Yet despite the importance of the olfactory system, humans still lack the proper vocabulary to describe odours with precision. Instead odours are described using vague or abstract terms which either relate to personal experience or similarity to other odours [36]. Not only can these odour terms vary among individuals but also the same individual may change his/her own perception of odours either because of a change in the sensing mechanisms (caused by common cold, aging) or psychological factors. In industrial applications where consumer products are evaluated by their odour characteristics, the problems of varying individual perception have created a need for a quantifiable and analytical method in which to analyse odours. In an attempt to respond to this need emerged the field of machine olfaction.
2.3. Electronic Noses
13
The first attempts to measure the presence of odours were recorded as early as the 1920’s where Zwaardemaker and Hogewind [156] measured the electrical charge of a small water sample which contained a solution of the odorant. Although these experiments noted a varied conductivity of different odourous water samples, no real instrumentation was developed. In 1962, Seiyama et al. [132] found that the surface conductance of semiconducting oxides, such as zinc oxide and tin dioxide, can be influenced by the composition of ambient gases. Shortly after in 1964, works on a actual experimental instrument were published by Hartman et al. [150] and found that using a metal wire microelectrode in contact with the surface of a rod saturated with a dilute electrolyte was also capable of measuring odours. They also discovered that by varying electrodes and electrolyte, a system of several sensors could be used to operate at the same time. Later, in 1972, the first patent for a tin dioxide based chemical gas sensor was made by N. Taguchi [141], which later developed into a commercialised product known as the Taguchi Gas Sensor (TGS) [44]. The first works published on odour detection that contained specific emphasis on the problem of data processing and pattern recognition appeared in 1982 in a publication by Persaud et al. [119] . The term “electronic nose” was introduced and the definition of an electronic nose was later refined by Gardner et al. [72] to require two components: a sensing mechanism comprising of an array of electronic chemical sensors with varying selectivity, and an appropriate pattern recognition system. Today, the use of chemical sensors to measure and analyse odours is a growing field that attracts interests from the sensor communities and pattern recognition communities. A variety of sensing technology is available and presently there are several different kinds of electronic noses which use these sensors for different commercial applications. An equal variety of data processing techniques have been used with electronic nose data such as artificial neural networks (ANN), principal component analysis (PCA) and fuzzy based techniques. The possibility to quickly and effectively analyse odours has given rise to a number of industrial and research applications such as the monitoring and control of industrial processes, medical diagnosis, and control of food quality.
2.3
Electronic Noses
The general accepted definition for an electronic nose is the following, “An electronic nose is an instrument which comprises of an
14
Chapter 2. A Review of Electronic Olfaction array of electronic chemical sensors with partial specificity and an appropriate pattern recognition system, capable of recognising simple or complex odours.” (Gardner, [47]).
The traditional architecture for the identification of odours is summarised in Figure 2.2. The basic principle is that each odour leaves a characteristic pattern or fingerprint of certain compounds. Based on this assumption, the process begins by collecting the signal responses from the each sensor, which occurs by converting the chemical reaction into an electrical signal. Many chemical sensors exhibit a response profile for several analytes. The degree of selectivity and the type of odours that can be detected largely depend on the choice and number of sensors in the sensor array. The sensors are often mounted in an air tight chamber containing gas inlets and outlets to control the gas flow. The signals from each sensor are measured and processed, usually by a analogue to digital conversion that is performed by a computer. After the signal processing, the data is transformed by a variety of pre-processing techniques designed to reduce the complexity of the multi-sensor response. From this point, pattern recognition can be applied to differentiate substances from one another or train a system to provide a classification based on a collection of known responses.
Figure 2.2: An Odour Classification System.
The term “electronic nose” is rather general and consequently can be misleading as far as its capabilities. In short, electronic noses are designed to mimic the human sense of smell by providing an analysis of individual chemicals or chemical mixtures. They offer an efficient way of analysing and comparing odours. Electronic noses have yet to reach the capability of decomposing odours into their chemical components.
2.4. Sensing Technologies
2.4
15
Sensing Technologies
Within the field of gas sensors there are several different types of sensing materials that contribute to the gas sensor. A short list of the commonly used gas sensors include, metal oxide semiconductor [44], conducting polymer sensors [45], acoustic wave sensors [54], field-effect gas sensors [103], pellistors [74], and fibre-optic sensors [131]. Many of these sensing technologies are explored in the research and have also been implemented on commercially available electronic noses [5, 7]. Each of these technologies have certain disadvantages and advantages over their counterpart and choosing the right kind of gas sensor, depends on the type of application. However, in the majority of electronic nose applications there are a set of desired properties that tend to be common: Rapid response - The sensors should be able to react to and recover from an exposed odour within an acceptable time frame. This is especially important in applications that integrate electronic nose with a robotic system, such as a mobile robot that should move around in an environment and measure odour gradient [124]. Low power consumption - In most realistic systems there is a limit of power, and thus the power consumption of the sensors should be relatively low. The headspace, containing the sensor array is likely to involve other electrical equipment such as pumps and valves which often share the same power supply. Compact size - Smaller sensor size facilitates the integration of sensors in a variety of platforms, including portable electronic noses [38]. High sensitivity - The sensors should exhibit a high sensitivity to different odorants and different concentrations of the same odorant. Reliability - Gas sensors should behave predictably particularly over long periods of time. Robustness - Unwanted effects, from humidity and physical motion, should not disturb the results from the sensor readings. In the following subsections a brief overview of commonly used gas sensing technologies is provided.
2.4.1
Gas Sensor Operation
The general principle behind the chemical gas sensor is based on the fact that analyte molecules come into contact with a chemically sensitive
16
Chapter 2. A Review of Electronic Olfaction
material which causes a change in the properties of the material. This change is sensed by a transducer and is converted to an electrical signal. Two types of reactions may occur depending on the construction of the sensor. The first type of reaction is a reversible process where the analyte binds to the surface of the sensing material. The binding is determined by the intermolecular forces between the analyte and the sensing material but is usually characterised by a hydrogen bonding. Essentially, the analyte does not change but will dissociate from the sensing material when the odour concentration is removed. This type of reaction is similar to the interaction between odours and receptor proteins in biological systems. An example of a sensor based on this type of reaction is the conducting polymer sensors. The advantage to these particular kind of sensors is that they exhibit both a rapid absorption and de-absorption to gases. A second type of gas sensor is based on an irreversible reaction. This occurs when an analyte undergoes a chemical change at the sensor surface, i.e. catalysis. A common example of this kind of sensor is the Taguchi type SnO2 sensor. The general advantages to the irreversible reaction is said to be a high sensitivity to specific odours. In these cases, the sensitivity to particular odours is determined by the choice of the catalytic surface. There have been several kinds of sensing material used for the chemical gas sensor. Some examples include inorganic crystalline or pollycrtalline, organic materials and polymers and although fairly rare even biologically materials such as proteins and enzymes have been used. In the next few subsections, a discussion of the more common types of sensors is given.
2.4.2
MOSFET
The general structure of the metal oxide semiconductor field effect transistor (MOSFET) is shown in Figure 2.3. The sensor contains three components: a catalytic metal, an insulator and a transistor (semiconductor). The basic principle of the device allows gaseous compounds to react with the catalytic metal and produce species that are able to diffuse through the metal film and absorb onto a metal insulator. The absorption will cause a voltage change, δV , and change the current-voltage characteristics of the sensor. To vary the sensitivities of the sensor, different catalytic metals may be used. For example, when palladium is used the sensing characteristics display high sensitivity to hydrogen. The operating temperature of the MOSFET is in the area of 50 ◦ C to 150 ◦ C in order to increase the rate of catalytic reactions on the metal surface and prevent the absorption of water molecules. In the case of the
2.4. Sensing Technologies
17
Figure 2.3: MOSFET structure and the current-voltage characteristics. The gate voltage is denoted by VG and drain current ID .
hydrogen sensor, the typical response time is approximately 5 seconds for 50 ppm hydrogen in air. Some physical effects can decrease the sensitivity of the catalytic metal. These effects include a fouling of the sensor caused by deposition of other substances on the surface. Poisoning may also occur in the presence of strong substances such as sulfur and lead that are absorbed at the active sites on the catalytic metal and block them from further reaction. Finally, migration may occur from exposure to an abnormally high temperature and cause a loss of active surface area.
2.4.3
Semiconducting Metal Oxide Sensors (MOS)
Gas sensing technology that is based on the chemical sensitivity of semiconducting metal oxides have been more used to make arrays for odour measurement than any other class of gas sensor. In much of the early research on electronic noses, the MOS was the dominating technology for building the sensor arrays. The MOS has a similar operating principle as the MOSFET and is composed of a heating element coated with a semiconductor, most typically tin dioxide (SnO2 ). The sensing material is then doped with small amounts of catalytic metal additives for e.g. palladium or platinum. The doping of the sensor changes its operating conditions by changing the particle size of the sintered material thus having an effect on the selectivity of the sensor to different substances. One of the major manufacturers of the MOS sensor is the Figaro Engineering company whose sensors were developed by N. Taguchi [44]. Some examples of different types of the Taguchi Gas Sensor (TGS) and their odour sensitivities are given in Table 2.4.3.
18
Chapter 2. A Review of Electronic Olfaction
O2
O SnO SnO Electron 2
2-x
2-x
Grain Boundary
Reducing Gas
O SnO SnO Electron 2
2-x
2-x
Grain Boundary
Figure 2.4: Model of the inter-grain potential barrier in the absence of a gas (left) and the presence of gases (right).
Category of Odours Detected Air Contaminants Combustible Gas Toxic Gases (Ammonia) Toxic Gases (Hydrogen Sulfide) Cooking Vapours
Table 2.2: Examples of commercially available SnO2 sensors provided by the Figaro Engineering Company.
2.4. Sensing Technologies
19
The TGS sensor operates by allowing electrical current to flow through the grain boundaries of the SnO2 micro-crystal surface. At the grain boundary, oxygen is absorbed and forms a potential barrier. The size of this potential barrier is reflected in the value of the sensor’s resistance, the higher the barrier the more resistance across the sensor (see Figure 2.4). Contrarily, exposure to a deoxidising gas, will deplete the grain boundaries and thus electrical charge flows more freely and the overall resistance is reduced. The relationship between the concentration of deoxidising gas and the sensor resistance can be expressed by the following equation, R = AC −α
(2.1)
Where R is the resistance of the sensor, A and α are both constants and C is the concentration of the sampling gas. Typically the relationship between the sensor resistance and the concentration of deoxidising gas is linear on a logarithmic scale within a range of concentration (up to several thousand ppm). Each sensor normally is sensitive to several deoxidising gases, with most sensitivity to an optimised gas determined by the fabrication of the physical sensing materials. A graphical representation of the sensor’s behaviour with relation to the varying concentration of gas is shown in Figure 2.5. A final requirement to sensor operation is to supply an input power source for the internal heater. Heating the sensors to a high temperature, between 300◦ C and 500◦ C increases the sensitivity characteristics. This is partly due to the fact that temperatures above 200◦ C increase the rate of reactions on the oxide surface. Additionally, to avoid covering the sensor surface with water particles, the temperature of the sensor should remain above 100◦ C. The MOS sensor is beneficial due to its high sensitivity to specific gases in the order of 10 − 500 ppm. Perhaps the most attractive feature of these gas sensor is the low effectiveness and a usable life span of 3 − 5 years, depending on the usage of the sensor. A few drawbacks of the sensors however, include a dependency between external humidity and temperature effects to the rate of reaction to a gas, and high power consumption.
2.4.4
Conducting Polymer Sensors
Conducting polymers are another popular sensing technology that is based on measuring the resistance of a thin film polymer (electropolymerization). Recall that two different kinds of gas sensing types were
20
Chapter 2. A Review of Electronic Olfaction
Figure 2.5: Sensitivity characteristics of the TGS822 sensor to ethanol vapour and other various gases and vapours. The ordinate is the ratio of the sensor’s resistances measured at various concentrations of gases compared to the sensor resistance at 300 ppm ethanol vapour, provided by [44].
2.5. Data Processing Techniques
21
mentioned, those based on reversible reaction and those based on an irreversible reaction with analyte molecules. Conducting polymers are an example of reversible reaction gas sensing type. The response given by the sensor is created by a chemical reaction that occurs on the surface of a polymer placed between two electrodes. Typical polymers are constructed from monomers such as pyrrole, aniline or thiophene. When a polymer comes into contact with particular gases, carriers on the polymer chain become mobile and produce an electrical conductivity which is then measured. Conducting polymers differ from the metal oxide sensor in that they are sensitive to a wide variety of organic vapours, however the actual level of sensitivity is approximately one order of magnitude less than the MOS. In addition, the devices are small with low power consumption and can operate at room temperature. The response to an odour is also rapid (2 to 20 sec) and due to the reversibility of the reaction mechanism, the recovery is rapid when the vapour is removed. In this thesis the conducting polymer sensors are mostly used in experimental validations.
2.5 2.5.1
Data Processing Techniques Pre-processing
When the array has sampled an odour and the information is stored in the PC, the data analysis can begin. In most applications, the data is often multi-dimensional and the objectives of the data pre-processing are the following: Noise Reduction may be used to reduce the noise and anomalous data readings that are traced back to both the sampling frequency and the analogue to digital conversion. Normalisation of the sensor data or re-scaling techniques may be used to fix the range of the sensor data, typically between 0 and 1. The motivations behind data normalisation are twofold. The first is to cope with multi sensor inputs that may be of varying magnitudes. Secondly, the processing time for many pattern recognition techniques such as an ANN can be decreased when the all input variables are of the same magnitude as the chosen weights (more about neural networks is given in Section 2.5.3). Compensation for the reference gas can be of benefit before the application of pattern recognition techniques. Compensation involves taking into account the variations in the initial or steady
22
Chapter 2. A Review of Electronic Olfaction state values that may be caused from drift or change in environmental conditions.
Data Reduction attempts to understand the relationship between each of the sensor’s response and the odour. Typically, the goal of the analysis is to reduce the complexity or dimensionality of the data by eliminating the co-dependencies. Many statistical techniques are available for this process and some of the techniques can also be applied to perform pattern recognition, such as principal component analysis which is mentioned in the next section.
2.5.2
Statistical Pattern Recognition
The major problem with analysing the data from an electronic nose is to determine the relationship that exists between a set of independent variables (the output from n sensors) and a set of dependent variables (odour classes) [48]. The processing of electronic nose data as it applies to multivariate analysis can be divided into two approaches. The first approach is a statistical one. The second approach is a biologically inspired approach using a manner similar to human cognitive processing, namely a neural network. The classification tree by [48] shown in Figure 2.6 is an illustration of the subdivision of the various pattern recognition techniques.
Figure 2.6: Classification scheme presented by [48] of the multivariate data processing techniques commonly employed to analyse data from electronic nose instrumentation.
2.5. Data Processing Techniques
23
There are many available statistical techniques that can be applied. The techniques below are those which have been previously used specifically on electronic nose data. Multiple Linear Regression (MLR) - is a statistical technique that uses the independent variables or in this case, the sensor responses to predict the dependent variables by performing a least-squares fit of the data to a function of the form csj = b1j x1j + b2j x2j + · · · + bij xij + bnj xnj + e
(2.2)
where bij are the partial regression coefficients and the discrepancy between the observed and calculated values is described by the residual e. For electronic nose data, the goal of MLR is to calculate the regression coefficients for the sensors so that the predicted component concentration values (which in turn is relative to a measure of voltage across a sensor), csj are as close to the measured values as possible. Successful application of a MLR have been outlined in [49], however, few drawbacks were mentioned such as the sensitivity to noise, and the treatment of co-linearity in the gas sensing array. Partial Least Squares (PLS) - is another linear regression technique that is based on both MLR and principal component analysis. Instead of representing the regression coefficient as a relation between X = {x1j , x2j , . . . , xij } and csj like MLR, a new function is used which contains variables W , P , and csj . W is the set of coefficient weights and P is a set of loadings to describe the relationship between the variables. Much in the same way as a mean value gains precision as the more measurements are made, PLS is also more precise than MLR. This is because unlike MLR, in PLS the variables in X do not need to be orthogonal. Instead they can partly contain the same information, or be collinear because PLS extracts the latent structures in the data which have the character of the weighted averages. The more collinear variables included in the calculations, the more precise the latent structures will be [76]. Cluster Analysis (CA) - The main objective of cluster analysis is to find the natural grouping in the individual observations within a data set. Cluster analysis is a non-supervised technique and can be further subdivided depending on how the clustering algorithm
24
Chapter 2. A Review of Electronic Olfaction is performed. Both agglomerative and divisive cluster analysis has been used in [118] for the discrimination of foods and perfumes. Agglomeration means that each point is initially considered as a cluster and then joined with neighbouring clusters to form larger groupings. Divisive is the opposite of agglomeration where the entire data set is considered as one cluster than further divided to form smaller groupings of data.
Principal Component Analysis (PCA) - performs a principal component or eigenvector analysis of the data and projects the samples into a new co-ordinate system. The most useful feature of PCA is that it describes major trends in the data by reducing the dimensionality of the data. In other words, the systematic variations that were present in the original data set is described using fewer variables. Consequently, PCA can been used as a pre-processing technique. The principal components are often used as inputs or features into further processing algorithms that take into account non-linear type classification [40].
2.5.3
Artificial Neural Networks
Artificial neural networks (ANN) are the most commonly used pattern recognition techniques with electronic noses. ANN’s have been applied in a variety of applications ranging from the detection of coffees [87], wines [56], other food sources [114, 17], toxicity detectors and medical applications [148]. ANNs offer the benefit over the traditional statistical processes to be able to detect a greater number of substances than the number of unique sensor types. Also, the less selective sensors which are generally less expensive can be rendered much more selective when used in conjunction with an ANN. This is due to the fact that once a network is trained for a particular odour, the consecutive recognition process can occur rapidly and effectively. Supervised Learning Perhaps one of the most common types of networks used to detect odours is the multi-layer feedforward (MLFF) network [48]. The MLFF network consists of a multi-layer perception which is comprised of three layers, an input layer, an output layer and a hidden layer. Each layer consists of several units. In Figure 2.7 a simple network architecture is shown, each unit in every layer is connected to every unit in the next layer. In the feed forward network there are no feedback loops so the units are connected from the input layer to the output layer. The units in
2.5. Data Processing Techniques
25
the hidden and output layer are information-processing elements called neurons. The neurons collect signals by summing both the excitatory (+) and inhibitory (-) influences called weights. If the excitatory influences are dominant, then the neuron may fire and send information along its connections to other neurons. The neuron output is modelled as a threshold function and most common functions are step, ramp, or sigmoidal functions.
Figure 2.7: The figure is an example of a 3 layer feed forward network with two inputs, X1 and X2 and two outputs , Y1 and Y2 . In an MLFF network the hidden unit is obtained by first forming a weighted combination of the values of the input unit given by aj =
d
(1)
(1)
wji xi + wj0 .
(2.3)
i=1 (1)
Where wji denotes the weight in the first layer, going from input i to (1)
hidden unit j, and wjo is a bias value for hidden unit j. The activation of the hidden unit can be obtained by the following transformation using the activation function g(·). zj = g(aj ).
(2.4)
The values continue to feed into the next layer until the output of each unit, k, in the output layer is calculated by the following, ak =
M j=1
(2)
(2)
wkj zj + wk0 .
(2.5)
26
Chapter 2. A Review of Electronic Olfaction
where there are M neurons in the output layer. There are two phases in the neural information process; a training phase and a testing phase. At the beginning of the training phase, all the weights are arbitrarily chosen. Through each iteration of the network, the weights are refined until they generate the desired output. This process determines a neural model, which is often called the ‘learning phase’ of the network. The ‘retrieval phase’ of the network occurs when the network is tested with unseen samples to yield classification results by selecting the appropriate output unit (more on neural networks can be found in [14, 63, 73]. Using electronic nose data the inputs are typically a featured selected version of the sensor data and the output units are the number of different sampled odours. One of the earlier experiments that successfully used a MLFF network with electronic nose data was recorded in late 1980’s by [39]. Eleven different common liquor brands found on the market were recognised using a two-layer network. The results were met with relative success generating an average classification rate of 73%. Since the 80’s, genetic algorithms (GA) have been applied to neural networks in order to correctly classify different fruit solutions [16]. A Radial Basis Functions network (RBF) was used in [140] to discriminate different potato chip flavours. A vector memory array, a technique [87] using single-layer neural network architecture, has been used to classify varying brands of coffee. Reinforcement neural networks have also been applied in the detection volatile compounds [1]. Unsupervised Learning Unsupervised algorithms have also been applied to electronic nose data. The main disadvantage to the MLFF and other networks mentioned in the previous section is that they require a priori knowledge of the correct number of classes or outputs the network from which the network can select. In supervised learning, this number is the total number of different odours one wishes to discriminate. However, there are particular cases in sampling odours when the number of possible outputs is unknown. In these cases an unsupervised learning algorithm is favoured. Combining an unsupervised network with electronic nose data was studied in [32, 111]. These experiments used a self-organised feature map (SOM), first developed by Teuvo Kohonen in the early 1980’s [83]. The SOM is a single-layer network that creates a topological structure among the neurons and processing units. The results from the SOM showed that the success of classification depended largely on the size of the network i.e., the size of the topological grid. The grid size that yielded the best
2.5. Data Processing Techniques
27
classification results was found by trial and error. In many aspects a SOM can be a powerful visualisation tool, however, since the topological map is two-dimensional there may be a sub-optimal placement of the vectors, especially if the input data is not intrinsically two-dimensional [14]. Other unsupervised techniques are discussed in the next section.
2.5.4
Fuzzy-Based Techniques
Lotfi Zadeh first introduced fuzzy logic in the 1960’s in an attempt to find a more ‘natural’ way to handle imprecise information [155]. The theory of fuzzy logic attempts to enable machines to deal with imprecise language used by humans in order to describe data that may not be exact or crisp. A fuzzy system can be represented by the following components, Figure 2.8. Rule Base
x
Fuzzification
Fuzzy Inference
Defuzzification
y
Figure 2.8: Operation of a fuzzy if-then system.
Fuzzification Fuzzification is the process by which crisp data is converted to non-crisp linguistic terms. Fuzzification attempts to find the degree matching between the input, xi to a set of linguistic terms defined by A = {Ai,1 , · · · , Aik }. For example, the feature of age may be described as {very young, young, middle-age, old}. Each of these linguistic terms can be represented by a set of values of the input xi . Fuzzification replaces xi with a set of Ki numbers which range from 0 to 1 and reflect how well xi matches each linguistic term. The degree of matching is represented by a membership degree defined µAi ,1 (xi ), · · · , µAi ,Ki (xi ) for Ki numbers, see Figure 2.9. Inference An inference system uses a set of fuzzy if-then rules called a rule base. Each rule contains an antecedent part and a consequent part. The antecedent may consist of Boolean expression that contains simple clauses, for example, “if x is small”. The consequent may be an output represented by linguistic terms, such as “then
28
Chapter 2. A Review of Electronic Olfaction Very Young
Young
Middle-Age
Old
1.0
0.8
0.6
0.4
0.2
Age
Figure 2.9: Linguistic terms for Age. Ai,1 = V eryY oung, Ai,2 = Y oung, Ai,3 = M iddle − Age, Ai,4 = Old.
y is big” (Mamdani-Assilian model), or a function of the input x, “then y = fx (x)”, (Takagi-Sugeno Kang model) [85]. Defuzzification Defuzzification involves calculating a single value to represent the fuzzy set. This is done in order to find the system’s output, y. There are many methods in which defuzzification occur. One common technique is the centre of gravity method which evaluates y by calculating the centre of gravity of the area under the fuzzy set. Fuzzy-based computational algorithms are attractive to researchers in machine olfaction for the following reasons: Several sources of fuzziness can be identified in the recognition of olfactory signals such as noisy data, imprecise measurement and odorant sample sets which overlap in the feature space. Often odours descriptors are used to describe odour samples (e.g., floral, fruity) and intensity (e.g, low, very, strong). Some examples of fuzzy-based systems that treat olfactory signals from electronic noses are described in [139, 135, 97, 37, 88, 31]. In [139] a fuzzy inference system was designed to predict the concentrations of two odours, carbon monoxide and methane. A Takagi-Sugeno type model was applied by using a grid-type input to estimate the outputs. In this case, every sensor response is partitioned into number of fuzzy sets. For this experiment a total of 5 input responses were partitioned
2.5. Data Processing Techniques
29
into 4 fuzzy sets. A disadvantage is that the number of rules required for the inference system can grow explosively as the number of sensor inputs is increased. However, the use of a fuzzy inference system offers the possibility to explore the integration of a sensing system with a microcontroller for fuzzy inference engines, which eventually could develop into an “electronic nose” on an integrated chip [91]. In other examples, fuzzy-based algorithms have been used together with artificial neural networks. In [97] a fuzzy ARTMAP neural network was used to classify electronic nose data from three different datasets, alcohol, coffee and cow’s breath. Fuzzy ARTMAP is a supervised pattern recognition method based on fuzzy Adaptive Resonance Theory (ART). It is a promising method since fuzzy ARTMAP is able to carry out on-line learning without forgetting previously learnt patterns, it can recode previously learnt categories (adaptive to changes in the environment) and is self-organising. The results obtained from the classification outperformed the comparative results from the backpropagation (BP) trained multilayer perceptron (MLP). Furthermore, the time necessary to train fuzzy ARTMAP was typically one order of magnitude faster than backpropagation. The result shows that this technique is promising in terms of its possibility for on-line learning, and ability to deal with uncertainty (in terms of measurement accuracy, noise rejection, etc.). In [37] a fuzzy hierarchical classification was explored. The odours samples were obtained using an electronic nose comprising of an array of conducting polymer sensors. The sensor responses were represented by the coefficients of their Fast Fourier Transform (FFT). Before the classification was performed, a feature reduction method was applied to reduce the feature space dimension. Then, an Unsupervised Fuzzy Divisive Hierarchical Clustering (UFDHC) method was used to establish the optimal number of clusters in the data set, as well as the optimal cluster structure. The output of UFDHC is a binary hierarchy of fuzzy classes where the branches in the tree may represent overlapping clusters. The binary hierarchy is then used to build a supervised fuzzy hierarchical classifier. At each level of the hierarchy a crisp separation of the two corresponding fuzzy classes was determined. Each region was then related to a specific expected output of the system. The system was validated using a set of preliminary experimental results which demonstrated good recognition performance, although no comparison to other non-fuzzy based methods was made. In [88] and [31] a fuzzy linguistic classifier was used to classify different types of olfactory signals. In the case of [88] an entire odour recognition architecture called a “Fuzzy logic-based Recogniser of Ol-
30
Chapter 2. A Review of Electronic Olfaction
factory Signals” (FROS) was presented. FROS consists of two fuzzy logic-based components. The first is a shape-based recogniser that uses linguistic fuzzy classification. The purpose of the shape-based recogniser is to examine the shape of the signal and describe it in linguistic terms. A linguistic model repository is used which contains all the odorant linguistic models that represent the odorant to be recognised. These models are generated during a training phase. The linguistic recogniser is designed to produce a normalisation of the signals so that they are analysed independently of their amplitudes. The consequence is that valuable information regarding the signal properties is lost. Therefore a second fuzzy based classifier is used. The second classifier builds a model to represent the dynamic range of the sensor responses. The results from the two classification systems are integrated using a ”combiner” that makes a final classification decision.
2.5.5
Techniques for Drift Correction
A reoccurring problem with gas sensors is a so-called drift effect. This means that the sensor response to an odour may vary over a period of time. There are two particular drift effects that have been studied using the gas sensors: long term drift and short term drift. The short term drift typically occurs over a period of weeks or days. To diminishes its effect, a reference gas may be used and the drift is compensated by subtracting the reference gas values from the other sensor responses. This assumes that the effect from the drift additive (independent of signal level). Other models are based on multiplicative techniques which take the signal level into consideration. Typically, the ratio of signal from a reference gas at two occasions (initial time and time of measurement) is used. Mathematical models are also able to handle the short term drift. These models may include updated training of an artificial neural networks may be trained to compensate for the shift in results [111] [66]. The effects of the long-term drift, also known as contamination drift, can been studied over a period of months or years [68]. It is difficult to obtain a general model of the drift phenomenon. Most likely the drifting occurs after a poisoning or weakening of the sensing material resulting from long term use. Using a neural network in order to compensate for the shift in values may not always work since each sensor may exhibit its own unique behaviour depending on the sensing history of the array. Another problem with the gas sensors occurs when a large number of samples are taken within a short period of time. Essentially, a memory effect is created where the samples are influenced by the previous sample
2.6. Applications
31
taken. An immediate step to solving this problem is to improve measurement procedures. To do this, enough time between samples should be allowed to allow the sensors to recover to steady state value (approx. 3 minutes). Also flushing the tubing and valves with a reference gas may prevent residual odours from affecting further tests. Recently some studies have also explored the possibility to shorten the restoration period between odour sampling by using temperature curing techniques to directly clean the sensor’s surface.
2.6
Applications
As mentioned at the beginning of this thesis, research in electronic olfaction has progressed on two fronts. The beginning of this chapter has focussed its attention on the first front dealing with the development of gas sensing technology1. The second research area discussed here are the number of applications which have emerged as a result of electronic nose technology. The application areas are broad and continuously expanding as commercial e-noses are becoming affordable and the reliability of the gas sensors are improving. In Table 2.3 we summarize a number of different application areas along with some practical implementations. From Table 2.3 it can be seen that e-noses for quality evaluation and control can be subdivided in two main areas, evaluation of food products and the evaluation of environmental conditions with a broad range odours being detected and discriminated. Environmental evaluation have considered air, water, and even soil quality. As e-nose technology improves and commercial devices are available, these ranges will only broaden to include a wider spectrum of odours. A new and emerging field of using e-noses is to aid in medical diagnosis. The e-nose provides a non-intrusive measurement of odours which have shown to be useful in the categorization of different human odour and bacteria. Although much can be said about quality evaluation and medical applications, these works have been mostly carried out on e-noses that have been considered as singular platforms. An exception however, has been a new application of mobile robot navigation. These works have considered the integration of gas sensors onto more complex systems namely indoor mobile robots 2 . The integration of machine olfaction onto mobile 1 It is debatable whether pattern recognition techniques can be considered as a development inherent to e-nose technologies since seldom are pattern recognition algorithms developed for the treatment of e-nose data per se. Rather, pattern recognition techniques are a research area on their own and e-nose data is often used as novel testbed for new algorithms. 2 Other platforms may include outdoor robots, flying robots, and even suggestions
32
Chapter 2. A Review of Electronic Olfaction
Application Area Food quality evaluation
Environmental monitoring Perfume and fragrance industry Automobile and space industry Detection of Explosives Medical Diagnosis
Mobile Robot Olfaction
Examples Discrimination of wines, fish freshness and potatoe chip flavours, etc.. [100, 118, 56, 114, 17, 16, 140, 112] Water quality, air quality and soil quality determination [30, 59, 148, 42] Identification of perfumes [41, 18] Monitoring of air quality[76, 154] Detection of landmines [153, 106] Bacteria identification and health quality assessment and quality control of pharmaceuticals [38, 11, 50, 4] Plume Tracking, Odour Source Localization, Trail Following [55, 70, 79, 86, 93]
Table 2.3: Application areas for e-noses.
robotic platforms has provided a number of applications for odour-based navigation. These include trail following, plume tracking, odour-source localization, creating concentration grid maps, modelling of gas distribution. The majority of mobile olfaction experiments do not include odour discrimination as part of the olfactory component. Rather, simple type gas sensing systems are used to detect a target gas known a priori. Since odour based navigation using mobile robots is very much a future application of the work presented here in this thesis, we will now say a few words about its current state of research.
2.6.1
Navigation By Smell for Mobile Robots
In larger scale environments which is often the case on robotic platforms, the patterns of airflow in a given environment and around a robotic platform influence greatly the ability to navigate using an odour stimulus. For odours which only exhibit molecular diffusion, chemotaxis can be taken (directed movement towards a source). However, transportation of molecules is often carried by laminar and turbulent flow and consequently cause sensor data readings to display chaotic fluctuations. For these reasons, different odour-based navigational behaviours are applied. of underground or tunneling robots [127].
2.6. Applications
33
For many plume tracking and trail following applications, many navigational strategies are based on biological models. Such models may use: Clinotaxis - Determination of a concentration gradient by comparison of successive stimuli. Anemotaxis - Using sensing devices such as anemometers to measure air speed and direction, and consequently approach an odour source in an upwind direction. Tropotaxis - Determination of concentration gradient by comparison of simultaneous stimuli from two or more receptors, such as two antennae. The validity of these techniques require that the odour plume has been established. However, in large search spaces often the chemical plume needs to first be found, in which case different techniques may be used. One such example is a technique called “casting” which consists of either a zigzag or spiralling motion each time increasing the explored area. Other techniques may use several robots cooperating in order to decrease search time [62] or random searching to locate an odour plume. In Figure 2.10 we show some examples of the different strategies to navigate to an odour’s source using combinations of the techniques mentioned. For the actual identification of the gas source, the concentration drop on the upwind side of the source can be used or the characteristics of the emitted plume such as the diminishment of plume width can indicate odour source location. However, to measure these properties, it is assumed that a strong wind source is present. In some experimental platforms [70, 126], a predominant airflow may be artificially introduced, however, in a realistic scenario this is not always the case. Therefore, a reliable means to declare an odour source is also required. Some suggestions have been made to use other sensing modalities such as tactile sensors indicating when the robot has hit the target whereas other techniques use machine learning to classify a pattern represented by the gas sensors temporal and spatial information as a gas source is approached [96]. Finally, other suggestions involve providing a concentration map or grid after thorough inspection of an area and then determining the source location [95]. The latter case in particular may be too time consuming to be considered in a realistic context. Mobile olfaction currently faces a number of open and difficult problems. For odour-source localization in uncontrolled environments airflow turbulences manifest in sporadic sensor readings combined with the
34
Chapter 2. A Review of Electronic Olfaction
Source
Source
Wind flow
Plume
Casting
Plume A
B
C
Figure 2.10: Different odour based navigation strategies. The robot is indicated as having two sensors where the larger sensor indicates that this sensor detects a target gas. (A) shows a case of tropotaxis combined with a zigzag technique [70] for gas source tracing in the presence of a strong unidirectional airflow; (B) demonstrates another reactive localization technique [94] using chemo-tropotaxis based on Braitenberg vehicle when no dominant air flow is present and (C) illustrates the technique of casting in this case used to acquire an odour plume.
2.7. Discussion
35
point measurement nature of the gas sensor make the identification of the source difficult and time consuming. Since we cannot rely that maximum concentration readings will correspond to the odour source location, appropriate navigational strategies need to be in place. In particular when odour sources correspond to physical objects, conceptually the location of the object is difficult to confirm using olfaction alone. A possibility is to integrate other modalities that can provide the spatial or structural properties of that object. It is also questionable whether the current gas sensing technologies are suited for odour-based navigation techniques. One limitation is the high concentrations ranges required for detection, many of the TGS sensors (often used for mobile olfaction) detect within a range from several ppm to several thousand ppm which is adequate for static platforms, however, in the context of a mobile robot searching in large open spaces for gases that use a surface sensing area of approx. 1 cm2 , detecting a gas using these sensors will typically occur only a few hundred centimetres from the source3 . This may be why many of the experiments done in odour-based navigation use special odours such as ethanol that have high volatility and low molecular weight. Also, mobile robots that use olfaction to navigate require fast reaction time of the sensors. Again the TGS sensor has a rise time in the order of a few seconds. This often limits the speed of movement of the robot to only a few meters per minute. Furthermore, using only one type of gas sensor does not exclude the possibility that other odours may be detected. Despite these challenges, the idea of tracking odours is inspiring and an important avenue to explore. Furthermore, the consideration of odour discrimination on a mobile platform raises new issues of interest to electronic nose community as well as those working on odour-based navigation techniques. These issues and how they apply to the integration of e-noses in intelligent systems are discussed in Chapter 6. For now, we have only presented some of the current difficulties in mobile olfaction platforms that do not consider actual odour identification, although some of these issues will reoccur in consideration of merging electronic noses onto mobile platforms.
2.7
Discussion
The possibility to measure and discriminate odours is particularly attractive to a number of existing application domains. Electronic noses offer this possibility by using an array of gas sensors with pattern recog3 This is of course dependent on the molecular weight of the target gas and the temporal factors.
36
Chapter 2. A Review of Electronic Olfaction
nition techniques. The major setbacks that we face today in electronic olfaction is a limitation in current sensing technologies for electronic nose devices. Gas sensors are subject to drift, noise, temperature and humidity effects. Each gas sensor is a specifically tailored device for certain target gases and the range of these gases are currently limited. Still, however, electronic noses have been tested for a large number of different odours and throughout this chapter we have highlighted the range of different odours used in actual experimental platforms. The key to discriminating among these odours is the pattern recognition technologies. As was shown, the number of different techniques applied to e-nose data are diverse. Moreover, no single technique has proven to be consistently more successful than another. Rather, the success of a technique depends largely on the type of odours being discriminated, the properties of the sensor data and the application. It is important to note that in many experiments seldom are more than 5 different odours discriminated. So even though electronic noses have been discriminating a wide range of different substances, it seems that the pattern recognition algorithms together with the sensor array can only cope with a small diversity in the training set. Electronic noses still require much development before they become generally accepted sensor devices and are integrated into industrial applications. For this integration to take place the sensors need to develop according to application needs. What is needed is the cooperation between the different application areas such as robotics, together with electronic nose researchers to promote the sense of smell into artificial systems for real applications.
Chapter 3
Abstraction of Olfactory Information What’s in a name? That which we call a rose By any other name would smell as sweet. William Shakespeare, Romeo and Juliet (II, ii, 1-2) When Shakespeare put pen to paper and wrote Romeo and Juliet he was probably unaware that the scent of a rose actually comprises of more than 400 different compounds [57]. Oddly enough, he was able to describe its odour in a single word: sweet. As discussed in Chapter 2, such descriptions for our perception of smell are not unusual. By using hedonic tone, emotional experience or verbal analogy humans successfully convey their impressions of odours to each other. The question then is how an electronic nose should express its perception of odour to a human? In this chapter, we address this question by investigating the specific challenges of communicating artificial olfactory data. We then present an odour classification algorithm which is able to capture on a symbolic level the response of the odour signal from an electronic nose. The approach is to start at the low-level sensor data and extend the recognition process of odours to a high level symbolic representation. Section 3.2.5 deals with the problem of developing good performance evaluation measures for this type of odour classification. In Section 3.4 we also consider an extension to a social agent, which takes as input the symbolic descriptions and generates as output a facial expression. The social agent constitutes an addition that facilitates communication between human and machine. 37
38
Chapter 3. Abstraction of Olfactory Information
To date many of the works that have considered the integration of symbolic representation with artificial sensing systems have mainly focussed on vision based sensing systems. However, unlike images where properties are detected such as shape, size and colour, the data from the gas sensors cannot be broken down into recognizable odour features. The challenge explored in this chapter is to therefore determine how the symbols representing odour descriptions can be mapped to the odour categories inherent in the sensor data. It is in this way we create an odour concept which can then be communicated to a human user. The relevant concept acquisition techniques currently available in the AI community can be divided in two directions: concept acquisition for autonomous agents and concepts acquisition for social agents. Concept acquisition for autonomous agents focuses mainly on how autonomous agents can learn to categorise and form internal representations using the observations from perceptual data. The theoretical background is heavily focussed on different machine learning techniques, pattern recognition and conceptual clustering systems. Discussions tend to revolve around the following four fundamental questions (1) how to determine when an agent should create a new concept (2) how prototype and probabilities theories can be used to better define category boundaries (3) how to develop an incremental systems which adjusts categories over time (4) how to develop a parallel system which creates concepts simultaneously [28, 107, 89, 46, 29]. Unfortunately, the significance of the symbolic description per se, as it relates to the concepts has not been explicitly emphasized. This is most likely due to the fact that the main goal of physically embedded autonomous agents is to interact effectively and independently with its environment via its own perceptors and therefore, concepts are often learned by observation [142]. Finding intelligent methods to accomplish given or self-generated tasks takes priority over human-robot communication. On the other hand, concept acquisition for social agents places more focus on the resulting symbolic output. There have been some notable examples of social agents interacting with humans, with other agents, and on mobile platforms [145, 136, 138, 137]. In these examples, language plays a causal role on concept formation. This means that categories and concepts are learned by example using a form of supervised learning. The supervision of concept acquisition occurs with a mediator who provides the agent with the appropriate symbols during training. Performance evaluation is then judged based on how much the output of the system matches those responses given by the mediator. In cases of mismatch, it is assumed that the agent should readjust its internal representations
3.1. Challenges of Communicating Olfactory Information
39
by forming new associations using the supervisory feedback from the mediator. In this chapter, we will investigate a system that attempts to create odour categories in accordance with its sensory information while using a lexicon that has been synchronized with a human user. Given the current technological state of the gas sensor and its operation properties, it is likely that not all odours sampled by the human nose can be equally discriminated by the electronic counterpart. Within concept acquisition for social agents, the problem of perceptual conflicts is often viewed as negligible provided that communication is successful (e.g., two agents can effectively point to the same object and say the same word). The following quote attests to this view: “...we cannot assume that different agents have exactly the same conceptualization and that they mean the same thing by the same words...There is no guarantee that a particular form maps onto the same meaning, even in the same language community. Despite quicksand foundations, communication is generally successful...”. (Steels, [136]) Our standpoint does not contradict the fact that different conceptualization can lead to the successful communication, however, we do add that by making the relations between concepts explicit, a deeper understanding of the internal representations of odours in the electronic domain can be gained. Our approach to achieve this goal is to map human odour names to the sensor data, and through manipulation of the symbols create a user-interpretable output.
3.1
Challenges of Communicating Olfactory Information
There are a number of challenges that need to be addressed when considering the representation of the perception of an electronic nose. First there is the problem of representing categories of odours in the absence of a clear and comprehensive structure odour relationship in human olfaction, i.e., the relationship between the molecular construction of an odour and its character [115]. Although some attempts to map simple chemical structures to odour character have been made, the majority of odours have a too complex structure to find any universal representation. Therefore few restrictions exist to describe an odour character. Odours can be described by either the emotional experience which they
40
Chapter 3. Abstraction of Olfactory Information
evoke (hedonic tone), the name of the object which emits the smell (e.g., lemon, chocolate, etc..), or through an associated memory [36]. Studies have also shown that the vagueness of the description of odours reflects a difficulty to express their recognition (e.g. ”smells sweet and a little fragrant”). Industrial applications which require a fixed standard for odour quality evaluation decide upon an appropriate vocabulary and experts within the field, chosen upon for their keen sensitivity, are trained to use this vocabulary to describe their perceptions. This absence of classification standards in human olfaction was perhaps one of the initial motivations to the notion of electronic noses. The ability to quickly classify odours in a quantitative manner with high reproducibility was and still is the driving force behind development of the compact gas sensor. However, the nature of the sensing mechanisms has revealed a complex data analysis component and a sensing platform subject to drift, temperature and humidity changes [48]. These factors combined with the inherent uncertainty in the human model of odour classification, have challenged the interpretation of electronic nose data. Often experts who are more familiar with the data processing than the odour characteristics are required to analyse and interpret the results. The consequence is that electronic olfaction has remained a research area and its presence in the industry is reluctant, despite commercial attempts to build compact and portable devices. However, the limitations of the current sensing technology aside, electronic noses have a unique potential to be “super-sensors”, or in other words, sensors whose technology empower us with an “extended reality” [133]. Existing examples include auditory sensors which can perceive frequencies outside of the human audible range, or thermal cameras which can detect infrared radiation which is invisible and intangible to the human eye. However, in order for humans to benefit from these sensing abilities some means to translate and communicate this new information is needed. In the example of auditory sensors, translations from the high frequencies are scaled down into the human audible range. In the case of thermal imaging, each pixel in an image contains a temperature value which is then mapped onto a value on a colour scale. Even though currently, the general performance of electronic noses are yet comparable to that of a human nose, already they can detect substances that are “odourless” to a human such as carbon monoxide. E-noses are also capable of better detecting varying levels of concentrations of certain odours. The problem is that finding the scales or the transformations to represent sensor data is not intuitive in olfaction. In addition to the challenges caused by the lack of a structure-odour relationship and the
3.2. A Symbolic Approach to Odour Profiling
41
complex analysis of the sensor data, a third challenge emerges. This challenge deals with the ability for artificial sensors to maintain perceptual integrity yet still communicate results that are coherent with the perceptions of its users.
3.2
A Symbolic Approach to Odour Profiling
We consider how to automatically create a correspondence between symbols, in this case linguistic names, and categories of e-nose data. To accomplish this task, we create a system that first creates categories using unsupervised clustering and then generalizes from these categories by evaluating how well an odour name applies to its own sensor representations. Since part of the objective is to facilitate human and machine interaction, the names outputted from the system are correlated and with those of a human user. To cope with the perceptual differences, the system is tuned to provide as explicit information about the categorization of odours as possible. We accomplish this task by adapting existing fuzzy-based algorithms to generate informative odour descriptions. Ultimately, the understanding of where and how perceptual differences may occur, allow us to better handle the sensing systems. In light of the challenges in both biological olfaction and electronic olfaction, linguistic concepts are used as a means to better promote this understanding, provided that the concepts adequately represent the sensor data to which they refer. In sensing systems, such as olfaction which currently, is absent of any quantifiable metric to classify odours, it is believed that this kind of representation can be equally appreciated from a human user interacting with the system. The system comprises of several modules, namely a pre-processor, cluster analyser, category generator and symbolic interpreter, shown in Figure 3.1. Each odour, o that is sampled by the electronic nose is represented by a response vector Rk for 1 < k < o. Each odour sample k is represented by a linguistic label lk , where lk ∈ D, a user specific discrete set. For example D may contain the collection of odour names (e.g. ethanol, vinegar) or odour qualifiers used for specific applications (e.g. resinous, nutty, phenolic, yeasty).
3.2.1
Pre-Processing
Pre-processing is performed in two steps: a baseline manipulation and a normalisation phase. Signal responses are manipulated using a fractional
42
Chapter 3. Abstraction of Olfactory Information Se nso r Da ta
Pre-Processor
Cluster analysis
Ca te g o ry Ge ne ra to r
Validation
Le ve l-1 Rule s
Symbolic Interpreter Symbolic Interpretter Le ve l-2 Rule s
Profile Pro file
Classifier Cla ssifie r
Figure 3.1: Overview of the odour profile generator. Input of the system is both labelled and unlabelled sensor data, the output is a symbolic representation of the artificial olfactory signals. manipulation technique where we subtract and divide the baseline thus generating both dimensionless and normalised responses. The signals are gathered in a response vector Rk where each sensor’s reaction is represented by: Rk,s =
Rmax,s − Ro,s Ro,s
(3.1)
for s=1 to 32. Rmax,s is the maximum possible signal strength and Ro,s is the calibrated baseline response. The heuristic between the maximum possible signal strength and the calibrated baseline response is used to represent a sensor’s reaction to an odourant.
3.2.2
Cluster Analysis
The sensor data stored in the repository is organised into clusters by implementing a Gath-Geva algorithm iteratively, each time with a varied number of clusters c until optimal partitioning is found [51]. The Gath-Geva algorithm is used over the more common fuzzy c-means algorithm [13] because of its ability to compensate for clusters of different shape and density. However, a potential problem is that the algorithm may converge in local minima. Therefore, to reduce this risk, the fuzzy c-means is used to initialise the membership matrix. Using odour signatures as input, the objective function for the c-means algorithm is computed as follows:
3.2. A Symbolic Approach to Odour Profiling
J(uik , vi ) =
c n
43
2 um ik dik
(3.2)
k=1 i=1
The classification vector defined by uik of a pattern Rk is the k th column vector of the partition matrix uik . And dik the Euclidean distance between each Rk and each cluster center, denoted by vi for 1 < i < c. To apply the Gath-Geva a fuzzification of statistical estimators is used which combines the fuzzy c-means with the adaptive Fuzzy Maximum Likelihood Estimation (FMLE) measure. The algorithm with the added c-means initialisation computes the FMLE distance between each olfactory feature vector and odour prototype using the fuzzy covariance matrix Fi and Pi , an estimate of the probability the normal distribution estimator of probability for the membership of Pi for each cluster.
d2Fi (xk , vi ) =
det(Fi ) 1 exp( (xi − vi )T Fi−1 (xi − vi )) Pi 2
(3.3)
Using the FMLE distance, the new partition matrix is computed as: uik =
1 c
d2F (xk ,vi )
2
(3.4)
j m−1 j=1 ( d2 (xk ,vj ) ) Fj
To determine the optimum number of clusters, a set of global validity measures are calculated after each evaluation. In this work, optimum criterion for cluster partitioning is defined as clusters with good separation, minimal volume and maximal number of points located in the vicinity of the cluster center. Three different global validity measures are applied to satisfy this criteria, namely: Compactness and Separation [152]; Partition Coefficient [13]; and Partition Entropy [67].
3.2.3
Category Generation
The category generator generalises from the organised data to form odour concepts in the electronic domain. A contingency analysis is performed which examines the relationship between the subjects’ scores on two categorical variables [121]. In the contingency table, the rows are the different odour names assigned by the human, and the columns are the unlabeled clusters found by the clustering algorithm. Since fuzzy cluster analysis was used, each point on our data space belongs to every cluster with a degree membership between 0 and 1 (although it should
44
Chapter 3. Abstraction of Olfactory Information
be noted that the Gath-Geva tends to provide slightly crisper clusters). Therefore, each entry of the table is represented by the number of prototypical points having label lk in cluster Cc . Prototypical points are points whose membership degree to a cluster is equal to 1. A cluster “wins” a specific label when prototypical members of a given name have the highest occurrence of an odour name within the prototypical region. Figures 3.2 illustrates an example of the clustering algorithm on the training data and the combined results from the category generator. For the sake of visualisation, a principal component analysis was done in order to represent the data on a 2-dimensional space (the first component vs. the second component). It should be noted that in the execution of the actual algorithm no principal component analysis was performed at the pre-processing stages. Figure 3.2 (Upper Left) shows the data prior to the fuzzy clustering, the assigned labels Lo are also indicated on the graph. Figure 3.2 (Upper Right) shows the results of the fuzzy clustering imposed on the previous graph. Shading in the graph represents the membership values to prototypes, membership values under 0.5 are not shaded in these graphs. The number of clusters have been determined by the validity measure evaluation shown in Figure 3.2 (Lower Left) where 3 clusters yield optimum partitioning. The contingency table is presented where the rows with the maximum value are the winning labels for the corresponding column, where the columns represent the unlabeled clusters.
Level-1 rules Level-1 rules are used to manage the results from the contingency analysis. They are fuzzy-based rules which examine every column of the contingency table and map the appropriate odour name to each of the clusters. Since there is no guarantee that clusters will not overlap, or that the e-nose can differentiate between two different odours, we create rules that allow for the possibility of using compound category names (e.g., sweet and fragrant). The rules can also be used to differentiate between two clusters that may have the same name. This is used in cases where the e-nose may be more sensitive and splits the same odour into two or more clusters. The fuzzy rules therefore allow the user to either better co-ordinate the names between human and electronic perception or to emphasise cases where the number of clusters exceeds the number of available odours.
3.2. A Symbolic Approach to Odour Profiling
45
Fuzzy clustering with 3 Clusters
PCA of 3 different Odours 1.5
1.2
1 1
0.8
0.6
yogurt vanilla lavender
0.5 PCA2
0.4
0.2 0
0
−0.2 −0.5
−0.4
−0.6 −1 −10
−5
0
5
PCA1
−10
−5
0
5
Validity Analysis 0.6
0.4
C1 Lavender 0.273 Y oghurt 0.0083 V anilla 0.0248
Compactness and Separation 0.2
0 Partition Entropy −0.2
−0.4
C2 0.00 0.463 0.0083 U (O, C)
C3 0.00 0.00 0.223 0.849
Optimum Partitioning −0.6
−0.8 Partition Coefficient −1
2
2.5
3
3.5
4
4.5
5
Figure 3.2: (Upper Right) Feature analysis of three different odours Yoghurt, Lavender and Vanilla. (Upper Left) Fuzzy clustering algorithm applied to the data where the darker shaded areas represent regions of higher membership. Three clusters are visible in the graph. (Lower Left) Validity measure analysis where 3 clusters yield optimum clustering for combination of the global measures. (Lower Right) Contingency analysis used to co-ordinate the clusters with the human labels.
46
3.2.4
Chapter 3. Abstraction of Olfactory Information
Symbolic Interpreter
Unknown points or new odours are classified by evaluating the degree of fulfilment β(x) where x is the unknown odour. The expression for the degree of fulfilment is given by, β(x) = µx,C1 (C1 ) ∧ µx,C2 (C2 ) ∧ . . . ∧ µx,C3 (Cc ).
(3.5)
where the membership degree of x is computed for each cluster Cc for c clusters found in the unsupervised algorithm and ∧ is the conjunction. If the categorical name for each cluster Lc replaces Cc , Equation 3.5 can be rewritten as, β(x) = µx,C1 (L1 ) ∧ µx,C2 (L2 ) ∧ . . . ∧ µx,C3 (Lc ).
(3.6)
A fully symbolic description is formed by applying a secondary set of rules called Level-2 rules. The level-2 rules better describe unknown odours by mapping an odour’s relation to the different clusters into symbolic adjectives. At this stage of the work, the rules are used to interpret the membership relations µx,Cc , into meaningful symbols. In the experiments described below, this is achieved using the following: ⎧ ⎨
M is now the set of linguistic descriptions that corresponds mathematically to a degree of similarity, and in the context of an odour profile, it corresponds to an odour quality scale [34]. Equation 3.6 can be rewritten as: β(x) = Mk,1 (L1 ) ∧ Mk,2 (L2 ) ∧ . . . ∧ Mk,c (Lc ).
(3.8)
The result from the classification is a odour description that contains explicit information on how an unknown odour relates to the known odours. A potential problem, however, is that the larger the repository of odours sampled by the electronic nose, the larger the descriptions may be. In this case, some form of trimming of the description is implemented, where the terms with the highest degree of similarity are preserved. For visualisation purposes, as will be shown in the experiments, µx,Cc is used directly to represent odours using a technique similar to that of the flavour wheels.
3.3. Experiments
3.2.5
47
Performance Evaluation
Evaluating the performance of a system that preserves its own representation of odours is non-trivial since misclassified points cannot necessarily be interpreted as erroneous. In supervised algorithms, the category boundaries are determined by linguistic concepts, however, in nonsupervised algorithms, the sensor data dictates how the categories are represented and thus a more individualistic learning approach is used. Therefore, in this work our performance measure is not only based on the number of correctly classified odours. Instead from the contingency table, a measure of dependency of the clusters to the human labels is calculated. This is obtained by calculating the entropy of the table also known as the coefficient of uncertainty U . H(C) − H(C|O) , H(C) poc H(C|0) = − poc ln , po• o,c
U (C|O) =
H(C) = −
po• ln po• ,
(3.9) (3.10) (3.11)
o
where p•o = c pco , pc• = o pco . The coefficient of uncertainty can range between 0 and 1. A value of U = 1 implies that there is a strong association between the human odour names and the clusters of sensor data. A value of U = 0 implies that there is no association between these two variables. This measure also indicates the coherence of cluster members to their prototypes.
3.3
Experiments
In this work, the odours were sampled using a commercially available electronic nose, Cyranose 320 [27]. This e-nose consisted of 32 thin-film carbon-black conducting polymer sensors of variable selectivity. The array of sensors are contained in a portable unit also consisting of pumps, valves and filters that are required to expose the sensor array to a vapour gas. The sampling of an odour occurs in 3 different phases. The first phase is a baseline purge, where the sensors are exposed to a steady state condition, for example the air in the room. The duration of the purge is 30 seconds. The second phase is a sampling cycle where a valve is switched to allow vapours from a sampling inlet to come into contact with the sensing array. The duration of this cycle is 20 seconds. Odours
48
Chapter 3. Abstraction of Olfactory Information Response Vector or Smell Print for a Sample
Sensor Response from one Sensor 5.05
0.3 Rmax=reaction 5
0.25
0.2
∆R/Ro
Sensor Response
4.95
4.9
0.15
4.85
0.1
Ro=baseline
Recovery
4.8
4.75
0.05
0
20
40
60 Time(s)
80
100
120
0
0
5
10
15 Sensor Number
20
25
30
32
Figure 3.3: (Left) One sensor’s reaction to an odour throughout the three phases of sampling. (Right) The smellprint for an odour showing the response vector over the sensing array. No. Clusters
Partition Coefficient
Partition Entropy
2 3 4 5 6 (best) 7
-0.998 -0.998 -1.00 -1.00 -1.00 -0.998
0.004 0.016 0.998 1.350 3.55 0.055
Compact. and Separation 0.118 0.150 0.230 0.231 0.650 0.652
Table 3.1: Validity Measures for different cluster patitions, where 6 clusters yield optimum partitioning.
samples are taken from a 50 ml vile. Finally, a sequence of purging cycles is used to remove the sampled odour from the unit and restore the sensor response to the baseline values. Figure 3.3, illustrates a typical response from one sensor and the response vectors (or smellprint) over the sensing array. Experiments are performed on a number of different odours extracted from an ASTM atlas of odour character profiles by Dravnieks[34]. The odour profiles were designed to develop odour character information of 168 chemical compounds. Dravnieks’ tables are useful for the purpose of our experiments since they contain a statistical analysis using a human panel. Of the total profiles listed in Dravnieks’ tables, 6 profiles were chosen to be used as subjects, shown in Table 3.2. These particular profiles were chosen in an attempt to represent as large as a spectrum of odour descriptions as possible. In the table only the principle descriptor
Table 3.2: The substances and their primary descriptions taken from Dravnieks profiles [34].
is shown (a profile relates a unknown odour to a table of known odours). A total of 300 odour samples were collected over a period of several weeks. The experiment consisted of two phases. The first phase examined how well the electronic nose can differentiate between the six profiles and recognise unseen samples of the same odours. The second phase explored how classifier manipulated the six profiles to generate symbolic description for unseen samples of five additional complex odours that are not in the profile list. The clustering performance from the first phase resulted in 6 optimum clusters in the validity analysis, shown in Table 3.1 and a corresponding uncertainty coefficient of 0.98. Classification of unseen samples of the data using “leave one” out validation generated a classification accuracy of 93 %. In the second phase, odours with some relation to existing descriptions contained in the profile list were sampled. For each odour, 10 samples were taken. Figure 3.4 summarises which substances were used for testing, their perceptions from a human user, and descriptions given by the e-nose. From these results, it can be seen that the electronic nose and the human panel agree on some of the substances (e.g. both describe chocolate as sweet substance). The level-2 rules shown in Table 3.3 have been used to better illustrate the relation between these points and the known categories, through the use of adjectives such as, very or moderate1 . For example, the Corn oil was considered moderately herbal and moderately oily, which may indicate that the point resides between these two categories. Therefore, from this linguistic information, it can also be deduced how the known concepts relate to each other, which is 1 Dravnieks profiles used an odour quality evaluation scale with increasing integer increments from 0 to 5, 0 corresponds to absent, 3 moderately, 5 extremely. In our experiments, odour quality scale ranges from 0 to 1 i.e. values of µ with three descriptors.
Chapter 3. Abstraction of Olfactory Information
lic
Alc
So
eet
Oi
Corn Oil
ly
So
lic
ly
So
oho
ur
Sw
Oi
Alc
ur
Herbal
Chocolate
ly
lic
oho
eet
ly
Herbal
Oi
Oi
Alc Herbal
Herbal
eet
Sw
eet
ur
Sw
Sw
Sw
ly
ur
Fragrant
Fragrant
Oi
oho
ur
Fragrant
So
eet
So
Alc
Fragrant
lic
oho
Alc
Sw
ly
lic
oho
ur
Herbal
Oi
So
Alc
Herbal
eet
lic
oho
ur
Herbal
Herbal
Sw
So
Alc
Fragrant
lic
oho
ur
Fragrant
So
Fragrant
lic
oho
Alc
Fragrant
50
Sw
eet
Oi
Vinegar
ly
eet
Oi
ly
Ethanol
Figure 3.4: Top Row represents the odour profiles generated from the electronic nose. Bottom Row represents the odour profiles generated from a human panel, to the same substances. Each of the rings represent a degree of certainty, lower degrees are closer to the center of the wheel. not also evident from the figures. In this experiment we were able to illustrate how a set of primary odours with associated character descriptions could be used to train an odour recognition system. We also showed how the character descriptions could then be manipulated in order to test unseen samples of new and more complex odours. Even in cases where the electronic nose does not cohere with the human perception on the test samples, an explicit description is generated which informs the user on the discrepancies. This description is most useful if the trained concepts are correlated with the human user. However, considering the sensing properties of electronic olfaction and the state of the technological development of the gas sensor, cases will emerge when even the trained concepts do not match. Cases where clusters overlap can be treated as lower sensitivity, and thus are compensated for in the contingency analysis. As previously stated, the Level-1 rules can be adapted to account for compound naming and thus inform the user of a sensitivity failure. Such an example was shown in the author’s previous work [99] where the same set of odours were tested but with a different variation of the algorithm causing an overlap between the 3-Hexanal and the Hexanoic Acid2 . The result was a compound category with Lc equal to alcoholic ∧ sour. This infor2 The primary difference in the algorithm was a variation in the validity measure evaluation
Table 3.3: The identification of new odours with respect to the trained concepts.
mation was useful since actual computational compensations could be made thus improving the results, but also hardware compensation could have been made by redesigning the sensing array to account for certain selectivity. Cases where number of clusters exceeds the trained concepts consider the electronic perception to be hypersensitive. The following experiment is specifically designed to evaluate this type scenario. Hypersensitive Electronic Perception In this experiment, we consider a situation where 12 containers contain 4 different substances. Each of these containers is given a number and the actual chemical contents of the container is shown in Table 3.3. A blindfolded human is allowed to sample (by smelling) the contents of each container only once. The task of the human is twofold. The first task is to discriminate among the odours, that is to say, determine which among the 12 samples are similar to each other. This is a process of categorisation. Once the odour categories are formed, the second task is to give each of the categories a name, Table 3.3. The experiment is then repeated but using an electronic nose to sample the odours. Unlike the human user, the electronic nose is capable of making a distinction between the 20% ethanol and the 30% ethanol. Four cluster prototypes are found using the unsupervised algorithm each corresponding to the different concentrations of ethanol. This is an obvious advantage to using an electronic nose since it is particularly sensitive to concentrations of certain gases such as ethanol. To handle the fact that the e-nose can detect more odour categories, three solutions are possible. The first solution is to acquire the category
52
Chapter 3. Abstraction of Olfactory Information Container 1-3 4-6 7-9 10-12
Table 3.4: Contents of the odour containers and the respective names assigned by the human user according to his/her perception. Note that two different ethanol contents are given the same name. Method MLFF GG
OdourA 98% 92%
OdourB 100% 99%
OdourC 98% 91%
Table 3.5: Correct classification results on the testing data on 20 new samples of each odour.
names according to the human perception, making no explicit emphasis on the extra cluster formation. The symbolic classification would be co-ordinated with the results from the human user. When the testing data is categorised, even though a single point may belong to any one of four different categories, only one of three different category names would be outputted. The classification result is comparable to that from a supervised algorithm such as the ANN (multi-layer feed-forward network, MLFF)as shown in Table 3.3. If the intention of the application is to match perceptions between the artificial nose and the human, then this approach would be sufficient. A secondary approach can be taken in order to enhance the classification and attempt to communicate the fact that there exists a discrepancy between the two categories which were merged in the human perceptual context. This is accomplished by using the parameters found from the fuzzy clustering process in order to better describe the unlabelled data not only in terms of the categories to which they belong but also in terms of the categories to which they possess a similarity. Finally, a third alternative would be to differentiate between two categories that have the same name using the level-1 rules. This would occur when the category names are being determined. For example, if two categories in the contingency table C2 and C3 both have a dominating association to OdourB, they could be differentiated from one another by the addition of a suffix to the name such as OdourB1 and OdourB2. This method explicitly tells the user that another category
3.4. Extension to a Social Agent
53
has been created, however, generating a more descriptive classification would be beneficial in order to understand the relationship between these different odour categories as presented in the previous approach. Which solution should be used depends upon the application and purpose of the experiment. However, the profiling framework is adaptable through the rule-based generation thus allowing some expert knowledge to be integrated into the unsupervised process of category generation as well as the process of odour identification.
3.4
Extension to a Social Agent
The presented symbolic method for classifying odours has been used to integrate a social agent to an electronic nose by [101]. The symbolic classification is a middle step between the raw sensory data and the generation of facial expressions. In [149] a demonstration was developed where the symbolic classifier presented in the previous sections generated an output that contained an odour classification and a degree of uncertainty for each unknown odour. For example, an output of 0.9 Yoghurt is given to an odour which has the highest degree of membership to the Yoghurt class and this degree has a numerical value of 0.93 . Using this output a facial expression driver (FED) maps the classified odour to a pre-determined facial expression (e.g. happiness) and the degree of membership to a stance behaviour. A stance behaviour is a behaviour that represents a degree of confidence, for example nodding of the head. How these mapping occur are outside the scope and contribution of this thesis, readers are referred to [149, 90] for more detail. What is worth noting here, is that the symbolic output contains enough information to further facilitate other means of communication. Uncertainty can still be represented through the stance behaviour in order to capture as much information about the raw sensor data as possible. Through supervised learning, specific odour classes can be used to trigger activation muscles in the model of the facial expressions as shown in Figure 3.5. A facial agent could also be useful to fuse the responses from several sensing modalities, such as the e-nose, an electronic tongue [61], vision, tactile, and sounds sensors. Figure 3.6 shows a number of examples from the demo where different facial expressions are generated based on the output from the e-nose classifier. 3 Such a description can be generated by altering the Level-2 rules in the classification module
54
Chapter 3. Abstraction of Olfactory Information
Figure 3.5: A model of the face topology, the activation muscles and the values to produce a different facial expressions.
Figure 3.6: Different primitive facial expressions. From left to right, the agent expresses happiness, disgust, fear. Notice how neck movements are used to better express a degree of confidence. In the expression of fear the neck is tilted reflecting some degree of uncertainty [149].
3.5. Discussion
3.5
55
Discussion
This chapter discussed one of the more challenging tasks in artificial olfaction: the communication of olfactory data to a human user. There are many factors which make this task difficult. Some of these factors stem from the challenges present on the user-end. Other factors are attributed to the complex nature of the sensor data from the gas sensor array. Regardless, the ability to convey perceptions of odours to a human user is essential if a system is to in any way interact with people. If the end user is a non-expert, the display of information should be intuitive and easily interpretable. If the end user has a good knowledge of the sensing device, the resulting information should be as informative as possible to facilitate proper interpretation of the results. This section discussed some of the challenges involved in the interpretation of artificial olfactory data and presented a system which generates a profile of an odour character using the input from an electronic nose. We applied unsupervised and fuzzy-based techniques to map linguistic descriptions to the e-nose sensor data. Expert knowledge could also be integrated into the system in order to better guide the manipulation of symbols to describe new odour concepts. Although unsupervised algorithms create a greater chance for perceptual conflicts than supervised techniques, we argue that perceptual differences can be highlighted and used as a means to inform the user. Furthermore, in sensing systems such as olfaction which are continuously evolving, this information can be used to make actual hardware implementations, such as the addition of a new sensor designed for special selectivity. Ultimately, the understanding of where and how perceptual differences may occur, allow us to better handle the sensing systems. In this work, linguistic concepts were used as a means to better promote this understanding, provided that the concepts adequately represent the sensor data to which they refer. Typically, fuzzy-based algorithms require the specifications of a number of design parameters. Another disadvantage to this approach is the inability to quantitatively evaluate the performance of the system. As was mentioned misclassified points cannot necessarily be interpreted as erroneous in a system which maintain its own perception of odours. Furthermore, a performance measure in this case would also need to capture how well the symbols correspond to the sensor data. This in itself is not a trivial task and is an intrinsic part of the symbol grounding problem [60]. Furthermore, we do not by any means advocate a symbolic approach to odour recognition for all applications. There are certain cases where a
56
Chapter 3. Abstraction of Olfactory Information
quantifiable output generated from supervised learning is preferable. In particular, cases where specific odours are being tested for quality control and a specific identification of concentration. However, our investigations have revealed that a symbolic output is an intuitive manner to interact effectively with non-expert users especially given the peculiarities in which humans communicate perceptions of odour. Further extensions from the symbolic output can be used to develop social agents which could generate appropriate facial expressions or create flavour wheels which are often used today by panellists. Perhaps in a future context, electronic noses could even be considered as legitimate panellists working together with humans for better quality evaluation.
Chapter 4
Management of Olfactory Information Up to now, we have considered the abstraction of olfactory information on a categorical level. In other words: how to organize the quantitative information from the gas sensors into a qualitative and meaningful representation in symbolic form. We then investigated how this information could be presented to a human user using different techniques. So far, we have considered a system which contains only an electronic nose and no other sensing modalities, and only a classification component and no other processes. The objective of this thesis, however, is to investigate the possibility of integrating electronic olfaction into a larger and complex system. In such systems, the symbolic classification of an odour may be a small ingredient to a larger task that the system is trying to solve. For example, odours may be associated to specific objects and those objects in turn may have other properties measured from other modalities. Another possibility to consider is that the odour character may be dynamically changing over time, which is a common attribute to measure for many quality evaluation applications. So the abstraction of olfactory information does not stop at a symbolic representation of an odour character, but rather we need to also consider how we shall maintain that information over time and across sensing modalities. To better illustrate this problem consider a scenario where a mobile robot is patrolling a corridor. Its task is to discover new objects and gather information about them using both traditional modalities such as vision and sonar and also non-traditional modalities such as the electronic nose. A user is able to monitor the robot, suggest actions to improve perceptual performance and request the robot to perform 57
58
Chapter 4. Management of Olfactory Information
specific tasks. Tasks are requested from the user using a symbolic representation consisting of natural language concepts. While no requests by the user are given, the robot autonomously prioritises tasks alternating between patrolling the corridor and inspecting objects or simply waiting on stand-by. In Chapter 6 we shall address this scenario more concretely, for now we want to focus our attention to a particular challenge which this and other similar scenarios present. The challenge to which we refer is the problem of perceptual management. That is how to maintain the correspondence between the symbolic representation of objects (e.g., Cup-22) and all the perceptual data that refers to that object. It also deals with the problem of collecting information from different sensing modalities working concurrently and/or sequentially and correctly attributing that information to its internal and symbolic representation of the object. Finally, it is in part the problem of preserving object consistency, so that when new information about previously seen objects is available it is also correctly attributed. In fields such as psychology, neuroscience, computational modelling and even philosophy, the very same challenge has been defined as the “the binding problem” [146]. Binding is the association between features to provide a unified representation of an object. For example, the binding of a visual feature of an object such as its shape to its location. Binding is also cross-modal and associates features of different modalities such as the sound of a ball striking a bat and the visual percept of it. Binding is also required across time for interpreting object motion. In addition of unifying aspects of percepts, binding can occur on the cognitive level, such as the link between the visual representation of an object (such as an apple) and all the semantic knowledge of it (edible, how it tastes, etc). For artificial systems, an important ingredient for maintaining perceptual information is an internal structure to store a representation of an object. Such an internal structure needs to satisfy a series of requirements. On one hand, perception management [123], an extension of sensor management [3], is required to integrate high-level information about the state of the object in order to make situation dependent decisions, direct control and select sensing actions. On the other hand, tracking and data association are also fundamental ingredients necessary to propagate information about coherent perceptions over time [130]. A third requirement, especially in the context of a cognitive robot, involves processes, which can create and maintain the link between high-level information (e.g. symbols) and low-level percepts.
4.1. What is Anchoring?
59
In this chapter, we borrow an existing concept of anchoring and show how it can confront some of the issues relating to perception management. The anchoring framework [24] aims at defining a theoretical basis for grounding symbols to percepts originating from physical objects. We present an extension of the framework that is able to cope with perception management considering multi-sensing resources and temporal factors. The pivot of this extension is the use of anchors as internal representations of objects that integrate symbolic and perceptual information across time and across sensing modalities. Our motives for using anchoring is to be able to unify the high level information about odours with the low-level sensor data from the e-nose and other modalities to facilitate the integration of olfaction into an intelligent system. A secondary aim is to shed light on the more general problem of perceptual maintainance, and its significance to cognitive robotics as applications extend into realistic environments involving human-robot interactions and lifelong acquisition of knowledge.
4.1
What is Anchoring?
Anchoring is the problem of how to create, and to maintain in time the connection between the symbol- and the signal-level representations of the same physical object. Previously, solutions to the anchoring problem have been implemented on a system by system basis and could therefore only be applied to restricted domains. In [25] an attempt to frame the anchoring problem and provide a theoretical groundwork to dealing with anchoring particularly on artificial systems was explored. For anchoring to occur certain assumptions are made. The first is that anchoring is concerned with the grounding of symbols that refer to specific object entities such as “a cup” or even more specifically “cup22”. Anchoring is the process of collecting all the sensor data that refers to that specific object and connecting it to its symbolic representation. Anchoring is not concerned with the process of grounding general properties such as “blue” or general concepts such as “difficult”. This is a symbol grounding problem [60] and anchoring is rather a subset of symbol grounding that requires physical sensor measurements of an object to make its connections. A secondary assumption is that anchoring must take the flow of continuously changing sensor input into account to allow for object persistancy in time and space. Even though some properties of an object may change while other remain static, the symbol-percept correspondence should remain in tact with the current and updated information. This
60
Chapter 4. Management of Olfactory Information
dynamic maintainance of information differentiates anchoring from pattern recognition which for the most part do not take into account this dynamic aspect nor the presence of symbols. Anchoring can be seen as a process which creates a shared representation to link several subsystems of an agent, such as the planner to the motion control. How anchoring occurs is unrestricted. Bottom-up approaches allow the sensor data to determine the initiation of an anchoring process, whereas top-down approaches may initiate an anchoring process upon request. Many notable contribution to progress the status on anchoring have been made in recent years. These includes association of words to visually recognised objects [81], managing of dynamic object anchoring for high-level reasoning [22], preliminary works considering people tracking applications [80] and different formulations of the anchoring problem using fluent calculus [43]. In current implementations of anchoring, experimental validations have only considered perceptual information coming from vision and spatial sensors. Furthermore, many implementations only consider either bottom-up or top-down approaches but not both occurring within the same system.
4.2
The Anchoring Framework
Before we present our extension of the framework, we summarise here the basic elements of the computational theory of anchoring. See [24] for a full account. The theory considers an autonomous system that includes a symbol system and a perceptual system, and it focuses on the problem of creating and maintaining a correspondence between symbols and percepts that refer to the same physical object. The main ingredients of anchoring are the following: A symbol system including: a set X = {x1 , x2 , . . .} of individual symbols (variables and constants); a set P = {p1 , p2 , . . .} of predicate symbols; and an inference mechanism whose details are not relevant here. A perceptual system including: a set Π = {π1 , π2 , . . .} of percepts; a set Φ = {φ1 , φ2 , . . .} of attributes; and perceptual routines whose details are not relevant here. A percept is a structured collection of measurements assumed to originate from the same physical object; an attribute φi is a measurable property of percepts, with values in the domain Di . The perceptual system generates percepts and associates each percept with the observed values of a set of measurable attributes and the
4.3. Anchoring for Perception Management
61
symbol-percept correspondence is reified in an internal data structure α, called an anchor. Since new percepts are generated continuously within the perceptual system, this correspondence is indexed by time. At every moment t, α(t) contains: a symbol, meant to denote an object; a percept, generated by observing that object; and a signature, a collection of property values meant to provide the (best) estimate of the values of the observable properties of the object.
4.3
Anchoring for Perception Management
In this section, we present our extension to the anchoring framework to also include percepts from different modalities that may be accessible at different times. To effectively present our extension, we refer to the three abstract functionalities defined by [24]. used to manage anchors, namely, Find, Reacquire, and Track. The functionalities have been developed for the consideration of top-down approaches for information acquisition (i.e. imposed a priori symbolic concepts). In this chapter, we revise these existing functionalities to include bottom-up approaches so that anchors can be created by perceptual observations derived from interactions with the environment. Bottom-up approaches have previously been considered on anchoring frameworks [81] but never in conjunction with top-down approaches. We advocate the presence of both approaches, in particular for robotic systems interacting with a human user. To accomplish this, an additional Acquire functionality is introduced. Creation of Anchors The creation of anchors can occur in both a top-down and bottom-up fashion. Bottom-up acquisition is driven by an event originating from a sensing resource (e.g. the recognition of a segmented region in an image) when perceptual information which cannot be associated to any existing anchor is perceived. Top-down acquisition occurs when a symbol needs to be anchored to a percept, such a call may originate from an external user or a top-level module (e.g. planner). Acquire Initiates a new anchor whenever a percept is received which currently does not match any existing anchor. It takes a percept π, and return an anchor α defined at t and undefined elsewhere. To make this problem tractable, a priori information is given with regards to which percepts to consider. In bottom-up acquisition, a
62
Chapter 4. Management of Olfactory Information randomly generated symbol is attributed to the anchor. Furthermore, information about the object and its properties are included into the world model used by the planner, in this way the object can be reasoned about and acted upon.
Find Takes a symbol x and a symbolic description and returns an anchor α defined at t (and possibly undefined elsewhere). It checks if existing anchors that have already been created by the Acquire satisfy the symbolic description, and in that case, it selects one. Otherwise, it performs a similar check with existing percepts (in case, the description does not satisfy the constraint of percepts considered by the Acquire). If a matching percept is found an anchor is created. Matching of anchor or percept can be either partial or complete. It is partial if all the observed properties in the percept or anchor match the description, but there are some properties in the description that have not been observed. Maintenance of Anchors At each perceptual cycle, when new perceptual information is received, it is important to determine if the new perceptual information should be associated to existing anchors. The following functionality addresses the problem of tracking objects over time. In this extension, we include the previous Reacquire functionality as an integral part of the Track and make no special distinction for it. Track The track functionality takes an anchor α defined for t−k and extends its definition to t. The track assures that the percept pointed to by the anchor is the most recent and adequate perceptual representation of the object. We consider that the signatures can be updated as well as replaced but by preserving the anchor structure we affirm the persistence of the object so that it can be used even when the object is out of view. This facilitates the maintenance of information while the robot is moving as well as maintaining a longer term and stable representation of the world on a symbolic level without catering to perceptual glitches. Deletion of Anchors By having an anchor structure maintained over time, it is possible to preserve the perceptual information even if the object is not currently perceived (caused by the object being out of view and/or by the inaccuracy in the measurement of perceptual data). The challenge is to
4.3. Anchoring for Perception Management
63
Symbolic System door5 table1 room23
cup-22
FIND TRACK
Anchoring Module Anch-1 Visual Description
Anch-1 Visual & odour
cup-22 Visual & odour
ACQUIRE
Perceptual System
Figure 4.1: Graphical illustration of the extended anchoring functionalities where bottom-up and top-down information is possible and different sensing modalities are used.
determine if the association of new percepts is justified or whether certain anchors should be removed. Mechanisms for destroying anchors when the corresponding object has been removed need to be in place. This is a difficult problem, because conceptually it is not clear when it is appropriate to remove anchors from the system. Anchors could be removed if they are not relevant for the current task, because the object to which it refers has been physically removed from the environment or the reliability of the perceptual information has expired. Anchors may also need to be removed if they have been associated to invalid perceptual data such as sensory glitches. We currently adopt simple solutions in which objects that are not perceived when expected decrease in a “life” value of the respective anchor. When the anchor has no remaining life, the anchor is removed. The converse could be implemented where anchors are created with initially lowlife values and persistent percepts increase its life value. The decreasing life of anchors is shown in Figure 4.3. A more adequate strategy to handle the maintenance of anchors may also be to include a “long term” memory where anchors may be stored for future use.
64
4.4
Chapter 4. Management of Olfactory Information
Integration of the Functionalities
The event-based functionalities are now restricted to the Find and Acquire while the Track functionality is regularly called. Figure 4.1 shows an overview and an example of the framework and its functionalities. In the example, anchors are created bottom-up from the visual percepts of a cup. Later, additional features of that object are required, for example, the olfactory property. These features are stored in the anchor. When a top-down request is sent to the anchor module to find a cup with matching properties denoted by the symbol “cup-22”, the Find functionality anchors the symbol to the perceptual data. As seen in the figure, properties can be collected at different time points using different modalities. Even when certain perceptual properties are updated, such as the smell property, which may change over time, other perceptual properties are maintained. Conversely, if the visual percepts of an anchor is replaced, the smell property previously obtained is not lost. In this way, the anchor is used to compensate for any dynamically changing features of an object. Furthermore, the perceptual description of anchors can be accessed by the planner to reason about perceptual knowledge. In certain cases, this may result in specific calls to perceptual actions in order to disambiguate between similar objects.
4.5
Example
Consider the corridor monitoring scenario introduced at the beginning of this chapter. Here we outline a brief example of how the anchoring module operates within such a system. In each corridor there may be several objects, in this case garbage cans. The robot automatically toggles between the task of patrolling the corridor, inspecting objects and waiting for commands from the user. Patrolling the corridor involves moving from corridor to corridor in a discovery for new objects and recognition of previous objects. When an inspect is invoked, the robot visits each object collecting the odour property. The inspect is usually autonomously invoked when new objects are detected. The robot is equipped with several heterogeneous sensing modalities such as a camera, sonar, tactile sensors, and an electronic nose. In this example, the modalities of interest are the vision, and e-nose. The vision component is trained to detect any visual signal matching garbage cans. For each found object we extract a number of properties such as color, size and relative position using different heuristics. The collection of the
4.5. Example
65
properties belonging to an object is called a percept. The electronic nose component is able to classify odours providing a symbolic categorical description (as explained in Chapter 3). The robot is also able to localize itself within the corridor using odometry and has a number of high level processes such as a planner which reasons about actions and a plan executor which can monitor motion control. Before we begin to outline the corridor example, let us first examine the structure of an anchor. Figure 4.2 shows two anchor structures that have been created bottom-up from a segmented image. The anchoring module updates anchors such that at every moment t, α(t) contains: Name - For top-down anchors a name is a symbol denoting the object in the planner (e.g., Silvia’s Cup). For anchors that have been created bottom-up the name is initially arbitrary. Symbolic description In general a symbolic description is given in a top-down fashion, however for bottom-up anchors the symbolic description may also be derived directly from the perceptual information of the object. For example, the odour classification module may populate the symbolic description with the linguistic odour name βx when classifying an odour associated to a particular object. Perceptual description The perceptual description is a vector which consists of the important properties of the object such as position (relative and global), colour, shape, and when available the odour x. signature, R In the figure the two anchors, Gar-36 and Gar-34, are visually similar however Gar-36 currently has an olfactory property. In the current implementation an anchor is “baptised” with the name of the percept which initially invoked the bottom-up process. As will be shown in the next example, the percept may be updated but the anchor persists using the tracking functionality. In Figure 4.3 The local space of the robot together with the visual image from the camera as well as the creation, deletion and updating of anchors is depicted during a run through the corridor. The figure contains four snapshots described as follows: Scene 1 - The robot begins patrolling the corridor, two visual percepts are detected and two anchors denoted by Gar-1 and Gar-2, are created. An inspect is performed and both anchors obtain olfactory properties, shown in the Figure by the grey colouring.
Figure 4.2: (Left) Segmented image from the vision module observing two green garbage cans at the center of the screen. (Right) The anchors created in a bottom-up manner for each object. Gar-34 refers to the garbage can on the left of the image and Gar-36 refers to the garbage can on the right.
1
1 0 1 0 1 0
2
Gar-1
Gar-1
track
Gar-2
Gar-2
track
Gar-3
Gar-1 Gar-16
3
4
track
Gar-27
track
Gar-35
Gar-24
Figure 4.3: The top row shows the camera images at different time points, the middle row shows the activity at the anchoring level. Grey bars indicate anchors with olfactory properties. The bottom row shows the corresponding local perceptual space given the changing representation of visual percepts.
4.6. Discussion
67
Since the anchors are created in a bottom-up fashion their labels are arbitrary. Scene 2 - As the robot continues its patrol, another object is inserted into the environment at a later time. Note however, that the previous two anchors are still maintained by the track functionality. Although the local space shows only the current percepts, the anchoring module updates the link between the anchor Gar-1 and the percept Gar-27. A new anchor is also created for the third object denoted by Gar-3 with visual percept Gar-24. Scene 3 - The robot approaches the object in order to acquire its odour property and the result is stored in the corresponding anchor. Some time later, the object is removed from the environment. The life of the anchor slowly decreases when an expected percept is no longer detected. Scene 4 - The anchor is removed from the system and unless it is perceived again, its properties cannot be accessed by the find functionalities described above. This scenario shows how the anchoring module is used to create an internal structure which can then maintain the perceptual coherence of objects, considering each object has both spatial and olfactory properties. Even when visual properties of anchors are being updated, the stored smell property remains until a new odour character is acquired by the next inspect action. The previous odour character is then stored in the odour repository.
4.6
Discussion
“Take a coin, toss it, and catch it again in your hand...you see a single object fly up in the air, twist over and over, and land in one piece on your hand. Bits don’t fly off. The silver doesn’t depart from the shape, and the shape doesn’t lag behind the motion.” [15] (p.244). In this chapter, we discussed how a modified anchoring framework could be used as a tool to maintain perceptual coherence of objects over extended periods of time. This tool requires the management of perceptual information from different sensing sources, tracking over time and maintaining object persistency. In Chapter 6 the corridor scenario will be revisited more extensively.
68
Chapter 4. Management of Olfactory Information
In sum, the anchoring problem needs to be solved for any robotic system that incorporates a symbolic component. This solution is nontrivial and is far from being fully understood. Furthermore, a number of open and immediate challenges exist. Currently, a definition of which processes should be handled at the symbolic level, versus the sensor level is missing and what processes are exclusive to the anchoring level need to also be defined. For example, hypothesis testing may be one method to handle uncertainties in the sensor data but where this should occur in the context of a general framework is not clear. Still, practical solutions to the anchoring problem are now starting to emerge. We have extended such a framework to include systems which consider olfactory information as part of the perceptual process. A secondary addition to the framework was the inclusion of a bottom-up approach to creating anchors suited for our corridor scenario. On one hand, the contribution to the field of anchoring includes the consideration of additional sensing modalities. On the other hand, the concept anchoring has contributed to the general idea of perception management to maintain a coherent notion of objects, which we shall show in the next chapter is an important ingredient for systems using planning and reasoning components to perform actions in the environment.
Chapter 5
Putting Olfaction into Action We have introduced two significant extensions to the development of electronic olfaction. The first dealt with an extension of the electronic nose data to a symbolic representation of odour categories. We then considered that this symbolic representation would be part of a more complex system and discussed the need to create and maintain a symbol to percept link when olfactory information is one ingredient to the entire perception of an object. Considering the general intelligent system, an important component to be discussed is the ability of the system to reach its objective by selecting the appropriate response. A response may be a single action or a set of several actions. The integration of an electronic nose in such a system would mean that the system could determine when olfactory information is needed, where in the environment this information could be found and how to use the e-nose to best collect this information (e.g., positioning of a robot). In Chapter 1, we had introduced this specific problem along a time axis where olfactory information may be anticipated . In the area of artificial intelligence automated planning techniques study this deliberation process that chooses and organizes such actions depending on specific outcomes. In this chapter, we shall investigate the integration of olfaction at the planning level by investigating a specific instance of a conditional planner on a mobile robot. In the literature, many forms of planning have been investigated from path and motion planning to economic planning. Here we are specifically interested in perception planning and generating plans that involve sensing actions in order to gather more information 69
70
Chapter 5. Putting Olfaction into Action
about the environment. Navigational planning as well as recovery planning are also needed particularly in the context of an embedded and mobile system with olfaction. To our knowledge, using an electronic nose with high-level reasoning has not been investigated by those in the electronic olfaction community nor those in the planning community. Though some works have a similar flavour to that being discussed here, such as active vision for object recognition [84, 110], it is hoped that the novel investigation with olfaction will be of benefit to both communities. It is worth emphasizing that the contribution here does not reside in the development of a planner. Instead, we use an existing planner developed by [77, 78] and integrate an olfactory component into that planning framework. We conclude this chapter with a brief example that shows how a mobilerobot uses active sensing to assess the odour character of an object.
5.1
Electronic nose as an Active Sensor
In Chapters 2 and 3 we investigated how an array of gas sensors can be used to detect and discriminate odours. The distinguished features of artificial olfaction that make e-noses suitable active sensors are: Sampling process of an odour is long and expensive. In our previous experiments the sampling of a single odour can take up to 2 minutes. The length of this procedure depends primarily on the method of sampling where a baseline is normally collected in order to reduce the effects of drift as well as capture and provide as much information about an odour, so that the pattern recognition process can distinguish among a small collection of different substances. Localization of the an odour source is a difficult problem with odour sensors. In systems with vision modalities, odour source candidates may be identified first using spacial information. Still however, there is a conceptual problem concerning the validity of associating a dispersed odour in the air to an object physically located at a distance from the actual point of detection. This problem becomes particularly difficult when multiple odour sources are present, see Figure 5.1. Resource Consumption for the operation of an electronic nose is high. The delivery system often consists of pumps and valves which rely on independent power sources. The sensing arrays also require
5.1. Electronic nose as an Active Sensor
71
I smell Coffee!!
Figure 5.1: A situation where active sensing may be needed. Here the robot in the middle of the image detects an odour using the gas sensors but sees several candidate odour sources. Planning can be used to first approach each candidate and then actively engage a thorough odour sampling process in order to determine the character of each object and resolve the ambiguity.
heating to high temperatures for correct operation. Particularly for mobile systems with limited battery, the sensing cost given the power requirements need to be considered. Although these are strong motivations it is not to suggest that all gas sensors should be used actively. Many odour-based navigational platforms choose to constantly sample an environment to detect increasing odour concentrations. By using this method it is possible to compare current samples with past samples to assess concentration levels. However, this technique often requires that the target odour is a specific molecular structure to which the sensors will exhibit a strong reaction. Rather, in odour classification, an odour’s signature contains each gas sensor’s reaction to an analyte with respect to a reference. Each odour sample is a comparison from the sensors’ baseline condition to the reaction to the gas. It is important that after sampling an analyte the sensors
72
Chapter 5. Putting Olfaction into Action
recover from exposure. Odours are learned and the success of identifying new samples depend upon the validity of a training set. Given that the sensors are prone to drift, training samples may need to be updated online. Moreover, new odours may be added to the repository. It is not enough that the sensors react to the presence of the gas but need to exhibit a similar pattern with respect to the one learned for correct identification.
5.2
Planning for Olfactory Actions
The symbolic reasoning and decision-making component of the robot is a planning system, called PTLplan developed by [77, 78]. The planner receives top-level tasks from the user, but it also used for recovering from problematic situations.
5.2.1
Conditional Planning under Uncertainty
PTLplan is a planner for partially observable domains with uncertainty (probabilities). It searches in a space of epistemic states, or e-states for short, where an e-state represents the agent’s incomplete and uncertain knowledge about the world at some point in time. An e-state can be considered to represent a set of hypotheses about the actual state of the world. Each sub-state in the e-state consists a set of value-assignments to properties or state variables, where a property might represent for instance the shape or smell of a particular object: (shape b1 = cup), (smell b1 = ethanol). Sub-states are also associated with degrees of probability. The planner can reason about perceptive actions, such as looking at or smelling an object, and these actions have the effect that the agent makes observations that may help it to distinguish between the different hypotheses. Each different observation will result in a separate new and typically smaller e-state, and in each such e-state the agent will know more than before. For instance, smelling a cup may result in two observations leading to two possible e-states: one where the agent knows there is ethanol, and one where it knows there isn’t. Figure 5.2 illustrates the effects on e-states of two actions: one for movement, and one for perception (smell). It would also be possible to represent sensor errors — observations need not always coincide with the actual state. PTLplan is a forward-chaining planner, which basically adds one action at a time, introducing conditional branches whenever several observations are possible (like after the smell action in Figure 5.2), and keeping the search space down by the use of strategic knowledge formu-
Figure 5.2: Epistemic states (thick borders) contain one or more substates with associated probabilities (hypotheses), and are transformed by the applications of actions (arrows). Note how different observations yield different new e-states.
lated as logical formulae. It searches for a plan with a probability of success above a given threshold, and returns such a plan as soon as it is found. We omit further details of how plans are generated here, as our approach is not dependent on the particular planning algorithm. Actually, another planner with corresponding power of representation could have been used instead (e.g. [116, 104, 75, 10]).
5.2.2
Recovery Planning for Ambiguous Situations
As mentioned before, the planner is used (among other things) for resolving ambiguous situations in anchoring, i.e. situations where the robot needs to gather more information about the different perceived objects in order to determine which one of them matches the given description. An ambiguity problem is solved in three steps: 1. The planner automatically formulates an initial situation (e-state) by considering the properties of the requested object and of the objects that are actually perceived, and generating different hypotheses about which of the perceived objects corresponds to the requested object. It also formulates a goal that the requested object should be identified, provided it is actually there. 2. The planner searches for a plan taking as parameters the e-state and the goal, using the available actions provided by the robot. 3. The plan is executed; either the requested object is found, identified and can be anchored, or it is established that it cannot be identified.
74
Chapter 5. Putting Olfaction into Action
We now present these steps in more detail.
5.2.3
Formulation of the Initial Situation and Goal
In an ambiguous situation, when there are several partially matching perceived objects, the agent needs to figure out which one of them actually matches the requested object (b1). Consequently, it generates a number of hypotheses relating the different ways b1 can be anchored, based on the known properties of b1 and the properties of the perceived objects. Based on the descriptions d for the requested object and di for each perceived object poi , two extra descriptions are formulated for every di : first, a description d+ i which completely matches d; and second, a non-matching description d− i which contains the different ways in which at least one incompletely specified property in di may not match with d. For instance, if d = (smell b1 = ethanol) and d1 = (smell po1 = ethanol none) (i.e. either ethanol or none), then − d+ i = (smell po1 = ethanol) and di = (smell po1 = none). Each hypothesis − then consists of the conjunction of one d+ i for one of the poi and dj for all remaining j = i. To each hypothesis is also added the statement (anchor b1 = poi ) denoting that b1 should be anchored to the object anchored by poi . There is also one hypothesis that no object matches: d− j for all j, and (anchor b1 = none). Finally, if the planner wishes to take a cautious approach and ascertain that no more than one object is matching, it + might also add a number of hypotheses consisting of d+ i , dj for two of the poi , poj and dk for all remaining k = i, j, and (anchor b1 = none). For instance, if b1 is known to be a cup containing ethanol — (smell b1 = ethanol) — and we perceive two cups anchor-1 and anchor-2, the (incautious) hypotheses might be: 0.4 0.4 0.2
Note that not-ethanol is defined as the complement of ethanol. The goal is achieved once a specific action (anchor b1 x) has been performed. This action has as a precondition that x is the only remaining anchor for b1: (nec (anchor b1 = x)). Thus, all other candidate anchors have to be eliminated before anchor is applied.
5.2. Planning for Olfactory Actions
5.2.4
75
Plan Generation
After the initial situation and the goal have been established, plan generation starts, using the appropriate domain description. The following action, for instance, lets the robot observe the smell of an object. (ptl-action :name (smell ?y) :precond ( ((?y) (perceived-object ?y) (robot-at = ?y)) ) :results (obsval (smell ?y)) :execute ((smell-behavior ?y)) )
In short, the precond part states that the action requires a perceived object ?y, which the robot must be positioned at. The result part states that the robot will be able to observe the smell of object ?y. The obsval form is a way to encode that the agent makes a specific observation. The right side of Figure 5.2 illustrates the effect of the smell action on one specific e-state. It is in the planner’s domain descriptions that information about the capacity, constraints and uncertainty of the e-nose and other sensors are modelled. For instance, the following more elaborated version of the smell action (used in an experiment involving a suspicious suitcase) takes into account that the robot might have failed to position itself in an adequate way before smelling: (ptl-action :name (smell ?y) :precond ( ((?y) (perceived-object ?y) (robot-near = ?y)) ) :results (and (cond ((positioned-to-smell = ?y) (cond ((smell ?y = TNT) (obs (smell ?y = TNT))) ((not (smell ?y = TNT)) (obs (smell ?y = air)))))) ((positioned-to-smell = false) (obs (smell ?y = air)))) (robot-near = false)) :execute ((smell-behavior ?y) (not-near me ?y)) )
Here, the property (positioned-to-smell = x) is determined when the robot attempts to approach a particular object: if successful, x becomes that object, and otherwise x = false. When the smell action then is performed, its outcome is conditional1 on positioned-to-smell. Note that after smelling, the robot will move away a short distance from the object it smelled: (robot-near = false) . The plans generated by PTLplan are conditional: after each action which has several alternative observation effects, the plan branches. The plan below is the solution to the problem of finding which one of three perceived cups (anchor-1, anchor-2, anchor-3) should be anchored to b1, 1 The syntax of conditional effects is to be read as follows: (cond ( condition1 result-1 ) ( condition-2 result-2 ) ... ). Each result-n may in turn contain new conditional results.
76
Chapter 5. Putting Olfaction into Action
which is known to smell of ethanol. Note how a conditional branching follows after each application of smell: the first clause “(smell = ethanol/notethanol)” of each branch is the observation one should have made in order to enter that branch, and the subsequent clauses are actions. The action (anchor b1 x) at the end of each branch represents the decision to anchor b1 to some specific perceived object (or to no object at all, if x = none). ((move-near anchor-1) (smell anchor-1) (cond ((smell anchor-1 = not-ethanol) (move-near anchor-2) (smell anchor-2) (cond ((smell anchor-2 = not-ethanol) (move-near anchor-3) (smell anchor-3) (cond ((smell anchor-3 = not-ethanol) (anchor b1 none) :success) ((smell anchor-3 = ethanol) (anchor b1 anchor-3) :success))) ((smell anchor-2 = ethanol) (anchor b1 anchor-2) :success)) ((smell anchor-1 = ethanol) (anchor b1 anchor-1) :success)))
5.2.5
Plan Execution
When the plan is then executed: the actions such as (smell anchor-1) are translated into executable perception and movement tasks (see field :execute in the definition of smell above). The anchor action has a special role: it causes the symbol of the requested object to be anchored to a specific perceived object. The robot can then continue performing the task in its top-level plan that was interrupted.
5.3
Example
Scenario description For objects of awkward and/or large size, the place where odour sampling occurs may not be the place of odour emission. To investigate how to contend with such a situation, a robot is required to investigate a suitcase, shown in Figure 5.3. The objective is to determine whether the suitcase contains traces of a “dangerous” substance. Furthermore, in this scenario we investigate how the probability of a false negative can be minimized. The risk of false negatives in this case can be reduced by attempting to approach the suitcase and sample it repeatedly, from several sides, thereby increasing the probability that at least one attempt is successful. In the planner’s model the electronic nose does not detect phantom odours, i.e. yields false positives. This is an accurate statement since in reality there are few occurrences in which the sensors will react when no
5.3. Example
77
Emission Side
90
180
270
Figure 5.3: The robot approaching a suspicious suitcase for careful inspection. odour is present. Consequently, the planner’s model currently does not take the possibility of false positives into account.2 Implementation The planner was equipped with a domain model including an action for approaching and docking to a suitcase or other objects from a given angle (0, 90, 180 or 270 degrees), with a certain probability that the robot is inadequately positioned after that action. There was also a smell action, which, if the robot was adequately positioned, would detect the “dangerous” substance. A cleaning agent that was lightly sprayed on one side of the suitcase was used as a stand-in for the “TNT” described in the planner. The side of the suitcase that contains the odour is referred to as the “emission side” (Figure 5.3). A black suitcase was positioned in the experiment room, and the robot was then given the goal to determine if this was a suitcase with TNT, and a desired certain maximum probability of failure (false negative). From the planner’s perspective, each sampling attempt has a certain probability, p, to fail and therefore the accumulated probability of n consequitive failures (1 ≤ n ≤ 4) is pn . The value of p is application dependent and in our experiments p = 20%. This value was chosen according to the expertise of the system designers, however in reality experimental investigations should be made to determine the probability. The plans generated by the planner had the following form: ((approach-from anchor-1 -90) (smell anchor-1) (cond ((Smell anchor-1 = TNT) (anchor s1 anchor-1) :success) 2 It is however well within the capacity of the planner to do so, and it might be desirable in situations where e.g. there are other suspicious suitcases very close the one being investigated, and the odour might come from one of those.
78
Chapter 5. Putting Olfaction into Action
Classification Rate
100 90 80 70 60 50 40 30 20 10 0 0
90
180
270
Rate of Successful Detections
Relative Docking Angle to Emission Side
60 50 40 30 20 10 0 Early Detection
Punctual Detection
Late Detection
Time of Detection
Figure 5.4: (Top) Successful rate of detection depending on the angle of approach. (Bottom) The rate of successful detection given a fixed emission angle and the fact that the robot may not necessarily choose that angle first in the execution of perceptual actions. ((smell anchor-1 = air) (approach-from anchor-1 0) (smell anchor-1) (cond ((smell anchor-1 = air) (approach-from anchor-1 90) (smell anchor-1) (cond ((smell anchor-1 = air) (anchor s1 none) :success) ((smell anchor-1 = TNT) :fail))) ((smell anchor-1 = TNT) (anchor s1 anchor-1) :success)))))
Results To examine how well the robot was able to correctly assess the odour of the suitcase depending on which side it sampled, 20 trials on each side were taken. Figure 5.4 (Top) shows successful rate of classification vs. the relative angle of approach from the emission side and indicates that the further the robot is from the emission side, the worse the classifica-
5.4. Discussion
79
tion rate. This result is expected since larger objects with more awkward shape will have an effect on the distribution of the odour around the source. Furthermore, the type of material of the object could effect classification depending on how the odour is absorbed. Of the 82% successes, about 37% of the trials found an odour at 90◦ from the emission point and only 18% at 180◦ . Another important factor to examine is how many perceptual actions were required before the correct assessment of the suitcase was given. This allows us to evaluate the benefit of using the planner. Of 50 trials which evaluated the suitcase, 82% provided correct asssessment when an odour was present and 100% when no odour was present. The benefit of calling additional smelling actions can be seen in Figure 5.4 (bottom). On the x-axis three terms are defined: early detection, punctual detection and late detection. Early detection occurs when the robot approaches any non-emitting side and provides a correct final assessment. Punctual detection occurs when the robot may have sampled other sides previously but when sampling the emission side it correctly resolves its goal. Finally late detection occurs when the robot may have sampled the emission side with no final assessment but in the examination of additional sides it is then able to provide correct evaluation. In the experimental runs, early detection occurs in about 29% of the successful trials and late detection occurs in approximately 23%. Therefore, planning for additional smelling actions in this case was able to increase the success rate by preventing the robot from drawing premature conclusions about the nature of the package before careful inspection.
5.4
Discussion
Olfaction is a valued sense in humans, and robotic systems, which interact with humans, and/or execute human-like tasks also benefit from the ability to perceive and recognise odours. While previously gas sensors were difficult to use and needed certain expertise to successfully implement odour recognition, commercial products have now made it possible to successfully employ electronic olfaction in new domains. The integration of electronic olfaction presents interesting challenges with respect to the use of AI techniques in robotic platforms. Some of these challenges are due to the properties of the sensing mechanism, such as long sampling time, and close proximity required for smelling an object. In this chapter, we advocate that the e-nose is explicitly called within the system at appropriate occasions. Planning is an essential AI component to accomplish this.
80
Chapter 5. Putting Olfaction into Action
Conditional planning is particularly useful when sensing actions are involved that do not always have certain outcomes and that the state of the world will not always be completely known. Other types of planners could have been used such as a partially observable markov decision process (POMDP) [75] but the purpose of our work here is not to promote one type of planning over another. Rather our objective is to illustrate how an existing planning method could make use of olfactory information.
Chapter 6
Odour Discrimination for a Mobile Robot In Chapter 4 and 5 we had presented two short examples of a mobile robot with an electronic nose that discriminated odours. We shall now extend these two examples and present in more detail a mobile olfactory platform. In general, gas sensing for mobile robotics is a large topic mainly due to the variety of potential applications. The common thread between the applications that shall be presented is that we deal explicitly with the notion of objects. In other words, we are interested in finding an odour property that can be associated to a specific source such as a cup or a garbage can. The object emitting the smell is also assumed to have other properties. These properties may be spatial such as shape and position, or tactile. To detect these properties we use other types of sensing modalities such as vision (camera) and sonar. These modalities together with processes for control, planning, information management and abstraction, form the cognitive robotic architecture that is presented in this chapter. By consideration of information from other sensing modalities we add the third and final axis presented in our approach in Chapter 1. In this chapter we include a number of experimental scenarios that show how our instantiation of a cognitive robotic system copes with an olfactory module. The diversification of tasks allows us to examine the ability of the same platform to be applied to a variety of contexts. We not only validate the odour classification performance but also examine the interaction between the different modules of the system. This interaction is validated both in the short-term context and in a long-term context. 81
82
Chapter 6. Odour Discrimination for a Mobile Robot
Green Cup
Snout of e-nose Baseline reference Figure 6.1: (Left) A view of Pippi the robot with the Cyranose 320 mounted underneath the camera (Right) The top view of Pippi when sampling from a green cup.
6.1
The Physical Robot
Let us introduce Pippi, a Magellan Pro indoor mobile robot, whose name is inspired by the Swedish fictional character Pippi Longstocking, best described as unconventional and assertive (not unlike our robot). Pippi (the robot) is approximately 0.25 m high with a diameter of 0.4 m. Figure 6.1 shows two different views of the robot. The motion of Pippi is driven by a 2-wheel drive with differential steering. The robot has a rear balancing caster and is equipped with a number of sensing modalities. Odometry sensors on the two wheel motors 16 tactile sensors placed at the lower base of the robot. 16 sonar ranging sensors placed at the upper base of the robot. Sonars can measure distances to obstacles with range of 0.15 - 7.0 m. A fixed color CCD camera fixed at the top of the robot 16 infrared (IR) proximity sensors which currently are not used. An electronic nose placed under the camera On Pippi, line of sight and line of ”smell” are coordinated. This is done by placing the e-nose below the camera and odour samples are drawn from a uni-directional airflow unit at the front of the robot, shown in Figure 6.1. The detection of an odour is therefore maximized when
6.2. System Architecture
83
the robot is directly facing the source of odour molecules. Pippi is also equipped with an on-board computer with a 550Mhz Pentium III processor running Linux. On this computer a number of processes for low level control and perception is running, communicating with high level decision making processes through a TCP/IP protocol. Furthermore, the robot is equipped with a wireless network so that we can monitor the robot and run the high level functions from other computers.
6.2
System Architecture
The system architecture is a sense-plan-act architecture with an added middle layer used for anchoring, see Figure 6.2. When an initial task is supplied to the system the planner described in Chapter 5 can generate top-level plans. The plan executor receives the plans and performs the required actions, while monitoring their execution. Actions involving movement are sent to the navigational planner and controller. The navigation planner in turn is built on top of a fuzzy behaviour-based controller. Its task is to generate low level plans describing which behaviours to activate under different conditions. The resulting behaviours are executed in parallel, using fuzzy blending for determining the actual control signal to use. This gives us a smooth execution allowing the robot to navigate to the target while, for instance, avoiding obstacles and making observations. For sensing actions the plan executor calls upon the perception routines, which include an anchoring module, image processing and processing of olfactory data. When sensing actions are requested, the anchoring module uses the current known state of the world and tries to provides a list of matching candidate objects in the world model. Recall from Chapter 4 that the this state is continuously being updated using the vision sensors with bottom-up anchoring. It may occur that the current knowledge of the world is insufficient to unambiguously find the correct objects and the system uses the planner in order to generate the right sequence of actions to collect the necessary information, in which case recovery planning may be needed. However, after the successful execution of such recovery plans the robot resumes the original task, until it has successfully completed all the wanted actions. In the following subsections we shall detail each of the processes in the system. The olfactory, anchoring and planning modules have been outline in Chapter 3, 4 and 5 respectively, and only certain changes that have been made specific for this platform will be described.
84
Chapter 6. Odour Discrimination for a Mobile Robot User goals & tasks
Planner (top-level & recovery) Interface Plan
Problem (state & goal)
Plan executor & monitor Request
Anchors,Status
Destination
Navigation Planner
Anchoring Commands
Classes
Odour Processing Commands
Data
E-Nose
Status
Percepts
Vision Processing Images
CCD Camera
Behaviours
Status
Fuzzy Control Commands
Status
Robot Control, Odometry, etc..
Figure 6.2: Overview of the complete system on the mobile robot.
6.2.1
Vision Module
In addition to the sonars used to navigate and detect obstacles, the system also uses vision to perceive objects. This is done by using a frame grabber card and taking snapshots from a CCD camera, which are processed on board the robot in order to identify objects and their properties. For the image processing we are using standard techniques. First the captured image is pre-processed using standard filters, segmented using mean shift segmentation in the Luv color space. After segmentation each segment is examined and pattern matched against the contours of a number of predefined objects. For each found object we extract a number of properties such as color, size and relative position using different heuristics. The collection of the properties belonging to an object is called a percept. The system works well for cases where we have single coloured, convex objects standing on the floor. Images are received once every second taking into account the total execution speed of the image processing when executed together with all other processes. The percepts from the vision processing is sent over TCP/IP to a perception process. This perception process is using the percepts in order to build a database of perceived objects, attempting to match percepts to
6.2. System Architecture
85
earlier seen objects or creating new objects for completely new percepts. Together with some simple filtering this gives a representation of time persistent objects as opposed to percepts only valid for one frame. In order to maintain and predict sensed objects currently outside the camera’s viewpoint an odometry based localization is used so that objects not currently perceived can be remembered. For limited movements, the system can easily reacquire objects based on their stored position. If movements are large, objects move or if the accumulation of odometry errors is too big this might lead to reacquisition ambiguities which can only be resolved using more complex methods [20].
6.2.2
Olfactory Module
The olfactory module consists of the electronic nose, a pattern recognition and classification component, and an electronic odour repository of odours stored in the on-board computer of the robot. The e-nose used is again a Cyranose 320 which contains 32 thin-film carbon-black conducting polymer sensors of variable selectivity [27]. Each sensor consists of two electrical leads on an aluminum substrate. Thin films are deposited across the leads creating a chemiresistor. Upon exposure to an analyte, a polymer matrix absorbs the analyte and increases in volume. This increase in volume is reflected in a increase in resistance across the sensor. Each polymer in the array is unique and designed to absorb gases to different degrees, creating a pattern of responses across the array. The array of sensors are contained in a portable unit also consisting of pumps, valves and filters that are required to expose the sensor array to a vapour gas. Odour Sampling Like the experiments conducted in Chapter 3, the sampling of an odour occurs in three phases. The first phase is a baseline purge, where the sensors are exposed to a reference gas. The reference gas is filtered air that has been prepared in a ventilation cabinet and is contained in a 50 ml sealed vile and fixed onto the robot. The duration of the purge is 30 seconds and is based on the manufacturer recommendation when a charcoal filer is used. The second phase is a sampling cycle where a valve is switched to allow vapours from a sampling inlet to come into contact with the sensing array. The duration of this cycle is 20 seconds and a substrate temperature is maintained at 42.0 ◦ C during sampling. Finally, a sequence of purging cycles is used to remove the sampled odour from the unit and restore the sensor response to the baseline values.
86
Chapter 6. Odour Discrimination for a Mobile Robot Sampling Inlet (snout) Stainless Steel
Baseline Filter Poly-paraxylene
(Charcoal)
Sensor Array
Pump
Valve Reference gas
Exhaust
Figure 6.3: Sampling setup for the electronic nose. A schematic of the sampling setup for the electronic nose is giving in Figure 6.3. Figure 6.4 (Left ) illustrates a typical response of all sensors throughout the phases of sampling. Odour Classification The odour classification component is connected to an olfactory interface described later. In the interface, different algorithms can be applied which also include the algorithm which was presented in Chapter 3. For pre-processing, the complete data set is normalized, by dividing each Rk,s by the maximum value for each sensor, in order to set the range of each sensor to [-1, 1]. The main difference in the pre-processing step here is that principal component analysis (PCA) is performed and the two first principal components are used as features for the classification algorithm. Recall that the signals are gathered in a response vector Rk where each sensor’s reaction is represented by: Rk,s =
Rmax,s − Ro,s Ro,s
(6.1)
for s=1 to 32. Rmax,s is the maximum signal during the measurement cycle and Ro,s is the baseline response. Each odour sample k is also represented by a linguistic label lk , where lk ∈ D is a user specific discrete set. For example D may contain the collection of odour names. In Chapter 3, we had presented an algorithm that was based on a semi-supervised fuzzy-clustering algorithm to perform classification. The use of a fuzzy algorithm provides for the explicit representation
87
Sensor Response (Rt-Rt=0)/Rt=0
6.2. System Architecture
Time(s)
Figure 6.4: Example of the raw sensor data throughout the three phases of sampling an odour. of uncertainty in the classification output (this uncertainty can later be used by the planner to initiate further perceptual actions); it also provides for easy integration of symbolic representation without the need to map specific outputs to categorical names. To assign each cluster an odour character, in the presented algorithm a contingency table is used that represents the probability of an odour with label lk occurring within a prototypical region around the cluster center Cc . Labels of points which most frequently occur within a cluster’s prototypical centre are then used as the naming convention for that cluster. Unlabelled data is identified by evaluating a degree of fulfilment β(x) for each odour odour x to known odour categories: β(x) = µx,C1 (C1 ) ∧ µx,C2 (C2 ) ∧ . . . ∧ µx,C3 (Cc ).
(6.2)
Only the label of the cluster and the membership value to which µx,Cc has the highest value is given as the classification result for the odour. This description is a trimmed version of the full odour description. The odour classification is then used by anchoring component and are stored within the symbolic description of an object. All odour records are then stored in the repository.
88
6.2.3
Chapter 6. Odour Discrimination for a Mobile Robot
Robot Control and Navigation Architecture
The control and navigation system is organized in a three layer hierarchical structure and consists of sensing modules (odometry, sonar, and vision, etc.), a fuzzy behaviour-based controller, and a navigation planner. The fuzzy behaviour based controller proposed by [129] uses fuzzy logic to determine activation levels of different behaviours such as obstacle avoidance. Fuzzy logic provides an ideal framework to merge notions from AI planning (symbolic representations) with notions from control (numeric form). Fuzzy behaviours are easy to define and provide a robustness with respect to sensor noise, modelling errors and imprecision [35]. The main behaviours used in our experiments are: “Go-Near” behaviour, which approaches an object to obtain a good odour sample; “Avoid” which performs obstacle avoidance; “Orient” used to orient the robot toward a particular object or corridor: “Follow-Corridor” which uses the sonar readings to remain centered within a corridor. The navigation planner is based on standard search techniques and decides what sequence of behaviours should be active to reach a specific target or goal. Plans are generated in the form of a set of “situation → behaviour” rules. These are called B-plans and are used by the top-level planner to formulate actions.
6.2.4
Anchoring
The anchoring module is an instantiation the anchoring processes described in Chapter 4 and besides creating and maintaining the anchor data structure it also serves a secondary purpose to function as a flag between top-level tasks given by a planner and low-level sensor data. Although not immediately evident this function is important in order to co-ordinate perceptual processes such as a smelling action which may take up to several minutes. In such a case, specific calls to perceptual actions e.g. “Smell Gar-22” are generated from the top-level planner, which then translates into several behaviours being activated e.g. “Go near to Gar-22, Touch Gar-22” and calls to an odour server which activates the respective pumps on valves on the e-nose. The odour classification provides the smell description e.g. “Gar-22 smells ethanol” and the perceptual information is updated. Here the anchoring module “polices” each event signalling to the respective modules when certain process need to be activated and when they have reached completion. In our instance of the anchoring module, the track functionality is achieved by performing a fuzzy matching algorithm between newly cre-
6.2. System Architecture
89
ated percepts and previously stored anchors in order to partly deal with sensor noise. However, the purpose of the anchoring framework allows different strategies for the track functionality. A part of the future aims to integrate more advanced solutions into the existing platform.
6.2.5
Planning on a Mobile Robot
The symbolic reasoning and decision-making component of Pippi is a planning system, PTLplan described in Chapter 5 [77, 78]. The planner receives top-level tasks from the user, but it also used for recovering from problematic situations, such as when the anchoring module fails to find an object corresponding to a description. The planning component has already been presented in Chapter 5.
6.2.6
Interfaces
In addition to the computations mentioned in the previous section for control, perception and autonomy, the system also has a number of processes for displaying the internal state of the robot as well as its current model of the external world. This model of the external world includes locally perceived objects and a gridmap of the environment built from sensor data. This is shown in Figure 6.6. In the LPS window different sensing modules can be selected, as well as tracing the movements of the robot. Furthermore, a live feed of the images viewed from the CCD camera is available and the robot can be either autonomously run or controlled through the cursor keys on the keyboard. A novel feature of the interface is an interface for olfactory data. The interface also elaborates on commands from a human into specific planning and sensing tasks on the robot. A snapshot of this interface is shown in Figure 6.5. The interface allows for different odour repositories to be loaded and stored, in the right most column. Each odour contains a signature, label lk , and a hidden field which represents its stored anchor to an object in the real world, if available1 . The classification process described in Section 6.2.2 is executed in the middle bar of the window. The process can be edited to adapt for cluster size, initialization algorithms, and logging outputs. As default the cluster size is set to the number of different names in the current repository. In the lower view a symbolic output of the clustering process is shown, where the clusters names are obtained from the contingency analysis. This gives explicit information about how the electronic nose categorizes the odours. Finally, a visualization of the clusters centers as well as the data points is shown in the upper left window (in 1 Some
odours signatures may be unanchored as shown in Section 6.4.4.
90
Chapter 6. Odour Discrimination for a Mobile Robot
2-dimensions). When a particular point on the data space is selected by clicking the mouse, a plan to visit the anchor that refers to the selected point is generated and the plan is executed. For example, objects which correspond to points located in clusters with high uncertainty (i.e. low density) can be re-sampled by the robot when selected by the user.
6.3
Experimental Setup
Experiments are conducted in a partially controlled environment at the ¨ Orebro University. Temperature and humidity are regulated by a conditioning system. A constant dominant airflow was not present any of the experiments, however, temporary increases in air currents may have been present to a small degree (caused by doors being opened or people moving in the environment). The odours used during experimentation are limited to a list taken primarily from an atlas of odour character profiles [34]. The atlas contains quality evaluations of a number of different reference substances evaluated by human panellists. In certain experiments, additional substances not in the profile list, namely ethanol and a commercial all-purpose cleaning fluid were also added to the odour repository. Two types of training methods were used in the collection of repository data. The first type of training method sampled from a sealed 50 ml container containing 20 ml of a test substance, this is called Headspace Sampling Method (HSM). The second training method used in cases of online acquisition of a training repository sampled objects directly containing traces of the odour in the open environment, this method we call Openspace Sampling Method (OSM). In OSM and all testing cases, the investigation of an object is done by moving the robot “close” to the object and taking an odour sample, in the next section the term close is empirically defined. Since errors in odometry and visual estimation of objects will propagate into positional errors, it is difficult to always sample at a fixed distance. All smellprints collected from OSM are also contained in specific anchors.
6.3.1
Effect of Distance on Classification
To determine a suitable range to collect odour samples, each substance was tested at varying distances from the robot. Figure 6.7 shows the reliability of classification as a function of the distance for each odour. At each of the marked distances, indicated by points in the graph, ten samples were collected, making a total of 60 samples for each odour. An important point worth highlighting is that even though the range
6.3. Experimental Setup
91
Figure 6.5: A screen shot of the olfactory interface used in the robotic system. On the two dimensional data space points can be selected by the user and plan to inspect the object whose anchor has a matching signature is generated and executed.
92
Chapter 6. Odour Discrimination for a Mobile Robot
Figure 6.6: The local perceptual space (LPS) of the Thinking Cap that displays the gridmap and objects detected from the image processing. The black dots represent the sonar readings, the shaded regions are regions that are not yet explored. In this image, two objects have been detected and are represented by rectangles and respective tags.
Effect of distance on classification with no wind present 100
90 1 80
Classification Rate (%)
70 Reliable Classification Threshold
60
2
50
40 3 30 4 20 Reliable Classification Distance
10
5
6 0
0
5
10
15
20 25 30 35 Distance from source (cm)
40
45
50
55
Figure 6.7: The effect of classification depending on the distance from a source in OSM sampling and HSM training. The results are shown for each odour where 1.(o) Ethanol 2. (∇) Octanol 3. (3) Linalool 4. (2) Hexanoic Acid 5. (*) cleaning agent 6. (+) Hexanal.
6.4. Experimental Scenarios
93
of sensors may be extended depending on the type of odour and the wind conditions, in classifying an odour character it is not enough for a sensor to simply exhibit a reaction. Rather for the reaction to be classified, certain properties need to be matched to the patterns in the training set. This is one major difference between odour classification and odourbased navigation where often small reactions from the sensors are enough to indicate the presence of an odour source. For odour classification it is important that the quality of the samples is maintained for both the training and testing sets. It is possible that the presence of strong wind will affect the range of classification and this affect depends upon the direction and speed of the wind. A strong wind also changes ambient temperature which may also have an effect on sensor performance. However, since the experimental room can be considered as windless, we accept as a design parameter that a classification rate of 70% is reliable and determine from Figure 6.7 that 20 cm is the maximum distance from the source that should be used to obtain this reliability.
6.4 6.4.1
Experimental Scenarios Acquisition of Training Data
Before any odours can be identified, training samples need to be collected. Bluntly said, this can be a tedious and monotonous process, especially in olfaction when the sampling of an odour including baseline, and purging cycles can take up to 2 minutes. An autonomous robot that could collect samples would be an attractive feature. However, expert knowledge is often needed to evaluate the results, identify anomalous readings, and in some cases actively select the next training sample. In the interface presented in the previous section, functionalities are available so that a human and the robot can co-operate to collect training samples. The question is whether classification improves using this process and can a robot conduct this process without the supervision of the human. To address these issues, an odour repository was built using odour samples collected over a period of 2 days. The experiment was repeated twice, in the first case a human user interacting with the system determined which samples to take and when. In the second case, the robot autonomously determines which samples to acquire based on a measure of the uncertainty in the existing training set. This measure, is based on the matching results, ui,k obtained from the classification algorithm. The next candidate sample is the k th sample with the lowest
94
Chapter 6. Odour Discrimination for a Mobile Robot Clustering Performance during Training Phase 1 human guided robot guided 0.98
Compactness and Separation
0.96
0.94
0.92
0.9
0.88
0.86
0.84
0.82
0
10
20
30 Sample Cycle
40
50
60
Figure 6.8: Clustering performance shown in the compactness and separation validity measure during the training of 4 different odour. Clusters improve quality both when the human guides the robot, and when it determines the next best sample autonomously.
ui,k (i.e. least membership to all clusters). The signatures of at least 4 different odours were contained in the repository, with n = 4 as a given parameter. Through each sampling cycle, the validity measures were evaluated where a total of 60 sampling cycles were performed. Each sampling cycle contains 5 samples of each odour. During the experiment, the coefficient of uncertainty averaged 0.97 ± 0.03, meaning that the categories from the human domain maintained correlation with the clusters in the electronic domain. Figure 6.8 shows the graphs of the Compactness and Separation measure mentioned in Section 6.2.2. Smaller values of Compactness and Separation measure indicates good clusters (dense, compact and wellseparated). When either the user or robot determine the next best sample, the global properties of the clusters improve as the number of samples increase. This result indicates that a robot can be used to collect the training samples for odours, and that this process can be adaptive based on the properties of the sensor data.
6.4. Experimental Scenarios Number of Odours 2 3 4 5
Table 6.1: Experimental results from Ambiguous Cases.
6.4.2
Disambiguation of Visual Similar Objects using Olfaction
To further test the capability of the planner to reason about sensing resources, an ambiguous situation containing visually similar objects is used. In this experiment, a user requests the robot to find an object given a list of properties. The properties are symbolic representation of perceptual concepts (e.g. green cup smelling coffee). The objects are first seen using vision but are only distinguishable by their olfactory property. The planner therefore creates a plan that determines how each of the objects are visited and when the smell action is executed. The following experiment shows the results when several visually similar objects are placed in a room in a variety of configurations. Up to 5 different objects are tested, where the only discriminatory feature between them is their odour character. The results from the runs are shown in Table 6.1. There are cases in which the e-nose misclassifies the odour independently of visual failures. These misclassification errors slightly increase when the number of different odours increase. The source of this error can be the e-nose’s inability to discriminate between classes of odours. However, in our case the error was actually due to the sensing parameters given in the sampling process. In particular, when two cups were separated by a short distance there was an inadequate recovery time between “sniffs” and this resulted in a misclassification of samples. This recovery time depends on the type of odour being sampled. However, the majority of olfactory failures are a direct result of visual and odometry errors. This demonstrates that in cases of odour classification it is important that the robot be correctly placed in relation to the odour source before taking smelling actions. Therefore the errors from one modality may propagate into another modality.
96
6.4.3
Chapter 6. Odour Discrimination for a Mobile Robot
Reacquiring Objects
In real situations, it is difficult to predict which substances the e-nose will need to classify. Therefore, to be able to adapt to new odours, the e-nose needs to be able to learn online and update its odour repository. To illustrate how this is possible, a series of experiments are done using the reacquire functionality presented in the anchoring module. In these scenarios, one cup is located on the floor at least 1.5m from Pippi’s starting position. Pippi is given the task to acquire as much information as possible about this specific cup. The robot acquires the colour, shape, size, position and odour character. For the olfactory property, an initial inspection of an object always includes more than one odour sample of that object. In this scenario, the planner was used to evaluate whether olfactory actions were necessary for reacquisition. For example, if the object could be reacquired based on position or visual information alone, then olfactory module was not used. However, since odometry errors would accumulate and additional objects were placed in the environment, reacquisition of an object would require the reacquisition of its olfactory property to reduce uncertainty. A total of 50 experimental runs were made using a repository containing six odours. For online learning, training data is obtained through OSM, which means that the robot may be placed anywhere close to the object when obtaining the olfactory information. In these experiments, only cups were used and it is assumed that a good quality sample can be obtained if approaching from any angle. However, in the case of more awkward objects the strategy presented in the suitcase scenario in Chapter 5, could be used. The task to reacquire an object was achieved in 90% of the trials. Thus showing that the training set obtained in an online and openspace context could effectively be used to identify new samples. To further evaluate the ability to collect good training data in an online context, we compare the quality of the data set to that obtained through HSM training. This is done by examining the quality of each odour cluster. Table 6.2 compares the average Euclidean distance of the total exposures (or samples) of an odour in the training set to the centroid of its class. The mean indicates how well the centroid is placed in relation to its members, however, the value of interest is the variance, s2 , which indicates how compact the cluster may be. To compare if the variance for an odours’ training set increases depending on OSM or HSM, an F-test statistic is effected. The last column of the table is the result when given the null hypothesis that the two variances are equal with 5% confidence. For four of the six odours training using the OSM and HSM did not
Table 6.2: Average euclidean distances of the total samples for an odour depending on training methods.
affect the quality of the data set. For Ethanol and Hexanal, however, this was not the case. At least for the Ethanol one reason for this could be its high volatility in OSM compared to HSM when the substance is contained in a sealed headspace. For these substances it is important that the training of the sensor data be consistent to the conditions used in identifying an unknown substance.
6.4.4
Finding Odours by Sample
The task is to acquire the odour of an object online, scan the environment for possible object candidates that may also have the same odour property and then to investigate each candidate until a match is found. The robot learns the odour by first requesting a smell, whereupon a human provides the robot with a sample (see Figure 6.10 Left ). The response vector from the odour signature is stored in the repository. In these cases, the symbolic name of the odour is not needed and therefore an arbitrary name is assigned by the anchoring module. Once stored, the robot proceeds into a room where the candidate objects are located. A scan behaviour is invoked and a number of objects are recognized by the vision system and their anchors are stored in the local perceptual space. In these experiments, the robot is able to recognize suitcases, balls, cups, and gasbottles. To determine which objects to visit, a plan is generated that first goes near each of the objects and then sends a request for the smell action. As soon as a matching object is found, Pippi moves away from the object one last time and performs no further action. An example of this experiment is shown in Figure 6.9 and Figure 6.10. A shirt soaked in cleaning agent is used as a target odour. The robot is then asked to enter a room and find an object with a matching odour. In Figure 6.9(Left) the robot enters a room and scans for all
98
Chapter 6. Odour Discrimination for a Mobile Robot
Figure 6.9: (Left) Finding odours by sample where a newly acquired smell is associated to an object in the room. Here the robot must determine which objects to sample and in what order they should be visited. (Right) The local perceptual space window shows the detected objects: two boxes, two whole gasbottles and one exploded bottle.
Figure 6.10: (Left) The online acquisition of an odour. (Right) The trace of the robot’s movement during the search for an object of a particular odour.
6.4. Experimental Scenarios
99
possible candidate objects. She recognizes 5 objects: two box-like objects and three gasbottle objects, which are shown on the corresponding LPS (Figure 6.9 Right ). She then makes a plan to visit each object and take an odour sample, until a match is found. A trace of the robot’s movements is shown in Figure 6.10 (Right). The robot’s actions are non-deterministic since the order in which each of the objects are visited depends on the order in which they are represented in the planning tree, usually the alphabetical order of the anchors. This is a point to improve, so as to find an optimum path to visit the objects, i.e. closer objects are visited first. In the trace of the movements it appears that the robot returns to the center of the room after each “sniff”. This behaviour is actually a consequence of the smelling action: once an odour is sampled Pippi moves away from the object in order to obtain a better approach to new objects. Of 27 trials, where a candidate object was present, Pippi found the object in 74% of the trials. Failures in this case were caused by the detection of mirage objects (18.5 %), misclassification of the target object’s odour (11.1%) and misdetection of objects in general (11.1%)2 . Throughout the experiments, the number and position of candidate objects were varied, as well as the starting position of the robot.
6.4.5
Patrolling and Inspecting an Environment Over Time
The aim of the last experiment is to demonstrate that the system can work in the context of a more complex and realistic environment for an extended period of time. The experiments test the system’s ability to collect both visually and olfactory information relating to objects. Also an odour classification with a variety of different odours is made with a collection of odour samples taken over a period time. The task executed in the experiment is to patrol an office environment and inspect objects. A user is able to monitor the robot, suggest actions to improve perceptual performance and request the robot to perform specific tasks mentioned in the previous section. Tasks are requested from the user using a symbolic representation consisting of natural language concepts. While no requests by the user are given, the robot autonomously prioritises tasks alternating between patrolling the corridor and inspecting objects or simply waiting on stand-by. 2 In certain trials candidates objects were missed and phantom were detected thus given an overlap in percentage errors.
100
Chapter 6. Odour Discrimination for a Mobile Robot
Room T1203
WC
Corridor 1
Room T1209
Room T1210 Conference
storage
x
Corridor 2
Room T1201
Corridor 3
Room T1204
Room T1206
Figure 6.11: The a priori map of the corridor which the robot navigates and where target objects are located. The robot starting position is shown at point X.
6.4. Experimental Scenarios
101
Environment ¨ All experiments were performed in the corridors at Orebro University. Temperature is regulated and maintained at approximately 20 ◦ C with a 65% RH and no dominant constant airflow was present. The experimental environment is shown in Figure 6.11. The workspace and objects to be inspected are located in the three main corridors shown in the map. This rough map is also provided to the system and contains the approximate position of all the doors, rooms, and corridors plus the topological information about their connectivity. It also contains a few metric properties such as the width of the corridors, and doors. Odour sources were dispersed at different locations in the three corridors. All odours sources were contained in green trash bins. The chemical substance was placed on a cloth-like material and placed inside each bin. Each odour source was separated by a 1.5 meter radial distance to minimize the effect of other source interference during sampling. Odour samples were refreshed at regular intervals approximately every 30 minutes. Classification Performance Over the four day period, over 75 odour samples were collected. Five different odours were used, Hexanal 98% (C6 H12 O), Linalool 98% (C10 H18 O), Ethanol (C2 H6 O), cleaning agent (H3 N) (common cleaning fluid) and Clean Air. The choice of this particular selection of odours was based primarily on availability. The entire odour set was clustered in an offline manner at the end of the experiment in order to evaluate the system’s classification performance and is shown in Figure 6.12. The clustering algorithm described in section 6.2.2, was applied using an unsupervised algorithm on half of the data set. The most appropriate number of clusters according to the sensor data was determined through validity measure analysis, in Figure 6.12 (Top). From the analysis, seven clusters yielded optimum partitioning and the labelling strategy generated clusters with labels given in Table 6.3. Notice that three clusters obtain the same label of Ethanol which may be due to a sensor drift, however this does not affect the classification performance on unknown samples. Classification performance on the second half of the training set yielded the results in Table 6.4.5 for each odour. The classification performance of the Hexanal and Linalool are due to samples which spread across clusters 4 and 7 , Figure 6.12 (Right ). To further inspect the odour analysis as the days varied, classification
102
Chapter 6. Odour Discrimination for a Mobile Robot
−3
4
Global Validity Analysis for Partition Determination
x 10
3.5
validity measure
3
2.5
2
1.5
1
2
3
4
5
6
7
8
9
prototypes
Figure 6.12: (Top)The validity analysis yielding optimum partitioning with seven clusters. (Bottom) The unsupervised clustering result of the data set collected over the 4 day run.
Table 6.4: Classification Performance during the Corridor Inspection.
process is repeated. This time each odour is labelled according to its character (e.g., ethanol) and time stamp. Figure 6.13 (Top), shows that the three Ethanol clusters can be decomposed according to the day it was sampled. This result can be either attributed to the fact that the Ethanol evaporates quickly or to a possible sensor drift. A similar pattern is exhibited in the Clean Air cluster but with a lesser degree and a in a different manner, Figure 6.13 (Bottom). This is a further area to consider. Although, not addressed here, a possibility for future works may be to include drift compensation in the classification algorithms [66, 111]. Task Execution During the Corridor Monitoring Three of the tasks described in the previous sections, finding an object by sample, disambiguating between objects and thorough inspection of an object (Chapter 5), was performed 20 times throughout different patrolling episodes. Table 6.5 Column 1, shows the performance success of the task as a singular task. Column 2 of the table shows performance success of the tasks when integrated into the corridor monitoring system. For Task 1, a success is considered when a matching object to the odour sample is found. For Task 2, a success is considered when a single object
104
Chapter 6. Odour Discrimination for a Mobile Robot
Figure 6.13: (Top) Enlarged image of Ethanol samples which show drift in the sensor data. (Bottom) Enlarged image of the clean air samples.
6.5. Summary and Conclusions
105
is correctly classified after thorough inspection. For Task 3, a success is registered when a correct object is found given a symbolic description in the presence of 5 visually identical objects. The performance of the task executed in the context of a corridor monitor is similar to the performance of the tasks executed individually. There is a correlation between successful performance and the average distance from an object when taking an odour sample. The further away we are from the odour source the less likely the odour is correctly identified. In the first task, the performance results of the individual task vs. the task performed in the corridor monitor are comparable. Meanwhile performing a thorough inspection of objects in the context of the larger experiments give a slightly worse performance compared to the task performed individually. The major source of error is an accumulation of odometric errors which increases in the corridor monitor experiments because of the extra distance required to navigate to the object. When re-approaching an object to take an additional sample, sometime the object is not seen again and its positional information in the anchor is not updated. The result is an accumulating odometric error for each new approach to the object resulting in a higher probability to fail. The same source of error is also responsible for the performance difference result in the third task. Both experiments demonstrate the ability to perform multiple tasks on a robotic system without the explicit need to fine tune the system for each task. The olfactory module has been integrated into the system in such a manner as to facilitate the collection of odours online and the execution of a variety of different olfactory tasks. These tasks are independent of one another yet rely on the same classification of odours and representation of objects. We have presented three specific tasks, however, the system has been constructed so that new tasks can be easily included.
6.5
Summary and Conclusions
The contribution of this work has been a mobile system equipped with an electronic nose and other sensing modalities that can detect and recognize objects based on olfactory and spatial properties. The implementation used an e-nose as an active sensor, that was explicitly called upon when smelling actions were required. The symbolic component included a planner, which used a symbolic model of the robot, its sensors and the environment in order to decide when and how to act. There is also an anchoring module which maintained the correspondence between symbols
106
Chapter 6. Odour Discrimination for a Mobile Robot
Table 6.5: Comparison of performance results from single tasks vs. tasks in a long term scenario.
denoting objects and the sensor data referring to those objects. The experimental results showed that a mobile robot could successfully use olfaction to discriminate between odours in a variety of different scenarios that detected objects. In the first case, we showed that the odour property could be used to enhance the current perceptual abilities of a robot, in that objects that were visually identical could be discriminated by smell. In the second scenario, we showed that the planner together with the electronic nose could be used to make inspection of an object, combining the uncertainties so as to perform an adequate amount of perceptual actions. This scenario also showed that by planning to move the robot in an appropriate manner better sensing actions could be performed. The last two scenarios presented cases where Pippi needed to acquire knowledge of odours online. One case investigated the possibility of using the robot much like a search dog with an affinity to inspect visually detectable objects. The other case, examined the reacquisition of objects whose odour was previously perceived. The ability to reacquire objects is an important feature especially for mobile robots that move in large areas and dynamic environments. We showed that by integrating electronic nose with other modalities a sensing strategy for odour discrimination could be used to identify the odour character associated with objects. This strategy involved vision to navigate to a potential odour source and taken the odour sample when close to an object. For the scenarios presented, this strategy was able to generate approximately an 80% accuracy in classification of new odours using a library of at least 6 different odours. To our knowledge, the work presented here is a first attempt to discriminate a larger set of odours using a mobile robot. Future work will attempt to address a number of interesting issues that arose during experimentation. One such issue contends with the methods used for collecting traning data. It could
6.5. Summary and Conclusions
107
be of benefit to employ methods from olfaction community and consider the dynamic response of the signal as we approach the odour source for the collection of training samples. Other open problems deal with the degradation of odours over time. The development that we expect is that electronic olfaction will grow to be a standard equipment on many mobile robotic systems, from field robots to humanoids robots [125]. Much in the same way as the computer vision community has developed different techniques and algorithms suited for mobile robots, an equal development is needed in electronic olfaction for smelling robots. This development should focus on all the aspects of artificial olfaction including the discrimination of objects based on their olfactory property, the detection and classification of odours unperceivable in the human domain, and the communication of an odour characteristic to intelligent systems, human users or other robotic units.
108
Chapter 6. Odour Discrimination for a Mobile Robot
Chapter 7
An Ecological Approach to Odour Recognition In this chapter we present a new approach for odour detection and recognition by integrating a network of gas sensors and a mobile robot in an intelligent environment. The environment can provide information regarding the location of potential odour sources, which is then relayed to a mobile robot equipped with an electronic nose. The robot can then perform a more thorough analysis of the odour character. This is a novel approach which alleviates some of the challenges in mobile olfaction techniques by single and embedded mobile robots. The environment also provides contextual information which can be used to constrain the learning of odours, which is shown to improve classification performance. As was shown in the previous chapter, mobile robots equipped with artificial olfaction capabilities have a great application potential in a wide spectrum of domains. An olfaction-capable robot could detect hazardous or anomalous situations in private, public, or industrial environments; it could help in risk assessment and victim search during rescue operations; it could be used for medical monitoring in a home or retirement house; and so on. In spite of this great potential, the integration of olfaction in a mobile robot is hindered by a number of technical difficulties. First, current devices for artificial olfaction are only able to perform reliable odour detection and classification when the concentration of odour molecules in the air is high. Thus, an interesting odour event would be missed unless the e-nose is close to the odour source. Second, today’s e-noses typically have a very slow reaction time. This, together with the complex nature of odour propagation, makes it very difficult for a mobile robot to correctly 109
110
Chapter 7. An Ecological Approach to Odour Recognition
localize and approach an odour source. Finally, odour classification can only be done reliably if the set of classes is very small. This limits the applicability of current e-noses to cases in which we have some a-priori information of the type of substance which is being smelled. Because of these limitations, the possibility of having a general olfactionenabled robot in our homes, which can detect and classify abnormal events everywhere in the home, is not realistic today. In this chapter, we explore a different road to approach this problem. Instead of assuming an all-mighty olfactory robot embedded in a passive environment, we assume a smart environment which contains a number of sensors and tagged objects, and which can provide the robot with the needed information. This approach is an instance of the general concept of PeisEcology (pronounced “piece”), which integrates autonomous robotics and intelligent environments. Here, we assume that the environment contains a number of very simple gas sensors placed at critical locations (e.g., inside the fridge, or near the cooker). These simple devices can detect an abnormal gas concentration, but they are unable to classify the type of odour. We also assume that objects in the environment (e.g., goods in the fridge) can have tags attached, which contain information about the object itself.1 When a simple gas sensor detects an alarm, its location is sent to a mobile robot equipped with a sophisticated e-nose. The robot navigates to that place, and smells the different objects there. The information stored in the object’s tags provides a context to restrict the classification problem. We begin our discussion with a presentation of more detail of the Peis-Ecology approach. We then describe an experimental system for odour recognition in a Peis-Ecology framework, and show a few experiments that demonstrate the viability of our approach. The Peis ecology described here is a joint effort work that has been done in cooperation ¨ with the Mobile Robotics Lab at Orebro University and the Electronics and Telecommunications Research Institute (ETRI) in Korea. Our contribution to this project has been the integration of olfaction in the Peis-Ecology. It is worth to note that its components are in the preliminary stages of its development. Nonetheless, the experimental results presented in this chapter present a potential application area for electronic olfaction which to our knowledge has not yet been explored. 1 This assumption is not unrealistic: for example, many consumer goods already have an RFID tag attached to them, and this practice is expected to extend to most goods in the next few years.
7.1. The Peis-Ecology Approach
7.1
111
The Peis-Ecology Approach
The concept of Peis-Ecology puts together insights from the fields of autonomous robotics and ambient intelligence [71] to generate a radically new approach to building assistive, personal, and service robots. The main constituent of a Peis-Ecology is a physically embedded intelligent system, or Peis. This is any computerized system interacting with the environment through sensors and/or actuators and including some degree of “intelligence”. A Peis can be as simple as a smart toaster and as complex as a humanoid robot. What differentiates a Peis-Ecology from other approaches in ambient intelligence is that emphasis is also placed on the performance of physical tasks rather than only on the exchange of information. A Peis-Ecology fuses together inspiration from three main fields: robotics, artificial intelligence and ubiquitous computing to incorporate the perception and manipulation of objects by communicating between sub-systems in the environment. Individual Peis in a Peis-Ecology can co-operate based on the notion of linking functional components: each Peis can use functionalities from other Peis in the ecology in order to compensate or to complement its own. The power of the Peis-Ecology does not come from the individual power of its constituent Peis, but it emerges from their ability to interact and cooperate. As an example, a robot which needs to grasp a bottle would not use its sensors to detect its position, shape, and weight — a task which proved to be surprisingly difficult in years of robotic research [109]. Instead, the bottle itself, enriched with a micro-Peis, would hold this information and communicate it to the robot. This idea of integrating robots and smart environments is still relatively unexplored [113, 143, 82]. The Peis-Ecology approach can help us to address the challenges of robot olfaction discussed above. To detect distant odours, the olfactory robot could rely on the pervasive sensing functionality provided by a network of simple Peis, each including a simple gas sensor, which are distributed at tactical locations in the environment. The same network of Peis would provide the mobile robot (itself a Peis) with location information about the potential odour source. When an odour alarm is generated, then the mobile robot, equipped with a much more sophisticated e-nose component, can move to that location and investigate the nature of the odour. The Peis-Ecology approach can also address the problem of providing domain information for classification. Each tagged object that needs to be investigated can be seen as a Peis in the ecology,
112
Chapter 7. An Ecological Approach to Odour Recognition
Figure 7.1: An example of olfaction in a Peis-Ecology. The gas sensors provide detection and location of candidate odour alarms, the objects provide classification context. and provide its identity to the mobile robot.2 This identity can be used as a context to load the correct training set from a large odour repository, thereby increasing the probability of correct classification. This Peis-Ecology is schematically illustrated in Figure 7.1. Compared to the tradition in electronic olfaction, the Peis-Ecology approach includes two interesting innovative aspects. First, this approach reverses the tradition in electronic olfaction by placing the recognition and identification on a mobile platform and the simple gas sensor in static locations in the environment. This is a cost effective strategy considering that the simple sensor are cheap (5-20 USD per sensor) and small and can be easily distributed where as an electronic nose device are often more costly. Second, this approach allows the introduction of context in olfaction.
7.2
A Peis-Ecology for Olfaction
We now proceed to describe a concrete system that applies the PeisEcology framework to the problem of mobile olfaction. The resulting system is able to perform detection and identification of a variety of 2 In the actual experiment described later, the fridge contains a RFID tag reader that can determine the identity of the (tagged) objects in it. Hence, the fridge, not the objects, is a Peis.
7.2. A Peis-Ecology for Olfaction
113
different odours in a large environment. In this section, we describe: the experimental platform used to implement and test Peis-Ecologies; the specific Peis-components used in our experiment; and the way in which they cooperate to solve our specific problem.
7.2.1
The Peis-platform
We have constructed a reference implementation of a Peis-Ecology, complete with olfactory capabilities, as well as built a demonstrator environment in which these tests can be performed. The basic building blocks of our Peis-Ecology is a “Peis-component”, that is, a logical process that implements an individual functionality inside a Peis. Some examples of individual functionalities are the software components that implement robotic algorithms for control, image processing, and planning. In order for these Peis’s to cooperate, the Peis-platform implements a distributed communication mechanism which allows the addition and removal of components from the ecology as needed. The main communication and synchronization mechanism is a distributed tuple-space enriched with an event mechanism. This is a generative communication method, which simplifies the dynamic reconfiguration of the Peis-Ecology. This idea is inspired by recent developments of the semantic web initiative [144], and allows the use of components whose responsibility is coordinating the functionalities of other components, using AI techniques for creating new configurations [102]. Tuple-spaces and events are increasingly used in ambient intelligence systems [8, 134], in sensor networks [2, 64] and autonomous robots [21]. To allow for more efficient and simple implementations of the distributed tuple space, all tuples are restricted to be of the form:
where peisID and compID are uniquely assigned to every Peis and every component inside it, and key is the tuple key. This convention simplifies the resolution of conflicts for multiple producers of the same key. The tuple-space is endowed with the usual insert and read operations [151]. In addition, we define event-based primitives subscribe and unsubscribe, by which a Peis can signal its interest in a given key. When an insert operation is performed, all subscribers are notified. Management of subscription and notifications is done by the middleware called the Peis-kernel. The middle-ware provides other important services like network discovery and routing of messages between
114
Chapter 7. An Ecological Approach to Odour Recognition
Peis lacking direct means of communications. To maintain a decentralized and distributed tuple-space, this library implements an ad-hoc P2P network using any available communication links between participating Peis. The Peis-kernel handles all communication issues by automatically discovering other running instances of itself on the local networks, by handling arbitrary types of networks, and by providing independences from the network type (TCP/IP, serial links, etc.). For more detail on the Peis-kernel and the general concept of the Peis ecology readers are referred to [128, 19].
7.2.2
TestBed
A physical test-bed facility, called the Peis-home, which looks like a typical bachelor apartment of about 25m2 is used to test the Peis-Ecology. It consists of a common room, a bedroom and a small kitchen. The PeisHome is equipped with communication and computation infrastructure, and with a number of sensors. The walls have been constructed to have a height of 1.40m so that observers can get a bird’s eye of the entire apartment. The ceiling has been lowered to standard height of 2.40m: the space above the ceiling is used for cables, computing equipment, and wireless access points. Four web-cameras have been mounted on the ceiling such that they cover the common room and the kitchen. The intention is to use the web cameras for localization [92], however, this functionality is not available in the current implementation. Figure 7.2 shows a schematic of the Peis-Home with a few snapshots of individual rooms. Beside the main apartment there is a “control deck”, a small elevated area from which the experimenters and visitors can observe and control the behaviour of the full PEIS-ecology in the PEIS-home.
7.2.3
The Refrigerator Peis
The refrigerator Peis consists of a apartment sized refrigerator (51cm × 53cm × 81cm), two simple TGS 2600 Figaro gas sensors (sensitive to air contaminants) and a laptop computer with data acquisition technology [44]. Under current experimental conditions the fridge is turned off and the inside temperature is approximately 20.5◦ C. Milk products used during the experiments are placed in the lower shelf of the fridge as shown in Figure 7.3 (left). One sensor is placed in the shelf door (marked Sensor A in the figure) and the other sensor is placed on the roof of the fridge (Sensor B).
7.2. A Peis-Ecology for Olfaction
115
Workbench
Bedroom
Kitchen
Common Room
doo r
Control Deck
Figure 7.2: (Upper Left) A rough sketch of the Peis-Home. (Upper Right) A picture of the control deck. (Lower Left) A picture of the kitchen with a fridge located under the workbench. (Lower Right) A picture of the common room.
116
Chapter 7. An Ecological Approach to Odour Recognition
Figure 7.3: (Left) The Peis fridge showing the placement of the two gas sensors and the products to be sampled (Right) The door mechanism which opens and closes the fridge door. Sampling both sensors at a rate of 4 Hz an alarm is triggered when the concentration gradient exceeds a pre-defined threshold ε for both sensors, see Figure 7.4 for an example the sensor data when an alarm is triggered. The Peis-fridge also incorporates an RFID tag reader (Texas Instruments IDISC MR 100). Each object stored in the fridge has an RFID tag attached, which stores the object’s unique ID together with other optional properties (not used in this experiment). The Peis-fridge inspects its contents and publishes it on the tuple-space at a fixed rate. All the communications between the fridge Peis and the other Peis in the ecology happens by exchanging tuples through the distributed tuplespace. In particular, the fridge produces tuples of type SmellAlarm, containing the gas sensor data, and of type Contents, containing the ID’s of all the objects inside the fridge. It responds to tuples of type DoorCommand, carrying requests to open or close the door.
7.2.4
The Mobile Robot Peis
The second Peis in our ecology is an iRobot’s Magellan Pro indoor research robot called Pippi that was described in chapter 6. Recall that in addition to the usual sensors, Pippi is equipped with a CCD color camera and with a Cyranose 320 electronic nose used to identify and discrimi-
7.2. A Peis-Ecology for Olfaction
117
Figure 7.4: Response from the two TGS 2600 sensors in the fridge when an alarm is triggered.
118
Chapter 7. An Ecological Approach to Odour Recognition
nate between odours. Also recall, the sensors on Pippi are mounted in such a way that line of sight and line of “smell” are coordinated. This is done by placing the e-nose below the camera and odour samples are drawn from a uni-directional air-flow at the front of the robot. The detection of an odour is therefore maximized when the robot is directly facing the source. and with a wireless network. When sampling an odour using the Cyranose 320 a three phase sampling technique is used, similar to that presented in Chapter 6. Recall that the first phase, is a baseline collection where the sensors are exposed to a reference gas. The reference gas is contained in a 50 ml vile and consists of clean air that was prepared and collected in a ventilation cupboard. The vile is placed and fixed onto the mobile robot. In these experiments, the baseline is collected for 30 seconds. The second phase is a sampling phase where odour samples are drawn into the e-nose unit and come in contact with the sensing array. The odour is sampled for 60 seconds. Finally, the third phase is a recovery where the sensing unit is purged of any reminiscent gases, this phase typically lasts for 60 seconds however, can be extended if the sensors do not exhibit a full recovery (i.e., return to approximate baseline values). Sensor data pre-processing involves: (1) a fractional baseline manipulation, where the baseline is subtracted and then divided from the sensor response; (2) local vector normalization, where the feature vector of each individual sniff is divided by its norm and lies on a hyper-sphere of unit radius; (3) global auto-scaling in which the distribution of values for each sensor across the entire database is set to have zero mean and unit standard deviation. Identification of new odours is performed using a classification component, which uses the raw sensor data provided by the e-nose together with domain information and provides classified data. For classification of an unknown odour, a statistical classifier located on-board Pippi is used. On-board Pippi also runs an instance of the Thinking Cap, an architecture for autonomous robot control based on fuzzy logic [129], and an instance of the player program [120], which provides a low-level interface between the robot’s sensors and actuators and the Peis-Ecology’s tuple-space. The planning and anchoring components described in Chapter 6 are no longer present on the robot. Instead deliberation is done by the Home Security Monitor Peis described below. Pippi responds to tuples providing commands and requests. These include tuples of type Goal, providing navigation goals, and of type Smell, providing smell commands. Pippi produces tuples that indicate the state of the navigation and the olfactory classification results.
7.2. A Peis-Ecology for Olfaction
7.2.5
119
The Home Security Monitor Peis
This is a Peis which consists of a static computer connected to a set of web-cameras mounted on the ceiling. In addition to other monitoring tasks, not relevant here, this Peis contains a high-level component responsible for detecting possible problems using the available olfactory resources: the simple gas sensors which are present in the fridge-Peis and possibly in other devices, and the electronic nose on Pippi. This component reacts to alarms published into the tuple-space by the simple gas sensing components. When an alarm is received, it uses a pre-compiled plan to send Pippi to the location of the alarm, and to perform an olfactory analysis.
7.2.6
Other Peis
In addition to the Peis described above, the Peis-Home contains several other Peis that can participate in the ecology. These include other mobile robots, a global localization system based on the information provided by the ceiling web-cameras, and an “observation” Peis used by the experimenter to observe and control the behaviour of the whole PeisEcology. These components are not relevant to our olfaction ecology, and they will not be described further.
7.2.7
Peis-Ecology Configuration
In our olfactory Peis-Ecology, we have connected the above Peis and their components as shown in Figure 7.5. The Peis-fridge provides olfactory alarms to the deliberator in the Home Security Monitor Peis, and it provides information about the fridge’s contents as context to the odour classifier inside Pippi. Pippi’s navigation component (Thinking Cap) receives navigation goals from the Home Safety Monitor, and returns execution states and odour classifications. Additional gas sensors, not considered here, could of course be distributed in the environment at other locations in order to extend the monitored area. In practice, the above configuration is generated automatically by the Home Security Monitor by informing each Peis-component of which tuples it should use as input and output. In general, the configuration of a Peis-Ecology can change dynamically depending on the context. For instance, when Pippi enters an area covered by the external localization system, it may switch to using the global localization functionality provided by that system instead of its own self-localization.
120
Chapter 7. An Ecological Approach to Odour Recognition Home Security Monitor
Pippi
Deliberator Classifier
Other components not relevant for this experiment
Thinking Cap
Cyranose Player
Fridge
Environment Gas sensor
RFID reader
Figure 7.5: The Peis, Peis-components, and connections used in our olfactory Peis-Ecology. For simplicity the components not relevant to the olfaction experiment have been omitted.
7.3
Experiments
Two of the main difficulties in odour classification using a mobile robot are (1) in order to obtain good classification of an odour it would be desirable to sample that odour close to the source [98] (2) good classification performance can usually be achieved if we strongly restrict the number of odour categories, but this requires to have some a-priori information about the odour source available. In this section, we show that our Peis-Ecology can help us to overcome these difficulties by: (1) using the location information provided by the distributed gas sensors to navigate to the odour source, and (2) using the identity information provided by the RFID-tagged objects to restrict the classification context. All the experiments reported here have been performed using the Peis-Ecology described in the previous section, and placing milk cartoons in the Peis-fridge.
7.3. Experiments
7.3.1
121
Approaching the Odour Source
This part of the experiment simply shows the viability of the PeisEcology approach to bring the e-nose close to the odour source. At start-up, Pippi started at a resting position in the living room. After some time, the gas sensors in the fridge triggered an alarms, and a tuple of type fridgeAlarm was placed in the tuple-space. This was notified to the deliberator component inside the Home Security Monitor Peis, thus triggering the generation of an investigation plan. This plan was executed by first sending to the Thinking Cap component in Pippi the goal tuple . When the Pippi had moved in front of the fridge, thus achieving this goal, it announced success by publishing the corresponding tuple. The deliberator then activated the electronic nose by publishing another tuple, and waited for the classification results. If these results signified no problem, the deliberator asked Pippi to return to its starting position and resumed its monitoring activity. In all other cases, the deliberator would ask Pippi to move to the bedroom to alert the occupants of the Peis-Home.
7.3.2
Context-dependent Training and Testing
In the second part of our experiment, we quantify the improvement in the classification performance obtained by using information about objects to provide a context for odour classification. To evaluate how the contextual information affects classification, the training results of a classifier using two different repositories were compared. The first repository used is shown in the first column of Table 7.1. This repository is not filtered according to the context and contains samples of a wide range of odours. The second repository which is a subset of the first contains samples that refer to the objects located in the fridge. The RFID reader in the fridge is able to detect which objects are present and publish their ID and information. In each of the repositories, each record or “odour-print” represents an individual sample of an odour. Each sample consists of: An odour signature or response vector Rk where each sensor’s reaction is represented by: Rk,s =
Rmax,s − Ro,s Ro,s
(7.1)
for s=1 to 32 sensors. Rmax,s is the maximum possible signal strength and Ro,s is the calibrated baseline response for odour o.
122
Chapter 7. An Ecological Approach to Odour Recognition A linguistic label lk , where lk ∈ D, a user specific discrete set. For example D may contain the collection of odour names (e.g. Ethanol, Hexanal, Milk).
A minimum of 10 samples were made for each odour used in each of the repositories. To validate context dependent training three different classification techniques were used in the classification module: K-means, KNN (with K=5 for both repositories), and canonical discriminant analysis. The results from the cross-validation are given in Table 7.2. For KNN and K-means cross validation is performed by removing one exposure at a time. For canonical discriminant analysis, cross validation is performed by removing one exposure from each class. All the exposures are left out once during the cross validation procedure. The partial training set remaining and the removed exposures are scaled. For all three techniques, better classification performance was achieved when contextual information was used. Notice that classification performance is greater for the canonical discriminant analysis, this is most likely due to the fact that the canonical analysis is using Mahalonobis distance to determine inter-class distance relationships, whereas KNN and K-means use the Euclidean distance. The Mahalonobis distance is said to be superior measure since it takes the distribution of the points into account [121]. When using canonical discriminant analysis to classify new samples, the sample with the shortest Mahalonobis distance between its centroid and the sample in the canonical space is used to determine the class of the new sample. Table 7.3 gives the discrimination results for varying qualities of milk. The amount of unknown samples classified is given as well as the success of classification in percentage for the first repository (no-context) in Column 3 and for the contexted repository in Column 4. Since it was difficult to get fresh samples of milk to match the exact temperatures given in the training set, two temperature ranges were used. Milk at cold temperatures is considered to be less than 15◦ C and above 10◦ C and milk at room temperatures is above 15◦ C and below 20.5◦C. Samples which do not fit in these ranges are not used in either the training or testing data sets in order to reduce the effect of temperature on the gas sensor reactions. Notice that the identification performance of Milk Quality-Good at Room temperature is low when using Repository A. The cause for this is a misclassification with other “mild” odours such as the clean air and the vanillin. However, by using context this classification result is greatly improved.
7.3. Experiments
123
Table 7.1: List of substances in each repository. Repository A (No-Context)
Table 7.2: Cross-validation results from non-context and context dependent training. Classification Method
No Context (%)
Context (%)
K-means
79.3
92.5
KNN (5)
89.5
96.2
Canonical
92.5
94.8
124
Chapter 7. An Ecological Approach to Odour Recognition
Table 7.3: Identification of unknown samples of milk quality. Quality
Samples
Repository A
Repository B
Quality-Good (Cold)
20
60 %
90 %
Quality-Good (Room)
18
45 %
90 %
Quality-Bad (Cold)
20
70 %
85 %
Quality-Bad (Room)
20
85 %
100 %
7.4
Discussion
The main result of this chapter is to show that we can use the concept of Peis-Ecology to provide a new and viable solution to the problem of mobile olfaction. The Peis-Ecology approach by-passes some of the difficult problems of mobile olfaction by exploiting the cooperation between the robot and an augmented environment. Although the scenario that we have considered concerns the monitoring of a domestic environment, it is easy to imagine a number of other applications for our approach, ranging from environmental monitoring in a factory with dangerous chemicals, to monitoring the health conditions of people in a hospital or retirement house. It should be noted that just using a network of artificial noses distributed everywhere in the environment would not be realistic: artificial noses able to correctly classify a reasonable range of odour are complex devices, which are still too massive and expensive to be distributed in the environment. By contrast, our approach relies on the combination of a network of simple, cheap and small sensors to detect a possible abnormal situation at a given location; and of a mobile robot equipped with a sophisticated artificial nose to move to that location and perform an in-depth analysis of the cause of the alarm. So far, the experimental validation done in the Peis-Home have been preliminary. A general point worth highlighting is that many of the ingredients in the mobile platform from Chapter 6 are still needed in the intelligent environment such as Home Monitor deliberator for dispatching robot to perform sensing actions (before the planner on the robotic system). Also in both cases the system can be exploited to assist in the odour recognition process such as aiding in the navigation to potential odour sources using other modalities and providing a context for odour classification. Future developments on the ecological approach could be to investigate the problems of odour-based navigation together with
7.4. Discussion
125
odour discrimination in an intelligent environment. This approach could be useful for large search areas (like a factory floor) with several odours.
126
Chapter 7. An Ecological Approach to Odour Recognition
Chapter 8
Conclusion This chapter concludes and discusses the results of our work. We shall begin with a summary of the main contributions of this thesis and proceed to discuss the strength and limitations of the proposed approach to integrate electronic noses in robotic and intelligent systems. Finally, we outline some potential directions for future work.
8.1
What has been achieved?
The thesis has focused on the problem of integrating electronic noses into intelligent and robotic systems. Three major sources of difficulties were addressed in this work (i) abstraction of the e-nose data into a symbolic representation for high-level representation (ii) Reasoning about sensing actions to determine when, how and where the e-nose should be activated (iii) the need to manage perceptual information that come from different sensing modalities at different times in a unifying structure in order to maintain coherent perceptual information of objects. In the work reported here, we have explored the use of current AI techniques used in robotic systems in order to address these difficulties. At the discussion of each chapter we have indicated the problems that the particular techniques could address, and have implemented these techniques on a number of different platforms. Two platforms in particular, the mobile robot and the ecological/intelligent room brought together our discussions in a unified system. More specifically, this thesis has made the following contributions. 1. We have used the concept of anchoring in order to provide a solution to the problem of managing the sensor data. We have exploited an existing framework by [24] and made essential contribu127
128
Chapter 8. Conclusion tions that take into account olfactory information and top-down and bottom-up information acquisition in the same system.
2. We have shown that automated planning can be used to contend with the problem of perceptual planning for olfactory actions. 3. We have proposed a solution to representing and communicating olfactory signals that relies on symbolic representation. We have also shown that this method can be used to maintain perceptual character of an odour when measured from an electronic nose.
8.2
Critical Assessment
The work in this thesis is meant to advance the state of the art in the field of electronic olfaction by fusing together e-nose research with research in autonomous robotics and artificial intelligence. In each chapter we concluded with a discussion which provided an assessment of its corresponding contribution. Here we discuss more general points that apply to our overall objective. Despite its importance, olfaction in general is still not well understood. The underlying physics is complex. Vision and audition also involve complex physical phenomena, but photons and sound waves are well-defined physical objects that follow well-known equations of a simple basic nature. Specifically, in both cases sensory quality may be related to a simple, one-dimensional scale of wavelength. On the other hand, the smell of an odorant, or an olfactory ligand, is determined by the complex, and only partially understood, forces between ligands and receptors. The underlying biology is also complex. The nose contains hundreds of different types of odor receptors, each of them interacting in different ways with different kinds of odorants. Thus, the dimensionality of the sense of smell is at least two orders of magnitude larger than that of vision, with only three types of color receptors. The above factors are strong motivations for creating an electronic counterpart which could eventually offer a deeper insight into the nature of an odourant. Without a good understanding of basic biological processes in our current examples of olfaction, duplicating the sense of smell electronically becomes difficult. Both at the sensor level, where good sensing technology is required, and at the data processing stage where proper interpretation of the data is necessary. Given the current situation, we developed a system that provided an output that was compatible with the human users. In some way, this type of output steers
8.2. Critical Assessment
129
away from the true motivation of having an electronic nose - quantification of odours. We see this as an important trade-off which makes electronic olfaction more useful, more understood, and facilitates integration. The major problem then resides with finding a suitable method of evaluation. With no ground truth, a symbolic output of odours is difficult to evaluate. This topic has been discussed both in the symbol grounding community and in the anchoring community without any general solution. Such a solution would be of benefit for many working on the correspondence between high-level representation and low-level sensor data. In our current definition of an intelligent system, automated process need to be in place that select appropriate responses and take corresponding actions. We have implemented a conditional planner that can reason about smelling actions. In the formal investigation, we have only considered the use of the conditional planner to perform sensing actions for performance improvement. The planner uses a current model of the world, and uncertainty in the sensing response and we have assumed such models to be provided to the system. In the practical investigations, we have implemented these models only superficially, using approximate estimation derived from a priori knowledge of the sensor. In fact, a deeper understanding of the uncertainty from the sensing unit is required in order to reason correctly about action. More work is required to investigate this issue. Finally, as the title of this thesis suggests, our primary measure of performance has been concentrated on the “odour recognition” component in the context of a larger system. The experimental validations have primarily focused on how well the classification components in the olfactory modules perform. These results, however, do not give a complete picture of the success of integration. As was seen in the experiments on the mobile platforms, visual errors propagate into olfactory errors and olfactory errors may mislead the reasoning component to take inappropriate action. The relationship between the components of the system, need to be experimentally validated in order to properly justify our approach. However, before such an investigation occurs the primary objective is to establish that odour discrimination is still feasible in the complex intelligent system and that was the goal of the work here. The work presented in this thesis is a part of a growing amount of applications areas that consider electronic olfaction. Many of these applications are discussed on a system by system basis, each system or application attempts to push the envelope to include a wider spectrum of odours for a wider range of fields. To the best of our knowledge,
130
Chapter 8. Conclusion
this thesis is the first attempt to identify and address a subset of general problems in olfaction applicable to a range of existing and future systems.
8.3
Future Work
The current technological state of gas sensors still require a long and cumbersome sampling procedure in order to achieve good data samples for discrimination. The robustness and reproducibility of the sensor data is poor. Short-term drift is present and although this can be addressed using a variety of techniques, long term drift over months depends largely on sensing history of each individual e-nose. Experience shows that each electronic nose that is integrated on a mobile robot needs to build its own training repository. It has yet to be shown that the training data from one type of e-nose is portable to another even when the gas sensing arrays are identical. So it is perhaps fair to say that it will take some years before e-nose start making an appearance in the average Swedish home. However, it is equally important to stress that the state of the art in sensing technology is constantly evolving and that electronic noses have transcended from a pure research platform to commercialized instruments. Given the current state of technology, a number of future endeavours are envisioned: Recognition, Navigation and the Ecology Work in this thesis has not considered any type of odour based navigation. Similarly odour based navigation techniques have more or less disregarded odour discrimination in the presence of multiple sources. However, these two applications for e-noses on mobile robots are closely related. The current mobile robotics system presented in Chapter 6 could be augmented to include path planning and develop an appropriate navigation strategy to an odour source. Placing such a mobile robot in an intelligent environment like that presented in Chapter 7 could also aid in developing better localization, and providing a fast generation of a rough gridmap of concentration for source declaration. Recognition, Communication and the Human The investigation of how well odour categories are represented by the electronic nose were investigated in cases with limited human expertise on olfaction. Further study with human interaction can be made to investigate the success of symbolically describing odours. Such a study would require inspirations from human-machine interaction,
8.3. Future Work
131
in order to formalize the effectiveness of communication and usability. However, this would constitute an important step forward in promoting olfaction particularly for industrial applications. Olfaction, Objects and the Binding Problem In an attempt to integrate olfaction onto a multi-sensing robot, the need to manage information from the electronic nose together with other sensing modalities was recognized. A further look into this challenge revealed an interesting problem called the binding problem discussed in Chapter 4. To address this problem on the mobile robot, we had borrowed the notion of anchoring and with some modifications to an existing framework, a naive implementation was made. As robotic systems in particular become more equipped with sensing modalities and cognitive processes, the relevance of the binding problem is expected to increase. We have scratched its surface, and a future direction will be too examine how the anchoring module can be refined to cope with aspects of perceptual management. For many of these future plans to occur a closer cooperation between research in the electronic nose community as well as the robotics/AI communities is needed. We believe that the work in this thesis constitutes an important contribution to the beginning of such a cooperation.
132
Chapter 8. Conclusion
Bibliography [1] H. Abdel-Aty-Zohdy. Artificial neural network electronic nose for volatile organic compounds. In Proceedings of the 8th Great Lakes Symposium on VLSI, pages 122–125, 1998. [2] W. Adjie-Winoto, E. Schwartz, H. Balakrishnan, and J. Lilley. The design and implementation of an intentional naming system. In Proc. of the 17th ACM Symp. on Operating systems principles, pages 186–201, 1999. [3] R. Adrian. Sensor management. In Proc of AIAA/IEEE Conference on Digital Avionics System, pages 32–37, 1993. [4] C. Alocilja, N. Ritchie, and D. Grooms. Protocol development using an electronic nose for differentiating e. coli strains. IEEE Sensors Journal, 3(6):801–805, 2003. [5] Alpha M.O.S Multi Organoleptic Systems, Toulouse (France). http://www.alpha-mos.com. (January 2006). [6] J. Amoore. Psychophysics of odor. In Cold Spring Harbor Symposia in Quantitative Biology, volume 30, pages 623–637, 1965. [7] AppliedSensor, Link¨ oping (Sweden), formely MoTech, Reutlingen (Germany). http://www.appliedsensor.com. (January 2006). [8] D. Arregui, C. Fernstrom, F. Pacull, and J. Gilbert. Stitch: Middleware for ubiquitous applications. In Proc. of the Smart Object Conf., Grenoble, France, 2003. [9] K. Arshak, G. Lyons, L. Cavanagh, and S. Clifford. Front-end signal conditioning used for resistance-based sensors in electronic nose systems: A review. Sensor Review, 23(3):230–241, 2003. [10] A. Atrash and S. Koenig. Probabilistic planning for behaviourbased robots. In AAAI ’01, pages 549–556, 2001. 133
134
BIBLIOGRAPHY
[11] R. Baby, M. Cabezas, E.Castro, R. Filip, and N. de Reca. Quality control of medicinal plants with an electronic nose. Sensors and Actuators B (Chemical), 106(1):24–28, 2005. [12] F. Bear, B. Connors, and M. Paradiso. Neuroscience Exploring the Brain. William and Wilkins, Baltimore, USA, 1996. [13] J. Bezdek. Pattern Recognition with Fuzzy Objective Function. Plenum Press, Oxford, 1981. [14] C. Bishop. Neural Networks for Pattern Recognition. Oxford University Press, Oxford, 1995. [15] S. Blackmore. Consciousness: Stoughton, Oxford, 2003.
An Introduction.
Hodder &
[16] P. Boilot, L. Hines, M. Gongora, and R. Folland. Electronic noses inter-comparison, data fusion and sensor selection in discrimination of standard fruit solutions. Sensors and Actuators B Chemical, B88(1):80–88, 2003. [17] T. Borjesson, H. Sundgren, and T. Eklov. Grain odour classification with an electronic nose. In 8th International Congress on Solid State Sensors and Actuators and Eurosensors, volume 3, pages 11–12, Stockholm, SWEDEN, 1995. [18] A. Branca, P. Simonian, M. Ferrante, E. Novas, and R. Negri. Electronic nose based discrimination of a perfumery compound in a fragrance. Sensors and Actuators B (Chemical), B92(1-2):222– 227, 2003. [19] M. Broxvall, M. Gritti, A. Saffiotti, B.S. Seo, and Y.J. Cho. Peis ecology: Integrating robots into smart environments. In Proc. IEEE Int. Conf. on Robotics and Automation (ICRA 06), 2006. Accepted. [20] M. Broxvall, L. Karlsson, and A. Saffiotti. Steps toward detecting and recovering from perceptual failures. In Proc of the 8th Int Conf on Inteligent Autonomous Systems (IAS), pages 793–800, Amsterdam, NL, 2004. Online at http://www.aass.oru.se/˜asaffio/. [21] D.A. C´ aceres, H. Mart´ınez, M.A. Zamora, and L.M. Tom´ as. A realtime framework for robotics software. In Int Conf on Computer Integrated Manufacturing, 2003.
BIBLIOGRAPHY
135
[22] A. Chella, S. Coradeschi, M. Frixione, and A. Saffiotti. Perceptual anchoring via conceptual spaces. In Proc. of the AAAI-04 Workhsop on Anchoring Symbols to Sensor Data, Menlo Park, CA, 2004. AAAI Press. Online at http://www.aass.oru.se/˜asaffio/. [23] R. Chouhan. A review of chemical sensors for electronic nose applications. MT&M-Manufacturing Technology and Management, 12(1-2):7–9, 2004. [24] S. Coradeschi and A. Saffiotti. Anchoring symbols to sensor data: preliminary report. In Proc. of the 17th AAAI Conf., pages 129–135, Menlo Park, CA, 2000. AAAI Press. Online at http://www.aass.oru.se/˜asaffio/. [25] S. Coradeschi and A. Saffiotti, editors. Robotics and Autonomous Systems, special issue on Perceptual Anchoring. Elsevier Science, 2003. [26] R. Costanzo and P. Graziadei. Development and plasticity of the olfactory system. In T. Finger and W. Silver, editors, Neurobiology of taste and smell. John Wiley, New York, 1987. [27] Cyranose Sciences Inc. The Cyranose 320 Electronic Nose - User’s Manual, Revision D. http://www.cyranose.com (August 2004). 2000. [28] P. Davidsson. Toward a general solution to the symbol grounding problem: combining machine learning and computer vision. In AAAI Fall Symposium Series, Machine Learning in Computer Vision: What, Why and How?, pages 191–202. AAAI Press, 1993. [29] M. de la Maza. A prototype based symbolic concept learning system. In Proc. of the Eighth International Workshop on Machine Learning, pages 41–45, Evanston, IL, 1991. [30] C. Delpha, M. Lumbreras, and M. Siadat. Discrimination of forane 134a and carbon dioxide concentrations in an air conditioned atmosphere with an electronic nose: influence of the relative humidity. Sensors and Actuators B Chemical, B80(1):59–67, 2001. [31] F. Di-Francesco, B. Lazzerini, A. Maggiore, F. Marcelloni, and D. De-Rossi. Electronic nose based on linguistic fuzzy classification. In 6th European Congress on Intelligent Techniques and Soft Computing,EUFIT-’98, volume 2, pages 1211–1215, 1998.
136
BIBLIOGRAPHY
[32] C. DiNatale, A. Macagnano, A. D’Amico, and F. Davide. Electronic-nose modelling and data analysis using a self-organizing map. Measurement Science and Technology, 8(11):1236–1243, 1997. [33] J. Dodd and V. Castellucci. Smell and taste: The chemical senses. In E.R. Kandel, J.H. Schwartz, and T.M. Jessel, editors, Principles of neural science. Elsevier, New York, 1991. [34] A. Dravnieks. Atlas of Odor Character profiles (ASTM Data Series Publication DS 61). American Society for Testing, USA, 2000. [35] D. Driankov and A. Saffiotti, editors. Fuzzy Logic Techniques for Autonomous Vehicle Navigation. Studies in Fuzziness and Soft Computing. Springer-Phisica Verlag, 2001. ISBN 3-7908-1341-9. [36] D. Dubois. Categories as acts of meaning: The case of categories in olfaction and audition. Cognitive Science Quaterly, 1:35–68, 2000. [37] D. Dumitrescu, B. Lazzerini, and F. Marcelloni. A fuzzy hierarchical classification system for olfactory signals. Pattern Analysis & Applications, 3:325–334, 2000. [38] R. Dutta, L. Hines, J. Gardner, and P. Boilot. Bacteria classification using cyranose 320 electronic nose. Biomedical Engineering Online, 1, 2002. [39] K. Ema, M. Yokoyma, T. Nakamoto, and T. Moriizumi. Odoursensing system using a quartz-resonator sensor array and neural networks pattern recognition. Sensors and Actuators, 18:291–296, 1989. [40] K. Esbensen, S. Schonkopf, and T. Midtgaard. Multivariate Analysis in Practice. Camo, Trondheim, 1994. [41] R. Feldhoff, C. Saby, and P. Bernadet. Detection of perfumes in diesel fuels with semiconductor and mass-spectrometry based electronic noses. Flavour and Fragrance Journal, 15(4):215–222, 2000. [42] R. Fenner and R. Stuetz. Application of electronic nose technology to environmental monitoring of water and wastewater treatment activities. Water Environment Research, 71(3):282–289, 1999.
BIBLIOGRAPHY
137
[43] M. Fichtner and M. Thielscher. Anchoring symbols to percepts in the fluent calculus. In U. Visser, G. Lakemeyer, G. Vachtesevanos, and M. Veloso, editors, Proc. of the 19th Int. Joint Conf. on Artificial Intelligence (IJCAI-05), Workshop on Agents in Real-Time and Dynamic Environments, July 30, 2005, pages 11–16, 2005. [44] Figaro Gas Sensor: http://www.figaro.com.
Technical
Reference,
1992.
[45] S. Fiorillo, S. Di Bortolomeo, C. Nannini, and D. De Rossi. Thin layers grown onto copper salt replica for sesor array fabrication. Sensors and Actuators-B Chemical, 7:399–403, 1992. [46] D. Fisher. Knowledge acquisition via incremental conceptual clustering. Machine Learning, 2:139–238, 1987. [47] J. Gardner and P. Bartlett. A brief history of electronic noses. Sensors and Actuators B, 18, 1994. [48] J. Gardner and P. Bartlett. Electronic Noses, Principles and Applications. Oxford University Press, New York, NY, USA, 1999. [49] J. Gardner, E. Hines, and C. Pang. Detection of vapours and odours from a multisensor array using pattern recognition: selforganizing adaptive resonance techniques. Measurement and Control, 29:172–178, 1996. [50] J. Gardner, H. Shin, and E. Hines. An electronic nose system to diagnose illness. Sensors and Actuators B (Chemical), B70(13):19–24, 2000. [51] I. Gath and A. Geva. Unsupervised optimal fuzzy clustering. IEEE Trans. on Pattern Analysis and Machine Intelligence, 11(7):773– 781, 1989. [52] R. Gesteland, J. Lettvin, W. Pitts, and A. Rojas. Odor specificities of the frog’s olfactory receptors. In Y. Zotterman, editor, Olfaction and Taste. Pergamon, New York, 1963. [53] T. Getchell, R. Doty, L. Bartoshuk, and J. Snow. Smell and Taste in healthy disease. Raven Press, New York, 1987. [54] J. Grate, S. Martin, and R. White. Acoustic wave microsensors, part i. Anal. Chemistry, 64:940–948, 1993.
138
BIBLIOGRAPHY
[55] F. Grazzo, J. Basil, and J. Atenna. Toward the convergence: robot and lobster perspectives of tracking odors to their source in the turbulent marine environment. In In proceedings of the IEEE Intl. Symp. on Intelligent Control (ISIC), pages 259–264, 1998. [56] A. Guardarrama, J. Fernandez, and M. Iniguez. Discrimination of wine aroma using an array of conducting polymer sensors in conjunction with solid-phase micro-extraction (spme) technique. Sensors and Actuators-B Chemical, B77(1-2):401–408, 2001. [57] I. Guterman, M. Shalit, N. Menda, D. Piestun, M. Dafny-Yelin, G. Shalev, E. Bar, O. Davydov, M. Ovadis, D. Emanuel, J. Wangand Z. Adam, E. Pichersky, E. Lewinsohn, D. Zamir, A. Vainstein, and D. Weiss. Rose scent: genomics approach to discovering novel floral fragrance-related genes. The Plant Cell, 14:2325–2338, 2002. [58] R. Gutierrez-Osuna. Pattern analysis for machine olfaction: A review. IEEE Sensors Journal, 2(3):189–202, 2002. [59] R. Gutierrez-Osuna, S. S. Schiffman, and H. T. Nagle. Correlation of sensory analysis with electronic nose data for swine odor remediation assessment. In Proceedings of the 3rd European Congress on Odours, Metrology and Electronic Noses, 2001. [60] S. Harnad. The symbol grounding problem. Physica D, 42:335– 346, 1990. [61] K. Hayashi, M. Yamanaka, and K. Toko. Multichannel taste sensor using lipid membranes. Sensors and Actuators B: Chemical, 2(3):205–213, 1990. [62] A. Hayes, A. Martinoli, and R. Goodman. Swarm robotic odor localization. In In Proc. of the IEEE Conf. on Intelligent Robots and Systems (IROS-01), pages 1073–1078, 2001. [63] S. Haykin. Neural Networks, A Comprehensive Foundation. Macmillan Colledge Publishing Company, New York, 1994. [64] J. Heidemann, F. Silva, C. Intanagonwiwat, R. Govindan, D. Estrin, and D. Ganesan. Building efficient wireless sensor networks with low-level naming. In Proc. of the 18th ACM Symp. on Operating systems principles, pages 146–159, 2001.
BIBLIOGRAPHY
139
[65] G. Henning, J. Brouwer, H. Van Der Wel, and A. Francke. The sweet-inducing principle from miracle fruit. In C. Pfaffmann, editor, Olfaction and Taste. Rockefeller University Press, New York, 1996. [66] M. Holmberg, F. Winquist, Y. Lundstr¨om, F. Davide, C. Di Natale, and A. D’Amico. Drift counteraction for an electronic nose. Sensors and Actuators-B, 35-36:528–535, 1996. [67] F. H¨ oppner, F. Klawnonn, R. Kruse, and T. Runkler. Fuzzy Cluster Analysis. John Wiley & Sons, Ltd, New York, USA, 1999. [68] K. Ihokura and J. Watson. The stannic oxide gas sensor. CRC Press, Florida, 1994. [69] H. Ishida, T. Nakamoto, and T. Moriizumi. Remote sensing of gas/odor source location and concentration distribution using mobile system. Sensors and Actuators B, 49:52–57, 1998. [70] H. Ishida, K. Suetsugu, T. Nakamoto, and T. Moriizumi. Study of autonomous mobile sensing system for localization of odor source using gas sensors and anemometric sensors. Sensors and Actuators A, 45:153–157, 1994. [71] IST Advisory Group. Ambient intelligence: from vision to reality. In G. Riva, F. Vatalaro, F. Davide, and M. Alca niz, editors, Ambient Intelligence. IOS Press, 2005. [72] Gardner J. Pattern recognition in the warwick electronic nose. In 8th International Congress of European Chemoreception Reasearch Organization, page 9, Coventry, UK, 1988. [73] A. Jain, M. Jianchang, and K. Mohiuddin. Artificial neural networks: a tutorial. Computer, 29(3):31–44, 1996. [74] E. Jones. The pellistor catalytic gas detector. In P Moseley, J. Morris, and D. Williams, editors, Techniques and mechanism in gas sensing, pages 17–31, Bristol, 1991. [75] L.P. Kaebling, M.L. Littman, and A.R. Cassandra. Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101(1–2):99–134, 1998. [76] E. Kalman. Gas-Sensors for Automobile Interiors. PhD thesis, Link¨ oping University, 2000.
140
BIBLIOGRAPHY
[77] L. Karlsson. Conditional progressive planning under uncertainty. In Proc. of the 17th Int. Joint Conferences on Artificial Intelligence (IJCAI), pages 431–438, 2001. [78] L. Karlsson and T. Schiavinotto. Progressive planning for mobile robots — a progress report. In M. Beetz, J. Hertzberg, L. Guibas, M. Ghallab, and M. Pollack, editors, Advances in Plan-Based Control of Robotic Agents, Lecture Notes in Artificial Intelligence. Springer, 2002. [79] S. Kazadi, R. Goodman, D. Tsikata, D. Green, and H. Lin. Autonomous water vapor plume tracking robot using passive resistive polymer sensors. Autonmous Robots, 9(2):175–188, 2000. [80] M. Kleinehagenbrock, S. Lang, J. Fritsch, F. L¨ omker, G. A. Fink, and G. Sagerer. Person tracking with a mobile robot based on multi-modal anchoring. In Proc. IEEE Int. Workshop on Robot and Human Interactive Communication (ROMAN), pages 423– 429, 2002. [81] A. Knoblauch, R. Fay, U. Kaufmann, H. Markert, and G. Palm. Associating words to visually recognized objects. In S. Coradeschi and A. Saffiotti, editors, Anchoring symbols to sensor data. Papers from the AAAI Workshop. Technical Report WS-04-03, pages 10– 16. AAAI Press, Menlo Park, California, 2004. [82] A. Knoll and M. de Kamps, editors. Roadmap of Neuro-IT Development. Online, 2004. www.neuro-it.net/NeuroIT/Roadmap/. [83] T. Kohonen. Self-organized formation of topologically correct feature maps. Biol. Cybernet, 43:59–69, 1982. [84] S. Kovacic, A. Leonardis, and F. Pernus. Planning sequences of views for 3-D object recognition and pose determination. Pattern Recognition, 31:1407–1417, 1998. [85] L. Kuncheva. Fuzzy Classifier Design: with 81 tables. PhysicaVerlag Heidelberg, New York, USA, 2000. [86] Y. Kuwana, I. Shimoyama, and H. Miura. Steering control of a mobile robot using insect antennae. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, volume 2, pages 530–535, 1995.
BIBLIOGRAPHY
141
[87] A. Larkin, E. Hines, S. Thomas, and J. Gardner. Supervised learning using the vector memory array method. In Proceedings of the Workshop on Neural Network Applications and Tools, pages 6–10, Los Alamitos, CA, 1994. [88] B. Lazzerini, A. Maggiore, and F. Marcelloni. Fros: a fuzzy logic-based recogniser of olfactory signals. Pattern Recognition, 34(11):2215–2226, 2001. [89] M. Lebowitz. Concept learning in a rich input domain: Generalization-based memory. In R. S. Michalski, J. G. Carbonell, and T. M. Mitchell, editors, Machine Learning: An Artificial Intelligence Approach: Volume II, pages 193–214. Kaufmann, Los Altos, CA, 1986. [90] Y. Lee, D. Terzopoulos, and K. Waters. Realistic modeling for facial animation. In In Computer Graphics ACM Anual Conference Series, pages 55–62, 1995. [91] S. Letant, S. Content, T. Tze Tsung, F. Zenhausern, and M. Sailor. Integration of porous silicon chips in an electronic artificial nose. Sensors and Actuators B Chemical, B69(1-2):193–198, 2000. [92] A. Lilienthal and T. Duckett. An absolute positioning system for 100 euros. In Proc. IEEE Int. Workshop on Robotic Sensing, ¨ Orebro, Sweden, 2003. [93] A. Lilienthal and T. Duckett. Experimental analysis of smelling braitenberg vehicles. In Proceedings of the IEEE International Conference on Advanced Robotics (ICAR 2003). IEEE, 2003. [94] A. Lilienthal and T. Duckett. Experimental analysis of gassensitive braitenberg vehicles. Advanced Robotics, 18(8):817–834, 2004. [95] A. Lilienthal, F. Streichert, and A. Zell. Model-based shape analysis of gas concentration gridmaps for improved gas source localisation. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA 2005), pages 3575 – 3580, 2005. [96] A. Lilienthal, H. Ulmer, H. Fr¨ohlich, A. St¨ utzle, F. Werner, and A. Zell. Gas source declaration with a mobile robot. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA 2004), pages 1430 – 1435, 2004.
142
BIBLIOGRAPHY
[97] E. Llobet, E. Hines, J. Gardner, P. Bartlett, and T. Mottram. Fuzzy artmap based electronic nose data analysis. Sensors and Actuators B Chemical., B61(1-3):183–190, 1999. [98] A. Loutfi, M. Broxvall, S. Coradeschi, and L. Karlsson. Object recognition: A new application for smelling robots. Robotics and Autonomous Systems, 52:272–289, 2005. [99] A. Loutfi and S. Coradeschi. Forming odour categories using an electronic nose. In Proc. of European Conference in Artificial Intelligence (ECAI 2004), pages 119–124, 2004. [100] A. Loutfi and P. Wide. Symbolic estimation of food odors using fuzzy techniques. In Information and Processing and Managment of Uncertainty, Annecy, France, 2002. [101] A. Loutfi, J. Widmark, P. Wide, and E. Wikstrom. Social agent: Expressions driven by an electronic nose. In International Symposium on Virtual Environments, Human Computer Interfaces and Measurement Systems, Lugano, Switzerland, 2003. [102] R. Lundh, L. Karlsson, and A. Saffiotti. Can Emil help Pippi? In Proc. of the ICRA-05 Workshop on Cooperative Robotics, Barcelona, ES, 2005. Online at www.aass.oru.se/˜asaffio/. [103] I. Lundstr¨ om, E. Hedborg, A. Spetz, H. Sundgren, and F. Winquist. Electronic nose based on field effect structures. In J. Gardner and P. Bartlett, editors, Sensors and sensory systems for an electronic nose, volume 212, pages 303–319, Kluwer, Dordrecht, 1992. [104] S.M. Majercik and M.L. Littman. Contingent planning under uncertainty via stochastic satisfiability. In AAAI ’99, pages 549–556, 1999. [105] L. Marques, U. Nunes, and A. de Almeida. Olfaction-based mobile robot navigation. Thin Solid Films, 418:51–58, 2002. [106] D. Martin. Electronic noses to sniff out rogue gases. Process Engineering, pages 65–67, 1975. [107] R. S. Michalski and R. E. Stepp. Learning from observation: Conceptual clustering. In R. S. Michalski, J. G. Carbonell, and T. M. Mitchell, editors, Machine Learning: An Artificial Intelligence Approach, pages 331–363. Springer, Berlin, Heidelberg, 1984.
BIBLIOGRAPHY
143
[108] Mintel International Group Ltd. Market Research and Consumer Intelligence. Fragrances & Cosmetics in Europe. http://www.mintel.com. (January 2006). [109] R. Molfino, editor. Proc. of the 1st Int. Conf. on Intelligent Manipulation and Grasping, Genova, Italy, 2004. [110] R. Murphy. Allocating sensing resources with incomplete information. In AAAI Spring Symposium, 1996. [111] C. Di Natale, F. Davide, and A. D’Amico. A self-organizing system for pattern classificiation: time varying statistics and sensor drift effects. Sensors and Actuators-B, 26-27:237–241, 1995. [112] C. Di Natale, A. Macagnano, F. Davide, A. D’Amico, R. Paolesse, T. Boschi, M. Faccio, and G. Ferri. An electronic nose for food analysis. Sensors and Actuators B: Chemical, 44(1-3):521–526, 1997. [113] Network Robot Forum. www.scat.or.jp/nrf/English/. [114] M. O’connell, G. Valdora, and G. Peltzer. A practical approach for fish freshness determinations using a portable electronic nose. Sensors and Actuators-B Chemical, B80(2):149–154, 2001. [115] G. Ohloff. Scent and Fragrances. Springer-Verlag, Berlin, UK, 1994. [116] N. Onder and M.E. Pollack. Conditional, probabilistic planning: A unifying algorithm and effective search control mechanisms. In AAAI ’99, pages 577–584, 1999. [117] T. Pearce. Handbook of Machine Olfaction. Wiley-VCH, 2003. [118] M. Penza, G. Cassano, F. Tortorella, and G. Zaccaria. Classification of food, beverages and perfumes by wo3 thin-film sensors array and pattern recognition techniques. Sensors and Actuators B Chemical, B73(1):76–87, 2001. [119] K. Persaud and G. Dodd. Analysis of discrimination mechanisms of the mamalian olfactory system using a model nose. Nature, 299:352–355, 1982. [120] Player/Stage Project. playerstage.sourceforge.net/. [121] W. Press, W. Vetterling, S. Teukolsky, and B. Flannery. Numerical Recipes. Cambridge University Press, 2nd edition, 1992.
144
BIBLIOGRAPHY
[122] Olivier Rochel, Dominique Martinez, Etienne Hugues, and Fr´ed´eric Sarry. Stereo-olfaction with a sniffing neuromorphic robot using spiking neurons. In 16th European Conference on Solid-State Transducers - EUROSENSORS, Prague, Czech Republic, September 2002. [123] L. Ronnie, M. Johansson, and N. Xiong. Perception management - an emerging concept for information fusion. Information Fusion, 4(3):231–234, 2003. [124] R. Russell. Odour Detection by Mobile Robots. World Scientific Publishing Co, London, UK, 1999. [125] R. Russell and A. Purnamadjaja. Odour and airflow: Complementary senses for a humanoid robot. In IEEE Int. Conf. Robotics and Automation (ICRA 2002), pages 1842–1847, 2002. [126] R. A. Russell, A. Bab-Hadiashar, R. L. Shepherd, and G. G. Wallace. A comparison of reactive chemotaxis algorithms. Robotics and Autonomous Systems, 45:83–97, 2003. [127] R.A. Russell. Chemical source location and the robomole project. In Proceedings Australian Conference on Robotics and Automation, 2003. [128] A. Saffiotti and M. Broxvall. PEIS ecologies: Ambient intelligence meets autonomous robotics. In Proc of the Int Conf on Smart Objects and Ambient Intelligence (sOc-EUSAI), pages 275–280, Grenoble, France, 2005. [129] A. Saffiotti, K. Konolige, and E. H. Ruspini. A multivalued-logic approach to integrating planning and control. Artificial Intelligence, 76(1-2):481–526, 1995. [130] D. Schulz, W. Burgard, D. Fox, and A. Cremers. People tracking with mobile robots using sample-based joint probabilistic data association filters. I. J. Robotic Res., 22(2):99–116, 2003. [131] W. Seitz. Chemical sensors based on fiber optics. Anal. Chemistry, 56:16–34, 1984. [132] T. Seiyama, A. Kato, K. Fujishi, and M. Nagatami. A new detector for gaseous components using semiconductive thin films. Anal. Chem., 32:1502–1503, 1962.
BIBLIOGRAPHY
145
[133] M. Siegel. The sense-think-act paradigm revisited. In International Workshop on Robotic Sensing ROSE, pages 1–5, 2003. [134] F. Siegemund. A context-aware communication platform for smart objects. In Proc. of the Int. Conf. on Pervasive Computing, 2004. [135] S. Singh. A single nearest neighbour fuzzy approach for pattern recognition, 1998. [136] L. Steels. The Talking Heads Experiment. Best of Publishing, Brussels, Belgium, 1999. [137] L. Steels and F. Kaplan. Aibo’s first words: The social learning of language and meaning. Evolution of Communication, 4(1):3–32, 2000. [138] L. Steels and F. Kaplan. Aibo’s first words. the social learning of language and meaning. Evolution of Communication, 4(1), 2001. [139] T. Sundic, S. Marco, A. Perera, A. Pardo, S. Hahn, N. Barsan, and U. Weimar. Fuzzy inference system for sensor array calibration: prediction of co and ch4 levels in variable humidity conditions. Chemometrics and Intelligent Laboratory-Systems, 64(2):103–122, 2002. [140] T. Sundic, S. Marco, A. Perera, A. Pardo, J. Samitier, and P. Wide. Potato creams recognition from electronic nose and tongue signals: feature extraction/selection and r.b.f neural networks classifiers. In Proc. of the IEEE 5th Seminar on Neural Network Applications in Electrical Engineering (NEUREL), pages 69–74, 2000. [141] N. Taguchi. U.S. Patent 3 695 848. (1972). [142] J. Taylor. Linguistic Categorization: Prototypes in linguistic theory. Clarendon Press, Oxford, UK, 1995. [143] The Europ Consortium. Building the European Robotics Platform EUROP. Online, 2005. www.euron.org/europ/. [144] The W3C Consortiom. www.w3.org/2001/sw/, 2001.
The
semantic
web
activity.
[145] P. Vogt. Lexicon Grounding on Mobile Robots. PhD thesis, Vrije Universiteit Brussel, 2000.
146
BIBLIOGRAPHY
[146] C. von der Malsburg. The correlation theory of brain function. Internal report, 81-2, Max-Planck-Institut f¨ ur Biophysikalische Chemie, Postfach 2841, 3400 G¨ottingen, FRG, 1981. Reprinted in E. Domany, J. L. van Hemmen, and K. Schulten, editors, Models of Neural Networks II, chapter 2, pages 95–119. Springer-Verlag, Berlin, 1994. [147] B. Wenzel. Chemoreception. In E. Carterette and M. Friedman, editors, Handbook of Perception, Volume III: Biology of perceptual systems. Academic Press, New York, 1973. [148] P. Wide, F. Winquist, and D. Driankov. An air-quality sensor system with fuzzy classification. Measurement and Science Technology, 1998. [149] J. Widmark. Social agent: Facial expressions driven by an e-nose. ¨ Master’s thesis, Orebro University, 2003. [150] W. Wilkens and J. Hartman. An electronic analogue for the olfactory process. Acad. Sci., 116:608–612, 1964. [151] P. Wyckoff, S.W. McLaughry, T.J. Lehman, and D.A. Ford. Tspaces. IBM Systems Journal, 37(3), 1988. [152] X. Xie and G. Beni. A validity measure for fuzzy clustering. IEEE Trans. on Pattern Analysis and Machine Intelligence, 13(8):841– 847, 1991. [153] J. Yinun. Detection of explosive by electronic noses. Analytical Chemistry, 75(5):98A–105A, 2003. [154] R. Young, W. Buttner, B. Linnell, and R. Ramesham. Electronic nose for space program applications. Sensors and Actuators B: Chemical, 93(1-3):7–16, 2003. [155] L. Zadeh. Fuzzy Logic and its applications. Academic Press, New York, 1965. [156] H. Zwaardemaker and F. Hogewind. On spray- electricity and waterfall-electricity. Proc. Academic Sci., 22:429–437, 1920.
