autonomous navigation. ... public robots, which are autonomy and human robot interaction. ... like an office building in KIST and the exhibit hall of Hyundai.
The Autonomous Tour-Guide Robot Jinny Gunhee Kim, Woojin Chung, Kyung-Rock Kim, Munsang Kim Intelligent Robotics Research Center, Korea Institute of Science and Technology, 39-1 Hawolgok-dong, Sungbuk-ku, Seoul, 136-791, Korea {knir38, wjchung, rock, munsang}@kist.re.kr Abstract—This paper explains a new tour-guide robot Jinny. The Jinny is developed by focusing on human robot interaction and autonomous navigation. In order to achieve reliable and safe navigation performance, an integrated navigation strategy is established based on the analysis of a robot’s states and the decision making process of robot behaviors. According to the condition of environments, the robot can select its motion algorithm among four types of navigation strategy. Also, we emphasized the manageability of a robot’s knowledge base for human friendly interactions. The robot’s knowledge base can be extended or modified intuitively enough to be managed by nonexperts. In order to show the feasibility and effectiveness of our system, we also present experimental results of the navigation system and some experiences on practical installations. Keywords-component; guide robots, navigation, human robot interaction
I.
INTRODUCTION
In recent years, there have been various trials to extend robotics technology to service applications in public spaces. Especially, many researchers have an interest in the guide robot since it is closely related to two critical issues in current robotic research, which are interaction with human and navigation in dynamic environments. So far, there have been many related research activities about the guide robots in human coexisting environment. Minerva [1] [2], one of the most famous tour-guide robots, was installed in the Smithsonian’s National Museum of American History during two weeks in 1998. It showed the feasibility and reliability of its navigation system in densely populated and unmodified environments. The navigation system adopts Markov localization, ceiling mosaics, and model based dynamic window approach. Also, it developed the spontaneous short-term interaction suitable in tour-guide applications. It is implemented by using facial expressions, the emotional state machine, and the adaptation method. The evolutionary Mobot Museum Robot Series [3] [4] are permanently installed robots which have operated in public spaces for many years. They focused on robust and reliable operation, and thus, they had relatively simple world modeling and navigation strategy. They defined two requirements of public robots, which are autonomy and human robot interaction. They adopted autonomy schemes like self-diagnosis and
Sangmok Han and Richard H. Shinn Intelligent Robotics Laboratory Bonavision, Inc. 1543-8 Seocho-dong, Seocho-ku, Seoul, 137-070, Korea {sangmok, shinn}@bonavision.net
interaction techniques such as an awareness of human presence and displays of robot’s emotional states. The entertainment robots at the “Museum fur Kommunikation” in Berlin [5] is another example of permanently installed guide robots. Three robots were created, and each of them had a specific character expressed through its appearance. Their tasks were welcoming visitors, leading a guide tour though the museum, and plying with a ball. The localization algorithm is Kalman filter based map-matching method using encoders, a gyroscope, and a laser scanner. The developed motion planning included simple ball tracking and preplanned path tracking based on a static map. RoboX [6] is one of the latest tour-guide robots. Ten RoboX systems had been applied at the Swiss National Exhibition Expo 02 during five months. It had several human interaction skills like face tracking using a color camera, a motion tracking using the laser range finders, speech recognition/synthesis, and facial expressions. It adopted probabilistic feature-based localization and the navigation system that integrates navigation function, elastic band, dynamic window approach [7]. At the KIST (Korea Institute of Science and Technology), a guide robot Jinny is under development toward permanent installation at National Science Museum of Korea in September 2004. The Jinny has been tested in various actual environments like an office building in KIST and the exhibit hall of Hyundai heavy industries. It has been also displayed in the exhibitions and appeared on television several times. Through several experiences in actual environments, we can summarize some major issues and design requirements of the Jinny as follows. 1) Adaptive navigation in dynamic and unmodified environment: As a guide agent, moving to desired exhibits and avoiding collisions with obstacles are basic capabilities. For robustness and reliability, we concluded that it is desirable that robot adaptively selects its motion according to the conditions of environments. For example, if the robot is in a broad area, the robot use a map based navigation. Else if it is in the narrow region between exhibits, the sensor based navigation, like a wall-following technique, is much dependable since it is less affected by localization accuracy and uncertainties of environments.
2) Interaction which attracts and engages people’s interest : Since our target environment is a museum, a robot doesn’t have a long-term relationship to users. The objective of our interaction system is spontaneous short-term interaction which can attract people around a robot and engages people’s interests for no more than 20 minutes at best. 3) Manageability of robot’s knoledge base: For permanent installation, the extension or modification of a robot’s knowledge base should be easy and straightforward enough to be managed by non-experts like the staffs of a museum. For example, the changes of exhibits, special events of anniversaries, and the visits of important figures happen very often in the museum. Therefore, our HRI (Human Robot Interface) system is developed by natural language based approach. It also has a graphically friendly user interface by applying widely used web technologies. 4) Reliability and safety: It is desirable that the robot mostly fulfill its assigned tasks. Also, it should operate safely both for people and for the robot itself. No mater what kinds of faults occur, for example, the robot looses its way, they do no critical harm to visitors and a robot.
The Fig.1 shows the hardware elements of the Jinny. Its diameter is 0.6m, and the height is 1.5 m. Two differentially driven wheels are located at the center of the robot, and two casters are used for stability. The robot is able to move at a speed of up to 1.0 m/s with 0.5 m/s2 of maximum acceleration. It has long lasting batteries for up to eight hour in order to recharge overnight and operate in the daytime without interruption.
The scope of this paper is to introduce our tour-guide robot Jinny by focusing on the navigation and human robot interaction systems. They are developed in order to meet the above described requirements. In order to show the feasibility and effectiveness of our system, we also present experimental results of the navigation system and some experiences about actual installations.
Table I summarizes the interaction elements in the Jinny system. The Jinny can recognize the speech of a registered user through wireless microphones. It can also receive a user’s command via a tough screen installed in the height where an average grown-up can easily use. Twelve LED buttons are distributed on the lower part of LCD display and on the body as shown in Fig.1. They are set up for robot’s short-term interesting reactions. The Jinny can recognize and synthesize voice sounds in Korean by using commercial packages manufactured by Voicetech [8]. Also, the Jinny can express its emotional states in the form of icons in the LED matrix and gestures using a mobile base, a 2-DOF neck system, and two 1DOF arm systems.
II.
SYSTEM OVERVIEW
A. Hardware platform
2-DOF neck system Two 1-DOF arms
12 LED buttons
LED matrix for expression of emotion
There are three PCs in the Jinny system. Two 3 GHz Pentium-IV PCs on Windows 2000 are used for navigation and human robot interaction respectively. Both computers communicate with each other over a 10Mbit/s wireless Ethernet. One 400MHz Pentium-III DOS PC in which motion boards are installed is used for control of mobile base, two arms, and the neck system. This computer is connected to the other computers through serial ports. The Jinny has several types of sensors for autonomous navigation and safety. Our approach is based on the rangesensors. Two laser range finders are set up on front and rear sides, and two infrared scanners are installed with different height. Also, an optical fiber gyroscope is used for better localization performance. The robot is equipped with rubber bumpers all around for an immediate stop.
TABLE I.
LCD display
Receptive elements
Speakers
Expressive elements
THE INERACTION ELEMENTS IN JINNY SYSTEM - Voice recognizer and microphones - Touch screen - 12 LED buttons - Voice synthesizer and speakers - LED matrix for expression of emotion - LCD display - Gestures (using mobile base, 2-DOF neck system, and two 1-DOF arm systems)
Two IR scanners
Battery indicator Two-wheel differential mobile base
Two laser range finders Bumpers
Figure 1. Hardware elements of the Jinny.
B. Software Architecture In our previous work [9][10], we proposed the Tripodal schematic control architecture as the solution to several architecture and system integration issues, which include information connectivity between a variety of components, scheduling of information processing, and the combination of reactivity and deliberation. It is developed for the PSR (Public Service Robot) systems, PSR-1, PSR-2, and the Jinny, which are multi-functional service robots in large scale indoor environments. Through the proposed architecture, we
successfully implemented four target service tasks, a delivery, a patrol, a guide and a floor cleaning task. Our strategy has two major advantages over pervious researches. First, the proposed architecture supports Petri net based formal description of tasks and error/fault handling schemes. Second, it provides three types of diagrams as easyto-use and straightforward guidelines of system integration issues. The integration process is intuitively completed by just following the proposed procedures. In [10], we described how to develop a new guide robot Jinny using Tripodal schematic approach. Most of modules developed for previous platforms, PSR-1 and PSR-2, are reused directly, although the hardware configuration of the Jinny is quite different. Only some hardware-related modules are different each other and is changed one to one. III.
NAVIGATION SYSTEM
A. Overview of developed navigation system The detailed description of whole navigation system of the Jinny is introduced in [11]. It includes the development of crucial navigation algorithms like map, path planning, and localization, and planning scheme such as error/fault handling. Experimental results are also given to show the feasibility of proposed navigation system. Major advantages of developed navigation system are as follows: 1) A range sensor based generalized scheme of navigation without modification of the environment: Range sensors are implemented for map construction, path planning, and localization. There is no fundamental limitation of using any type of range sensors. Multiple roles of the map were successfully implemented for environmental representations and a reference database of a localization as well as optimal path generation to the goal. Also, navigation is carried out without any artificial landmarks. 2) Intelligent navigation-related components: In our navigation system, each navigation-related module has its own intelligence. It means each component has not only its own function but also the ability to analyze internal states and environmental information. For example, the path planner generates a reference trajectory, and provides important information like a goal occupation by an object in forms of events. These events have an influence on the selection of a robot behavior. 3) Framework supporting the selection of multiple behaviors and error/fault handling schemes: As the eventgenerating modules and the concerned events increase, it becomes troublesome to manage these situations. Thus, we developed general framework, which is the Petri net based configuration. By using a proposed scheme, several significant issues can be solved, for example, task decomposition, error/fault detection and recovery, and handling many events received from several modules. Basically, we consider the events from the localizer, the path planner, and behaviors.
B. Localization Our localization method is a probabilistic map-matching scheme based on Monte Carlo localization [12]. It can use any kinds of range sensors, and handle both local tracking and global position estimation. The details of the reliable position estimation method of the PSR are introduced in [13] [14]. The advantages of the proposed localization scheme as follows. 1) Two measure functions, Range Image Similarity Measure Function (RISF) and Angular Similarity Measure Function (ASF), are developed for both polygonal and nonpolygonal environments. ASF provides fast sample convergence and high accuracy in polygonal environments. RISF makes it possible to estimate the position in nonpolygonal surroundings. 2) Our smart localization considers not only position estimation but also extraction of state information by analyzing environmental uncertainties. It helps a robot take appropriate action like human beings by introducing discrete event control concept. C. Robot motion We implemented four types of motions into the Jinny. The Jinny selects its motion algorithm according to the conditions of environments. We learned by experience that this approach, in uncertain and dynamic surroundings, is more robust and reliable than the method using one navigation strategy. As mention before, it is easily implemented and managed in virtue of our Petri net based formal architecture. 1) AutoMove: The AutoMove is our fundamental navigation strategy. Our path planning algorithm is developed by modifying Konolige’s gradient method [15]. It deals with two exceptional cases, path blocking and goal occupation unlike original Konolige’s gradient method. Such situations happen very often in a real environment. The detailed description of our path planning algorithms is introduced in [11]. The advantages of the AutoMove is generality and optimality. It is applicable in any situations, and it also make it possible for the robot to move the desired position with shortest collsition-free trajectory. The Automove is mostly used when the robot is in a broad area in our application. 2) AutoMove without path updating: During AutoMove, the optimal path is continuously updated in run-time according to the dynamic change of an environment. However, the path updating for obstacle avoidance is not desirable in some cases. The typical example is when the robot travels with a group of visitors from one exhibit to the next during the course of a tour. A stop-and-wait motion with voice notification is safer and more human-friendly than an avoidance motion when many peoples are around the robot. 3) Virtual balloon wall following: Our wall following technique was based on the full-coverage algorithm designed for a cleaning task [9]. It generates velocity commands by using on raw sensing data in every sampling time. Thus, it is more dependable with respect to localization errors and
uncertainties of environments than the other navigation methods. This motion is selected when the robot is in a narrow region between exhibits, one of the typical cases critical to localization accuracy. 4) Remote controlled motion: The robot may lose its way or fail to find out the path to the goal in some tough cases. A central computer over wireless network is allowed to monitor the state of a robot and control directly by using a keyboard or a joystick.
IV.
activated. For example, if the response is a simple question such as “How’s weather today?” or “Introduce yourself,” only speech synthesizer would run to react a user’s request. Otherwise, if the request is a long tour-guide, the robot should activate navigation related modules like a path planner and a localizer.
HUMAN ROBOT INTERACTION
A. Basic Function The Jinny can autonomously navigate the environment and explain assigned exhibits to visitors. It can express its basic emotional states through LED matrix and gestures generated by a mobile base, a 2-DOF neck, and two 1-DOF arms. It can also perform several interesting services in response to user’s requests. The Jinny plays a simple game with visitors, and dances to the music. It can dialogue with registered users about limited class of topics. Also, it provides the information about today’s coverage and weather gathered from the web sites. B. Human Robot Interface(HRI) system The Jinny’s HRI system is developed by natural language based approach which provides an effective way to communicate with untrained users. The voice input was converted to a text string by commercialized speech recognizer. Then, the strings are decomposed into several keyword patterns since we assume that most conversation between a guide robot and visitors are very short and keyword-oriented. Thus, we built a specialized simple matching algorithm to find the most probable response to an input. For example, two questions like ‘Where is a toilet?’ and ‘Where can I find a toilet’ are equally interpreted since the keyword pattern of ‘where’ and ‘toilet’ would be extracted from both cases. The weighted vector space model, which was introduced in [14] and [15], is used to calculate the similarity among keyword patterns. In the case of a touch screen and LED buttons, this process is rather simple. Each touch input is directly mapped to the pre-defined keyword pattern. For example, if a user push button 11, this input may convert to the keyword pattern of ‘go to exhibit A’ straightly without complex recognition process. Then, the keyword patterns are scored against the possible responses in knowledge base and the most appropriate one is selected. Additional constraints called ‘context’ were used to restrict the search space during the pattern matching process. Each response in knowledge base was tagged with a ‘context’ value, which tells when this response can be a valid one. For example, all response about the dinosaurs is tagged with ‘at the dinosaurs section’, because the response is valid when the robot is located near the exhibit of dinosaurs. As a result, the Jinny could make fewer mistakes in voice recognition, and could search the response in a shorter time. When the HRI planner processes a response selected in knowledge base, it decides which components should be
Figure 2. The robot knowledge management system.
The manageability of knowledge base is one of the most important requirements for the HRI system. The knowledge base is required to be not only reused in several places and but also customized for a specific environment. And, the administration tool should help non-technical staffs to handle with the frequent updates of knowledge base. The Jinny provides the web-based administration tool called Robot Knowledge Management System (RKMS), which enables users to manipulate knowledge base with an intuitive interface. Fig.2 shows the graphical user interface of the system. Even nontechnical users can specify the expected robot actions using a text string containing pre-defined action tags by following XML standard. Any changes made to knowledge base are applied immediately without compiling or downloading processes. Moreover, the system supports monitoring and administration from a remote place since it is implemented as a web server for a secure access control. As shown Fig.3, The HRI software was built using Java, ActiveX, and JavaScript technologies not only to reuse the prebuilt components but also to improve the portability of the system. The keyword pattern matching component and knowledge base component were developed in a Java language, and other HRI components were wrapped as ActiveX components. JavaScript was used to integrate Java Applets and ActiveX components. The knowledge base component stores its data using MySQL database system, since it has a reliable performance for handling a large amount of knowledge data
[17]. Finally, the web-based interface of RKMS was developed with Java Server Page technologies and was deployed in a web application server, Tomcat [18].
tracking such as an acceleration filter. The results of the AutoMove with run-time path update were shown in [11].
Node 2 Planned Node 1
Actual Trajectory
Figure 3. The deployment of the HRI software components.
V.
Node 0
EXPERIMENTS AND RESULTS
A. Navigation experiments Experiments are conducted to show the feasibility of the navigation system of the Jinny. Fig.4 shows experimental environment, a conventional office building in KIST. The mission is to guide a visitor from a hallway to the front of the room. The start point is node0, and goal is node2 as shown in Fig.4.
(a) The results of path planning.
Scan data
Estimated position
Reference data
(a) The results of localization.
Node2 (19.2, 15.8)
Figure 5. Experiment of navigation. Node1 (27.3, 14.8) Node0 (36.0, 11.3)
Figure 4. An experimental environment.
Fig.5.(a) presents a planned path and an actual trajectory during the navigation. The AutoMove without path updating is applied in this experiment since the environment is a narrow hallway. Thus, the robot doesn’t change a reference path. When the robot meets a person, it stops moving and say “Step aside, please.” As shown in Fig.5.(a), the robot’s actual trajectory has the discontinuity which mainly results from position updates by the localizer. The robot can move smoothly since the behavior also contains several schemes for stable
Fig.5.(b) shows the results of localization. It represents the local map, laser scan data, reference data, sample distributions, and estimated position. The data for path planning and localization is gathered simultaneously with a single experiment. The reference measurements of an estimated robot location are mostly consistent with the scanned measurements of an actual robot position. It means that the accurate position estimation is accomplished. Although the environment is slightly changed and a user disturbs the Jinny’s way, the proposed localization algorithms work successfully. B. Experience in actual environments The Jinny was demonstrated at the 2003 Korea Science Festival from August 13th to 21st. Also, it was tested in the exhibit hall of Hyundai Heavy Industries as shown in Fig.6. The Jinny autonomously navigated the crowded environment and explained 25 exhibits to visitors. Fig.7 shows the information map in HRI of the Jinny. A user can select the scenario by selecting the sequence of exhibits to be explained.
REFERENCES [1]
[2]
[3]
[4] Figure 6. The guide robot Jinny in the exhibit hall of Hyundai Heavy Industries.
[5]
[6]
[7]
[8] [9] Figure 7. Information map in HRI. [10]
VI.
CONCLUSION
This paper introduces a new tour-guide robot Jinny. We described a system overview, a navigation approach, and a human robot interface. Our experiments clearly show that the developed system is feasible and useful as a tour-guide agent. Major advantages of the proposed system can be summarized as follows; 1) Adaptive navigation in dynamic and unmodified environment: According to the conditions of environments, the robot can select its motion algorithm among four types of navigation strategy. Also, navigation is carried out without any artificial landmarks. 2) Manageability of robot’s knoledge base: The robot’s knowledge base is extended or modified intuitively enough to be managed by non-experts since we adopted natural language based approach and widely used web technologies. ACKNOWLEDGMENT The authors gratefully acknowledge Hyundai Heavy Industries Co., Ltd and JoyMecha Co., Ltd for prototyping the Jinny systems.
[11]
[12]
[13]
[14]
[15]
[16]
[17] [18] [19]
S. Thrun, M. Bennewitz, W. Burgard, A. B. Cremers, F. Dellaert, and D. Fox, “MINERVA: A Second-Generation Museum Tour-Guide Robot,” in Proceeding of the IEEE International Conference on Robotics and Automation, Detroit, Michigan, USA, pp. 1999-2005, 1999. J. Schulte, C. Rosenberg, and S. Turun, “Spontaneous, Short-term Interaction with Mobile Robots,” in Proceeding of the IEEE International Conference on Robotics and Automation, Detroit, Michigan, USA, pp. 1999-2005, 1999. T. Willeke, C.Kunz, and I. Nourbakhsh, “The History of the Mobot Museum Robot series: An Evolutionary Study,” in Proceeding of FLAIRS 2001 Conference, 2001. I. Nourbakhsh, J. Bobenage, S. Grange, R. Lutz, R. Meyer, and A.Soto, “An affective mobile robot educator with a full-time job,” Artificial Intelligence, vol. 114, pp. 95-124, 1999. B. Graf, and O. Barth, "Entertainment Robotics: Examples, Key Technologies and Perspectives," in proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems - Workshop "Robots in Exhibitions," 2002. B.Jensen, G.Froidevaux, X.Greppin, A.Lorotte, L.Mayor, M.Meisser, G.Ramel, R.Siegwart, “The Interactive Autonomous Mobile System RoboX,” in Proceeding of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Lausanne, Switzerland, pp. 1221-1227, 2002. K. Arras, R. Philippsen, N. Tomatis, M. Battista, M. Schilt, and R. Siegwart, “A Navigation Framework for Multiple Mobile Robots and its Application at the Expo.02 Exhibition,” in Proceeding of the IEEE International Conference on Robotics and Automation, Taipei, Taiwan, pp. 1992-1999, 2003. http://www.voicetech.co.kr/ Gunhee Kim, Woojin Chung, Munsang Kim, and Chongwon Lee, “Tripodal Schematic Design of the Control Architecture for the Service Robot PSR,” in Proceeding of the IEEE Conference on Robotics and Automation, Taipei, Taiwan, pp.2792-2797, 2003. Gunhee Kim, Woojin Chung, Munsang Kim, and Chongwon Lee, " Implementation of Multi-Functional Service Robots Using Tripodal Schematic Control Architecture," Proceedings of the 2004 International Conference on Robotics and Automation, New Orleans, LA, 2004. Woojin Chung, Gunhee Kim, Munsang Kim, and Chongwon Lee, "Integrated Navigation System for Indoor Service Robots in Large-scale Environments," Proceedings of the 2004 International Conference on Robotics and Automation, New Orleans, LA, 2004. D. Fox, W. Burgard, F. Dellaert, and S. Thrun, “Monte Carlo Localization: Efficient Position Estimation for Mobile Robots”, in Proceeding of the National Conference on Artificial Intelligence (AAAI'99), Orlando, Florida, 1999. Dongheui Lee, Woojin Chung, Munsang Kim, “Autonomous Map Building and Smart Localization of the Service Robot PSR,” in Proceeding of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Las Vegas, Nevada, 2003. Dongheui Lee, Woojin Chung, Munsang Kim, “A Reliable Position Estimation Method of the Service Robot by Map Matching,” in Proceeding of the IEEE Conference on Robotics and Automation, Taipei, Taiwan, 2003. Kurt Konolige, “A Gradient Method for Realtime Robot Control,” in Proceeding of the IEEE/RSJ Conference on Intelligent Robots and Systems, Takamatsu, Japan, pp.639-646, 2000. Gerard Salton, Chris Buckley, “Term Weighting Approaches in Automatic Text Retrieval,” in Technical Report TR87-881, Department of Computer Science, Cornell University, 1987. Salton, G. and McGill. “Introduction to Modern Information Retrieval,” McGill-Hill:1983. http://www.mysql.com/ http://jakarta.apache.org/tomcat/index.html
