INTELLIGENT PARKING SYSTEM DESIGN USING ... - IEEE Xplore

3 downloads 0 Views 295KB Size Report
INTELLIGENT PARKING SYSTEM DESIGN USING FPGA. Insop Song, Keith Gowan, Jason Nery, Henrick Han, Tony Sheng, Howard Li, Fakhreddine Karray.
INTELLIGENT PARKING SYSTEM DESIGN USING FPGA Insop Song, Keith Gowan, Jason Nery, Henrick Han, Tony Sheng, Howard Li, Fakhreddine Karray Department of Electrical and Computer - University of Waterloo 200 University Ave. W, Waterloo, ON Canada {isong,khgowan,jnery,hahan,tlsheng,h23li,karray}@uwaterloo.ca ABSTRACT In this research, we introduce FPGA based fuzzy logic controller (FLC). The benefit of using FPGA based FLC compare to software FLC is that the computation time reduction. Using this FLC, we design automated car back parallel parking system also with complete FPGA based controller. We build a small-scaled robot car and test on a real environment with VHDL code for wall following and parking. This paper describes the background of fuzzy logic system, the design of fuzzy logic system with FPGA and the experimental results. 1. INTRODUCTION For a novice driver, parallel parking is a very challenging task. This will become even more so, since the number of car grows annually, and parking space in urban area becomes tighter. To assist drivers, industry and academia have researched parking assistance systems [1,2]. In this research, we present a fast hardware-based (FPGA-based) automated parking assistant system. We design (1) a hardware accelerator (co-processor) for fuzzy logic controller that controls car movement and (2) a finite state machine that controls operation mode. This research develops two important hardware-based components and accomplishes a challenging task, a robot car parallel parking control. Both the hardware accelerator for fuzzy logic controller (FLC) and the FSM machine can be applied to other different types of applications. The hardware accelerator for FLC can be used many different types of automatic controller as a supporting co-processor to a main CPU or as a stand-alone component. There are various potential applications of this FLC hardware accelerator, since numerous products use FLC, such as a rice cooker to a steel pressing machine. Also a complicated FSM can be used for many consumer and industrial devices, and it can replace a microcontroller. As a matter of fact, the automotive industry started look at FPGA devices of its low silicon cost and existing reference designs [3]. In our previous work, a software-based (C++ program) real-time parking system was developed [2]. Multi-thread application was designed based on real-time OS (RTOS).

c 1-4244-0 312-X/06/$20.00 2006 IEEE.

To accomplish RTOS based multi-thread real-time application, a CPU and memory are essential. In order to guarantee that all (or most) of the tasks’ deadline are to be met, a priori schedulability test should be performed, in case of hard (or semi hard) real-time applications. In addition, sharing resource protocol and interprocess communication are also carefully designed and and tested to avoid deadlock and priority inversion. Our application scenario, a robot car control, does not require very high-speed interactions among tasks (threads); however, this does not mean that we can be free from the above mentioned real-time design requirements: schedulability test and resource protocol test. One of the common mis-concepts of real-time systems are a real-time system runs very high-speed not slow-speed. It is not; a real-time system is the system that should meet tasks’ deadline, either fast or slow. In this regards, our robot car system is a real-time system, i.e. the robot should turn within certain time to avoid crash, a sensor data should be processed periodically as examples. We did use a CPU and memory, and real-time design tests were performed in our previous research [2]; however, in this research we want to carry out the same task using hardware-based method in order to use low-cost silicon chip and fulfill real-time related design requirements with less efforts. These methodological reasons are the main reasons that we perform this hardware-based research. Also hardware-based design reduces throughput time dramatically compared to softwarebased design. Based on previous design experiences [2] of fuzzy logic controller and parking expert knowledge, we re-design the system with a complete hardware-based way in order to accomplish more prompt and reliable system outputs, in addition to the above mentioned methodological reasons. One of the important advantages of designing hardware-based system is that it responses much faster then software-based system in most cases. It is because that computation is performed in parallel with logic gates on a silicon unlike the way of sequential computations in software-based system. Hardware based system is built on FPGA (Field Programmable Gate Array) chips with VHDL (Very high speed integrated circuits Hardware Description Language) language.

VHDL describes hardware logics in gate level circuits. We use Altera’s Startix II FPGA board [3]. The parking assistant system is fully autonomous and intelligent. It consists of two main components. A central controller keeps track of external environmental information as well as internal states. A Fuzzy Logic Controller (FLC) [4] controls car to drive straight with predetermined distance from external objects, the FLC is used as a basic module for the central controller. A reduced-scale robot car is built for experimental test. The car is equipped with steering mechanism, speed control mechanism, and distance measuring sensor data acquisition systems. Thus, the car steers exactly as a full scale car (with differential gear), and it moves forward and backward with different speeds. Five sensors and ADC measure the distance between the car and external environment, such as other parked cars or curbs. A FLC simulator is also designed on PC to develop FLC. The simulator has two modules: first module is for FLC simulation, and the second module is for FLC VHDL code generator. The simulator is designed to test FLC and parking operation. Once we find proper FLC parameters based on simulations, the simulator is able to generate FLC VHDL code based on predefined code template with corresponding parameters. The VHDL template is designed in advance, and the code generator is used to expedite integration time. In experimental test, first we develop the fuzzy logic control for wall following, which maintains constant distance from environments. We design two different types of wall following: short distance wall following and far distance wall following for different environmental situations. We test FLC wall following on different shapes of wall: straight, curved, or corner. Then a finite state machine (FSM) of the central controller is designed for back parallel parking. We design and test back parallel parking with various sizes of parking spaces, and the car performs reliable and fast parallel parking. In this paper we consider a complete FPGA-based intelligent parking system deign using FLC and central controller. The organization of this paper is as follows: In Section II, we describe the background of Fuzzy system in detail. In Section III, we explain how do we design FLC for the robot car system. In Section IV, we show a FPGA design and the robot car design, and in Section V, we describe experimental results. Finally, in Section VI we discuss the results of the paper and provide possible future research areas.

is not easy to exactly modeled by the various complexity of the system parameters. Thus, in such a complexity involved case, FLC is very suitable. At the beginning of designing fuzzy systems, we should collect fuzzy IF-THEN rules from human experts or domain knowledge. The fuzzy logic system shown in Figure 1 is a simple diagram of fuzzy system that is used in this research. Its inputs and outputs are fuzzy sets; without loss of generallity, the problem of this type is that most of the engineering systems have crisp-valued variables, not fuzzy value. Thus, we need fuzzifier that converts crisp input to fuzzy input. Then fuzzy inferencing is performed using the fuzzified input values and fuzzy rule base. Afterwards defuzzifier finally converts fuzzy value to crisp output. We use Takagi-Sugeno-Kang (TSK, or also called Sugeno model) proposed real-valued variables inputs and outputs [4] as shown a Weighted Average block in Figure 1. TSK fuzzy system can be defined as a weighted average of the values in the THEN parts. In short, an important contribution of fuzzy systems theory is that it provides a systematic procedure for transforming a knowledge base into a nonlinear mapping [5]. The applications of the fuzzy systems are countless, such as domestic appliances, an digital image stabilizer, automobile control systems, subway control systems, and manufacturing control systems. The fuzzifier maps from a crisp point, x ∈ U , in to a fuzzy set A in U . A fuzzy rule base consists of a collection of fuzzy IF-THEN rules as (2). The defuzzifier maps from fuzzy sets in V to a crisp point y ∈ V . There are many defuzzifiers, but followings are three possible choices, such as Maximum, Center of gravity, Center average, and Modified center average. Since we use TSK type FLC, the defuzzifier is as in (4). The details of this equation will be explained in the next Section. In this section, we briefly review the fuzzy system. Fuzzy system composed as follows: Fuzzy rule base, Fuzzifier, and Defuzzifier. The importance of fuzzy system is its incorporation with linguistic information in a natural and systematic

2. BACKGROUND: FUZZY SYSTEMS Fuzzy logic controller (FLC) can produces an approximate control output for systems that are complex or non-linear, which normally are difficult to modelled and solved analytically. In addition, FLC can use an inferencing method of human expert knowledge [4]. In our system, car driving task

Fig. 1. Basic configuration of fuzzy systems with fuzzifier, fuzzy rule base, and defuzzifier

membership function

Z

NEG

POS 1.0

-8

-4

0

4

8

Fig. 2. Sensor location on the side of the robot car.

Fig. 3. Fuzzy membership function of input variable.

way. Following section describes a the fuzzy controller design and its experimental results.

to the side wall. f4 has this value; we obtain these zi values from expert knowledge and simulations.

3. INTELLIGENT PARKING SYSTEM DESIGN In this section, we describe FLC design of wall following module, which is a main part of parking task. This wall following task controls the robot car to maintain a given distance from side objects using FLC. Then, in the next subsection, a central controller is explained. The central controller is a finite state machine that include driving experts knowledge of performing back parallel parking. 3.1. Fuzzy Logic Controller Design for Wall Following Fuzzy logic controller is designed and simulated to maintain given distances from side objects, such as curbs or other cars. This wall following task is accomplished by measure the side distance from two locations of the car: front side and back side. As show in Figure 2, there are three sensors on the side of the robot, and there are two other sensors on each front and the back of the robot car. The main sensors that are used in wall following task are D3 , front side sensor and D4 , back side sensor. Dw is defined a given reference distance to be maintained as in Figure 2. With these three values, we define two input variables for FLC as in (1). x in (1) and in Figure 2 is a measure of the angle of the car reference to the side wall. It tells us that how steep the robot is with the angle towards the side wall. On the other hand, y in (1) and in Figure 2 is a measure of a distance shows how far the car is off from the given reference distance, Dw . x = D4 − D3 (1) y = D4 − DW Membership function is designed for two inputs, x and y, as shown in Figure 3, horizontal axis is measured distance. It is intentionally designed simple for hardware implementation of fuzzificaiton, so we only use three fuzzy variables: Z is zero, POS is positive, and NEG is negative. These membership functions are simple triangular type membership functions. Using human expert knowledge, fuzzy rule base is designed as in (2). θ is a steering angle of the robot car. For example, as in R4 if x, an angle measure, is ZERO and y, distance measure, is P OS, then we can turn steering much

R1 R2 R3 R4 R5 R6 R7 R8 R9

: IF : IF : IF : IF : IF : IF : IF : IF : IF

x x x x x x x x x

is ZERO and y is ZERO, T HEN θ is f1 is P OS and y is ZERO, T HEN θ is f2 is N EG and y is ZERO, T HEN θ is f3 is ZERO and y is P OS, T HEN θ is f4 is P OS and y is P OS, T HEN θ is f5 is N EG and y is P OS, T HEN θ is f6 is ZERO and y is N EG, T HEN θ is f7 is P OS and y is N EG, T HEN θ is f8 is N EG and y is N EG, T HEN θ is f9 (2) After fuzzification and fuzzy inferencing using fuzzy rule base, a final step has to be done, which is converting fuzzy values to crisp values in order to actually steer the robot to follow wall, i.e. defuzzification step. Defuzzificaiton is performed as in (4). fi are from fuzzy rule base and a weighting parameter wi is calculated from the membership operation shown in (3). wi is obtained by either “min” operation or “product” operation shown in (3). θ, crisp value, is obtained by (4). (3) wi = min[µAi (x), µB i (y)] ,where µAi (x) is the membership function of x and µB i (x) is the membership function of y. 9 i=1 wi · fi θ=  (4) 9 i=1 wi Using the FLC wall following, the robot car can follow short distance as well as far distance wall following task. 3.2. Central Controller Design for Parallel Parking To accomplish full automated parallel parking, we designed a central controller. It is composed with FSM that checks corresponding sensor values and controls the car steering and direction. It is a very complicated FSM; however, we try to simplify and show only important state operations and state transition conditions as shown in Figure 4. State 1 performs short distance wall following using FLC until a gap (a parking space candidate) is detected, which is transition condition A. Then in State 2, the robot changes its mode to far wall following also using FLC until the gap is ended, which is transition condition B. After the gap is finished, in State 3, based on time taken and the speed of the car, we can find out how big is the parking size. Thus, if the gap size is big enough for parking, then

C

Start

If space is not enough for parking

1

2

Short distance wall following

A If gap is detected

FLC is used

Far distance wall following

3 B

Measure gap size

If gap is end

FLC is used

D 6

5

Stop & move foward Front sensor triggered

G End

If space is enough for parking

FLC is used

F Back sensor triggerred

Move back & steer straight & left FLC is used

If too far from side curb

E Back & side back sensor triggerred

4

Move back & turn hard right

H

Fig. 4. A very simplified finite state machine of central controller. state moves to State 4. In State 4, the car moves back with hard right turn steering until it detects certain distances on back sensor and back side sensor. State 5, then, has initiated with straight and then gradual left steering, which is kind of wall following in backward movement. State 5 is finished when the back sensor detects very close distance. In State 6, the car moves forward using shorter wall following until front sensor is triggered. In short, the automated parking task is accomplished by the combination of FSM and FLC, and these are designed based on using human expert knowledge. 4. SYSTEMS DESIGN WITH FPGA In this section, design of robot car and VHDL for fuzzy logic control (FLC) and central controller are described. 4.1. Robot Car Design A RC (radio control) car frame is purchased and other additional circuits are integrated as shown in Figure 5. A main control FPGA board is Altera Stratix II board [3]. A daughter board is designed for power distribution and ADC & MUX circuits. Steering control is done by radio control servo controller, and its input is PWM (Pulse Width Modulation) signal. VHDL code is designed for PWM generation. Direction and speed control is done by a built in speed controller that comes with the car frame and body. It is also controlled by PWM signal, so another PWM generation VHDL code is designed. A 10-bit resolution high-speed parallel output ADC (ADC 10461 from National Semiconductor) is used to convert distance sensor signal. Since ADC10461 does not have enough channels for five sensors, we use multiplexer (MC14051B). ADC read timing and multiplexer selection are also done by VHDL code, so that FLC and FSM can access distance values from each sensors. Five infrared distance measuring sensors are used. Two kinds of sensors are used, one kind is a short range sen-

Fig. 5. A cut out picture of the robot car. Inside the robot, there are two boards: Altera board and custom made daughter board. sor (GP2D120 from Sharp) and another kind is a long range sensor (GP2D12 from Sharp). Three sensors are installed on side of the car, and two sensors are installed on front and back of the robot car, respectively. 4.2. VHDL Design for Intelligent Parking System There are two main VHDL modules: FLC and central controller. Also there are few other VHDL modules: ADC control, direction control & speed control, steering conversion from FLC output, and operation mode selection & display. We also add a moving average filter at sensor module to eliminate noisy inputs. In Figure 6, controller and FLC can be seen, and ADC control module that feeds distance information to the controller and FLC. fi (i fi ) values is feed by using lpm constant for easy debugging. Due to the limitation of the space, we only show FLC block in Figure 7. Two fuzzifiers are located in the bottom in Figure 7, and there are also min, multiply, adder, and divider modules, which computes final steering output. It takes only 21 clock cycles to produce fuzzy output. This is much reduced processing time compared to that of software based FLC in the previous research; it takes around hundreds of cycles. Much more importantly, in hardware accelerated FLC, the processing time is fixed and deterministic unlike the software couterpart. The hardware-based system uses 8,900 logic elements. We use many debugging purpose lpm constant blocks to modify parking parameters on the fly without recompile VHDL files. Also, we use SignalTap [3] for real-time monitoring of internal data. If we take out these development related modules, the number of logic elements will be reduced. 5. EXPERIMENTAL RESULTS In this section, experimental results are presented. First, simulator and code generator are presented, and actual robot run is showed. 5.1. Simulation and VHDL code generation With a simulator, a researcher can choose how many membership functions for input 1 and input 2, as well as their shapes. After the parameters are set, a simulated robot can

ADC_CNTL i_int o_RD i_data[9..0] o_SH i_reset o_data[11.0] i_clk o_mux[2..0]

FLC

i_f1[9..0] i_f2[9..0] o_steer[9..0] i_f3[9..0] i_f4[9..0] from i_f5[9..0] lpm_const i_f6[9..0] controller i_f7[9..0] i_f8[9..0] i_adc[9..0] i_f9[9..0] i_reset o_distance[9..0] i_distance[9..0] i_clk o_angle[9..0] i_angle[9..0] to i_param i_clk o_speed[9..0] direction i_reset pwm to mux

min_zero1_zero2 z1 i_clk i_input1[9..0] o_min[9..0] i_input2[9..0] to z2 min_zero1_pos2 steering i_clk pwm z1 i_input1[9..0] o_min[9..0] i_input2[9..0] p2 min_zero1_neg2 i_clk z1 i_input1[9..0] o_min[9..0] i_input2[9..0] n2 min_zero2_zero1 i_clk z2 i_input1[9..0] o_min[9..0] i_input2[9..0] z1 min_zero2_neg1 z2 i_clk i_input1[9..0] o_min[9..0] i_input2[9..0] n1 min_pos1_pos2 p1 i_clk i_input1[9..0] o_min[9..0] i_input2[9..0] p2 min_pos1_neg2 p1 i_clk i_input1[9..0] o_min[9..0] i_input2[9..0] n2 min_neg1_pos2 i_clk n1 i_input2[9..0] o_min[9..0] i_input1[9..0] p2 min_neg1_neg2 i_clk n1 i_input1[9..0] o_min[9..0] i_input2[9..0] n2

Fig. 6. VHDL modules for intelligent parking task.

multiply i_clk i_input1[9..0] o_val[9..0] i_input2[9..0] i_f1[9..0] multiply i_clk m2 i_input1[9..0] o_val[9..0] i_input2[9..0] i_f2[9..0] multiply i_clk m3 i_input1[9..0] o_val[9..0] i_input2[9..0] i_f3[9..0] multiply i_clk m4 i_input1[9..0] o_val[9..0] i_input2[9..0] i_f4[9..0] multiply i_clk m5 i_input1[9..0] o_val[9..0] i_input2[9..0] i_f5[9..0] multiply i_clk m6 i_input1[9..0] o_val[9..0] i_input2[9..0] i_f6[9..0] multiply i_clk m7 i_input1[9..0] o_val[9..0] i_input2[9..0] i_f7[9..0] multiply i_clk m8 i_input1[9..0] o_val[9..0] i_input2[9..0] i_f8[9..0] multiply i_clk m9 i_input1[9..0] o_val[9..0] i_input2[9..0] i_f9[9..0]

m1

be tested on a simulator. After few testing, the parameters are fixed, then FLC VHDL code is generated. This FLC VHDL code is integrated with other VHDL modules of the fuzzifier1 i_clk o_neg[9..0] robot’s central controller. Figure 8(a) shows a screen capture i_data[9..0] o_zero[9..0] o_pos[9..0] of one of the parameter setting menus. Figure 8(b) shows i_distance a screen capture of a robot run on simulator on a IBM PC. [9..0] fuzzifier2 o_neg[9..0] i_clk Figure 8(c) shows a screen capture of FLC VHDL code geni_data[9..0] o_zero[9..0] o_pos[9..0] eration. i_angle [9..0]

5.2. Back Parallel Parking First, we test wall following by FLC. In Figure 9(a), a short distance wall following mode is activated as long as there is no big parking space is existed on the side, such as up to number 1 in the Figure 9(a), the car maintains short distance from the side curb using FLC. After passing number 1 in Figure 9(a), the car changes its mode to the far distance wall following until it passes number 3, another close distance object. As shown in Figure 9(a), the car tries to maintain the predefined distance. After the number 3 in Figure 9(a), the car calculates a parking space that just passed, if it is long enough for parking or not. If it is long enough, the car changes its from state 3 to state 4 as in Figure 4. As in Figure 9(b), starting from number 4, the car backs with hard right turn until it has objects detected on side and back side sensors. Then it releases its steering to straight and to soft left as at number 5 in Figure 9(b). Then, FLC shorter wall following in backward direction is triggered and steers the car. It keeps backing up till an object is detected by the back sensor as at number 6 in Figure 9(b). As shown in Figure 9(c), the car moves forward till front sensor is triggered. It stops in the middle of the parking space. This forward movement is also helped by shorter distance wall following FLC. 6. CONCLUSIONS We design, simulate, and test FPGA-based automated parking system with both a simulator and an actual robot car. The advantages of using hardware vs. software-based design are as follows: (1) the computation time of hardware

m1 n1 m2 z1 m3 p1 m4 m5 n2 m6 z2 p2 m7 m8 m9

Adder:NumAdder i_clk i_input1[9..0] o_val[31..0] i_input2[9..0] i_input3[9..0]

Adder:NumAdder i_clk i_input1[9..0] o_val[31..0] i_input2[9..0] i_input3[9..0]

Adder:NumAdder i_clk i_input1[9..0] o_val[31..0] i_input2[9..0] i_input3[9..0]

Adder:NumAdder i_clk i_input1[9..0] o_val[31..0] i_input2[9..0] i_input3[9..0]

division i_clk i_num[9..0] o_val[9..0] FLC output i_den[9..0] o_steer [9..0]

Adder:DenAdder i_clk i_input1[9..0] o_val[31..0] i_input2[9..0] i_input3[9..0] Adder:DenAdder i_clk i_input1[9..0] o_val[31..0] i_input2[9..0] i_input3[9..0]

Adder:DenAdder i_clk i_input1[9..0] o_val[31..0] i_input2[9..0] i_input3[9..0]

Adder:DenAdder i_clk i_input1[9..0] o_val[31..0] i_input2[9..0] i_input3[9..0]

Fig. 7. VHDL modules of FLC. Inside details of FLC block in Figure 6 based design is much reduced compared to that of software based design. It is because that, in hardware designs, most of the logics are synthesized in parallel; thus operations can be processed parallel. (2) real-time design requirements are easier to met then ones with multi-thread software based design under RTOS. In future work, we want to integrate the FLC hardware accelerator (FPGA-based FLC), as a hardware thread, with a software & CPU based automatic control (or other FLC reqired) systems. Then, both flexible and fast response control system are expected, and less complicated real-time design requirements should be tested. 7. REFERENCES [1] E. Batista, “Park without dings – or drivers,” Wired News, August 25 2003. [Online]. Available: http://www.wired.com/ news/technology/0,1282,60154,00.html [2] I. Song, F. Karray, Y. Dai, M. Masmoudi, and B. Ghaddar, “An intelligent car-like robot parking system design and implementation,” in Proceedings of Third IEEE International Conference on Conference on Systems, Signals Devices, SSD, 2005. [3] “www.altera.com.” [4] F. Karray and C. W. D. Silva, Soft Computing and Tools of Intelligent Systems Design: Theory and Applications. Addison Wesley, 2003. [5] L.-X. Wang, A Course in Fuzzy Systems and Control. Saddle River, New Jersey: Prentice Hall, 1997.

Upper

2

1

3

(a) Simulation program screen shot. Rule base input menu.

(a) State 1, 2, 3.

4 5

6

(b) Simulation run screen shot.

(b) State 4 and State 5.

7

8

(c) State 6.

Fig. 9. Overlayed picture of automated parking operation. (c) FLC VHDL code generation screen shot.

Fig. 8. FLC simulation and FLC VHDL code generator screen shots.