A Real-Time Indoor Positioning System Based on RFID and Kinect

A Real-Time Indoor Positioning System Based on RFID and Kinect Ching-Sheng Wang, Chien-Liang Chen and You-Ming Guo

Abstract Global navigation satellite system is fully well developed in outdoor positioning nowadays; however, it cannot be applied in indoor positioning. The research of indoor positioning is rapidly increasing in recent years, and most researchers have paid much attention to RFID technology in indoor positioning; however, RFID is restricted by hardware characteristics and the disturbance of wireless signals. It is difficult to deal with the RFID positioning method. Therefore, this paper proposed an indoor real-time location system combined with active RFID and Kinect. Based on the identification and positioning functions of RFID, and the effective object extraction ability of Kinect, the proposed system can analyze the identification and position of persons accurately and effectively. Keywords RFID

Kinect Indoor Positioning Localization

1 Introduction Location-based service (LBS) has received considerable attention in recent years. Taking the popular outdoor positioning system, GPS, as an example, many practical positioning applications have been developed. However, GPS performs too poorly inside buildings to provide usable indoor positioning, thus, studies on indoor positioning systems have also received considerable attention. A comparison of the common indoor positioning systems using the methods of RFID, WiFi,

C.-S. Wang (&) C.-L. Chen Y.-M. Guo Department of Computer Science and Information Engineering, Aletheia University, Taipei, Taiwan, ROC e-mail: [email protected] C.-L. Chen e-mail: [email protected]

J. J. (Jong Hyuk) Park et al. (eds.), Information Technology Convergence, Lecture Notes in Electrical Engineering 253, DOI: 10.1007/978-94-007-6996-0_61, Ó Springer Science+Business Media Dordrecht 2013

575

576

C.-S. Wang et al.

Ultrasounds, Bluetooth, and WIMAX, [1] found that RFID is the best choice for indoor positioning in consideration of accuracy and cost. The research of RFID indoor positioning is increasingly in recent years. For example, The SpotON system first applied the positioning concept of GPS to indoor positioning and used the principle of Received Signal Strength Indication (RSSI) to calculate distance [2]. And Ni et al. proposed the famous LANDMARC positioning system [3]. Then, the VIRE, as proposed by Zhao et al. [4], the LEMT, as proposed by Yin et al. [5], are RFID positioning systems that aim to improve the positioning effect of LANDMARC. Besides, Wang et al. [6, 7] have proposed different positioning improvement mechanisms. However, even RFID is applicable to the indoor positioning, due its hardware characteristics, namely, wireless signals are likely to be disturbed by indoor environments and furnishings, the positioning accuracy limits of the RFID positioning system are difficult to address. Therefore, this paper implemented an accurate indoor real-time location system using the motion-sensing function of Kinect released by Microsoft. As the Kinect can accurately extract persons, even for different heights, body types, and skin colors, it is very applicable to indoor real-time location positioning. Although Kinect is limited by infrared detection distance, the proposed system is complemented by the embedded Kinect RGB color camera; thus, it still can implement dynamic tracking of remote persons. It is also integrated with the ID function of RFID, and a precise person positioning and identification system is completely implemented. The remainder of this paper is organized as follows. Section 2 reviews past works on indoor positioning, including RFID positioning, video tracking, and Kinect positioning; Sect. 3 introduces the overall architecture and positioning mechanisms of the proposed positioning system; Sect. 4 describes the implementation and experimental results of the proposed system; Sect. 5 gives conclusions.

2 Related Works The research on dynamic video tracking has become more mature in recent years. Furthermore, some positioning studies have combined RFID and video tracking. Germa et al. [8] proposed a system for identifying persons using RFID and video tracking. They validated the feasibility of combining images with RFID techniques, and designed a system for tracking and identifying persons. However, this system requires a functionally complicated mobile robot and RFID reader equipped with multidirectional antenna, which has high cost. Mandeljc et al. [9] proposed a system using UWB (ultra-wideband) and video for indoor person positioning. Although this system has good positioning effect, multiple UWB equipment and cameras are required to be installed at the scene, thus, the system construction cost is very high. Wang et al. [10, 11] have designed a system for real-time tracking and positioning of indoor persons by successfully combining image with RFID technologies.

A Real-Time Indoor Positioning System Based on RFID and Kinect

577

However, it is found in experimental experience that there remain person overlapping and shielding problems, and the results of positioning are likely to be influenced by body height and type, thus, the implementation effect requires improvement. The occurrence of Kinect solves the aforesaid problems. As Kinect can accurately extract persons, it will not have failed identification resulted from different heights, body types, and skin colors, and the image processing chip embedded in Kinect can greatly increase the efficiency image processing, which renders it very applicable to the implementation of an indoor person positioning system. Some studies have used Kinect to design person positioning systems. Schindhelm [12] proved that the Kinect was very applicable to indoor positioning of public buildings. Nakano et al. [13] also proposed using Kinect for indoor person positioning. However, the aforesaid systems lack the function of person identification. This paper combines the accurate person positioning ability of Kinect with the person identification function of RFID to complete an indoor positioning system for accurate positioning and person identification.

3 System Architecture and Positioning Mechanisms The positioning system proposed in this paper is consisted of two parts, including an active RFID positioning mechanism and a Kinect positioning mechanism (system architecture is as shown in Fig. 1). For the RFID positioning part, we placed multiple active RFID Repeaters in the scene in advance, and then the tester carried the active Tag while moving among the various blocks, meanwhile, the Reader integrated the Tag signals forwarded by all the Repeaters, and analyzed the signal strength scales of the Tag signals. Afterwards, the system defined the signal strength reference values of various positioning blocks according to the collected signal strength information, and stored related data in the database to complete the initialization analysis of a positioning environment. Through immediate addressing, the preliminary positioning of RFID can be completed by analyzing the Tag signals collected by all the Repeaters and matching the signal strength reference values in the database. In the Kinect positioning system part, provided that Kinect is mounted high in the positioning area, the position information of moving persons can be instantly extracted by means of the accurate person detection function of Kinect, and then the actual positions of persons in the scene are determined by coordinate transformation equation. Finally, the preliminary positioning result of the RFID positioning system is compared with the positioning result of the Kinect positioning system, and pairing is completed on the principle of minimum distance, meaning the personal information in RFID is perfectly integrated with the precise positioning result of Kinect.

578

C.-S. Wang et al.

Fig. 1 Schematic diagram of system architecture Reader

RFID Signal Database

Repeater

RFID Positioning Mechanism

Tag

Kinect Positioning Mechanism

3.1 RFID Positioning Mechanism When the Repeaters are denser in the RFID positioning system the positioning accuracy is higher; however, this increases cost and signal collision. Therefore, the proposed system only adopts appropriate RFID equipment that considers both accuracy and construction costs, uniformly places equipment within the positioning environment, and first conducts preliminary person positioning. Afterwards, the positioning system for accurate positioning and identification is completed by the precise positioning ability of Kinect. The main mechanisms of RFID positioning in this paper include a signal classification mechanism and a sensing overlap area analysis mechanism, as described below: Signal Classification Most RFID positioning systems directly use signal strength as the basis of positioning; however, this method sometimes neglects the differences among the signal strengths of various Tags, whereas, the signal classification mechanism proposed in this paper can set appropriate signal strength intervals according to the individual differences among environments and Tag signal strengths, thus, effectively enhancing the stability of the positioning system. The proposed system classifies signal strength into four levels according to the distance between RFID Repeater and Tag, and the signal strength values received by various Tags at 0.5 m, 1.5 m, and 3 m equidistance, are tested in the positioning field (as shown in Fig. 2). According to the data in Fig. 2, the signal strength value of individual Tags have a different RSSI value due to the differences in Tag and in environment; therefore, using fixed signal strength as the standard to estimate distance often results in considerable positioning errors. Therefore, the proposed system receives the signal strength of different Tags at different distances, and filters occasional too strong or too weak unstable signals as an effective reference frame. When the signal strengths of all the Tags are collected and analyzed, the appropriate signal strength interval for each Tag can be set (as shown in Table 1) as a reference for subsequent determination of sensing range, thus, enhancing the stability of RFID positioning.


579

Fig. 2 Records of signal strengths of different tags at different distances. (a) Tag1 signal strength (b) Tag2 signal strength

Table 1 Signal strength intervals corresponding to signal levels of individual tags Signal level Distance between RSS of Tag1 RSS of Tag2 RSS of Tag3 repeater and Tag (dBm) (dBm) (dBm) L0 L1 L2 L3

\0.5 m 0.5–1.5 m 1.5–3 m [3 m

-40 -58 -65 -68

-45 -50 -58 -65

-45 -55 -60 -70

Sensing Overlap Area Analysis Figure 3 shows the schematic diagram of the overlapped sensing areas of the proposed system, with an active RFID Repeater laid at intervals of 3 m within the positioning space, where the circular area in a radius of 3 m centered in the position of Repeater is the sensing range, and the positioning space is divided into multiple independent sensing overlapped areas. Afterwards, the positioning of the sensing overlap areas is completed by analyzing and matching the signal strength scales received by adjacent Repeaters. When the system locates sensing overlap areas, the centers of various blocks will be used as locating points to complete the preliminary positioning of relative positions, which are then integrated with the results of Kinect to determine the final locating points. There are five types of sensing overlapped areas in the proposed system, as described below. Red block. It means the position is very close to a certain Repeater, thus, the signal strength detected by the Repeater is level L0; Brown block. It means the signal strength detected by Repeater2 has not reached level L0, but has reached level L1; the signal strength detected by Repeater1 and Repeater4 is level L2; while that detected by Repeater3 is level L3; Green block. It means the signal strength detected by Repeater1, Repeater2, Repeater3, and Repeater4 is level L2. Blue block. It means the signal strength detected by Repeater3 and Repeater4 is level L2; the other Repeaters have not detected signals above level L2. Yellow block. It means the signal strength detected by Repeater1 is level L2; the other Repeaters have not detected signals above level L2.

580

C.-S. Wang et al.

Fig. 3 Schematic diagram of sensing overlap areas of RFID positioning

3.2 Kinect Positioning Mechanism Microsoft Kinect has three lenses, the middle lens is an RGB color camera, with an infrared transmitter and an infrared camera placed on both sides, forming a depth sensor. In terms of the Kinect positioning mechanism in this paper, a Kinect is mounted in the positioning scene to instantly extract the position information of moving persons, and then, the actual positions of persons in the scene are calculated by the coordinate transformation equation, matched with the preliminary positioning result of the RFID positioning system, finally, personal ID is integrated with accurate positioning. The proposed system divides Kinect positioning into three positioning modes, as based on the operating characteristics of Kinect, as described below: Within 4.5 m from Kinect (Including Skeleton Information) When a person enters into the 4.5 m sensing range of Kinect, the proposed system uses Kinect SDK to efficiently extract the skeleton information of the person, and uses the Ankle Left of skeleton as the positioning coordinates. When person positioning is completed, the system marks it with a blue frame (as shown in Fig. 4a). Within 4.5 m from Kinect (Excluding Skeleton Information, Including Depth Information) Since the officially released SDK of Kinect only extracts the skeleton information of two persons, when there are more than two persons in the 4.5 m sensing range of Kinect, the proposed system uses the depth information extracted by Kinect to determine the lowest left endpoint of the persons as the positioning coordinates. When person positioning is completed, the system marks it with a green frame (as shown in Fig. 4b). Beyond 4.5 m from Kinect (Excluding Skeleton and Depth Information) When a person is beyond the 4.5 m sensing range of Kinect, the proposed system uses the RGB color camera embedded in Kinect for person positioning by image processing. Traditional image processing modes often require complex and time consuming calculations, while the proposed system excludes the aforesaid two kinds of located persons before image processing-based positioning, which can effectively enhance system effectiveness. When person positioning is completed, the system marks it with a yellow frame, and the center point of the lower margin of the frame is used as the positioning coordinates (as shown in Fig. 4c). When the


581

Fig. 4 Schematic diagram of Kinect positioning result. (a) Skel information (b) Depth information eton (c) RGB video (d) Integration result

three modes of positioning are completed, the system will automatically integrate all the positioning information (as shown in Fig. 4d).

4 Experimental Results The implementation environment for the proposed system is an 8M 9 5M laboratory (as shown in Fig. 4a). The experimental equipment includes a notebook computer equipped with an active RFID Reader, four RFID Repeaters mounted within the scene, several RFID Tags carried by users, and one Kinect. The RFID equipment of the proposed system adopts the RFID chip module of Texas Instruments, and indoor layout and signal loss tolerance must be tested and set before implementing positioning (including signal collision, attenuation model,

582

C.-S. Wang et al.

interference of electromagnetic wave, etc.). The Kinect positioning system is composed by WPF (Windows Presentation Foundation). First, the proposed system uses the RFID positioning mechanism proposed in this paper to obtain the ID information of persons and preliminary positioning results. Afterwards, the accurate positions of persons are obtained by the Kinect positioning mechanism, and the result of Kinect positioning is used as the final positioning coordinates. Finally, the pairing is completed on the principle of minimum distance, where the personal information in RFID is perfectly integrated with the precise positioning result of Kinect. The steps and experimental results of combining RFID positioning with Kinect positioning are as described below: Step 1: RFID positioning: Fig. 5a shows the actual positioning scene, Fig. 5b shows the positioning result of RFID. The blue squares in Fig. 5b are the positioning results calculated by the RFID positioning system (including ID information). It is found in Fig. 5 that, the positioning results of RFID are slightly different from the actual positions, thus, the proposed system integrates the positioning results of RFID with the positioning results of Kinect in order to improve positioning accuracy. Step 2: Kinect positioning: Fig. 6b shows the positioning results of Kinect. The red squares in Fig. 6b are the positioning results calculated by the Kinect positioning system (excluding ID information). The proposed system uses Kinect SDK to effectively extract the skeletons of persons, depth information, and RGB video in the scene, and determines the actual positions of persons in the scene by coordinate transformation. When person positioning is completed, the system uses green, blue, and yellow frames to mark the persons identified by the three kinds of Kinect positioning modes (as shown in Fig. 6a).

(a)

(b)

ID 01 ID 03 ID 02 Fig. 5 Schematic diagram of RFID positioning results

ID 04


(a)

583

(b)

Fig. 6 Schematic diagram of Kinect positioning results

(b)

(a)

ID ID 03 ID 02

01 ID 04

ID 01 ID 03 ID 04 ID 02

Fig. 7 Schematic diagram of pairing and integration result

Step 3: Integration of positioning results: Fig. 7 a shows the pairing process of RFID and Kinect positioning. The proposed system completes the pairing of RFID positioning results (blue squares) on the principle of minimum distance, as based on the positioning results of Kinect (red squares), and integrates the corresponding ID information. After pairing, the final positions displayed by the system are the coordinates calculated by Kinect positioning and the corresponding ID information (as shown in Fig. 7 (b)).

584

C.-S. Wang et al.

5 Conclusions This paper proposed an indoor real-time location system combined with active RFID and Kinect. RFID signal classification and sensing overlap area mechanisms were used to complete the preliminary positioning of persons, combined with the stable and accurate person extraction ability of Kinect, in order to implement accurate personal identification and positioning. The experimental results proved that, in addition to implementing real-time personal identification, the proposed system can effectively enhance the accuracy and stability of person positioning. In addition, the proposed system only uses one Kinect, one RFID Reader, and several Repeaters and Tags, thus, effectively reducing the construction costs of an indoor positioning system. Acknowledgments The authors would like to thank the National Science Council of the Republic of China, Taiwan for financially supporting this research under Contract No. NSC-1012221-E-156-013 and NSC-99-2632-H-156-001-MY3.

References 1. Benavente-Peces C, Moracho-Oliva VM, Dominguez-Garcia A, Lugilde-Rodriguez M (2009) Global system for location and guidance of disabled people: indoor and outdoor technologies integration. In: The fifth international conference on networking and services, IEEE Press, Valencia, p 370–375. 2. Hightower J, Want R, Borriello G (2000) SpotON: an indoor 3D location sensing technology based on RF signal strength. UW CSE Technical Report (2000) 3. Ni LM, Liu Y, Patil AP (2003) LANDMARC: indoor location sensing using active RFID. In: The first IEEE international conference on pervasive computing and communications, Texas, pp 407–415 4. Zhao Y, Liu Y, Ni LM (2007) VIRE: active RFID-based localization using virtual reference elimination. In: The 2007 international conference on parallel processing, Xian, China 5. Yin J, Yang Q, Ni LM (2008) Learning adaptive temporal radio maps for signal-strengthbased location estimation. IEEE Trans Mob Comput 7(7):869–883 6. Wang CS, Huang CH, Chen YS, Zheng LJ (2009) An implementation of positioning system in indoor environment based on active RFID. In: The second IEEE international conference on Ubi-media computing, Taipei, Taiwan, p 71–76 7. Wang CS, Huang XM, Hung MY (2010) Adaptive RFID positioning system using signal level matrix. World Acad Sci Eng Technol 46(135):746–752 8. Germa T, Lerasle F, Ouadah N, Cadenat V (2010) Vision and RFID data fusion for tracking people in crowds by a mobile robot. In: The computer vision and image understanding, vol 114, no. 6. Elsevier, New York, pp 641–651 9. Mandeljc R, Kovacˇicˇ S, Kristan M, Perš J (2013) Tracking by identification using computer vision and radio. Sensors 13(1):241–273 10. Wang CS, Chen CC (2011) Flexible localization and identification system based on RFID and vision tracking technologies. In: 2011 international symposium on computer science and society, Kota Kinabalu, Malaysia, p 356–360 11. Wang CS, Cheng LC (2011) RFID & vision based indoor positioning and identification system. IN: The third IEEE international conference on communication software and networks, Xian, China, pp 506–510


585

12. Schindhelm CK (2012) Evaluating SLAM approaches for Microsoft Kinect. In: The eighth international conference on wireless and mobile communications, Venice, Italia, p 402–407 13. Nakano Y, Kai K, Izutsu K, Tatsumi T, Tajitsu K (2012) Kinect positioning system (KPS) and its potential applications, 2012. In: International conference on indoor positioning and indoor navigation, Montbeliard-Belfort, France