An Approach for Object Detection in Android Device - IEEE Xplore

4 downloads 302464 Views 417KB Size Report
objects in an image implemented on android platform. Color images are converted to binary images using the thresholding technique. Morphological opening ...
2014 2014 Fifth Fifth International International Conference Conference on on Signals Signal and and Image Image Processing Processing

An Approach for Object Detection in Android Device Savitha G Asst. Professor Dept of MCA, NMAMIT, Nitte

Venugopal P S Asst.Professor Dept of CSE, NMAMIT, Nitte

Dr. Sarojadevi Professor, Dept of CSE NMIT, Bangalore

Abstract-Object detection is a computer technology related to computer vision and image processing that deals with detecting instances of semantic objects of a certain class in digital images and videos. An attempt is made in this study to detect and extract objects in an image implemented on android platform. Color images are converted to binary images using the thresholding technique. Morphological opening and closing filters are used in sequence for object detection. Contour based learning techniques are adopted for drawing contours of the objects detected. In the process objects are extracted and stored in an array for further analysis. OpenCV functions are used to implement these algorithms on android.

To accomplish this, objects in images should be identified and detected. Computer vision techniques are used for detecting and tagging objects in the image. a. Scope In this study object tagging application is proposed. Users can instantly take a picture through a camera and any objects in picture can be tagged with the associated information. Users are allowed to send the image to any other users without attaching the information manually. Receivers when taps on any object in image the associated information are visible. This application is very useful for travelers, who can instantly send the tagged images to any other user.

Keywords- Android, Morphological filters, Image tagging, Least Significant Byte technique

II. RELATED WORK Object detection using image processing algorithms on Android is an emerging concept. Hence only few published literature were available. However, the available literatures mainly concentrate on desktop applications.

I. INTRODUCTION Mobile phones are becoming convergent platform for personal sensing, computing and communication. Integration of these facilities in the mobile will make communication more efficient and effective. Android is a development platform for the mobile application which is having the maximum market share among the mobile technologies.

There are many methods proposed for contour based component extraction for desktop application. Ommar et.al have analysed contour based object detection and have used logical representation of contours for object detection and also presented a non-parametric representation of contours, curvatures and junctions which enables their accurate localization [1]. ChengEn lu and others have studied and stated that contour based object detection can also be done from cluttered environments. Given a contour model for a class of objects, it is first decomposed into fragments hierarchically. It is then grouped into part bundles and finds the similarity between these bundles. Based on the similarity object is detected [2].

Detecting objects in a given image is the first step in image tagging applications. The information to be associated with the image is tagged to the detected objects. Thus user can get the information as and when required. So far object detection is done by various methods for desktop applications. Identification of objects in an image on android platform is not fully developed. The present study is focused on detecting objects in an image which later can be used for various other purposes. Objects once detected can be saved and is available for further use. In the present study image processing algorithms are considered for detecting objects.

Jammie shotten and others have studied that object detection can be done using only local contour-based features. Here a two stage supervised learning architecture is proposed [3]. For inserting information in various formats, there exists a large variety of steganography techniques. Darshan Mistry has done a detailed study of LSB technique. The Least Significant Bit (LSB) embedding technique suggests that data can be hidden in the least significant bits of the cover image. This technique can be used for inserting text in 24-Bit, 8-Bit, Gray scale format. Different embedding techniques are

Tagging refers to attaching the identified information or data to the object. Image tagging refers to attaching the related text with the image which helps users as and when required. Though applications related to tagging in maps have been developed, there is still scope for tagging objects in images on mobiles. Tagging should be done in appropriate place in the captured image. It should be done in such a way that the tagged information is available for users anywhere, anytime. 978-0-7695-5100-5/13 $31.00 © 2013 Crown Copyright DOI 10.1109/ICSIP.2014.6

Dr. Niranjan Chiplunkar Principal, NMAMIT Nitte

9

analysed and the evaluation for various file formats are presented [4]. Nagaraj V Dharwadakar have studied the method of LSB watermarking scheme and proposed a method of storing variable number of bits in each pixel based on the actual color value pixel. Equal or higher the color value of channels with respect to intensity of pixel stores higher number of watermarking bits [5].

algorithms have been used for the purpose of object detection and extraction. After the object is extracted individual objects are placed in an array for purpose of tagging using LSB technique of steganography. III. METHODOLOGY Structural element is the fundamental part in mathematical morphology. It is defined as configuration of pixels on which an origin is defined. Morphological filter is applied which consists of probing each pixel of the image using structuring element. When the origin of structuring element aligns with given pixel and all pixels intersecting the structuring element are included in the set. Structuring element is of simple shapes like square, circle etc. Erosion and Dilation are two fundamental morphological operators.

Luis Malagon-Borja, Olac Fuentes has presented an object detection system and its application to pedestrian detection in still image, without any prior knowledge of image. Here first classifier examines each location in the each location in the image at different scales. Then the system tries to eliminate false detections based on heuristics. The classifier works on the basis of Principal Component Analysis (PCA) which can compress the images that are used to compute the principal components [6]. Joel Nelson David has worked on tagging features in open street map using text recognition with an android smart phone. This application extracts identifying text from image taken with the phone and uses its GPS to tag the object on the map. In this application text recognised is important since they are extracted and used for tagging. [7] Tagging is basically done using the text in the image which is a drawback since images without text cannot be tagged.

Erosion replaces current pixel with the minimum pixel value found in defined pixel set where as Dilation replaces the current pixel value with the maximum pixel value found in defined pixel set. Since the binary image contains only black (0) and white (255) pixels, each pixel is replaced by either white or black pixel. Erosion and dilation filters are used in cascade to produce other operators namely Open and Close filters etc.. In this study opening and closing filters are used for detecting the objects in an image. i.e. separating foreground objects and background objects.

All the above published works are carried out in desktop. TagIT is an android application which tags the image on android platform (on mobile). It takes an image and tags information on different locations of image. User can write text or click an image and tag it to the present image. Tags can be moved within the area of the image. Tags can be edited or deleted as and when required. When the image is opened it highlights the region in the image where tags are available. Type of tags can also be changed anytime during tagging.

Open filter: It is obtained by the erosion of an image which is then followed by dilation. It is useful for removing small objects in the given image. Dst=open (src,element)=dilate(erode(src,element));

However the individual object tagging is not attempted, instead entire image is tagged. The information of individual objects is not revealed. Hence there is a scope for developing a methodology for tagging individual objects in an image.

Close filter: It is obtained by the dilation of an image which is then followed by erosion. It is used to remove small holes (dark regions) in image. Dst= close (src,element)=dilate(erode(src,element));

Tagged image can also be sent to the receiver. There is a need to maintain the position of tag at receiver side. But in TagIT application tags can be moved within the area of image. However, there is a limitation, since user at the receiver side may not know for which objects the tagging is done. In the present study it has been attempted to tag the individual objects within the image and maintain their position using computer vision techniques on android platform.

The closing filter connects together objects erroneously fragmented into smaller pieces, while the opening filter removes the small blobs introduced by image noise. Contour means edge or line that bounds a shape or object. In order to detect objects, contour needs to be drawn for the objects which are then extracted from the image and placed in an array. An image is obtained from a thresholding operation which was followed by open and close

An attempt is made to detect and extract the objects on android platform for the purpose of tagging. The study has been carried out using computer vision techniques. Different 10

morphological filters. Next, connected components are extracted by drawing contours.

thresholding operation is used which is followed by the application of open and close morphological filters. OpenCV offers a list of function to extract the contours of the connected components of an image.

The contours are extracted which consists of systematically scanning the image until a component is hit. From this starting point on the component, its contour is followed marking the pixels on its order. When the contour is completed scanning resumes at the last position until a new component is found.

Object detection on android involves the following steps: 1. 2.

Following are the steps involved for detecting and extracting the objects from the image.

3.

A binary image is produced using thresholding method Morphological opening and closing filters are used in sequence for object detection Contour based learning technique is used for drawing the contours for the objects detected and extracting them for further analysis

Input image Images contain a set of objects. One of the objectives of image analysis is to identify and extract those objects.

Thresholding function applied

For this purpose the first step is to produce the binary image showing where certain objects of interest could be located which is done using the thresholding function. An OpenCv function is applied on an image in order to create a binary image. Here, we threshold the image at the minimum value just before the increase toward high peak of histogram.

Binary image Use of Morphological open filter

Cv:Mat threshold Cv::threshold(image, threshold,60,255,cv::THRESH_BINARY);

Use of Morphological close filter

After obtaining the binary image, morphological filters are used for object detection. Morphology is used for differentiating the foreground objects and background objects. Foreground objects are represented by high (white) pixel values and background by low (black) pixel values need to be differentiated for detecting objects present in the scene.

Finding contours

Contours drawn (Object identified)

CV::morphologyEx function is used to apply higher-level morphological filters. Opening and closing filters of morphology are used for segmenting the image. First closing filter is applied which fills out the small holes of white foreground objects. This filter also connects several of the adjacent objects.

Objects extracted (Placed in array)

Function for using the closing operator is: cv::Mat element5(5,5,CV_8U,cv::Scalar(1)); cv::Mat closed; cv::morphologyEx(image,closed,cv::MORPH_CLOSE, element5); //Here 5X5 structuring element is considered for identifying the objects Then the opening filter operator is applied which eliminates small objects in the preview. All the small objects which are very small to contain structuring element is removed.

Image with objects identified Fig: 1. Detection of objects in an image An attempt is made to detect objects in a given image on android platform using contour based technique. Here connected components are extracted, that is shapes made of connected pixels in a binary image. For this purpose a simple 11

The function used to calculate opening filter is: cv::Mat opened; cv::morphologyEx(image,opened,cv::MORPH_OPEN, element5);

different intensities of light on different areas of an image. 24 Bit BMP image is used to tagging. This type of image files is best for applying the LSB technique since it is the largest type of file and is of highest quality. Because of its high resolution and quality it is easier to insert the text within the image. The least significant bit (means the 8th bit) of some or all of the bytes inside the selected rectangular box is changed to a bit of the message. When using a 24-bit image, a bit of each of red, green and blue colour is used for replacing the text.

Once the morphological operation is applied all the prominent objects in the preview are identified. Objects identified need to be extracted from the preview and contour based learning technique is used. This extracts the connected components, that is, shapes made of set of connected pixels in a binary image. OpenCV has a function which extracts the contours of the connected components of an image.

After inserting text data in appropriate position in the image, the encoded image is sent to the receiver. At the receiving side decoding of the image is done to extract the contents from the image. Text extracted is displayed at appropriate position in the image without causing any major distortion. Therefore objects are efficiently tagged with the related information which also can be sent to the receiver where it is extracted and displayed.

Function for this is: It is the cv::findcontours function. std::vector contours; cv::findContours(image, contours, // a vector of contours CV_RETR_EXTERNAL, // retrieve the external contours CV_CHAIN_APPROX_NONE); // all pixels of each contours Output here is a vector of contours, where each contour is represented as vector of cv::points. Also two flags are specified. The first one indicates that only external contours are required meaning holes will be ignored. The second flag specifies the format of contour. With this the vector will list all the points in the contour. With the flag CV_CHAIN_APPROX_SIMPLE, only the end points would be included for horizontal, vertical, or diagonal contours.

IV. EXPERIMENTAL SETUP Edge detection algorithms of computer vision techniques are used to detect the edges and tag objects. All the edges in the image were detected. But it is not useful for identifying individual objects in image. Identifying those objects from the detected edges is difficult and long process. (This sentence is to be removed) In the present approach it has been attempted to convert the colour image to binary image using thresholding function. Morphological open and close filters are used to distinguish between foreground and background objects. In this process objects are identified. Boundary for identified objects is drawn using contours. Contours are extracted and placed in an array.

Function for drawing the contours on an image is given as: // Draw black contours on a white image cv::Mat result(image.size(),CV_8U,cv::Scalar(255)); cv::drawContours(result,contours, -1, // draw all contours cv::Scalar(0), // in black 2); // with a thickness of 2

Android does not allow linking of other languages. However all other computer vision techniques are implemented in OpenCv. OpenCv functions can be called from android. After the application is developed it can be ported from laptop to mobile using USB chord. A mobile device driver is to be installed on android mobile to make it compatible with laptops.

Objects identified are placed in the array list (contour) consisting of all the objects extracted from the scene. i.e. contour[0] represents first object, contour[1] second object and so on . After object detection and extraction, tagging can be done for image using the following the method. A point is to be selected inside the detected object. The coordinates of this point are extracted. A rectangular box of defined size is drawn from the top left corner of the point selected.

V. RESULTS AND DISCUSSION In the present study object extraction is done on android using image processing algorithms. Binary image obtained from the color image is given as input to the morphological open and close function. Foreground and background objects are distinguished based on their intensity values. After image segmentation, all the major objects are extracted from the scene using contour based learning technique. This yielded an

Information is inserted inside the rectangular box in the image using LSB (Least Significant Byte) method of steganography. LSB is an efficient method to be used since it causes less distortion to the image. Image file consists of

12

In figure 4 a laptop is detected and a neat boundary is drawn for it. The document inside the laptop is also identified because of the difference of intensities if light. Higher intensity and lower intensity pixels are identified and grouped together which defines an object.

image in which objects detected were marked with a boundary which corresponds to each object in the image. These objects are extracted and put in an array. It is seen that small objects are eliminated and are not detected using these algorithms. Boundaries for major objects are drawn and these are considered for further analysis.

Original Image

Original Image

Resultant Image

Fig: 5. Trees detected from android cell

Fig: 2. Window detected from android cell

Figure 5 represents an outdoor image where all the trees are detected and marked with a boundary. Boundary is for the entire area of trees which proves to be efficient since whole lot of trees can be easily tagged further. It is seen that objects are identified based on the pixel intensity values. Similar pixel values are grouped together which ultimately form a object.

Figure 2 is a simple image where picture of the window in a plain back ground is taken. Boundaries are detected and marked for the windows.

Original Image

Resultant Image

Resultant Image

Fig: 3. Palm detected from android cell Original Image

Figure 3 is a simple image where a palm is detected in a plain background by simply observing the boundaries.

Resultant Image

Fig: 6.Objects are detected from android cell Figure 6 is a complex image where there are many objects in the image like monitor, keyboard, boxes etc. Each object is individually identified by drawing the contours at the edges of the object. An attempt is also made to determine the time required for processing the image by considering the CPU time taken for different types of image.

Original Image

The complexity arises because of large number of smaller objects present in an image. It is observed that as the image complexity increases the CPU time also increases.

Resultant Image

Fig: 4. Laptop detected from android cell

13

operators are successively used on binary images, then an image is obtained showing only the major objects in the scene. Contours drawn on the objects included all the major objects present in the scene or preview. It can be seen that complexity of the images is not the matter. Objects are efficiently identified and extracted in any situation. Foreground and background objects are effectively detected.

Table: 1. Processing time for different images

Image Type

Image Name

CPU Time Taken(secs)

Simple Image

Window image

6.00

Simple Image

Hand Image

6.683

Complex Image

Laptop Image

10.120

Complex Image

Trees Image

10.897

Memory utilization increases with the presence of more number of objects in an image. Also time consumed to process increases with presence of smaller objects in an image. This work can be further extended for tagging the objects which are detected. Tagging of objects can be done using the LSB (Least Significant Bit) technique of steganography.

Table: 2. Memory Utilization by Images

Image Name

Actual Heap Size

Allocated Space

Window Image

5.504

2.707

REFERENCES

Hand Image

5.504

[1] Joseph Schlecht, Björn Ommer “Contour-based Object Detection” Interdisciplinary Center for Scientific Computing University of Heidelberg , Germany, 2011

2.784

Laptop Image(Indoor Image)

5.504

2.800

Trees Image

5.504

2.899

[2] ChengEn Lu, Nagesh Adluru, Haibin Ling, Guangxi Zhu, Longin Jan Latecki, “Contour Based Object Detection Using Part Bundles”, Computer Vision and Image Understanding, March 2010 [3] Jamie Shotton, Andrew Blake, Roberto Cipolla, “ContourBased Learning for Object Detection”, Tenth IEEE International Conference on Computer Vision, 2005

(Outdoor Image)

[4] Darshana Mistry, “Comparison of Digital Water Marking methods”, International Journal on Computer Science and Engineering, 2010

Memory is one of the important criteria in android mobiles as the space available is limited. Hence it is relevant to determine the memory used by the application.

[5] Nagaraj V. Dharwadkar and B. B. Amberker, “Secure Watermarking Scheme for Color Image Using Intensity of Pixel and LSB Substitution”, Journal of computing, 2009

The memory utilized in various types of images is presented in Table 2. It is observed that the memory utilization increases as the number of objects are more in an image.

[6] Luis Malagon-Borja , Olac Fuentes, “Object detection using image reconstruction with PCA” Article in press published by Elsevier B.V. March 2007

In all the cases small holes or small objects are discarded and only the main objects in the image are detected and marked which are later used for tagging.

[7] Joel Nelson David DeBaun Steven Bell, “Mobile Address Tagging for OpenStreetMap” published by IEEE computer society,2009

V CONCLUSION AND FUTURE WORK Object detection is achieved using OpenCv and Java language and implemented on android device by using image processing algorithms. Larger objects got detected and indicated by marking their boundary. The extracted objects are stored in array list which can be used for further processing. It is observed that if opening and closing

14