a review: from keyword based image retrieval to ontology based ...

International Journal of Reviews in Computing 31st December 2012. Vol. 12 © 2009 - 2012 IJRIC & LLS. All rights reserved.

ISSN: 2076-3328

www.ijric.org

IJRIC E-ISSN: 2076-3336

A REVIEW: FROM KEYWORD BASED IMAGE RETRIEVAL TO ONTOLOGY BASED IMAGE RETRIEVAL 1

V.VIJAYARAJAN, 2M.KHALID, 3P.V.S.S.R. CHANDRA MOULI

1

Assistant Professor (Senior), School of Computing Science and Engineering, VIT University, Vellore. 2 Pro Vice-Chancellor, KIIT University, Bhuvanewar. 3 Professor, School of Computing Science and Engineering, VIT University, Vellore. E-mail: [email protected], [email protected], [email protected]

ABSTRACT This paper confers the evolution of the various image retrieval techniques starting from Keyword based Image Retrieval (KBIR) to Ontology Based Image Retrieval. It addresses various outstanding issues in both past and present. The rapid advancement of image data demands continuing the research and development of Image Retrieval. The exploration in image retrieval has its basis in keyword based retrieval and then evolved to retrieval techniques employing content based low-semantic features approach is that the complete knowledge cannot be mapped and hence there exists some gaps known as semantic gaps. To overcome these gaps, currently, the researchers are interested in Knowledge based retrieval using Ontology. The focus is towards the semantic features because the key words are very subjective, time consuming to search and also the low level features are very tough to map into high level concepts in user minds. Keywords: Keyword Based Image Retrieval (KBIR), Content Based Image Retrieval (CBIR), Semantic Gap, Ontology Based Semantic Image Retrieval.

model. The model is also largely controlled by factors such as dissimilarity of user base and retrieval time. Besides this extent, search data can be divided into the following types:

1. INTRODUCTION Image retrieval is a technique for searching images in large databases. It is the study concerned with the technique of searching and retrieving digital images from a collection of image databases. To perform image retrieval, user gives query as images, text as a keyword(s), and image links, then the system will retrieve images “similar” to the query. The comparison used for searching criteria could be meta tags, color spreading in images, region or shape attributes. In image search, searching of images is carried out based on combined metadata such as keywords and text that are annotated with each image. The Content-based image retrieval (CBIR) is one of the earlier technique of computer vision for image retrieval. CBIR targets at averting the use of textual descriptions. Instead, it fetches the images based on relationship in the content (textures, colors and shapes) with a user given query image or user denoted image features. It is important to perceive the scope and essence of image data in order to actuate the complexity of image search system

Archives: Ample amount of structured or semi structured similar data be relevant to particular topics in the past. Domain Specific Collection: A similar collection giving admittance to controlled users with very particular objectives. Instances of such accumulations are biomedical and satellite image databases. Enterprise Collection: A dissimilar collection of images disclosed to an arrangement and has access to users within the intranet. Personal Collection: A similar collection and is normally a small size, accessible initially to its owner, and stored on a local storage media.

1


ISSN: 2076-3328

www.ijric.org

Web: The www images can be easily accessed using internet. We can store these semi-structured, non-homogeneous images in large disk arrays.


properties that can be understood by perceiving the image itself. These two parts cannot be linearly separated, and both have to be taken into account when analyzing an image. Text-based methods can easily adopt with all paradigms up to some degree. Everything can be described with natural language but it is hard to solve the intended meaning of a textual description automatically by computers. For this reason, different semantically oriented techniques have been created to support text based information retrieval.

The image retrieval attracts researchers from the fields of image processing, multimedia, database applications, remote sensing, digital libraries, astronomy and other related areas [1, 2]. The computer vision and data base researchers are the two major research groups focusing on image retrieval from different perspectives wherein the former focuses on text based image retrieval and the latter on visual based image retrieval [3]. In the text based image retrieval, text can be used to explain the content of the image while in the visual based or content based image retrieval (CBIR) the visual features are used to explain the contents of images.

In the actual context of retrieval, plain text annotated photos respond similarly to plain text documents because both contain text, which can be exploited by conventional text-based retrieval techniques. The generic text based information retrieval is performed in such a way that initially a user gives a query that has 1 to m keywords. In metadata based image retrieval, the search engine compares the keywords with a set of images collected from a database and gives them priority values. For example, if the keyword is book, image ‘A’ contains two instances of book and image ‘B’ contains only one instance, then ‘A’ gets higher priority. The image annotated keywords are performed to the user in the reducing order of the priority. When the size of the image retrieval increases, it is similar to the classical problems of text based information retrieval. Irrelevant images are retrieved and the user has to use time filtering the information again, usually by browsing through the search results.

2. KEYWORD BASED IMAGE RETRIEVAL In conventional image retrieval system, the keywords are used as descriptors to index and retrieve an image. The key words does not convey better than the content of an image mean. Text based image retrieval techniques use text to describe the content of the image which often creates ambiguity and inadequacy in query processing and performing an image database search. The process of assigning meta data with captions or keywords to a digital image is known as automatic image annotation or automatic image tagging. This type of text based information retrieval is lexically motivated rather than conceptually motivated, which leads to irrelevant search results in information retrieval. Lexically motivated information retrieval means that text based retrieval operates on the word level but not on the meaning of words. But the very basic idea of ontology is that it is conceptually motivated. That means it can be applied to specify the actual meaning of things but not like words or strings.

The fitness of generic text-based retrieval is depicted in Figure 1 and the Table 1 describes the precision and recall for the same. When the recall gets higher the precision gets lower and vice-versa. The graph represents a usual relation between recall and precision. The fitness of recall and precision can be calculated with the equations (1) and (2). The main principle of the vector-model [4, 5] is to index documents that consist of text into a matrix including statistical information such as the amount of appearances of different words in a document and the place of the words in the document. This is fundamentally faster than to search directly from the documents and also with optimized character parsing techniques [6, 7]. The vector-model also provides the search algorithms a structured basis to start the reasoning with. It is a lexically motivated type of retrieval and keywords used are user given text only. These keywords are used to annotate the images.

2.1. Text Based Image Retrieval The basic document retrieval techniques may be used to metadata based image retrieval without modification, and also some techniques and methods in the aim of image retrieval can be utilized to document retrieval in general. In key word based image retrieval, the metadata that describes images can be roughly categorized into two parts. One part concerns with the tools used in the process of creating the image, art style of the image and the artist, price, and other explicit properties of the image. The other part describes what is actually in the image, the implicit 2

International Journal of Reviews in Computing

IJRIC

31st December 2012. Vol. 12 © 2009 - 2012 IJRIC & LLS. All rights reserved.

ISSN: 2076-3328

www.ijric.org

E-ISSN: 2076-3336

description, subject, creator and title. Qualified versions of DC have been created for particular domains like for describing art objects in museums [9]. However, an average user who is not-interested in the properties of the image uses mostly the topic field. In this case the results are similar to text based retrieval. The only difference is the way to set values for a number of other fields, which can have other kinds of value types. The planning of the fields is again very close to ontology based planning.

2.2. Field Based Image Retrieval The extension of text based retrieval is seen as field based retrieval in which only one field is used in annotation and retrieval. The field based approach (also called attribute based and feature based) describes and retrieves an item using one or more field value pairs. Commonly a metadata schema is

2.3. Structure Based Image Retrieval The extension of field based retrieval is seen as structure based retrieval paradigm [10]. In this method, field based approach is used which mainly uses an even structure of attribute value pairs. This method permits more complex descriptions implicating relations. For an instance, a definition of a part of a car can include a specification of its components. Each object elements can be specified again using various attributes like shape, size and material. Elements can even have elements themselves, for a instance a table have legs, and its sub elements can move up to the depth where an element cannot get any more particular sub elements. Ontology based design is the best way to construct structure based systems, and in general structure based systems can be called ontology based systems and will be discussed at the end of this paper. Table 2 will discriminate the types in keyword based retrieval which includes text based, field based and structure based.

Figure 1. Relation between Precision and Recall Table 1. Precision and Recall

Found x+y Discarded m+n Sum x+y+m+n

Relevant x

Irrelevant y

m

n

x+y

m+n

x x+n 0 ≤ recall ≤ 1

(1)

x x+ y 0 ≤ precision ≤ 1

(2)

recall =

precision =

3. CONTENT BASED IMAGE RETRIEVAL The CBIR (a query by image content QBIC or content based visual information retrieval CBVIR) is one of the computer vision methods for image retrieval, means to retrieve necessary digital images from image databases. “Content based search” will perform the analysis with the real contents of the image instead of metadata such as tags, keywords or descriptions annotated with the image. The word ‘content’ here may deal with shapes, color, textures, or some other details that can be obtained within the image itself. The web related image search engine relies on meta data and so this generates a lot of garbage results. Hence, CBIR is desirable in this case. Giving manual keywords to search images in a large database may retrieve wrong results. Also the process is expensive and may not identify every keyword that specifies the

described that gives a set of fields and few indication is given about the type of values that can be chosen to a particular field. The widely used schema and metadata template for describing online documents in general is the Dublin Core (DC) [8]. The fields of DC version1.1 are rights, coverage, relation, language, source, identifier, format, type, data, contributor, publisher, 3


ISSN: 2076-3328

www.ijric.org

image and hence it is inefficient. By providing a good indexing technique based on the actual

Text based image retrieval Text are annotated to describe the image content

Field based image retrieval One or more fields (image attributes) are annotated to describe the image content

Structure based image retrieval Complex descriptions involving relations are used to describe the image object contents

2.

Lexically motivated retrieval

Lexically motivated retrieval

Conceptually motivated retrieval

3.

Keywordstexts

Keywordsone or more fields

Keywordssentences having conceptual meanings

1.

E-ISSN: 2076-3336

level, measuring the closeness and ‘semantic gap’ contraction. Few models are proposed for a particular domain like finding the units inside a CAD-CAM database by using shape matching algorithms.

Table 2. Comparison within Keyword Based Image Retrieval

Sl. No

IJRIC

In CBIR, feature detection and extraction are low-level image processing operations. The process examines each pixel to detect if there exists a possible feature at that pixel, perform an initial operation on that image. Before performing feature detection, use the Gaussian kernel in a scale space to mild the input image, and estimate one or more image features usually represented using confined derivative operations. In certain circumstances if the image feature detection process looks and results with high computational time, then go for an another best algorithm to find only some image parts for the searching features. Once if the features have been obtained, a confined image segment surrounding the feature could be extracted using some of the image processing techniques. This process results in producing feature descriptor or feature vector. This extracted features help to perform a similarity matching in the CBIR retrieval process. The low-level feature extraction is defined in the following section.

contents of images may retrieve and produce accurate results. The content based image retrieval enriches the main utility of Picture Archiving and Communication System (PACS). It retrieves pictures with patterns of images instead of using alpha numeric indices.

3.1. Low-Level Image Feature To perform CBIR low-level features should be extracted first. The feature extraction may be done on the entire image or it may be done as region of interest. The simple technique used in image retrieval depends on global features. The human perception closely matches with the representation of images at region level [13]. To perform region based image retrieval, the foremost step is image segmentation. From the segmented region, the low level features like texture, color, shape or spatial location can be easily taken out. Based on the region features, it can easily be found the closeness matching within two images. The process of region based image retrieval has three parts.

The CBIR systems mostly suffer from “semantic gap” [11]. It is a gap between the high level image grasping of a human mind and the low level image estimation from computers. Recent CBIR techniques include the low level features like color, texture and shape and high level features like facial expressions. The modern techniques[12] used for diminishing the ‘semantic gap’ has the following varieties: describing high level semantics by utilizing object ontology, combining the query ideas with low level features with the help of machine learning tools, using relevance feedback (RF) as key to encircle them within the retrieval process for studying the users’ real motivation, producing semantic template (ST) to reinforce the best possible image retrieval process, combining the visual information of the image with textual details got from the web for providing better web based image retrieval process. There are some basic ideas that are mostly used in various CBIR techniques like image feature extraction at low

3.2. Image Segmentation Automatic process of performing image segmentation is a hard task. The academic techniques used in image segmentation are curve energy diffusion [14], evolution [15], and graph partitioning [16]. Most of the methods can be suitable only for images that have similar color regions, such as direct clustering methods in color space [17]. Such methods can suit for the retrieval systems that work with colors. But natural scenes 4


ISSN: 2076-3328

www.ijric.org

contain colors as well as textures. Applying segmentation in texture is difficult [18]. Even in texture based segmentation the estimation of texture model parameter is difficult. To overcome this ‘JSEG’ segmentation algorithm is used. Another algorithm called Blobworld segmentation is widely used one [19, 20]. Some segmentation algorithms use segmentation based on color, texture or both [20, 21, 22, 23, 24, 25]. These algorithms use K-Means for classification purpose. Same class blocks are grouped into the same region. The K-means with connectivity constraint (KMCC) is a segmentation work [25] for segmenting objects i n t h e i m a g e s . This algorithm utilization is trust on the necessity of the system and the usage of data set. It is hard to determine which algorithm provides better result. The result of JSEG is a texture color of similar regions, but the result of KMCC produces objects that are dissimilar. Still now, KMCC is computationally much comprehensive than JSEG. So, Blobworld and JSEG are mostly used algorithms.


identical, but in few cases they look very different. Depending on the segmentation results only the color features are selected. It is observed that average color is not a desirable choice whether the segmentation results objects that do not have similar colors. In most of the CBIR works the color images are not preprocessed. The appropriate color filters [26, 35, 36] are essential to enhance the retrieval efficiency because the color images are always damaged with noise. 3.3.2. Texture feature Few systems do not make use of texture features [25, 27, 33, 37, 38, 39] for image retrieval as like the color features. The texture is an important feature for describing high level semantics in image retrieval, because it provides essential details in image catalogue as it defines the context of many real world images like clouds, fruit skin, bricks, fabric and trees. The result of applying Wavelet transform [40] or Gabor filtering [41], confined statistical measures like world features proposed by [42] and six Tamura texture features [43] are the widely using texture features in image retrieval process. The regularity, line likeness, roughness, directionality, contrast and coarseness are the various Tamura features. Among them coarseness, directionality and regularity are the most important [43] features. These three are related to other are less effective with respect to texture description.

3.3. Various Low-Level Image Features In the various categories of algorithms, very few can be applied in real time image retrieval with high level semantics that are: 3.3.1. Color feature It is the commonly adopted feature in image retrieval. Various color spaces are used for defining colors. Those color spaces are used depending on different applications. Lot of different color space description is discussed in [ 26] they contain RGB, LAB, LUV, HSV (HSL), YCrCb and the hue-min-max-difference (HMMD) [20, 25, 27, 28, 29]. The color covariance, color histogram and color moments [13, 21, 22, 30, 31] are mostly used color features in RBIR. The leading color, scalable color, color structure and color layout are the mainly used color features in MPEG-7 [32]. With the origin of three color features, hue-hue pair and hue are estimated and the color invariants are built. The high level semantics are not straight related to the above said color features. For mapping the region colors to high level semantic color names, the color average of all pixels in a region could be used as its color feature [25, 33, 34]. If the segmentation is erroneousness then that will end up as the original region is visually different from the average color. From [28], it is understood that in many cases, the dominant color and average color are very

The texture browsing descriptors [29, 32] are obtained from MPEG-7. They are regularity, directionality and coarseness. It is found that Brodatz texture [44] will perform outstanding with word features like randomness, directionality and periodicity. The Tamura features fail to work for multiple resolutions that are considered for measurement. The world features are get affected by image distortions like orientation differences due to viewpoint distortion [24] and scale. If the texture regions in the image are not organized and similar, [24], it would result in poor retrieval response for natural scene images. But for Brodatz textures, it works well. The human vision study [32, 40, 41] may match well with Wavelet and Gabor features in most of the image retrieval. But the actual design of Gabor filter and wavelet transform is only meant for rectangular images. But in RBIR, the region of image is having erratic-shapes. Hence, in such cases, the texture features are effectively used[25, 41]. But for natural image representation [32], the edge histogram descriptor (EHD) is most suitable and effective. 5


ISSN: 2076-3328

www.ijric.org


contents is still not solved in many CBIR systems. Once the preprocessing of feature extraction is over, the extracted visual properties are used to find concept detection, annotation or similarity estimation. The Figure 2 demonstrates the method of producing image signatures and the core research problems involved. The recent research interest focus on region based visual signatures that contain segmentation as the initial step. However, without performing segmentation for future extraction and signature forming is the recent research attraction [52].

3.3.3. Shape feature One of the most distinct concepts is shape feature. This feature has consecutive boundary segments, aspect ratio, Fourier descriptors, circularity and moment invariants [45]. The color and texture features are more useful in domain particular images like manmade objects. Still, the shape features are essential features but they do not have that much popularity in RBIR like texture and color features. The erroneousness of segmentation have been resulting them as not famous as texture and color features. To explore the inherent benefits of RBIR, the shape features could be used as evaluators by some system. For instance, the orientation and eccentricity features are used for this purpose [25]. 3.3.4. Spatial location Not only texture, color and shape features are important but also spatial location is very much useful with region cataloguing. For an instance an image containing trees with grass on ground could have similar color and texture features, but their spatial locations are different with tree leaves normally appears at the top of an image, while gross leaves at the bottom. So, it is very easy to define the spatial locations as ‘left, right, top and bottom’ depending on the place of the region in an image [46, 47]. The minimum bounding rectangle and region centroid are utilized to find the spatial location details are found in [48]. Also the center spatial of a region has been applied to define the spatial location details were discussed in [25].

Using segmentation we can easily acquire the region based signature. Trustable segmentation is really tough for distinguishing shapes within images because the shape estimates are purely depend on that discrimination. The K- means clustering was the classical approach used for segmentation. At earlier times it was the faster method for segmentation, but later on better methods had been evolved. By using normalized cuts criterion, the segmentation process can be very much improved [53]. In this, the segmentation is graphed into a weighted graph partitioning problem in which the vertices are considered as image pixels and the similarity denoted as the edge weight between two pixels.

In semantic feature extraction, relative spatial affinity is more essential than complete spatial location. The directional affinity between objects such as ‘right1left’ and ‘above below’ have been easily described using 2D-string [49] and with its alternative. Only directional affinities are not enough by without considering the topological affinities while representing the semantic image contents. The algorithm in [50] which refers the touch, front, right, up, left, down with spatial context modeling is used to provide performance improvement in semantic related image retrieval. The most desirable method was given by Smith in [23]. The spatial structuring of regions could be made available using (CRT) composite region template and every semantic group is constituted by a collection of image samples [23]. There are other issues in image retrieval such as problems of image annotation, dimensionality reduction and so on. The problem of retrieving images based on pixel

Figure 2. Image Signature Formation Overview

The normalized cut segmentation method is expanded in to texture image segmentation with texture dissimilarities and cues of contour [54], and 6


ISSN: 2076-3328

www.ijric.org

to combine known incomplete classing priors by answering a restrained optimization problem [55]. The other has inherent capability for appending real world application specific priors like size of cues of parts and their place in pathological images. The medical image searching became very important research problem because of it requires high dimension, throughput and resolution in the result. In this area, expectation maximization (EM) algorithm and hidden Markov random fields [56] are applied to segment the 3D brain magnetic resonance (MR) images, vertebral bodies from sagittal MR images [12] got segmented using spectral clustering approach. The new segmentation using the mean shift procedure [57], multiresolution [58] segmentation for low depth of field images, the Markov chain Monte Carlo technique [59] used Bayesian framework based segmentation, and a Gaussian mixture model [60] used EM algorithm depended segmentation, creating blobs matchable for image querying and retrieval. Starting the segmentation initially with texture features and end up with color features is a sequential approach discussed in [61]. Without any support for segmenting images that have similar texture or color regions has given in [62].


methods is shown in Figure 3 like supervised, semi or unsupervised learning, region based, global based similarity or both, segmentation based closeness matching and computation, using hypothetical, considering vectors, nonvectors or aggregate of features, deterministic closeness or fuzzy and computing closeness over linear space or non-linear regions. The mostly applied method to image retrieval is content based. It retrieves images depends on image content using image meta data or human attached meta data. But human annotation is a tough and time absorbing process, and hence the retrieval process has to be automated. By participating the user in retrieval process to refine the image search by asking them to continuously mark each result as ‘relevance’, ‘irrelevant’ or ‘neutral’ and this approach in CBIR systems is called relevance feedback method. To compare a given image with an image in database, the CBIR relies only on distance measure. It examines in contrast the closeness of two images in various ranges such as texture, color, shape and spatial locations. Hence, zero value of distance measure means perfect match of images with the given query by considering the above said dimensions. If it is higher than zero, then different types of similarities will exit between images.

It is important to obtain good segmentation with the achievement of reliability, increased quality and determining less complexity. By decreasing the dependence on reliable segmentation [60], also involving every generated image segment in the process of similarity matching to produce soft measures [40] and using multi resolution hidden Markov models for spatial structuring of texture and color [63, 64] will helps to achieve the above said qualities in image retrieval process. The other method that uses the non-cognitive grouping rules to extract images in pyramid structure [65], probabilistic modelling of group wise color segment co operations has been performed for the goal of image type separation and retrieval, to decrease acuteness to segmentation.

Figure 3. Various Types of Image Comparison Measures, their Mathematical Foundations And Methods for Estimating them

3.4. Image Similarity using Visual Signatures After obtaining image signature, the next step is to focus on accurate image retrieval. Various basic frameworks have been already defined for image similarity. The most useful features are local linearity (using triangle inequality in a neighbourhood), concord with semantics, and invariance to background (region dependent querying) and robustness to noise (in large scale and in real time). A design by incorporating various

With respect to the distance measure obtained, the retrieved result images with respect to query have been sorted. CBIR uses variety of user query types. The method of giving an image as an input to retrieve images from database is referred to as Query by example. In this technique the resultant images should be similar, that means they should have common objects within the image but different algorithms may be applied in retrieval 7


ISSN: 2076-3328

www.ijric.org

process depending on application. Different alternatives of input query images in this scheme are:


users’ focus is in a particular image region rather than the entire image. It has been found that most of the users are normally interested in specific regions of an image instead of entire image. This results with region based CBIR as the current trend. The easiest image retrieval is based on global features. The human idea based system has a strong similarity matching with region level image retrieval systems [13]. In RBIR, initially the spatial locations and other features are extracted from the regions that are already segmented. This above said step can help to perform the closeness match between two images. The other way of computing the closeness measure is based on color histograms of two images. In recent RBIR techniques, the segmentation is achieved with the help of color proportion or combinations and spatial relation between two color regions. Color based segmentation is mostly used because it never depends on size or orientation of the image. Apart from the above technique, various definitions for color spaces are discussed in [71]. Because human ideas are more close to color spaces, they are applied in region based retrievals. They normally have color spaces like (HMMD) hue-min-maxdifference, LUV, RGB, YCrCb and HSV (HSL). The color definitions and features in RBIR systems contain color histogram, covariance matrix, coherence vector and moments. The color structure, dominant color, color layout and measureable color are discussed in MPEG-7.

The user can provide the query image as a file, or allowed to be taken from random group, or a drawn image supplied by the user that is similar to images to be retrieved. This approach avoids the disadvantage of defining images with keywords. The other type of query methods are searching for images using web browser, image region based queries, image feature based queries, multimodality type of queries like combination of two or more features. Hence, by adapting some emerging fields with machine learning ideas and digital image processing concepts may try to shrink the gaps in the above discussed issues. The texture, color, shape and spatial location are considered as the low level image features. The focus of indexing and image retrieval is for identifying semantic information to provide a direct correlation between the low level features as discussed above. The real world CBIR application areas include photographic and art collections, retail achieves, intellectual property rights, military, building architectural design, mechanical CAD/CAM engineering design, medical imaging processing, computer science engineering, criminology department, Global positioning system (GPS), Mobile applications, remote sensing, educational training, Global information system (GIS), etc., To overcome the drawbacks in text based image retrieval, the CBIR is the best choice. The CBIR systems called IBM QBIC [66] performs the image retrieval that depends on image visual contents like percentage of color, layout of color and occurrences of texture in the image. The other system used for content based retrieval in videos is called Virage [67]. An interactive learning agent called photobook [68] is applied well in image retrieval. To frame a query using spatial structuring of diagrammed color regions, we can go for VisualSEEk [69]. The relevance feedback method can be applied in multimedia image retrieval using MARS (Multimedia Analysis and Retrieval System) [70].

Figure 4. Overall CBIR Architecture

Main advantage of using texture measure is that it will give an analysis of visual patterns are spatially described in an image. As textures are described with pixels, they can be easily grouped into sets. Adding a texture into a group depends on the count of identified textures in the image. The texture feature is a tough one to describe. With the help of 2D grey level differences, it can be easily identified as a particular texture in an image. To compute various degrees of directionality, contrast,

The architecture of CBIR Retrieval System is as follows in Figure 4: In CBIR technique the basic level image feature extraction is the initial step. The feature extraction in CBIR is done for the total image or for a particular region in the image. Most of the end 8


ISSN: 2076-3328

www.ijric.org

coarseness and regularity [72], the comparative brightness of pixel pairs is very important. Even though, the hard thing is that to attach a specific texture into groups of textures like “soft” or “hard” because of the differences in the neighbouring pixel values. The texture property in images is the important feature because they help in image classification process. Different real-world images have different texture features. Thus, we can conclude that texture is a unavoidable feature to map the high level concepts with low level.

4. ONTOLOGY BASED IMAGE RETRIEVAL Ontology means a particular description of a conceptualization. It designs a domain in a formal way. With the help of textual information at the surrounding only the web image retrieval is achieved. There are some text dependent image retrieval engines are already available for the web such as Yahoo and Google. They use text features like file names as indices for searching images in the web. Many image retrieval engines are under construction. The low level descriptors of these engines are far from semantic notions. The other type of systems only relies on human-annotations. Hence, there is a necessary to define and intermediate approach to image understanding. Some systems may define a specific domain from domain experts by identifying vocabularies used to describe objects of interest. The most desirable thing in image retrieval is domain-independent visual concept ontology. This type of ontology driven description supports automatic recognition based on image processing techniques. The visual concept ontology is described in [88]. The ontology driven knowledge acquisition is necessary for building the visual concept ontology. In this, a domain is specified using a tree structure with class hierarchy of its sub elements at each level. We can take an example domain in Medical Pathology or biological organisms.

Table 3. Various Image Retrievals Based on Visual Content

Shape

Texture

Chad Carson [ 75] Moore [76 ] J. Sawhney & Hafner [74] Stricker & Orengo [77] Kankanhalli [78] Wang [ 79] MichaelOrtega [80] F. Mokhtarian [ 81] Sougata Mukherjea [82] J. R. Smith [ 83] J. Michel[ 84] B. S. Manjunath [ 85] George Tzagkarakis & Panagiotis Tsakalides [86]

E-ISSN: 2076-3336

gaps is discussed in [87]. By representing, how ontology helps in image retrieval by shrinking the semantic disparity between low level and high level features will provide a better solution for the CBIR system.

Shape means the form of a specific region in an image. Various edge detection methods combined with segmentation are used to compute shapes. Along with them, shape filters [73] can also detect image shapes. Due to the drawback of unachieved fully automated image segmentation, human interception is necessary for concluding shape detection accuracy. A brief survey on the research in CBIR is highlighted in Table 3.

Low-level Researches features W. Niblack [74] Color

IJRIC

Approaches Histogram and color moments Region histogram Dominant color Color histogram Color moment Color cluster Bit signature Fourier transform Curvature scale Template Matching Wavelet transform Edge statistic Gabor filters Statistical

The major categories of CBIR gaps defined in the literature are: A disparity between the low level constituents that are extracted by computers from the image and the high level understanding of human image cognition is called semantic gap. A disparity between the methods of image objects capturing and object that present in the real world is called sensory gap. A disparity between the levels of CBIR integration with general purpose image retrieval system is known as integration gap. Automation of feature extraction gap is an algorithm generated gap. The catalogue of various

Figure 5. The Visual Idea Ontology

In ontology based retrieval, the knowledge gathering process is done as follows. The visual idea ontology in Figure 5 has three parts such as Color, Spatial Temporal and texture concepts.

9

International Journal of Reviews in Computing

IJRIC

31st December 2012. Vol. 12 © 2009 - 2012 IJRIC & LLS. All rights reserved.

ISSN: 2076-3328

www.ijric.org

E-ISSN: 2076-3336

Table 4. The spatial concepts could be defined using RCC-8 spatio-temporal relations and geometric concepts in Table 6. The texture concepts could be defined using cognitive science community experiments in Table 5. Table 4. Color Concepts (flattened view) Derived from the ISCC-NBS Color Dictionary

Figure 6. Architecture of Ontology Applied Image Retrieval Process

The architecture of ontology applied image retrieval process is given in Figure 6. The image feature extraction done by computer will result with meaningful image concepts. These concepts may be color, texture, shape or spatial locations. Mapping those one or more resultant concepts into ontology will interpret the conceptual meaning of an image. If the retrieval query captures the actual users’ intention from this ontology representation, it will definitely reduce the semantic disparity between man and machine. In an intelligent image retrieval process, different type of indexing schemes has been applied starting from text based, keyword annotated, field based, structure based, content based to ontology based. Still, image retrieval is in its infant stages only because of the semantic disparity. Various algorithms are evolving to fill this gap but sometime that may create other gaps in image interpretation and understanding. This means that the similarity matching process of image concepts with human perception fails to achieve its precision. This precision could be measured only through the user satisfaction in the image retrieval process. To improve accuracy in this context, a fuzzy mechanism with the closeness measure computation can be adopted. The knowledge representation could be the first step in a concept ontology formation. This representation should provide a conceptual relationship between the extracted features in terms of numerical information. The color concepts have been defined using ISCC-NBS color dictionary in 10

Red

Purple

Reddish Orange

Reddish Purple

Orange

Purplish Red

Orange Yellow

Purplish Pink

Yellow

Pink

Greenish Yellow

Yellowish Pink

Yellow Green

Brownish Pink

Yellowish Green

Brownish Orange

Green Reddish Brown

Brown

Bluish Green

Yellowish Brown

Greenish

Blue

Blue

Olive Brown

Purplish Blue

Olive

Violet

Olive Green

Table 5. Texture Concepts: Developed Based on the Experimental Results of Cognitive Science Community

Smooth Texture Regular

Weaved Texture

Texture

Oriented Texture

Images

Periodic Texture Granulated Texture Irregular

Veined Texture

Texture

Marbled Texture 3D Texture


ISSN: 2076-3328

www.ijric.org

Table 6. Subset of Geometric Structures

Curve Point Set

Surface


diploma, graduate, masters or doctorate and lifting or promoting them from one level to the other.

Affine Curve (e.g. Segment) Broken Line (e.g. Polygon) Ellipse (e.g. Circle) Polygonal Surface (e.g. Rectangular Surface) Elliptical Surface (e.g. Circular Surface)

The image context description could be framed using ontology from the above said image concepts. Applying Description Logics (DL), the knowledge representation could be formed. The DAML (DARPA Markup Language) and OIL (Ontology Interface Language) are used for this implementation which is available with OWL (Web Ontology Language). Rules for describing relation between image features in ontology can be defined using the DL also. Once the concept ontology in Fig.7 and Fig.8 has been framed (for example spatial ontology), the similarity matching of user query with extracted image feature is estimated through the ontology hierarchy. This provides more closeness to user query with images in database.

Figure 8. Manifestation of Spatial Relation Ontology

Considering photos taken in these functions and maintaining them in a database, an appropriate image retrieval system may produce better search results for the photos. The ontology annotated image retrieval system could achieve accuracy in retrieval results. This is because ontology concept reduces the semantic gap in the retrieval process. This case study design has three parts. The first part deals with the creation of ontology for a function or event happening in university building. The essential point in ontology formation is that it never contain too much of concepts unless they are really needed. While fixing classes and subclasses for ontology, the confusion and relation increases exponentially with the count of used concept on it. For an instance, how a system introduces complexity to convey a message to a friend A? The second part deals with annotating more than 500 photos taken in the function with this created semantic concept based ontology. Then these annotations should be transferred into RDF formats. An annotation editor can be used for this purpose where the actual description of images could be given in text. Once the annotation schema is over, this will relate each classes and instances in the domain ontology with the photos. The third part deals with user query given based on conceptual hierarchy choices designed in the UI.

Figure 7. Ontological Concept Description in Image Interpretation

There are some tools have been developed namely “OntoVis” [51] which perform three tasks namely domain knowledge acquisition, ontology driven visual acquisition and image example management. The benefit of using visual idea ontology is to fill the semantic gap as much as possible between low and high level concepts. 5. ONTOLOGY USED IMAGE RETRIEVAL: A CASE STUDY ANALYSIS A Case Study example [89] to reduce the semantic disparity in image retrieval is described in this section. The Convocation is a ceremony of awarding degree to students that may be for

11


ISSN: 2076-3328

www.ijric.org

5.1. The Ontology Formation Model Using Protégé The prototype model is used here because ontology formation cannot go with design and implementation phase as in Figure 9.


5.2. The Ontology Annotation Model This part shown in Figure 11 has done to combine the structure dependent annotation with frame dependent ontologies. Implementation of this will define the schema for annotating images with this convocation ontology. In this, the rdf:Property could be used as structure based and field based. Each annotation schema should have to insert values through one field.

Figure 9. Development Model for Ontology Formation

The initial step in ontology framing is knowledge acquisition and hence the domain knowledge could be acquired from domain experts. Then it is refined by clients with the help of their feedbacks. For complex ontologies the user acceptance is the main one. In this formation process, the top level classes’ selection could be done using RDFS. These classes should tell what image context is described and the remaining ontology should tell how it is described. The higher level class alone will not describe a total image, but it should serve as a whole to annotate and retrieve them. The created structure of ontology is shown in Figure 10.

Figure 11. Annotation of Convocation and Various Events on Left are annotated with Objects at Right

The values to be inserted are chosen from category tree. Hence, classes and instances are called categories in this case. This category tree represents a group of distributed ontologies, with a group of a single ontology or combination of all. 5.3. The Image Retrieval Model After the image annotation, the ontology dependent photographic demonstration system is developed. The user interface representation of this system is in Figure.12. 6.

CONCLUSIONS

The classical research interest in image retrieval was done with CBIR, low-level feature extraction, etc. In CBIR technique, the low level image features cannot always represent high level semantic concepts in the users’ mind. Hence, the CBIR systems should provide maximum support in bridging the ‘semantic gap’ between low level visual features and the richness of human semantics. This paper provides a comprehensive survey of recent work towards narrowing down the ‘semantic gap’. The high level semantics can be combined with CBIR system for shrinking the

Figure 10. Ontology Structure of Different Persons, Roles and Universities

12


ISSN: 2076-3328

www.ijric.org

semantic gap using some computer vision and machine learning techniques.


combine query ideas of user with basic level extracted features of an image, getting the relevance feedback of users to transparent their real expectation, providing a meaningful template for user to enhance the high level interpretations of user expectations, intermix the proofs from and the visual web image retrieval and HTML text. Although there is no generic and automated algorithm for image retrieval without the above said gaps. The discussion of CBIR with high level meanings and classical systems with low level features, this paper gives an idea for ‘semantic gap’ reduction using ontology shown in Figure 13. To implement fully automated image retrieval system with high level semantics requires some type of visual concept ontology to be built for low level feature extraction and map them into high level semantics. This case study analyses the ontology dependent image annotation system for image retrieval. The domain model could not be built with a natural language, but can be represented with natural language. But tools and semantic web languages can help to develop an ontology domain or thesaurus. The very good ontology domain modeling is purely depending on expert’s hand who frames the ontology. If text based annotation is used then good thesaurus could be used for choosing keywords to be annotated with images. If a complex ontology is defined then annotation process will also become complex. But without domain knowledge it could be useable. Once if design rules of domain ontology changes, the annotation schema would also get changes. It is still remain unsolved. Definitely the ontology based retrieval perform search based on finding the constraints, classes that actually define the images. In this system the precision and recall rates are comparatively high than comparing with text based systems.

Figure 12. Different Categories Representation of Retrieving Images Based on their Attributes

REFERENCES [1]

[2] Figure 13. Semantic Gap Reduction using Ontology

Different solutions to decrease the semantic gap may be by using ontology representation of image objects that will interpret the high level ideas in user mind, machine learning approaches which

[3]

13

Rosenfeld, A. (1969). Picture processing by computer. ACM Computing Surveys (CSUR), 1(3), 147-176. Tamura, H., & Mori, S. (1977, April). A data management system for manipulating large images. In Proceedings of Workshop on Picture Data Description and Management (pp. 45-54). Rui, Y., Huang, T. S., & Chang, S. F. (1999). Image retrieval: Current techniques,


ISSN: 2076-3328

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

www.ijric.org

promising directions, and open issues. Journal of visual communication and image representation, 10(1), 39-62. Chowdhury, G. (2010). Introduction to modern information retrieval. Facet publishing. Buckley, C., & Salton, G. (1995, July). Optimization of relevance feedback weights. In Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval (pp. 351-357). ACM. Knuth, D. E., Morris Jr, J. H., & Pratt, V. R. (1977). Fast pattern matching in strings. SIAM journal on computing, 6(2), 323-350. Boyer, R. S., & Moore, J. S. (1977). A fast string searching algorithm.Communications of the ACM, 20(10), 762-772. Lanzenberger, M., Sampson, J., & Rester, M. (2009, March). Visualization in ontology tools. In Complex, Intelligent and Software Intensive Systems, 2009. CISIS'09. International Conference on (pp. 705-711). IEEE. Genesereth, M. R., & Fikes, R. E. (1992). Knowledge interchange format-version 3.0: reference manual. Schreiber, A. T., Dubbeldam, B., Wielemaker, J., & Wielinga, B. (2001). Ontology-based photo annotation. Intelligent Systems, IEEE, 16(3), 66-74. Liu, Y., Zhang, D., Lu, G., & Ma, W. Y. (2007). A survey of content-based image retrieval with high-level semantics. Pattern Recognition, 40(1), 262-282. Carballido-Gamio, J., Belongie, S. J., & Majumdar, S. (2004). Normalized cuts in 3D for spinal MRI segmentation. Medical Imaging, IEEE Transactions on, 23(1), 3644. Jing, F., Li, M., Zhang, L., Zhang, H. J., & Zhang, B. (2003). Learning in region-based image retrieval. Image and Video Retrieval, 199-204. Ma, W. Y., & Manjunath, B. S. (1997, June). Edge flow: a framework of boundary detection and image segmentation. In Computer Vision and Pattern Recognition, 1997. Proceedings., 1997 IEEE Computer Society Conference on(pp. 744-749). IEEE. Feng, H., Castanon, D. A., & Karl, W. C.

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

14


(2001). A curve evolution approach for image segmentation using adaptive flows. In Computer Vision, 2001. ICCV 2001. Proceedings. Eighth IEEE International Conference on (Vol. 2, pp. 494-499). IEEE. Sumengen, B., & Tesic, J. (2000). Normalized Cuts and Image Segmentation.ECE, 278, 1-7. Comaniciu, D., & Meer, P. (1997, June). Robust analysis of feature spaces: color image segmentation. In Computer Vision and Pattern Recognition, 1997. Proceedings., 1997 IEEE Computer Society Conference on (pp. 750-755). IEEE. Liu, Y., Zhang, D., & Lu, G. (2008). Regionbased image retrieval with high-level semantics using decision tree learning. Pattern Recognition, 41(8), 25542570. Feng, H., & Chua, T. S. (2003, November). A bootstrapping approach to annotating large image collection. In Proceedings of the 5th ACM SIGMM international workshop on Multimedia information retrieval (pp. 5562). ACM. Fan, J., Gao, Y., Luo, H., & Xu, G. (2004, July). Automatic image annotation by using concept-sensitive salient objects for image content representation. InProceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval (pp. 361-368). ACM. Wang, J., Li, J., Chan, D., & Wiederhold, G. (1999). Semantics-sensitive retrieval for digital picture libraries. D-Lib Magazine, 5(11). Town, C., & Sinclair, D. (2000). Content based image retrieval using semantic visual categories. AT&T Laboratories Cambridge Report, 2000. Smith, J. R. (1998, June). Decoding image semantics using composite region templates. In Content-Based Access of Image and Video Libraries, 1998. Proceedings. IEEE Workshop on (pp. 9-13). IEEE. Leow, W. K., & Lai, S. Y. (2000). Scale and orientation-invariant texture matching for image retrieval. SERIES IN MACHINE PERCEPTION AND ARTIFICIAL INTELLIGENCE, 40, 151-164. Mezaris, V., Kompatsiaris, I., & Strintzis, M. G. (2003, September). An ontology approach to object-based image retrieval. In Image


ISSN: 2076-3328

www.ijric.org

Processing, 2003. ICIP 2003. Proceedings. 2003 International Conference on (Vol. 2, pp. II-511). IEEE. [26] Plataniotis, K. N., & Venetsanopoulos, A. N. (2000). Color image processing and applications. Springer. [27] Stanchev, P., Green Jr, D., & Dimitrov, B. (2003). High level color similarity retrieval. [28] Liu, Y., Zhang, D., Lu, G., & Ma, W. Y. (2005). Region-based image retrieval with perceptual colors. Advances in Multimedia Information Processing-PCM 2004, 931938. [29] Manjunath, B. S., Ohm, J. R., Vasudevan, V. V., & Yamada, A. (2001). Color and texture descriptors. Circuits and Systems for Video Technology, IEEE Transactions on, 11(6), 703-715 [30] Chang. E, Tong. S, (2001). SVM activesupport vector machine active learning for image retrieval, Proceedings of the ACM International Multimedia Conference, pp. 107–118. [31] Zheng, X., Cai, D., He, X., Ma, W. Y., & Lin, X. (2004, October). Locality preserving clustering for image database. In Proceedings of the 12th annual ACM international conference on Multimedia (pp. 885-891). ACM. [32] Salembier, P., Sikora, T., & Manjunath, B. S. (2002). Introduction to MPEG-7: multimedia content description interface. John Wiley & Sons, Inc. [33] Hua, K. A., Vu, K., & Oh, J. H. (1999, October). SamMatch: a flexible and efficient sampling-based image retrieval technique for large image databases. InProceedings of the seventh ACM international conference on Multimedia (Part 1) (pp. 225-234). ACM. [34] Wang, W., Song, Y., & Zhang, A. (2003). Semantics retrieval by content and context of image regions. TECHNICAL PAPERSSOCIETY OF MANUFACTURING ENGINEERS-ALL SERIES. [35] Plataniotis, K. N., Androutsos, D., & Venetsanopoulos, A. N. (1999). Adaptive fuzzy systems for multichannel signal processing. Proceedings of the IEEE,87(9), 1601-1622. [36] Lukac, R., Smolka, B., Martin, K., Plataniotis, K. N., & Venetsanopoulos, A. N. (2005). Vector filtering for color imaging. Signal Processing Magazine,

[37]

[38]

[39]

[40]

[41]

[42]

[43]

[44]

[45]

[46]

[47]

15


IEEE,22(1), 74-86. Sethi, I. K., Coman, I. L., & Stan, D. (2001, March). Mining association rules between low-level image features and high-level concepts. InAerospace/Defense Sensing, Simulation, and Controls (pp. 279-290). International Society for Optics and Photonics. Permuter, H., Francos, J., & Jermyn, I. (2006). A study of Gaussian mixture models of color and texture features for image classification and segmentation.Pattern Recognition, 39(4), 695-706. Stanchev, P. (2003, May). Using image mining for image retrieval. In IASTED Conference “Computer Science and Technology,” Cancun, Mexico (pp. 214218). Wang, J. Z., Li, J., & Wiederhold, G. (2001). SIMPLIcity: Semantics-sensitive integrated matching for picture libraries. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 23(9), 947-963. Ma, W. Y., & Manjunath, B. S. (1997, October). Netra: A toolbox for navigating large image databases. In Image Processing, 1997. Proceedings., International Conference on (Vol. 1, pp. 568-571). IEEE. Liu, F., & Picard, R. W. (1996). Periodicity, directionality, and randomness: Wold features for image modeling and retrieval. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 18(7), 722-733. Tamura, H., Mori, S., & Yamawaki, T. (1978). Textural features corresponding to visual perception. Systems, Man and Cybernetics, IEEE Transactions on,8(6), 460-473. Brodatz, P. (1966). Textures: a photographic album for artists and designers(Vol. 66). New York: Dover. Mehrotra, R., & Gary, J. E. (1995). Similarshape retrieval in shape data management. Computer, 28(9), 57-62. Song, Y., Wang, W., & Zhang, A. (2003). Automatic annotation and retrieval of images. World Wide Web, 6(2), 209-231. Mojsilovic, A., Gomes, J., & Rogowitz, B. (2002, May). Isee: Perceptual features for image library navigation. In Proc. 2002 SPIE


ISSN: 2076-3328

www.ijric.org

Human Vision and Electronic Imaging. Tong, S., & Chang, E. (2001, October). Support vector machine active learning for image retrieval. In Proceedings of the ninth ACM international conference on Multimedia (pp. 107-118). ACM. [49] Chang, S. K., Shi, Q. Y., & Yan, C. W. (1987). Iconic indexing by 2-D strings.Pattern Analysis and Machine Intelligence, IEEE Transactions on, (3), 413-428. [50] Ren. W., Sing. M., & Singh. S. (2002). Image retrieval using spatial context. In IWSSIP'02: international workshop on systems, signals and image processing (pp. 44-49). [51] Alani, H. (2003). TGVizTab: an ontology visualisation extension for Protégé. [52] Datta, R., Joshi, D., Li, J., & Wang, J. Z. (2008). Image retrieval: Ideas, influences, and trends of the new age. ACM Computing Surveys (CSUR), 40(2), 5. [53] Shi, J., & Malik, J. (2000). Normalized cuts and image segmentation. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 22(8), 888-905. [54] Malik, J., Belongie, S., Leung, T., & Shi, J. (2001). Contour and texture analysis for image segmentation. International Journal of Computer Vision,43(1), 7-27. [55] Yu, S. X., & Shi, J. (2004). Segmentation given partial grouping constraints.Pattern Analysis and Machine Intelligence, IEEE Transactions on, 26(2), 173-183. [56] Zhang, Y., Brady, M., & Smith, S. (2001). Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm. Medical Imaging, IEEE Transactions on, 20(1), 45-57. [57] Comaniciu, D., & Meer, P. (2002). Mean shift: A robust approach toward feature space analysis. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 24(5), 603-619. [58] Wang, J. Z., Li, J., Gray, R. M., & Wiederhold, G. (2001). Unsupervised multiresolution segmentation for images with low depth of field. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 23(1), 85-90. [59] Tu, Z., & Zhu, S. C. (2002). Image segmentation by data-driven Markov chain Monte Carlo. Pattern Analysis and Machine


Intelligence, IEEE Transactions on,24(5), 657-673. [60] Carson, C., Belongie, S., Greenspan, H., & Malik, J. (2002). Blobworld: Image segmentation using expectationmaximization and its application to image querying. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 24(8), 1026-1038. [61] Chen, J., Pappas, T. N., Mojsilovic, A., & Rogowitz, B. (2002, June). Adaptive image segmentation based on color and texture. In Image Processing. 2002. Proceedings. 2002 International Conference on (Vol. 3, pp. 777-780). IEEE. [62] Deng, Y., & Manjunath, B. S. (2001). Unsupervised segmentation of color-texture regions in images and video. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 23(8), 800-810. [63] Li, J., Gray, R. M., & Olshen, R. A. (2000). Multiresolution image classification by hierarchical modeling with two-dimensional hidden Markov models.Information Theory, IEEE Transactions on, 46(5), 1826-1841. [64] Li, J., & Wang, J. Z. (2003). Automatic linguistic indexing of pictures by a statistical modeling approach. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 25(9), 1075-1088. [65] Iqbal, Q., & Aggarwal, J. K. (2002). Retrieval by classification of images containing large manmade objects using perceptual grouping. Pattern recognition, 35(7), 1463-1479. [66] Flickner, M., Sawhney, H., Niblack, W., Ashley, J., Huang, Q., Dom, B., ... & Yanker, P. (1995). Query by image and video content: The QBIC system.Computer, 28(9), 23-32. [67] Gupta, A. (1996). Visual information retrieval technology: A virage perspective.Virage Inc., TR A, 3, 1996. [68] Pentland, A., Picard, R., & Sclaroff, S. (1995). Photobook: Tools for content-based manipulation of image databases. International Journal of Computer Vision. [69] Smith, J. R., & Chang, S. F. (1997, February). VisualSEEk: a fully automated content-based image query system. In Proceedings of the fourth ACM

[48]

16


ISSN: 2076-3328

www.ijric.org

international conference on Multimedia (pp. 87-98). ACM. [70] Rui, Y., Huang, T. S., & Mehrotra, S. (1997, October). Content-based image retrieval with relevance feedback in MARS. In Image Processing, 1997. Proceedings., International Conference on (Vol. 2, pp. 815-818). IEEE. [71] Dorairaj, R., & Namuduri, K. R. (2004, November). Compact combination of MPEG-7 color and texture descriptors for image retrieval. In Signals, Systems and Computers, 2004. Conference Record of the Thirty-Eighth Asilomar Conference on (Vol. 1, pp. 387-391). IEEE. [72] Tamura, H., Mori, S., & Yamawaki, T. (1978). Textural features corresponding to visual perception. Systems, Man and Cybernetics, IEEE Transactions on,8(6), 460-473. [73] Tushabe, F., & Wilkinson, M. (2008). Content-based image retrieval using combined 2D attribute pattern spectra. Advances in Multilingual and Multimodal Information Retrieval, 554-561 [74] Hafner, J., Sawhney, H. S., Equitz, W., Flickner, M., & Niblack, W. (1995). Efficient color histogram indexing for quadratic form distance functions. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 17(7), 729-736. [75] Carson, C., Thomas, M., Belongie, S., Hellerstein, J., & Malik, J. (1999). Blobworld: A system for region-based image indexing and retrieval. In Visual Information and Information Systems (pp. 660-660). Springer Berlin/Heidelberg. [76] Deng, Y., Manjunath, B. S., Kenney, C., Moore, M. S., & Shin, H. (2001). An efficient color representation for image retrieval. Image Processing, IEEE Transactions on, 10(1), 140-147. [77] Stricker, M., & Orengo, M. (1995, February). Similarity of color images. In Proc. SPIE Storage and Retrieval for Image and Video Databases (Vol. 2420, No. 381-392, p. 4). [78] Kankanhalli, M. S., Mehtre, B. M., & Wu, R. K. (1996). Cluster-based color matching for image retrieval. Pattern recognition, 29(4), 701-708. [79] Katz, J., & Wang, N. (2003, October). Efficiency improvements for signature schemes with tight security reductions. In Proceedings of the 10th ACM conference


on Computer and communications security (pp. 155-164). ACM. [80] Rui, Y., Huang, T. S., Ortega, M., & Mehrotra, S. (1998). Relevance feedback: A power tool for interactive content-based image retrieval. Circuits and Systems for Video Technology, IEEE Transactions on, 8(5), 644-655. [81] Mokhtarian, F., & Mohanna, F. (2001). Enhancing the curvature scale space corner detector. In Proceedings of the Scandinavian Conference on Image Analysis (pp. 145152). [82] Mukherjea, S., Bamba, B., & Kankar, P. (2005). Information retrieval and knowledge discovery utilizing a biomedical patent semantic web. Knowledge and Data Engineering, IEEE Transactions on, 17(8), 1099-1110. [83] Smith, J. R., & Chang, S. F. (1994, November). Transform features for texture classification and discrimination in large image databases. In Image Processing, 1994. Proceedings. ICIP-94., IEEE International Conference (Vol. 3, pp. 407-411). IEEE. [84] Sidenbladh, H., & Black, Michel. J. (2003). Learning the statistics of people in images and video. International Journal of Computer Vision, 54(1), 183-209. [85] Manjunath, B. S., & Ma, W. Y. (1996). Texture features for browsing and retrieval of image data. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 18(8), 837-842. [ 8 6 ] Tzagkarakis, G., & Tsakalides, P. (2004, April). A statistical approach to texture image retrieval via alpha-stable modeling of wavelet decompositions. In5th International Workshop on Image Analysis for Multimedia Interactive Services, Lisboa, Portugal (pp. 21-23). [87] Deserno, T. M., Antani, S., & Long, R. (2009). Ontology of gaps in content-based image retrieval. Journal of Digital Imaging, 22(2), 202-215. [88] Maillot, N., Thonnat, M., & Boucher, A. (2004). Towards ontology-based cognitive vision. Machine Vision and Applications, 16(1), 33-40. [89] Styrman, A. (2005). Ontology-based image annotation and retrieval (Doctoral dissertation, Master thesis).

17