Hence, another requirement arises, which we call the visibility requirement. .... We find that presenting an image as a circle with center point and radius R = w. 2.
Similarity based visualization of image collections G.P. Nguyen
M. Worring
Intelligent Sensory Information Systems, University of Amsterdam, Kruislaan 403, 1098SJ Amsterdam, The Netherlands E-mail: {giangnp,worring}@science.uva.nl Abstract In literature, few content based multimedia retrieval systems take the visualization as a tool for exploring the collections. However, when searching for images without examples to start with, one needs to explore the data set. Up to now, most available systems just show random collections of images in 2D grid form. More recently, advanced techniques have been developed for browsing based on similarity. However, none of them analyze the problems that occur when visualizing large visual collections. In this paper, we make these problems explicit. From there, we establish three general requirements: overview, visibility, and data structure preservation. Solutions for each requirement are proposed. Finally, a system is presented and experimental results are given to demonstrate our theory and approach.
1
Introduction
Together with the development of multimedia technologies and the availability of cheap digital cameras, the size of image and video collections is growing tremendously. For managing, exploring and searching through the collections effective visualization is essential. Many works have been contributed to this interesting area [18]. A major issue in content based retrieval is the semantic gap between the system’s automatic annotation capability and the user’s requirements in terms of conceptual access of the collection. Improving the system performance can be done from the systems perspective or from the user side or from the combination of these. In any approach the visualization of the collections is an important element since it is the best way to create a link between the user and the system. In literature, few content based multimedia retrieval systems take the visualization as a tool for exploring the collections. However, when searching for images without examples to start with, one needs to explore the data set. Up to now, most available systems just show random collections of images in 2D grid form. Examples are described in [18], [20], [22]. For searching based on descriptions querying based on features or examples is most appropriate. Browsing depends on the relations between images. Hence, it should be based on similarity. More recently, advanced techniques have been developed for browsing based on similarity such as [15], [2], [9], [4], [14], [3], [21], [11]. However, none of them analyze the specific problems that occur when visualizing visual collections. For example, as the size of image collections is very large, randomly selecting a set of images from the collection [22] is not a proper way. Using this selection set only the user cannot get the feeling of what is inside the database. On the other hand, the limitation of the display (i.e. either size or resolution) does not allow any system to show the whole collection. Besides, showing all images does not even provide more information to the user, but also lets him/her easily get lost in the crowded net of images. Some systems have made an effort to relief this limitation by showing the whole collection to the user [3] as a point set. Each image is then represented by a point on the display, and once the user selects the point, they will get a visualization of the real image [4]. However from a practical point of view, this approach is not easy at all for the user when looking at sets of more than a thousand points. Moreover, an image is a visual object hence its content should be visible to the user.
In this paper, we make all problems explicit. The paper is organized as follows. In section 2, we analyze the general requirements for visualizing large image collections. From there, a set of requirements is proposed. Solutions for each requirement are then presented in section 3. Finally, section 4 shows experimental results with real data.
2
Problem analysis
In this section, we analyze in more detail the problems occurring when visualizing a large visual collection. From there general requirements for a common visualization system are defined. The first problem in visualizing large collections is the limited display size of the equipment to display them, which is called the visualization space, due to its limitation in size and resolution. Meanwhile, the size of the collections is often much larger than the capability of the visualization space. Secondly, since an image is a visual object the ultimate goal of any visualization tool is to show the content of the images. Because of this limitation, only a small set of images can be displayed at one time. Random selection of those images is certainly not a good approach as it is not able to show the distribution of the whole collection. To overcome this problem, the selected set should be able to represent the whole set or at least give the user an overview of the set that he/she is interested in. This problem has been studied in [16], where the author pointed out that overview is the most important criteria in designing information visualization systems. Therefore, the first criteria for an optimal visualization tool is the overview requirement. In the collection, images normally have relations with each other such as they are taken as the same date, by the same author, or share the same visual features [4]. It means each image is represented as a feature vector. For examples, if the RGB histogram of each image is extracted with 32bins for each color channel, a feature vector is a 96 dimensional vector. The whole collection therefore builds up a high dimensional feature space. Normally the number of feature dimensions is much larger than 3, the maximum number of dimensions human vision is able to capture. A mapping algorithm from feature space to the visualization space is needed. Some of the well-known techniques for this mapping are PCA(Principle Component Analysis), MDS(Multi Dimensional Scaling), or the more recent methods ISOMAP(ISOmetric MAPping)[19], SNE(Stochastic Neighbor Embedding)[7], LLE(Local Linear Embedding)[13]. For example in [10], [2] the authors use PCA for mapping data to 2D, meanwhile in [12] the ISOMAP is used. However, an exact match between data driven distances and the screen distances in general is impossible. Moreover, in feature space, image collections often follow certain nonlinear structures. For example, given a video sequence, the set of frames of the sequence will have a linear structure if the camera captures a car moving along the street, or a cyclic structure if the video captures the hour-pointer movement on a clock [12]. Therefore, when mapping data to visualization space, the structure of the collection should be preserved. This is the second requirement, which is called structure preservation requirement. Finally, when a set of images is displayed, even though systems use thumbnails, which have smaller size compared to the original images they tend to overlap each other partially or fully [10]. The overlap between images up to now has not been considered properly even though it can influence the quality of a visualization tool greatly. As an example, from the user’s point of view, the overlap in visualization introduces the possibility of missing images when they are overlapped by other images, hence restrains the communication between the user and the system. It must be stressed here that images are visual objects, so the overlap between them should be reduced as much as possible. Hence, another requirement arises, which we call the visibility requirement. In summary, we have three general requirements: overview, structure preservation, and visibility. However, it should be noted here that the three requirements are not all independent of each other. In particular, to increase the visibility, images should be spread out from each other. This has a negative effect on the structure preservation requirement. Consequently the structure is not preserved. Moreover, the more images are selected as representatives the better the overview, but the visibility decreases because of the limitation in the visualization space. Figure 1 illustrates the relations between the requirements. Hence, there needs a balancing functions between the requirements. This means we want to keep the original relation as well as increase the visibility, and a good overview together with the visibility is kept. In the following sections, we introduce the approach for each requirement. Then for each solution we define a cost function. From there, the balancing functions are proposed for optimal visualization.
Limited space Overview Large set Visibility Image collection
Visual objects High dimension
Structure preservation
Non−uniform
Figure 1. General requirements for visualizing image collections.
3 3.1
Approach From high to 2-dimensional space
Assume each image is represented as a point in the feature space. A mapping is needed to map these points to the visualization space. Many methods exist for this purpose. For a good overview of the whole data as well as keeping the real structure of data after the mapping, we need an algorithm that preserves the relations between all the data points locally and globally. Common techniques like PCA and MDS are designed for linear manifold embedding. More recent techniques take the non-linear structure of the dataset into account. For instance, ISOMAP [19] first re-calculates the similarity between points in the high dimensional space as the length of the shortest path between any two points, and then applies the MDS on the new distances computed. LLE [13] works on local neighbors aiming to preserve the local geometry of the dataset. SNE [7] is somewhat in the middle of those two methods. This method computes the probability that two points take each other as neighbors in the high and the lower dimension, and then tries to match between the two probability distributions. Hence, it works for local geometry structure preservation but also keeps points distant in the high dimensional space distant in the visualization space. Because of its advantage, in this paper, we employ the SNE algorithm. Besides, to overcome the problem of linear distance computation, the distance computation step in the ISOMAP algorithm is combined to extract graph based distance. The combination is called ISOSNE. In the original ISOMAP, there are three main steps. First, the algorithm builds the neighborhood graph between every points using k-nearest neighbors or -nearest neighbors. If there is an edge between all points, the distance between them is set as the dissimilarity value between two corresponding objects. The second step uses Dijkstra’s algorithm to find shortest paths between every pair of points, the distances between them are then redefined as the sum of all the edges linking from one point to the other. After the distances are recomputed, the MDS is applied to the new distance matrix. Figure 2 shows an example of the advantage of the ISOMAP method. Assume an image collection follows an S-curve in 3-dimensional space. For a normal distance computation (the left figure), such as Euclidean distance, the nonlinear manifold of the data as shown in the right image is ignored. This means points far from each other along the manifold can yield short distances. The ISOMAP algorithm in the right image defines distance following the structure of the manifold, which is the real distance between two points in this manifold. In the ISOSNE, for any given data set, similar to the first two steps in the ISOMAP, we recalculate the distances between images. Finally, the SNE is then applied to the new distance matrix. The working principle of the SNE can be briefly described as follows. Given a dissimilarity matrix D = {dij }, for each object in the original space the algorithm computes the probabilities that a point would take others to be its neighbors P = {pij }:
Isomap algorithm: redefine distance as shortest path in neighborhood graph
Example of high dimensional feature space
High dimensional feature space with redefined distance
Figure 2. Example of nonlinear structure in high dimensional feature space
exp(−d2ij ) 2 . k6=i exp(−dik )
pij = P
(1)
with pii = 0. Assuming each image is represented as a point. In the visualization space, the SNE initializes all points at random positions {yi }N i=1 . The induced probability Q = {qij } is then calculated for every pair of points: exp (−||yi − yj ||2 ) , qii = 0. 2 k6=i exp(−||yi − yk ||
qij = P
(2)
Finally, using the Kullback-Leibler distance between two probability distributions P and Q, the SNE aims at minimizing the following function: XX pij C= pij log . qij i j We experimented with the ISOSNE with some synthetic nonlinear manifold sets, as well as real data sets to proof the performance of the algorithm. Figure 3 shows the results of the test with some synthetic data. The ISOMAP tends to keep the global data hence in some cases fails in assuring local structure preservation. The SNE itself uses direct distance between points, hence loses the real structure of the data. The combination of ISOMAP and SNE takes the advantage of the two methods and gives the best result. We also experimented with LLE but it does not give satisfactory results. For real data, image sets are taken from different video sequences. Figure 4 shows results of mapping video frames represented by Lab color histograms (96 dimensions) to the visualization space. Figure 4a is an example of a linear structure with a camera capturing the moving of a car along a street, the video sequence then has a linear structure. Figure 4b shows a repeated structure where the video captures the conversation between two people. It first shows the whole scene of two talking people, then the camera moves from one person to the other, and finally shows the whole scene again. Clearly, the mapping results indeed keep the original structures.
3.2 Overview As discussed above, showing all images to the user is not appropriate since the visualization space is limited and the user will get lost in a crowded net of images. Hence, we need to select a representative set of images to show to the user. To that end, we apply the k-means algorithm in clustering the dataset. In order to preserve the structure of the data, distances between points are derived using graph-based distance computation as presented in the previous section. With this, the k-means is modified to adapt with our requirement. Assume we classify the collection into k clusters. The k-means is first initialized with random positions for k center points. Each center point is one of the existing points in the dataset. Once the center points are defined, we then find points belonging to the same cluster. Within each cluster Mi , a new center point for the next iteration (t+1) mi is computed as:
Original data ISOMAP SNE ISOSNE
Figure 3. Example of mapping synthetic nonlinear data structure from high to lower dimension using ISOMAP, SNE, and combination of ISOMAP and SNE (ISOSNE)
Figure 4. Example of mapping video sequence from high to lower dimension using ISOSNE
(t+1)
mi
= min
xk ∈Mi
X
d(x, xk )
x∈Mi
The process is repeated until the maximum number of iterations is reached. In figure 5, we show an example of using original distance in k-means and the improved k-means. The given data has a spiral form. As the structure of the data is preserved using ISOMAP, the improved clustering also follows the structure of the data, hence performs better than the original ones.
Figure 5. Example of k-means clustering, from left to right are results with and without graph based distance computation. The lines indicate clustering results. On the left, distances are obtained follow the graph of the data, hence the clustering performs better than the one on the right. On the right, as direct distance is computed points should be far from each other are now clustered into the same group.
The evaluation of the clustering certainly depends on the given number of clusters, and how well the center points cover the whole data set. Evaluating the performance of a clustering algorithms is still a difficult question [8], [6], [1], [24]. For later purpose, we need a normalized evaluation function, hence we use the modified Hubert statistic [6] as clustering measure. It is given as follows: CO =
r − M p Mc σ p σc
where r = (1/M )
XX
d(i, j)d(mi , mj ) XX Mp = (1/M ) d(i, j) XX Mc = (1/M ) d(mi , mj ) XX σp2 = (1/M ) d2 (i, j) − Mp2 XX σc2 = (1/M ) d2 (mi , mj ) − Mc2 and M = k(k − 1)/2, d(i, j) is the distance between i and j, mi is the center of ith cluster.
(3)
3.3
Visibility
So far we only considered point sets. When each point is represented as an image, we need to obey the visibility requirement. In this section, we show how to give the visibility to displayed images. The overlap depends on the number of images displayed, and on the relations between them. The closeness on the visualization space shows the similarity of an image compares to the others. This is because of the result of mapping from the high to 2-dimensional space (section 3.1). Figure 6 shows an example of displaying 50 and 150 images in the visualization space. When there is a lot of overlap it is very difficult for the user to see the content of the overlapped images.
Figure 6. Example of overlap between images in the visualization space in case of 50 and 150 images.
To calculate the overlap between displayed images, we define the overlap function, which will be described now. For simplicity, assume that all images have width w and height h, which are equal. Finding the overlap between two squares is not difficult when their center points and sizes are known. However, many cases have to be distinguished. We find that presenting an image as a circle with center point and radius R = w2 (figure 7a) is simpler in finding the overlap area, also in defining general mathematical formula. The overlap or the intersection area between two circles i and j is as: ( dij dij R2 2 arccos 2R − sin 2 arccos 2R , if dij < 2R, Oij = (4) 0 , otherwise. where dij is the distance between image i and j. On the other hand, the circle area represents an image cover π4 ' 80% of the image area. This means that if two images overlap outside the circle area and inside the image area (figure 7b), the viewable area of the image is larger than 75% of the image area, which we assume is sufficient for the user to interact with. Therefore, if two images are overlapped in the area outside the circle, the overlap between them is then set to 0. Besides, the less overlap the better the visibility. Hence, if the number of displayed images is k, the total visibility cost is defined as: k X k X 1 Oij CV = 1− . k(k − 1) i ΠR2 j6=i
(5)
h
R = w/2
w Overlap := 0
Figure 7. (a) Image and its circle area. (b) Overlap between two images
3.4
How to balance the requirements?
There are two main relations between the three requirements. First of all is the selection of number of representative images. From equation 3, the more images, the better the overview, while the visibility is reduced (equation 5). Hence, balance these two requirements finds the optimal number of selected images to be displayed. We have: C1 = λ1 CO + (1 − λ1 )CV
(6)
where 0 ≤ λ1 ≤ 1. The aim is for given λ1 to find k where C1 (k) has the maximum value. For that target, since this step is done offline, we calculate for the given data CO and CV with a range of k from 2 to kmax (normally we select kmax = 300), then calculate C1 for final decision of selecting k: C1 (k)→k max . The second relation is the relation between visibility and structure preservation requirement. Obviously simply spreading images to increase the visibility of displaying the images will break the original structure. This means the more visibility, in other words less overlap, the less structure preservation. Hence, we add a constraint to the overlap function as the balance between the two requirements. The problem boils down to finding the best optimal positions where the cost of overlap and structure preservation reach the minimum value. For a given set of displayed images, similar to the minimize function in SNE, we first compute the distribution of images in the original positions, then the distribution of images in the new position with the overlap reduction. The combination of the two constraints is: C2 = λ2 CS + (1 − λ2 ) (1 − CV ) P Q,
P P
(7)
where 0 ≤ λ2 ≤ 1 and CS = i j P log with P , Q are the distributions of points in old and new positions, respectively. Gradient descent method is applied for finding the optimal solution for C2 , which requires to compute y (t+1) = (t) 2 y − ∂C(t) where: ∂yi
The differentiation of CV is given in [7]: X ∂CS =2 (yi − yj )(pij − qij + pji − qji ). ∂yi j Given the function 5 above, we then derive:
∂CV 1 = ∂yi k(k − 1)ΠR2
X
(yi − yj )
q − 4R2 − d2ij
j
dij
.
3.5 Scheme
Feature space Image collection
Isomap+Sne Visualization space
Balancing function Overview
Visibility Offline process
Representative set
Visualization process
Balancing function Next Visibility
Structure preservation
Visualization Figure 8. Scheme of the image collection visualization system.
Up to this point, we have analyzed the requirements and how to solve them. We now present the scheme of the visualizing image collections system, which is shown in figure 8. The scheme includes two main process: offline processing and visualization. In the offline process, we precompute the features for all images in the collection. From there, the ISOSNE is applied to find the mapping to the visualization space. The computation complexity of this step is O(N 2 ), where N is the size of the collection. When the positions are found, we calculate CO and CV with different values of k ∈ [2..kmax]. After that, the balancing function in eq.6 returns the number of images should be displayed at one times. This number is then the input for the graph based k-means clustering algorithm. Each image in the collection is assigned to the corresponding cluster. This step is done once before-hand. The second process is to adjust the positions of the current set of images displayed. This set contains images which are closest to the corresponding center points and have not been displayed before.
4
Experiment and results
In this section based on the proposed scheme, we present the procedure to construct a visualizing image collection system. The collection contains 10000 Corel images with different scenes, and objects. In the offline stage, each image is represented by its Lab color histogram with 32 bins at each color channel. The ISOSNE is applied for finding 2D positions. Since the SNE uses gradient descent method to find the optimal positions, with a large size of the collection, this step takes quite some time. In particularly, with 10000 points the total time for finding the final result is approximately 10 hours. For the visualization, we only store the final positions. Figure 9a shows the mapping results of 10000 points to the 2-dimensional space.
Figure 9. a. Value of C1 with λ1 = 0.5 over changes of k. b. Result of mapping 10000 Corel images from 96 Lab feature space to 2-dimensional space, red points represent for selected center points.
With the result, we calculate the CO and CV with k ∈ [2..300] and λ1 = 0.5, which means the two requirements are equally important. Applying the obtained results in equation 6, we then have k = 53. Figure 9a shows the results with different values of k, the red circle points out the selected k with maximum value of C1 . With known number of clusters, the collection is classified into 53 clusters. Each image is then assigned the corresponding cluster’s index. In the second process, in the first round of visualization, images closest to the center points are selected to be displayed. Each image represents one cluster. The second balancing function C2 with λ2 = 0.5 is used to find the optimal positions for the displayed set. This process is repeated to display the next set of images. Figure 10 shows the interface of the system. The upper-left corner shows the whole collection as a point set with red points representing the currently displayed set. The bottom-left corner shows the bigger size image of the current selected thumbnail. The main screen shows the representative set as images.
5
Discussion
When the size of the collection is getting larger, the number of representatives should also increase to be able to give the overview of the data. Since the visualization space is fixed, increasing the number of displayed images will reach the limitation of the screen space. This increase leads to the difficulty in finding the optimal solution for eq.7. Let us assume that an image is called viewable if its visible area occupies k% of the image. Every displayed images has the same size of w and h, and the visualization space has size W and H. The maximum number of images that the proposed method can afford to reduce the overlap will be H×W h×w × k. On the other hand, the value of λ2 also affects the result. With λ2 equal to 1.0, only the structure preservation is concerned. When λ2 is closer to 0, images are more spread out, as well as less structure is preserved. An example is shown in figure 11a with 100 images in case λ2 is 1, 0.5 and 0, respectively. Figure 11b, 11c show examples of adjusting value λ2 ∈ [0..1] in eq.7. For example, in case of only 50 images, the overlap between them is small hence
Figure 10. A screendump of the visualization system.
the balancing function easily finds a solution for structure preservation as well as visibility. However, in case of 200 images, even there is almost no structure preservation, the visibility overall does not increase very much. However, in fact, if there are too many images displayed it will be confusing for the user. Hence the number of displayed images should be chosen such that it is sufficient to represent the data set as well as suited for interaction.
6
Conclusion
Visualization is the best support tool for exploring visual collections. In literature, problems in visualizing those collections have not been made explicit, especially in content based retrieval. Since the size of the collections are getting much larger showing the whole collection is not a proper way. Hence there needs a sufficient dissection of all those problems. In this paper, we analyzed and proposed three general requirements for similarity based visualization systems: overview, visibility, and structure preservation. However, these requirements are not independent from each other. In order to give the best results of visualizing there must be compromises among them. We proposed a novel cost function and algorithm that can be used to define the importance of these requirements to the overall visualization goal. Using a rather large data set (i.e. 10000 points) we conducted an experimental work to ascertain the proposed algorithm frame work. The results support the theory analyzes quite well and it shows that the given solutions give better visualization for exploring large image collections.
References [1] U. Brandes, M. Gaertler, and D. Wagner. Experiments on graph clustering algorithms. Proc. 11th European Symp. Algorithms, LNCS 2832:568–579, 2003. [2] G. Caene, G. Frederix, A. Kuijk, E. Pauwels, and B. Schouten. Show me what you mean! pariss: A cbir-interface that learns by example. Proceedings of the 4th International Conference on Advances in Visual Information, pages 257–268, 2000.
(a)
Percentage of 75% viewable area over changes of λ2
Relation between CV and CS over changes of λ2
1
50 images 100 images 150 images 200 images
0.9
1 0.8 0.7
0.8
percentage
CV
0.6 0.5
0.6
0.4
0.4 0.3 0.2
0.2 50 images 100 images 150 images 200 images
0.1 0
0
0.1
0.2
0.3
0.4
0.5 CS
(b)
0.6
0.7
0.8
0.9
1
0
0
0.1
0.2
0.3
0.4
0.5 λ
0.6
0.7
0.8
0.9
1
2
(c)
Figure 11. (a) Example of displaying 100 images with λ2 = 1, 0.5, and 0, respectively. (b) Example of the changing of CS and CV values by adjusting λ2 in case of showing 50, 100, 150, and 200 images. (c) Example on percentage of images with visible area is 75% with changes of λ2 .
[3] C. Chen, G. Gagaudakis, and P. Rosin. Content-based image visualization. IEEE International Conference on Information Visualization, pages 13–18, 2000. [4] L. Cinque, S. Levialdi, A. Malizia, and K. Olsen. A multidimensional image browser. Journal of Visual Language and Computing, 9:103–117, 1998. [5] J. Costa and A. Hero. Geodesic entropic graphs for dimension and entropy estimation in manifold learning. IEEE Transactions on Signal Processing, 52(8):2210–2221, 2004. [6] R. Dubes. How many clusters are best? - an experiment. Pattern Recognition, 20(6):645–663, 1987. [7] G. Hinton and S. Roweis. Stochastic neighbor embedding. Neural Information Processing Systems 15., 2002. [8] A. Jain, M. Murty, and P. Flynn. Data clustering: A review. ACM Computing Surveys, 31(3):264–323, 1999. [9] W. Leeuw and R. Liere. Visualization of multidimensional data using structure preserving projection methods. Data Visualization: The State of the Art Editors, pages 213–224, 2003. [10] B. Moghaddam, Q. Tian, N. Lesh, C. Shen, and T. Huang. Visualization and user-modeling for browsing personal photo libraries. International Journal of Computer Vision, 56(1-2):109–130, 2004. [11] M. Nakazato and T. Huang. 3D MARS: Immersive virtual reality for content-based image retrieval. Proceedings of International conference on Multimedia Expo, 2001.
[12] R. Pless. Image spaces and video trajectories: Using isomap to explore video sequences. IEEE International Conference on Computer Vision, pages 1433–1440, 2003. [13] S. Roweis and L. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), pages 2323–2326, 2000. [14] Y. Rubner, C. Tomasi, and L. Guibas. The earth mover’s distance as a metric for image retrieval. International Journal of Computer Vision, 40(2):99–121, 2000. [15] S. Santini, A. Gupta, and R. Jain. Emergent semantics through interaction in image databases. IEEE Transaction of Knowledge and Data Engineering, 13(3):337–351, 2001. [16] B. Shneiderman. Information visualization: Dynamic queries, starfield displays, and lifelines. short paper: http://www.cs.umd.edu/hcil/members/bshneiderman/ivwp.html. [17] V. Silva and J. Tenenbaum. Global versus local methods in nonlinear dimensionality reduction. Advances in Neural Information Processing Systems, 15, pages 705–712, 2002. [18] A. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain. Content-based image retrieval at the end of early years. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(12):1349–1380, 2000. [19] J. Tenenbaum, V. Silva, and J. Langford. A global geometric framework for nonlinear dimensionality reduction. Science 290 (5500), pages 2319–2322, 2000. [20] R. Veltkamp and M. Tanase. A survey of content-based image retrieval systems. In O. Marques and B. Furht (Eds.), Content-based image and video retrieval, Kluwer Academic Publishers, pages 47–101, 2002. [21] C. Vertan, M. Ciuc, C. Fernandez-Maloigne, and V. Buzuloiu. Browsing image databases by 2d image similarity scatter plots: Updates to the iris system. Proceeding of Intl. Symp. Communications, 2002. [22] J. Wang, J. Li, and G. Wiederhold. Simplicity: Semantics-sensitive integrated matching for picture libraries. IEEE Transaction on Pattern Analysis and Machine Intelligence, 23(9):947–963, 2001. [23] L. Yang. Distance preserving projection of high dimensional data for nonlinear dimensionality reduction. IEEE Transactions on pattern analysis and machine intelligence, 26(9):1243–1246, 2004. [24] Y. Zhao and G. Karypis. Criterion functions for document clustering: Experiments and analysis. Technical report tr 01–40, Department of Computer Science, University of Minnesota, Minneapolis, MN, 2001.
