3D Res. 04, 01(2013)3 10.1007/3DRes.01(2013)3
3DR EXPRESS
w
A hybrid-optimization method for assessing the realizability of wireframe sketches
Philip Azariadis • Sofia Kyratzi • Nickolas S. Sapidis
Received: 15 July 2012 / Revised: 25 October 2012 / Accepted: 12 December 2012 © 3D Research Center, Kwangwoon University and Springer 2013
Abstract * This paper introduces an optimization strategy for evaluating the realizability of a 2D wireframe sketch that conveys geometric and topological information about a 3D solid model. Applying the cross-section realizability criterion, one is able to assert whether a wireframe sketch is a true orthogonal projection of a 3D-solid. In this work, we first review current sketch interpretation methods and realizability criteria, and then we focus on an algebraic system derived from the cross-section realizability criterion. A two-phase hybrid-optimization approach for deriving cross-sections of a given wireframe sketch is introduced. In the first phase, a Genetic Algorithm is employed to produce an initial solution (i.e., an initial cross-section), which is refined by a Conjugate Gradient method in the second phase of the proposed approach. The final cross-section is an accurate solution of the aforementioned algebraic system. Then we are able to test sketch’s realizability utilizing four criteria which are derived from the crosssection realizability criterion and the applied optimization procedure. The proposed optimization strategy is tested on wireframe sketches with accurate geometry and also on wireframe sketches with inaccurate geometry. Experimental numerical results are presented to illustrate the effectiveness and robustness of the proposed method. Keywords wireframe sketches, Conjugate Gradient method, Cross-section realizability criterion
Philip Azariadis1 ( ) • Sofia Kyratzi1 • Nickolas S. Sapidis2 1 Department of Product and Systems Design Engineering, University of the Aegean, Hermoupolis, Syros, Greece 2 Department of Mechanical Engineering, University of Western Macedonia, Kozani, Greece E-mail:
[email protected] Web site: www.syros.aegean.gr/users/azar
1. Introduction This research focuses on “sketches”, which are a primary tool in preliminary/concept design and for communicating ideas between different working groups. Besides the evolution of Computer-Aided Design programs, there is still lack of an efficient sketch-based application to support the conceptual phase of product design. This is due to the fact that, for a computer-based application, a sketch is considered as a 2D configuration of lines intersecting in junctions without any depth (3D) information. In contrast, humans have the ability to intuitively realize the 3D shape from a given 2D sketch, even if the latter contains geometric errors. It is exactly this human ability that researchers (from diverse fields, such as Geometric Modeling, Computer Vision, and Artificial Intelligence) try to emulate by developing methods and tools for the interpretation and algorithmic “translation” of a sketch into a 3D geometric model. This research is related to the “sketch-to-solid” problem, i.e., to the construction of a polyhedron from a given single sketch1-6. A sketch is considered to be a line drawing, i.e., a set of straight lines on a plane meeting at junctions. Handwritten sketches are regarded to have been transformed into line drawings. A sketch is either a complete wireframe sketch (Figure 1(a)) or a natural sketch, i.e., a sketch without hidden lines (Figure 1(b)). Loops of lines and junctions form regions of the sketch (see the bold-marked line path in Figure 1(c)). There is an “one-to-onecorrespondence” between the lines (L), junctions (J), and regions (R) of a sketch and the edges (E), vertices (V), and faces (F) of the polyhedron to-be-constructed. The present research is limited to sketches of trihedral polyhedra (where exactly three faces meet at each vertex), without holes. Although this is the simplest case of polyhedra, there is still no robust method for the solution of the “sketch-to-
2
3D Res. 04, 01(2013)3
solid” problem for this kind of solids. Researchers dealing with the “sketch-to-solid” problem4-6 focus on the topological, geometric or semantic correlations between a sketch and the corresponding 3D model. For this reason, numerous heuristic- and/or mathematics-based methods have been developed for the interpretation of a sketch as a solid model. Sketch interpretation techniques are
(a)
based on the fact that the ability of humans to correctly realize the 3D models depicted in sketches relies on the visual experience that they obtain by observing the patterns that exist between the 3D geometry and its 2D projections7. Thus, methods of this kind focus on extracting sketch information that concerns the shape of the depicted 3D model.
(b)
(c)
Figure 1 (a) Α wireframe sketch, (b) a natural sketch, (c) a region R of a sketch.
In parallel, many researchers have been studying the realizability problem of a sketch. In this case, the junctions of a sketch should be drawn in special positions that establish the existence of a 3D solid from the sketch8. During the last decades, numerous geometric and algebraic criteria have been introduced in order to evaluate whether a sketch identifies with the projection of a 3D solid model, i.e., if a sketch is realizable. Focusing on criteria/methods of this kind, one encounters the cross-section criterion introduced by Whiteley9. The cross-section criterion is a necessary and sufficient geometric condition for the realizability of a sketch and is applied to wireframe sketches (i.e., sketches that include both visible and hidden lines) that depict trihedral solids. This paper focuses on the realizability problem of sketches and develops an optimization strategy that can be utilized for testing the realizability of wireframe sketches with accurate and inaccurate geometry. Within a “sketch-tosolid” algorithm, the realizability test of a sketch provides an accurate tool for distinguishing sketches that correspond to a real 3D model from those that depict nonsense 3D structures. The proposed method is based on the crosssection criterion9 and on the algebraic model10, 11 and solves, simultaneously, both the cross-section generation problem and sketch realizability problem. Particularly, the algebraic model10, 11 has been improved to represent more precisely all feasible cross-sections that can be generated by a single sketch. The proposed method uses a hybrid optimization approach that explores the advantages of Genetic Algorithms (GAs) and Conjugate Gradient (CG) method. The structure of the paper is as follows: Sections 2 and 3 review current sketch interpretation methods and realizability criteria, Section 4 describes the cross-section criterion and its algebraic model, and Section 5 presents the proposed hybrid optimization approach for the realizability evaluation of a sketch. The paper concludes by presenting and discussing test examples derived by applying the proposed approach to sketches of accurate geometry and also on sketches with inaccurate geometry.
2. Sketch Interpretation Techniques Interpretation of a sketch as a 3D object refers to the
identification of sketch features and 2D lines-junctions configuration that convey information concerning the shape, the geometry and the topology of the corresponding 3D solid model. The analysis of these sketch characteristics enables the emulation of human ability to realize the 3D shape of a sketch. In other words, sketch interpretation techniques are developed on an effort to capture the “design intent” appearing in a sketch. Company et. al.12 classify sketches to “thinking sketches”, “talking sketches” and “perspective sketches” and reviews the different methods and characteristics concerning their interpretation and employment in a Computer-Aided Sketching program. The research on sketch interpretation originates in the beginning of 1970, when Huffman13 and Clowes14 studied all possible projections from different views of a 3D corner in a trihedral object and introduced a labeled Junction Catalogue that includes for each junction type all possible line label assignments (Figure 2). L-junctions
Arrow junctions
Y- junctions
T- junctions
Figure 2 The Junction Catalogue shows the different types of junctions and the valid labels of their adjacent lines.
Each line in the sketch can be assigned with one of the four labels {+, −,Æ,Å}.The label “+” (“−”) implies, in the polyhedron-to-be-constructed, a convex (concave) edge (Figure 3). The labels “Æ,Å” indicate that the corresponding edge is associated to one visible and one hidden face of the polyhedron. Given a junction with three adjacent lines, the lines should be labeled so that the
3D Res. 04, 01(2013)3
resulting configuration around the junction belongs to the Junction Catalogue. A sketch is called labelable if and only if it can be associated to at least one such valid set of labels8. Many researchers extended the Junction Catalogue for wireframe sketches8, for natural sketches of tetrahedral solids3, for sketches with missing boundaries15, for sketches of curved objects16 and sketches with shadows17. Line labeling (LL) as a sketch interpretation method facilitates the construction of the frontal (visible) geometry induced by natural sketches18, 19. On the basis of line labels, one is able to determine for an underlying 3D model: its internal and boundary edges, information regarding its 3D shape (i.e., concave or convex parts – see Figure 3), its spatial position, and its visible, partial visible and hidden faces. Moreover, line-labels indicate occluding parts of the object and combined with the Junction Catalogue facilitate the construction of the hidden topology. Although line labeling method is not necessary for the interpretation of sketches derived from trihedral solids18, it becomes a significant tool for distinguishing occluding T-junctions from non-occluding T–junctions in sketches depicting tetrahedral objects (i.e., objects where there is at least one vertex that belongs to four faces).
(a)
(b) Figure 3 (a) A labeled natural sketch, (b) the corresponding 3D model follows the shape that is implied by line labels.
Topology and geometry of a sketch convey valuable information about the topology and geometry of the polyhedron-to-be-constructed. On the basis of this correspondence, topological relations and geometric features of a sketch allow the development of a sketch interpretation method that is mainly based on mathematical tools. The regions of a sketch, which correspond to the faces of the depicting 3D model, can be determined according to graph theoretical tools combined with wellestablished theorems of Euclidean Geometry and Solid
3
Modeling10, 11, 20, 21, 3 or Gestalt theory22. Solid modeling rules enable the identification of boundary (occluding) lines (edges) and imply the existence of hidden regions (faces) in a natural sketch. Tools and theorems of Graph Theory, Topology, Euclidean and Projective Geometry, and Solid Modeling are also employed for the definition of the hidden part of a sketch resulting either in a minimum solution18, 19 or in a plausible one21. An alternative sketch interpretation technique is based on “image regularities”23-25. Image regularities are heuristic tools employed for the geometric definition of the corresponding solid. These heuristics are based on “correlations” that exist between the geometry of the sketch and the 3D geometry of the corresponding polyhedron. They are categorized to regularities reflecting a spatial relationship among (a) individual entities (e.g., parallel lines in a sketch correspond to parallel edges of the polyhedronto-be-constructed), (b) a selected group of entities (e.g., symmetry), and (c) all entities in the sketch (e.g., all line lengths in the sketch are proportional to all edge lengths in the polyhedron-to-be-constructed)23. Company et al.25 classify the image regularities to “true” and “false”, according to their probability to correspond to a plausible feature in the 3D model. Each “image regularity” detected in the sketch is weighted according to the probability that the identified geometric feature corresponds to a real geometric feature in the polyhedron (e.g., the “parallel lines” regularity is assigned a high “weight”, because one usually draws – in orthographic projections – parallel edges of the polyhedron as parallel lines in the sketch). All detected “regularities” are modeled in a “compliance function”, whose optimization calculates the geometry of the polyhedron’s vertices20. The result of the optimization process depends highly on the set of the employed regularities and the weights assigned to each of them. This, combined with the fact that the regularity set is based on experimental observations, may conclude in local minima, and thus to a non-valid solid model20, 25, 26. In order to generate an optimal compliance function, Yuan et. al.26 proposes a method that automatically determines the most effective regularity set for a given sketch. Their selection method is based on neural network and data mining combined with a 3D object library that includes different models and their parallel projection from multiple views. In particular, the objects’ projections produce numerous line arrangements that provide a good measurement tool for the possibility of a selected regularity set.
3. Sketch Realizability A sketch is realizable if it is identical with the projection of a polyhedron. This means that a sketch must have its junctions at “special locations” in order for this sketch to correspond to the projection of a valid polyhedron8 (Figure 4). Since the late seventies, a large number of heuristic27-29 algebraic8, 11, 30 or geometric criteria9, 31, 32 have been presented dealing with “sketch realizability.” The Line labeling method (see Section 2) is a typical example of a heuristic sketch-realizability criterion. Its major disadvantage is that it provides only a necessary condition for the existence of a polyhedron corresponding
4
3D Res. 04, 01(2013)3
to a sketch11. Varley et. al.18 reconsider the usefulness of Line Labeling and conclude that this is only valuable as a sketch interpretation tool for sketches of tetrahedral objects. Sugihara8 proposes a necessary and sufficient condition for the realizability of a wireframe sketch that is based on the Line Labeling method. In particular, on the basis of Line Labeling, he develops two methods to acquire the “incidence” structure (i.e., which vertices lie on which edges or faces of the polyhedron) and the “spatial” structure of a sketch (identifying, e.g., pairs of regions of the sketch corresponding to polyhedron faces where the first face is in front of the second). Then, he constructs a linear system of equalities and inequalities, which is generated with respect to these structures, and whose solution corresponds to the geometry of a polyhedron. The same author introduces30 the “resolvable sequence” of a wireframe sketch; this is a specific order by which all elements of a sketch can be “lifted” in space so that finally a polyhedron is produced. According to this method, a sketch is realizable if there is at least one “lift” for a given sketch. The works31, 33, 34 present a series of geometric criteria that are based on necessary “line-concurrence” conditions for the realizability of a sketch. Given the fact that the projection preserves the concurrences of the projected edges,
(a)
the “line-concurrence conditions” are formed with respect to the “edge-alignment” and “edge-concurrence” conditions that exist among the edges of a polyhedron. On the basis of these conditions, Ros and Thomas34 developed a graphbased method that employs Delta/star reductions to complicated sketches in order to evaluate the consistency of sketch lines with respect to the aforementioned conditions. On the basis of the above line-concurrence conditions Whiteley9 proposes a geometric criterion that checks the realizability of a sketch with the help of a “cross-section” that is constructed from the sketch. This criterion is a necessary and sufficient condition for a sketch to be realizable. According to this criterion, a cross-section is constructed from a given sketch so that each region/line of the sketch is represented by a line/point on the cross-section (see Figure 4). The sketch is realizable if and only if the cross-section is compatible with the sketch (see an analytic description of this criterion in Section 4). Ros and Thomas32 revisit Whiteley’s “cross-section criterion” and develop a simplified description of it. We employ this criterion in a “sketch-to-solid” method2, 11 and propose an algebraic model (Section 4.1) that can be successfully used for the realizability evaluation of a sketch.
(b)
(c)
Figure 4 The (a)“edge-alignment” and (b-c)“edge – concurrence” conditions of a polyhedron provide necessary line-concurrence conditions for the realizability of a sketch.
(a)
(b)
(c)
Figure 5 The cross-section criterion for: (a) a polyhedron and (b) a polyhedron’s projection. (c) The polyhedron’s wireframe sketch and its compatible cross-section.
4. The Cross-Section Criterion Given a trihedral polyhedron P (Figure 5(a)), consider an arbitrary plane Ф which is "in general position" with respect to P, i.e., Ф is not parallel to any face of P and also Ф does not intersect P. Intersecting the planes of P’s faces with Ф produces an arrangement of lines comprising the crosssection of P. If each pair of cross-section lines Lfi and Lfj (corresponding to the two faces Fi and Fj) intersect at a point Pij on (the extension of) the common edge eij of Fi and
Fj, the cross-section is called compatible. The above concurrence conditions hold true also for the wireframe sketch produced by the projection of the polyhedron onto the plane Ф, because projection preserves collinearity of points and all incidence relations. Thus, given a wireframe sketch S (Figure 5(b)) with L lines, J junctions and R regions, a cross-section of S is an arrangement of lines {Lfk ; k = 1,...R} that represents the regions {Rk ; k = 1,..., R} of S (Figure 5(c)). In complete analogy to the above discussion, if each pair of cross-section lines Lfi and Lfj
3D Res. 04, 01(2013)3
intersect at a point Pij on (the extension of) the common line eij of Ri and Rj, the cross-section is called compatible with S32. On the basis of the “sketch-to-solid” problem, this paper focuses on the development of a hybrid optimisation method that evaluates the realizability of a given sketch. The generation of a cross-section from a sketch can be considered as an intermediate step necessary for the application of the sketch realizability conditions. Whiteley9 was the first to introduce a cross-section theorem establishing the realizability of a wireframe sketch. Ros & Thomas32 rewrote Whiteley’s theorem as follows: Theorem 1: (Cross-Section Criterion) A wireframe sketch is realizable if and only if it has a compatible crosssection, where the cross-section lines Lfi and Lfj of the adjacent regions Ri and Rj are not identical. Theorem 1 establishes the existence of more than one cross-section for a given sketch11. According to the aforementioned geometrical definition of the cross-section criterion, the cross-section that is produced from a given polyhedron P identifies with the cross-section generated by the polyhedron’s projection (which is a wireframe sketch) on plane Ф.
5
sketch. Theorem 2: A set of lines {Lfi} satisfies the properties (B), (C) of Definition 1 if and only if the following hold true: (C1) ai b j ≠ a j bi , (C2) kr ai − mr bi ≠ 0 , where i, j ∈[0,..., R − 1] correspond to the two regions Ri and Rj that are adjacent to the sketch line eij : kr x + mr y + nr = 0 , r = 0,..., L - 1 . Property (D) is related to the compatibility of the crosssection with a sketch. According to it, each intersection point c a − c j ai c j bi − ci b j (1) Pij = i j , ai b j − a j bi ai b j − a j bi of two cross-section lines Lfi and Lfj that correspond to two adjacent regions Ri and Rj, must be on the extension of the regions’ common sketch line eij : kr x + mr y + nr = 0 . This constraint forms each equation Fr, with r = 0,..., L - 1 of the following cross-section system est ↔ F0 : (ct as − cs at )k0 + (cs bt − ct bs )m0 + (at bs − as bt )n0 = 0
4.1 Algebraic model of the cross-section criterion This section presents an algebraic model of the crosssection criterion along with the constraints establishing its definition from a wireframe sketch. The following Definition is a revised version of Definition 111. With the new definition the cross-section lines Lfi are allowed to pass through the origin of the coordinate system, which, in turn, results in the generation of a richer set of cross-sections that comply with the above geometric definition of the criterion. Indeed, although Theorem 1 implies the existence of numerous cross-sections for a realizable sketch, the restriction of rejecting cross-section lines that pass through the origin, limits the number of the feasible cross-sections that can be derived from a given sketch. Definition 1: (Cross-Section Compatibility) Let S be a wireframe sketch with R regions, L lines and J junctions (Figure 5(b)). A compatible with S cross-section is a set of lines {Lfi} such that: (A) Each cross-section line Lfi that corresponds to region Ri, is written in the form: bi x + ai y + ci = 0 , with i = 0,..., R − 1. (B) The cross-section lines Lfi and Lfj of two adjacent regions Ri and Rj are not identical. (C) For each region Ri, its adjacent sketch lines eij intersect the cross-section line Lfi. (D) Each line eij of S that is adjacent to regions Ri and Rj, and the corresponding to these regions cross-section lines Lfi and Lfj intersect at a point Pij. Property (B) is based on the constraint of Theorem 1 for the cross-section lines of adjacent regions, while Property (C) is a prerequisite for the existence of a cross-section, i.e., the geometric definition of a cross-section from a polyhedron implies that the sketch lines eij of a region Ri must also intersect the cross-section line Lfj11. Properties (B) and (C) are introduced, with the following theorem1, as constraints for the generation of a cross-section from a
.............................................................................. e pq ↔ Fr −1 : (c p aq − cq a p )kr −1 + (cq bp − c p bq )mr −1 + (a p bq − aq bp )nr −1 = 0 eij ↔ Fr : (ci a j − c j ai )kr + (c j bi − ci b j )mr + (ai b j − a j bi )nr = 0 e pn ↔ Fr +1 : (cn a p − c p an )kr +1 + (c p bn − cnbp )mr +1 + (anbp − a p bn )nr +1 = 0 eqn ↔ Fr + 2 : (cn aq − cq an )kr + 2 + (cq bn − cn bq )mr + 2 + (an bq − aq bn )nr + 2 = 0 ............................................................................... ekp ↔ FL −1 : (c p ak − ck a p )k L −1 + (ck bp − c p bk )mL −1 + (a p bk − ak bp )nL −1 = 0
(2) System (2) includes 3*R unknowns and L equations. The unknowns of the system are the cross-section line coefficients (bi , ai , ci ) . System (2) combined with the constraints of Theorem 2 defines the “Cross-Section Problem” (CSP), whose solution determines the realizability of a sketch.
5. The Proposed Optimization Methodology to Solve the CSP We presented an iterative algorithm (“ILA”)11 for the solution of the CSP that proceeds in complete analogy to the geometric construction of a cross-section32. In particular, ILA incrementally linearizes the equations of the “crosssection system” provided that two cross-section lines Lfs and Lft corresponding to two adjacent regions Rs and Rt are initially determined. As long as Lfs and Lft comply with the constraints of the CSP problem, the rest of the cross-section lines (corresponding parameters bi , ai , ci = 1 ) are iteratively calculated. In this work, the “cross-section” system (2) include one more set of variables
ci , and, therefore, the number of
unknowns is increased to 3*R. ILA, as it is currently designed, cannot provide a unique solution for the variables sets (bi , ai , ci ) of system (2). Thus, an alternative numerical optimization method is employed that contrarily to ILA exploits simultaneously all the equations of system (2).
6
3D Res. 04, 01(2013)3
5.1. Basic assumptions and requirements of the optimization methodology CSP is a multi-modal nonlinear system with nonlinear inequalities. Here, we present some basic assumptions and requirements that should be taken into account for solving this problem using a nonlinear optimization method. Every solution (bi , ai , ci ) , i = 0,..., R − 1 should satisfy all equations of system (2), thus the adopted optimization method should be capable of providing a global optimal solution where all equations of system (2) equal to zero and all constraints of Theorem 2 are satisfied. Hence, the selected optimization strategy should be able to perform a global search of the solution space in order to avoid local minima. In addition, the provided solution should be of high accuracy in order to satisfy the “equal to zero” condition required by system (2). To achieve the aforementioned goals we adopted - after testing several alternative approaches - a hybridoptimization approach, where a Genetic Algorithm is employed in order to perform a global search of the solution space and provide an initial solution “close to a global optimal”. Then, the Conjugate Gradient method, provided with analytic gradient, is commenced in order to “fine-tune” the existing solution and conclude to a global optimal of high accuracy. GAs are population-based heuristics that have been found to perform better than the gradient-based methods, especially when addressing the problem of optimizing multi-modal, non-differentiable, or discontinuous functions35. The parameters of the adopted GA are analyzed and described in the following subsections. This strategy proved to be robust since all our test cases showed that with this approach one is able to process both accurate and inaccurate sketches which include small computational errors in their junctions’ coordinates (“noisy junctions”).
5.2. Formulation of the optimization problem We have studied several approaches for solving system (2) with the constraints of Theorem 2 and we concluded to the following nonlinear optimization problem with nonlinear constrain: r r min F ( x ); x = ( b0 , a0 , c0 ,..., bi , ai , ci ,..., bR −1 , aR −1 , cR −1 ) r x∈ 3 R (3) subject to ai b j ≠ a j bi (4) kr ai − mr bi ≠ 0
constraints in the optimization problem. The minimum of objective function F is zero, and, every solution set that satisfies this condition is a solution to system (2). By incorporating the constraint C1 in the denominator we make sure that the solution set which minimizes F is also a solution to the CS problem, since, by definition, the Fr equation of system (2) is satisfied when the corresponding sketch line ei intersects the associated cross-section lines Lfi and Lfj. After extensive experiments, we found out that use of objective function (5) with the constraints (4) enables a fast convergence of the GA. However, when the CG is commenced we only use the objective function (5). In order to enable a fast and accurate minimization of r F ( x ) with the CG method (during the second part of the optimization approach) we have calculated the gradient r r vector g = ∇F using the partial derivatives of the objective function as follows: r ∂F ∂F ∂F , , ,... g = ..., ∂bn ∂an ∂cn L −1 ∂ Fr 2 L −1 ∂ Fr 2 L −1 ∂ Fr 2 (6) = ..., ∑ 2 , ∑ 2 , ∑ 2 ,... b C a C c C ∂ ∂ ∂ r =0 n r r =0 n r r =0 n r with i = 0,..., R − 1 . The factor Fr 2 Cr 2 includes two triplets
of
unknowns
( bi , ai , ci )
and
(b , a , c ) j
j
j
,
with
i, j ∈ [ 0,..., R − 1] . The corresponding partial derivatives are
given as follows: ∂ ∂bi ∂ ∂ai
2 Fr 2 2 ( c j mr − nr a j ) Fr Cr + 2a j Fr 2= 3 Cr Cr 2 Fr 2 2 ( b j nr − kr c j ) Fr Cr − 2b j Fr 2= 3 Cr Cr
∂ ∂ci
Fr 2 2 ( a j kr − mr b j ) Fr 2= Cr 2 Cr
∂ ∂b j
Fr 2 2(ai nr − mr ci ) Fr Cr − 2ai Fr 2 2= Cr 3 Cr
∂ ∂a j
Fr 2 2(ci kr − nr bi ) Fr Cr + 2bi Fr 2 2= Cr 3 Cr
∂ ∂c j
Fr 2 2(bi mr − kr ai ) Fr 2= Cr 2 Cr
(7)
(8)
5.3 Design and parameters set up of the GA
3R
Here, F: R →R, is the objective function given as
F 2 L−1 ( ci a j − c j ai ) kr + ( c j bi − b j ci ) mr + ( ai b j − a j bi ) nr F = ∑ r2 = ∑ 2 r = 0 Cr r =0 (a b − a b ) L −1
i
j
2
(5)
j i
The subscripts i, j ∈[0,..., R − 1] correspond to the regions Ri and Rj that are adjacent to sketch line eij. Each function Fr ,
r = 0,..., L − 1 , is the left-hand side of the corresponding equation in system (2) and
Cr corresponds to the constraint
C1 of Theorem 2, i.e., Cr = ai b j − a j bi . Objective function (5) incorporates constraint C1 of Theorem 2, while both C1 and C2 are included as separate
Four major decisions should be taken in order to effectively adopt and setup a Genetic Algorithm35: the type of the GA, the data structure representation, the genetic operations and the fitness function. ● Type of GA: We have adopted a GA with multiple populations. Each population evolves using a steady-state genetic algorithm, but in each generation the best individuals of each population migrate from one population to its neighbour. We found that this kind of GA provides, for the present problem, an effective global search of the solution space. Specific values: in all our experiments we used 12-30 parallel populations and each population
3D Res. 04, 01(2013)3
7
consists of 150 individuals. At every generation the best 12 individuals migrate from one population to its neighbour population. ● Data structure representation (chromosomes representation): According to Michalewicz36, floating-point (FP) representation is more effective than binary representation when the GAs are used to solve highprecision numerical problems. Some of the features that make FP representation attractive for use with these problems are the following: (a) FP representation is faster and more consistent from run to run. (b) It provides a higher precision. (c) The performance of a real-coded GA is much faster compared to a binary-to-decimal coded GA. Specific values: in all our experiments we used 24 bits floating-point accuracy, since no higher accuracy is required in the first phase of the proposed optimization strategy. ● Genetic operators: We have adopted uniform crossover, a Gaussian mutation operator and a Roulette Wheel selection. Specific values: in all our experiments we used crossover = 70%, mutation =0.75% - 0.95%. The fitness function that corresponds to the minimization problem (3) subject to the constraints (4) is given as:
1 , if no constraints are violated F = F 0, otherwise 젨
(9)
( a b − a b )2 > 0 i j j i constraints = 2 ( kr ai − mr bi ) > 0 In order to satisfy both constraints of Theorem 2 we are “punishing” those individuals that violate one or both of the constraints by assigning them a zero fitness score. In this way, we make sure that “bad” individuals do not take part in the reproduction phase. This tactic enables a fast convergence of the GA. Finally, although, GAs are known for their ability to avoid local minima we strengthen the proposed algorithm by incorporating the following two mechanisms: (a) The GA is restarted if no significant progress has occurred during the first 5000 populations (with this we avoid cases with “bad” initial individuals). (b) We restart the GA if no convergence occurs for three repetitive times. In this way, we are able to obtain a cross-section that corresponds to a global optimum of the aforementioned optimization problem.
5.4 Terminating conditions of the hybridoptimization method As it is explained earlier the proposed optimization methodology consists of two phases. At the first phase, the aforementioned GA is applied to provide a near global optimal solution. We derive a near global optimal since we use a 24 bits floating-point representation (8 significant digits), which allows for a fast execution of the GA. Then, we use this solution as an initial guess for the CG algorithm. Three issues are stressed out in this section: (a) the termination criteria of the GA, (b) termination criteria of the CG, and (c) the numerical accuracy of the calculations. For
the termination criteria of the GA, one has many options, such as: (i) “Terminate upon generation”: if the number of generations exceeds the maximum number of generations the algorithm is terminated. (ii) “Terminate upon convergence”: if after a maximum allowed number of generations there is no improvement of the score of the best individual the algorithm is terminated. (iii) “Terminate upon population convergence”: if the population average converges to the score of the best individual in the population then the algorithm is terminated. In the present implementation, we have adopted the second termination criterion. In fact, if after a number nConv = 1000 of generations all new solutions are not improved compared to the last best one, then the GA is terminated. This criterion implies that the resulting solution will correspond to the best individual of all parallel populations. For the termination of the CG algorithm we use the norm of the gradient vector and a maximum number of iterations. If the gradient vector’s norm is below a predefined threshold or if the number of iterations is more than 30R the algorithm is terminated. The iterations threshold is defined to avoid infinite loops of the optimization algorithm in the case where no solution is found that makes the gradient vector’s norm equal to zero. Since we are dealing with a numerical analysis procedure we have to assign appropriate tolerances to indicate when a numerical value is “zero”. In this work, we have used two tolerance variables GRDTOL and ERRTOL . The variable GRDTOL is used to indicate when the norm of the gradient vector is vanished, while variable ERRTOL is used as a threshold to indicate when a value x should be considered as zero (i.e., x is “zero” iff x < ERRTOL ).
5.5 Evaluating the resulting cross-section The minimization of the problem defined by equations (3) and (4) provides the values of each unknown triplet (bi , ai , ci ) , i = 0,..., R − 1 , which correspond to a crosssection on the plane z = 0 of the given wireframe sketch. The next step in the proposed optimization methodology is to test whether the generated cross-section is compatible with the sketch. The cross-section is compatible with the sketch, if, for r = 0,..., L − 1 , the calculated values satisfy the following criteria: ci a j − c j ai c j bi − b j ci (10) kr + mr + nr < ERRTOL ai b j − a j bi ai b j − a j bi (11)
ai b j − a j bi > ERRTOL
kr ai − mr bi > ERRTOL ∧ r g < GRDTOL
kr a j − mr bj > ERRTOL
(12) (13)
Criteria (10)-(12) explicitly evaluate the accuracy of the solution of the Cross-Section Problem (CSP), while ineq. (13) ensures that the CG algorithm has converged according to the gradient vector’s norm criterion described in the previous section. If all the aforementioned criteria (10)-(13) are satisfied, the given sketch is considered as realizable, since there is a compatible cross-section corresponding to it.
8
3D Res. 04, 01(2013)3 Table 1: Indicative results and parameters’ values of the genetic algorithm.
Sketch
Lines
1 2 3 4 5 6 7 8 9
21 24 30 33 36 36 54 53 107
Number of Populations 12 12 16 16 16 20 30 30 30
Population Size 150 150 150 150 150 150 150 150 150
Mutation Rate 0.0075 0.0075 0.0085 0.0085 0.0085 0.0095 0.0095 0.0095 0.0095
Generations
Fitness Value
Time (sec)
74600 34647 150933 196069 119756 176400 243012 354369 51189
1.4436 3.1614 4.8659 0.7053 8.8772 0.0581 1.9931 0.0092 0.0131
57.158 25.537 156.672 206.842 135.144 292.46 571.09 1108.85 279.41
Table 2: Compatibility evaluation results with inaccurate sketches.
Sketch error
E
F max r Cr
Is the generated Cross-Section compatible?
C1 Ineq.(11)
C2 Ineq.(12)
Gradient Norm Ineq.(13)
7 *10−9 1.68*10−1
√
2*10−6 5.9*10−6
√
4.5*10−6 9.9*10−2
√
5.3*10−6 1.3*10−4
√
3.2*10−7 3.3*10−5
√
√
Ineq.(10) 1
O(10-6) O(10-1)
1.71*10−7 4.45*10−2
√
√
x
√
O(10-6) O(10-1)
1.37 *10−6 2.09*10−2
√
√
x
√
O(10-6) O(10-1)
1.3*10−6 1.4*10−2
√
√
x
√
O(10-6) O(10-1)
3.2*10−6 1.4*10−2
√
√
√
√
O(10-6) O(10-1)
8.6*10−7 1.9*10−2
√
√
√
√
O(10-6) O(10-1)
1.5*10−6 5.1*10−2
√
√
√
√
1.7*10−6 2.4*10−3
O(10-6) O(10-1)
3.59*10−6
√
√
4.8*10−6
√
1.31
x
√
67.06
x
O(10-6) O(10-1)
1.01*10−6 6.87*10−1
√
√
√
x
√
5.2*10−6 1.6*10−1
O(10-6) O(10-1)
1.08*10−6 1.3*10−2
√
√
√
x
√
6.4*10−6 9.9*10−1
x
2 x
3 x
4 x
5 x
6 x
7
8 x
9
6. Results and Discussion The proposed hybrid-optimization approach is tested using a number of accurate as well as inaccurate sketches in order to evaluate its stability and effectiveness. In fact we perform three sets of experiments using: (a) accurate sketches, (b) inaccurate sketches with an acceptable error level, and (c) inaccurate and not acceptable sketches. In all cases, we investigated whether the proposed method is able to conclude to correct results with respect to the realizability of the corresponding sketch. In other words, the proposed method determines whether a given sketch depicts a valid 3D model or if it includes geometric errors that do not allow
x
for a direct definition of a 3D shape. The realizability evaluation of each sketch depends on the error tolerance variables GRDTOL and ERRTOL that can be adjusted according to the accepted accuracy level for each sketch.
6.1 Cross-sections of accurate sketches With respect to accurate sketches, a set of indicative results and GA’s parameter values are listed in Table 1. All crosssections have been generated by the new hybrid method and are evaluated as “compatible” complying with all the criteria (10)-(13), with GRDTOL = 1e-9 and ERRTOL = 1e-6.
3D Res. 04, 01(2013)3
9
After several trials we have established some basic rules for adjusting the parameters of the GA. These rules depend on the number of unknown variables (or cross-section lines) in the optimization problem. In particular, fixing the “population size” at 150 individuals, the “number of populations” is ranged from 12, like e.g., in Sketch 1, to 30 for complicated sketches, like e.g., Sketch 9. In addition, we slightly increase the mutation rate in cross-sections with
many unknown lines in order to have a more effective search of the solution space. In practice, we haven’t noticed any improvement in the GA behavior with population size over 150 individuals and with a mutation rate more than 0.95%. However, according to our experiments, the number of parallel populations plays an important role in establishing a global optimum.
Figure 6: The GA convergence data for two test sketches.
sketch 1
sketch 2
sketch 3
sketch 4
sketch 5
sketch 6
sketch 7
sketch 8
sketch 9
Figure 7: Cross-sections produced by the proposed hybrid-optimization approach.
Although, due to the probabilistic nature of GAs, it is not possible to derive safe conclusions in terms of the number of required generations and the corresponding execution time, we found out that usually the higher the number of the unknown cross-section lines, the higher the execution time. In some cases, however, when the number of parallel populations is increased, we have noticed a faster convergence speed (see, for example, the results of Sketch
9). All given times in Table 1, have been measured in an Intel Core i7 2.67 GHz PC. The two graphs in Figure 6, present the score of the fitness function (9) in terms of the “number of generations” for the sketches 4 and 5, respectively. Similar, “good convergence behavior” has been denoted in the rest of the test sketches. In all tested cases, the CG algorithm concludes to a
10
3D Res. 04, 01(2013)3
highly accurate global optimal by minimizing the objective function (5) in milliseconds. In fact, for each sketch in Table 1, the CG converged with an accuracy level of O(10 ) generating the final compatible cross-sections that are shown in Figure 7. −9
6.2 Cross-sections of inaccurate sketches An analysis of inaccurate sketches is performed in terms of a given error E associated with the coordinates of the sketch’s junctions; i.e., the “noisy” junction coordinates are in the form (x + E, y + E). Preserving the GA parameters of the corresponding accurate sketches, the performance of the hybrid-optimization method is explored on the basis of different accuracy levels for each sketch. In particular, two cases are considered: In the first case, all test sketches include, in their coordinates, a small perturbation error O(10-6). This kind of sketches is considered as “acceptable” since the error conveyed is quite small compared to the bounding box of the sketches (square of unit size). The second case, considers sketches that their coordinates are associated with a higher and not-acceptable error level O(10-1). In either case we validate the capacity of the proposed hybrid-optimization methodology to correctly evaluate the realizability of the corresponding sketches. In order to be able to assert correctly these two sets of inaccurate sketches we set GRDTOL = 1e-5 and ERRTOL = 1e-5. In other words, we set the required accuracy equal to the maximum acceptable error level. If one desires to allow sketches with higher inaccuracies then the ERRTOL variable must be adjusted to cover such a case. For each one of the sketches of Table 1 there are two alternative inaccurate sketches as explained above. All these sketches are processed by the proposed algorithm, and the results are reported in Table 2. In fact, Table 2 presents the evaluation calculations associated to the four criteria (10)-(13). A tick “√” is assigned when the cross-section corresponding to an inaccurate sketch complies with the associated criterion in each column (a mark “x” is assigned otherwise). In all test cases, the proposed hybrid-optimization method is able to assert correctly the sketches, with both acceptable and inacceptable error level, allowing a robust evaluation of their realizability.
function given in eq.(5) has proved to be robust in terms of the consistency noted in the obtained results. In addition, since the overall optimization problem (see eq.(3)-(4)) is a multi-modal nonlinear problem with nonlinear constraints, we selected a hybrid-optimization approach in order to ensure that the final solution will correspond to a feasible cross-section. In this way, we are able to exploit the benefits of GA for getting a near-optimal solution and the CG for getting a final solution of high accuracy. In the near future we aim to employ this method to a “sketch-to-solid” algorithm and in particular to exploit its robustness by applying it to intermediate wireframe sketches (i.e., wireframe sketches where the conveyed hidden geometry is unknown) for the geometric definition of the hidden junctions. In parallel, we will explore the effectiveness of the hybrid-optimization approach for the algorithmic calculation of the 3D geometry of the depicted solid model.
Acknowledgment The research of N. Sapidis and Ph. Azariadis has been (partially) co-financed by the European Union (European Social Fund - ESF) and Greek national funds through the Operational Program “Education and Lifelong Learning” of the National Strategic Reference Framework (NSRF) Research Funding Program: THALIS. Finally, the authors sincerely thank the two anonymous referees of the original submission for their many and very valuable comments/questions that led to significant improvement of this paper.
References 1. 2.
3. 4.
7. Conclusions and Future Applications In this work, we have adopted the cross-section realizability criterion of Ros and Thomas32 in order to evaluate if a given wireframe sketch is the true projection of a 3D solid model. An algebraic system is derived which is solved using a hybrid-optimization approach consisting of two phases. In the first phase, a GA is employed to provide an initial global solution which is improved in the second phase through a CG method. The result of the proposed approach is an accurate cross-section of the given sketch that it is tested for compatibility using the four criteria depicted in Section 5.5. As with all optimization methods, a key issue in the proposed approach is the selection of the objective function and the type of numerical method for minimizing it. After considering several functions, the selected objective
5.
6. 7.
8. 9.
S. Kyratzi, N. S. Sapidis (2009b) From sketch to solid: An algebraic cross-section criterion for the realizability of a wireframe sketch, Computing, 86(2-3): 219-234. S. Kyratzi, P. Azariadis, N.S. Sapidis (2011) Realizability of a Sketch: An Algorithmic Implementation of the CrossSection Criterion, Computer Aided Design and Applications, 8(5): 665-679. P. A. C. Varley (2002) Automatic Creation of BoundaryRepresentation Models from Single Line Drawings, PhD. Thesis, University of Wales. P. Company, A. Piquer, M. Contero, F.Naya (2005) A survey on geometrical reconstruction as a core technology to sketchbased modelling, Computer & Graphics, 29(6): 892-904. M. A. Fahiem, S. A. Haq, F. Saleemi (2007) A Review of 3D Reconstruction Techniques from 2D Orthographic Line Drawings, GMAI’07: Proceedings of Geometric Modeling and Imaging: 60-66. L. Olsen, F. F. Samavati, M.C. Sousa, J. A. Jorge (2009) Sketch-base modelling: A survey, Computers & Graphics, 33(1): 85-103. H. Lipson, M. Shpitalni (2002) Correlation-Based Reconstruction of a 3D Object from a Single Freehand Sketch, AAAI Spring Symposium on Sketch Understanding: 99–104. K. Sugihara (1986) Machine Interpretation of Line Drawings, The MIT Press. W. Whiteley (1991) Weavings, Sections and Projections of Spherical Polyhedra, Discrete Applied Mathematics, 32(3): 275-294.
3D Res. 04, 01(2013)3 10. S. Kyratzi, N. S. Sapidis (2009) Extracting a polyhedron from a single view sketch: Topological construction of a wireframe sketch with minimal hidden elements, Computers and Graphics, 33(3): 270-279. 11. S. Kyratzi, N. S. Sapidis (2011) 3D Object Modeling using Sketches, Information Resources Management Journal, 24(4): 27-49. 12. P. Company, M. Contero, P. Varley, N. Aleixos, F. Naya (2009) Computer-aided sketching as a tool to promote innovation in the new product development process, Computers in Industry, 60(8): 592-603. 13. D. A. Huffman (1971) Impossible Objects as Nonsense Sentences, Machine Intelligence, 6(1): 295-323. 14. M. B. Clowes (1971) On Seeing Things, Artificial Intelligence, 2(1): 79-116. 15. Y. Ding, T. Y. Young (1998) Complete Shape from Imperfect Contour: A Rule-Based Approach, Computer Vision and Image Understanding, 70(2): 197-211. 16. J. Malik (1987) Interpreting Line Drawings of Curved Objects, International Journal of Computer Vision, 1(1): 73103. 17. D. L. Waltz (1972) Generating Semantic Descriptions from Drawings of Scenes with Shadows, Technical Report AI-TR271, MIT. 18. P. A. C. Varley, R. R. Martin, H. Suzuki (2005) Frontal Geometry from Sketches of Engineering Objects: Is Line Labelling Necessary, Computer-Aided Design, 37(12): 12851307. 19. I. J. Grimstead, R. R. Martin (1995) Creating Solid Models from Single 2D Sketches, Proceedings of the third ACM Symposium on Solid Modeling and Applications: 323-337. 20. H. Lipson (1998) Computer Aided 3D Sketching for Conceptual Design, PhD Thesis, The Israel Institute of Technology. 21. L. Cao, J. Liu, X. Tang (2008) What the Back of the Object Looks Like: 3D Reconstruction from Line Drawings without Hidden Lines, IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(3): 507-517. 22. M. Wuersch, M.J. Egenhofer (2008) Perceptual Sketch Interpretation, Headway in spatial data handling: 13th International Symposium on Spatial Data Handling: 19-38.
11 23. H. Lipson, M. Shpitalni (1996) Optimization-Based Reconstruction of a 3D Object from a Single Freehand Line Drawing, Computer-Aided Design, 28(8): 651-663. 24. C. Tian, M. Marsy, H. Lipson (2009) Physical sketching: Reconstruction and analysis of 3D objects from freehand sketches, Computer-Aided Design, 41(3): 147-158. 25. P. Company, M. Contero, J. Conesa, A. Piquer (2004) An Optimisation-Based Reconstruction Engine for 3D Modelling by Sketching, Computers & Graphics, 28(6): 955979. 26. Y. Sun, Y. T. Lee, J. Sun (2008) Regularity selection for effective 3D object reconstruction from a single line drawing, Pattern Recognition Letters, 29(10): 1486-1495. 27. D. A. Huffman (1977) Realizable Configurations of Lines in Pictures of Polyhedra, Machine Intelligence, 8: 493-509. 28. P. A. C. Varley, R. R. Martin, H. Suzuki (2004) Making the Most of using Depth Reasoning to Label Line Drawings of Engineering Objects, Proceedings of the Ninth ACM Symposium on Solid Modeling and Applications: 191-202. 29. R. Martin, P. Varley, H. Suzuki (2005) Perpendicularity as a Key to Interpreting Line Drawings of Engineering Objects, Proceedings on Digital Engineering Workshop: 5th JapanKorea CAD/CAM Workshop: 115-120. 30. K. Sugihara (1999) Resolvable Representation of Polyhedra, Discrete and Computational Geometry, 21(2): 243-255. 31. J. Stolfi (1987) Oriented Projective Geometry, ACM Press New York, USA. 32. L. Ros, F. Thomas (2005) Geometric Methods for Shape Recovery from Line Drawings of Polyhedra, Journal of Mathematical Imaging and Vision, 22(1): 5-18. 33. W. Whiteley (1979) Realizability of polyhedral, Structural Topology, 1: 46-58. 34. L. Ros, F. Thomas (1998) Analysing Spatial Realizability of Line Drawings through Edge-Concurrence Tests, Proceedings of IEEE International Conference on Robotics and Automation: 3559-3566. 35. D. E. Goldberg (1989) Genetic Algorithms in Search, Optimization and Machine Learning, Addison Wesley Publishing Company. 36. Z. Michalewicz (1992) Genetic Algorithms + Data Structures = Evolution Programs, Springer-Verlag Publications.
