Off-line Identification With Handwritten Signature Images: Survey and ...

2 downloads 6855 Views 2MB Size Report
Production Automatisee, 4750 Henri-Julien, Montreal QC, Canada H2T 2C8 ... extraction of a novel representation of handwritten signature images.
Off-line Identification With Handwritten Signature Images: Survey and Perspectives Robert Sabourin 1 , Rejean Plamondon 2 , and Guy Lorette3 1

2

3

Laboratoire Scribens, Ecole de Technologie Superieure, Departement de Genie de la Production Automatisee, 4750 Henri-Julien, Montreal QC, Canada H2T 2C8 Laboratoire Scribens, Ecole Poly technique de Montreal, Departement de Genie Electrique, C.P. 6079, Succ. "A," Montreal QC, Canada H3C 3A7 IRISA, Universite de Rennes 1, Campus de Beaulieu, 35042 Rennes Cedex, France

The first part of this paper presents a survey of the literature on automatic handwritten signature verification systems using binary or graylevel images, and focuses primarily on preprocessing techniques, feature or primitive extraction methods, comparison processes, and performance evaluation. With these previous studies in mind, we propose, in the second part of this paper, an image-understanding system based on the extraction of a novel representation of handwritten signature images. This approach is text insensitive. A structural match between a reference primitive set Pr and a test primitive set Pt takes into account the geometric shape and spatial relations between primitives. Finally, the local comparison of gray levels between pairs of primitives next to each node of the static solution path N results in a pseudo dynamic similarity measure iJd(Pr, Pt). This scheme allows the elimination, with a certain degree of success, of skilled forgeries such as tracings and photocopies, showing marked gray-level dissimilarity along the signature line.

1 Introduction Off-line signature verification problems are generally considered to be more difficult than on-line ones [7]. A signature image can easily be copied optically or mechanically. Moreover, timing and dynamic information are highly degraded in a static specimen. Although part of this information can be recovered by expert document analysts using specific skills and techniques [23, 24, 25], most of their methods cannot be implemented easily in a computer environment. A few papers proposing ways of dealing with this problem have been reported in the literature. Sakai et al. have proposed an interactive hybrid system to collect and display the static and dynamic features of a test and a reference signature on a CRT screen, using a color code [3]. This system was used to help a clerk examine specific features of a signature, and the final decision about the specimen's authenticity was left to him. Ammar et al. have tried recently to extract indirectly and use pressure information from a signature image [1, 2]. Apart from this recent paper, most of the work done in this area thus far has been concerned with static feature analysis only. H. S. Baird et al. (eds.), Structured Document Image Analysis © Springer-Verlag Berlin Heidelberg 1992

220

R. Sabourin et al.

The design of off-line signature verification systems requires the solution of five types of problems: data acquisition, preprocessing, feature or primitive extraction, the comparison process, and performance evaluation.

2 Data Acquisition Different types of acquisition devices are used for off-line signature verification systems, including TV vidicon cameras, CCD matrix cameras, and electrooptical scanners. The size of the image may be up to 512 x 512 pixels in area and with up to 256 gray levels of quantization. Off-line data entry systems may also be simulated by ordering (X,Y) coordinate pairs issued from a digitizing tablet as with a raster scanning.

3 Preprocessing This step deals with the preparation of the relevant information: nonuniformity correction for sensor elements, localization of the signature in the picture, extraction of the signature from the background, slicing, solving thresholding and filtering problems, segmentation, and data reduction. Since most of the applications covered by these studies are related to the automatic validation and processing of specific paper forms (such as checks), the finding of a signature in this context is not a difficult problem and may be solved with the help of a window operator [4]. However, as suggested by Nagel [4, 5, 6], this problem might become more serious if the form standard permits a large number of variations. If no artifice is used (e.g. signing with special ink), the problem of extracting a signature image from its background is not a trivial one. Thresholding and slicing techniques have been proposed [1, 6, 8] but are generally insufficient in practice in a noisy background for which restoration techniques are also necessary [1]. Indeed, the efficiency of slicing and thresholding techniques is directly related to the selection of the cut point from a gray-level histogram of the picture. The problem is then reduced to finding a point on the frequency distribution that separates the peaks of a bimodal curve, where the peak representing the signature is almost lost in noise. The use of the Laplacian or the Sobel operator has been proposed as a border or stroke detector. In the first case, the threshold is fixed to select pixels above the 85% valve in the cumulative Laplacian distribution [5, 6]. In the second, the modulus gradient vector of the gray-level signature is used, with the threshold fixed at the point where the modulus variations in the pixel distribution function become less than 1% [8]. This latter approach offers the advantage of being independent of the thickness of the strokes and of the overall length of the signature. However, for practical cases where specimens are noisy, these techniques have to be improved. A four-step preprocessing operation has recently been proposed by Ammar et al. [1] (background equalization and reduction, noise reduction by averaging, automatic thresholding and image extraction). This approach has

Off-line Identification of Handwritten Signatures

221

been found to be successful in removing the overlapping in a signature as well as in the signing line, at the cost, however, of eliminating some parts of the signature in about 2% of samples. Depending on the features needed at the next stage, the preprocessing algorithms can perform more specific operations on the extracted image. Prescreening operators are used to eliminate points that are judged to contain little information on writer identity [14, 15, 16]. Window operators have been used by Brocklehurst to improve the image [17]. A tracking algorithm has been applied by Nagel and Rosenfeld to identify and label all connected components in the picture [6]. Based on the hypothesis that high-pressure regions are indicated by higher gray levels in an image, a high pressure threshold selection algorithm has been proposed by Ammar et al. [1] to extract the high-pressure regions of a signature. This mechanism is based on the evaluation of a pressure factor, i.e. the ratio of the high-pressure area to the binary signature area. Recently, Sabourin and Plamondon [10, 11] have proposed a new version of a centroidal linkage region-growing-with-merging algorithm to perform signature segmentation. Using the statistics of directional data, atomic regions characterized by local uniformity in the orientation of the gradient are extracted. This scheme allows the extraction of signature areas characterized by gray levels with a very low signal/noise ratio.

4 Feature or Primitive Extraction The division of feature or primitive extraction methods into those involving textsensitive and those involving text-insensitive features, as has been proposed for document expertise [18], cannot be strictly applied to signatures since the number of characters or symbols in a specimen is limited and often irrelevant (in an alphabetical context). Similarly, since the time information is not available from the image data, the function approach to feature or primitive extraction is not a really an appropriate distinction for static techniques [7]. Attempts at feature or primitive selection are better categorized by their global or local approaches to the problem, although an efficient system will probably incorporate both global and local feature or primitive extraction [13]. Moreover, none of the feature or primitive sets selected thus far preserves enough information to allow a reconstruction of the handwritten specimens. In the global approach, the signature image is taken and processed as a whole; features or primitives are deduced from the global aspect of a signature. In other cases, features are evaluated from local parameter values which are averaged over the signature. In both cases, structural aspects encountered in signatures are taken into account in the local and global descriptions. Nemcek and Lin [16] have used the Hadamard transform on binary images as a means for data reduction and feature selection, the resulting spectrum constituting the feature set. A Sobel operator has been applied by Sabourin and Plamondon [8] to graylevel handwritten signature images, and the angular behavior of the resulting intensity gradient (assumed to reflect the stroke orientation) has been used as

222

R. Sabourin et al.

a feature set. A close-fitting polygon has also been computed to evaluate the overall shape of a signature and to serve, once normalized, as basic features [15]. Sabourin and Plamondon have proposed a feasibility study on the use of some graphometric techniques, and, in particular, the rhythmic line of the signature has been used [9]. Chuang [21] has divided the signature into upper, middle, and lower zones, initial and end strokes (a process familiar to graphoanalysts), and performed a sequential analysis of each sub-image to extract features related to each resulting zone and its proportionalities. A single study has been reported to date describing the extraction of some global dynamic features from a static image. Indeed, Ammar et at. [1, 2] have worked with a set of features mainly related to pressure measurements, as extracted from gray-level images: the vertical position of the baseline of the high-pressure and of the binary image, a pressure factor and threshold, the highest gray level in the extracted signature image, the dynamic range on the gray scale for a signature, and its area measured as the number of pixels. In the second approach, a set of local attributes is computed. Many of these features are known to be stable, according to document examiner expertise [23, 24, 25]. However, most of the studies dealing with local features also incorporate some global ones. For example, Brocklehurst [17] has proposed the use of the overall length of the signature (global) with a set of local features likes slope measurements, the distance from the left-hand end of the designated space provided for a signature to the point at which the signature begins, concavity measures, etc. Nagel and Rosenfeld [4, 5,6] have also worked with characteristics extracted from signatures segmented into vertical and horizontal zones. Assuming that a signature spelling was known, these authors detected tall letters in a specimen. Two global features (the ratio of signature width to short-stroke height and to long-stroke height) and two types of local features (the ratio of the height of a long stroke to the height of the short strokes immediately preceding it and the slope features of appropriate long letters) were retained for their studies, the number of these local features being dependent on the number of tall letters in a signature. Most of the work done so far in the area of off-line signature verification has been concerned mainly with the statistical approach. Few studies deal with structural approaches using primitives. In his work, Requier [30] has used the linear strokes of signatures as primitives in a simulated off-line verification system. Nouboud et at. [19, 20] used as primitives the segments of the polygonal envelope of a signature. In a recent paper, Ammar et at. [22] described signature features and relations among them using a character string (the Global Description of a Signature (GDS)) and a hierarchical tree structure as a local description.

5 Comparison Processes and Performance Evaluation As for the on-line system [7], the comparison process is based on the assumption that feature sets or structural descriptions extracted from genuine signatures

Off-line Identification of Handwritten Signatures

223

are more stable than those of forgeries: that is, that intrapersonal variations are smaller than interpersonal variations. So, an unknown signature may be recognized or rejected according to its similarity to the reference signature representation first specified by an indirect identifier. To evaluate this similarity, three approaches have been tried, two involving statistical or data analysis and one involving structural analysis. Considering signature verification as a non-standard two-class problem (since the class of forgeries cannot be specified), a first group of researchers [1, 6, 21] has used weighted distances. These methods compute a distance measured between the reference set and the test set. The decision is taken according to the value of this distance criterion with respect to a decision threshold, this threshold being pre-established with the help of a training set of genuine signatures. Assuming some a priori probability [16], or by limiting their studies to random forgeries [8], some authors have used maximum likelihood classifiers and linear discriminant functions. This second approach also assumes statistical interdependence of gaussian distributed features. The third approach is typically a structural one. Requier [30] has proposed a method using graph and subgraph isomorphism for signature verification. Nouboud et al. [19, 20] have used dynamic programming to compare segments related to the polygonal envelope of a signature. Table 1 summarizes the results obtained thus far by the research groups involved in off-line signature verification. Since the performance evaluation method differs greatly from one approach to another, these experimental results are again difficult to compare. The table shows the type of input data used in each author's study, a description of the data base used for the training or test experiments or both, the number and type of features with the comparison method, the error rates as reported according to type I (cd and type II (c2) definitions. The last column reports a few comments that might be helpful in analyzing the results. Generally speaking, it has been observed that type I and type II errors of the order of a few percent can be obtained at the present time for systems working with simple and random forgeries. Only one system has been tested with skilled forgeries, and these specimens were not reliably detected [17]. According to expert document analysts, skilled forgery detection will not be efficient with these systems as long as dynamic and static features are not exhaustively extracted from a written specimen [23, 24, 25]. This is corroborated by observations from human perception experiments that knowledge of the drawing methods can greatly influence the recognizability of distorted characters [26]. Some authors have commented on their system failures [16, 17], two major causes of which are reported: first, scanning problems or signal quantization errors, resulting in incomplete specimen acquisition, due to the thinness of the trace or the color of the ink used [16]; second, excessive natural variations in the signatures of a few users [17].

6 ANew Approach Incorporating some knowledge from expert document analysts [23, 24, 25] seems to us the best way of obtaining a significant improvement in the performance of

R. Sabourin et al.

224

Table 1. Performance of static signature verification systems. AUTHORS

INPUT IMAGE RESOLUTION FEATURE DESCRIPTION

=6A, FUKUMURA (1986) [I] BROCKLEHURST

256 x 1024 grid 256 gra~ levels 7 g loba pressure features

TRAINING AND/OR TEST DATA BASE (Specimens x Writers)

m~e~uIBee)

200 forgeries (20 S x 10 imitators)

60 pixe ls/cm binary 7 local cMracterist ics

2820 genu i nes !60 S x 47 W) equa 1 number of unseen forgeries)

100 x 300 grid binary g loba 1 features

2400 genuines (6 S x 400 W) 1600 forgeries (4 S/user)

NAGEL, ROSENFELD (1977) [6]

500 pixels/inch 64 gray leve Is 2 t~pes of loca 1 g 10 a 1 features

NEMCECK, LIN (1974)

128 x 256 grid binary 14 features from Hademar spectra

(1985~

17]

CHUANG (1977) [21]

[16]

COMPARISON METHOD

~!¥~~~~

ERROR RATES

f1 • 6" f2 • 4"

COMMENTS

~~:e~~~~e!~~¥e~;~d

with thresho ld for maximizing performances

f1 • 5" f2 • 5"

6 S/W as reference acceptation training thresho ld at 95" true acceptat i on

weighted distance

f1 • 20X f2 • 20X

3 S/W as refarence freehand forgeries

~e~uln~s+ 5S x lW) 14 forgeries (9 5 x I W + 55 x lW)

~!¥~~~

f1 f2 f1 f2

600 ~enu i nes (40 x 15 W) 120 forgeries 110 users imitated Sluser, 4 imitators

maximum likelihood classifier

f 1 - 11" f2 - 41"

1000 genuines

f1 • 2" f2 • 8"

n

t~~:~~Uo~i~~~i~~n25" genuine acceptat ion

• 8" • OX - 12" - OX

freehand for~eries average resu t of single and double jacknife method 30 S/W training 10 S/W test freehand forgeries training threshold at 96" genu ine acceptat ion

with persona 1 threshold

NOUBOUD (1989~ 20]

384 x 510 64 gray levels sig. envelope

~10 S x 100 W)

dynamic ~'i~gram-

PHELPS (1982)

reso lut ion not sracified c ose-fitting polygon

3 genuines ~3 S x 1 W) forgeries (3 S x 2 W)

similarity of overlapping area

incomp lete resu Its

128 x 512 grid 256 ~ray leve Is angu ar histogram image gradient

760 ~enu ines (40 x 19 W)

maximum f1 • 1.5" likelihood f2 - 7" and linear discrim.

random forgeries on ly 15 S/W for tra ining 25 S/W for test

800 ~enu i nes (40 x 20 W)

nearest f1 -0.57" neighbour f2 -0.03" claSSifier

5 ref ./signer

63 genuines from 1 W 117 simulated forgeries from 6 amateur forgers

nearest f1 • 3.5" neighbour f2 • 0" classifier

features ext racted interactively at a graphic terminal

[15] SABOURIN PLAMONDON (1986) [8] SABOURIN PLAMONDON (1987) [9]

rhythmic 1ine

S/W for reference

6 ref./signer

17 sk i lled for~eries from 1 professIonal forger SABOURIN PLAMONDON (1990~ 14]

128 x 512 grid 256 gray-levels set of primitives

pen II x B W) tracing forgeries Sluser, pen II! Sluser, pen 12

224 FB 60 110 10 24 (28

~enuines,

224 (28 1.60 10 10 24 (28

~enuines,

!

~hotocoPies

luser)

pen II x 8 W) traCing forgeries S/user. pen II! 51 user , pen '2 ~hotocoPies

luser)

f1 - 1.8" f2 -47.1"

structura 1 f1 -0.0" matching f2 =1.34" (static)

f2 - 100X f2 ·98.9" f2 - 96"

structura 1 f1 '8.93" matching f2 -I. 79% (stat ic + f2 -6.25" pseudof2 -78.8" dynamic) f2 =61.2"

6 ref ./signer 3 S/W as reference random forgeries B - .0 comparison threshold se lected from the random forgeries experiment, tracing forgeries from 1 writer 3 S/W as reference randomBf~r~~Oies compar i son thresho ld se lected from the random forgeries experiment, tracing forgeries from 1 writer

Off-line Identification of Handwritten Signatures

225

Automatic Handwritten Signature Verification Systems (AHSVS) [13]. As stated in [12, 13], Locard [25] has proposed many characteristics belonging to genuine handwritten signatures which are imperceptible to the average forger and very difficult to imitate. These characteristics are related to local variation in aspect ratio, orientation, relative position, shading, etc., between pairs of characters. This scheme seems to be more powerful than the individual letter's shape or general design which is very easily perceived by the average forger and therefore easier to imitate [23, 24, 25]. In the light of these previous studies [1,6,8,9,15,16,17,20,21]' we propose an image-understanding system [10, 11, 12, 13, 14, 32] based on the extraction of a novel signature representation from gray-level images [10, 11] for automatic signature verification. By an image-understanding system we mean the separation of the local and global evaluation functions [28] for the analysis of the handwritten signature and the detection of skilled forgeries. The approach considered in the present work [10, 11, 12, 13, 14] is text insensitive because no attempt is made to segment specific letters from the semantic part of the signature. Two classes of features, static and pseudo-dynamic, are taken into account for the representation of the scene. The former includes the geometric shape and spatial relations (N,S,E,O,N-E,N_O,S-E,S_O, and ADJ_TO) between some primitives extracted from the signature line [12]. The latter is associated with the gray-level variation inside the primitive [14]. By considering simultaneously these two classes of features, and the separation from the local interpretation of the primitives followed by the global interpretation of the scene, we have designed a general-purpose image-understanding system for the interpretation of handwritten signature images. This system is view-point dependent [27] and tailor-made for this purpose. The spatial sampling of handwritten signatures produces an image format of 128 x 512 pixels. The preprocessing stage of the AHSVS (see Figure 1) is responsible for producing the signature image where each pixel from the graylevel image has a label Ls(m, n) signifying that the pixel at location (m, n) is either a background or a signal picture element. The preprocessing phase is subdivided into two stages: the gradient computation and the background elimination processes [10, 11]. The gradient at location (m, n) is evaluated over the entire gray-level image with the Sobel operator. The resulting filtered image is then analyzed by the background elimination process. Because the handwritten signature is characterized by a high gradient activity, the density function F(IV'I) is computed and a threshold T is automatically settled. Finally, those pixels at location (m, n) where the gradient activity is high, that is to say 1V'I(m, n) 2: T, are associated with a label Ls(m, n) = 1, and Ls(m, n) = 0 otherwise. The signature image is thereafter analyzed by the primitive extraction process that is responsible for the production of the primitive sets necessary for the structural analysis of the handwritten signature by the comparison process. The strategy adopted here takes into account the collinearity of neighboring signalpixels (labeled as Ls(m, n) = 1) in the directional plane 8(m, n) of the gradient space. This task is performed in two steps [10]. First, the signal-pixels are merged into atomic regions characterized by the

226

R. Sabourin et al.

....

........

10 NUMBER

........

....

10 NUMBER

/'

--/

/

",I

/.

~\ENROLMEI'IT \ PROCESS / '-

.,.

/'

J REFERENCE SET

Fig. 1. The data-flow diagram of the proposed AHSVS using gray-level images.

uniformity of a local property, i.e. the orientation 8( m, n) of their gradient vectors. The resulting atomic regions belong to the signature line or represent spurious noise elsewhere in the scene. The initials, "ar", depicted in Figures 2(a) and 2(b) (reference and test images), will serve as a case study to illustrate the various stages of the proposed AHSVS. The enlargement of the letter "r" (Figure 2( a)) depicted in Figure 2( c) shows the atomic region partitions in the gradient space. The second step of the primitive extraction process is responsible for generating the primitive set [11]. The High-Level-Merging process (HLM) starts with the elimination of sparse atomic regions. The HLM process continues with a hierarchical merging strategy in growing collinear atomic regions. The growth is therefore limited by the resulting circular variance R. The latter sub-process is used repeatedly in varying a growing constraint Rm , considering a fixed directional constraint related to the local organization in the directional data located at the common border of neighboring atomic regions. This scheme acts as a zoom procedure focusing the attention of the HLM process on the biggest homogeneous regions from the atomic region set. The resulting primitive set is therefore characterized by a collection of arbitrarily shaped primitives. The degree of merging in the HLM process depends on the enrollment flag status (Figure 1). When this status has a TRUE value, the resulting set is a reference primitive set Pr where the growth is limited only by the presence of directional discontinuities along the handwritten signature line. In the case of a test primitive set Pt,

em

Off-line Identification of Handwritten Signatures

227

b)

a)

QJ\. d)

c)

e)

IIIrIIIIICI PBIIIl'l'IVI SEl' Pr

Fig. 2. Reference (a) and test (b) images. The atomic region set is shown in the gradient space (C) with the enlargement of the letter "r" from (a), followed by the corresponding reference primitives (d). Finally, the representation of the reference (e) and the test (f) primitive sets Pr and Pt.

the HLM process limits the growth depending on a circular constraint Rm (for example, the reference primitives Pri E Pr depicted in Figure 2(d), resulting from the application of the HLM process to the atomic regions of Figure 2(c)). The final results obtained from the extraction of the reference primitive set Pr from the reference image (Figure 2(a)) and the test primitive set Pt from the test image (Figure 2(b)), are shown in Figures 2(e) and 2(f) with cardinalities of N r = 6 and Nt = 11 primitives. Examples of reference primitive sets Pr taken from handwritten signature images are also depicted in Figure 3. The comparison process is therefore responsible for the structural match between a reference primitive set Pr and a test primitive set Pt, resulting in a global similarity measure {}(Pr, Pt). The first stage of the comparison process, the Local Interpretation of Primitives (LIP), is related to the final merging of test primitives Pt v E Pt. The LIP process is responsible for the labeling of all test primitives Pt v E Pt, given a reference primitive set Pr. The A* algorithm previously proposed [29] is replaced by a partially informed best-first BF* strategy governed by a new evaluation function f(n) = f(g(n), h(n)), where heuristics are embedded in f(n) [12]. The

R. Sabourin et al.

228 \

a)

b)

Fig. 3. Examples of reference primitive sets Pr for two writers enrolled in the proposed AHSVS data base [13]. evaluation function f(n) is now defined by the relation

* (1 - SHAPE(Pt v:n , Pri))] + [W2 * AREA(Ptv:n , Pri)] +[W3 * DIST(Ptv:n , Pri)] + [W4 * A_R(Ptv:n , Pri)],

f(n) = [WI

(1)

where Lk=I Wk = 1, and 0 ~ f(n) ~ l. Each term used in the definition of the evaluation function f(n) is related in some way to graphometric features used by the expert document analysts [13]. The first term SHAPE(Ptv:n,Pri) gives the similarity of shape between the collection of test primitives Pt v:n E Pt and the reference primitive Pri E Pro The shape factor is related to the measure of planar shapes using binary shape matrices obtained from the polar sampling of the silhouette of primitives Pt v:n and Pri. The next term AREA(Pt v:n , Pr;) is related to the difference in area between the test primitive subset Pt v:n centered around the test primitive Pt v and tentatively interpreted as reference primitive Pri' The relative difference in area is justified by the fact that all test and reference images for each writer are taken under similar experimental conditions. The AREA(Pt v:n , Pri) feature should permit the rapid elimination of primitives with the same shape but with

Off-line Identification of Handwritten Signatures

229

great dissimilarity in proportion. In the same way, DIST(Pt v :n , Pri) enables discrimination between two primitives on the basis of their local position. Two primitives similar in shape and equally prominent must be discriminated finally by their local positions. For example, if two lines with the same orientation have to be matched, DIST(Pt v :n , Pri) acts as a local constraint in the intelligent local template matching, limiting the merge of test primitives on the basis of their respective X positions from the start of the handwritten signature. This is also carried out by the term A_R(Pt v :n , Pri) related to the relative differencein-aspect ratio between Pt v :n and Pri. The local confidence rating between a test primitive subset Pt v :'! E Pt, tentatively labeled as primitive Pri E Pr, is given by

(2) where 0 :::; Fi(V) :::; l. The local confidence rating Fi(V) takes a value near unity for a perfect match between the subset of test primitives Pt v :'! E Pt in the neighborhood of test primitive Pt v , and the reference primitive Pri E Pr. The local interpretation of Pt v as Pri terminates with the set of primitives

Pt v :'! = Pt v , ... , Pt'! E Pt

(3)

The second stage of the comparison process, the Global Interpretation of the Scene (GIS), completes the analysis with the evaluation of a solution path N, produced by the structural matching of the primitive sets Pr and Pt, and followed by the validation of the resulting solution with the comparison of corresponding pseudo-dynamic features taken along the solution path N. The first stage of the GIS process involves the evaluation of a static similarity fJs(Pr, Pt) between the test primitive set Pt and a reference primitive set Pr. The state space graph G' = (V', E') is defined as an oriented graph where nodes (states) are grouped in k ~ 2 disjoint subsets Vi,l :::; i :::; k (phases). So, the set of nodes V' may be defined as the union of all subsets, as V' = VI U V2 U ... U Vi u ... Vk. As previously stated [12, 14], a greedy algorithm seems a sufficient search mechanism for the global interpretation of handwritten signature images. This scheme is computationally less costly than the dynamic programming used in [29]. The static cost (similarity) at node v E Vi may be defined as

(4) The factor Ri-I,i (u, v) represents the similarity in spatial relations between pairs of reference primitives Pri-I and Pri taken from the reference handwritten signature image represented by the primitive set Pr, and the spatial relations between pairs of corresponding test primitives Pt u and Pt v from the test primitive set Pt. Finally, node v E Vi, where the accumulated cost COSTi(V) is maximum, is pushed on list Ni = V. Assuming the use of a greedy algorithm at the GIS stage, the cost COSTi(V) is therefore defined by the expression [12, 14]:

(5)

R. Sabourin et al.

230

where u

=

Ni-

1, V

E

Vi, < u,v

>E

EI,

3::;

i::;

k -1, and

(6)

Ni = argmax COSTi(V) vEV,

Note also that

0::; Ri - 1,i(U, v) ::; 1 (the similarity in spatial relations) 0::; }Ci(V) ::; 1 (the local confidence rating)

0::;

(the normalization factor)

(7) (8) (9)

Moreover, the initial conditions are as follows:

COST1(1) N1 COST2(V) N2

=0 =

(10) (11) (12) (13)

S

= st2 * }C2(V), 'Iv E V2 = argmax COST2(v) VE V 2

The final step is therefore:

Nk = t COSTk(t) = COSTk-1(Nk-d

(14)

(15)

The normalization factor defined by sti = AREA(Pr;)/AREA(Pr) takes into account the ratio, in area, of the reference primitives related to each phase (Pri == Vi) E VI to the entire area of the reference primitive set Pro This permits an additional penalty for primitives not yet included in the definition of VI. Finally, sti emphasizes prominent reference primitives in set VI in the evaluation of a global static similarity measure iJs(Pr, Pt), defined as follows:

(16) where of course 0 ::; iJs(Pr, Pt) ::; 1. A pseudo-dynamic similarity measure iJd(Pr, Pt) is therefore computed at the end of the GIS process with the use of the resulting solution path N = S, Vb V2,"', Vk-1, t. A pseudo-dynamic feature is evaluated from the local analysis of the gray levels inside pairs of primitives Vi = Pri, Pt v :"{ located on the solution path N (Figure 4). The pseudo-dynamic feature adopted here is the proportion of "black-labeled pixels" obtained from a discriminant analysis of gray-level histograms computed for each of the primitives Pri E Pr and Pt v :"{ E Pt, where Pri, Pt v :"{ E N [14]. This feature is called pseudo dynamic because it reflects the effect of the writing process dynamics, which produces gray-level variations along the signature line. The pseudo-dynamic similarity measure iJd(Pr, Pt) is defined as:

k-1

iJd(Pr, Pt) =

(st 2 * }C2( v) * Dg (v)) + Z)sti * Si-1,i( u, v) * D? (v)) i=3

(17)

231

Off-line Identification of Handwritten Signatures

RIlIRINCE

TlST

~~

~~

~~



~~

~A

Q~



~~

@1~

~~

~~

Fig. 4. Primitives pairs located on static solution path N and considered in the evaluation of the pseudo-dynamic similarity measure iJd(Pr, Pt).

where u = N i - l E Vi-I. o~

Fi(V)

~1

V

= Ni E Vi, INI = k, Nl = S, Nk = t, and (the local confidence rating)

0::; Si-l,i(U,V) ::; 1 (the static similarity measure) 0::;

Di(V)

O~

(J

O~

ili

(the pseudo - dynamic similarity measure) ~5 (the scaling factor) ::;1 (the normalization factor) ~1

(18) (19) (20) (21) (22)

The pseudo-dynamic factor D i ( v) acts as an additional penalty when the cost of the static solution path N is revalued. Define the pseudo-dynamic features

DYN(i) DYN(v)

= wi(ki) = wv(k;)

(23) (24)

where wi(ki) (wv(k;)) represents the proportion of black pixels defined at graylevel threshold k; (k;), which is evaluated automatically with a discriminant

232

R. Sabourin et al.

analysis of the gray level density functions proposed by Otsu in [31], and computed from pixels belonging to the reference primitive Pri (Ptv:'Y)' Di (v) is defined as the relative similarity in the proportion of black pixels between pseudo-dynamic features DYN(i) and DYN(v) belonging to the primitive pair Pri, Ptv:'Y = Vi E N, i.e. D

_ _ IDYN(i) - DYN(v)1 ,(v) -1 MAX(DYN(i),DYN(v))

(25)

with of course 0 :S Di (v) :S 1. The resulting pseudo-dynamic cost Di (v) is used in equation (17) for the evaluation of the final similarity measure 'l3 d (Pr, Pt) between the test and reference primitive sets Pt and Pr. An experiment was conducted [14] with 224 genuine specimens from eight writers, 224 random forgeries, 160 tracing forgeries from one forger using two kind of tracing pens and 224 photocopies of the genuine specimens. Let WI be the class of genuine handwritten signatures and W2 be defined as the class of forgeries. A test image from class WI or W2 is compared to three reference signature images for a specific writer enrolled in the AHSVS, and identified by a specific ID number (Figure 1). The best similarity 'l3 d (Pr, Pt) obtained from the comparison process is therefore dispatched to the decision process. The probability functions of resulting maximum similarity measures 'l3 d (Pr, Pt) are therefore computed considering an equal a priori probability for each class WI and W2, say P[WI] = P[W2] = 0.5. A decision threshold T ra is settled with the help of random forgeries at the value corresponding to the total minimum error rate Etmin [14]. In the static case ({3 = 0 in equation (17)), experimental results with random forgeries show a type I error rate of 101 = 0.0%, a type II error rate of 102 = 1.34%, resulting in a total minimum error rate of Etmin = 0.67% for the proposed AHSVS. Considering the same decision threshold T ra , tracing forgeries and photocopies were not eliminated (see Table 1). Using the proposed pseudo dynamic scheme with {3 = 5 in equation (17), a new decision threshold T ra is computed with the random forgeries. Using this threshold (Table 1), tracing forgeries (pen number 1) showing a thicker signature line than the genuines can be eliminated with a type II error rate of 102 = 6.25%, at the expense of a higher type I error rate of 101 = 8.93%. The corresponding type II error rate for the random forgeries experiment is now equal to 102 = 1.79%.

7 Conclusion A new scheme is advocated for the design and analysis of automatic handwritten signature verification systems using gray-level images [32]. The use of a novel handwritten signature representation allows local analysis of gray levels along the signature line, enabling the elimination of skilled forgeries such as tracings and the photocopies showing great gray level dissimilarity along the signature line. The concept of combining the structural matching of gray-level images with the separately considered local interpretation of primitives and the global

Off-line Identification of Handwritten Signatures

233

interpretation of the scene is not new in the field of computer vision [27, 29], but major improvements to the method have been presented [10, 11, 12, 13, 14, 32].

Acknowledgements This work was supported by grant A0915 to Rejean Plamondon from NSERC CANADA, and by FIR grant to Robert Sabourin from Ecole de Technologie Superieure. The preparation of this paper was completed when Rejean Plamondon was a Fellow of the Netherlands Institute for Advanced Studies.

References [1]

M. Ammar, Y. Yoshida, and T. Fukumura, "A New Effective Approach For OffLine Verification of Signatures by Using Pressure Features," Proc. 8th ICPR, Paris, France, pp. 566-569, October 1986. [2] M. Ammar, Y. Yoshida, and I. Fukumura, "Feature Extraction and Selection for Simulated Signature Verification," Proc. 3rd Int'l Symp. on Handwriting and Computer Applications, Montreal, pp. 167-169, 1987. [3] T. Sakai, T. Kanade, and Y. Ariki, "Multi-Feature Display of On-line Signatures by Color TV Display," Proc. 2nd Int'l Joint ConI on Pattern Recognition, pp. 303-304, Copenhagen, 1974. [4] R. N. Nagel and A. Rosenfeld, "Steps Toward Handwritten Signature Verification," Proc. 1st Int'l Joint ConI Pattern Recognition, pp. 59-66, 1973. [5] R. N. Nagel, "Computer Detection of Freehand Forgeries," Ph.D. thesis, Univ. Maryland, 1976. [6] R. N. Nagel and A. Rosenfeld, "Computer Detection of Freehand Forgeries," IEEE Trans. on Computers, vol. C-26, no. 9, pp. 895-905, September 1977. [7] R. Plamondon and G. Lorette, "Automatic Signature Verification and Writer Identification, The State of The Art," Pattern Recognition, Vol. 22, No.2, pp. 107-131, 1989. [8] R. Sabourin and R. Plamondon, "Preprocessing of Handwritten Signatures From Image Gradient Analysis," Proc. 8th ICPR, Paris, France, pp. 576-579, October 1986. [9] R. Sabourin and R. Plamondon, "On the Implementation of Some Graphometric Techniques for Interactive Signature Verification: A Feasibility Study," Proc. The Third Int'l Symp. on Handwriting and Computer Applications, Montreal, Canada, pp. 160-162, 1987. [10] R. Sabourin and R. Plamondon, "Segmentation of Handwritten Signature Images Using The Statistics of Directional Data," Proc. 9th ICPR, Rome, Italy, pp. 282285, November 1988. [11] R. Sabourin and R. Plamondon, "Segmentation of Handwritten Signature Images: A Structural Approach," Tech. Report TR89a, Ecole de Technologie Superieure, Montreal QC, Canada, 1989. [12] R. Sabourin and R. Plamondon, "Structural Interpretation of Handwritten Signature Images," Tech. Report TR90a, Ecole de Technologie Superieure, Montreal QC, Canada, 1990.

234

R. Sabourin et al.

[13] R. Sabourin and R. Plamondon, "Progress in the Field of Automatic Handwritten Signature Verification Systems Using Gray-level Images," Proc. International Workshop on Frontiers in Handwriting Recognition, Montreal, April 1990. [14] R. Sabourin and R. Plamondon, "Steps Toward Efficient Processing of Handwritten Signature Images," Vision Interface 90, Halifax, Nova Scotia, May 1990. [15] R. I. Phelps, "A Holistic Approach to Signature Verification," Proc. ICPR, vol. 2, p. 1187, 1982. [16] W. F. Nemcek and W. C. Lin, "Experimental Investigation of Automatic Signature Verification," IEEE Trans. on SMC, pp. 121~126, January 1974. [17] E. R. Brocklehurst, "Computer Methods of Signature Verification," NPL Report DITC 41/84, pp. 1~12, May 1984. Also in: J. of the Forensic Science Society, Vol. 25, pp. 445~457, 1985. [18] V. Klement, R. D. Naske, and K. Steinke, "The Application of Image Processing and Pattern Recognition Techniques to the Forensic Analysis of Handwriting," Int'l Con/. on Security Through Science and Engineering, Berlin, pp. 5~11, September 1980. [19] F. Nouboud, "Contribution it l'etude et it la Mise Au Point d'un Systeme d'Authentification de signatures Manuscrites," These de Doctorat, Universite de Caen, France, 1988. [20] F. Nouboud, F. Cuozzo, R. Collot, and M. Achemlal, "Authentification de Signatures Manuscrites par Programation Dynamique," Proc. Congres PIX 1M, Paris, pp. 345~960, 1988. [21] P. C. Chuang, "Machine Verification of Handwritten Signature Images," Proc. Int'l Con/. on Crime Countermeasures, Sciences and Eng., pp. 105~1O9, 1977. [22] M. Ammar, Y. Yoshida, and I. Fukumura, "Description of Signature Images and Its Application to their Classification," Proc. 9th ICPR, Rome, Italy, pp. 23~26, 1988. [23] W. R. Harrison, Suspect Documents, Their Scientific Examination, Chicago, Nelson-Hall, 1981. [24] O. Hilton, Scientific Examination of Questioned Documents, Elsevier, NY, revised edition, 424 Pages, 1982. [25] E. Locard, Les Faux en Ecriture et leur Expertise, Payot, Paris, 1959. [26] J. J. Freyd, "Representing The Dynamics of a Static Form," Memory Cognition 11, pp342~346, 1983. [27] T. O. Binford, "Survey of Model-Based Image Analysis Systems," Int'l J. of Robotics Research, Vol. 1, No.1, Spring 1982. [28] M. A. Fischler and R. A. Elschlasser, "The Representation and Matching of Pictorial Structures," IEEE Trans on Computers, C-22, No.1., pp. 67~92, 1973. [29] M. D. Levine, "A Knowledge-Based Computer Vision System," In E. M. Riseman and A. R. Hanson (eds.), Computer Vision Systems, Academic Press, New York, 1978. [30] J. P. Requier and G. Lorette, "Reconnaissance de Signatures Manuscrites," Actes du 2e Congres A FCET-IRIA , Reconnaissance des Formes et Intelligence Artificielle, Tome III, pp. 298~307, 1979. [31] N. Otsu, "A Threshold Selection Method from Gray-Level Histograms," IEEE Trans. on SMC, Vol. SMC-9, No.1, pp. 62~66, 1979. [32] R. Sabourin, "Une Approche de Type Comprehension de Scene Appliquee au Probleme de la Verification Automatique de I'Identite par l'Image de la Signature Manuscrite," Ph.D. Thesis, Ecole Poly technique de Montreal, 1990.