Test Sequencing Problems Arising In Test ... - Semantic Scholar

5 downloads 314 Views 419KB Size Report
Index Terms—AND/OR graph search, heuristics, minimal stor- age testing, minimax .... Thus, the HEF for minimax optimization does not depend on ..... She is working on the simulation engine of Simulink, an ... He has served as a Consultant.
IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART A: SYSTEMS AND HUMANS, VOL. 29, NO. 2, MARCH 1999

153

Test Sequencing Problems Arising in Test Planning and Design for Testability Vijaya Raghavan, Member, IEEE, Mojdeh Shakeri, Member, IEEE, and Krishna R. Pattipati, Fellow, IEEE

Abstract— In this paper, we consider four test sequencing problems that frequently arise in Test Planning and Design For Testability (DFT) process. Specifically, we consider the following problems: 1) How to determine a test sequence that does not depend on the failure probability distribution? 2) How to determine a test sequence that minimizes expected testing cost while not exceeding a given testing time? 3) How to determine a test sequence that does not utilize more than a given number of tests, while minimizing the average ambiguity group size? 4) How to determine a test sequence that minimizes the storage cost of tests in the diagnostic strategy? We present various solution approaches to solve the above problems and illustrate the usefulness of the proposed algorithms. Index Terms—AND/OR graph search, heuristics, minimal storage testing, minimax test sequencing, multi-objective optimization, test planning.

I. PRELIMINARIES

B

EFORE we consider the test sequencing problems arising in Test Planning and Design for Testability (DFT) process, we discuss the formulation of the basic test sequencing problem and the associated top-down algorithms based on AND/OR graph search. Once the basic notation and algorithms are explained, we go on to present the variations needed on these algorithms to solve the proposed problems. The test sequencing problem, in its simplest form, consists of: system states 1) a set of associated with the system, where denotes the faultdenotes one free state of the system and potential faulty states in the system; of the 2) the prior conditional probabilities of the system states where is the conditional probability that no fault exists in the system denotes the probability that and has occurred; with an 3) a set of available tests where application cost vector denotes the usage cost of test , measured in terms of time, manpower requirements, or other economic factors;

4) a diagnostic dictionary matrix , where is 1 if test detects a failure state , and 0 otherwise. The problem is to design a test algorithm that is able to unambiguously identify the occurrence of any system state in using the tests in the test set , and that minimizes the expected testing cost, , given by (1) is an by binary matrix such that where is if test is used in the path leading to the identification of system state , and is zero otherwise. This problem belongs to the class of binary identification problems that arise in medical diagnosis, nuclear power plant control, pattern recognition, and computerized banking. The optimal algorithms for this problem are based on dynamic programming (DP) and AND/OR graph search procedures [11]. The DP technique is a recursive algorithm that constructs the optimal decision tree from the leaves up by identifying successively larger subtrees until the optimal tree rooted at the initial node of complete ambiguity is generated. Suppose we are given an ambiguity subset, i.e., a suspected . Upon applying a test , the set of failure states ambiguity set can be reduced based on the outcome of test and the diagnostic dictionary matrix . If the test fails, would consist of then our reduced ambiguity set those failure sources from which can be detected by , i.e., . Similarly, if the test passes, would consist of then our reduced ambiguity subset which cannot be detected by , i.e., those failures from . Let denote the cost of the optimal decision strategy starting from an ambiguity set . , We can now write the DP recursion relating and as follows: (2) where the conditional probabilities of the ambiguity subsets and are given by (3)

Manuscript received September 9, 1996; revised November 1, 1998. V. Raghavan and M. Shakeri are with Mathworks, Inc., Natick, MA 017601500 USA (e-mail: [email protected]). K. R. Pattipati is with the Department of Electrical and Systems Engineering, University of Connecticut, Storrs, CT 06269-3157 USA (e-mail: [email protected]). Publisher Item Identifier S 1083-4427(99)01450-2.

(4) and

1083–4427/99$10.00  1999 IEEE

(5)

154

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART A: SYSTEMS AND HUMANS, VOL. 29, NO. 2, MARCH 1999

The DP recursion is initiated with the known terminal con. The DP technique has storage and ditions for the basic test secomputational requirements of quencing problem [10]. Efficient top-down algorithms based on AND/OR graph search were developed in [11] to contain the computational is a directed graph explosion of DP. An AND/OR graph with a root (or initial) node , and a nonempty set of . The initial node terminal leaf (or goal) nodes, represents the given problem to be solved, while the terminal leaf nodes correspond to sub-problems with known solutions. The intermediate nodes of are of three types: OR, AND, and intermediate leaf. An OR node is solved if any one of its successor nodes is solved, but an AND node is solved only when all of its immediate successor nodes are solved. An intermediate leaf node has no successors and is unsolvable. The AND/OR graph associated with the test sequencing problem has the following properties: 1) the initial node of complete ignorance represents the original test sequencing problem to be solved; 2) the intermediate nodes of residual ambiguity correspond to test sequencing subproblems that must be solved in order to obtain a solution to the original problem; 3) the goal nodes of zero ambiguity represent primitive sub-problems with known solution (that is, system state identified) and zero cost; 4) if the solution tree contains an AND node, all its successors (representing the resulting ambiguity subsets) are also in the solution tree; 5) if an OR node is in the solution tree, then only one successor of the node is in the solution tree and represents the optimal test at that node. Since the generation of an optimal test algorithm is an NPcomplete problem [11], it is necessary to explore heuristic approaches for guiding the AND/OR graph search. These heuristic approaches use problem domain knowledge, in the form of an heuristic evaluation function (HEF), to avoid enumerating the entire set of potential solution trees. The HEF of the optimal is an easily computable heuristic estimate from any node of ambiguity set to the goal cost-to-go, nodes of zero ambiguity. Various HEF’s based on Huffman and feasible codes were derived for the basic test sequencing problem in [11], [13]. II. MINIMAX TEST SEQUENCING A common criterion that is minimized in most test sequencing problems is the expected cost of diagnosis. Minimization of the expected cost can sometimes result in inordinately expensive sequences of tests to isolate faults of very low probability of occurrence. This may not be acceptable since the estimates of the mean time to failure (MTTF’s) of components are often inaccurate. Typically, the theoretical estimates of MTTF’s may be off by as much as a factor of 10 from the actual MTTF’s under field conditions [1], [2]. In these cases, the dependence of the cost function on the underlying probability distribution can result in diagnostic strategies that are not truly optimal. For this problem, we consider the

so-called Minimax (minimizing the maximum testing cost) criterion to construct robust diagnostic strategies. Formally, the problem is to devise a sequential testing strategy that minimizes the maximum testing cost (i.e., diagnostic cost) defined by (6) is an by binary matrix such that where is if test is used in the path leading to the identification of system state , and is zero otherwise. Suppose we are given an ambiguity subset, i.e., a suspected . Upon applying a test , the set of failure states ambiguity set can be reduced based on the outcome of test and the diagnostic dictionary matrix . If the test fails, would consist of then our reduced ambiguity set those failure sources from which can be detected by , i.e., . Similarly, if the test passes, would consist of then our reduced ambiguity subset which cannot be detected by , i.e., those failures from . Let denote the cost of the optimal decision strategy starting from an ambiguity set . , We can now write the DP recursion relating and as follows: (7) at the solution The recursion is initialized with nodes (i.e., leaf nodes of no ambiguity). Note that the above DP recursion is very similar to that obtained for the basic test sequencing problem. Thus, we can use AO to solve this problem, provided an admissible and consistent HEF can be found to approximate the optimal cost-to-go. A. Minimax Coding Problem tests are available and the test costs are identical, If all then this test sequencing problem is identical to the Minimax coding problem; that is, the problem of generating a prefixfree binary code of a set of binary messages for transmission over a noiseless channel. The problem is to find the minimal for a set of binary maximum length of code . Note that there is no messages, dependence on the prior probabilities of messages. The analogy between the Minimax test sequencing and the noiseless Minimax coding is as follows: the system states correspond to the binary messages, the sequence of test results are similar to the message code word, and the maximum number of tests required to isolate a failure correspond to the length of the longest code word. The only differences are that the generation of a test algorithm is constrained by the availability of tests, whereas no such constraint exists for the coding problem, and the tests may have unequal costs in the test-sequencing problem. Using the following lemmas, we can construct such a code for a given value of . , then there is no prefixLemma 1: If free code with maximum code word length less than or equal . to

RAGHAVAN et al.: TEST SEQUENCING PROBLEMS

155

Proof: If the maximum code word length is less than or , then the maximum number of such distinct equal to . Hence, such a code binary code words is cannot exist for this message set. Lemma 2: A prefix code exists with a maximum code word length of . be Proof: Let pseudomessages an expanded message set with . This message set can be encoded by a length binary code. By dropping the code words for the ), we then have a code pseudomessages (in case for where all code words have length . From the above lemmas, it is seen that the minimal maxiis mum code word length for a message of cardinality where denotes the smallest integer given by greater than or equal to .

Since the test costs are in ascending order, we have (13a) where (13b) That is, the cost of the optimal tree must be greater than the smallest test costs. Note that cost of a tree that uses the is monotone increasing, convex function of . Now

(14) B. HEF for Minimax Test Sequencing We derive the appropriate HEF for this problem by appealing to the analogy between the minimax test sequencing problem and the minimax coding problem discussed earlier. for Let us denote the minimal maximum code word length any node of ambiguity subset as follows: (8) is the cardinality of the ambiguity set . where for any node of The minimal maximum code word length ambiguity subset provides a lower bound on the maximum length of any test algorithm rooted at (including the optimal test algorithm with maximum length). Formally (9) if test is used by a test algorithm rooted where at to identify the system state and is zero otherwise. The above property of Minimax code can be used to derive an admissible HEF as shown in the following theorem. Theorem 1: Assume, without loss of generality, that the test . Then costs are in ascending order is given by a lower bound (10) Proof: The cost of the optimal tree rooted at

is given by (11)

if test is used in the subtree of the optimal where be the algorithm rooted at and is zero otherwise. Let in the optimal test algorithm length of the code word for rooted at . That is (12)

is the maximum length of the optimal test where algorithm rooted at . From the monotonicity of and the property of Minimax code in (9), we have (15) completing the proof of the theorem. Thus, the HEF for minimax optimization does not depend on the probability distribution of failure sources in the ambiguity group . It has been shown in [12] that the application AO in conjunction with an admissible HEF is sufficient to yield optimal solutions. We now show that the above HEF has the additional property of consistency, thus ensuring that the AO algorithm with this HEF monotonically converges to an is consistent, if for each node optimal solution. An HEF in an AND/OR graph (16) First, we consider the case where the test costs are identical to derive a key result which will then be used to prove the general case where test costs are not identical. Consider the and . Let the two minimax trees corresponding to lengths of the Minimax code-words for each of these trees and , respectively. One can construct be denoted by a prefix-free code (although not necessarily optimal) for the subset by connecting the two minimax trees by a binary link. Since the length of each code-word is increased by 1 for , the maximum code-word length of each , is the tree denoted by (17) . Without loss of Let us now denote . Consider generality, let us assume that the following two cases. • Case 1: Note that, if is an admissible test for the ambiguity and set (i.e., splits into two nonempty subsets ), then it will not be an admissible test for and , since and .

156

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART A: SYSTEMS AND HUMANS, VOL. 29, NO. 2, MARCH 1999

Hence, and are computed using sorted test costs which do not include corresponding to test . If , then

TABLE I PERFORMANCE OF HUFFMAN CODE HEF VERSUS MINIMAX HEF

FOR

m = 10

TABLE II PERFORMANCE OF HUFFMAN CODE HEF VERSUS MINIMAX HEF

FOR

m = 15

TABLE III PERFORMANCE OF HUFFMAN CODE HEF VERSUS MINIMAX HEF

FOR

m = 15

(18) Hence

(19) • Case 2: Since is computed using the first tests sorted in the ascending order of costs. Hence

(20) Thus, we see that (21) proving (16). C. Simulation Results In order to demonstrate the robustness of the diagnostic strategy computed via Minimax approach, we generated several random diagnostic dictionary matrices of varying sizes number of faults, number of tests) and computed ( the diagnostic strategies due to Minimax HEF and Huffman code based HEF1 described in [11] (using a randomly generated prior probability distribution for failure sources). We then computed the following parameters for these strategies averaged over a set of 1000 randomly generated probability distributions. Projected expected cost of AO diagnostic strategy based on the initial probability distribution. Projected maximum testing cost of AO diagnostic strategy based on the initial probability distribution. Mean expected testing cost of AO diagnostic strategy averaged over 1000 random fault probability distributions. 1 Assume, without loss of generality, that the test costs are in ascending order 0 c1 c2 cn . The Huffman code based lower bound (HEF) for the optimal cost-to-go h(x) at an ambiguity node x is given by





 111  w (x)

h (x ) = j =1

cj + [ w 3 (x )

0

w 0 (x)]c

w (x)+1

(22)

where w 3 (x) is the Huffman code length computed using the normalized conditional probabilities of failure sources belonging to the ambiguity group x and w 0 (x) is the integer part of w 3 (x).

Standard deviation of expected testing cost of AO diagnostic strategy over 1000 random fault probability distributions. Projected expected cost of Minimax diagnostic strategy based on the initial probability distribution. Projected maximum testing cost of Minimax diagnostic strategy based on the initial probability distribution. Mean expected testing cost of Minimax diagnostic strategy averaged over 1000 random fault probability distributions. Standard Deviation of expected testing cost of Minimax diagnostic strategy over 1000 random fault probability distributions. Tables I–IV show the results for various system sizes ( number of faults, number of tests). It is seen that and are less than and for most cases. Also, the projected for Minimax is very close to the actual testing testing cost for most of the cases; thus, with a Minimax strategy, cost we have a reliable estimate of the actual testing cost when the probability distribution is unknown. III. CONSTRAINED OPTIMIZATION OF TEST SEQUENCE Most reasonably complete formulations of real world problems involve multiple, conflicting, and noncommensurate objectives. In the context of test sequencing, optimizing the diagnostic strategy with respect to both expected testing cost with a constraint on the expected testing time is a very important problem that has not been addressed so far. The

RAGHAVAN et al.: TEST SEQUENCING PROBLEMS

TABLE IV PERFORMANCE OF HUFFMAN CODE HEF VERSUS MINIMAX HEF

157

FOR

m = 25

problem of constrained optimization of diagnostic strategy can be formally written as

TABLE V PERFORMANCE OF VARIOUS NODE SELECTION HEURISTICS FOR

m = n = 15

TABLE VI PERFORMANCE OF VARIOUS NODE SELECTION HEURISTICS FOR

m = n = 20

(23) such that (24) is an by binary matrix such that where is if test is used in the path leading to the identification are the test of system state , and is zero otherwise and are the test times. costs and The following argument shows that a variation of AO algorithm can be used to solve this problem optimally. Let be an ambiguity node in the AO graph and be an admissible be the estimate (information giving) test at . Let of cost-to-go (based on test times) at the root OR node before splitting the node using test . When the ambiguity node is split using test , the cost-to-go estimate (based on test times) at can be revised using the HEF values for the successors and of . The admissibility and consistency of HEF ensures that the cost-to-go estimate can only increase. This cost increase is propagated all the way up to the root OR node, and at the root suppose that this entails a cost estimate of , then it clearly indicates OR node. If that the inclusion of test at the ambiguity node results in an infeasible solution. Thus, can be deemed inadmissible at and the search space can thus be reduced to avoid infeasible solutions. Note that more and more infeasible directions can be identified and pruned as the estimates become more accurate as node expansion proceeds. A nice feature of this approach is that the computational requirements of this algorithm can only be less than those of unconstrained AO . Also note that, when all tests are pruned at an ambiguity node (that is not a leaf), this indicates that a feasible diagnostic strategy does not exist for the specified threshold on expected test time. An important implementation issue in AO algorithm variants is that of node selection strategy for expansion. We considered the following node-selection strategies and evaluated their performance (in terms of the number of nodes expanded before the solution is obtained). While traversing the graph from the root OR node in search of an expandable node, if both children of the current node are unsolved. Always pick the left child. Always pick the right child. Pick the child node with minimum test cost based . HEF

Pick the child node with maximum test cost based . HEF Pick the child node with minimum test time based . HEF Pick the child node with maximum test time based . HEF Table V shows the number of node expansions required for these strategies for various values of the test time constraint for a random system having 15 faults and 15 tests. Table VI shows the results for a system with 20 faults and 20 tests. and resulted in We see that the strategies minimal node expansions. This is in contrast to unconstrained is found to be the best node-selection AO where strategy. Fig. 1 shows the set of nondominated solutions2 obtained by invoking constrained AO with various values of the test time system considered above. Note constraint for the that it is possible to obtain solutions that linearly interpolate the given set of nondominated solutions via randomizing the test strategy selection over any two given points. In order to understand this concept, consider any two solution points and that are adjacent to each other in Fig. 1. Suppose, we employ a randomized testing strategy that involves choosing the strategy corresponding to the first solution point with a probability and choosing the strategy corresponding to the second solution point with a probability . The expected testing time and testing cost of this . This strategy are is equivalent to the existence of a solution point that divides the line segment joining the original two solution points in a ratio.

2 In multi-objective optimization, a nondominated solution is one for which no other solution exists which is better in terms of all the objective functions which we are trying to optimize. In this case, since we are trying to optimize expected test-cost and expected test-time, a nondominated solution is a diagnostic strategy for which no other strategy exists which yields both a better expected test-cost and expected test-time.

158

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART A: SYSTEMS AND HUMANS, VOL. 29, NO. 2, MARCH 1999

Fig. 1. Nondominated solution via constrained optimization.

IV. TEST SEQUENCING WITH A CONSTRAINT ON THE NUMBER OF TESTS USED An important problem that arises in the Design For Testability is one of constructing a test sequence that utilizes less than a specified number of tests, while minimizing the average ambiguity group. This is especially true for systems with limited accessibility to test points, and/or very expensive tests. In this case, the system designer is burdened with the task of selecting the best set of tests that can be used to diagnose the system with minimal average ambiguity group size. Formally, let represent the AND/OR graph denoting the be a Boolean variable that diagnostic strategy and let if test represents the presence of test in . That is, is used in the diagnostic strategy , and otherwise. be the set of leaves in (a leaf node Let is one which is not split by a test, i.e., no successors). The problem is to determine a diagnostic strategy minimizing (25) represents the average ambiguity group size for a where . diagnostic strategy resulting in the leaf set The solution approach to this problem consists of two steps. First step involves constructing a complete diagnostic strategy based on a limited lookahead heuristic without explicitly enforcing the constraint on the number of distinct tests used. The second step is to prune the diagnostic strategy, removing tests on an incremental basis, until the constraint on the number of distinct tests used is satisfied. The limited lookahead strategy in Step 1 to select the next best test at every unexpanded

ambiguity node is based on a Figure of Merit (FOM). This FOM represents the effectiveness of a test in producing a good diagnostic strategy in terms of the number of distinct tests used. Clearly, if a diagnostic tree is constructed in such a way that many tests are used more than once in the tree (i.e., in multiple branches), then such a strategy results in a low cardinality of the set of tests used. This enables us to define two different heuristic functions for limited lookahead strategies. Let be the ambiguity node for which the next best test is to and be the child ambiguity nodes of be chosen. Let following Pass and Fail outcomes of test , respectively. The limited lookahead strategy is to pick the test that minimizes the average ambiguity group size at the next step (26) represents the cardinality of the set , and where are the failure probabilities. This is a greedy heuristic that is expected to produce reasonable diagnostic strategies by locally optimizing the next test. Another heuristic is to pick the test that maximizes (27) denotes the set of all admissible tests at where an ambiguity node . This heuristic is intuitively appealing because, by maximizing the cardinality of the set of common tests available to the successors of the ambiguity node , we can expect the same tests to be used in multiple branches of the diagnostic strategy, thus utilizing a smaller number of distinct tests in the diagnostic tree. In addition to the above

RAGHAVAN et al.: TEST SEQUENCING PROBLEMS

TABLE VII PERFORMANCE OF THE TWO FOM’s

FOR

159

VARYING

nmax

heuristics, we can further ensure maximal reuse of tests in parallel branches of the diagnostic strategy by giving higher priority to tests that are already used in parallel branches. Once a diagnostic tree is constructed using all the available tests without enforcing the constraint on the number of tests, as a post-processing step, we prune the tree removing extra tests in Step 2. Once again, we employ greedy heuristics to discard the tests in an incremental fashion. Note that pruning, i.e. discarding a test that splits an ambiguity node that is not a leaf, results in the removal of all tests that follow the Pass and Fail branches of that test. This entails a decrement in the number of distinct tests used, by more than one, and at the same time, it also results in a larger increment of average ambiguity group size for the tree. Hence, we need to weigh these factors appropriately for the pruning choice. split by a test in the Consider an ambiguity node be the number of distinct tests to be diagnostic tree. Let removed from the diagnostic strategy in order to meet the be the number of tests that are discarded constraint. Let is pruned (the tests that follow the successor when test be the set of leaf nodes of the subtree nodes of ). Let rooted at . Pruning results in an increment of the average ambiguity group size given by

TABLE VIII PERFORMANCE OF THE TWO FOM’s

FOR

VARYING

nmax

TABLE IX PERFORMANCE OF THE TWO FOM’s

FOR

VARYING

nmax

TABLE X PERFORMANCE OF THE TWO FOM’s

FOR

VARYING

nmax

(28) V. DIAGNOSTIC STRATEGIES Given this, we can choose the next test to be discarded as one that minimizes the incremental average ambiguity group size per pruned test, i.e., (29) This incremental pruning process is continued until the constraint on the number of distinct tests used is met. Tables VII–X show a comparison of the performance of number the two FOM’s for various random systems ( number of tests, maximum number of faults, Average ambiguity group of tests allowed, Average ambiguity size obtained using FOM1, group size obtained using FOM2). FOM2 is seen to be clearly a better criterion than FOM1, since it resulted in lower ambiguity group sizes almost all the time.

WITH

MINIMAL STORAGE COST

The problem of minimizing the storage (the number of tests in the tree) required for a diagnostic strategy is of considerable interest. In many situations, a decision algorithm residing in the primary memory can perform more efficiently than an algorithm whose components must continually be swapped between the primary and the secondary storage. Pollack [3] and Press [4] have given heuristics for computing storage efficient decision trees for certain types of decision tables (i.e., fault-test point dependency relationships). The General Optimized Testing Algorithm (GOTA) of Hartmann et al. [5] can also be applied to the storage problem. In addition, Reinwald and Soland [6] have presented a branch and bound algorithm for designing decision tables with minimum storage. Recent work by Murphy and McCraw [7] considered this problem and presented a suboptimal heuristic approach to this problem which was shown to be faster and more efficient than the previous approaches, yet achieving reasonably nearoptimal solutions.

160

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART A: SYSTEMS AND HUMANS, VOL. 29, NO. 2, MARCH 1999

In the following, we present a formulation of the generalized minimal storage problem and derive an optimal AO based algorithm. The top-down nature of AO algorithm readily extends to produce a series of near-optimal solutions which provide a nice tradeoff between optimality and computational effort. We present extensive simulation results to demonstrate that our optimal and near-optimal approaches are superior to the heuristic approaches presented in [7]. A. Problem Formulation system states 1) Set of associated with the system, where denotes the faultdenotes one free state of the system and potential faulty states in the system. of the modules 2) Finite set of comprising the system states, where denotes the fault free state of the system. , where 3) Mapping , if corresponds to a of the system. faulty state in module available tests with a 4) Set of where denotes storage cost vector the cost of storing test in a diagnostic strategy. , where is 1 5) Diagnostic dictionary matrix if test detects a failure state and 0 otherwise. The problem is to devise a sequential testing strategy such that the storage cost defined by (30) is the number of times test is used is minimized where in the diagnostic strategy. to represent the ambiguity subset, If we denote and to denote the resultant ambiguity sets based on the outcome (pass or fail) of test , then the DP technique for the test sequencing problem employs the recursion (31) is the optimal cost-to-go (i.e., the storage cost) where for the diagnostic strategy rooted at the ambiguity group . at the solution The recursion is initialized with nodes (i.e., leaf nodes of no ambiguity). Noting that the above DP recursion is very similar to that obtained for the basic test sequencing problem, we can see that AO can be used to solve this problem, provided an admissible and consistent HEF can be found to approximate the optimal cost-to-go. B. NP-Completeness of the Storage Problem A simplified storage minimization problem would be to are identical. In this case, assume that all the storage costs the objective is to seek a strategy with the minimal number of tests. In the following, we show the NP-completeness of the simplified problem and then go on to consider the optimal solution procedure for the general problem.

Hyafil and Rivest [8] have shown that the problem of designing decision trees that minimize the processing time (average and worst case path length) is an NP-complete problem. It is shown in Comer and Sethi [9] that designing storage optimal trees is NP-complete for cases where each module is associated with a single failure source provided that the tests may have ternary outcomes. The result does not hold for the binary case since, trivially, every decision algorithm that identifies each of the failure sources will require tests. It is also known that designing storage optimal full binary decision trees is NP-complete [9]. The Decision Tree Storage Problem for the reachability and the integer [denoted by ] is to matrix with determine whether there exists a decision tree for storage cost less than or equal to . is NP-complete. Theorem 2: is in NP since a nondeterministic Proof: Turing machine can guess the decision tree and then determine in polynomial time if it has storage cost less than or equal to . To complete the proof, it will be shown that the problem Vertex Cover reduces to DTS. (The vertex cover problem asks for the graph if there is a subset of the set of vertices such that every edge in the set is incident and ). For the graph, on at least one element of , where , we construct the testing problem as follows. Define where . (Let us ignore and since it does not will result in any loss of generality.) The set of modules is defined by for have cardinality 2 where and . tests, one for each vertex, where the test There are associated with the vertex is described by the set is incident on . Note that the tests must be used from the remaining failure to partition the failure source and ). Therefore, if has a sources (i.e., partition vertex cover of size , then there is a decision tree for with tests that discriminate from every one of the edges. with Alternatively, if there exists a decision algorithm for tests, then the vertices associated with those tests form a vertex cover for . An obvious corollary of Theorem 2 is that DTS remains NPcomplete if the set of failure sources is partitioned into two modules or if the system has only a single module containing more than one failure source. As noted, the set of instances of DTS where there is a single module associated with each action can be solved in polynomial time. Let denote the set of instances of the problem DTS where there are no more than two objects associated with any action. Theorem 3: is NP-complete. Proof: It shall be shown that the NP-complete problem CLIQUE reduces to DTS2. (The decision problem CLIQUE asks if there is a clique (complete subgraph) of size for .) For a given instance of the a given graph problem CLIQUE, a corresponding instance of the problem can be constructed in polynomial time as follows. vertices in the set , we shall add For each of the two failure sources to the set . Therefore, define . The set of modules will

RAGHAVAN et al.: TEST SEQUENCING PROBLEMS

161

have cardinality where is defined by for . This definition essentially assigns the two failure associated with the vertex to module . sources tests associated with the decision table. There will be , there will be an associated test For each failure source . These tests can be used exclusively to construct . The a testing algorithm requiring a storage cost of remaining tests will be associated with the vertex set . For , there will be a test where if each vertex or if all three of the following properties hold: , . Alternatively, if and . associConsider the decision problem ated with the above construction. If the answer to is yes, it implies that there are modules that have not been partitioned by the tests of some decision tree for , tests which do not split these which means that there are actions. A test associated with vertex will split these actions corresponding to vertices not adjacent to . This implies that the vertices associated with the unpartitioned modules in the decision tree form a clique of size in the corresponding graph. Alternatively, if has a clique of size , then there is with vertices. To see a decision algorithm for this, note that only a path constructed of vertex tests will leave modules undivided. The tests associated with each of the individual failure sources can then be used to discriminate the failures in divided modules from those in the undivided modules. C. HEF for Minimal Storage Test Sequencing Consider an ambiguity set whose cardinality in terms of . Clearly, the minimal number of tests required modules is . We need a lower bound on the storage for isolation is cost of the optimal diagnostic tree rooted at . Assume, without loss of generality, that the test costs are in ascending order . is given by Theorem 4: The lower bound

implies that it be used at all nodes at depth . Thus, by extending the above rule, we can assign the least cost tests to nodes at various depths for this diagnostic tree resulting in a storage cost of

(34) . For this , the Now consider the case where maximum number of times the least cost test can appear is (depth = ) and the maximum of depth . Note that this tree is balanced only up to a . Now that the least cost test has been depth of nodes, can be used at used at nodes. From this point on, tests can be used at nodes, resulting in a storage cost given by (32), completing the proof of the theorem. The above proof of admissibility of the HEF is sufficient for obtaining optimal solutions when used with AO algorithm. In addition, we also show that the above HEF is consistent, thus ensuring that AO algorithm with this HEF monotonically is consistent, converges to an optimal solution. An HEF if for each node in an AND/OR graph (35) Consider the two minimal storage trees corresponding to and . Note that one can construct a valid diagnostic strategy (although not necessarily optimal) for the subset by connecting the two minimal storage trees with a binary link. The storage cost of this feasible tree at is given by (36) is clearly a lower bound, since for which bound for the optimal tree storage cost.

is a lower

D. Heuristic Storage Measures (32)

is an indicator function which is 1 when the logical where is an integer satisfying expression is true and 0 otherwise. (33) . An Proof: Let us first consider the case obvious and crude lower bound would be to take . However, this bound can be very weak for , times in a tree rooted since test can be used at most at . Such a tree is necessarily balanced, i.e., there are tests at depth from the root . Now that we have used the least , the maximum number cost test at all nodes at level , which of times any other test can appear in this tree is

In the following, we briefly present the heuristic algorithms considered in [7] for the minimal storage test sequencing which problem. Consider an ambiguity node and a test and based on its outcomes. Let splits into be the cardinality of be the cardinality of , and be the cardinality of . The heuristic storage measure is defined as follows: if Parent of

is root

otherwise (37)

is the test used to arrive at if is not the root where node. The single step look-ahead algorithm would select the test maximizing the average storage measure per-unit cost of

162

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART A: SYSTEMS AND HUMANS, VOL. 29, NO. 2, MARCH 1999

TABLE XI PERFORMANCE OF VARIOUS STORAGE MINIMIZING METHODS FOR

n = 10

3) test sequencing with a constraint on the number of distinct tests used; 4) the problem of minimal storage test sequencing. We developed optimal and near-optimal solutions for these problems based on AO and other heuristic top-down graph search techniques. ACKNOWLEDGMENT The authors would like to thank an anonymous reviewer for valuable comments on an earlier version of this paper and V. Rajan for help in the preparation of the final manuscript. REFERENCES

PERFORMANCE OF VARIOUS

TABLE XII STORAGE MINIMIZING METHODS FOR

n = 12

test (38) A multistep look-ahead algorithm expands the node into a partial tree up to a given depth (look-ahead value) for each admissible test, and picks the test that maximizes the sum of values over all nodes in the partial tree. Tables XI and XII show the relative suboptimality of the solutions obtained by various limited search AO algorithms Limited Search AO with the number of best tests ( retained at every OR-node during the AO cost propagation and pruning steps) and the Heuristic Storage Measure based Heuristic Algorithm of [7] with depth algorithms ( ). The AO based algorithms are clearly far superior to the heuristic methods of [7]. A nice feature of the limited search AO is that they result in monotonically better solutions, unlike the heuristic techniques where no monotonicity is guaranteed. VI. CONCLUSION In this paper, we considered a set of four test sequencing problems arising in Design for Testability. Specifically, we considered 1) robust test sequencing; 2) test sequencing with a constraint on one of the objectives;

[1] D. J. Klinger et al.., AT&T Reliable Manual. New York: Van Nostrand Reinhold, 1990. [2] D. P. Sieworek and R. Schwarz, The Theory and Practice of Reliable System Design. Bedford, MA: Digital, 1982. [3] S. L. Pollack, “Conversion of decision tables to computer programs,” Commun. ACM, vol. 8, pp. 677–682, 1965. [4] L. I. Press, “Conversion of decision tables to computer programs,” Commun. ACM, vol. 8, pp. 385–390, 1956. [5] C. R. Hartmann, P. K. Varshney, K. G. Mehrothra, and C. L. Gerberich, “Application of information theory to the construction of efficient decision trees,” IEEE Trans. Inform. Theory, vol. 28, pp. 565–577, July 1982. [6] L. T. Reinwald and R. M. Soland, “Conversion of limited entry decision tables to optimal computer programs II: Minimum storage requirement,” J. ACM, vol. 14, pp. 742–755, 1967. [7] O. J. Murphy and R. L. McCraw, “Designing storage efficient decision trees,” IEEE Trans. Comput., vol. 40, Mar. 1991. [8] L. Hyafil and R. L. Rivest, “Constructing optimal binary decision trees is NP-complete,” Inform. Process. Lett., vol. 5, no. 1, pp. 15–17, 1976. [9] D. Comer and R. Sethi, “The complexity of trie index construction,” J. ACM, vol. 24, pp. 428–440, 1977. [10] M. R. Garey, “Optimal binary identification procedures,” SIAM J. Appl. Math., vol. 23, no. 2, pp. 173–186, 1972. [11] K. R. Pattipati and M. G. Alexandridis, “Application of heuristic search and information theory to sequential fault diagnosis,” IEEE Trans. Syst., Man, Cybern., vol. 20, pp. 872–887, July 1990. [12] A. Mahanti and A. Bagchi, “Admissible heuristic search in AND/OR graphs,” Theor. Comput. Sci., vol. 24, pp. 207–219, July 1983. [13] V. Raghavan, M. Shakeri, and K. R. Pattipati, “Optimal and near-optimal test sequencing algorithms with realistic test models,” IEEE Trans. Syst., Man, Cybern., vol. 29, pp. 11–26, Jan. 1999.

Vijaya Raghavan (M’88) received the B.E. degree from Osmania University, India, in 1990, and the Ph.D. degree in controls and communication systems from the University of Connecticut, Storrs, in 1996. His research interests as a doctoral student included array signal processing, numerically robust target tracking algorithms, and automated fault diagnosis algorithms. He was with Qualtech Systems, Inc., Storrs, from 1995 to 1997, where his primary focus was to develop and implement efficient, near-optimal algorithms for a range of large-scale, computationally intensive problems in the area of system health management and automated fault diagnosis. He is currently with The Mathworks, Inc., Natick, MA. His main focus is the application of graph algorithms for production quality code-generation from hierarchical, directed-graph representations of reactive systems. He is responsible for the code-generation module of the Stateflow product, a software tool for the executable specification of control-flow algorithms, and design, simulation, and implementation of complex, event-driven systems. His current research interests include development of visual programming languages, code-generation, coverage and automatic test-data generation for hybrid (data-flow and control-flow) simulation systems. Dr. Raghavan received the Best Technical Paper award and the Best Student Paper award at the 1994 and 1995 IEEE Autotest Conferences.

RAGHAVAN et al.: TEST SEQUENCING PROBLEMS

Mojdeh Shakeri (M’97) received the B.S. and M.S. degrees in electrical engineering from Tehran University, Tehran, Iran, in 1988 and 1990, respectively, and the Ph.D. degree in control and communication systems from the University of Connecticut, Storrs, in 1997. She was with Qualtech Systems, Inc., Storrs, from 1995 to 1997. Her primary focus was to develop and implement efficient, near-optimal algorithms for a range of large-scale, computationally intensive problems in the area of system health management and automated fault diagnosis. Currently, she is with The Mathworks, Inc., Natick, MA. She is working on the simulation engine of Simulink, an interactive environment for modeling, analyzing, and simulating a wide variety of dynamic systems. Her main focus is to develop efficient algorithms for simulation of large-scale, hierarchical systems arising in the areas of controls and digital signal processing. Her research interests include measurement scheduling, automated testing, system fault diagnosis, system simulation, and real-time applications. Dr. Shakeri received the Best Technical Paper award and the Best Student Paper award at the 1994 and 1995 IEEE Autotest conference.

163

Krishna R. Pattipati (S’77–M’80–SM’91–F’95) received the B.Tech. degree in electrical engineering with highest honors from the Indian Institute of Technology, Kharagpur, in 1975, and M.S. and Ph.D. degrees in control and communication systems from the University of Connecticut, Storrs, in 1977 and 1980, respectively. He was with Alphatech, Inc., Burlington, MA, from 1980 to 1986, where he supervised and performed research on artificial intelligence and systems theory approaches to human decision modeling, multitarget tracking, and automated testing. He has served as a Consultant to Alphatech, Inc. and the IBM Thomas J. Watson Research Center, Yorktown Heights, NY. Since September 1986, he has been with the University of Connecticut, where he is a Professor of Electrical and Systems Engineering. He is also President of Qualtech Systems, Inc., Storrs, a small business specializing in software tools and solutions for testability, maintainability, and quality control. Dr. Pattipati was selected by the IEEE Systems, Man, and Cybernetics Society as the Outstanding Young Engineer of 1984 and received the Centennial Key to the Future Award. He won the Best Technical Paper awards at the 1985, 1990, and 1994 IEEE AUTOTEST Conferences and at the 1997 Command and Control Symposium. He served as the Vice-Chairman for invited sessions of the IEEE International Conference on Systems, Man, and Cybernetics, Boston, MA, 1989. He is the Editor of the IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS and is the Vice President for Technical Activities of the IEEE SMC Society (1998–2000).