a speci cation of the software under test, rather than from the implementation. ...... resentation fsm is an abstraction of the class's implementation and is used to.
Formal Derivation of Finite State Machines for Class Testing Leesa Murray, David Carrington, Ian MacColl, Jason McDonald and Paul Strooper Software Veri cation Research Centre Department of Computer Science and Electrical Engineering The University of Queensland, St Lucia Australia 4072
leesam, davec, ianm, jasonm, pstroop @csee.uq.edu.au
Abstract. Previous work on generating state machines for the purpose
of class testing has not been formally based. There has also been work on deriving state machines from formal speci cations for testing non-objectoriented software. We build on this work by presenting a method for deriving a state machine for testing purposes from a formal speci cation of the class under test. We also show how the resulting state machine can be used as the basis for a test suite developed and executed using an existing framework for class testing. To derive the state machine, we identify the states and possible interactions of the operations of the class under test. The Test Template Framework is used to formally derive the states from the Object-Z speci cation of the class under test. The transitions of the nite state machine are calculated from the derived states and the class's operations. The formally derived nite state machine is transformed to a ClassBench testgraph, which is used as input to the ClassBench framework to test a C++ implementation of the class. The method is illustrated using a simple bounded queue example.
1 Introduction The purpose of speci cation-based testing is to derive testing information from a speci cation of the software under test, rather than from the implementation. Although it is possible in theory to formally re ne such a speci cation into an implementation, this rarely happens in practice. However, when the implementation is developed informally from a formal speci cation, the speci cation can assist with testing, by allowing us to derive test inputs and the expected outputs for the implementation. Although a considerable amount of work has been done in the area of class testing [2], most of this work is not formally based. In this paper, we present a method for speci cation-based class testing, in which we use an Object-Z speci cation [7,8] of the class under test to generate a state machine that allows us to test the class implementation.Explicitly, we focus on the use of the formally derived nite state machine for practical work, that is, actually using the nite
state machine to generate and execute test cases, and evaluate test results. In previous work it is not clear that nite state machines derived from speci cations have actually been used to test implementations. In [22], we introduce object-orientation to the Test Template Framework [25,26] | a framework originally designed for non-object-oriented speci cationbased testing. We achieve this by deriving testing information (test inputs and expected outputs) from Object-Z speci cations and by inheriting testing information from parent classes. However, that work involves testing at the operation level. Our goal here is to focus on testing the class as a whole. We achieve this by deriving a class's nite state machine from its speci cation, which allows us to formulate sequences of the class's operations to conduct testing. The Test Template Framework does not execute tests; it is used to generate testing information. By using it to formally derive nite state machines for classes, we are providing a basis for test execution in a framework such as ClassBench [12,13]. ClassBench is a testing framework that models the class under test as a nite state machine. This model is referred to as a testgraph and represents a subset of the states of the class and the transitions between them. Currently a testgraph is developed informally in the ClassBench framework. ClassBench executes tests by traversing the testgraph and comparing the state of the class under test to the expected state supplied by an oracle class, to evaluate test results. We present the initial results of work on transforming the class's formally derived nite state machine to a ClassBench testgraph, which we use to test the class implementation. The Test Template Framework [22] and ClassBench [17] both provide support for genericity and inheritance, and therefore do support objectoriented testing. However, the derivation of nite state machines for classes, which is the focus of this paper, deals with class testing. In Sect. 2 we specify class Bounded Queue [G ], which is used throughout to illustrate our method. The Test Template Framework is introduced in Sect. 3. Section 4 details the process of deriving the states and transitions of a class's nite state machine from its Object-Z speci cation. This section concludes with the derived nite state machine of our example class. Section 5 addresses the reachability of states in nite state machines derived from speci cations. The process of transforming a nite state machine to a testgraph for use in the ClassBench framework is presented in Sect. 6. Related work is compared to our work in Sect. 7. In Sect. 8 we conclude with a discussion of our achievements and plans for future work. Throughout this paper, when referring to speci cation items we use italics | Extend , and implementation items are presented in typewriter font | Extend.
2 Example - Speci cation of Bounded Queue [G ] Our example class, Bounded Queue [G ], is a generic class representing a rst-in rst-out abstract data type. Bounded Queue [G ] is contrived to demonstrate the problem of unreachable states in nite state machines discussed in Sect. 5. The Object-Z speci cation of Bounded Queue [G ] is shown in Fig. 1.
Bounded Queue [G ] Max : N
j
items : seq G INIT items = hi Extend (items ) v? : G
items = items a hv ?i #items Max Wipe out (items ) items = hi Item result ! : G items = 6 hi result ! = head (items ) 0
Remove
(items )
items 6= h i items = tail (items ) 0
0
0
Has v? : G result ! : B result ! , v ? 2 ran items
Fig. 1. The Object-Z Speci cation of Bounded Queue [G ]. An Object-Z class is represented as a named box with optional generic parameters (here class Bounded Queue with generic parameter G ). A class contains an unnamed state schema, an initialisation schema (INIT ) and zero or more operations ( ve in this case). Inputs to operations are decorated with ? and outputs are decorated with !. Pre-state variables are unprimed and post-state variables are primed ( ). Operations that are permitted to change the state of the class have a delta list (denoted by ) in their declarations. The list identi es those variables that are 'open', that is, that do not have their unprimed and primed forms equated. An Object-Z operation is disabled outside its precondition, in contrast to Z where precondition violation gives an unde ned result. We represent a queue as a sequence of elements, items . The INIT schema speci es that a Bounded Queue [G ] is initially empty. The Extend operation appends an element to the queue provided the size of the queue is less than Max , a constant whose value is set by the environment. The speci cation of Extend is contrived to illustrate steps in our method for deriving nite state machines. Such a restriction on the addition of elements to a queue would usually form part of the class invariant. Remove removes the rst element of a non-empty queue, leaving the rest of the queue as the result. Wipe out deletes all elements from 0
the queue. Item returns the rst element of a non-empty queue. Has checks if a given element, v ?, is an element of the queue, returning true if it is and false otherwise. Item and Has do not change the state of the queue.
3 Test Template Framework Our approach to speci cation-based testing is based on the Test Template Framework (TTF) [15,25,26]. The TTF is a formal, abstract model of non-objectoriented testing, used to derive a hierarchy of test information from a modelbased formal speci cation. The hierarchy includes test inputs and expected outputs (oracles), but is not used directly for test execution. Most of the work with the TTF has used the Z speci cation notation. We have extended the TTF to accommodate object-oriented features such as those available in Object-Z [22]. We use the Extend operation from Fig. 1 to illustrate the TTF. The TTF uses the valid input space (VIS) of an operation as the source of all tests since the operation is unde ned for any inputs that are not in the VIS. The VIS is the precondition of the operation's schema and is computed by hiding the nal state and any outputs. For the example, VISExtend =b [items : seq G ; v ? : G j #items < Max ] The input space (IS) of an operation is the signature of its VIS. ISExtend =b [items : seq G ; v ? : G ] The basic unit for de ning test data in the TTF is a test template (TT). A TT is a constrained subset of the VIS and is expressed as a Z schema. We de ne a type for all TTs derived for the operation Extend : TTExtend == PVISExtend Test templates are organised into a hierarchy called the test template hierarchy (TTH). The root of this hierarchy is the VIS. The hierarchy is created by applying testing strategies to existing TTs to derive additional TTs. A strategy in the TTF identi es a particular technique for deriving test cases. The TTF encourages the use of multiple strategies to build the TTH, both traditional techniques such as input partitioning and boundary analysis, and specialised techniques that exploit the speci cation notation [26]. The TTH for Extend is formally declared as: j TTHExtend : TTExtend STRATEGY ! 7 PTTExtend where STRATEGY is the set of all possible testing strategies. For our example, we choose to apply a type-based strategy to partition the VIS into cases where the queue is empty and where it is nonempty. The resulting templates are: TB1 =b [VISExtend j items = h i] TB2 =b [VISExtend j items 6= h i] TTHExtend (VISExtend ; Type based ) = fTB1; TB2g
Naming of the TTs re ects the testing strategy used. The second strategy applied is boundary analysis for the cases where the cardinality of items is either the maximum permitted (Max ? 1) or less than the maximum. Only template TB2 can be partitioned so we obtain two further templates: BA2 1 =b [TB2 j #items = Max ? 1] BA2 2 =b [TB2 j #items < Max ? 1] TTHExtend (TB2; Boundary analysis ) = fBA2 1; BA2 2g :
:
:
:
At this stage our test template hierarchy is: VIS Extend
TB 1 TB 2
BA 2.1 BA 2.2
TTs are derived until the tester is satis ed that the subdivisions of the VIS are adequate to achieve satisfactory testing. From each leaf TT, we derive an instance template that uniquely de nes a single test case. If we instantiate the queue to store integer values with Max = 5, possible instance templates for Extend are: IT1 =b [TB1 j items = hi ^ v ? = 0] IT2 =b [BA2 1 j items = h1; 2; 3; 4i ^ v ? = 0] IT3 =b [BA2 2 j items = h1; 2i ^ v ? = 0] The formal speci cation can also be used as an oracle to determine the success or failure of each test. We derive oracle templates to precisely describe suitable output for a given input. An oracle template is calculated by (operation ^ T ) OSoperation where T is replaced by a test or instance template and OSoperation is the output space of operation . For example, the output space of Extend and the oracle for IT1 are: OSExtend =b [items : seq G ] oracleIT1 =b (Extend ^ IT1 ) OSExtend [items : seq G j items = h0i] The derivation of the test template hierarchies for the other operations of Bounded Queue [G ] appears in [21]. No testing strategies were used for Remove and Item , as we chose not to partition their valid input spaces. The cause e ect strategy [26] was used in the derivation of the test template hierarchy for Has and partition analysis [26] was used for Wipe out . :
:
0
0
0
4 Derivation of Finite State Machines This section details the derivation of a nite state machine from an Object-Z speci cation. We address the derivation in two parts: derivation of the states of the class's nite state machine and then the transitions. The process is illustrated using Bounded Queue [G ] and the section concludes with the complete nite state machine of Bounded Queue [G ]. 4.1
Derivation of States
We use the Test Template Framework to derive test templates and oracles for the class's operations. From the class's INIT schema, and each operation's test templates and oracles, we derive the states of the class's nite state machine. When deriving test templates from an operation's speci cation, we partition the operation's input space. The states that result from test templates only guarantee that some operation of the class can be executed in each state. However, there is no guarantee that the state reached after the operation terminates will be a member of the set of states derived from test templates. Therefore it is also necessary to use the oracles of the operations to derive the states of the nite state machine. We only consider oracles for operations that change the state of the class, as operations that do not change the state cannot contribute any new states. Considering test and oracle templates gives more equivalence classes, that is, a ner grained partition of the class's state space. The collection of states gained from the INIT schema, test templates and oracles provides the states for the class's nite state machine, and these are referred to as state templates. We derive the states from the leaf test templates of each operation's test template hierarchy by using schema hiding to restrict the signatures of the templates to be only the state variables. Hiding involves removing the input variables from the template's declaration and existentially quantifying them in the template's predicate. To derive states from oracles, we rename the primed state variables to their unprimed equivalents. Any output variables in the oracle templates are hidden and the templates simpli ed, so only state variables appear in the templates' declarations. Once the set of state templates is derived, we consider whether the templates are disjoint. If they are not, it is necessary to resolve any overlap of states so that we have a maximal partition1 of the class's state space. To achieve this, we apply Dick and Faivre's [5] transformation of a disjunction into disjoint components. This method uses equivalences of the form: A_B A^B _:A^B _A^:B
to obtain non-overlapping partitions. 1
By maximal partition, we mean there is no overlap of states and together the states completely cover the class's state space.
If the INIT schema is partitioned in the process of formulating disjoint templates, the subscript INIT is added to the names of the resulting templates. This allows us to track those templates that are initial states of the class's nite state machine. This process for deriving the states of a nite state machine from an Object-Z speci cation is illustrated using our example class Bounded Queue [G ]. Bounded Queue [G ]. The state template derived from Bounded Queue [G ]'s INIT schema is
Deriving the States of
STInit =b [items : seq G j items = h i]
The derivation of test templates for Bounded Queue [G ]'s Extend operation is shown in Sect. 3. The test template derivations for the other operations are similar and appear in [21]. The resulting leaf test templates of Bounded Queue [G ]'s operations are: TTExtend1 =b [items : seq G ; v ? : G j items = h i] TTExtend2 =b [items : seq G ; v ? : G j items 6= h i ^ #items = Max ? 1] TTExtend3 =b [items : seq G ; v ? : G j items 6= h i ^ #items < Max ? 1] TTRemove1 =b [items : seq G j items 6= h i] TTWipe out1 =b [items : seq G j items = h i] TTWipe out2 =b [items : seq G j items 6= h i] TTItem1 =b [items : seq G j items 6= h i] TTHas1 =b [items : seq G ; v ? : G j v ? 2 ran items ] TTHas2 =b [items : seq G ; v ? : G j items = h i] TTHas3 =b [items : seq G ; v ? : G j v ? 62 ran items ^ items 6= h i]
To formulate state templates, we take each test template and hide the input variables, which results in schemas that have only the (unprimed) state variables in their declarations. We assume that G , the generic parameter of Bounded Queue [G ], contains sucient elements to allow simpli cation of our state templates in the nite state machine. Upon re nement of G to a concrete type, the states are reviewed [26]. For the same reason, we assume that Max > 4 2. 2
At this point it is only necessary to assume Max > 2, but in subsequent sections we need to assume Max > 4.
ST1 =b TTExtend1 n (v ?) [items : seq G j items = h i] ST2 =b TTExtend2 n (v ?) [items : seq G j #items = Max ? 1] ST3 =b TTExtend3 n (v ?) [items : seq G j items 6= h i ^ #items < Max ? 1] ST4 =b TTRemove1 [items : seq G j items 6= h i] ST5 =b TTWipe out1 [items : seq G j items = h i] ST6 =b TTWipe out2 [items : seq G j items 6= h i] ST7 =b TTItem1 [items : seq G j items 6= h i] ST8 =b TTHas1 n (v ?) [items : seq G j items 6= h i] ST9 =b TTHas2 n (v ?) [items : seq G j items = h i] ST10 =b TTHas3 n (v ?) [items : seq G j items 6= h i]
We calculate oracles for the operations that change the state of the class. This ensures that all post states of operations are included in our nite state machine. The calculation of Extend 's output space is shown in Sect. 3. Calculation of the output spaces of the other operations is similar and is shown in [21]. After the calculation of the oracles, state templates are formed by renaming the primed state variables to their unprimed equivalents. Recall that we only calculate oracles for operations that change the state: Extend , Remove and Wipe out in this case. OracleST1 =b (Extend ^ ST1 ) OSExtend [items : seq G j #items = 1] ST11 =b [items : seq G j #items = 1] OracleST2 =b (Extend ^ ST2 ) OSExtend [items : seq G j #items = Max ] ST12 =b [items : seq G j #items = Max ] OracleST3 =b (Extend ^ ST3 ) OSExtend [items : seq G j #items > 1 ^ #items < Max ] ST13 =b [items : seq G j #items > 1 ^ #items < Max ] OracleST4 =b (Remove ^ ST4) OSRemove [items : seq G ] ST14 =b [items : seq G ] OracleST5 =b (Wipe out ^ ST5 ) OSWipe out [items : seq G j items = h i] ST15 =b [items : seq G j items = h i] OracleST6 =b (Wipe out ^ ST6 ) OSWipe out [items : seq G j items = h i] ST16 =b [items : seq G j items = h i] One reason for the contrived speci cation of Extend is to illustrate the important role that oracles have in the formation of states. If we did not use oracles, classes might not be fully tested as operations might not be executed in all possible states. Without oracles, we would not have the state template ST12, which 0
0
0
0
0
0
0
0
0
0
0
0
speci es a full queue. Without ST12 , there is no guarantee that we will test a full queue. As a result, we may not generate tests from the nite state machine to examine the class's behaviour when the queue is full. Other templates such as ST5 include ST12 as a special case, but there is no guarantee that this special case will be chosen, so we need ST12 to ensure the Extend operation is fully exercised. The set of state templates for Bounded Queue [G ] so far is f STInit =b [items : seq G j items = h i], ST2 =b [items : seq G j #items = Max ? 1], ST3 =b [items : seq G j items 6= h i ^ #items < Max ? 1], ST4 =b [items : seq G j items 6= h i], ST11 =b [items : seq G j #items = 1], ST12 =b [items : seq G j #items = Max ], ST13 =b [items : seq G j #items > 1 ^ #items < Max ], ST14 =b [items : seq G ]g We now transform this set into a set of disjoint state templates. STInit and ST4 partition ST14 (ST14 STInit _ ST4 ) so ST14 is discarded. STInit is disjoint from all other state templates. Templates ST2 and ST3 are disjoint, but ST2 and ST4 are not, so we apply Dick and Faivre's [5] transformation. ST2 _ ST4 ST2 ^ ST4 _ : ST2 ^ ST4 _ ST2 ^ : ST4 [items : seq G j #items = Max ? 1] ^ [items : seq G j items 6= h i] _ [items : seq G j #items = 6 Max ? 1] ^ [items : seq G j items = 6 h i] _ [items : seq G j #items = Max ? 1] ^ [items : seq G j items = h i] [items : seq G j #items = Max ? 1] _ [items : seq G j items = 6 h i ^ #items = 6 Max ? 1] _ [items : seq G j false ] ST2 _ [items : seq G j items = 6 h i ^ #items = 6 Max ? 1]
Therefore we have a new state template, ST17 =b [items : seq G j items 6= h i ^ #items 6= Max ? 1]
and ST4 is excluded from our set of templates as it is partitioned by ST2 and ST17 (ST4 ST2 _ ST17).
Details of the remainder of the transformation are shown in [21]. Our resulting set of state templates for Bounded Queue [G ] is f STInit =b [items : seq G j items = h i], ST2 =b [items : seq G j #items = Max ? 1], ST11 =b [items : seq G j #items = 1], ST12 =b [items : seq G j #items = Max ], ST18 =b [items : seq G j #items > 1 ^ #items < Max ? 1], ST19 =b [items : seq G j #items > Max ]g
4.2
Derivation of Transitions
In the previous section, we partitioned a class's state into state templates to be used as the nodes of the class's nite state machine. We derive the transitions of the nite state machine by considering each pair of states and checking whether the pair is related by an operation. In this section, we present a formal model of this process and give examples. The model is extended in Sect. 5 to include reachability. We take the set of all states of the class under test as given, we de ne operations as relations on states, and we identify a class as a set of operations. [State ] Op == State $ State class : POp
A valid transition in a nite state machine occurs when a pair of states (s1 ; s2), is related by an operation. For each valid transition identi ed, an arc labelled with the operation's name, beginning in state s1 and terminating in state s2 , is added to the class's nite state machine. Each operation requires n 2 (where n is the number of states) valid transition calculations. This work is mechanical, but detailed and tool support would be of great assistance. However, automation of this process is beyond the scope of this paper. 0
0
Bounded Queue [G ]. We choose two possible Bounded Queue [G ] transitions to illustrate the derivation of valid and invalid transitions of a class. To show that (STInit ; STInit ) are not related by Extend and therefore do not form a valid transition in Bounded Queue [G ]'s nite state
Deriving the Transitions of
0
machine, we prove 9 ISExtend ; OSExtend STInit ^ Extend ^ STInit 9 items ; items : seq G ; v ? : G j items = h i ^ items = items a hv ?i ^ #items Max ^ items = h i false with ISExtend and OSExtend as the input and output spaces of Extend ([items : seq G ; v ? : G ] and [items : seq G ]) respectively. On the other hand, to show that (STInit ; ST11) are related by Extend and therefore form a valid transition, we prove 9 ISExtend ; OSExtend STInit ^ Extend ^ ST11 9 items ; items : seq G ; v ? : G j items = h i ^ items = items a hv ?i ^ #items Max ^ #items = 1 true 0
0
0
0
0
0
0
0
0
0
0
0
The complete derivation of the Bounded Queue [G ] transitions appears in [21]. STInit is the only initial state of Bounded Queue [G ] as it was not partitioned during the derivation. We represent initial states by an incoming arc that does not originate in a state and has no label. The resulting nite state machine of class Bounded Queue [G ] is shown in Fig. 2. Note that the nite state machine maintains assumptions made during the generation of templates, namely, that Max is at least 5 and that the generic type G contains sucient elements. Wipe_out Item, Has Extend STInit
Extend, Remove, Item, Has Extend
ST11 Remove Wipe_out
Item, Has
Extend
Extend
ST18 Remove
ST2 Remove
ST12 Remove Item, Has
Wipe_out Has
Remove
Wipe_out ST19
Wipe_out
Wipe_out Remove, Item, Has
Fig. 2. Class Bounded Queue [G ]'s Finite State Machine.
5 Reachability As Bounded Queue [G ] illustrates, it is possible to derive a nite state machine from a class's speci cation that contains unreachable states. By unreachable, we mean states that cannot be reached from the initial state via a sequence of transitions. ST19 is an example in Fig. 2. The goal of generating a nite state machine for a class is so we can construct sequences of tests to exercise the class. We are not only interested in testing each operation in isolation but also testing interactions between operations. While a class's speci cation may generate unreachable states, and it may be possible to execute operations in these states and perform legal transitions, if the interaction of the class's operations does not allow us to reach these states they cannot be tested. To formally identify reachable states, we rst specify all possible paths that are present in the class's nite state machine. Path == fss : seq State j 8 i : 2 :: #ss 9 op : class (ss (i ? 1); ss (i )) 2 op g
We are interested in those paths that begin in an initial state of the nite state machine. We call these realisable paths. initial : PState Realisable == fp : Path j head (p ) 2 initial g
where initial is the set of initial states of the nite state machine and head (p ) returns the rst state of the path p , thus giving us a set of paths where each path begins at an initial state. From Realisable , we can identify those states of the class's nite state machine that are reachable, since all states in a realisable path are reachable states. Reachable == ran([ Realisable ) Reachable therefore consists of the states that occur in at least one realisable path. Any states of the nite state machine that are not elements of Reachable
are eliminated along with any arcs that originate or terminate in them. The resulting nite state machine for Bounded Queue [G ] is shown in Fig. 3. Wipe_out Item, Has Extend
Extend, Remove, Item, Has Extend
ST11
STInit Remove Wipe_out
Item, Has
Extend ST18
Remove
Extend ST2
Remove
ST12 Remove Item, Has
Wipe_out Has
Wipe_out Wipe_out
Fig. 3. Class Bounded Queue [G ]'s Finite State Machine with Reachable States.
6 Testing Classes A formallyderived nite state machine for a class allows us to identify the allowed interactions of its operations and possible sequences of tests. This section shows how such a nite state machine can be used as the basis of a test suite for a state-based testing framework such as ClassBench [12,13].
For each ClassBench test suite, the tester must provide three components: a
testgraph, an Oracle class, and a Driver class. The testgraph is a nite state
machine that models a subset of the possible states and transitions of the Class Under Test (or CUT). The testgraph nodes correspond to states of the CUT and the arcs to the transitions; there is one distinguished node, the start node, that represents the initial state of the CUT. The Oracle class is derived from the Object-Z speci cation of the CUT [18]. The oracle inherits the CUT's implementation and augments its operations to check the CUT's state is always consistent with the speci cation. The oracle also checks the values of any outputs, reporting an error if they do not comply with the speci cation. The Driver class is called by the framework as the testgraph is traversed. The Driver must provide two functions. The Driver::arc() function performs the state transitions associated with each testgraph arc. The Driver::node() function checks that the CUT behaviour is correct for the state corresponding to the current testgraph node. At run-time, the framework automatically traverses the testgraph, calling Driver::arc() each time an arc is traversed, and calling Driver::node() each time a node is reached. The ClassBench traversal algorithm guarantees arc and node coverage of the testgraph. Before we can test the class with ClassBench, we must implement the class and transform the formally derived nite state machine into a ClassBench testgraph. Since ClassBench tests C++ implementations, we have chosen C++ as the implementation language. In the implementation, we must also decide on values for the class constants and generic parameters. Even though the class can be implemented as a generic class in C++ using templates, we cannot test it unless we instantiate the generic parameters. For this example, we have given Max a value of 5, and chosen the type int (integer) to replace the generic parameter G. We must also transform the state machine to a ClassBench testgraph. In the formally derived nite state machine, a state typically represents a set of states that is considered to be an equivalence class. In a ClassBench testgraph, a node represents one state. Therefore, we must transform the nite state machine states to testgraph nodes. To achieve this, we choose one state for each set of states in the nite state machine. Bounded Queue [G ]'s nite state machine has a single initial state, which becomes the start node of the testgraph3 . The result of transforming the states of Bounded Queue [G ]'s nite state machine (shown in Fig. 3), to testgraph nodes is shown in Fig. 4. The initial node is distinguished by an incoming arc that has no source node. Above each node in the testgraph we identify the queue that it 3
A nite state machine may have more than one initial state, but a testgraph has only one start node, and this node must correspond to the unique initial state of the class implementation. Note that this may mean that some of the other initial states of the nite state machine become unreachable, which can be addressed by adding arcs to the testgraph.
represents, and below each node we identify the nite state machine state from which it is derived. [] Empty ST Init
[1]
[1,2,3]
One
Multiple
ST 11
ST 18
[1,2,3,4]
[1,2,3,4,5]
Almost_Full ST 2
Full ST 12
Fig. 4. Class Bounded Queue [G ]'s Testgraph Nodes (for Max = 5). In a testgraph, all nodes should be reachable. An arc, or transition, in a nite state machine represents the execution of one operation call. An arc in a testgraph represents a sequence of calls. For example, Extend2 in Fig. 5 is implemented by calling the Extend operation of Bounded Queue [G ] twice; the rst call adds 2, and the second adds 3. We must also cover the nite state machine's transitions that change the state of the class. Transitions that do not change the state, such as the Has and Item transitions in Fig. 3, are tested by including them in Driver::node(). At present, ClassBench does not allow an arc to have the same source and destination nodes. ClassBench also does not permit two arcs to have the same source and destination nodes, as Remove and Wipe out do between states ST11 and STInit in Fig. 3. To overcome these problems, we duplicate nodes in our testgraph as we have done to node Empty in Fig. 5. The complete testgraph for Bounded Queue [G ] that achieves coverage of the transitions of its nite state machine in Fig. 3 is shown in Fig. 5. Of particular interest is arc Extend2, because it represents two transitions in Bounded Queue [G ]'s nite state machine. Extend2 begins in node One (derived from state ST11) and terminates in node Multiple (derived from state ST18 ). In the nite state machine, there is an Extend transition from ST11 to ST18, and one from ST18 to itself. Arc Extend2 in the testgraph absorbs both of these transitions, as it has two calls to the Extend operation. The rst call translates to the ST11 to ST18 Extend transition in the nite state machine. The second call translates to the Extend transition from ST18 to itself. The derived testgraph was used to test Bounded Queue [G ]'s implementation using ClassBench. During the testing, the ClassBench traversal algorithm generated 41 calls to Driver::node() and 36 calls to Driver::arc(). This testing achieved 100% statement coverage of Bounded Queue [G ]'s implementation, showing that the choice of one concrete state in the testgraph for each equivalence class (state) in the nite state machine, adequately exercised the class's implementation.
Extend1
[]
[1]
Empty
One
Wipeout
Wipeout
[1,2,3]
Extend3
Multiple Remove2
Remove1 None
Extend2
Wipeout
[1,2,3,4]
Extend4
Almost_Full Remove3 Wipeout
[1,2,3,4,5] Full
Remove4 Wipeout
Empty1
Fig. 5. Class Bounded Queue [G ]'s Testgraph.
7 Related Work In this section, we discuss related work on the formal derivation of nite state machines from speci cations for testing purposes, the generation of nite state machines from Object-Z speci cations, and the use of nite state machines for testing classes. Our work builds on that of Dick et al. [1,5] who provide a method for generating test cases and for constructing a nite state machine from a model-based speci cation. Using partition analysis, they reduce the speci cation to disjunctive normal form. The transitions of the nite state machine are a set of expressions, termed sub-operations, that are derived from the partition analysis. The states of the nite state machine are the disjoined before and after states of the sub-operations. The sub-operations correspond to our consideration of pairs of states in Op in Sect. 4.2. We extend their work by using an object-oriented notation, Object-Z, and by using the resulting nite state machine as input for a test execution framework. Hierons [11] describes the generation of a nite state machine from a Z speci cation. The input domain is partitioned using the category-partition method of Ostrand and Balcer [24]. The Test Template Framework, in contrast, permits use of a variety of testing strategies, including category-partition. Hierons calculates states for the nite state machine by rewriting the speci cation as disjoined input and output predicates (e ectively preconditions and postconditions). The transitions are determined by pairwise consideration of states as operation preand postconditions. The e ect is similar to the machine constructed by Dick et al. Hierons also notes previous work on test control and test sequencing from nite state machines but does not pursue these aspects. We are using ClassBench for test control and sequencing. Dong, Zucconi and Duke [6] informallygenerate a nite state machine from an Object-Z speci cation. Their motivation is to highlight the system behaviour in a nontrivial speci cation, that is, supporting comprehension rather than testing. States are formed from a permutation of the cross product of the possible values of the critical state variables of the class. Transitions are Object-Z operations with instantiated preconditions.
Bosman and Schmidt [3] use nite state machines to test classes. Their method uses state machines that result from Statecharts [10] used in objectoriented analysis and design techniques. A design fsm for a class is generated from the state diagrams and de nes the expected outcomes of test cases. A representation fsm is an abstraction of the class's implementation and is used to drive the testing of the class. It also provides a mapping between the design and implementation of the class. Their approach is similar to ours but starts from a behavioural rather than a model-based speci cation. Object-Z is richer for constraining underlying state, whereas Statecharts are stronger for constraining behaviour. Their tool support appears less developed than ClassBench. Hong et al. [14] apply conventional data ow testing techniques to class testing. A nite state machine is used to specify class behaviour and is transformed into a owgraph of data member de nitions and uses. The nite state machine is developed as a component of the overall system model in contrast to our approach of deriving the state machine from a formal speci cation. Application of data ow testing techniques to speci cation-based testing is a novel aspect of this work. Our approach permits use of a range of test derivation strategies rather than being restricted to a single class of techniques, and uses a model-based notation rather than a behavioural notation. McGregor and Dyer [19,20] describe a technique for constructing the nite state machine of a subclass from the nite state machine of its parent class. They use Objectcharts [4], an object-oriented variant of Statecharts, and restrict the inheritance relationship of classes to subtyping. The focus of this work is test case generation from behavioural notations, with a particular emphasis on reuse through inheritance. Transition coverage is considered as a criterion for test adequacy but use of the state machine for test execution is not discussed. Turner and Robson's [28] state-based testing technique uses nite state machines for test case generation. Finite state machines are constructed to model the internal representation of a class, in contrast to Bosman and Schmidt, and McGregor and Dyer, who use a state machine constructed for object-oriented analysis and design. This work highlights the importance of considering state in object-oriented testing. In our approach, the Test Template Framework provides access to both the state of the class and the local parameters of an operation. Testing strategies can be applied to either. Gao et al. [9] also construct a state model speci cally for testing. The model is similar to Objectcharts and it is unclear how it is derived from a class design speci cation.
8 Conclusions We have described a method for the formal derivation of a nite state machine for a class from its Object-Z speci cation. Test inputs and outputs derived from the class's operations using the Test Template Framework comprise the states of the nite state machine. Transitions are calculated to determine operation execution and termination. The nite state machine provides us with a mechanism for
testing the class as it details the possible interactions of the class's operations. By using the nite state machine in a testing framework such as ClassBench, the tests are executed as ClassBench traverses the nite state machine. The derived nite state machine must undergo a transformation to be suitable for use with ClassBench. Initial details of this transformation process are provided and a formalisation of this transformation is work in progress [16,23]. Others have formally derived nite state machines from model-based speci cations, but not from object-oriented speci cations. Finite state machines generated from object-oriented analysis and design diagrams are used to aid in the testing of classes, but the generation of the nite state machine is not formal. By formally deriving the nite state machine from the class speci cation, we have a representation of the expected behaviour of the class. Executing this expected behaviour by way of a transformed nite state machine in ClassBench enables us to test an implementation e ectively. The example used to illustrate the formal derivation of a nite state machine is deterministic. However, Object-Z allows the speci cation of non-deterministic operations, the Test Template Framework and nite state machines handle non-determinism, as does ClassBench [23]. Work currently under investigation is the development of a tool to support and partially automate the Test Template Framework. Plans for future work include an open tool set that allows us to link the Test Template Framework and ClassBench, so that we have integrated tool support for testing object-oriented software based on formal speci cations.
Acknowledgements The authors wish to thank the anonymous reviewers for their constructive comments and suggestions. This work is supported by Australian Research Council grant A4-96-00282. Ian MacColl is supported by an Australian Postgraduate Award and a Telstra Research Laboratories Postgraduate Fellowship. Jason McDonald is supported by an Australian Postgraduate Award.
References 1. J. Bicarregui, J. Dick, B. Matthews and E. Woods. Making the most of formal speci cation through animation, testing and proof. Science of Computer Programming, 29:53-78, 1997. 2. R. Binder. Testing object-oriented software: a survey. Software Testing, Veri cation and Reliability, 6:125-252, 1996. 3. O. Bosman and H. Schmidt. Object test coverage using nite state machines. In C. Miggins, R. Duke and B. Meyer, editors, Technology of Object-Oriented Languages and Systems (TOOLS 18), pp. 171-178, Prentice Hall, 1995. 4. D. Coleman, F. Hayes and S. Bear. Introducing Objectcharts or how to use Statecharts in object-oriented design. IEEE Transactions on Software Engineering, 18(1):8-18, 1992.
5. J. Dick and A. Faivre. Automating the generation and sequencing of test cases from model-based speci cation. In J.C.P. Woodcock and P.G. Larsen, editors, FME '93: Industrial Strength Formal Methods, 5th International Symposium of Formal Methods Europe, Volume 670 of Lecture Notes in Computer Science, pages 268-284, Springer-Verlag, 1993. 6. J. Dong, L. Zucconi and R. Duke. Specifying Parallel and Distributed Systems in Object-Z: The Lift Case Study. International Workshop on Software Engineering for Parallel and Distributed Systems, pages 140-149, IEEE Computer Society, 1997. 7. R. Duke, G. Rose and G. Smith. Object-Z: A speci cation language advocated for the description of standards. Computer Standards and Interfaces, 17:511-533, 1995. 8. R. Duke and G. Rose. Formal Object-Oriented Speci cation and Design Using Object-Z, Academic Press, Formal Methods Series, to appear 1998. 9. J.Z. Gao, D. Kung, P. Hsia, Y. Toyoshima and C. Chen. Object State Testing for Object-Oriented Programs. Computer Software & Applications Conference (COMPSAC 95), pp. 232-238, 1995. 10. D. Harel. Statecharts: A visual formalism for complex systems. Science of Computer Programming, 8(3):231-274, June 1987. 11. R. Hierons. Testing from a Z Speci cation. Software Testing, Veri cation and Reliability, 7:19-33, 1997. 12. D.M. Ho man and P.A. Strooper. The testgraph methodology: Automated testing of collection classes. Journal of Object-Oriented Programming, 8(7):35-41, November-December 1995. 13. D.M. Ho man and P.A. Strooper. ClassBench: A methodology and framework for automated class testing. Software Practice and Experience, 27(5):573-597, May 1997. 14. Hyoung Seok Hong, Yong Rae Kwon and Sung Deok Cha. Testing of object-oriented programs based on nite state machines. In Proceedings Asia-Paci c Software Engineering Conference (APSEC '95), pp. 234-241, 1995. 15. I. MacColl, D. Carrington and P. Stocks. An experiment in speci cation-based testing. In Nineteenth Australasian Computer Science Conference Proceedings (ACSC '96), pages 159-168, 1996. 16. I. MacColl, L. Murray, P. Strooper and D. Carrington. Speci cation-based objectoriented testing: A case study. Technical Report 98-08, Software Veri cation Research Centre, The University of Queensland, 1998. Submitted to ICFEM'98. 17. J. McDonald and P.A. Strooper. Testing inheritance hierarchies in the ClassBench framework. In Proc. Technology of Object-Oriented Languages and Systems (TOOLS USA '96), 1996. In press. 18. J. McDonald. Translating Object-Z speci cations to passive test oracles. Technical Report 98-04, Software Veri cation Research Centre, The University of Queensland, 1998. Submitted to ICFEM'98. 19. J. McGregor and D. Dyer. A Note on Inheritance and State Machines. SIGSOFT Software Engineering Notes, 18(4):61-69, October 1993. 20. J. McGregor. Constructing Functional Test Cases Using Incrementally Derived State Machines. 11th International Conference and Exposition on Testing Computer Software, pp. 377-386, 1994. 21. L. Murray, D. Carrington, I. MacColl, J. McDonald and P. Strooper. Formal Derivation of Finite State Machines for Class Testing. Technical Report 98-03, Software Veri cation Research Centre, The University of Queensland, 1998.
22. L. Murray, D. Carrington, I. MacColl and P. Strooper. Extending Test Templates with Inheritance. In P. Bailes, editor. Proceedings of 1997 Australian Software Engineering Conference (ASWEC '97), pages 80-87, IEEE Computer Society, 1997. Available in extended form as SVRC TR 97-18 at http://svrc.it.uq.edu.au/Bibliography/svrc-tr.html?97-18
23. L. Murray, J. McDonald and P. Strooper. Speci cation-based class testing with ClassBench. Technical Report 98-12, Software Veri cation Research Centre, The University of Queensland, 1998. Submitted to APSEC'98. 24. T. Ostrand and M. Balcer. The category-partition method for specifying and generating functional tests. Communications of the ACM, 31(6):676-686, June 1988. 25. P. Stocks and D. Carrington. A framework for speci cation-based testing. IEEE Transactions on Software Engineering, 22(11):777-793, November 1996. 26. P. Stocks. Applying formal methods to software testing. PhD Thesis. Department of Computer Science and Electrical Engineering, The University of Queensland, 1993. 27. C. Turner and D. Robson. State-Based Testing and Inheritance. Technical Report TR-1/93, Computer Science Division, University of Durham, 1993. 28. C. Turner and D. Robson. A state-based approach to the testing of class-based programs. Software | Concepts and Tools, 16:106-112, 1995.
