Model checking embedded software of an industrial

186

Int. J. Information Technology, Communications and Convergence, Vol. 1, No. 2, 2011

Model checking embedded software of an industrial knitting machine Thomas Reinbacher* and Martin Horauer Department of Embedded Systems, University of Applied Sciences Technikum Wien, Höchstädtplatz 5, A-1200 Vienna, Austria E-mail: [email protected] E-mail: [email protected] *Corresponding author

Bastian Schlich and Jörg Brauer Embedded Software Laboratory, RWTH Aachen University, Ahornstraβe 55, D-52074 Aachen, Germany E-mail: [email protected] E-mail: [email protected]

Florian Scheuer TEXION Software Solutions, Rotter Bruch 26a, D-52068 Aachen, Germany E-mail: [email protected] Abstract: Model checking is a promising approach for the verification of embedded systems software. The [mc]square approach for verification of binary code provides several improvements compared to other existing methods: the system model is automatically derived from the binary code using dedicated microcontroller simulators and state spaces are reduced by applying automatic abstraction techniques. In this paper, we survey the involved mechanisms and assess the overall approach by conducting an industrial case study – the verification of the embedded software of a monitoring device of a knitting machine. Keywords: model checking; static analysis; binary code; formal verification; embedded systems; embedded software; industrial knitting machine. Reference to this paper should be made as follows: Reinbacher, T., Horauer, M., Schlich, B., Brauer, J. and Scheuer, F. (2011) ‘Model checking embedded software of an industrial knitting machine’, Int. J. Information Technology, Communications and Convergence, Vol. 1, No. 2, pp.186–205. Biographical notes: Thomas Reinbacher received his MSc in Electrical Engineering from the University of Applied Sciences Technikum Wien, Vienna, Austria in 2009. Currently, he is doing his PhD at the Vienna University of Technology. His research interests focus on formal verification and testing strategies for embedded systems software.

Copyright © 2011 Inderscience Enterprises Ltd.

Model checking embedded software of an industrial knitting machine

187

Martin Horauer received his PhD from the Faculty of Electrical Engineering and Information Technology, Vienna University of Technology, Austria in 2004. He is the Head of the bachelor degree programme in Electronics Engineering and is the Associate Head of the master programme in embedded systems at the University of Applied Sciences Technikum Wien. His research interests include modelling, design, test, and verification of embedded systems with a focus on dependable automotive electronics. Bastian Schlich received his PhD in Computer Science from the RWTH Aachen University in 2008 with distinction. For his thesis, he was awarded with the Borchers Plakette. He heads the formal verification group of the Embedded Software Laboratory at the RWTH Aachen University. His research focuses on the application of different formal methods such as static analysis, abstract interpretation and model checking to software for embedded systems. Jörg Brauer received his Diploma degree in Computer Science from the Christian-Albrechts University of Kiel, Germany in 2008, before he joined the Embedded Software Laboratory of the RWTH Aachen University as a PhD student. His research focuses on the development and application of abstract interpretation techniques for binary code verification. Florian Scheuer received his Diploma in Computer Science from the RWTH Aachen University in 2007. Currently, he is working as a Development Engineer for embedded systems with TEXION Software Solutions.

1

Introduction

Embedded systems are becoming ubiquitous and an inherent part of our daily life. The verification of software used in these systems – integrated for example in today’s automobiles, medical devices, and industrial applications – is extremely important since failures caused by software are costly in almost every regard. Traditional methods such as testing are often infeasible in order to establish correctness of safety-critical applications since the covered test space is only a subset of all possible system configurations. Whereas testing examines only a limited number of specified test cases, formal verification methods such as model checking (Clarke et al., 1999; Baier and Katoen, 2008) examine all reachable states of a system. Model checking is an automatic approach with the goal to verify (possibly infinite) systems that essentially performs an exploration of system states in order to check whether the given system model satisfies a certain specification. While model checking gained some momentum in the context of software development and is frequently used in hardware verification, it plays a minor role in context of embedded systems software so far. Applying model checking to embedded systems code brings along some difficulties. These are: 1

the additional effort of creating a system model

2

the difficulty of keeping the model consistent with the software design

3

the large state spaces preventing the approach from scaling to industrial-sized projects.

188

T. Reinbacher et al.

The [mc]square binary code model checker addresses these limitations. Model checking at the level of binary code has to deal with a lot of details, and hence, faces larger state spaces compared to model checkers that operate on intermediate representations. With regard to embedded software, however, it also reveals failures introduced by the compiler and the implementation that may not be visible on higher levels of abstraction. Particularly in the embedded systems domain, it is not unknown for compilation itself to introduce errors. Model checking is applied to the application’s binary code as deployed on the real target hardware, by using a tailored target MCU simulator to build state spaces, thereby, alleviating issues due to 1 and 2. The process of model creation is therefore shifted from the test engineer towards the verification tool. Furthermore, [mc]square features various automatic abstraction techniques to deal with the restrictions related to 3. Some of these abstraction techniques are based on static analyses, which are used to examine the application code prior to model checking and provide additional information. In this paper, we present our experiences with model checking a monitoring application for an industrial knitting machine in order to assess the maturity of binary code model checking using [mc]square. The remainder of this paper is structured as follows. First, related work is discussed in Section 2. Next, a brief overview of [mc]square is given in Section 3. Then, details of the case study application, the derived specification, details of the verification process, and findings are presented in Section 4. Finally, lessons learnt and future work are discussed in Section 5.

2 Related work Numerous case studies have been conducted in order to evaluate the effectiveness of model checking in practise. Dong et al. (1999) describe the application of different model checkers – including Spin, Murphi and SMV – in order to detect a bug in the i-protocol. Ribeiro et al. (2005) describe model checking of embedded systems with Promela and the Spin model checker. Unlike our approach, they model the embedded system with a variant of Petri Nets in the Promela language. In contrast to our work, all these approaches rely on manual translations. Spin was also used in another case study conducted by Havelund et al. (2000). In this case study, a deep space autonomy flight software called remote agent was verified using an early version of Java path finder, which translated the source code of the software into Promela. The model checker Uppaal, as described by Bengtsson et al. (1996), is suited for automatic verification of safety and bounded liveness properties of real-time systems modelled as networks of timed automata extended with data variables. Uppaal has been frequently used to verify industrial systems, for instance, by Lindahl et al. (1998) and Havelund et al. (1999). In contrast to [mc]square, time can be modelled in Uppaal and is considered in the verification process. This, however, can aggravate the state-explosion problem. Uppaal focuses on the verification of systems modelled as networks of timed automata. If the system is not modelled as such a network, a manual translation is needed, which can be time-consuming and error-prone. Related model checkers targeting embedded systems are MCESS as detailed by Schlich et al. (2006) and ESTES as in Mercer and Jones (2005). ESTES model checks assembly code for the 68HC11 microcontroller by building the state space with help of the GNU debugger. In practise, this approach is only feasible for very small


189

programmes. MCESS, in contrast, translates the assembly code of ATMEL ATmega16 microcontrollers into hardware-independent bytecode for a specific virtual machine that is able to check LTL properties. Most hardware issues, however, are coarsely abstracted, possibly removing essential information. This may invalidate the entire verification process. Several verification and validation processes of safety-critical systems using static analysis have been described in the past. For instance, Bouissou et al. (2009) describe the validation of a real-time embedded space software using the Astrée analyser. Similarly, Kästner et al. (2008) describe their experiences with timing validation of automotive software using AIT and SYMTA/S, two commercially available tool-suites for analysing worst-case execution times and worst-case response times. To the best of our knowledge, our work is the first case study that combines model checking and static analysis of embedded systems software directly on the binary code. Most of the previously conducted case studies with [mc]square (c.f., Noll and Schlich, 2008; Schlich, 2008; Reinbacher et al., 2008b; Reinbacher et al., 2009) were of smaller code size, less complex, or for different target platforms.

3 The [mc]square framework [mc]square is an explicit-state binary code model checker for computation tree logic (CTL) tailored to formal verification of software for microcontrollers. It features model checking and static source code analysis for various target platforms, including the ATMEL ATMega series, the Intel MCS-51, the Infineon XC16x, and programmable logic controllers, as detailed in Schlich (2008), Reinbacher et al. (2008a), Scheuer (2007) and Schlich et al. (2009). Figure 1

The [mc]square model checking framework Binary . code

.

C code . and Dbg. files

Program parser

.

Static analyser

CTL . formula

.

Environment . model

CTL parser

.

Environment parser

.Simulators

.

State space

.AVR .

.C51 .PLC

.Model checker . Counterexample generator .[mc]square

.C167

. . ...

190


3.1 Approach Model checking using [mc]square works as follows: First, the user provides a specification in CTL and a binary-code file (a compiled and linked Intel-Hex or ELF file), as shown in Figure 1. If possible, a mapping between the binary code and the high-level programming language is established using debug information. This allows using, for instance, C variables in the specification. Then, [mc]square uses an accurate and customised microcontroller simulator to build state spaces, which rules out the manual and often error-prone process of model creation. In our approach, the system model is part of the model checker itself. An advantageous side-effect of the simulator-based approach is a full MCU simulator, allowing the user to analyse and debug the code from within the model checker. Later on, if required, [mc]square generates a counterexample to pinpoint the user to an execution path that disproves the specification, which is a valuable support in fixing the defect. If the specification is satisfied, it generates a witness to demonstrate that the property is fulfilled by the application. A thorough overview of [mc]square is given by Schlich (2008).

3.2 Abstraction A major challenge in model checking is the state-explosion problem (see Clarke, 2008) caused by the exponential growth of state spaces due to parallelism and large domains of variables. In order to successfully verify the monitoring application at hand, we had to mitigate the state-explosion problem using various concepts. Regarding microcontroller applications, the main reason for the state-explosion problem is nondeterministic behaviour, which is introduced whenever the microcontroller application interacts with its environment, for instance, by reading values over I/O lines or due to the occurrence of interrupts. Table 1 3-valued memory representation Internal representation

Address @0xA @0xB @0xC @0xD

Binary

ND-mask

Ternary

11110000 00001111 00000000 10101010

00000001 11110000 11111111 01010101

1111000* ****1111 ******** 1*1*1*1*

[mc]square uses 3-valued logic for modelling the microcontroller memory in order to handle nondeterministic behaviour. Every single bit of the simulator memory is valued in the set {0, 1, ND}. This means that it can take the values false (0), true (1), and nondeterministic (ND), that, is perhaps true or perhaps false. For the actual implementation, an additional nondeterministic mask (ND-mask) accompanies every memory location to realise this concept (see Table 1). From state-space view, the 3-valued memory representation introduces so-called lazy states as introduced by Noll and Schlich (2008). A lazy state combines explicit and symbolic parts in order to provide a more compact representation of the state space, but [mc]square still uses explicit-state model checking algorithms. Any state that includes nondeterministic memory locations is a lazy state, and thus, a single lazy state represents a set of explicit states. The following three abstraction techniques are based on this representation.


191

• Delayed non-determinism (DND) was first described by Noll and Schlich (2008). It is an approach to tackle the state-explosion problem by delaying the instantiation of nondeterministic values with their values as long as possible. Successor states are not necessarily instantiated when they are created, but only in case the values are needed to prove a given system property or for subsequent computations. A subsequent computation step, for instance, is any conditional branch instruction, which requires the actual value of a nondeterministic memory location in order to resolve the jump condition or to determine the target location the branch leads to. DND is restricted to data-transfer operations. Whenever data is transferred, the ND-mask is copied from the source to the target location. • Delayed non-determinism with look ahead (DND1A) as in Reinbacher et al. (2009) is an extension of DND, which preserves the advantages of DND. It is applied to logic instructions of the microcontroller. The approach of DND1A centres around the coherence among the Boolean operators ∧, ∨, and ¬ with respect to the 3-valued memory model of the Intel MCS-51 simulator. • Non-deterministic programme status word (NDPSW) is a technique that executes a single instruction only partially without evaluating all the details defined by its semantics. In contrast to DND and DND1A, this abstraction technique can also be applied to arithmetic operations. The generation of successor states is avoided by generating a coarse over-approximation. NDPSW works as follows. Consider the instruction ADDC [A, R0] (add register R0 to accumulator with carry). The operation affects the following registers and flags: the accumulator A, containing the first operand and serving as destination register after the calculation the working register R0, and the carry flag C, the overflow flag OV, the auxiliary carry flag AC, and the parity flag P. All flags in the programme status word are implicitly set when A is written. Whenever one of the memory locations read by the instruction has a nondeterministic value, the abstraction technique sets the accumulator A and the flags C, OV, AC, and P to nondeterministic, too. The second operand R0 is not modified since it is not written by the operation. Setting registers to nondeterministic rather than generating the actual value combinations effectively avoids the creation of successor states. Nevertheless, NDPSW introduces behaviour and programme paths which may not exist in reality. In addition to these purely dynamic abstraction techniques, [mc]square also uses adaptations of several general-purpose techniques. These are: • Dead variable reduction (DVR) aims at finding dead memory locations, that is, memory locations containing values that are never read before they are overwritten. The results obtained through DVR are used by the model checker to reset dead memory locations. In consequence, states that only differ in dead memory locations are merged during model checking. DVR is based on Yorav and Grumberg (2004), details on the required adaptations for binary code are given by Schlich et al. (2007). • Path reduction (PR) is an abstraction technique used to compress single-successor chains, which are paths of states that only have single successors, into single states (see Yorav and Grumberg, 2004; Schlich et al., 2007). Although, PR significantly reduces state spaces, a minor drawback is that the validity of the next operator


192

is lost due to the compression of successor chains into single states. This means that PR only establishes a stuttering equivalence with respect to the original transition system. Both DVR and PR use information from a static analysis framework. The performed analyses include standard data-flow analyses such as reaching definitions analysis and live variables analysis, as well as hardware-specific analyses for stack usage (SA), the status of the interrupt flag (IFA), and the usage of register banks (RBA). Details on these analyses are given by Schlich (2008), Brauer et al. (2009) and Reinbacher et al. (2009).

3.3 Workflow and environment modelling The workflow, as applied to the present case study, is shown in Figure 2. Figure 2

The model checking workflow of [mc]square

0101010001001000 . 0100111101001101 0100000101010011

.

.S.3

.S 1

Hex and debug files assembly source

Environment automaton U

.

.

.yes

.

System specification φ

[mc]square . model checker . .

.Witness

(1) AG event U abort .(2) EF event > 100 (3) AF event = 20

.S 2

System model M

.M |= φ?

.no

.Counterexample

We use [mc]square with three inputs, namely the binary code (e.g., in form of an IntelHex file), a model of the microcontroller environment, and the system specification. For modelling the environment, we made use of [mc]square’s user defined environment (UDE) feature introduced in Schlich et al. (2008). A communicating finite state machine as defined in Brand and Zafiropulo (1983) is used to define input values as well as interrupt and value transitions. In the case study, an UDE automaton is used to define the sequence of values read from the serial interface as dictated by the communication protocol (see Section 4.1.4). By modelling the environment, we tackle fairness issues induced by reentrant interrupts. The target application allows for reentrant interrupts, and therefore, the model checker always finds a counterexample that shows a stack collision, which is caused by storing too many return addresses of interrupt service routines on the stack. This behaviour is called unfair because it is very unlikely when executing the code on the actual target hardware. The serial interrupt, for example, only occurs


193

at multiple time instances of the selected serial baud rate, and thus, is very unlikely to exhibit this behaviour. By modelling the environment, we rule out this unfair behaviour.

4 Case study In the following, we introduce a real-life embedded systems application and show how we model checked it with [mc]square. A formal specification is extracted out of a behavioural, textual system description provided by our industry partner. The model checking process is supported by the abstraction and static analysis techniques for the Intel MCS-51 target, as described in Section 3. All numbers were generated on a SUN Fire x4600 M2 server equipped with 8 dual-core AMD OpteronTM 8220 processors (2.80 GHz) and 256 GB of RAM, running a 64 bit Windows Server Enterprise Edition and a JavaTM server virtual machine version 1.6.0 (with settings -server -Xmx200G -Xss120M). Revision 4338 of [mc]square was used.

4.1 The target application Textile knitting machines produce various types of knitted fabrics of varying degrees of complexity. Modern knitting machines usually contain highly complex electronics controlling the needles and the yarn. The target application of our case study is the software for a knitting machine monitoring device. It targets the Intel MCS-51 microcontroller, uses several on-chip peripheral modules and interrupt sources, interacts with its environment, and consists of about 700 lines of tested C code. The deployed application consists of 1,400 lines of disassembled binary code. The hardware building blocks

Figure 3

Knitting . machine

.

.Power supply

Microcontroller .

.

87C51 . 12 MHz

.Watchdog

.

Serial interface

.

Input module

.Knitting machine monitoring device

Host . . application

.LED module

4.1.1 Hardware structure The monitoring device is composed of the hardware modules depicted in Figure 3. The input module connects the monitoring device with the knitting machine using eight dedicated input lines. The Intel MCS-51 microcontroller executes the software that is subject to verification. The serial interface provides a physical link to the host application trough a RS232 interface. Moreover, a separate watchdog module triggers the reset input of the microcontroller whenever the watchdog timer overflows. A power supply module provides an inverse-polarity protection and a filtered and stabilised power supply. In addition, this module contains a quartz-controlled clock generation. A LED module is used to indicate liveness of the application.


194

4.1.2 Software structure The application is designed according to the foreground/background design pattern (see, e.g., Pont, 2001). The software design is shown in Figure 4. The programme consists of an infinite loop that calls individual functions to perform the desired operations (the background part). Asynchronous events (the foreground part) are handled through interrupt service routines (ISRs). Thus, all time-critical operations are performed by ISRs in order to ensure that they are served within the given timing constraints. Timing correctness is met by interrupting the background part of the software at predefined points in time, for instance, when a timer expires or a character is received over the serial interface. Figure 4

The foreground/background design pattern void main (void){ InitBoard(); SendTxt(STX,'R',ETX); SetTime(); while(1){

.Foreground

.

.Extern ISR()

Watchdog();

. Loop

Liveness();

. UpdateInputs();

.

.Timer ISR()

EvaluateRPM(); RSM(readCmd()); }

.

}

..

.Serial ISR()

.Circular buffers

From a conceptional point of view, the background part can be divided into six building blocks. Watchdog() periodically resets the external watchdog module. Liveness() toggles the liveness LED and performs some housekeeping. UpdateInputs() reads eight digital input lines connected to the knitting machine. EvaluateRPM() calculates the variable Rev, representing the current RPM of the knitting machine. RSM() represents the receiver state machine and implements the serial communication protocol. It writes to . the transmit buffer whenever the application replies to host commands. readCmd() reads the receive buffer to fetch commands sent by the host application. The foreground part is composed of three ISRs. Extern ISR() counts pulses from external rotary encoders. The pulse count is internally mapped to a 16-bit wide unsigned data type. Timer ISR() uses the Timer 0 peripheral module of the microcontroller to provide a system tick and four additional software timers. Serial ISR() initialises the serial communication device of the microcontroller and uses dedicated circular buffers to manage receive and transmit queues.

4.1.3 Serial receive and transmit buffers A circular buffer is a data structure that uses a single, fixed-size buffer as if it is connected end-to-end. The case study uses two dedicated circular buffers, one for receiving and one for sending characters. According to Table 2, a single CircBuffer element consists of ten bytes. The column Formula name in Table 2 refers to the actual expressions that are used in the CTL specifications of the application – introduced later in this section.


195

Table 2 Circular buffer elements and their size Element

Type

#Bytes

Formula name

ReadIndex WriteIndex MaskIndex Data Buffer Sum

Word Word Word Char

2 2 2 4 10

Tx|RxBuf {0, 1} Tx|RxBuf {2, 3} Tx|RxBuf {4, 5} Tx|RxBuf {6...9} Tx|RxBuf {0...9}

4.1.4 The communication protocol The communication between the knitting machine monitoring device and the host application follows a master-slave protocol where the host application operates as the master. The communication is initiated by the host application, with one exception: the knitting machine monitoring device sends a status message to the master after power-up. The communication protocol does not include any data integrity checks such as checksums. Table 3 states the specified commands and the expected replies. RPM1 denotes the first byte of the Rev variable, RPM2 refers to the second byte, respectively. Table 3 The master-slave communication protocol Master 1 2 3 4 5

Slave

Cmd

#Bytes

Response

#Bytes

$D# $R# $Z# $E# $V#

3 3 3 3 3

$D[RPM1 RPM2 ]# $R# $Z[CNT1 CNT2 ]# $E[INP]#

5 3 5 4 6

$V[VER1 VER2 VER3 ]#

Command Cmd1 returns the current revolutions per minute, Cmd2 sets the device into sleep mode (system reset by watchdog), Cmd3 returns the current pulse counter value, Cmd4 returns the current input representation, and Cmd5 returns the software’s version number as a string.

4.2 Implementation For the case study, we used the source code as provided by our industry partner. For the sake of clarity, however, we introduced a dedicated byte-sized variable termed mark that can be included in the specification. Note that all CTL formulas could also be expressed by involving the program counter in the property, such as AG(EF PC = 0×C0FFEE). Hence, [mc]square is able to check the given unmodified code. The case study was compiled and linked with the Keil C51 Compiler C8.01. [mc]square is able to additionally parse the generated debug files permitting the use of C variables in the specification. This provides a mapping between the C code and the assembly code, which is especially useful when studying counterexamples, as they can be traced both in the C and the assembly code.

196


4.2.1 Notation An initial behavioural description of the application is part of the project. As the given specification is informal and in textual form, we identified relevant properties first and translated them to CTL. The specification is divided into four parts: initialisation, reachability, the receiver state machine, and the communication protocol. Results obtained from [mc]square are given in the form of: Results template Prop. #

M

State space size Stored

Created

Time (mm:ss) Revisits

Initialisation Reachability Receiver state machine and circular buffers Communication protocol with UDE

The item states stored comprises the states that are stored in main memory. It is compiled out of the total states created shortened by the number of state revisits. The item states created refers to the overall states that are created by [mc]square. State revisits are caused by states that are already stored in the state space. The column time refers to the time needed to complete the model checking run. The item M states whether the postulated property was verified by [mc]square or not (M |= φ?). In this paper, we present only a subset of the derived specification. In the following, CTL properties are given in the form of: Property #Explanation #Explanation Textual representation As stated in the textual specification of the application. CTL representation ([mc]square notation) . . The derived CTL formula, given in the notation used in [mc]square. Comment Additional information and explanation of the formula and meaning of the included variable names.

4.2.2 Initialisation The following CTL properties cover the initialisation of the essential data structures and investigate the behaviour of the startup code.

4.2.2.1 Specification and description In particular, properties Init#1 to Init#4 cover: are the static variables and circular buffers correctly initialised? Does the monitoring device send the startup sequence ‘$’,‘R’,‘#’ to the host application after power-up?


197

Property #Init1 #Init1 Textual representation The variable Rev is initialised to 0xffff. CTL representation ([mc]square notation) AG(startUpCodeFinished=0 & Rev=0x0000 ⇒ A Rev=0x0000 U Rev=0xff00) AG(startUpCodeFinished=0 & Rev=0xff00 ⇒ A Rev=0xff00 U & . . Rev=0xffff) & EF Rev=0x0000 & EF Rev=0xff00 Comment If Rev is 0x0000, it remains 0x0000 until it becomes 0xff00. If Rev is 0xff00, it remains 0xff00 until it becomes 0xffff. There is a path where Rev is 0x0000 and one where Rev is 0xff00. The initialisation must be completed before the startup code is left.

Property #Init2+3 #Init2+3 Textual representation Initialisation of the receive|transmit circular buffers. CTL representation ([mc]square notation) (2) AF (RxBuf 0=0x0 & RxBuf 1=0x0 & RxBuf 2=0x0 & RxBuf 3=0x0 & RxBuf 4=0x0 & RxBuf 5=0x3 & RxBuf 6=0x0 & RxBuf 7=0x0 & RxBuf 8=0x0 & . . RxBuf 9=0x0 & startUpCodeFinished=1) (3) AF (RxBuf 0=0x0 & RxBuf 1=0x0 & RxBuf 2=0x0 & RxBuf 3=0x0 & RxBuf 4=0x0 & RxBuf 5=0x3 & RxBuf 6=0x0 & RxBuf 7=0x0 & RxBuf 8=0x0 & RxBuf 9=0x0 & startUpCodeFinished=1) Comment On all paths in the startup-code there is a state where all bytes of Rx|TxBuf i where i = {0 . . . 9} are initialised to 0x0, except Rx|TxBuf 5 which is initialised to 0x3, since it acts as the circular buffers mask (see Table 2).

Property #Init4 #Init4 Textual representation The monitoring device sends ‘$’,‘R’,‘#’ to the host after power-up. . . CTL representation ([mc]square notation) EF (TxBuf 6=‘$’ & TxBuf 7=‘R’ & TxBuf 8=‘#’ & TxBuf 9=‘0’) Comment There is a path where the transmit circular buffer is filled with ‘$’,‘R’,‘#’.

4.2.2.2 Results Note that property #Init1 reveals one of the strengths of our approach. As the verification process is based on machine instructions, it is possible to verify the exact initialisation sequence of the 16-bit variable Rev. Property #Init1 requires the high-byte to be initialised prior to the low-byte as usual on little-endian architectures. Model checkers for C are not able to verify properties of this granularity due to missing details of the target platform. For example, the statement AF Rev = 0xffff & startUpCodeFinished = 1 is a property suited for C code model checkers, but it only claims that Rev will reach the value 0xffff during startup, excluding details on how the initialisation is accomplished. For example, it is possible that Rev is first set to 0xfc0f and later on set to 0xffff. As a result, AF Rev= 0xffff & startUpCodeFinished = 1 will

198


evaluate to true since the initialisation sequence is not sufficiently specified. In this case, [mc]square will reveal the erroneous behaviour. The model checking run evaluated properties #Init1 to #Init4 to true, thus, with respect to the postulated CTL properties, the variable Rev and the circular buffers are initialised correctly. Results for initialisation properties Prop. # Init1 Init2 Init3 Init4

State space size

M

X X X X

Time (mm:ss)

Stored

Created

Revisits

2,107,785 230 230 1,519

22,627,672 634 634 344

721,167 0 0 14

05:11 00:01 00:01 00:01

4.2.3 Reachability We use reachability checks to verify that crucial parts of the target application are eventually executed. The formulae are rather simple as they only make use of the EF operator. They are a practical indicator that more complex CTL properties investigated later on are not trivially satisfied.

4.2.3.1 Specification and description Typical properties we checked, amongst #Reach1 to #Reach5, are: Do crucial system variables ever take on certain checkpoint-values? Does the monitoring device ever establish a connection with the host system? Does the monitoring device ever send the commands specified in Table 3? Property #Reach1+2+3+4+5 #Reach1+2+3+4+5 Textual representation It is always possible to reach the (1)“sleep”, (2)“send version”, (3)“send inputs”, (4)“send pulse count”, and (5)“send RPM” state. CTL representation ([mc]square notation) (1) AG(EF mark=M SLEEP) . . AG(EF mark=M SENDVERSION) (2) (3) AG(EF mark=M SENDINPUTS) (4) AG(EF mark=M SENDPLSCNT) (5) AG(EF mark=M SENDRPM) Comment On every path the application finally reaches the (1) “sleep”, the (2) “send version”, the (3) “send inputs”, the (4) “send pulse count”, and the (5) “send RPM” state.

4.2.3.2 Results During early stages of the software development process, reachability checks are useful to rule out conceptual errors and specification misunderstandings. They are easy to establish by users and specification errors are unlikely to occur. Property #Reach1 is valid, thus, it is always possible to reach the ‘sleep’ state of the application. Properties #Reach2+3+4+5 were disproved by [mc]square. The counter example shows that it is not possible to reach several states in case the host application previously sent the command ‘$’,‘R’,‘#’ (go to the sleep state) to the microcontroller.


199

The sleep state is implemented as a non-terminating loop and can only be left through an external reset by the watchdog module. Since we do not consider the watchdog module in the verification process, the endless loop cannot be left. Hence, these properties are disproved by [mc]square. Results for reachability properties Prop. # Reach1 Reach2 Reach3 Reach4 Reach5

M

X × × × ×

State space size

Time (mm:ss)

Stored

Created

Revisits

2,325,333 20,459 20,456 23,893 27,328

20,099,919 150,896 150,890 173,084 195,275

525,454 3,247 3,247 3,676 4,106

04:30 00:03 00:02 00:02 00:03

4.2.4 Receiver state machine and circular buffer The receiver state machine is responsible for handling communication requests sent by the host and the properties specify its internal state transitions.

4.2.4.1 Specification and description Representative properties are #RSM1 to #RSM4, covering: Does the receiver state machine follow the specified transitions? Do the state variables of the circular buffers remain within range? Do the circular buffer pointers remain within range? Property #RSM1 #RSM1 Textual representation The receiver state machine may only reside in states 0, 1, or 2. . . CTL representation ([mc]square notation) AG(CState=0 | CState=1 | CState=2) Comment On all paths the actual state of the state machine is either 0, 1, or 2.

Property #RSM2 #RSM2 Textual representation Changes in the receiver state machine can only follow the patterns 0 → 1 → 2 → 0 or 0 → 1 → 0. All other transitions are invalid. CTL representation ([mc]square notation) AG(CState=0 ⇒ A CState=0 U CState=1 | CState=0) & AG(CState=1 ⇒ A CState=1 U CState=0 . | CState=2 | CState=1) & . AG(CState=2 ⇒ A CState=2 U CState=0 | CState=2) & AF CState=0 Comment If CState = 0, CState remains 0 until it changes to 1, if CState = 1 then CState remains 1 until it changes to 0 or 1, if CState = 2, CState remains 2 until it changes to 0. There is a path where CState initially becomes 0 and the receiver state machine may always remain in its current state.

200


4.2.4.2 Results The circular buffer implementation is covered by properties #RSM3+4 and #Init1+2. Whereas #Init1 and #Init2 refer to the correct initialisation of the circular buffers, properties #RSM3+4 make propositions about the ranges of the read and write pointers. These properties are successfully verified on the analysed program code. Thus, with respect to the CTL properties, the circular buffer implementation is flawless. Property #RSM3+4 #RSM3+4 Textual representation The serial receive|transmit circular buffers read and write pointer may never exceed their bounds. CTL representation ([mc]square notation) . . AG(RxBuf 10 ×$],?,#}) is received. The root cause of the error lies in source code line 19. The application waits for either a start byte (STX = ‘$’) or a stop byte (ETX = ‘#’), thus, a repeated even number of start bytes resets the variable CState to 0 – over and over again – in source code line 20. In order to correct the receiver state machine implementation, the existing caseg clause is adapted as shown in lines 18–28 of Figure 5 (right). Whenever the host now sends sequences of form {[2n]n>0 ×$],?,#}, the variable CState is not erroneously reset


203

to 0, but set to 1, forcing the state machine to properly wait for the following command byte.

4.2.6 Static stack analysis We performed a static stack analysis with [mc]square in order to detect stack corruptions. For the case study application, no stack corruptions were detected, and hence, the stack is safe. This means, all those bytes pushed onto the stack are again popped from the stack in the correct order. The upper stack bound of the application is 81 and the lower stack bound evaluated to 73. Hence, the maximum stack size is eight. The results obtained from static analysis were cross-proved using model checking. Here, the maximum stack size evaluated to eight as well.

4.3 Performance Table 4 shows the performance of [mc]square using the same hardware configuration. Herein, we assess the contribution of the individual abstraction techniques to state-space reductions. Table 4 Case study results for plain state space building State space size

Run # 1 2 3 4 5 6

Stored

Created

2,311,895 2,316,882 15,271,404 15,378,086 162,471,807 >391,789,933

20,015,571 20,072,077 15,796,063 15,902,718 172,839,103 >6,981,334,953

Time (hh:mm:ss)

Disabled abstractions

00:04:27 00:04:30 00:05:01 00:05:41 01:33:10 >48:00:00

none DVR PR PR, DVR PR, DVR, IFA, RBA, SA DNDlA, NDPSW

According to the outcome of the state space comparison in Table 4, we derive the following conclusions. For the case study, it is not feasible to build the state space without any abstraction techniques. This is not surprising since the application heavily interacts with its environment. Model checking the case study with only DND enabled (run 6) was cancelled after running for two days. Thus, DND does not provide enough abstraction to build the state space within reasonable time- and resource-constraints. The state space could only be built when enabling DND1A and NDPSW (runs 1–5). These options drastically reduced state spaces and run-time. Enabling static analysis helps alleviating the state-explosion problem. In particular, PR greatly reduces state spaces (runs 1–2). For the case study, DVR only leads to a minor reduction of system states. This can be explained by indirect accesses to the circular buffers in ISRs, which prohibits resetting more memory locations in order to preserve a valid over-approximation.

5 Conclusions In this paper, we have described our experiences with the formal verification of a monitoring device used in an industrial knitting machine. In particular, using model checking revealed a serious bug in the implementation of the communication protocol.

204


As expected, the generated counterexample provided useful information on how to fix this defect. During this case study, however, we had to overcome several obstacles. First of all, we had to develop multiple abstraction techniques in order to alleviate the state-explosion problem, which reduced the runtime of the verification process from more than two days down to approximately five minutes. From this experience, we believe that further combining static and dynamic state-space reduction techniques is a promising approach for tackling the state-explosion. Moreover, some specifications could either not be expressed in CTL or only using lengthy and complicated formulae. Using LTL would help for some specifications, but then, other specifications could not be expressed. It turned out that all specifications could easily be expressed using nondeterministic Büchi automata (NBA). The drawback is that model checking NBA is computationally more expensive than model checking CTL.

Acknowledgements The work of Thomas Reinbacher and Martin Horauer has been supported within the FHplus project Design Methods for Embedded Control Systems (DECS) managed by the Austrian Research Agency FFG under Grant 811414.

References Baier, C. and Katoen, J-P. (2008) Principles of Model Checking, The MIT Press, ISBN 026202649X. Bengtsson, J., Larsen, K., Larsson, F., Pettersson, P. and Yi, W. (1996) ‘Uppaa – a tool suite for automatic verification of real-time systems’, in DIMACS/SYCON Workshop on Hybrid Systems III: Verification and Control, Springer, pp.232–243. Bouissou, O., Conquet, E., Cousot, P., Cousot, R., Feret, J., Ghorbal, K., Goubault, E., Lesens, D., Mauborgne, L., Miné, A., Putot, S., Rival, X. and Turin, M. (2009) ‘Space software validation using abstract interpretation’, in Data Systems in Aerospace (DASIA 2009), Vol. SP-669, ESA, pp.1–7. Brand, D. and Zafiropulo, P. (1983) ‘On communicating finite-state machines’, J. ACM , Vol. 30, No. 2, pp.323–342. Brauer, J., Schlich, B., Reinbacher, T. and Kowalewski, S. (2009) ‘Stack bounds analysis for microcontroller assembly code’, in 4th Workshop on Embedded Systems Security (WESS 2009), Grenoble, France, pp.1–9. Clarke, E. (2008) ‘The birth of model checking’, in 25 Years of Model Checking, Vol. 5000 of LNCS, pp.1–26. Clarke, E.M., Grumberg, O. and Peled, D.A. (1999) Model Checking, The MIT Press, ISBN 0262032708. Dong, Y., Du, X., Ramakrishna, Y.S., Ramakrishnan, C.R., Ramakrishnan, I.V., Smolka, S.A., Sokolsky, O., Stark, E.W. and Warren, D.S. (1999) ‘Fighting livelock in the i-protocol: a comparative study of verification tools’, in Tools and Algorithms for the Construction and Analysis of Systems, Vol. 1579 of LNCS, Springer, pp.74–88. Havelund, K., Larsen, K.G. and Skou, A. (1999) ‘Formal verification of a power controller using the real-time model checker UPPAAL’, in Formal Methods for Real-Time and Probabilistic Systems (ARTS 1999), Vol. 1601 of LNCS, Springer, pp.277–298.


205

Havelund, K., Lowry, M.Park, S.J., Pecheur, C., Penix, J., Visser, W. and White, J.L. (2000) ‘Formal analysis of the remote agent before and after flight’, Technical report, NASA Ames Research Center. Kästner, D., Wilhelm, R., Heckmann, R., Schlickling, M., Pister, M., Jersak, M., Richter, K. and Ferdinand, C. (2008) ‘Timing validation of automotive software’, in Leveraging Applications of Formal Methods, Verification and Validation, Vol. 17 of Communications in Computer and Information Science, Springer, pp.93–107. Lindahl, M., Pettersson, P. and Yi, W. (1998) ‘Formal design and analysis of a gear controller’, in ‘Tools and Algorithms for the Construction and Analysis of Systems (TACAS 1998), Vol. 1384 of LNCS, Springer, pp.281–297. Mercer, E. and Jones, M. (2005) ‘Model checking machine code with the GNU debugger’, in SPIN Workshop on Model Checking Software, Vol. 3639 of LNCS. Noll, T. and Schlich, B. (2008) ‘Delayed nondeterminism in model checking embedded systems assembly code’, in Haifa Verification Conf. (HVC 2007), Vol. 4899 of LNCS, pp.185–201. Pont, M.J. (2001) Patterns for Time-triggered Embedded Systems: Building Reliable Applications with the 8051 Family of Microcontrollers, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA. Reinbacher, T., Brauer, J., Horauer, M. and Schlich, B. (2009) ‘Refining assembly code static analysis for the Intel MCS-51 microcontroller, in Symposium on Industrial Embedded Systems (SIES 2009), pp.161–170. Reinbacher, T., Horauer, M. and Schlich, B. (2009) ‘Using 3-valued memory representation for state space reduction in embedded assembly code model checking’, in Design and Diagnostics of Electronic Circuits and Systems (DDECS 2009), pp.114–119. Reinbacher, T., Kramer, M., Horauer, M. and Schlich, B. (2008a) ‘Challenges in embedded model checking – a simulator for the [mc]square model checker’, in Symposium on Industrial Embedded Systems (SIES 2008), pp.245–248. Reinbacher, T., Kramer, M., Horauer, M. and Schlich, B. (2008b) ‘Motivating model checking for embedded systems software’, in Mechatronic and Embedded Systems and Applications (MESA 2008), pp.546–551. Ribeiro, O., Fernandes, J. and Pinto, L. (2005) ‘Model checking embedded systems with PROMELA, in Engineering of Computer-Based Systems (ECBS ’05), pp.378–385. Scheuer, F. (2007) ‘Extending the model checker [mc]square to handle the Infineon XC167 microcontroller’, Diploma thesis, RWTH Aachen University, Department of Computer Science 11, (in German). Schlich, B. (2008) ‘Model checking of software for microcontrollers’, Dissertation, RWTH Aachen University, Aachen, Germany, available at http://sunsite.informatik.rwth-aachen.de/Publications/AIB/2008/2008-14.pdf. Schlich, B., Brauer, J., Wernerus, J. and Kowalewski, S. (2009) ‘Direct model checking of PLC programs in IL’, in Dependable Control of Discrete Systems (DCDS’09), To appear. Schlich, B., G¨ uckel, D. and Kowalewski, S. (2008) ‘Modeling the environment of microcontrollers to tackle the state-explosion problem in model checking’, in Formal Methods for Automation and Safety in Railway and Automotive Systems (FORMS/FORMAT 2008), L’Harmattan, Budapest, Hungary, pp.27–34. Schlich, B., L¨ oll, J. and Kowalewski, S. (2007) ‘Application of static analyses for state space reduction to microcontroller assembly code’, in Formal Methods for Industrial Critical Systems (FMICS 2007), Vol. 4916 of LNCS, Springer, pp.21–37. Schlich, B., Rohrbach, M., Weber, M. and Kowalewski, S. (2006) ‘Model checking software for microcontrollers’, Technical report, Technischer Bericht AIB-2006-11, RWTH Aachen. Yorav, K. and Grumberg, O. (2004) ‘Static analysis for state-space reductions preserving temporal logics’, Formal Methods in System Design, Vol. 25, pp.67–96.