a case study at BOSCH - Springer Link

0 downloads 0 Views 481KB Size Report
Jan 24, 2012 - Automotive behavioral requirements expressed in a specification pattern system: a case study at BOSCH. Amalinda Post • Igor Menzel • Jochen ...
Requirements Eng (2012) 17:19–33 DOI 10.1007/s00766-011-0145-9

REFSQ 2011

Automotive behavioral requirements expressed in a specification pattern system: a case study at BOSCH Amalinda Post • Igor Menzel • Jochen Hoenicke Andreas Podelski



Received: 16 August 2011 / Accepted: 12 December 2011 / Published online: 24 January 2012  Springer-Verlag London Limited 2012

Abstract To allow an automatic formal analysis of requirements, the requirements have to be formalized first. However, logical formalisms are seldom accessible to stakeholders in the automotive context. Konrad and Cheng proposed a specification pattern system (SPS) represented in a restricted English grammar that can be automatically translated to logics, but looks like natural language. In this paper, we investigate whether this SPS can be applied to automotive requirements of BOSCH, in the sense that it is expressive enough to specify automotive behavioral requirements of BOSCH. We did a case study over 289 informal behavioral requirements taken from automotive BOSCH projects. We evaluated whether these requirements could be formulated in the SPS and whether the SPS has to be adapted to the automotive context. The case study strongly indicates that the SPS, extended with 3 further patterns, is suited to specify automotive behavioral requirements at BOSCH. Keywords Real-time

Automotive  Requirements  Formalization 

A. Post (&)  I. Menzel Robert Bosch GmbH, Corporate Research, Stuttgart, Germany e-mail: [email protected] I. Menzel e-mail: [email protected] J. Hoenicke  A. Podelski Department of Computer Science, University of Freiburg, Freiburg, Germany e-mail: [email protected] A. Podelski e-mail: [email protected]

1 Introduction In this work, we investigate whether the restricted English grammar provided by Konrad and Cheng [1] suffices to express behavioral requirements taken from automotive projects at BOSCH. The grammar looks like natural language, however, it allows an automatic translation into linear time logic (LTL) [2], computational tree logic (CTL) [2], graphical interval logic (GIL) [3], metric temporal logic (MTL) [4], timed computational tree logic (TCTL) [4], and real-time graphical interval logic (RTGIL) [5]. We are only interested in requirements specifying the behavior of the system, i.e., we do not consider any other kind of requirements. Thus, in the following, every time we speak of ‘‘requirements’’ we mean in fact behavioral requirements [6], i.e., requirements specifying the behavior of the system. We consider behavior without exact timing bounds (e.g., ‘‘If the system is in diagnostic mode then previously the diagnostic request DiagStart held.’’) and behavior with exact timing bounds (e.g., ‘‘If the diagnostic request IRTest appears then the infrared lamps are turned off after at most 6 s’’). The case study is motivated by the automotive domain. In this context, sets of requirements often comprise several hundred pages. At BOSCH, the majority of these requirements are behavioral requirements. The requirements are mainly written in natural language. There are a lot of involved stakeholders, many of them spread over different companies [7]. Furthermore, requirements specifications are manually checked for errors, e.g., by peer reviews [8]. The problem addressed in this work is that such a manual check is a considerable effort since requirements affect each other and cannot be analyzed in isolation [9]. As it is labor-intensive to check every requirement with every other requirement, automatic checks are desirable

123

20

already for small sets of requirements [10–12]. However, to allow automatic checks, the requirements need to be available in a formal language, such as, e.g., a logic like LTL. In addition to that, the new functional safety standard for Automotive Electronic Systems ISO26262 (currently under development) states that for safety critical systems, at least a semi-formal specification of safety requirements is highly recommended [13]. Thus, in the automotive domain, a need arises for methods developing semi-formal or formal requirements. Unfortunately, such formalizations are rarely accessible to the stakeholders who need to read them [1, 14]. Thus, before working with a formal specification, all the stakeholders would need to be trained in the chosen logic, even beyond company boundaries. In practice, this is clearly unrealistic. Another possibility is to develop both a formal and an informal requirements document. But this is double work, and it will be difficult to hold both documents consistently over the development time of 2–5 years. Thus, the idea is to use a language that retains the mathematical rigor but uses a vocabulary and syntax very close to natural English language. Such a language is provided by Konrad and Cheng [1]. They propose a restricted English grammar that represents a specification pattern system (SPS), such that every pattern looks like natural language but can be translated into a logical formula in LTL, CTL, TCTL, RTGIL or MTL. The drawback of this language is that it covers only a subset of the statements possible to express in the particular logic. For example, the following requirement cannot be expressed in the SPS as the SPS provides no pattern that expresses such a behavior: ‘‘For a codeword with 5 digits, the time between entering the first digit and the last one is \5 s’’. However, the SPS does not claim to be complete. Thus, the question is not whether all possible requirements can be expressed in this SPS but: Does in practice the SPS suffice to express automotive requirements? If this would be true, then automotive requirements could be expressed in SPS, i.e., they would be still readable for all stakeholders (as they look like natural language) but could be automatically checked by a computer (as they can be automatically translated into logical formulas). To investigate that question, we did a case study over 289 requirements taken from five automotive projects of BOSCH. This way, we cannot generalize our results for the whole automotive domain, but we can generalize to BOSCH. We believe that the results obtained with specifications of BOSCH also provide an insight in the automotive domain. BOSCH is one of the biggest suppliers in the automotive domain. The company business units span several sub domains, e.g., engine control, car multimedia, driving assistance, body electronics, brake control systems up to automotive after market solutions.

123

Requirements Eng (2012) 17:19–33

The requirements were given in informal prose. For the case study, we checked for every requirement in this sample whether it could be reformulated in the SPS without loss of meaning. 1.1 Remark This paper is an extended version of [15]. It provides a more detailed description of the case study planning and analysis. In addition, it extends the results with respect to the usage frequency of the scopes and patterns of the SPS in Sects. 5.1 and 5.2. The case study strengthens the belief of Dwyer et al. [16] that some very few patterns and scopes suffice to express the majority of requirements. For example, in our case study, about 50% of the requirements were expressed with only 2 patterns. Further, in Sect. 5.3, based on our experiences at BOSCH, we elaborate the applicability of the SPS in the sense of how many requirements are typically behavioral requirements, and about the implications on the coherence of requirements documents when using different notations.

2 Related work Various tabular notations aim to provide requirements that are both accessible and suitable for formal analysis. For example, Heitmeyer et al. [11] have built a variety of tools for checking consistency, completeness, and safety properties of requirements expressed in the tabular SCR notation. In contrast to the tabular notation, formalisms like B or Z [17] aim to express requirements in a mathematical way. For example, the formalism Z defines rigid notations for logical operations, quantifiers, sets, and functions. Whereas Z is strictly designed as a specification language, the goal of B is to use mathematical proofs to verify consistency between refinement levels and to enable an automatic transformation of requirements into some executable code. However, neither B nor Z offers a natural language representation. Some research, such as the Attempto Controlled English project [18] and the work of Han [19], attempts to construct formal specifications from natural language requirements. However, automatic interpretation of natural language is still error-prone, as natural language is often vague and the information is often implicitly stated. The use of natural language in the work described here is much less ambitious. This work investigates whether the restricted English grammar proposed by Konrad and Cheng [1] can be applied to the automotive context. The grammar depicted in Table 1 represents a specification pattern system (SPS) [1] with 16 non-recursive patterns. Every pattern can be mapped to logical statements in, e.g., the logics MTL,

Requirements Eng (2012) 17:19–33

21

Table 1 Restricted English grammar given by Konrad and Cheng in [1] Requirement

::=

Scope, Specification

Scope

::=

Globally | Before R | After Q | Between Q and R | After Q until R

Specification

::=

Qualitative type | Real-time type

Qualitative type

::=

Absence | Universality | Existence | Bounded existence | Precedence | Response | Precedence chain 1-2 | Precedence chain 2-1 | Response chain 2-1 | Response chain 1-2 | Constrained chain

Absence

::=

it is never the case that P holds

Universality

::=

it is always the case that P holds

Existence

::=

P eventually holds

Bounded existence

::=

transitions to states in which P holds occur at most twice

Precedence

::=

it is always the case that if P holds, then S previously held

Precedence chain 1-2

::=

it is always the case that if S holds and is succeeded by T, then P previously held

Precedence chain 2-1

::=

it is always the case that if P holds then S previously held and was preceded by T

Response

::=

it is always the case that if P holds then S eventually holds

Response chain 1-2

::=

it is always the case that if P holds then S eventually holds and is succeeded by T

Response chain 2-1

::=

it is always the case that if S holds and is succeeded by T, then P eventually holds after T

Constrained chain

::=

it is always the case that if P holds then S eventually holds and is succeeded by T, where Z does not hold between S and T

Real-time type

::=

Min duration | Max duration | Periodic category | Bounded response | Bounded invariance

Minimum duration

::=

it is always the case that once P becomes satisfied, it holds for at least c time unit(s)

Maximum duration

::=

it is always the case that once P becomes satisfied, it holds for less than c time unit(s)

Periodic category

::=

it is always the case that P holds at least every c time unit(s)

Bounded response

::=

it is always the case that if P holds, then S holds after at most c time unit(s)

Bounded invariance

::=

it is always the case that if P holds, then S holds for at least c time unit(s)

TCTL, and RTGIL. The patterns without exact timing bounds were originally proposed by Dwyer et al. in [16]. Konrad and Cheng extend these patterns in their SPS. The patterns consist of non-literal terminals (given in a italic font) and literal terminals. For example, the Minimum duration pattern consists of ‘‘it is always the case that once P becomes satisfied, it holds for at least c time units’’, with P, c as non-literal terminals and the rest as literal terminals. The non-literal terminal P denotes a boolean propositional formulae that describes properties of states and is used to capture properties of the system. The non-literal terminal c is instantiated with integer values. Both Dwyer et al. [16] and Konrad [20] evaluate their approach with a case study. However, Dwyer evaluates the applicability only for patterns without exact timing bounds, and Konrad only for patterns with exact timing bounds. To our knowledge, there are no case studies that evaluate the applicability of the SPS for mixed requirements. The work of Dwyer et al. has been extended in a number of directions. Grunske extends the patterns to express also probabilistic quality properties [21]. This extension might be useful to express availability or reliability requirements, however in our case study, we consider only behavioral requirements, thus we base our work on Konrad’s SPS.

Cobleigh et al. developed the tool PROPEL (PROPerty ELucidator) that aims to guide users through the process of creating property specifications in supporting them in selecting a suited pattern and scope [22]. The tool provides both a finite-state automaton representation and a natural language representation. However, the tool currently only supports patterns without exact timing bounds. Further restricted English grammars are proposed by, e.g., [23, 24]. However, these grammars are semi-formal and thus do not allow automatic translation to logical formalisms.

3 Planning of the case study 3.1 Study goals and questions In order to assess whether the SPS proposed by Konrad and Cheng can be suitably applied in the automotive domain, two questions should be raised: first, is the SPS expressive enough to express automotive requirements? And, second, do developers, requirements engineers, and sub-suppliers accept the SPS, in the sense that they think it easy to use and useful?

123

22

In this case study, we address only the expressiveness of the SPS, not its acceptance. However, in order to determine whether BOSCH requirements engineers and developers can understand requirements formalized in SPS and can apply it to their requirements, we did an initial informal survey with four requirements engineers, two SW-developers and one tester (all of them working for BOSCH). We showed them requirements formalized in SPS and asked them to explain their meaning to us. After that we asked them to apply the SPS on some of their behavioral requirements. The results indicate that requirements formalized in SPS are easy to understand, as they could explain the meaning of these requirements, with the meaning intended by us. Further, the results indicate that the SPS is easy to apply, as they were all able to express their requirements in SPS. The requirements engineers were especially intrigued by the idea that the SPS allows automatic tests, e.g., for consistency. The developers liked the idea of the SPS as guidance in writing requirements. The SW-developers at BOSCH have to specify requirements for their SW-modules. The tester thought that the requirements in SPS were easy to test and emphasized that informal requirements are often ambiguous and thus difficult to test, whereas requirements in SPS are unambiguous. However, in this initial survey, we did not include stakeholders from marketing, sales, or the customer. A further survey is needed to investigate whether they would accept requirements in SPS. Furthermore, many experts asked for tool support. Therefore, we plan to develop a tool and do a further case study to address the question of ease of use in a bigger context—presuming the present case study indicates the SPS is expressive enough. Another property we investigate in this case study is the pattern complexity: every pattern in the SPS can be expressed in logic, but not every pattern can be expressed in every logic. We define pattern complexity as a function that maps a pattern to the least expressive logic of a given finite set of logics in which it can be expressed. We call the pattern complexity of a set of requirements small, if the patterns needed to express these requirements are mapped

Fig. 1 In the case study we investigate the relation between requirements and SPS, and between requirements and pattern complexity

123

Requirements Eng (2012) 17:19–33

to the least expressive logic of our given set of logics. We are interested whether our case study contains subsets with a small pattern complexity, as we expect that the computational effort for an automatic requirements analysis grows with the expressiveness of the logic used to express the requirements. Therefore, we identified four subgoals of the case study: Goal 1a

For a representative sample of automotive requirements, is it possible to express all these requirements in the SPS by Konrad and Cheng? Goal 1b If there are automotive requirements, that cannot be expressed in the SPS, what is the reason? I.e. could such requirements be expressed in another formalism? Goal 2 Are all patterns in the SPS by Konrad and Cheng relevant for the automotive domain? I.e. are there patterns that are never needed? Goal 3 What is the relation between the requirements development phase, the application domain (e.g., human machine interface, engine controller,...), and the pattern complexity? As depicted in Fig. 1, the subgoals 1a, 1b, and 2 investigate the relation between requirements and SPS, whereas subgoal 3 relates requirements (via the SPS) with pattern complexity.

3.2 Selection of the sample 3.2.1 Selection criteria for documents In the first step, we selected requirements documents from different BOSCH projects of the automotive domain. To get a representative sample, we decided to apply stratified sampling over the automotive application domains car multimedia, driving assistance, engine controlling, power train development, and catalytic converter development. Moreover, we decided to use projects from different development stages, i.e. platform and customer projects [25] from concept phases to development phases close to

Requirements Eng (2012) 17:19–33

23

Fig. 2 Case study design

start of production.1 We then used convenience sampling to select a project out of every stratum. Each project had several requirements documents, some consisting of more than 100 pages. In order to get a representative sample, we asked the corresponding requirements engineers to give us a requirements document such that, first, the document contained behavioral requirements, and, second, the document was representative for the application domain. This way we obtained the following five requirements documents: a document specifying a Human Machine Interface (HMI) D1 (of a car multimedia project), a document specifying an error-handling concept D2 (of a driving assistance project), one of a controller of a heater of an oxygen sensor D3 (of a catalytic converter project), one of a power train controller D4 (of a power train project), and one of an engine injection device D5 (of an engine control project). 3.2.2 Selection criteria for requirements Every document D1, …, D5 consisted of several chapters. In the documents process requirements (e.g., ‘‘The testing coverage shall be at least 99% over all code statements’’) or other nonfunctional requirements were mostly grouped together into chapters, thus separated from the behavioral requirements. In this work, we are only interested in behavioral requirements. Therefore, we scanned the chapters of the documents and made a list with the chapters containing such requirements. After that we randomly chose a chapter out of this list and selected all requirements in the chapter. In this way, we obtained the set of requirements R1 out of D1, …, D5 out of D5. R, the union of these sets, consisted of 289 requirements. We randomly chose chapters instead of the requirements itself, as in our experience a requirement should not be interpreted out of its context. After that we preprocessed the initial data set: we deleted 11 headings, and 22 statements, that were either mere descriptions but not real requirements (e.g., ‘‘Most drivers prefer a smooth deceleration’’), or requirements that did not specify behavior (e.g., ‘‘All failure thresholds 1

Platform projects develop a collection of reusable artifacts, such as requirements, software components, test plans, etc. These artifacts are then reused in customer projects in order to build applications.

shall be defined and documented by the developers.’’). Furthermore, we deleted 11 redundant requirements. In order to ensure that these redundant requirements, headings, and statements could be safely deleted, we discussed every deletion with 2 further experts. After that our sample consisted of 245 informal requirements in prose. 3.3 Case study design The setting of the case study is depicted in Fig. 2: As input we used 245 informal requirements. These are analyzed in a content analysis with the category system defined below. The category system consists of three main groups: phenomenon requirements, requirements expressible in SPS and requirements not expressible in SPS. The category requirements expressible in SPS is further refined in the patterns defined in [1]. We defined a requirement as phenomenon requirement if it specified not behavior but data. An example for such a requirement is ‘‘An error is built up as following: error name (2 Byte), error status (1 Byte), odometer value when the error occurred (2 Byte)’’. Note that a phenomenon requirement cannot be directly mapped to a pattern, instead the data it specifies is mapped to non-literal terminals. Thus, phenomenon requirements are indirectly expressed in the SPS. E.g., in the example the phenomenon requirement would lead to three variables errorName, errorStatus and errorOdometerValue. The behavior requirements then refer to these variables, i.e., the variables are used in the instantiated patterns. Thus, part of the information of phenomena requirements is indeed expressed in the instantiated patterns. However, there are information types that cannot be expressed in the SPS, e.g., the number of bytes or the data type. We defined a requirement as expressible in SPS if and only if the SPS provides one or more suitable patterns, and there is an assignment to the non-literal terminals of these patterns, such that the conjunction of the instantiated patterns expresses the meaning of the requirement. We defined a requirement as not expressible in SPS, if it was not a phenomenon requirement and could not be reformulated in the SPS without loss of meaning. In this step, we used the assistance of four requirements engineers of BOSCH. We asked one requirements engineer

123

24

Requirements Eng (2012) 17:19–33

to determine for every requirement in the sample whether it was a phenomenon requirement, a requirement expressible in SPS or a requirement not expressible in SPS. For requirements expressible in SPS, the engineer should further give instantiated patterns expressing the meaning of the requirement. Hence, the requirements engineer had to decide for every informal requirement if it was possible to express it in SPS without loss of meaning. The other requirements engineers had two responsibilities: first, they assisted with clarifications if the meaning of an informal requirement of their domain was not clear. Second, they validated the results with random tests. The majority of the informal requirements could be reformulated into exactly one instantiated pattern. However, for some requirements, a conjunction of multiple instantiated patterns was needed to express the meaning of the initial requirement. For example, consider the informal requirement ‘‘If the transition condition ChangeGear comes true, the system shall stay for at least 50 ms in state F and then change into the state 0 (after at most 100 ms)’’. This requirement was expressed in SPS as a conjunction of a bounded invariance and a bounded response pattern: ‘‘Globally, it is always the case that if ChangeGear holds, then StateF holds for at least 50 time unit(s)’’ and ‘‘Globally, it is always the case that if ChangeGear holds, then State0 holds after at most 100 time unit(s)’’. We obtained a set size for the resulting requirements expressible in SPS, phenomena and not expressible in SPS of 307. The following statistics concerning subgoal 1 and subgoal 2 relate to the initial number of requirements (245), the statistics for the subgoal 3 relate to the 307 SPS requirements.

4 Analysis of the results 4.1 Goal 1: expressivity of the SPS In a first step, we investigated whether it is possible to reformulate all requirements of the case study in SPS.

Fig. 3 The number of not expressible requirements is strongly reduced by Konrad and Chengs’ patterns. It can be further reduced by extending the grammar with possibility patterns

extended

Therefore, we first measured how many requirements could be expressed in the SPS by Dwyer et al., which is limited to patterns without exact timing bounds. After that we compared these results with the measurements for the SPS by Konrad and Cheng. Figure 3 depicts the results. The figure shows that for the requirements in the case study, using the extension of Konrad and Cheng strongly reduces the number of not expressible requirements. However, 39 (16%) requirements could not be reformulated. Twenty-five of these requirements needed a branching time concept, not provided in the given patterns. The branching concept was needed to allow the specification of possible behavior. Consider the requirement If the gear is in P then it must be possible to start the engine. In the later development phases, this requirement will certainly be split into further more precise and deterministic requirements, e.g, If the gear is in P and the ignition is turned on then the system starts the engine. If the gear is in P and the ignition is turned off, then the engine stays off…. One might argue that it is not desirable to have requirements specifying possible behavior, as these requirements are too coarse. However, as [26], we think that in the early development phases, it is desirable to allow also a less precise specification, as the information needed to specify the requirements in a deterministic way is usually not available. Obtaining this information is usually either very costly or impossible in this phase (e.g. if parts are to be developed by subcontractors). Furthermore, delaying the definition of the requirements can pose a risk for the committed schedule and is thus undesirable. We believe that also in the early development stages, a formalization of the requirements is desirable. Thus, for early development phases, we think that an extension with branching time concept patterns is needed. Therefore, we propose to extend the grammar by Konrad and Cheng with possibility patterns depicted in Table 2. However, in later development phases, we recommend to check that there are no possibility patterns used anymore.

13 14

SPS SPS by Konrad/ Cheng

13

SPS by Dwyer et al

13

0

phenomenon

218

130

50

expressible in SPS

102

100

150

number of requirements

123

not expressible in SPS

193

39

200

250

Requirements Eng (2012) 17:19–33

25

Table 2 Grammar of the extended patterns ::=

if P holds then there is at least one execution sequence such that S eventually holds

::=

if P holds then there is at least one execution sequence such that S holds after at most c time unit(s)

Possible bounded invariance pattern

::=

if P holds then there is at least one execution sequence such that S holds for at least c time unit(s)

25 20 15 10 5

number of requirements

Possibility pattern Possible bounded response pattern

not expressible in extended SPS expressible in extended SPS

5

powertrain controller (R4)

unclear description of appearance

5

20

0

out of context

project 6 1 HMI (R1)

2 4 0 error handler (R2)

oxygen sensor (R3)

1 8

1 engine controller (R5)

Fig. 4 The left figure depicts the number of requirements not expressible in the language by Konrad/Cheng but expressible via extended patterns. On the right a further classification of requirements not expressible in extended SPS

With the help of the possibility patterns, it is possible to reduce the number of not expressible requirements from 39 (16%) to 14 (6%), as depicted in Fig. 3. Figure 4 depicts that in greater detail for R1, …, R5, every bar represents the number of requirements that were not expressible in the SPS by Konrad and Cheng. For example, 25 (10%) requirements of R4 were not expressible. The bars are further divided into two classes: the number of requirements that could be expressed by the new possibility patterns is depicted in light gray, the number of requirements that are still not expressible is depicted in dark gray. Note that in the case study the possibility patterns were especially needed for the requirements of the projects in an early development phase, particularly in R4. This is plausible as in later development phases such requirements will be split into more precise requirements. Table 3 depicts our translation of the possibility patterns to LTL, CTL, and TCTL (the logics used in [1, 16]). Note that not every pattern could be translated into every logic, e.g., the possible bounded invariance pattern cannot be expressed in LTL or CTL. However, even with this extension, it was not possible to find a reformulation in SPS for 14 (6%) requirements. Thus, we investigated what reasons made a requirement not expressible. We identified three reasons, depicted in Fig. 4. One not expressible requirement was only not expressible because it was beyond the system context: ‘‘If failures are detected in multiple electronic control units (ECUs) the same classification of faults shall be used in all ECUs’’.

This requirement was given to a project developing a single ECU, but clearly this requirement cannot be solved on the ECU level as in this context it is not known what faults are detected by other ECUs. However, on the context level of the whole car, the requirement could be expressed in SPS. Five requirements could not be formalized as they did not specify any behavior. Instead, they described the appearance of the product, e.g., ‘‘the warning icon is an image of two cars with a star in between’’. As in this work we wanted to investigate only behavioral requirements, these requirements were wrongly selected into our sample. Due to our scope behavioral requirements, these five requirements should be ignored. Finally, the majority of requirements became not expressible because the meaning of the requirement was not clear or just too vague. Neither the requirements engineer who formalized the requirements in the first step, nor the evaluators understood what the requirement wanted to specify, thus, it was clearly impossible to formalize it. Examples for such requirements are ‘‘The drag torque and the activation torque depend on the operating state.’’ (how?) and ‘‘warning in central line of vision’’ (how is the ‘‘line of vision’’ determined? E.g., adaptive to the size of the driver?). Thus, in these 8 cases, the problem was not that the SPS was not expressive enough but instead that the requirements were unclear. So, in fact, the SPS helped to identify requirements that needed to be revised. Thus, we come to the conclusion that the extended version of the SPS is well suited to express behavioral requirements

123

26

Requirements Eng (2012) 17:19–33

Table 3 Translation of the possibility patterns to LTL, CTL and TCTL Pattern

Scope

LTL

CTL

TCTL

Possibility

Globally

n.a.

AGðP ! EF SÞ

AGðP ! EF SÞ

Before R

n.a.

AðððP ! E½ð:RÞUðS ^ :RÞÞ _AGð:RÞÞW RÞ

AðððP ! E½ð:RÞUðS ^ :RÞÞ _AGð:RÞÞW RÞ

Possible bounded response

Possible bounded invariance

After Q

n.a.

Að:Q WðQ ^ AGðP ! EFðSÞÞÞÞ

Að:Q WðQ ^ AGðP ! EFðSÞÞÞÞ

Between Q and R

n.a.

AGðQ ^ :R ! A½ððP ! E½:RU ðS ^ :RÞÞ _ AGð:RÞÞW RÞ

AGðQ ^ :R ! A½ððP ! E½:RUðS ^ :RÞÞ _AGð:RÞÞW RÞs

After Q until R

n.a.

AGðQ ^ :R ! A½ðP ! E½:RU ðS ^ :RÞÞW RÞ

AGðQ ^ :R ! A½ðP ! E½:RUðS ^ :RÞÞW RÞ

Globally

n.a.

n.a.

AGðP ! EF  c S Þ

Before R

n.a.

n.a.

A½ððP ! E½:RU  c ðS ^ :RÞÞ _ AGð:RÞÞW R

After Q

n.a.

n.a.

Að:Q WðQ ^ AGðP ! EF  c ðSÞÞÞÞ

Between Q and R

n.a.

n.a.

AGðQ ^ :R ! A½ððP ! E½:RU  c ðS ^ :RÞÞ _AGð:RÞÞW RÞ

After Q until R Globally

n.a. n.a.

n.a. n.a.

AGðQ ^ :R ! A½ðP ! E½:RU  c ðS ^ :RÞÞW RÞ AGðP ! EG  c SÞ

Before R

n.a.

n.a.

AððP ! EG  c ð:R ^ SÞ _ AGð:RÞÞW RÞ

After Q

n.a.

n.a.

Að:Q WðQ ^ AGðP ! EG  c ðSÞÞÞ

Between Q and R

n.a.

n.a.

AGðQ ^ :R ! A½ððP ! EG  c ðS ^ :RÞÞ _AGð:RÞÞW RÞ

After Q until R

n.a.

n.a.

AGðQ ^ :R ! A½ðP ! EG  c ðS ^ :RÞÞW RÞ

from automotive projects at BOSCH. The majority of the behavioral requirements could be directly expressed in the SPS by [1]. With an extension of only three patterns, all behavioral requirements could be expressed as long as their meaning was clear. As only three further patterns were needed, the case study indicates that Dwyer et al. may be right with their belief that in practice only some few patterns are needed to express requirements. The effort to extend the SPS was low, i.e., we had to add only three patterns with their formalization. Furthermore, the SPS helped to identify requirements that needed to be revised. Thus, it seems that for behavioral automotive requirements of BOSCH, the SPS is well suited. However, for requirements that are not behavioral requirements, this claim does not hold. In fact, the requirements specifying the appearance of the product were nearly impossible to formalize in any formalism. Thus, we think this is an indication that not all kind of requirements can be formalized. It seems that behavioral requirements are suitable for formalization, but for the other kinds of requirements, this needs to be investigated. In particular, methods are needed that separate between requirements that can be formalized and requirements that need to be validated via other methods. 4.2 Goal 2: pattern relevance Next, we evaluated whether all patterns in the SPS by Konrad and Cheng are relevant for the automotive

123

application domain. This question needs to be asked to find a minimal pattern set. We assume that a less complex SPS with fewer patterns is easier and faster to use for developers than one with many patterns. However, we still want to express all automotive requirements. Thus, we evaluated whether the SPS by Konrad and Cheng contains patterns that were never needed in the case study. We identified six patterns that were not needed to express any requirement in the case study: the bounded existence, precedence chain 1-2, precedence chain 2-1, response chain 1-2, response chain 2-1 and constrained chain pattern. Thus, the case study indicates that for the system level in automotive projects at BOSCH, these patterns might be omitted. We do not know whether this indication is valid for the whole automotive domain. Hence, it would be interesting to investigate that question in another case study.

4.3 Goal 3: pattern complexity versus application domain The SPS by Konrad and Cheng (and also the extended version) can be automatically transformed to the logics LTL, CTL, GIL, MTL, TCTL, and RTGIL. However, not every pattern can be translated into every logic. All provided patterns can be translated into TCTL. But, e.g., only the patterns without any reference to quantitative time or

Requirements Eng (2012) 17:19–33

possible behavior can be expressed in LTL. For example, the absence pattern (‘Globally, it is never the case that P holds’) has no reference to quantitative time and can be expressed in LTL as ‘G(:P)’. In contrast, the Minimum duration pattern (‘Globally, it is always the case that once P becomes satisfied it holds for at least c time unit(s)’) refers to the explicit time bound c. It is not possible to express explicit time bounds (for continuous time) in LTL [4], thus this pattern cannot be expressed in LTL. For LTL, CTL, and TCTL, tool support is available [27, 28], therefore we focus in the following work on these logics. However, even for this subset of logics, there is a trade-off between expressivity of the logic and its computational class. For example, a consistency check for a set of requirements can be transformed into the satisfiability problem. However, the satisfiability problem is undecidable for TCTL [4], while decidable for LTL and CTL. Thus, we are interested in the pattern complexity and its relation to the application domain (e.g., human machine interface, engine controller, …), and the requirements development phase. We define pattern complexity as a function that maps a pattern to the least expressive logic [ {LTL, CTL, TCTL} in which it can be expressed. The result of the mapping is depicted in Fig. 5. Note that all LTL patterns can be expressed in CTL as well, and both CTL and LTL patterns can be also expressed in TCTL. Generally, LTL is no subset of CTL. However, in this case, all patterns that can be expressed in LTL can be expressed in CTL as well. We counted for every set of requirements R1, …, R5 the number of LTL, CTL and TCTL patterns. As visible in Fig. 6, TCTL patterns were needed in every project. The only exception was the HMI-project: in this set of requirements, the number of TCTL patterns was negligible. However, this project was still in its concept phase, hence we assume that in the later development phases, the need for requirements with exact timing bounds will arise as well. Hence, we believe that for the

Fig. 5 The patterns are classified according to the least expressive logic they can be expressed in. The extended patterns are depicted in italic font

27

specification of the whole system functionality, the need for TCTL patterns is inevitable. However, it might be the case that individual components of the system could be solely expressed in less complex patterns. Further investigations are needed to prove or refute that thesis.

5 Observations To express the informal requirements in the SPS, the requirements engineers needed in average 2–3 min per requirement. Most of the time was needed to understand the meaning of the informal requirement, the reformulation itself was then quickly done. However, even 2 min per requirement may scale to a considerable amount of time as in the automotive domain, there are often thousands of requirements for one product. However, if the method is integrated within the development processes (i.e., the requirements are directly formulated in the SPS when developing the requirements), then these costs could be omitted. Another result of the case study is that neither the patterns nor the scopes were equally used. For example, as depicted in Table 4, more than 80% of the requirements formulated in SPS are formulated using the ‘‘Globally’’ scope, and more than 50% of the requirements formulated in SPS are formulated using the Absence or Bounded response pattern. 5.1 Usage of scopes As depicted in Table 1, the SPS offers five scopes that determine the extent of the program execution over which the pattern must hold. Figure 7, taken out of [29], illustrates the meaning of the scopes: ‘‘Globally’’ (the entire program execution), ‘‘Before R’’ (the execution up to a given state/ event R), ‘‘After Q’’ (the execution after a given state/event Q), ‘‘Between Q and R’’ (any part of the execution from one given state/event to another given state/event), and ‘‘After Q until R’’ (like between but the designated part of the execution continues even if the second state/event does not occur). ‘‘Before R’’ and ‘‘After Q’’ scopes are interpreted relative to the first occurrence of the designated state/event. Dwyers et al. state that ‘‘Scope operators are not present in most specification formalisms […]. Nevertheless, our experience strongly indicates that most informal requirements are specified as properties of program executions or segments of program executions. Thus a pattern system for properties should mirror this view to enhance usability’’ [29]. This matches our experiences in the case study. As depicted in Table 4, more than 80% of the requirements in our case study were formulated using the ‘‘Globally’’ scope—however for the remaining requirements, the other scopes were needed to express them.

123

28

Requirements Eng (2012) 17:19–33

Fig. 6 TCTL patterns were needed in every project

140 120 100

TCTL patterns CTL patterns LTL patterns

36

80 27 60

1

40

2

25 5

53 42

20

27 0 powertrain controller (R4)

HMI (R1)

error handler (R2)

28

23

4

5

oxygen sensor (R3) engine controller (R5)

Table 4 Evaluation results with regard to the pattern frequency displayed in the case study Patterns

Globally

Before

After

Between

Until

Total

Absence

63

0

5

0

1

69

Universality

23

0

0

2

2

27

Existence

2

0

0

0

0

2

Bounded existence

0

0

0

0

0

0

Response

27

0

0

1

0

28

Precedence Response chain

4 0

0 0

0 0

0 0

0 0

4 0

Precedence chain

0

0

0

0

0

0

Minimum duration

2

0

0

0

0

2

Maximum duration

3

0

0

0

0

3

Periodic category

4

0

1

0

10

15

Bounded responce

59

2

0

1

2

64

Bounded invariance

30

0

0

1

0

31

217

2

6

5

15

245

Total

In our opinion, many of the requirements with ‘‘Globally’’ scope were in fact not required for the whole execution sequence. Instead, they were only required after a successful startup and until the shutdown-sequence starts. However, the requirements engineers simplified the requirements to reduce their complexity. As the majority of requirements specify behavior that needs to be fulfilled after startup and before shutdown, instead of writing for every such requirement ‘‘Between Startup and ShutDown’’,

123

Fig. 7 Pattern scopes

the requirements engineers used the ‘‘Globally’’ scope and check the requirements for the normal operation mode separately from the Startup- and the ShutDown-Mode. Through that simplification nearly all requirements are then valid for the whole execution sequence (i.e., the time interval after Startup and before ShutDown). Another reason for the high frequency of requirements with ‘‘Globally’’ scope is that one normally wants to specify a system that has always the same reaction to an event, even if the event occurs several times. To specify such a behavior, the ‘‘Globally’’ scope is needed. For example, consider the following two requirements in SPS: ‘‘Globally, it is always the case if IRTest holds then IRLampsOn eventually holds’’ and ‘‘After IRTest, IRLampsOn eventually holds’’. Although the two requirements describe related behavior, probably only the first requirement specifies the desired behavior. The first requirement is fulfilled if every time IRTest occurs, IRLampsOn occurs later on. In contrast, the second requirement is already fulfilled if after the first IRTest the signal IRLampsOn occurs once and then never occurs again (even if IRTest appears again).

Requirements Eng (2012) 17:19–33

5.2 Usage of patterns Table 4 shows that some patterns were more frequently used than others. The most frequently used patterns in the case study were the Absence, Response, Bounded response, and Bounded invariance pattern. The Absence pattern (‘it is never the case that P holds’) was mainly used to link two variables. Automotive requirements specification typically contains requirements of the form ‘if P holds then S holds as well’. For example, when specifying a system state machine, there are requirements that specify the variables that have to hold in a certain state. For example, one might want to express that ‘if the system is in SleepMode then the microcontrollers are turned off’. Such requirements can be expressed with the absence pattern, in writing ‘it is never the case that SleepMode ^: MicroControllersTurnedOff holds’. However, feedback of the requirements engineers implies that they consider it less intuitive to express invariant behavior with the help of the Absence pattern. As a consequence of the frequent application of the construct, they suggested to add an additional pattern of the form ‘it is always the case that if P holds then S holds as well’. We call it invariant pattern. The pattern does not extend the expressiveness of the SPS (as it can be mapped to the absence pattern, as shown above). However, the requirements engineers expect that the intention of such a requirement gets clearer to a reader when written in the form of the new pattern. Further patterns that were frequently used are the Response pattern (‘it is always the case that if P holds then S eventually holds’) and the Bounded response pattern (‘it is always the case that if P holds then S holds after at most c time units’). Many requirements in the automotive domain specify requirements in describing the system as a black box, i.e., they specify how the system shall react to the input signals. This kind of requirements can be expressed by the Response and Bounded response pattern. The above-mentioned invariant pattern describes simultaneous assignment of variables. In contrast, the (bounded) Response pattern specifies behavior in such a way that time may pass before the reaction to an event occurs. The Bounded invariance pattern (‘it is always the case that if P holds then S holds for at least c time units’) was mainly used to formulate technical constraints. For example, say one wants to specify how fast the gears in a vehicle may be changed. Say there is a requirement ‘it is always the case that if nextGear holds then ShiftCollarInRightPosition holds after at most 10 time units’. A shift collar needs some time to position itself. This technical constraint may be expressed with the bounded invariance pattern, e.g., via ‘it is always the case that if nextGear holds then : ShiftCollarInRightPosition holds for at least 7 time units’. This way it is possible to express technical constraints.

29

The other patterns were used to a much lesser extent. Thus, the case study indicates that some few patterns are extensively used, whereas a lot of patterns are only sparsely used. This finding strengthens the belief of Dwyer et al. that some few patterns suffice to express the majority of the properties of a system [16]. This is only a belief and cannot be proven. However, we think it might be interesting to refute or strengthen that belief with further studies from other domains. A further strengthening of the belief could vastly improve the applicability of formal methods on requirements. 5.3 Applicability Only behavioral requirements can be expressed in SPS. Thus, it would be interesting to know how many requirements are typically behavioral requirements (at least for the automotive domain). To our knowledge, there exists no study on that topic. In the following, we give a rough estimation, derived from our experiences at BOSCH. Note that these numbers are just estimations. We did not do a case study on that topic, the given numbers are only the result of discussions with the requirements engineer(s) that participated in this study. At BOSCH requirements are typically specified on 3 abstraction levels. On the customer level, all requirements given by the customer, by norms and standards are listed. On this level, there are also many project requirements specifying how the system shall be developed (e.g., what test coverage is expected, the release plan, …). In our experience, on this level, the behavioral requirements make up about a third of the requirements. The other requirements are mainly project requirements, non-functional requirements, links to norms and standards, and technical constrains (e.g., that only certified CAN-bus transceivers may be built-in). On system level there are derived requirements for the system, mostly detailing the expected behavior of the system, the interfaces, and the mechanical constraints. The project requirements are not listed here, but instead are integrated into test management plans, project release plans, and so on. Thus, on this level, the percentage of behavioral requirements increases to about 60–70% of the requirements in the specification documents. On the next level, the requirements are divided into Software (SW) and Hardware requirements. For the SW, the behavior is described in greater detail (dependent on the input and output variables of the SW). Thus, on this level, there are mainly behavioral requirements, with some additional data structures. Hence, the SPS may be less suited on the customer level, but more suited for the later abstraction levels. We do not know whether these numbers are typical for the automotive domain or just for BOSCH. Further, we do not know whether different domains have similar

123

30

distributions. We think that it would be very interesting to do further research on that topic. Another aspect is the question about the implications for the overall coherence of a requirements document, if the behavioral requirements are expressed in SPS, whereas other requirements are expressed in different notations. In our experience, existing requirements documents already mix different notations. Although the majority of requirements in the automotive domain are written in natural language, some requirements are already written in different notations [7, 30]. For example, operating states and the transitions between them are often expressed via state machines, interface data are mostly expressed in tables, etc. Not every notation is suited for every purpose. For example, it is possible to express the interface of two modules in a graphical way, but for defining the range of values, the cycle time, the bit order, etc., at BOSCH tables are found to be more suited. Thus, mixing different notations is always necessary and unavoidable. However, using different notations makes it more difficult to allow automatic analysis. We want to use the SPS to allow an automatic analysis of behavioral requirements. This way, we can automatically check the requirements expressed in SPS for, e.g., consistency. However, even if the requirements in SPS are consistent with each other, they may still be in conflict with other requirements that are not expressed in SPS. In our experience, not all kinds of requirements depend on each other. For example, process requirements have no impact on behavioral requirements and vice versa. Thus, in our experience, these kinds of requirements can be analyzed independently of each other. In contrast, non-functional requirements may have an impact on behavioral requirements. Hence, the interdependency of non-functional requirements with behavioral requirements needs to be analyzed. Reviews are needed to find these errors. To reduce the effort of reviews, we think that it is important to investigate the interdependencies between different kinds of requirements, like, e.g., done by [9].

6 Threats to validity In this section, we analyze threats to validity defined in Neuendorf [31], Krippendorff [32], and Wohlin [33]. 6.1 External validity 6.1.1 Sampling validity [31] This threat arises if the sample is not representative for the requirements. In order to minimize this threat, we used the selection procedure described in Sect. 3.2, thus getting

123

Requirements Eng (2012) 17:19–33

representative requirements of every application domain. A limitation of the case study is that we only used requirements of BOSCH projects. Thus, we cannot extend our results to the whole automotive domain but only for BOSCH’s automotive domain. 6.1.2 Interaction of selection and treatment [33] This threat arises if the requirements engineer in this study (see Sect. 3.3) is not representative for BOSCH requirements engineers. However, the reliability analysis in Sect. 6.4 suggests that the application of the patterns is sufficiently independent of the evaluator. 6.2 Internal validity 6.2.1 Selection [33] This threat arises due to natural variation in human performance. The requirements engineer in this study (see Sect. 3.3) could have been especially good in formalization. The reliability analysis in Sect. 6.4 suggests that the application of the patterns is sufficiently independent of the evaluator. Another issue that could have impacted our study is that the requirements engineer might omit a requirement that can be formalized, partially formalize the requirement, or change the meaning of the requirement. We therefore asked a team of 3 requirements engineers to check a randomly selected subset of the coded requirements to ensure that the meaning of the original requirement was not changed. Thus, we expect variation in human performance not to be an issue. 6.3 Construct validity 6.3.1 Experimenter expectancies [33] Expectations of an outcome may inadvertently cause the evaluators to view data in a different way. However, the evaluators have no benefit from a good or bad outcome for the applicability of the SPS as they did not invent it. Thus, such psychological effects probably did not affect the evaluators. 6.3.2 Semantic validity [32] This threat arises if the analytical categories of texts do not correspond to the meaning these texts have for particular readers. In the case study, the categories are clearly defined in Sect. 3.3 and through the patterns in the SPS [1]. However, when reformulating an informal requirement in SPS, there are several possibilities to instantiate a pattern with a phenomenon that shall correspond to the description

Requirements Eng (2012) 17:19–33

in the text. Thus, in the formalization, the requirements engineer might invent phenomena without clearly defining their meaning. Discussion with experts showed that a data dictionary is a potential candidate to minimize this threat. 6.3.3 Face validity [31] Face validity is the extent to which a measure addresses the desired concept, i.e. the question whether it measures what it is supposed to measure. In order to ensure face validity, we discussed with experts, without mention of the case study, whether the instantiated patterns are a good representation of their concept of behavioral requirements. The discussion indicated that the patterns seem to capture typical behavioral requirements. 6.4 Conclusion validity 6.4.1 Intercoder reliability [31] Unreliable coding limits the chance to make valid conclusions based on the results. Especially in this case study, in which only one requirements engineer coded the requirements, there is a risk that the assessment was subjective. In order to minimize this threat, we asked a requirements engineer with high experience in the application domain and another one with little experience and external to the project to code a set of 30 requirements. The requirements were randomly chosen out of the sample defined in Sect. 3.2. In order to minimize the threat of different interpretations of the phenomena, we gave both evaluators a data dictionary. The results yielded a reliability of 0.86/0.86 according to Cohen’s Kappa/Scott’s pi. In consequence, the measure seems to be very reliable. Even though the reliability is good, we were interested in the reasons for different codings. We identified that the requirements engineer tended to use his domain knowledge when formalizing the requirements. For example, consider the following informal requirement: ‘‘If the locally measured voltage is not available, the voltage value as received from the bus shall be used’’. Using the phenomena localVoltageNotAvailable, busVoltage and internalVoltage with their obvious meanings, the evaluator with low experience in the application domain expressed the requirement as following ‘‘Globally, it is always the case that if localVoltageNotAvailable holds, then internalVoltage == busVoltage holds as well’’. However, the requirements engineer specified instead ‘‘Globally, it is always the case that if localVoltageNotAvailable and errorGetsActive and : busOff holds, then errorVoltage == busVoltage holds as well’’. The engineer used the additional system knowledge, that (first) the locally measured voltage is only

31

needed to be stored if an error appears, and (secondly) that if the bus is off, then this requirement does not apply. Thus, differing knowledge of the system context might lead to unreliable results. We believe that this threat can be further minimized if the coders discuss their interpretation of the informal requirements prior to the formalization.

7 Outlook 7.1 Evaluating the ease of use and usefulness As described in Sect. 3.1 in this case study, we only addressed the question of expressivity of the SPS. Thus, the next step is to address the ease of use and usefulness: Is the application of the SPS natural to practitioners? How well can they apply it? What are the risks? What is the benefit of the formal analysis enabled by the SPS? We think that before addressing these questions, two prior tasks need to be done. First, tool support is needed that supports the practitioners in writing requirements in SPS and that syntactically checks the correct application of the SPS. Second, tool support is needed that exploits the full potential of the SPS and executes formal analysis on the requirements. The first task is needed, as we expect that it is very difficult to comply to the strictures of the SPS if there is no external check. One requirements engineer compared the SPS to coding rules for code. He asked for tool support as he experienced that the coding rules are only constantly applied as long as there are periodic code reviews or automatic coding rule checks. As soon as the reviews or checks stop, the developers start writing again in their own style. We expect to have a similar behavior with requirements engineers. Thus, we think an automatic syntactical check is needed. The second task is needed to exploit the full potential of the SPS. In our eyes, a key advantage of the SPS is that the requirements (though looking like natural language) can be automatically translated into logical formulae. Thus, they allow a computer to formally analyze the requirements. However, to assess the benefit of such a formal analysis, we first need some unambiguous properties to check. We developed tool support for checking the requirements’ consistency (the specification is unsatisfiable by any system, e.g., because it contains two contradicting requirements), rt-consistency [34] and vacuity [35]. The property rt-consistency reflects that the requirements specify apparently inconsistent timing constraints. The property vacuity reflects that a requirement is only vacuously satisfied in a given set of requirements, i.e., it is only satisfied in a non-interesting way. To identify further unambiguous

123

32

properties for requirements remains an active research topic; see, e.g., [8, 9, 36]. We used the SPS as input language for our formal analysis. Our tools, which check for consistency, rt-consistency, and vacuity, are based on the logic Duration Calculus. Thus, we invented a translation from the SPS into Duration Calculus formulae. First results showed that the formal analysis generates high benefit. We applied the formal analysis on requirements of 16 components. Although all requirements had undergone a thorough review before the formal analysis, the automatical check detected errors in 8 of the 16 sets of requirements. Thus, first results (regarding the second task) are promising. Still, the current tools were developed to evaluate the benefit of the formal analysis. Before starting an evaluation of the ease of use by requirements engineers, we need to develop a user interface suited for not-formally minded practitioners. 7.2 Extending the SPS for non-functional requirements The specification pattern system by Konrad/Cheng can only be applied to behavioral requirements. Non-functional requirements cannot be expressed in this SPS. However, Grunske developed a SPS that captures probabilistic quality properties. In this SPS, it is possible to express, e.g., availability, reliability, security, performance, and safety requirements [21]. For example, a safety requirement might be expressed in specifying that the probability that a certain hazard occurs is less than the tolerable hazard rate. A performance requirement might be expressed in specifying that a certain percentage of requests must be handled within specified time bounds. Integrating the SPS by Grunske with the SPS by Konrad and Cheng, it would be possible to express both behavioral requirements and the kind of non-functional requirements covered by probabilistic quality patterns. Adding the patterns to the SPS is easily done. However, to allow an automatic analysis, the patterns should be translated into the same logic, i.e., a probabilistic real-time logic. There we see some drawbacks: At present, we do not know of any tool solving such kind of logic that is suitable for economic use. Second, in [34, 35], we experienced that already for behavioral requirements we suffer the state explosion problem during formal analysis, although these requirements are expressed in a non-probabilistic real-time logic. We expect that using a probabilistic real-time logic to analyze requirements, the state space will explode even faster. However, research on probabilistic logics is ongoing, and there are already some promising results on model-checking probabilistic systems [37, 38]. Thus, we think that in future an extension of the SPS by [1] with the SPS by [21] might be beneficial.

123

Requirements Eng (2012) 17:19–33

Note, that even with an extended SPS, not all nonfunctional requirements can be expressed in the SPS. We expect that there will be always requirements that cannot be formalized. For example, we do not see a possibility to formally express a requirement like ‘‘The system shall look great’’. For this kind of non-functional requirements, we think that, e.g., the PLanguage given by Tom Gilb and evaluations with the Technology Acceptance Model might be well suited [39, 40].

8 Conclusion This case study investigates the question whether in practice the SPS suffices to express automotive behavioral requirements. Based on the results of Sect. 4.1, we think that at least for automotive requirements at BOSCH, this question can be answered with yes. The belief of Dwyer et al. is that some few patterns suffice to express the majority of the properties of a system. This is only a belief and cannot be proven. Nevertheless, we think that Dwyer et als’, Konrad’s and our case study strengthens that belief: in every case study, the majority of requirements could be reformulated in the SPS. Furthermore, the distribution of the patterns indicates the same: some few patterns are extensively used, whereas a lot of patterns are only sparsely used. Thus, it seems that in practice, some few patterns suffice to express automotive behavioral requirements at BOSCH. However, further studies with requirements of other automotive suppliers are needed to refute or strengthen that belief for the whole automotive domain. Furthermore, the case study indicates that for BOSCH the SPS may be even reduced, as six patterns were not needed. It might be beneficial to investigate whether developers can apply such a less complex SPS even faster. Moreover, the case study shows that the SPS can be easily adapted to a certain application domain. The adaptation to the BOSCH automotive domain required the addition of three patterns to the SPS and their translation into the logical formalisms CTL and TCTL. This effort stays reasonable as only few additional patterns are needed. Last, regarding the pattern complexity, it seems that on system level there are always requirements that can be solely expressed in TCTL patterns. It would be interesting to investigate whether this is also true on the component level. Concluding, the results indicate that it is worth developing tool support for the SPS that we believe prerequisite to allow the next stage of evaluation. In the next stage, it should then be evaluated whether in practice automotive requirements engineers accept the strictures of SPS and how strong they rate the benefit of formal reasoning.

Requirements Eng (2012) 17:19–33

References 1. Konrad S, Cheng BHC (2005) Real-time specification patterns. In: ICSE ’05: Proceedings 27th International Conference on Software Engineering, New York, ACM, pp 372–381 2. Emerson EA (1990) Temporal and modal logic. In: Handbook of theoretical computer science. Volume B. Elsevier, Amsterdam, pp 995–1072 3. Ramakrishna YS, Melliar-Smith PM, Moser LE, Dillon LK, Kutty G (1996) Interval logics and their decision procedures. TCS 170(1–2):1–46 4. Alur R (1992) Techniques for automatic verification of real-time systems. PhD thesis, Stanford University, Stanford 5. Moser LE, Ramakrishna YS, Kutty G, Melliar-Smith PM, Dillon LK (1997) A graphical environment for the design of concurrent real-time systems. ACM Trans Softw Eng Methodol 6(1):31–79 6. Davis AM (1993) Software requirements: objects, functions, and states. Prentice-Hall, Inc., Upper Saddle River 7. Heumesser N, Houdek F (2004) Experiences in managing an automotive requirements engineering process. In: RE, IEEE Computer Society, pp 322–327 8. Walia GS, Carver JC (2009) A systematic literature review to identify and classify software requirement errors. Inf Softw Technol 51(7):1087–1109 9. Dahlstedt AG, Persson A (2003) Requirements interdependencies—moulding the state of research into a research agenda. In: REFSQ, pp 71–80 10. Heimdahl MPE, Leveson NG (1995) Completeness and consistency analysis of state-based requirements. In: IEEE Transaction on SW Engineering, pp 3–14 11. Heitmeyer CL, Jeffords RD, Labaw BG (1996) Automated consistency checking of requirements specifications. ACM Trans Softw Eng Methodol 5(3):231–261 12. Yu L, Su S, Luo S, Su Y (2008) Completeness and consistency analysis on requirements of distributed event-driven systems. In: TASE, Washington, pp 241–244 13. ISO26262 (2010) Road vehicles—functional safety, Part 8, Baseline 17 14. Hall A (2007) Realising the benefits of formal methods. J Univers Comput Sci (JUCS) 13(5):669–678 15. Post A, Menzel I, Podelski A (2011) Applying restricted english grammar on automotive requirements: does it work? A case study. In: Proceedings of the 17th international working conference on Requirements engineering: foundation for software quality. REFSQ’11, Springer, Berlin, pp 166–180 16. Dwyer MB, Avrunin GS, Corbett JC (1999) Patterns in property specifications for finite-state verification. In: ICSE, ACM, New York, pp 411–420 17. Abrial JR (2006) Formal methods in industry: achievements, problems, future. In: ICSE, ACM, New York, pp 761–768 18. Kuhn T (2007) Acerules: Executing rules in controlled natural language. In: International conference on web reasoning and rule systems. Springer, Berlin 19. Han B, Gates D, Levin L (2006) From language to time: a temporal expression anchorer. TIME, pp 196–203 20. Konrad S (October 2006) Model-driven development and analysis of high assurance systems. PhD thesis, Michigan State University, East Lansing 21. Grunske L (2008) Specification patterns for probabilistic quality properties. In: ICSE, ACM, New York, pp 31–40

33 22. Cobleigh RL, Avrunin GS, Clarke LA (2006) User guidance for creating precise and accessible property specifications. In: FSE, ACM, New York, pp 208–218 23. Rupp C (2004) Requirements Engineering und Management— Professionelle, iterative Anforderungsanalyse fu¨r die Praxis. Carl Hanser Verlag, Mnchen 24. Mavin A, Wilkinson P (2010) Big ears (the return of ‘‘easy approach to requirements engineering’’). In: RE’10. 277–282 25. Pohl K, Bo¨ckle G, Linden FJvd (2005) Software product line engineering: foundations, principles and techniques. Springer, USA 26. Sibay G, Uchitel S, Braberman V (2008) Existential live sequence charts revisited. In: International Conference on Software Engineering. ICSE ’08, ACM, New York, pp 41–50 27. Behrmann G, David A, Larsen KG (2004) A tutorial on uppaal, pp 200–236 28. Cimatti A, Clarke E, Giunchiglia E, Giunchiglia F, Pistore M, Roveri M, Sebastiani R, Tacchella A (2002) NuSMV: an opensource tool for symbolic model checking. In: CAV. Vol 2404 of LNCS., Springer, Copenhagen 29. Dwyer M, Avrunin G, Corbett J (1997) A system of specification patterns. http://patterns.projects.cis.ksu.edu/ 30. Houdek F (2003) Requirements Engineering Erfahrungen in Projekten der Automobilindustrie. Softwaretechnik-Trends 23(1) 31. Neuendorf KA (2002) Content analysis guidebook. Sage Publications, Thoushand Oaks 32. Krippendorff KH (2003) Content analysis: an introduction to its methodology. 2nd edn. Sage Publications, Inc., Thoushand Oaks 33. Wohlin C, Runeson P, Ho¨st M, Ohlsson MC, Regnell B, Wessle´n A (2000) Experimentation in software engineering: an introduction. Kluwer, Norwell 34. Post A, Hoenicke J, Podelski A (2011) rt-inconsistency: a new property for real-time requirements. In: Fundamental Approaches to Software Engineering. FASE’11/ETAPS’11, Springer, Berlin, pp 34–49 35. Post A, Hoenicke J, Podelski A (2011) Vacuous real-time requirements. In: International conference on requirements engineering. RE2011. Springer, Berlin 36. Hayes JH (2003) Building a requirement fault taxonomy: Experiences from a nasa verification and validation research project. In: ISSRE ’03: proceedings of the 14th international symposium on software reliability engineering. IEEE Computer Society, Washington, pp 49 37. Kwiatkowska M, Norman G, Parker D(2011) PRISM 4.0: Verification of probabilistic real-time systems. In: Gopalakrishnan G, Qadeer S (eds), Proceedings 23rd International Conference on Computer Aided Verification (CAV’11). Vol. 6806 of LNCS, Springer, Berlin, pp 585–591 38. Teige T, Eggers A, Fra¨nzle M (2011) Constraint-based analysis of concurrent probabilistic hybrid systems: an application to networked automation systems. Nonlinear Anal: Hybrid Syst 5(2):343–366 39. Gilb T (1989) A planning language (a Planguage). In: Conference proceedings on APL as a tool of thought. ACM, New york, pp 169–177. http://doi.acm.org/10.1145/75144.75168 40. Davis FD (1989) Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly 13(3):319–340

123