Modeling safety and security interdependencies with BDMP (Boolean ...

Modeling safety and security interdependencies with BDMP (Boolean logic Driven Markov Processes) Ludovic Piètre-Cambacédès1,2, Marc Bouissou1,3 1

Electricité de France (EDF) R&D, Clamart, France Institut Telecom, Telecom ParisTech, CNRS LTCI, Paris, France 3 Ecole Centrale Paris, Châtenay-Malabry, France {ludovic.pietre-cambacedes, marc.bouissou}@edf.fr

Keywords— Security modeling, safety engineering, fault trees, attack trees, BDMP, risk analysis.

I.

INTRODUCTION

Safety and security have long been considered as two separate issues and dealt with by two distinct and isolated communities [1]. In fact, they have many points in common and share intimate connections. Already identified in the early 90’s (e.g. in [2, 3, 4]), these links are being progressively more widely recognized and investigated [5]. Such an investigation is indeed becoming crucial by the convergence of security and safety issues on systems that used to be concerned by only one of the two aspects. This is particularly the case in risk-prone industries, where the systems in charge of safety, e.g. protection systems in power plants or safety-instrumented systems (SiS) in oil refineries or in chemical plants, are being threatened by new security risks associated to digital technologies and their growing interconnectivity [6]. Beyond the simple identification of similarities and differences between safety and security, a careful analysis of the impacts of their cohabitation is needed. Safety and security have strong interdependencies that can no longer been ignored; nevertheless, their study is still in its early stages. This paper proposes a way to better characterize such relations by using the BDMP (Boolean logic Driven Markov Processes) formalism [7]. Graphically close to fault trees and attack trees, they enable the modeling of dynamic aspects like sequences,

detections and reactions as well as efficient and diverse quantifications. In Section 2, we define the concepts of safety and security used in this paper, and present the stakes and the state of the art regarding the study of their interdependencies. Section 3 gives a brief description of BDMP before explaining how they can help into a better characterization of safety and security relations. Section 4 presents complementary tracks on how such a tool could be leveraged in a more systematic way. Finally, Section 5 identifies different limits but also discusses perspectives related to the proposed approach. II.

CONVERGING AND INTERDEPENDENT ISSUES

A. Definitions Before trying to capture safety and security interdependencies, these two concepts should be defined as they can have different meanings depending on the context in which they are used. In fact, a review of the normative, technical and scientific literature results in dozens of definitions, ranging from slightly different to completely reversed. To address this confusion, we make use of the SEMA referential framework, described in [8]. This conceptual tool does not aim at replacing the terms safety and security, but rather to make their meanings and respective limits explicit in a given context, to avoid misunderstandings. With this objective, safety and security are graphically mapped on a conceptual grid representing the two most common distinctions between safety and security found in the literature. The first axis distinguishes between accidental and malicious risks; the second axis differentiates risks depending on their origin and target. This second axis separates risks originating from the environment and impacting the system, from those coming from the system and impacting the environment. A system to system dimension is added to complete the coverage. As shown in Fig. 1, in our paper, security is related to malicious risks, whereas safety addresses accidental ones. Env.ÖSyst.

Syst.ÖEnv.

Syst.ÖSyst.

Mal.

Abstract— Safety and security issues are increasingly converging on the same critical systems, leading to new situations in which these closely interdependent notions should now be considered together. Indeed, the related requirements, technical and organizational measures can have various interactions and side-effects ranging from mutual reinforcements to complete antagonisms. A better characterization of these interdependencies is needed to ensure a controlled level of risk for the systems concerned by such a convergence. This paper describes the state of the art on this open issue and presents a new approach based on BDMP (Boolean logic Driven Markov Processes), allowing graphical modeling and advanced characterization of safety and security interdependencies. A simple use-case is used through diverse modeling variants, illustrating the capabilities, the contributions but also the limits with respect to other works dealing with safety and security interdependencies. We believe the proposed approach constitutes an original and valuable tool which could find its place in the ongoing research aiming at tackling this open and challenging task.

Defense

Safeguards

Self-protection

Acc.

2

Robustness

Containment Ability

Reliability

Security, as used in this paper

Safety, as used in this paper

Figure 1. Safety and security in the SEMA referential framework

B. Reciprocal influences on methods and tools The similarities between safety and security have been already discussed in the literature [3, 9, 10], fostering the idea to adapt tools and methodologies from one domain to the other. Such cross-fertilization has been particularly fruitful, as illustrated by Table I, with mostly adaptations from the safety engineering domain to the security area. This may be partially explained by the historical precedence of safety as a rigorous engineering discipline. Globally, the adaptations span from architectural concepts (e.g. kernels, use of diversity) to formal methods, passing by testing practices and risk analysis methodologies and tools. On this last aspect, the attack trees are probably the most visible example of such adaptations, they have been largely recognized by the security community since their adaptation from safety [11]. TABLE I.

EXAMPLES OF CROSS-FERTILIZATION RESULTS From safety to security

Safety methodology/tool

Adaptation in security

References*

Fault trees GEMS HAZOP BDMP Zonal analysis Fault tolerance (Diversity, N-Variant...)

Threat trees, attack trees GEMS for security HAZOPs BDMP for attack modeling Security zonal analysis Intrusion-tolerant architectures Diversity-based intrusion detect. From security to safety

[11, 12] [13] [14, 15] [16, 17] [15] [18, 19, 20, 21] [22]

Adaptation in the safety domain

References

Security methodology/tool

Security kernel Non-interference Formal integrity models

Safety kernel Formal fail-safe property Multi-criticality levels policy

[23] [24, 25] [26]

2) Characterization of safety & security interdependencies It is commonly recognized that safe operations may be conditioned by security [5]: for instance, malicious modifications of sensor data or control programs may prevent safety systems from protecting an installation in accidental conditions. Nevertheless, there are subtler and more diverse relations that need to be covered when dealing with safety and security. A good way to illustrate this is to use the classical example of an automatic door shutting system, as found in different versions in [33, 34]. On a safety point of view, such a system should be designed with a fail-safe behavior, leaving the door open in case of failure, allowing evacuation in case of fire or other emergency conditions. From a security point of view, it should be designed with a “fail-secure” behavior, leaving the door locked in case of failure, preventing malicious activity in case the failure had been intentionally provoked. This illustrates possible antagonistic requirements or measures that may apply to a same system if safety and security are considered separately. From a general standpoint, it is possible to identify four categories of interdependencies: •

•

*The references given are related to the adaptation work; they are not exhaustive.

C. The need to go further 1) Converging issues If reciprocal inspiration has produced interesting and useful results, it is now necessary to go beyond and to fully consider the convergence of safety and security issues and its associated consequences. In a growing number of industries, safetycritical systems are becoming exposed to new security risks, originating from the combination of a massive conversion to digital technologies and the generalization of standard network interconnectivity. These evolutions make the historical approach to security that used to be based on physical and logical segregation, separating safety from security issues, obsolete. For instance, Jaatun et al. discuss the emerging security risks on safety-critical systems in the oil and gas industry [27], related to growing remote monitoring and control capabilities, while proposing an architectural approach to address such issues. Aeronautics provides also numerous examples, as identified by [28, 29], whereas similar issues can be found for instance in railways [30], automotive [31] or electricity [32] industries. If safety and security issues used to concern different and separated systems, this is no longer true in many industries; the two issues should now be considered jointly, not only because they concern the same systems in an increasing number of sectors, but also because they have strong interconnections that have to be taken into account.

• •

Conditional dependencies. The most obvious aspect deals with security as a condition to safety, as mentioned in the introduction of this sub-section. Nevertheless, the reciprocal situation, in which safety is a condition for security, exists as well. A generic illustration can be given by unmanaged catastrophic conditions weakening the security posture of a system or an organization, and leading to opportunistic malicious acts. This is particularly relevant when the same resources are shared to ensure both safety and security [35]. Reinforcement. Fortunately, safety and security measures can be complementary and strengthen each other. This is for example the case of event and activity logging, which may be used both for attack detection and accident anticipation, as well as post-event analysis. Antagonism. The shutting door system is a direct example. Ref. [34, 35, 36] provide other relevant illustrations. Independence. Of course, safety and security may also have in some cases no interaction at all.

Such categories can be considered from a design standpoint but also from an operational perspective. They can also be relevant from an attacker point of view. In particular, an attacker can take opportunistic advantage of safety incidents; accidental failures can contribute directly towards the attack objective, may ease the attacker’s task, or be leveraged to lessen the risks of being detected. An attacker can also analyze the safety scenarios and safety cases to design attack plans, not only for the previous reasons but also to increase the consequences of the attack. Beyond examples and generic descriptions, the real challenge is to recognize and characterize such interdependencies at the earliest stage, during the specifications or design phases, in order to manage their consequences and optimize organizational resources and system performance. The challenge is still wide open, and the state of the art can be categorized as follows:

A first category of contributions deals with the organization and the generic processes related to specification and development of systems. In 1999, Eames and Moffett [33], acknowledging the potential conflicts or complementarity between safety and security requirements, proposed an integrated requirement specification methodology to take them into account. In 2005, Lautieri et al. described SafSec, a unified risk assessment methodology aiming at reducing the effort, cost and timescales associated with certification of modular systems [37]. In 2008, Novak et al. proposed a complete life-cycle model [38]. All these contributions stay at a very generic level and do not deal with modeling and quantifications aspects. A second category regroups more targeted investigations. For example, the effects of diversity on safety and security have been studied by Littlewood and Strigini in [39], and more recently by Levitin and Hausken [40] or Komari et al. [41]. Even more targeted, Cho et al. analyze for example the effect of intrusion detection systems on the reliability of missionoriented group communications [42]. These works cover only very specific aspects of the safety-security interaction issue. A third category is related to formal methods. In 2005, Zafar and Dromey applied the Genetic Software Engineering method and the behavioral trees formalism to design a system which must satisfy safety and security properties [43]. In 2009, Sun et al. used the Maude formalism to model the shutting door system previously discussed, and automatically spot contradictory requirements [34]. Unfortunately, these two approaches seem adapted only to simple systems. Finally, to the best of our knowledge, there has only been a single attempt to model safety and security interactions in a rigorous graphical formalism supporting quantification: Fovino et al. described a method to integrate fault trees, capturing accidental risks, with attack trees, capturing malicious risks in [44]. Our proposal described in the rest of the paper can be seen as an alternative approach, going beyond the limits of Fovino’s approach as discussed later in the paper. III.

state spaces. In particular, it allows obtaining relevant qualitative information in the form of the list of sequences leading to the occurrence of the top event. Ref. [7] gives the formal and original definition of BDMP, the demonstration of their mathematical properties, and several examples of their modeling power and ease of use. Fig. 2 represents three small BDMPs modeling in three simplified ways a system adapted from the use-case examined by Fovino et al. [44]. It is constituted by a pipe, transporting a polluting substance (which could be chemical following Fovino’s example) and monitored by a safety-instrumented system (SiS) in charge of stopping the flow in case of problem. Fig. 2 is based on the classical leaves defined in [7]; their macroscopic definition is recalled in Table II. Each BDMP captures a different dynamic behavior of this two component system (Pipe and SiS) which could not be differentiated in classical fault trees, intrinsically static. In Fig. 2a), the SiS has a probability γ to fail when it is activated by a pipe breakdown. In Fig. 2b), the SiS can stop being operational before the pipe breakdown (silent failure, maintenance etc.), the Priority AND gate making the pollution possible only in this order. In Fig. 2c), both alternatives are taken into account. TABLE II. Representation !

I !

Modeled behavior This leaf is used to model a failure in operation, when the modeled component is active. Failure occurs after a time exponentially distributed (parameter λ) and can also be repaired in a time exponentially distributed (parameter μ). This leaf is used to model a failure on demand, likely to arise instantaneously when the leaf changes of mode (activated or not), with a probability γ. Failure can be repaired in a time exponentially distributed (param. μ).

b)

a) Pollution

Pollution

AND

USING BDMP AS A UNIFYING FORMALISM

A. BDMP in reliability engineering and system safety Generally speaking, BDMP can be seen as a formal graphical model which assigns new semantics to the traditional representation of fault trees [7], augmenting it by a new kind of links, the “triggers”. Represented by dotted arrows, triggers enable modeling of sequences and simple dependencies by “activating” sub-trees of the global structure based on the states of some other leaves: the sub-tree pointed to by a given trigger is activated only if the element at the origin of the trigger is true. Moreover, the leaves, modeling system components, are associated to Markov processes, which capture the component behavior depending on the leaf activation (the component is in stand-by or is required). Based on these two principles, the BDMP formalism allows the definition of complex dynamic models while remaining nearly as readable as fault trees, inheriting their hierarchical structure. Common situations like for instance standby redundancies, common cause failures or mutually exclusive failure modes can be modeled very simply and quickly. Moreover, efficient processing can be done for BDMP equivalent to Markov processes with potentially huge

TWO TYPICAL BDMP LEAVES FOR SAFETY ENGINEERING

Accidental_failure Malicious_failure

THEN_(PAND)

I !

!

!

!

Pipe_accidental_breakdown SiS_ondemand_ failure SiS_accidental_failure Pipe_accidental_breakdown

c) Pollution Pollution

OR OR_1

AND THEN_(PAND) THEN_1

!

SiS_accidental_failure SiS_accidental_failure

Ondemand_accidental_failure

!

Pipe_accidental_breakdown Pipe_accidental_breakdown

I !

SiS_on_demand_failure SiS_on_demand_failure

Figure 2. Pipe and SiS accidental failure

B. Security-oriented BDMP The authors have recently adapted BDMP to attack modeling [16, 17]. BDMP also bring, in this context, an advantageous trade-off between readability, modeling power, scalability and quantification capabilities. In particular, security-oriented BDMP inherit the hierarchical and scalable structure from attack trees [11], allowing different depths of analysis and ease of appropriation, but go far beyond by taking into account the dynamics of security and enabling diverse and efficient time-domain quantifications. Similarly to their original definitions, the general idea of BDMP applied to security is to associate a Markov process to each leaf of an attack tree, and introduce the use of triggers, as described in Section III.A. Here, the leaves represent attacker actions, or in some cases security events, as recalled in Table III. They have also several modes, corresponding for example to the fact that the attacker actions that they model are on-going or not yet undertaken, have been detected or can be attempted without notice. At any time, the choice of the mode, corresponding to a Markov process, depends on the value of a Boolean function of some other leaves. These dependences are graphically represented by the triggers. Ref. [17] presents a complete theoretical description and different use-cases of attack modeling. Note that we have chosen different icons in order to differentiate safety and security-oriented leaves. TABLE III. Representation

BASIC BDMP LEAVES FOR SECURITY MODELING Modeled behavior The Attacker Action (AA) leaf models an attacker’s step towards the realization of his/her objective. In Idle mode, the action has not yet been tried. Active mode corresponds to attempts with a time to success exponentially distributed. The Timed Security Event (TSE) leaf models an event the realization of which impacts the attacker’s progress, but which is not under the attacker’s direct control. The time needed for its realization is exponentially distributed. The Instantaneous Security Event (ISE) leaf models an event that can happen instantaneously with a probability γ, when the leaf switches from the Idle to Active mode.

The BDMP represented on Fig. 3 is based on the same system as the one considered previously, with the same undesired top event, but on a malicious perspective: in other words, the SiS and pipe failures leading to the undesired event are now considered as maliciously provoked. One can note that this simple change of perspective has a concrete impact on the modeling: the only logical way to proceed for an intelligent attacker is to disable the SiS first, before attacking the pipe itself, whereas accidental modeling implies the consideration of different sequence orders as illustrated by Fig. 2.

Pollution Pollution

AND Malicious_failure Malicious_failure

SiS_disabled_maliciously SiS_disabled_maliciously

Pipe_broken_maliciously

Figure 3. Pipe and SiS malicious failure

C. Safety and security interactions through BDMP modeling With their capabilities of modeling both safety and securityoriented scenarios, BDMP provide a common basis to consider these traditionally separated issues in integrated graphical models, offering the opportunity to better capture how safety and security interact. We illustrate such an approach in this section on the basis of the simple “Pipe and SiS” use-case adopted previously. When considering the same undesired event, but in a global perspective covering both safety and security, it can be considered through three modalities, discussed in the sections below. Note that for the sake of simplicity, while changing and enriching the models to serve our purpose, they are kept at a macroscopic level; nevertheless, the hierarchical nature of BDMP would make it easy to develop the leaves and make the diversity of the attacker’s alternatives and/or accidental sub-sequences explicit. 1) Pure models: Pure models correspond to either purely accidental or purely malicious situations. They are illustrated by Fig. 2 and 3. While not directly addressing safety and security interdependencies, some relative remarks can be made: • As seen in Section III.B, the change of perspective in building the models leads to different constraints when considering the events order. This stresses the relevance and importance of the ability to capture such ordering aspects, offered by BDMP contrarily to static models like fault trees or attack trees. • The purely accidental scenario is modeled in three different ways, but this corresponds to three different behaviors which the analysts have to choose from, depending on how their wishes to model the system. On the contrary, the purely malicious scenario is presented in a single BDMP as in the chosen breakdown, there’s only one logical sequence to model; nevertheless, it could have been developed further, reflecting the different alternatives and steps offered to the attacker. 2) Hybrid models: Hybrid models imply a combination of accidental and malicious basic events leading to the undesired event. Typical examples are represented in Fig. 4, building upon the previous use-case. Fig. 4a) models an opportunistic behavior of the attacker: as described in Section II.C.2, the attacker waits for the SiS to be out of order to attack the pipe. Several assumptions are needed: firstly, the pipe has to stay in operation despite the outage of the SiS, which may depend on

the level of safety and procedures in place; secondly, the outage of the SiS has to last for at least the time needed for the attacker to succeed in the pipe attack; finally, the attacker has to be able to detect such an outage, which may imply collaboration of insiders or of third parties involved in the maintenance activity. The BDMP of Fig. 4b) models a hybrid scenario in which the attacker disables the SiS, but chooses to let accidental conditions trigger the final step towards his/her final objective, the pollution. This may be done to lessen the risk of detection and depends as well on the SiS outage duration. Note that like in Fig. 2b), we use a Priority AND gate to model the fact that the pollution occurs only if the SiS is disabled first. b)

a) Pollution

Pollution

OR Failure Failure

AND THEN_(PAND)

Hybrid_failure Hybrid_failure I !

OR

OR

SiS_disabled SiS_disabled

Pipe_breakdown Pipe_breakdown !

SiS_disabled_maliciously

SiS_accidental_failure

SiS_ondemand_failure SiS_ondemand_failure !


Pipe_accidental_breakdown

Pollution Pollution

Figure 5. Pipe and SiS integrated models

AND Hybrid__accidental_start Hybrid_accidental_start

THEN_(PAND)

!

!

SiS_accidental_failure

b)




Figure 4. Pipe and protection hybrid models

3) Integrated models: It is finally possible to combine pure and hybrid models into integrated ones, as illustrated by Fig. 5. In Fig. 5a), the model allows to distinguish two different probabilities of instantaneous failure for the SiS, depending whether it has been attacked or not; this is somehow finer than the previous models where a SiS compromise leads to a sure failure. Fig. 5b) does not take this into account, but covers a larger scope of scenarios, including the opportunistic one corresponding to Fig. 4a) and both on-demand and in operation SiS failure. Moreover, it should be stressed that the origin and target of the trigger in the left side of Fig. 5b) could be changed, reflecting different scenarios. For instance, making the trigger start from the leaf “SiS disabled_maliciously” instead of the “SiS_disabled” OR gate would exclude the opportunistic scenario of Section III.C.2. In the present model, the attacker tries to corrupt the SiS but would also notice and take advantage of an accidental failure. a) Pollution Pollution

AND Failure Failure

OR

OR

Pipe_breakdown Pipe_breakdown

SiS_failure SiS_failure SiS_compromised SiS_compromised

!

Pipe_accidental_breakdown Pipe_accidental_breakdown

I !


SiS_ondemand_failure

I !

SiS_attacked_ondemand_failure

4) Quantification capabilities and related considerations: Generally speaking, the interest of BDMP is not only related to the representation of dynamic aspects, but also to the diverse time-domain quantifications that they enable. This includes the probability for the undesired event to occur in a given time or the overall mean time for this event to happen. Moreover, BDMP analysis leads to the enumeration of all the possible sequences leading to the undesired event, ordered by their probability of occurrence in a given time. Such results can be efficiently computed thanks to an original analytical method developed for large Markov models as explained in [16]. Models construction and analysis are made with the KB3 workbench [45]. In addition to the time domain analysis, timeindependent quantifications usually associated to attack trees can also be made [16]. They reflect for instance monetary cost or indicators of specific needs (e.g. insider support, tools). In our case, such capabilities can be used to quantify and compare pure, hybrid and integrated models. These comparisons can be made between alternatives belonging to the same category, changing parameter values or structure configurations, but also in a cross-category setting. In all cases, dynamic and domainrelated specificities such as reparability and maintenance for accidental aspects, and detection/reaction aspects for malicious parts, can be taken into account. Ref. [7] and [17] respectively detail how such aspects are modeled with BDMP. Finally, when dealing with hybrid or integrated models, it is possible to consider different time references to take into account the difference of nature between accidental and malicious events with respect to stochastic modeling. Concretely, the models can be based on a time reference set on the start of the attack, as done in our previous examples. But it is also possible to change this reference, and integrate in the model the time needed before such an attack is started: this is made thanks to an extra leave “attack occurrence” and a trigger, as depicted in Fig. 6. This can introduce a notion of frequency of attack by modeling the related mean time before malicious acts are started. Such an approach is more homogeneous with accidental events stochastic modeling.

a)

b) Pollution Pollution

Pollution Pollution

Attack_occurrence THEN_(PAND)

THEN_(PAND)

!


!




Figure 6. Different time and probabilistic references

5) Examples of quantifications We illustrate in this section the quantification capabilities discussed previously, but also their relevance for safety and security interaction analysis, based on the simple models formerly presented. In this perspective, we parameterize the different leaves following their formal specifications described in [7] and [17], respectively for safety-related and securityrelated events. Such parameterization consists mainly in defining values for in-operation or on-demand failure rates (λs and γs), and attacker’s actions success rates. Other parameters are described in [7] and [17], enabling a finer modeling taking into account for instance repairs, detections and reactions. Mainly for article size reasons, we only add here the initial detection modeling for attacker’s actions when the action is started (probability γD(I)), and the on-going detection modeling, during the attacker’s attempts (parameter λD(O)). The values are arbitrarily chosen for illustration purposes, but they should be defined along classical reliability and statistical indicators for accidental leaves, and along security expert opinion for security-oriented ones. Reasoning in terms of Mean-Time-ToFailure (MTTF) and Mean-Time-To-Success (MTTS) is often useful in this process. Table IV gives the parameter values chosen to characterize the purely malicious model, the hybrid models and one of the integrated models, respectively found in Fig. 3, 4a), 4b) and 5b). TABLE IV. Leaf

Fig.

SiS disabled maliciously

3 4b) 5b)

SiS accidental failure (in op.) Pipe broken maliciously Pipe accidental breakdown SiS on-demand failure

PARAMETERS OF THE MODELS

Parameters (time unit = hour) If not-detected: λS/ND= 4.166x10-2 (MTTS~1 day); Detection parameters: γD(I)=0.5; λD(O)=5.952x10-3 (MTTD~1 week); Once detected: λS/D = 1.377 x10-3 (MTTS~1 month)

4a) 5b) 3 4a) 5b) 4b) 5b)

If not-detected: λS/ND=5.952x10-3 (MTTD~1 week); Detection parameter: γD(I)=0.1; λD(O)= 5.952x10-3 (MTTD~1 week); Once detected: λS/D=0 (stop)

5b)

γ = 10-4

λ = 5.741x10-5 (MTTF~2 years)

λ = 1.148 x10-4 (MTTF~1 year)

With such parameters, we can compute the probability of pollution for each model, for different durations (i.e. mission times). The mission time depends on the frame and the situation modeled: in our case, for instance, the polluting substance may flow in the pipe only for a restricted period, or the attacker may have a deadline to respect. Fig. 7 represents such a probability for the purely malicious model and the two

hybrid models, with three distinct mission times. Mission time has in our case a significant impact, especially regarding the pollution probability in the hybrid scenarios. In the perspective of an attacker targeting pollution, it is more efficient to behave along the purely malicious model when the mission time is short, whereas a hybrid approach may be more interesting when the attacker has no delay: the SiS is compromised, but a pipe accidental failure leads to the pollution. This situation can be explained by the choices made in terms of detection/reaction parameters: in our case, pipe attack detection cancels all chance of success, reflecting for instance a radical change of defense. The SiS attack detection leads only to a raise in difficulty. This may be justified if we consider for example the first SiS attack as a cyberattack, reflecting the attacker’s privileged vector, and substituted by physical attack, more difficult but necessary after the SiS logical new protection, due to the cyberattack detection. 0.7 5.88E-01

0.6

In a month

4.50E-01

0.5 0.4

In a week

In a year

3.44E-01

0.3 0.2

1.91E-01

1.75E-01 5.03E-02

0.1

8.44E-03

0 Purely malicious

Hybrid malicious start

1.62E-02 2.44E-03 Hybrid accidental start

Figure 7. Probability of the undesired event in malicious and hybrid models

In addition, BDMP can yield other kinds of results. In particular, as previously mentioned, all the possible sequences leading to the undesired event can be automatically listed, ordered by their probability of occurrence in a given time. The pure and hybrid models of our use-case are too simple for such results to be of interest, but sequence enumeration provides valuable qualitative and quantitative information in more complex cases. The integrated model of Fig. 5b) is more appropriate to illustrate this capability. Its analysis leads to 27 possible sequences to pollution, taking into account detection aspects, and ordered by their contribution to the overall probability of the undesired event in the chosen mission time. For a mission time of one year, pollution has a global probability of 0.75 to occur. This high value is partly linked to the all-hazard perimeter of the analysis, but also to the time reference adopted, set on a determinist and immediate start of attack. As discussed in Section III.C.4, the notions of attack frequency and mean time before attack could also be modeled. Table V shows a representative excerpt of the complete list of sequences. The two first sequences are purely malicious and weigh around 52% of the overall pollution probability. The two next ones are hybrid with a malicious start and contribute for 32%. Then one has to wait sequence #9 before finding the first hybrid sequence with accidental start, followed by the first purely accidental one (#10). Quite naturally, purely accidental sequences have here a very limited contribution to the pollution odds in a year. Globally, most of the sequences include one or more unnecessary actions or events that have no effect on the pollution occurrence. A finer analysis would imply focusing on the consolidated contributions of the minimal sequences.

TABLE V.

SELECTION OF SEQUENCES AND ASSOCIATED QUANTIFICATION FOR THE INTEGRATED MODEL OF FIG. 5B) Average duration 7.23E+02 1.04E+02 8.80E+03 9.42E+03

Contrib.

SiS attack initial detection, SiS attack success (detected), Pipe attack success (undetected) SiS attack success (undetected), Pipe attack success (undetected) SiS attack success (undetected), Pipe attack ongoing detection, Pipe accidental breakdown SiS attack initial detection, SiS attack success (detected), Pipe attack ongoing detection, Pipe acc. breakdown

Probability in a year 1.981E-01 1.943E-01 1.222E-01 1.187E-01

SiS attack initial detection, SiS accidental failure, Pipe attack success (undetected) SiS attack initial detection, SiS accidental failure, Pipe attack ongoing detection, Pipe accidental breakdown

8.20E-03 4.92E-03

7.29E+02 9.42E+03

1.1 % 0.6 %

SiS attack initial detection, SiS accidental failure, Pipe attack initial detection, Pipe accidental breakdown

1.11E-03

9.34E+03

0.1%

#

Sequences

1 2 3 4

26.2 % 25.7 % 16.2 % 15.7%

…

9 10 …

14 …

D. Position with respect to the state of the art As stated in Section II.C.2, the state of the art in terms of graphical models taking into account safety and security interactions is limited, to the best of our knowledge, to the approach proposed by Fovino et al. [44]. It describes how fault trees and attack trees can be merged to obtain extended fault trees, integrating accidental basic events and malicious acts in a logical tree leading to a given undesired top event. Our approach can be seen in some way similar, but it has several advantages. It captures dynamic dimensions such as simple dependencies and sequences, repairs, detections and reactions, staying graphically close to fault trees and attack trees. These capabilities offer more accurate modeling of safety and security scenarios but also, serving the objective of this paper, modeling a richer scope of interactions between accidental and malicious events. Such diversity is reflected by the use-cases developed in Section III.C. Finally, our approach supports diverse timedomain quantifications in addition to the ones classically offered by static trees. It produces an ordered list of sequences as described in Section III.C.4, and allows different time references for the stochastic parameters, as developed also in Section III.C.4. It may also be of interest to adopt a broader perspective and examine the other categories of work dealing with safety and security interactions described in Section II.C.2. Regarding the generic specifications and development methodologies, our approach may actually be used into different phases of such methodologies, as a tool enabling finer characterization of safety-security interactions. When it comes to the more targeted studies such as diversity effects on safety and security, they could take advantage of our proposal in order to model and quantify different options for a given architecture. Finally, BDMP modeling does not pretend to identify automatically conflicting situations or to cover exhaustively the scope of possible interactions like the model-checking based approaches. Nevertheless, such approaches are limited to small and simple systems, whereas BDMP are scalable and well adapted to complex systems: models with more than a hundred leaves are commonly processed for reliability studies [45].

IV.

TOWARDS A SYSTEMATIZED APPROACH

A. Going back to the SEMA decomposition As shown in Section III.C, there are many ways to combine safety and security events in relevant configurations during a risk assessment; it may be difficult to cover such diversity in a rigorous and complete way. Moreover, it may be necessary to consider other aspects of safety and security, implying different configurations with respect to accidental vs. malicious events, and system vs. environment impacts than the one used in this paper. Help can be found in the SEMA referential framework, used in Section II.A to define safety and security. As described in [8], in addition to avoiding ambiguities when dealing with safety and security, this framework can also be seen as a way to decompose the risk space from a holistic point of view. Although it is not demonstrated in this paper, it is possible for elaborate systems to build BDMP models corresponding to each of the six sub-notions of the SEMA referential framework, and then go beyond by considering hybrid and integrated BDMP models covering incrementally several sub-notions in a systematic way. Throughout this process, the top event considered can be changed: the objective is not to end up with a big and unique model but rather to obtain a set of relevant ones, covering the diversity of the risk scenarios. B. Supporting the analyst into side-effects identification Hybrid and integrated BDMP models are helpful in modeling and characterizing situations where safety and security interact closely. They can provide a valuable tool in a risk assessment process aiming at covering safety and security issues [33, 37, 38]. Once the risks are identified, risk management decisions have to be made, ranging from risk acceptance to risk reduction supported by the definition of counter-measures. Unfortunately, hybrid and integrated BDMP models as presented in Section III.C do not directly help to identify unforeseen side-effects and hidden interdependencies when counter-measures are designed. We propose in this section a simple approach to help the analyst in this objective. Risk reduction strategies can be quantified and compared by changing specific leaves parameters in the corresponding BDMP, modeling the effect of counter-measures. We propose to add a system component dimension to the models, allowing the analyst to consider the impact of a counter-measure deployed in a given model on the other BDMPs covering different aspects of risk. Fig. 8 gives a simple illustration of this

idea. The upper part schematically represents the components of the new system modeled, which corresponds to the SiS of the previous use-cases. It is made of a programmable logic controller (PLC, labeled P), a sensor (S), enabling the detection of abnormal flow conditions in the pipe, and an actuator (A) capable of stopping the flow in case of incident. The left side and the right side BDMPs model respectively accidental failures and attack scenarios of the SiS, with two new aspects: •

•

Under each leaf, a list of the components at stake is indicated. For the safety-oriented BDMP, this is rather straightforward, as a leaf models a single component; for the security-oriented BDMP, the correspondence may be less straightforward, even if the present case is simple. In a notation similar to the one proposed by Bistarelli [46] for defense trees and found in other references [47, 48], dotted boxes indicate the counter-measures decided by the analyst in his attempts to reduce risk when examining a given BDMP model. Such dotted boxes are graphically linked to the leaves of which they change the parameters.

SiS_disabled SiS_accidental_failure

OR

OR

!

!

!

Sensor

Actuator Actuator < A>

PLC PLC < P>

OR Spoof_Sensor

PSTN Modem

Actuator_DoS

< S, P>

PLC_oriented

< A>

Sensor Authentication PLC_DoS

Thresholds_changed

Figure 8. Augmenting the models to better identify interdependencies

We can now illustrate the use of these two features to help identify safety and security interdependencies in our very simple example. Let’s assume that the analysis of the safetyoriented BDMP has shown that the SiS accidental failure risk can be largely reduced by enhancing the PLC mean time to repair (μ parameter). This can be achieved by a better monitoring and a faster access for the experts. In this perspective, the analyst proposes to replace the legacy and closed PLC remote diagnostic connection by a dial-up modem connected on the public switching telephone system (PSTN), as represented in Fig. 8. Before any decision is taken, the impact of such a change can be analyzed in the security-

oriented BDMP by leveraging the component lists associated to its leaves. The impact of this change is analyzed on the leaves of the security-oriented BDMP specifically implying the PLC: they reduce the level of security and should lead to a parameter change reflecting such a fact. It is then possible to make an informed decision about the dial-up modem deployment, knowing qualitatively that the change will impact both safety and security, but also if needed, with quantitative inputs computed thanks to the BDMP models. Conversely, the analysis of the security-oriented BDMP may lead to the idea of deploying authentication between the PLC and the Sensor to reduce spoofing possibilities: a similar process as the one previously described will enable us to identify and if necessary quantify potential impacts on the safety-oriented BDMP. V.

PERSPECTIVES

A. Perspectives on BDMP as a modeling tool There are several enhancements to the BDMP formalism but also to its treatment that may enhance the characterization of safety and security interactions. A first perspective lies in the use of multi top-events BDMP, integrating in the same model distinct attacker objectives and safety-related undesired events; for the time being, our models are based on a unique common top event. A second perspective deals with the native Markovian framework of BDMP, enabling efficient and controlled quantification by sequence exploration algorithms, but which may not be appropriate in some situations. Ref. [16] discusses its relevance for security applications and mentions the possibility of generalizing BDMP into BDSP (Boolean logic-Driven Stochastic Processes), allowing the use of other distributions than instantaneous or exponential ones. This generalization is still to be formalized: quite straightforward, it would mainly imply basing quantifications on Monte-Carlo simulation rather than on sequence exploration. Finally, support tools helping the analyst to take advantage of the results yielded by the BDMP models are under-development. A particular example deals with sensitivity analysis of the different leaves, for which the methods are not as well established as for static models [49]. B. General limits and the way forward Model-based evaluation methods like the one described in this paper assume that quantitative parameters can be chosen to reflect the behavior of the system under study. This is a common practice in the safety and reliability area, where failure rates and other parameters can be evaluated on historical and statistical basis, and for which the stochastic nature of the modeled events is admitted. This is rather different in the security domain. Data are scarce, and from a more epistemological point of view, the relevance of capturing an attacker behavior with stochastic modeling is still an open issue (cf. [16] for more elements on this debate). Nevertheless, if quantitative methods cannot play the same pivotal role in security as in the safety area, they already provide a helpful tool for the security analyst and have found their place in different risk assessment methodologies (cf. [50, 51] for attack trees).

Moreover, as mentioned previously, the proposed approach cannot pretend to exhaustively cover safety-security interactions scenarios; in fact, the model construction is highly dependent on the analyst skills, experience and rigor. This limit is clearly inherited from the attack tree and fault-tree formalisms and should be kept in mind when used in practice. It also underlines the fact that BDMP modeling should be considered within broader risk assessment and system development methodologies, like the ones mentioned in Section III.D.

[4]

Finally, in this paper, we have based our examples on a concrete and physical top event, but it may be of interest to explore more abstract ones dealing with security and safety properties, such as integrity, availability or confidentiality. This will be the object of future work.

[8]

VI.

[5]

[6]

[7]

[9]

CONCLUSION

Modeling and characterizing safety and security interdependencies is a challenging and crucial topic. In the past treated by two different communities and considered for distinct systems, safety and security issues are rapidly converging. The progressive recognition of their intimate link has lead to several fruitful reciprocal inspirations in terms of modeling tools, architectural concepts or risk assessment methodologies. Nevertheless, these cross-fertilization efforts do not address the core issue related to safety and security convergence: such a convergence gives rise to new situations in which the interactions between safety and security influence directly, and sometimes in a surprising way, the overall level of risk. The scope is wide, ranging from mutual reinforcement to frontal antagonism, but the state of the art regarding the characterization of these interactions is still limited. We propose to use the BDMP dynamic formalism to model risk scenarios which combine safety and security aspects, supporting qualitative and quantitative characterization. The proposed graphical modeling formalism is visually close to fault trees and attack trees, but goes far beyond by allowing modeling of complex dynamic situations (including repairs, attack detections and reactions, sequences) and supporting extensive time domain quantifications. We believe these capabilities can contribute to a better characterization of the safety and security interdependencies, and a better control of the associated consequences.

[10]

[11] [12]

[13]

[14]

[15] [16]

[17]

[18]

[19]

ACKNOWLEDGMENT L.P.C. would like to thank Claude Chaudet, from Telecom ParisTech, for his support and feedback.

[20]

REFERENCES

[21]

[1]

[2]

[3]

E. Schoitsch, “Design for safety and security of complex embedded systems: a unified approach,” in Proc. NATO Advanced Research Workshop on Cyberspace Security and Defense: Research Issues, Gdansk, Poland, Sep. 2004, pp. 161–174. E. Jonsson and T. Olovsson, “On the integration of security and dependability in computer systems,” in Proc. IASTED Int. Conf. Reliability, Quality Control and Risk Assessment, Washington, D.C, USA, 1992, p. 93–97. D. F. C. Brewer, “Applying security techniques to achieve safety,” in Proceedings of the Safety-Critical Systems Symposium (SSS’93). Bristol, U.K., Feb. 1993.

[22]

[23] [24]

[25]

J. Cullyer, “The technology of safety and security,” The Computer Bulletin, vol. 5, no. 5, 1993. M. B. Line, O. Nordland, L. Røstad, and I. A. Tøndel, “Safety vs. security?” in Proc. 8th International Conference on Probabilistic Safety Assessment and Management (PSAM 2006), New Orleans, USA, 2006. T. Stauffer and C. Fialkowsi, “Safety & security: Can you have the best of both worlds?” in Proc. ISA 64th Annual Instrumentation Symposium for the Process Industries, Texas, USA, 2009. M. Bouissou and J.-L. Bon, “A new formalism that combines advantages of fault-trees and Markov models: Boolean logic driven Markov processes,” Reliability Engineering & System Safety, vol. 82, no. 2, pp. 149–163, 2003. L. Piètre-Cambacédès and C. Chaudet, “The SEMA referential framework: avoiding ambiguities when dealing with security and safety issues,” in International Journal of Critical Infrastructure Protection, vol. 3, no. 2, 2010. B. Littlewood, S. Brocklehurst, N. Fenton, P. Mellor, S. Page, D. Wright, J. Dobson, J. McDermid, and D. Gollmann, “Towards operational measures of computer security,” Journal of Computer Security, vol. 2, p. 211–229, 1993. J. Rushby, “Critical system properties: Survey and taxonomy,” Reliability Engineering & System Safety, vol. 43, no. 2, pp. 189–219, 1994. B. Schneier, “Attack trees: Modeling security threats,” Dr. Dobb’s Journal, vol. 12, no. 24, pp. 21–29, 1999. J. D. Weiss, “A system security engineering process,” in Proc. 14th National Computer Security Conference, Washington D.C., USA, Oct 1991. S. Brostoff and M. A. Sasse, “Safe and sound: a safety-critical approach to security,” in Proceedings of the 2001 Workshop on New Security Paradigms (NSPW’01), Cloudcrofl, USA, Sep. 2001, pp. 41–51. R. Winther, O.-A. Johnsen, and B. A. Gran, “Security assessments of safety critical systems using HAZOPs,” in Proceedings of SAFECOMP 2001, LNCS 2187, Budapest, Hungary, Sep. 2001, pp. 14–24. T. Srivatanakul, “Security analysis with deviational techniques,” Ph.D. dissertation, University of York, 2005. L. Piètre-Cambacédès and M. Bouissou, “Beyond attack trees: dynamic security modeling with Boolean logic Driven Markov Processes (BDMP),” in 8th European Dependable Computing Conference (EDCC), Valencia, Spain, Apr. 2010, pp. 199–208. L. Piètre-Cambacédès and M. Bouissou, “Attack and defense dynamic modeling with BDMP,” in Proc. 5th Int. Conf. on Math. Methods, Models, and Architectures for Computer Networks Security (MMMACNS-2010), LNCS 6258, St Petersburg, Russia, Sep. 2010, pp. 86–101. J. E. Dobson and B. Randell, “Building reliable secure computing systems out of unreliable insecure components,” in Proc. IEEE Symposium on Security and Privacy (S&P’86), Oakland, USA, Apr. 1986, pp. 187–193. Y. Deswarte, L. Blain, and J.-C. Fabre, “Intrusion tolerance in distributed systems,” in Proc. IEEE Symposium on Security and Privacy (S&P’91), Oakland, USA, May 1991, pp. 110–121. D. Powell (ed.), “Delta-4: a generic architecture for dependable distributed computing,” Research Reports ESPRIT series, SpringerVerlag, 1991. Y. Deswarte and D. Powell, “Intrusion tolerance for internet applications,” in Proceedings of the IFIP World Computer Congress, IFIP Volume 156/2004, Toulouse, France, Aug. 2004, pp. 241–256. E. Totel, F. Majorczyk, and L. Mé, “COTS diversity based intrusion detection and application to Web servers,” in Proc. Int. Conf. on Recent Advances in Intrusion Detection (RAID’05), Seattle, USA, Sep. 2005. J. Rushby, “Kernels for safety?” in Proc. Safety-critical Systems Symposium (SSS’86), Glasgow, U.K., Oct. 1986, pp. 210–220. V. Stavridou and B. Dutertre, “From security to safety and back,” in Proc. Computer Security, Dependability and Assurance: From Needs to Solutions (CSDA’98), Washington, D.C., USA, Nov. 1998, pp. 182–195. A. Simpson, J. Woodcock, and J. Davies, “Safety through security,” in Proceedings of the 9th International Workshop on Software Specification and Design (IWSSD ’98), Japan, Apr. 1998, pp. 18–24.

[26] E. Totel, J.-P. Blanquart, Y. Deswarte, and D. Powell, “Supporting multiple levels of criticality,” in Proc. IEEE Symp. on Fault Tolerant Comp. Systems (FTCS-28), Munich, Germany, Jun. 1998, pp. 70–79. [27] M. G. Jaatun, M. B. Line, and T. O. Grotan, “Secure remote access to autonomous safety systems: A good practice approach,” International Journal of Autonomous and Adaptive Communications Systems, vol. 2, no. 3, pp. 297–312, 2009. [28] R. Robinson, M. Li, S. Lintelman, K. Sampigethaya, R. Poovendran, D. von Oheimb, J.-U. Bußer, and J. Cuellar, “Electronic distribution of airplane software and the impact of information security on airplane safety,” in Proceedings SAFECOMP’07, LNCS 4680, Nuremberg, Germany, Sep. 2007, pp. 28–39. [29] N. Neogi, “Safety and security in the next generation air transportation system,” in Proc. National Workshop on Aviation Software Systems, Alexandria, USA, Oct. 2006. [30] R. S. Smith, J. and M. Looi, “Security as a safety issue in rail communications,” in Proc. of the 8th Australian Workshop on Safety Critical Systems and Software, Canberra, Australia, 2003, pp. 79–88. [31] R. G. Herrtwich, “Automotive telematics - road safety vs. IT security? (invited talk),” in Proc. SAFECOMP’04, LNCS 3219. Potsdam, Germany, 2004, p. 239. [32] E. Johansson, T. Sommestad, and M. Ekstedt, “Security isssues for SCADA systems within power distribution,” in Proc. of the Nordic Distribution and Asset Management Conference (NORDAC’08), Bergen, Norway, Sep. 2008. [33] D. P. Eames and J. Moffett, “The integration of safety and security requirements,” in Proceedings of SAFECOMP’99, LNCS1698, Toulouse, France, Sep. 1999, pp. 468–480. [34] M. Sun, S. Mohan, L. Sha, and C. Gunter, “Addressing safety and security contradictions in Cyber-Physical Systems,” in Proceedings of the 1st Workshop on Future Directions in Cyber-Physical Systems Security (CPSSW’09), Newark, USA, Jul. 2009. [35] G. Deleuze, E. Châtelet, P. Laclémence, J. Piwowar, and B. Affeltranger, “Are safety and security in industrial systems antaonistic or complementary issues?” in Proceedings of ESREL’07, Valencia, Spain, Sep. 2007. [36] J. Jalouneix, P. Cousinou, J. Couturier, and D. Winter, “A comparative approach to nuclear safety and nuclear security,” IRSN, Tech. Rep. 2009/117, Apr. 2009. [37] S. Lautieri, D. Cooper, and D. Jackson, “SafSec: Commonalities between safety and security assurance,” in Proc. 13th Safety Critical Systems Symposium (SSS’05), Southampton, UK, Feb. 2005, pp. 65–75. [38] T. Novak and A. Treytl, “Functional safety and system security in automation systems - a life cycle model,” in Proc. 13th IEEE Conf. on Emerging Technologies and Factory Automation (ETFA’08), Hamburg, Germany, Sep. 2008, pp. 311–318.

[39] B. Littlewood and L. Strigini, “Redundancy and diversity in security,” in Proc. European Symp. on Research in Comp. Security (ESORICS’04), LNCS 3193, Sophia Antipolis, France, Sep. 2004, pp. 423–438. [40] L. Gregory and H. Kjell, “Redundancy vs. protection in defending parallel systems against unintentional and intentional impacts,” IEEE Transactions on Reliability, vol. 58, no. 4, pp. 679–690, Dec. 2009. [41] I. E. Komari, V. Kharchenko, A. Romanovsky, and E. Babeshko, “Diversity and security of computing systems: Points of interconnection (part 1 and 2),” MASAUM Journal of Open Problems in Science and Engineering, vol. 1, pp. 28–41, 2009. [42] J.-H. Cho, I.-R. Chen, and P.-G. Feng, “Effect of intrusion detection on reliability of mission-oriented mobile group systems in mobile ad hoc networks,” IEEE Transactions on Reliability, vol. 59, no. 1, pp. 231– 241, Mar 2009. [43] S. Zafar and R. Dromey, “Integrating safety and security requirements into design of an embedded system,” in Proc. 12th Asia-Pacific Software Engineering Conference (APSEC’05), Taipei, Taiwan, Dec 2005. [44] I. N. Fovino, M. Masera, and A. De Cian, “Integrating cyber attacks within fault trees,” Reliability Engineering & System Safety, vol. 94, no. 9, pp. 1394–1402, Sep. 2009. [45] M. Bouissou, “Automated dependability analysis of complex systems with the KB3 workbench: the experience of EDF R&D,” in Proc. International Conference on Energy and Environment (CIEM’05), Bucharest, Romania, Oct. 2005. [46] S. Bistarelli, F. Fioravanti, and P. Peretti, “Defense trees for economic evaluation of security investments,” in Proc. 1st Int. Conf. on Availability, Reliability and Security (ARES’06), Vienna, Austria, Apr. 2006, pp. 416–423. [47] W. J. Caelli, D. Longley, and A. B. Tickle, “A methodology for describing information and physical security architectures,” in Proc. 8th IFIP TC11 International Conference on Information Security (SEC’92), ser. IFIP Transactions, vol. A-15, Singapore, May 1992, p. 277–296. [48] M. Howard and D. LeBlanc, Writing Secure Code, 2nd ed. Microsoft Press, 2002. [49] Y. Ou and J. B. Dugan, “Approximate sensitivity analysis for acyclic Markov reliability models,” IEEE Transactions on Reliability, vol. 52, no. 2, pp. 220–230, Jun 2003. [50] N. Mead, E. Hough, and T. Stehney, “Security quality requirements engineering (SQUARE) methodology,” Carnegie Mellon University, Tech. Rep. CMU/SEI-2005-TR-009, 2005. [51] S. Evans, D. Heinbuch, E. Kyule, J. Piorkowski, and J. Wallner, “Riskbased systems security engineering: stopping attacks with intention,” IEEE Security and Privacy, vol. 2, no. 6, p. 59–62, 2004.