Nondeterminism: Unsafe? - Semantic Scholar

14 downloads 417 Views 262KB Size Report
of one such tool is discussed here. Index ... Optimization packages may use some random restarts ..... LURCH is a temporal search engine for models of finite-.
Nondeterminism: Unsafe? Tim Menzies Member,IEEE, David Owen, Mats Heimdahl Member,IEEE, Jimin Gao, Bojan Cukic Member,IEEE

Abstract— A class of software is identified where a nondeterminism analysis is not inherently less safe than a deterministic analysis. Tools built for that class of software can be simple to build, fast to run, and scale to very large problems. An examples of one such tool is discussed here. Index Terms— Nondeterminism, randomization, LURCH, verfication,design debugging, temporal logic

spaghetti:

safety,

I. I NTRODUCTION ONVENTIONAL algorithms are deterministic and contain the hard-wired decision paths of a one-way function for converting inputs to outputs. A nondeterministic algorithm lacks such hard-wired paths and, hence, the same inputs at different times can give rise to different outputs. There are many applications for nondeterministic algorithms. Optimization packages may use some random restarts to avoid local maxima. Model-based diagnosis tools may nondeterministic explore mutually exclusive, but equally plausible, solutions [1]. Stochastic algorithms can design circuits [2] and nondeterministic genetic algorithms can learn predictors for software development times [3]. Design debugging tools like LURCH [4] can perform a stochastic search for property violations in a formal model. The outputs of a nondeterministic algorithm cannot be predicted with complete accuracy. Many test engineers therefore advise against nondeterministic analysis since it introduces a degree of uncertainty into our assessment of software. For example, Leveson cautions that “nondeterminacy is the enemy of reliability” [5]. Despite these cautions, nondeterministic is widely used. One reason for this is that the nondeterministic algorithm may be the simplest algorithm available, or the fastest, or both [6]. Probabilistic skip lists are far simpler to code than AVL trees [7]. Randomized min-cut is simple to code and runs faster than the best known algorithms based on network flow [6]. Large and complex planning problems that defeat best known deterministic methods can be solved via stochastic search [8]. LURCH’s implementation is simpler than standard model checkers, runs much faster (seconds and minutes as opposed to hours); uses less memory (1-10MBs as opposed to 10-100MBs); and can still find the vast majority of faults. The widespread use of nondeterministic analysis prompts the question : is a nondeterministic analysis always less safe

C

Tim Menzies is with the Department of Computer Science, Portland State University, P.O. Box 751 Portland, Oregon 97207-0751 [email protected]. Dr. Menzies’ web site is http://menzies.us David Owen and Bojan Cukic are with Lane Dept. of Computer Science and Electrical Engineering West Virginia University Morgantown, WV 26506 [email protected], [email protected] Mats Heimdahl and Jimin Goa are with the Dept. of Computer Science & Engineering, University of Minnesota, Minneapolis, MN 55455, heimdahl@ cs.umn.edu,[email protected]

simpler: Fig. 1.

a −→ b −→ c −→ d −→ e input1 −→f −→ g −→ h −→ i −→ j −→ goal1 input2 −→k → ¬g −→ l −→ m → ¬j −→ goal2 ¬a −→ n −→ o −→ p −→ ¬e

Dependencies between variables in two programs.

than a deterministic one? At its core, this is an issue of sampling. Any nondeterministic analysis samples some subset of the possible behaviors of a piece of software. Nondeterministic analysis is less safe than deterministic analysis when nondeterministic sampling misses detects that could be found by deterministic analysis. There is at least one case where a nondeterministic analysis will sample as much of a space as a deterministic analysis. Suppose the number of choices in a system are few. In that case, randomly exploring just some of the choices will find as much as thoroughly exploring all choices. For example, a nondeterministic selection of choices within the complex spaghetti system of Figure 1 may never find all the reachable states. On the other hand, the topology of the simpler model in Figure 1 means that it is easy to find the goals. The reason for this is that most of the 215 = 32768 possible states in the simpler model are unreachable. For example, when searching for the goals from the inputs, the (a or ¬a) and (e or ¬e) choices are unreachable and ignorable. The reachable choices are (g or ¬g) and (j or ¬j). Since {j, ¬j} are fully dependent on {g, ¬g} (respectively), then there are only two real choices in this system: (g or ¬g). This article argues that software is usually simpler, and not spaghetti. Often, the reachable states of software clump around a small number of very frequently reached states. For software with clumps, the implications of the nondeterministic choices are very few- just one of the clumped states. In the simpler model there are two clumped states: 1) g enables the state input1, f, g, h, j, goal11 ; 2) ¬g enables the state input2, k, ¬g, l, m, ¬j, goal2. If it could be determined that a particular piece of software has clumps, then it would be safe to use very simple nondeterminisitic tools like LURCH. This article will present analytical and empirical evidence that clumps occur frequently. Hence, nondeterministic tools like LURCH can as compentent as deterministic tools. We will show that such tools are not only easier to build but, by some measures, are be superior 1 For simplicity sake, this state does not show the variables uninfluenced by the inputs which would keep their default assignments.

MENZIES,OWEN,HEIMDAHL, GAO, CUKIC NONDETERMINISM: UNSAFE?; SUBMITTED TO IEEE TSE, MAY 14, 2004

2

Y=% modules needing X paths 100 75 50 25 0