An Artificial Neural-Network-Based Approach to ... - IEEE Xplore

An Artificial Neural-Network-Based Approach to Software Reliability Assessment Yu-Shen Su, Chin-Yu Huang, Yi-Shin Chen, and Jing-Xun Chen Department of Computer Science National Tsing Hua University Hsinchu, Taiwan Abstract—In this paper, we propose an artificial neuralnetwork-based approach for software reliability estimation and modeling. We first explain the network networks from the mathematical viewpoints of software reliability modeling. That is, we will show how to apply neural network to predict software reliability by designing different elements of neural networks. Furthermore, we will use the neural network approach to build a dynamic weighted combinational model. The applicability of proposed model is demonstrated through four real software failure data sets. From experimental results, we can see that the proposed model significantly outperforms the traditional software reliability models.

Index-term—Neural Network, Software Testing, Software Reliability, Combinational Model.

I. INTRODUCTION In modern society, computers are used for many different applications, such as nuclear reactors, aircraft, banking systems, and hospital patient monitoring systems. As the demand of the application quality becomes higher and higher, the research of the computer software reliability becomes more and more essential. The software reliability is defined as the probability that the software will operate without a failure under a given environmental condition during a specified period of time [1]. To date, the software reliability assessment is one of the most important processes during the software development. Since 1970, many software reliability growth models (SRGMs) [2-4] have been proposed. In general, there are two major types of software reliability models: the deterministic and the probabilistic. The deterministic one is employed to study the number of distinct operators and operands in the program. The probabilistic one represents the failure occurrences and the fault removals as probabilistic events. The probabilistic models can be further classified into different classes, such as error seeding, failure rate, and non-homogeneous Poisson process (NHPP). Among these classes, the NHPP models are the most popular ones. The reason is the NHPP model has ability to describe the software failure phenomenon. The first NHPP model, which strongly influences the development of many other models, was proposed by Goel and Okumoto [6]. Later, Ohba [7] presented a NHPP model with S-shaped mean value function. Yamada and Osaki [8, 9] also made further progress in various S-Shaped NHPP models. Although these NHPP models are widely used, they impose certain restrictions or a priori assumptions about the nature of software faults and the stochastic behavior of software failure process. To overcome this problem, several alternative

solutions are introduced. One possible solution is to employ the neural network, since it can build a model adaptively from the given data set of failure processes. Many researchers [10-20] have been successfully adapted neural networks to software reliability issues. Motivated by these successful cases, we employ the neural network to solve the problems for software reliability assessment. The major contributions of this paper are: • Derive mathematic expressions that can be applied to neural networks from traditional software reliability models. • Propose a generic model for all software projects. This proposed model is adaptive based on the characteristic of the given data set. The rest of this paper is organized as follows. Section II gives an overview of neural networks. In Section III, we present our methodology. Section IV depicts the results of our evaluation as compared to the existing SRGMs. Section V concludes the paper.

II. OVERVIEW OF NEURAL NETWORKS The software reliability models are widely employed to assess the software reliability. However, since most software reliability models embed certain restrictions or assumptions, selecting an appropriate model based on the characteristics of the software projects is challenging. In order to locate the suitable model, two approaches are adapted. The first one is to design a guideline, which could suggest fitting models for software projects. The other is to select the one with the highest confidence after various assessments. Basically, these two approaches both require users to manually opt for candidates. The decision-making processes would be a huge overhead while the software projects are huge and complicated. In order to reduce such overhead, researchers proposed an alternative approach that can adapt the characteristics of failure processes from the actual data set by using neural networks. For example, Karunaithi et al. [11] applied some kinds of neural network architecture to estimate the software reliability and used the execution time as input, cumulative the number of detected faults as desired output, and encoded the input and output into the binary bit string. The results showed that the neural network approach was good at identifying defect-prone modules software failures. Khoshgoftaar et al. [14-15] ever used the neural network as a tool for predicting the number of faults in programs. They introduced an approach for static reliability modeling and concluded that the neural networks produce models with better quality of fit and predictive quality. In addition, Cai et al. [19]

examined the effectiveness of the neural network approach in handling dynamic software reliability data overall and present several new findings. They found that the neural network approach is more appropriate for handling datasets with `smooth' trends than for handling datasets with large fluctuations and the training results are much better than the prediction results in general. Next, we briefly introduce the concept of the neural networks. Neural networks are learning mechanisms that can approximate any non-linear continuous functions based on the given data. In general, neural networks consist of three components as follow: 1. Neurons: each neuron can receive signal, process the signals and finally produce an output signal. Fig. 1 depicts a neuron, where f is the activation function that processes the input signals and produces an output of the neuron, x are the outputs of the neurons in the previous layer, and w are the weights connected to the neurons of the previous layer. n

x1 x2

xn

neti = b + ∑ x j w j

w1 w2

j =1

∑

I

f (neti )

wn Activation function Bias b Fig. 1 A neuron

2. Network architecture: The most common type of neural network architecture is called feed-forward network shown in Fig. 2. This architecture is composed of three distinct layers: an input layer, a hidden layer, and an output layer. Note that the circles are represented as neurons and the connection of neurons across layers is called the connecting weight. Input layer

Hidden layer

xi xN

wij1

hH

w0jk

,M ,

(2)

j =1

where w0jk is the output “weight” from hidden layer node j to output layer node k, bk is the bias of the node k, hj is the output of the hidden layer, and g is an activation function in output layers. The values of the hidden layer are given by N

h j = f (b j + ∑ wij1 xi ), j = 1, , H ,

(3)

i =1

where wij1 is the input “weight” from input layer node i to hidden layer node j, bj is the bias of the node in the hidden layer, xi is the value at input node i, and f is the activation function in hidden layer. The approximated function of the neural networks can be considered as some compound functions, which can be rewritten as nested functions, such as f(g(x)). Because of this feature, the neural network can be applied to software reliability modeling since software reliability modeling is likely to build a model to explain the software failure behavior.

III. A NEURAL-NETWORK-BASED APPROACH FOR SOFTWARE RELIABILITY MODELING

yj

yM

A. The Derivations of Proposed Approach in Software Reliability Modeling

y1

hi

H

yk = g ( bk + ∑ w0jk h j ), k = 1,

As we mentioned in the previous section, the objective function of the neural network can be considered as compound functions. In other words, if we can derive a form of compound functions from the conventional software reliability models, we can build a neural-network-based model for software reliability. In Section III.A, the deriving processes are described one by one. In Section III.B, we will further demonstrate a generic neural-network-based model that can combine existing models. The implementation details of this approach are illustrated in Section III.C.

Output layer

x1

the neural network in a mathematical form. The objective of the neural networks is to approximate an non-linear function that can receive the vector (x1, …, xn ) in RN and output the vector (y1 ,.., ym) in RM. Thus, the network can be denoted as: (1) y = F ( x) , where x=(x1, …, xn ) and y= (y1,.., ym). The values of yk are given by

Fig. 2 A feed-forward network

3. Learning algorithm: The algorithm describes a process to adjust the weights. During the learning processes, the weights of network are adjusted to reduce the errors of the network outputs as compared to the standard answers. The back-propagation algorithm is the most widely employed one. In back-propagation algorithm, the weights of the network are iteratively trained with the errors propagated back from the output layer. Subsequently, we describe the learning algorithm of

We first consider the logistic growth curve model [3]. This model simply fits the mean value function with a form of the logistic function. Its mean value function is given by: a (4) m(t ) = , a > 0, b > 0, k > 0 . 1 + ke − bt We can derive a form of compound functions from its mean value function as the following: Replace k with e-c: a a a . = = m(t ) = 1 + ke − bt 1 + e − c e− bt 1 + e− ( bt +c ) Assume that (5) g ( x ) = bx + c ,

f ( x) = and

1 , 1 + e− x

(6)

(7) m( x) = ax . Therefore, we can get m( f ( g ( x ))) = m( f (bx + c )) 1 a . (8) )= = m( 1 + e −( bx +c ) 1 + e −( bx +c ) This means that the mean value function of logistic growth curve model is composed of g(x), f(x), and m(x). Subsequently, we derive the compound functions from the viewpoints of neural network. Consider the basic feed-forward network shown in Fig. 3. Note that the network has only one neuron in each layer and w111 , w110 are the weights and b1 , b0 are the biases. When the input, x(t), at time t is fed to the input layer, we can derive the following form. The input of the hidden layer is: 1 (9) h _ in(t ) = w11 t + b1 . The output of the hidden layer is: (10) h(t ) = f (h _ in(t )) , where f(x) is the activation function in the hidden layer. The input of the output layer is (11) y _ in(t ) = w110 h(t ) + b0 .

The output of the output layer is (12) y (t ) = g ( y _ in(t )) , where g(x) is the activation function in the output ayer. 1 w11

h(t )

x(t )

w110

y (t )

Fig. 3 Feed-forward neural network with single neuron in each layer

After the derivation above, we find that if we assume the activation functions f(x) and g(x) as: 1 , f ( x) = 1 + e− x g ( x) = x . Furthermore, we remove the bias in the output layer. We can consequently get y (t ) = y _ in (t ) = w110 h ( t )

w110 . (13) 1 1 + e − ( w11t +b1 ) According to Eq. (13), we have successfully derived the neural network into a logistic growth curve model. By the same process, we can derive the neural network into many other existing models. For example, we can construct a neural network with an activation function as 1 − e − x in 1 the hidden layer, w11 =b, w110 =a, and there is no bias in both hidden layer and output layer. Thus, we have (14) h(t ) = 1 − e−bt , and (15) y(t ) = a(1 − e−bt ) . Obviously, it is the Goel-Okumoto (GO) model [6] which was firstly proposed by Goel and Okumoto and the model has strongly influenced the development of other models. We show another example to confirm that neural network can be applied for software reliability modeling. = w110 f ( h _ in (t )) =

If we construct a network with an activation function as 1 =b, w110 =a, and 1 − (1 + x )e − x in the hidden layer, w11 there is no bias in both hidden layer and output layer. Then, we have (16) h(t ) = 1 − (1 + bt )e− bt , and (17) y (t ) = a (1 − (1 + bt )e − bt ) . It is the Yamada Delay S-Shaped model [8, 9]. The model describes the fault detection process as a learning process in which testing members become familiar with the test environment or testing tools. In other words, their testing skills gradually improved. B. Extended Modeling We have mentioned that selecting a particular model is very important to software reliability assessment. But sometimes, software projects can not fit the assumptions of a unique model. To overcome this problem, Lyu and Allen [21] have proposed a solution by combining the results of different software reliability models. This approach inspired us to use the neural-network-based approach to combine the models. Next, we demonstrate an application of proposed approach to reach combinational models. We construct the neural network with single input single output but more than one neuron in the hidden layer. The network is depicted in Fig. 4. Note that we determine the number of neurons in the hidden layer by the number of models which assumptions are partially suitable to the software project. We use different activation functions in the hidden layer at the same time to achieve combinational models. More models selected will result that the combinational model has more accurate but it will take much more computation time. In this example, we will try to combine the GO model, the Delay-S-Shaped, and the logistic growth curve model. We use 1 − e − x , 1 − (1 + x )e − x , 1 as the activation functions in the hidden layer. and 1 + e− x The output of the network is: w21 , (18) y(t) = w21 (1 − e− w11t ) + w22 (1 − (1 + w12t)e− w12t ) + 1 + e− w13 where w1 j , w2 j are the weights of the network. Note that w2 j are represented as the weight of each individual models and they are determined by the training algorithm (refer to Appendix for the proof of this deduction).

w11

x

w21

w12 w13

w22

hj

y

w23

Fig. 4 the network architecture of the combinational model

This combinational model can adapt the characteristic of the given data set. Hence, this model itself can be considered as a general model for all software projects. Note that this method differs from those proposed

C. An Implementation of Proposed Approach As mentioned eariler, we have shown that the proposed approach is achievable. In this section, the implementation of the proposed neural-network-based models will be described. The following steps are required to apply our approach for software reliability prediction. 1. Analyze the assumptions of the software project and if there is a suitable model, select the model. If there is no suitable model, select those models which partially meet the assumptions of project. 2. Construct the neural network of selected models by designing the activation functions and bias. 3. Given n fault-detection time interval data x1, x2, …, xn, we accumulate the execution time and divide it into 100 time units t1, t2, …, t100 , and then calculate the number of failures, T1, T2 , …, T100. 4. Feed the pairs of { ti, Ti } to the network to train the network by the back-propagation algorithm which is mentioned in the previous section. 5. When the network trained, feed the future testing time to the network, and the network output is represented as the forecasting number of faults in the future. By these five steps, we can use the neural-network-based models to assess the software reliability, predict the total faults to be detected in the future. Note that in the Steps 2, the activation functions should be designed under some conditions which are (1) continuous and (2) differentiable everywhere, since we use the back-propagation algorithm in the Steps 4. And we will prove in the appendix that the activations which are used in the paper are satisfied these conditions.

IV.PROFORMANCE EVALUATION To validate the performances of our proposed approach, we use a real software failure data set. The data set was collected by John D. Musa of Bell Telephone Laboratories for a Real Time Command & Control (S1) which consists 21,700 lines of code (LOCs). During the testing phase, 136 faults were detected [2]. We first analyze the data set by using Laplace trend test [22] because software reliability studies are usually based on the application of growth models to obtain various measures. Let the time interval [0, t] be divided into k units of time, and let n(i) be the number of faults observed during time unit i. The Laplace factor, u(k), is given by k ( K − 1) k (i − 1)n(i ) − ∑ ∑ n (i ) . 2 i =1 (19) u( k ) = i =1 k 2 −1 k ∑ n (i ) 12 i =1 Note that negative value of u(k) indicates decreasing failure intensity, and thus reliability growth, while positive value indicates increasing failure intensity, and reliability decrease. Fig. 6 shows the Laplace trend test for the S1 data set. We can find that the reliability of this system is

growth. If the trend of the data set is shown as reliability growth, then this application of data set is suitable to use exponential type of model, for example, the GO model. If the trend of the data set is shown as reliability decay followed by growth, then the data set is suitable to use the S-Shaped model, for example, the Delay S-Shaped model or the Inflection S-Shaped model. 0

-1

-2

-3 Laplace factor

by Lyu et al., since their model only combined the results from various models based on assigned weights. Our proposed approach automatically determines the weight of each model based on the characteristic of the given data set.

-4

-5

-6

-7

-8

-9

time 0

10

20

30

40

50

60

70

80

90

100

Fig. 6 Laplace trend test

A. Criteria for Model's Comparison The comparison criteria we engage to compare various models’ performance are described as follows. 1. The root Mean of Square fitting errors (RMSE) [21] 2

1 k (20) ∑[m(ti ) − mi ] k i =1 Notice that m(ti) is the predicted number of faults at ti , and mi is actual number of faults at ti. The lower RMSE indicates less fitting error and better performance. The Relative Error (RE) [2] 2. M (t ) − k (21) Ri = i end k where k is detected faults at the end of testing, and Mi is the predicted mean value functions at testing time i (i