A Parametric Empirical Bayes Model to predict

0 downloads 0 Views 767KB Size Report
Software Development (waterfall, agile sprint). Quality: Standards. ISO-IEC 9126 (1991). ISO-IEC 14598 (1995). ISO-IEC 15504 (1998). CMMI (1991). Analysis.
A Parametric Empirical Bayes Model to predict Software Reliability Growth

A Parametric Empirical Bayes Model to predict Software Reliability Growth Néstor R. Barraza [email protected]

March - 2015 Universidad Nacional de Tres de Febrero

A Parametric Empirical Bayes Model to predict Software Reliability Growth

Contents 1

Software Reliability

A Parametric Empirical Bayes Model to predict Software Reliability Growth

Contents 1

Software Reliability

2

Software Reliability Growth Models

A Parametric Empirical Bayes Model to predict Software Reliability Growth

Contents 1

Software Reliability

2

Software Reliability Growth Models

3

Poisson Processes

A Parametric Empirical Bayes Model to predict Software Reliability Growth

Contents 1

Software Reliability

2

Software Reliability Growth Models

3

Poisson Processes

4

Empirical Bayes Estimator

A Parametric Empirical Bayes Model to predict Software Reliability Growth

Contents 1

Software Reliability

2

Software Reliability Growth Models

3

Poisson Processes

4

Empirical Bayes Estimator

5

Simulation

A Parametric Empirical Bayes Model to predict Software Reliability Growth

Contents 1

Software Reliability

2

Software Reliability Growth Models

3

Poisson Processes

4

Empirical Bayes Estimator

5

Simulation

6

Conclusions

A Parametric Empirical Bayes Model to predict Software Reliability Growth

Contents 1

Software Reliability

2

Software Reliability Growth Models

3

Poisson Processes

4

Empirical Bayes Estimator

5

Simulation

6

Conclusions

7

Bibliography

A Parametric Empirical Bayes Model to predict Software Reliability Growth Software Reliability

Software Development (waterfall, agile sprint) Quality: Standards ISO-IEC 9126 (1991) ISO-IEC 14598 (1995) ISO-IEC 15504 (1998) CMMI (1991) Analysis Reliability Metrics

A Parametric Empirical Bayes Model to predict Software Reliability Growth Software Reliability

Software Reliability (waterfall, agile sprint).

Models Growth Models Markov Chains Clusters ...

A Parametric Empirical Bayes Model to predict Software Reliability Growth Software Reliability

Software Reliability Growth

A Parametric Empirical Bayes Model to predict Software Reliability Growth Software Reliability

Software Reliability Growth

A Parametric Empirical Bayes Model to predict Software Reliability Growth Software Reliability Growth Models

Independent Increments Stochastic process P(N(t) = n). Remaining number of failures µ(t, s) = E[N(s)|N(t)].

A Parametric Empirical Bayes Model to predict Software Reliability Growth Software Reliability Growth Models

Non homogeneous Poisson Process

P((N(s) − N(t)) = n) =

µ(t, s)n −µ(t,s) e n!

µ(t, 0) = a(1 − e−bt ) Goel − Okumoto µ(t, 0) =

1 ln(µ0 θt + 1) Musa − Okumoto θ

A Parametric Empirical Bayes Model to predict Software Reliability Growth Software Reliability Growth Models

Other Models

Model Delayed S-shaped Log Power Gomperts Yamada Exponential

µ(t, 0) a(1 − (1 + b t)e−b t ) α lnβ (1 + t) t a(bc ) (−d t) ) a(1 − e−b c (1−e )

A Parametric Empirical Bayes Model to predict Software Reliability Growth Poisson Processes

Compound Poisson Process

P(N(t) = n) =

m X (λ t)k k =1

k!

e−λt f ∗k (X1 + X2 + · · · + Xk = n)

E[N(t)] = λ t E[X ]

A Parametric Empirical Bayes Model to predict Software Reliability Growth Empirical Bayes Estimator

Empirical Bayes Estimator

Estimate a binary probability based on a previous number of occurrences. Applications:

A Parametric Empirical Bayes Model to predict Software Reliability Growth Empirical Bayes Estimator

Empirical Bayes Estimator

Estimate a binary probability based on a previous number of occurrences. Applications: Speech recognition

A Parametric Empirical Bayes Model to predict Software Reliability Growth Empirical Bayes Estimator

Empirical Bayes Estimator

Estimate a binary probability based on a previous number of occurrences. Applications: Speech recognition Word processing

A Parametric Empirical Bayes Model to predict Software Reliability Growth Empirical Bayes Estimator

Empirical Bayes Estimator

Estimate a binary probability based on a previous number of occurrences. Applications: Speech recognition Word processing Arithmetic compression

A Parametric Empirical Bayes Model to predict Software Reliability Growth Empirical Bayes Estimator

Empirical Bayes Estimator

Estimate a binary probability based on a previous number of occurrences. Applications: Speech recognition Word processing Arithmetic compression Reliability

A Parametric Empirical Bayes Model to predict Software Reliability Growth Empirical Bayes Estimator

Empirical Bayes Estimator

Try to estimate the probability of success θ, given n samples with r successful events.

A Parametric Empirical Bayes Model to predict Software Reliability Growth Empirical Bayes Estimator

Empirical Bayes Estimator

Try to estimate the probability of success θ, given n samples with r successful events. Maximum likelihood: θˆ = r n

A Parametric Empirical Bayes Model to predict Software Reliability Growth Empirical Bayes Estimator

Empirical Bayes Estimator

Try to estimate the probability of success θ, given n samples with r successful events. Maximum likelihood: θˆ = r → Problems when r = 0 n

A Parametric Empirical Bayes Model to predict Software Reliability Growth Empirical Bayes Estimator

Empirical Bayes Estimator

Try to estimate the probability of success θ, given n samples with r successful events. Maximum likelihood: θˆ = r → Problems when r = 0 n

Smoothing θˆ =

r +a n+b

A Parametric Empirical Bayes Model to predict Software Reliability Growth Empirical Bayes Estimator

Family of estimators

A Parametric Empirical Bayes Model to predict Software Reliability Growth Empirical Bayes Estimator

Family of estimators Laplace law of succession θˆ =

r +1 n+2

A Parametric Empirical Bayes Model to predict Software Reliability Growth Empirical Bayes Estimator

Family of estimators Laplace law of succession θˆ = Lidstone law θˆ = r +b n+s b

r +1 n+2

A Parametric Empirical Bayes Model to predict Software Reliability Growth Empirical Bayes Estimator

Family of estimators Laplace law of succession θˆ = Lidstone law θˆ = r +b Good-Turing θˆ =

n+s b r +1 Nr n Nr +1

r +1 n+2

A Parametric Empirical Bayes Model to predict Software Reliability Growth Empirical Bayes Estimator

Family of estimators Laplace law of succession θˆ = Lidstone law θˆ = r +b Good-Turing θˆ = Discount θˆ =

r −b n

n+s b r +1 Nr n Nr +1

r +1 n+2

A Parametric Empirical Bayes Model to predict Software Reliability Growth Empirical Bayes Estimator

Family of estimators Laplace law of succession θˆ = Lidstone law θˆ = r +b Good-Turing θˆ = Discount θˆ =

r +1 n+2

n+s b r +1 Nr n Nr +1

r −b n

New approach, to introduce a non linear function of r : θˆ = f (rn )

A Parametric Empirical Bayes Model to predict Software Reliability Growth Empirical Bayes Estimator

P(r /λ) = θ=

λr −λ e . r!

(1)

λ n

(2)

A Parametric Empirical Bayes Model to predict Software Reliability Growth Empirical Bayes Estimator

ˆ λ E[λ|r ] θˆ = = n n

θˆ = =

(3)

R r 1 λ λr ! e−λ dS(λ) R n λr !r e−λ dS(λ) r + 1 P(r + 1) n P(r )

(4)

A Parametric Empirical Bayes Model to predict Software Reliability Growth Empirical Bayes Estimator

θ: Probability of failure occurrence r : The number of failures up to time n P(r ): Prior probability of the number of failures Reliability Growth → A decreasing probability of failure θ >1 θˆ =

r +1 n

P(r +1) P(r )

P(r )

r

r n

A Parametric Empirical Bayes Model to predict Software Reliability Growth Empirical Bayes Estimator

θ: Probability of failure occurrence r : The number of failures up to time n P(r ): Prior probability of the number of failures Reliability Growth → A decreasing probability of failure θ >1 θˆ =

r +1 n

P(r +1) P(r )

P(r )

r

r n

A Parametric Empirical Bayes Model to predict Software Reliability Growth Empirical Bayes Estimator

θ: Probability of failure occurrence r : The number of failures up to time n P(r ): Prior probability of the number of failures Reliability Growth → A decreasing probability of failure θ >1 < θˆ =

r +1 n

P(r +1) P(r )

P(r )

r

r n

A Parametric Empirical Bayes Model to predict Software Reliability Growth Simulation

Simulation procedure s ← initial count of seconds r ← initial number of failures for i = s to the total testing time do r + 1 P(r + 1) {Estimate the probability of failure} p= i P(r ) GENERATE a failure with probability p if a failure arrived then r ←r +1 end if print number of arrived failures r and seconds i end for (Failure generation code)

A Parametric Empirical Bayes Model to predict Software Reliability Growth Simulation

Number of failures 40

30

20

10

secs 5,000,000

10,000,000

15,000,000

A Parametric Empirical Bayes Model to predict Software Reliability Growth Simulation

Number of failures 40

θˆ0 =

30

20

10

secs 5,000,000

10,000,000

15,000,000

r0 +1 P(r0 +1) n0 P(r0 )

A Parametric Empirical Bayes Model to predict Software Reliability Growth Simulation

Number of failures 40

θˆ0 =

30

20

10

secs 5,000,000

10,000,000

15,000,000

r0 +1 P(r0 +1) n0 P(r0 )

A Parametric Empirical Bayes Model to predict Software Reliability Growth Simulation

120

Number of failures 100

80

Actual Data 60

Simulation 40

20

secs 5,000,000

10,000,000

15,000,000

A Parametric Empirical Bayes Model to predict Software Reliability Growth Simulation

120

Number of failures 100

80

Actual Data 60

Simulation 40

20

secs 5,000,000

10,000,000

15,000,000

A Parametric Empirical Bayes Model to predict Software Reliability Growth Simulation

Same Mixing Distribution Different Starting Points 120

Number of failures

100

80

60

40

20

secs 5,000,000

10,000,000

15,000,000

A Parametric Empirical Bayes Model to predict Software Reliability Growth Simulation

Same Mixing Distribution Different Starting Points 120

Number of failures

100

80

60

40

20

secs 5,000,000

10,000,000

15,000,000

A Parametric Empirical Bayes Model to predict Software Reliability Growth Simulation

Same Mixing Distribution Different Starting Points 120

Number of failures

100

80

60

40

20

secs 5,000,000

10,000,000

15,000,000

A Parametric Empirical Bayes Model to predict Software Reliability Growth Simulation

Same Mixing Distribution Different Starting Points 120

Number of failures

100

80

60

40

20

secs 5,000,000

10,000,000

15,000,000

A Parametric Empirical Bayes Model to predict Software Reliability Growth Simulation

Mean Time Between Failures (Secs)

SS1 Project ·105

6

+ + Actual Data 4

ML 2

Our Model

0 0

20

40

60

80

Failure Number

100

120

A Parametric Empirical Bayes Model to predict Software Reliability Growth Simulation

Mean Time Between Failures (Secs)

SS1 Project ·105

6

+ + Actual Data 4

ML 2

Our Model

0 0

20

40

60

80

Failure Number

100

120

A Parametric Empirical Bayes Model to predict Software Reliability Growth Simulation

Mean Time Between Failures (Secs)

SS1 Project ·105

6

+ + Actual Data 4

ML 2

Our Model

0 0

20

40

60

80

Failure Number

100

120

A Parametric Empirical Bayes Model to predict Software Reliability Growth Simulation

Mean Time Between Failures (Days)

Naval Tactical Data System Actual Data

100

50

0 5

10

15

20

Failure Number

25

30

A Parametric Empirical Bayes Model to predict Software Reliability Growth Simulation

Mean Time Between Failures (Days)

Naval Tactical Data System Actual Data Mazzuchi Model I 100

50

0 5

10

15

20

Failure Number

25

30

A Parametric Empirical Bayes Model to predict Software Reliability Growth Simulation

Mean Time Between Failures (Days)

Naval Tactical Data System Actual Data Mazzuchi Model I Mazzuchi Model II 100

50

0 5

10

15

20

Failure Number

25

30

A Parametric Empirical Bayes Model to predict Software Reliability Growth Simulation

Mean Time Between Failures (Days)

Naval Tactical Data System

100

Actual Data Mazzuchi Model I Mazzuchi Model II Our Model

50

0 5

10

15

20

Failure Number

25

30

A Parametric Empirical Bayes Model to predict Software Reliability Growth Conclusions

Conclusions

A Parametric Empirical Bayes Model to predict Software Reliability Growth Conclusions

Conclusions A new method in order to predict software failures production was proposed.

A Parametric Empirical Bayes Model to predict Software Reliability Growth Conclusions

Conclusions A new method in order to predict software failures production was proposed. This method is based on The Empirical Bayes estimator usually used in other areas of Engineering.

A Parametric Empirical Bayes Model to predict Software Reliability Growth Conclusions

Conclusions A new method in order to predict software failures production was proposed. This method is based on The Empirical Bayes estimator usually used in other areas of Engineering. In order to model reliability growth, a specially smoothing nonlinear factor was introduced in the estimate.

A Parametric Empirical Bayes Model to predict Software Reliability Growth Conclusions

Conclusions A new method in order to predict software failures production was proposed. This method is based on The Empirical Bayes estimator usually used in other areas of Engineering. In order to model reliability growth, a specially smoothing nonlinear factor was introduced in the estimate. This factor comes from a mixed Poisson distribution.

A Parametric Empirical Bayes Model to predict Software Reliability Growth Conclusions

Conclusions A new method in order to predict software failures production was proposed. This method is based on The Empirical Bayes estimator usually used in other areas of Engineering. In order to model reliability growth, a specially smoothing nonlinear factor was introduced in the estimate. This factor comes from a mixed Poisson distribution. A simulation of the cumulative failures curve using a Poisson mixed by a Generalized Inverse Gaussian probability density function has been performed with good agreement with reported data.

A Parametric Empirical Bayes Model to predict Software Reliability Growth Bibliography

Bibliography V. Almering, M. van Genuchten, G. Cloudt, and P. J. Sonnemans, “Using software reliability growth models in practice,” IEEE Software, vol. 24, no. 6, pp. 82–88, 2007. N. R. Barraza, “The empirical bayes estimator and mixed distributions,” AIP Conference Proceedings, vol. 1073, no. 1, pp. 103–110, 2008. [Online]. Available: http://scitation.aip.org/content/aip/proceeding/aipcp/10. 1063/1.3038987 ——, “Parameter estimation for the compound poisson software reliability model,” International Journal of Software Engineering and its Applications, vol. 7, no. 1, pp. 137–148, 2013.

A Parametric Empirical Bayes Model to predict Software Reliability Growth Bibliography

A. Nadas, “On Turing’s formula for word probabilities,” IEEE Trans. Acoust. Speech and Signal Processing, vol. 33, pp. 1414–1416, Dec. 1985. J. S. Maritz, “Empirical Bayes estimation for the Poisson distribution,” Biometrika, vol. 56, pp. 349–359, 1969. S. Vaithyanathan, J. Mao, and B. Dom, “Hierarchical bayes for text classification,” in PRICAI Workshop on Text and Web Mining, 2000, pp. 36–43. R. E. Barlow and F. Proschan, Mathematical Theory of Reliability. Philadelphia, PA 19104-2688: SIAM, 1996. F. D. B. Klaus Ries and A. Waibel, “Class phrase models for language modeling,” in Proceedings of the ICSLP 96, October 1996.

A Parametric Empirical Bayes Model to predict Software Reliability Growth Bibliography

Q. Mei, H. Fang, and C. Zhai, “A study of poisson query generation model for information retrieval,” in SIGIR, 2007, pp. 319–326. T. A. Mazzuchi and S. R., “A Bayes Empirical-Bayes Model for Software Reliability,” IEEE Trans. on Reliability, vol. 37, pp. 248–254, Jun. 1988. G. Willmot, “Mixed Compound Poisson Distributions,” Astin Bulletin, vol. 16, pp. 59–79, 1986. H. Ney, U. Essen, and R. Kneser, “On the estimation of ’small’ probabilities by leaving-one-out,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 17, no. 12, pp. 1202–1212, 1995. K. Lynn and Y. Y. Tae, “Bayesian computation for nonhomogeneous poisson processes in software reliability,” Journal of The American Statistical Association, vol. 91, no. 434, pp. 763–773, 1996.

A Parametric Empirical Bayes Model to predict Software Reliability Growth Bibliography

I. J. Good, “The population frequencies of species and the estimation of population parameters,” Biometrika, vol. 40, no. 3-4, pp. 237–264, 1953. [Online]. Available: http://biomet.oxfordjournals.org/content/40/3-4/237.abstract S. F. Chen and J. Goodman, “An empirical study of smoothing techniques for language modeling,” in Proceedings of the Thirty-Fourth Annual Meeting of the Association for Computational Linguistics, A. Joshi and M. Palmer, Eds. San Francisco: Morgan Kaufmann Publishers, 1996, pp. 310–318. E. S. Ristad, “A natural law of succession,” Princeton University, Tech. Rep. TR-495-95, 1995. B. Far, “Software reliability engineering for agile software development,” in Electrical and Computer Engineering,

A Parametric Empirical Bayes Model to predict Software Reliability Growth Bibliography

2007. CCECE 2007. Canadian Conference on, April 2007, pp. 694–697. J. D. Musa, “Cyber security & information systems. information analysis center,” https://sw.thecsiac.com/databases/sled/swrel.php, 1970. M. Xie, G. Hong, and C. Wohlin, “Software reliability prediction incorporating information from a similar project,” Journal of Systems and Software, vol. 49, no. 1, pp. 43 – 48, 1999. [Online]. Available: http://www.sciencedirect.com/ science/article/pii/S0164121299000655 M. Xie, G. Y. Hong, and C. Wohlin, “A practical method for the estimation of software reliability growth in the early stage of testing,” in Software Reliability Engineering, 1997. Proceedings., The Eighth International Symposium on, Nov 1997, pp. 116–123.

A Parametric Empirical Bayes Model to predict Software Reliability Growth Bibliography

H.-L. Wu, Y. Zhong, and Y. Chen, “A software reliability prediction model based on benchmark measurement,” in Information Management and Engineering (ICIME), 2010 The 2nd IEEE International Conference on, April 2010, pp. 131–134. Y. K. Malaiya, “Software reliability and security,” Encyclopedia of Library and Information Science, 2005. I. Sharma and M. ParveenBano, “A combined approach of software metrics and software fault analysis to estimate software reliability,” IOSR Journal of Computer Engineering, vol. 11, no. 6, pp. 1 – 14, 2013.