Software Quality Assessment using Flexibility

4 downloads 1976 Views 318KB Size Report
articles identified as primary paper. Traceability metrics and mind map used to locate the current research and resolved problem in the field of software flexibility.
International Review on Computers and Software (I.RE.CO.S.), Vol. 7, n. 5 ISSN 1828-6003 September 2012

Software Quality Assessment using Flexibility: A Systematic Literature Review Hema Subramaniam1, Hazura Zulzalil 2, Abstract – Software flexibility has been viewed as one of the software quality determinant factors apart from usability, reliability and maintainability. Software expected to achieve high level of flexibility in order to comply with current dynamic nature of user requirement. However, there is no effort on systematically collect, review and synthesize article that relate to this topic. This trigger a need for a systematic literature review whereby it covers software flexibility concept in the context of software quality assessment and various perspective of quantifying it. Literature search related to software quality and flexibility reveal 360 papers, and after further reading, 29 articles identified as primary paper. Traceability metrics and mind map used to locate the current research and resolved problem in the field of software flexibility. Information regarding software flexibility measurement in assessing software quality also been extracted through literature review. This paper serves as tool for software developer to anticipate the available software flexibility measurement method and tools in the process of assessing software quality.

Keywords: Software quality assessment, flexibility measurement, extensibility, maintenance evaluation, systematic literature review

I.

Introduction

Software quality is one of the knowledge areas in Software Engineering body of knowledge that got a wide attention among software development industries nowadays. Due to increasing complexity of software requirement, software quality becomes a crucial thing to evaluate which motivate researcher to produce software quality models. Software quality being anticipated differently based on the viewpoint of the quality itself which can be categorized into transcendental view, user view, product based view, value based view and manufacturing view [1]. There are plenty of software quality models has been developed and the models can be categorized into 3 different category (prediction model, definition model and assessment model) [2]. Software quality assessment model define as analytical models that provide quantitative assessment of selected quality characteristic or sub-characteristic based on measurement data form software projects [3]. Software quality models play important role in assessing software quality from various perspectives including requirement which can be used as an essential characteristic in order achieve software quality assurance [4]. Most of the software quality models are created based on software quality determinant factors. Among the listed quality factors, there are number of quality factors which overlapping across the model [5]. This indicates the importance of the characteristic in determining the software quality. Even though, each software quality models using different terms to describe the quality characteristics [6][7][8][9], Manuscript received and revised August 2012, accepted September 2012

it’s do refer to same characteristics. These review aim to investigate the usage of flexibility and its affiliated terms in assessing software quality. The various definition and understanding of flexibility covered in background and motivation section. Meanwhile, research methodology section outlines the searching strategy and ways to conduct systematic literature review. The outcome of the systematic literature review presented in traceability matrix format in results and discussion section.

II.

Background and Motivation

Flexibility is one of quality determinant factors as specified in the McCall quality model which is grouped under product revision perspective [3]. Bug fixes and complies with newly added requirement are among the reasons for a software to be released by versions. Versions control is also influenced by the flexibility of software. According to IEEE standard glossary, flexibility means “the ease with which a system or component can be modified for use in applications or environments other than those for which it was specifically designed” [10]. The word flexibility also referring to ability to modify the section/part that needs to be changed while the unchanged sections being remained [11]. Software flexibility can be referred in two perspectives; Software process flexibility and software product flexibility. Software process flexibility involve from the requirement phase throughout to implementation phase always need to adhere to the environment/business process changes. Software process flexibility becomes a mediator between software process standardization and Copyright © 2012 Praise Worthy Prize S.r.l. - All rights reserved

S.Hema, Z.Hazura

project performance [12]. Meanwhile software product flexibility can be achieved by designing software that able to change from one state to another state. Software product flexibility always refers to requirements modification and software internal structure changes. A considerable amount of literature has been published on maintainability metrics in open source environment which focus on the maintainability affiliated terms [13]. There are wide varieties of affiliated terms associated with flexibility in software measurement and information system terminologies [14]. Even though the terms refer to software measurement, this terms also closely related to software quality assessment as well. Portability, extensibility, extendibility, adaptability are among the terms that identified by author that considered as affiliated terms of flexibility. Affiliated terms of flexibility help in refining the results for our literature search strategies. Different author refer to flexibility in different terms which give same meaning.

III. Research Method The method used to conduct this systematic literature review based on the approach proposed by Barbara Kitchenham in their article title systematic literature review in software engineering [15]. There are few steps proposed by author in this article that manage the literature in systematic manner to facilitate the finding of the most relevant information in the field of research. Steps to conduct systematic literature review including developing reviewing protocol, identification and selection of primary studies, data extraction and lastly reporting the results. The reported result is the main activity that leads to the knowledge gap tracking. Knowledge gap is important in research to identify the researchable problem. Research always starts with a problem and questions. Software flexibility research relies on the following research questions that need to be answered during the literature review process. Q1: What are the inter relation between available software quality models and what are the overlapping characteristic between the software quality models? Q2: What are the available measurements models on measuring software flexibility? Q3: Where is the flexibility point relying on software/application? Q4: Flexibility had been covered by previous research in how many perspective and what are they? Paper article searching strategy started with finding journal names that relate to software quality assessment and the relation of software flexibility. The identification of Journal name and their associated publisher is the initial step that helps in developing review protocol as mention by Barbara Kitchenham [15]. Article searching strategies become cumbersome as the improper searching strategies reveal many searching results and give Copyright © 2007 Praise Worthy Prize S.r.l. - All rights reserved

difficulties in identifying and selecting primary studies. Adaptability, expendability, extendibility, extensibility and portability have been identified as affiliated terms of flexibility [14]. Searching strategy divided into 2 sections; flexibility and measurement. Table 1 shows the search terms sections and it’s affiliated terms. Affiliated terms play an important role in locating the paper, as different author using different term but pointing to same meaning. Searching has been conducted in cross section manner. For example adaptability term combined with measurement, metrics and evaluation. TABLE I SEARCH TERM USED Section Affiliated Terms /Keyword

Section ID 1

Software Flexibility

2

Measurement

Flexibility Adaptability Expandability Extendibility Extensibility Portability Measurement Metrics Evaluation

Our searching strategies have revealed 360 papers that associated with software flexibility and software quality assessment. There is certain filtration process used in the process of finding the primary paper. Figure 1 shows the stage by stage filtration process that help in displaying primary result that closely related to software flexibility measurement towards achieving software quality assessment. As an initial stage article searched by keyword in a cross section manner using available electronic databases such as IEEE, ACM, Wiley and etc. 360 articles have been identified using these searching strategies which later refined by article publication year. Article which is published in latest 5 years used to locate the latest finding on the field of research. Search result displayed 100 articles from various journals and conference preceeding which then refine to dedicated journal and conference proceeding as stated in Table 1. First level reading has been conducted in order to obtain primary article that reflects the idea of software flexibility measurement in measuring the software quality assessment. Stage A: Search by keyword using cross section searching strategies (eg: flexibility measurement)

360

Stage B: Refine result by displaying latest 5 year back articles under all the identified publisher

100

Stage C: Refine result by displaying related journal and proceedings that identified earlier

80

Stage D: 1st level reading: abstract filtration by identifying is the objective answer the research questions that identified earlier

29

Obtain Primary articles

Fig. 1. Primary paper filtration process flowchart

International Review on Computers and Software, Vol. 7, N. 5

S.Hema, Z.Hazura

IV. Result and Discussion Twenty nine primary articles or also known as anchor papers used in identifying the knowledge gap in software quality assessment. The result of reading represented in two illustration format; mind map and traceability matrix. Both representation help in identifying the area of research discussed in the paper. Based on literature review, roadmap as illustrated in Figure 2 produced which consist of area that related to software flexibility. The area includes the discussion or interested area on software flexibility process, measurement method, concept and terminology used in software flexibility, management and evaluation and assessment method of software flexibility. Anchor papers that identified through filtration process, become the major reference to produce the following roadmap. Due to the author limitation, article filtration activity and reviewing process only involve authors of the paper instead of group of reviewers. Research questions that identified earlier have been answered after the implementation of review process. Interrelation between quality models and their overlapping characteristic been identified by anticipating the meaning of affiliated terms. There is variety of software flexibility models has been identified using traceability matrix. However, the flexibility point that accommodate in the software/application is less discussed by authors. There are 5 perspectives (as shown in Figure 2) that related to flexibility measurement in relation with software quality assessment being derived from the literature review activity.

Fig. 2. Software Flexibility Area

[2]

F. Deissenboeck, E. Juergens, K. Lochmann, S. Wagner, T. Universit, and G. M, "Software Quality Models : Purposes , Usage Scenarios and Requirements," Critique, 2009, pp. 914.

[3]

D. Samadhiya, S. Wang, and D. Chen, "Quality Models : Role and Value in Software Engineering," Technology Management, 2010, pp. 320-324.

[4]

Y.A. Alsultanny and A.M. Wohaishi, "Essential Characteristics of Software Model that Provide the Software Quality Assurance," International Review on Computers and Software, vol. 4, 2009, pp. 577-585.

[5]

B. Kitchenham and S.L. Pefleeger, "Software Quality: an elusive target," IEEE Software, 1996.

[6]

Boehm, "Software Quality Models and Philosophies," Management, 1978.

[7]

I. JTC 1/SC, "ISO/IEC 9126-2: Software EngineeringProduct Quality-Part 2: External Metrics," 2002.

[8]

ISO9126-2-INTERNAL METRICS).

[9]

R.G. Dromey, "A Model for Software Product Quality," IEEE Transactions on Software Engineering, Vol. 21, No., vol. 2pp, pp. 146-162.

[10]

Standard Coordinating Commitee IEEE, "IEEE Standard Glossary of Software Engineering Terminology," IEEE, vol. 121990, 1990.

[11]

R. Martinho and D. Domingos, "A Two-Step Approach for Modelling Flexibility in Software Processes," 23rd IEEE/ACM International Conference on Automated Software Engineering, 2008, pp. 427-430.

[12]

J.Y. Liu, V.J. Chen, C. Chan, and T. Lie, "The impact of software process standardization on software flexibility and project management performance : Control theory perspective," Journal of Systems and Software, vol. 50, 2008, pp. 889-896.

[13]

A.D. Bakar, A.B. Sultan, H. Zulzalil, and J. Din, "Review on ‘Maintainability’ Metrics in Open Source Software," International Review on Computers and Software, vol. 7, 2012.

[14]

O.T. Pusatli and S. Misra, "A Discussion on IS and Software Measurement Terminology Flexibility as an Example," Internation Conference on Computation Science and Its Applications, IEEE, 2010, pp. 250-254.

[15]

B. Kitchenham, O.P. Brereton, D. Budgen, M. Turner, J. Bailey, and S. Linkman, "Systematic literature reviews in software engineering – A systematic literature review," Information and Software Technology, vol. 51, 2009, pp. 715.

[16]

J. Niu, L. Shen, S. Peng, and F. Li, "A Measurement Method of Software Flexibility Based on BP Network," International Workshop on Intelligent Systems and Applications, 2009, pp. 1-4.

[17]

D.D. Zeng and J.L. Zhao, "Achieving Software Flexibility via Intelligent Workflow Techniques," Proceeding of the 35th Annual Hawaii International Conference on System Sciences, IEEE, 2002, pp. 606- 615, 7-10.

Appendix Refer to appendix A for traceability metrics on software quality assessment using flexibility

References [1]

D.A. Garvin, "What does product quality really means," Sloan Management Review, 1984.

Copyright © 2007 Praise Worthy Prize S.r.l. - All rights reserved

International Review on Computers and Software, Vol. 7, N. 5

S.Hema, Z.Hazura

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

S. Peng, L. Shen, H. Liu, and F. Li, "User-Oriented Measurement of Software Flexibility," 2009 WRI World Congress on Computer Science and Information Engineering, 2009, pp. 629-633.

[32]

Q.P. Thi, D.T. Quang, and T.H. Quyet, "A Complexity Measure for Web Service," 2009, pp. 226-231.

[33]

V. Stoicu-Tivadar and L. Stoicu-Tivadar, "Software Flexibility Improvement for Document Oriented Applications," Health (San Francisco), 2009, pp. 21-26.

S. Wang and X. Liu, "A Study on Flexibility of ERP System Based on Grey Evaluation Model," Technology, 2010, pp. 36.

[34]

K. Rautiainen, P.O. Box, L. Vuornos, and C. Lassenius, "An Experience in Combining Flexibility and Control in a Small Company ’ s Software Product Development Process," Empirical Software Engineering, 2003.

M. Rauterberg, "An Empirical Validation of Four Different Measures to Quantify User Interface Characteristics Based on a General Descriptive Concept for Interaction Points," 1996.

[35]

B.P. Barbosa, E.A. Schmitz, and A.J. Alencar, "The Case for Managerial Flexibility in MMF-based Software Development Projects," 2008, pp. 114-115.

J.H. Yahaya, "Measuring Unmeasurable Attributes of Software Quality Using Pragmatic Quality Factor," Methodology.

[36]

J. Osbeck and P. Roden, "INVESTIGATION OF AUTOMATIC PREDICTION OF SOFTWARE QUALITY," Meta, 2011.

[37]

X. Wang, Y. Bai, and C. Cai, "A Production Rule-based Knowledge System for Software Quality Evaluation," Engineering and Technology, 2010, pp. 208-211.

[38]

W. Xiao-jun, "Metrics for Evaluating Coupling and Service Granularity in Service Oriented Architecture," Measurement, 2009.

[39]

A. Kumar, U. Pradesh, and P.S. Grover, "A Quantitative Evaluation of Aspect-Oriented Software Quality Model ( AOSQUAMO ) ACM SIGSOFT Software Engineering Notes," Software Engineering Notes, vol. 34, 2009, pp. 1-9.

F. Mårtensson, H. Grahn, M. Mattsson, P.O. Box, and S. Ronneby, "Evaluating Software Quality Attributes of Communication Components in an Automated Guided Vehicle System," Engineering, 2005.

[40]

S.K. Dubey and A. Rana, "Assessment of Maintainability Metrics for Object-Oriented Software System," Software Engineering Notes, vol. 36, 2011, pp. 1-7.

P. Qualitative and A. Using, "Predicting Qualitative Assessments Using Fuzzy Aggregation," Computer Engineering, pp. 267-272.

[41]

P. Warren, C. Gaskell, and C. Boldyreff, "Preparing the Ground for Website Metrics Research," World Wide Web Internet And Web Information Systems, 2001.

Y. Wang, M. Jia, J. Guo, and B. Zhang, "Evaluating Model of Software Flexibility of Domestic Foundational Software," International Conference on Electrical and Control Engineering, 2011, pp. 5906-5909. N.M. Villegas, G. Tamura, L. Duchien, and R. Casallas, "A Framework for Evaluating Quality-Driven Self-Adaptive Software Systems," Proceedings of the 6th International Symposium on Software Engineering for Adaptive and SelfManaging Systems (SEAMS '11), New York: ACM, 2011, pp. 80-89.

[26]

C. Babu, R. Vijayalakshmi, S.S. College, and O. Engineering, "Metrics- Based Design Selection tool for Aspect Oriented Software Development," Software Engineering Notes, vol. 33, 2008.

[27]

W. Royce, "Software Development Models of Code ( SLOC ).," Development.

[28]

S. Barrett and Y. Khattak, "Primitive Components : Towards more flexible Black Box AOP," Proceedings of the 1st International Workshop on Context-Aware Middleware and Services: affiliated with the 4th International Conference on Communication System Software and Middleware, ACM, 2009, pp. 24-30.

[29]

D. Perez-palacin, P. Milano, and D. Elettronica, "Software Architecture Adaptability Metrics for QoS-based," Proceedings of the joint ACM SIGSOFT conference -- QoSA and ACM SIGSOFT symposium -- ISARCS on Quality of software architectures -- QoSA and architecting critical systems -- ISARCS (QoSA-ISARCS '11), New York: ACM, , pp. 171-175.

[30]

Y. Singh, P.K. Bhatia, and O. Sangwan, "Software reusability assessment using soft computing techniques," Software Engineering Notes, vol. 36, 2011.

[31]

E. Reisner, C. Song, K. Ma, J.S. Foster, and A. Porter, "Using Symbolic Evaluation to Understand Behavior in Configurable Software Systems," Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering, New York: ACM, , pp. 445-454.

Copyright © 2007 Praise Worthy Prize S.r.l. - All rights reserved

Authors’ information Hema Subramaniam is a PhD student at the faculty of Computer Science and Information Technology, Putra University of Malaysia. She holds a Master Degree in Software Engineering from Selangor University (UNISEL). Her research interest includes software quality and software measurement.

Hazura Zulzalil holds a PhD from Putra University of Malaysia. She holds a Master Degree in Software Engineering from UPM. Her research interest includes software quality and quality for web based application.

International Review on Computers and Software, Vol. 7, N. 5

S.Hema, Z.Hazura

Copyright © 2007 Praise Worthy Prize S.r.l. - All rights reserved

International Review on Computers and Software, Vol. 7, N. 5

International Review on Computers and Software (I.RE.CO.S.), Vol. 7, n. 5 September 2012

ISSN 1828-6003

Appendix A: Traceability matrix on software quality assessment using software flexibility Paper Reference Process Model Measurement metrics Measurement Model Evaluation / Assessment Management and concept Paper Reference Process Model Measurement metrics Measurement Model Evaluation / Assessment Management and concept

1: [11] ×

2: [16]

3: [17] ×

4: [18]

5: [19]

6: [12]

7: [20] ×

8: [14]

9: [21] ×

10: [22]

11: [23]

12: [24]

× ×

20: [32]

×

21: [33]

22: [34]

23: [35]

×

24: [36]

×

×

28: [40]

29: [41]

× 25: [37]

×

26: [38]

27: [39]

×

× ×

×

×

Manuscript received and revised August 2012, accepted September 2012

×

×

Copyright © 2012 Praise Worthy Prize S.r.l. - All rights reserved

15: [27]

16: [28]

×

×

× ×

14: [26]

×

× 19: [31]

13: [25]

17: [29]

18: [30]

× ×

×

×