an empirical study - Semantic Scholar

Applicability of IEEE Maintenance Process for Corrective Maintenance Outsourcing - An Empirical Study Baru S. Rao Syntel, Unit No. 112, SDF IV Seepz, Andheri (East), Mumbai 400 096 India 0091-22-829-0181 [email protected] Abstract We establish the context of maintenance outsourcing of mission critical applications by Fortune 500 organizations. We present the results of empirical studies that were conducted at Syntel, a NASDAQ listed application management and e-business solutions company, on 46 software maintenance projects that belonged to various lines of business on the IBM mainframe platform using an automated data collection tool EQUIP. After establishing that corrective maintenance activities form a significant component of the overall maintenance efforts, we examine the applicability of the IEEE standard maintenance process for corrective maintenance by measuring the efforts spent on the various activities. We conclude that a) the processes for each type of maintenance need to be fine tuned especially in the context of outsourcing b) analysis, testing form significant part of the corrective maintenance effort c) teams need to carry out other activities such as database reorganization and configuration management that are not defined by the IEEE maintenance process. Categories and Subject Descriptors D.2.7 [Software Engineering]: Distribution, Maintenance, and Enhancement - Corrections, Enhancement, and Restructuring General Terms Management, Measurement, Third Party Maintenance Keywords Software Maintenance, Outsourcing, Types of Maintenance, Maintenance Process

N. L. Sarda Department of Computer Science & Engineering, Indian Institute of Technology Powai, Mumbai 400 076 India 0091-22-576-7710 [email protected] 1. Introduction 1.1 Current Business Context of Outsourcing Several Fortune 500 organizations worldwide are focussing on their core businesses and are exploring the possibilities of outsourcing both the development and the maintenance activities to software companies. For maximizing the returns on the dollars spent, organizations are also looking for software companies that have offshore capabilities. Considering that maintenance accounts for about 40% to 90% of the software life cycle costs [3], it is reasonable to expect that the market for outsourcing of maintenance is at least $5 billion. This is amply reflected in the growth in software exports from India. The exports have grown from $485 million in 1995 to about $4.0 billion in the year 2000. The number of data communication circuits has gone up from 10 in 1992 to 1200 in the year 2000. In the year 2000, 185 of the Fortune 500 companies used Indian companies for their software needs as compared to 10 in 1990 [1,12]. Even after the Y2K related work has come to a conclusion, the Indian software industry has registered a 52% revenue growth in Q1 2001 in comparison to the same period in Q1 2000[2].

1.2 Trends in Outsourcing However the following trends are seen in this maturing market: • The number of players in the outsourcing market is on a dramatic rise. The number of sales offices of Indian software companies at overseas locations has gone up from 167 in 1995 to 582 in the year 2000[1]. • The work that is outsourced to offshore companies continues to be in the latter phases of the software

Proceedings of the International Conference on Software Maintenance (ICSM’02) 0-7695-1819-2/02 $17.00 © 2002 IEEE Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY BOMBAY. Downloaded on April 24, 2009 at 05:54 from IEEE Xplore. Restrictions apply.

development life cycle, namely, coding, testing and documentation. • Most of the software maintenance work that is outsourced is in the area of maintenance of custom developed systems as opposed to product/package maintenance [2]. • While there is an increased spending in the outsourcing area, the rates for onsite and offshore work are decreasing by 8 and 25% respectively. The possibility of more cost effective services from countries such as China coupled with the recent slow down experienced by the IT sector globally, is bringing rate pressure on the outsourcing vendors [2,11]. • Both the infrastructure and the salary costs of providing the software services are rising continuously. The software professionals’ salaries in India have gone up by an average of 18% every year during 1996-2000 [1]. The trends listed above are forcing the vendor organizations to scope maintenance engagements at finer levels of detail and focus on continuous process improvements to provide cost effective delivery.

1.3 Brief Description of Outsourcing Engagements When an organization decides to outsource its application maintenance to third parties, they either go through a tendering process or short list a few vendors based on their track records in this area. Depending on the specific requirements, organizations evolve a set of criteria for evaluation and conduct a due diligence. The due diligence may involve written responses to questionnaires, vendor site visits to meet the potential teams that could be involved in the project and reference checks from the existing customers of the vendor. Once the vendor is chosen, the applications are transitioned to the vendor based on a project plan. Based on the size of the portfolio being outsourced, its complexity, volatility, geographical spread, stability and the number of pending enhancements, vendors could take between a few weeks to a few months to complete the transition. After the transition is completed, the vendors are held accountable for various service levels associated with the engagement. The service levels are normally decided as part of the contractual negotiations and include items such as time limit to service a severity one level error, the guaranteed up times of a particular application, etc. Post transition, the vendor organization may carry out various types of maintenance activities [3], namely, Adaptive: Changes as a result of the system’s movement to a new operating environment Corrective: Changes necessitated by actual errors Perfection Related: Changes and enhancements made to a system to meet the evolving needs

Preventive: Changes carried out to prevent the problems before they occur

1.4 Related work Details of software maintenance methodologies and processes in the industrial context have been given by Pigoski in [3]. He presents issues related to methodologies, metrics, tools and training. He concludes that outsourcing of perfection related maintenance is unlikely to happen in near future. The IEEE standard for software maintenance [4] discusses the phases and associated activities but does not provide guidance on the variations in process flow for different types of maintenance. Mira Kajko-Mattson [5,6] has developed the maintenance maturity model and the taxonomy of one of the activities of corrective maintenance. Ed Yourdon in his paper [7] argues that we will have to establish a rational balance between various project parameters such as cost and quality. Bennett and Rajlich [8] while laying out the road map for software maintenance and its evolution, identify the impact of outsourcing on application management as an area of research. The area of software maintenance outsourcing has not received the attention it deserves despite so many Fortune 500 organizations currently being engaged in these projects. A good understanding of the impact of outsourcing on software maintenance methodologies, processes and metrics can significantly contribute to both the researchers and practitioners.

1.5 Details of the Work Undertaken One of the authors, Baru Rao, is currently employed in Syntel (an ISO 9001 certified and SEI CMM level 5 application management and e-business solutions organization [NASDAQ: SYNT]) as Head of Delivery. Syntel carries out several maintenance engagements for Fortune 500 companies on multiple platforms and has an established program for continuous process improvement and quantitative management of metrics. The work presented here has been carried out during September 2000 - January 2001 when Syntel was focusing on finetuning its maintenance processes to service its global clients. The objective of the work was to: • Understand the maintenance type mix in typical maintenance outsourcing projects; • Examine the effort distribution in various activities associated with different types of maintenance; • Evaluate the applicability of the industry standard processes for maintenance outsourcing; and • Improve the process models for maintenance.


A set of software maintenance projects that belonged to various lines of business, such as finance, insurance, manufacturing and predominantly on the IBM mainframe platform were considered as candidates for the study. All the systems were outsourced to Syntel after their development and Syntel was not involved in the development of any of these systems and the systems were under maintenance in Syntel for a period of at least 6 months. All the systems were in-house systems developed by large multinational corporations based out of the USA and Europe. Table 1 shows the brief profiles of the projects for which the empirical data was collected.

manager also specifies the expected effort for completing the activity for each team member. Each team member at the end of the day completes an activity-based time sheet where he enters the time spent on each activity that he was assigned to. The empirical data was collected using a Syntel home grown tool, Enterprise Quality Information Processing (EQUIP). The data model for EQUIP is presented in Figure 1. EQUIP also provides various reports that summarize the efforts taken at the level of activity, task, maintenance request across the projects. All activities as listed by the IEEE maintenance process were defined as the possible activities of maintenance in EQUIP. Programmers were asked to fill the actual times spent by them on the listed activities and were also encouraged to record the effort spent on activities that were not defined as part of EQUIP. At the end of every week, the project managers of the various projects met together to analyze the efforts that team members spent on the new activities and created the new activity definitions where relevant in EQUIP. The data collected consisted of the following: 1. Maintenance type mix details of the various engagements. 2. Duration and efforts for the maintenance requests. 3. Distribution of efforts for various activities of maintenance.

2.2 Data Collection Process

3.0 Maintenance Mix of the Projects

A brief description of the maintenance process followed in Syntel is given below: Every maintenance request, either originating from the customer or raised by the project team (to fix a known problem), is analyzed by the project manager and is broken into one or multiple maintenance tasks. Each task is then broken down to several activities and one or more team member(s) is assigned an activity. The project

The maintenance type mix for each project was collected by identifying each type of maintenance request. For each maintenance request, the total effort spent in hours was computed by summing up the time spent by all the team members on all the activities for all the tasks that constituted the maintenance request. Table 1 presents the maintenance mix of the various projects.

1.6 Organization of the Paper Section 2 presents the methodology of the study by providing details of project profiles and the data collection process. Section 3 presents the maintenance mix of the various projects. Section 4 presents a brief outline of the IEEE standard maintenance process and the distribution of efforts and duration for corrective maintenance tasks. The analysis of the empirical study is presented in Section 5 and conclusions are provided in Section 6.

2. Methodology of the Study 2.1 Profiles of the projects

Employee

Employee -Project

Project

Task-Assignment

Activity-Time Sheet

Tasks

Fig 1. Data Model Of EQUIP


No. of Hours of No. of Hours No. of Hours of Total Hours Age of the Lines of No. of Hours System Code in of System of Adaptive Preventive of Corrective Perfection Related Maintenance Maintenance (In Years) '000 No Maintenance Maintenance Maintenance 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46

9

224 15 10 112

265 99 168 194

951 203 1308 301

24 4

84 19 500 3311 400 2809 39 2651 2649

57 235 1419 2306 289 1688 52 1288 1271 2063 489 1123 362 428 1147 920 64 26 328 274

1449 317 1486 607 227 165 258 1919 7363 689 5505 153 4105 4132 2939 499 1544 1366 2674 6302 3072 736 1841 1493 1238 16 502 1658 2843 442 90 3576 335 166 642 233 455 459 399 1259 711 695 426 402 101 36

227

1646

100

50 22 54

1008 12 144 158 876

21 379 55 17 249 93 2

264 216 870 1081 252 336 222 74 111 107

1648

10 421 740 2009 3906 1016 403 1479 694 797 16 266 1362 1345 235 78 1928 6 80 122 345 321 348 676 564 456 332 322 101 36

123 189 1498 207 12 329 166 562 111 110 138 51 583 147 239 94 80

15 15 7 15 7 7 3 18 20 1 15 22 10 6 22 10 6 20 20 20 15 15 12 12 20 20 8 12 1 9 2 9 12 5 20 5 1 6 8 7 9 7 12 5 2 5

2756 844 278 1166 370 18 90 4744 10000 50 3630 110 14500 1800 110 600 11100 2500 3500 1720 890 1500 1700 1600 2500 2500 3020 550 80 300 46 300 200 40 550 50 50 70 25 80 55 80 200 20 320

Platform Details Microsoft, Mainframe Mainframe Mainframe, MS Microsoft Microsoft Mainframe, MS Mainframe Mainframe Mainframe Mainframe Mainframe Mainframe Mainframe Mainframe Mainframe MS, AS400, Web Mainframe Mainframe Mainframe Mainframe Mainframe Mainframe Mainframe Mainframe Mainframe Mainframe Mainframe Microsoft Mainframe Mainframe Mainframe Mainframe Mainframe Mainframe Mainframe Mainframe Mainframe Mainframe Mainframe Mainframe Mainframe Mainframe Mainframe Mainframe Web

Table 1. Project Profiles and Maintenance Mix

•

The average effort spent on corrective maintenance was about 49.7%. This is high in comparison to the 20% reported in studies [3].

•

Preventive maintenance constituted approximately 11% of the efforts and was carried out by the project


teams when they were not busy with corrective maintenance requests. • The average effort spent on adaptive maintenance was about 4%. This is very low in comparison to the 25% reported in studies [3]. The adaptive maintenance efforts being carried out as part of the maintenance projects were due to minor changes in the versions of the supportive software used in the system environment. We also observed that a major platform change for a system was normally considered a migration project and was not combined with the maintenance activities. • The average effort spent on perfection related maintenance was about 34.4%. This is a little low compared to the 55% reported in studies [3]. Most of the perfection related maintenance requests in these projects were minor functional enhancements. Detailed discussions with the project teams led us to the conclusion that major functional enhancements were being managed as separate development projects. Considering that corrective maintenance constitutes close to 50% of the total efforts spent, it was decided to focus on corrective maintenance activities and arrive at the possible improvements in the corrective maintenance process.

4. Corrective Maintenance Efforts and Duration Before proceeding with the details of the effort and duration, a brief description of the IEEE maintenance process is presented.

however, distinguish between the processes that need to be followed for different types of maintenance.

4.2 Effort and Duration for Corrective Maintenance Tasks For all the projects, details such as task with minimum/maximum effort/duration, the total effort spent on all the tasks was collected. Emergency patches that were applied to the system to ensure that the system continues to run were considered as corrective maintenance tasks as the same team was responsible both for the corrective maintenance and the production support process. Duration was computed as the time between the assignment of the task to the team by the project lead and the closure of the task by the project lead. The average, both for the effort and duration, was computed by dividing the total effort (duration) by the number of tasks. Table 2 presents the details of the corrective maintenance efforts and duration. For all the tasks, the effort spent on various maintenance activities as defined in Section 2.2 was collected. Table 3 shows the effort spent on various activities.

5. Analysis of the Details • • •

4.1 IEEE Definition of the Software Maintenance Process IEEE definition of the software maintenance process includes the following phases that need to be carried out in sequence. • Problem Identification, classification and prioritization • Analysis • Design • Implementation • Regression/System testing • Acceptance testing and • Delivery For each of the phases, the IEEE process defines the inputs, process, control and outputs. In addition, IEEE also classifies the problem to be of any of the following types, namely, corrective, adaptive, perfection related and emergency. Each phase consists of a set of activities that need to be performed. The IEEE process does not,

•

•

•

•

Of the total of 2777 tasks, 520 were emergency patches. The minimum effort tasks were mostly emergency patches or usage clarifications provided to the users. The minimum effort for a task varied between 17 minutes to 24 hours. It was found that for projects that had few maintenance requests, the minimum effort was not a representative value. The scatter plot in Fig 2 shows that the minimum corrective maintenance effort was usually less than 4 hours. The average effort varied from 1 hour to 33 hours and there was no correlation between the number of tasks and the average effort. The nature of the maintenance requests carried out on the system has a significant bearing on the effort. Similar conclusion can also be drawn for the maximum effort. The average duration varies from 2 hours to 119 hours across the projects. But as the clustered column graph of Figure 3 shows, almost 70% of the tasks have duration of less than 20 hours. The duration for a task was normally higher than the effort as presented in Fig 3. This is due to the reason that a team member may need a clarification or a resource before implementing the required change. Except in case of emergency patches that had to be applied anywhere within 15 minutes to 4 hours, all activities involved a peer review and the person


Average Total Effort Task of Min. Duration Average Task of Task of Total Task Of (In Duration Max. Duration (In Duration System No of Min Effort Max Effort Effort No Tasks (In hours) (In hours) (In hours) hours) (In hours) (In hours) hours) (In hours) 1 38 0.5 17 265 6.97 0.5 112 176 25.24 2 9 5 24 99 11.00 5 32 64 5.82 3 23 1 19 168 7.30 1 32 168 23.00 4 28 1 36 194 6.93 1 120 296 42.72 5 6 11 8.5 10 84 7.64 8.5 40 224 29.33 7 6 3 5 19 3.17 3 56 56 17.68 8 53 0.55 34 500 9.43 0.55 34 678 71.87 9 136 0.38 54 3311 24.35 0.38 1432 4328 177.77 10 33 0.14 105 400 12.12 0.14 39 698 57.59 11 166 3 147 2809 16.92 8 736 2736 161.69 12 11 2 10 39 3.55 2 112 224 63.18 13 209 3.85 57 2651 12.68 3.85 424 7984 629.44 14 80 1.1 122 2649 33.11 1.1 808 9520 287.50 15 16 1 10 10 10 10.00 10 10 10 1.00 17 29 1.6 23 421 14.52 1.6 216 880 60.62 18 60 0.37 25 740 12.33 0.37 248 1424 115.46 19 77 1 233 2009 26.09 1 800 3864 148.10 20 291 1.1 74 3906 13.42 1.1 248 5040 375.48 21 301 0.28 167 1016 3.38 0.28 168 952 282.04 22 51 5 16 403 7.90 5 144 1032 130.60 23 66 2.02 34 1479 22.41 2.02 248 2216 98.89 24 62 0.5 309 694 11.19 0.5 248 1632 145.80 25 37 1 233 797 21.54 1 304 2288 106.22 26 4 2.5 6 16 4.00 2.5 16 16 4.00 27 82 2 5 266 3.24 2 40 216 66.59 28 87 0.5 42 1362 15.66 0.5 248 1472 94.03 29 45 4 36 1345 29.89 4 36 1345 45.00 30 45 0.5 5.5 235 5.22 0.5 19 235 45.00 31 17 1.1 6 78 4.59 1.1 22 108 23.54 32 108 1 233 1928 17.85 1 280 3248 181.94 33 6 0.4 1.2 6 1.00 0.4 32 32 32.00 34 35 4 17 22 80 20.00 7 56 216 10.80 36 20 1.5 7 122 6.10 1.5 24 107 17.54 37 32 0.4 24 345 10.78 0.4 24 126 11.69 38 55 1.3 6.5 321 5.84 1.3 16 438 75.05 39 33 8 15 348 10.55 8 34 653 61.92 40 87 1.8 42 676 7.77 1.8 42 860 110.68 41 132 2.4 5.7 564 4.27 2.4 24 980 229.36 42 69 1.5 9 456 6.61 1.5 56 830 125.59 43 84 2.5 6 332 3.95 2.5 120 568 143.71 44 76 3.5 5 322 4.24 3.5 32 445 105.03 45 5 24 42 101 20.20 8 120 256 12.67 46 8 4 6 36 4.50 9 16 16 3.56 Table 2. Efforts and Duration for Corrective Maintenance


Serial No. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47

Activity Assign an identification number Classify the type of maintenance Analyze the modification to determine accept, reject or evaluate Make preliminary estimates Prioritize the modification Assign a MR to a block of Modifications for implementation Analyze impact of modification Analyze alternate solutions Analyze conversion requirements Analyze safety and security requirements Analyze human factors Analyze short and long term costs Conduct cost benefit analysis of modification Define firm requirements for modification Identify the elements of modification Identify safety and security issues Define test strategy Develop an implementation plan Identify affected software modules Modify software module documentation Create test cases for new design Identify/create regression tests Identify documentation update requirements Update modification list Perform coding Perform unit testing Perform risk analysis Conduct test readiness review Perform system functional test Perform interface test Perform regression test Conduct test readiness review for acceptance testing Perform acceptance test at functional level Perform interoperability testing Perform regression testing Conduct a physical configuration audit Notify the user community Develop archive Perform installation and training at customer facility Internal communication and reporting Communication with client staff Reorganize the databases and files Correction of data errors Apply emergency patch Explain the system usage Configuration management related activities Miscellaneous

Total Effort 28 43 992 255 71 28 8654 371 0 200 0 0 0 1263 569 200 834 463 375 765 1323 150 162 1654 2321 0 199 370 106 154 412

157 42 106 654 1078 2116 2539 2173 1075 1124 576

Table 3. Distribution of Efforts for Various Activities of Corrective Maintenance


Fig 2. Corrective Maintenance Minimum Efforts – Tasks

Fig 3 Duration – Effort in Corrective Maintenance

• •

•

conducting the peer review clocked his effort against the same activity, such as coding, testing, etc. There was no effort spent on conversion requirements and human factors during the corrective maintenance activity. There was no effort spent on cost benefit analysis. This is probably due to the reason that corrective maintenance is normally an activity that needs to be carried out due to a system malfunction. It is probably more applicable in case of perfection related maintenance. There was very little effort that was spent on safety and security related issues. It was found on discussions that any changes to the authorization schemes to the systems normally happen due to functional changes/enhancements.

•

•

•

Due to most of the changes being localized to a single software component, there was normally no need to conduct the system test, except in isolated cases where the maintenance request had an effect on a significant number of system components. To this extent, a corrective maintenance request never resulted in a need to conduct a regression test or interoperability test for the system. More significantly, project teams were spending significant efforts, close to 25%, on some activities that were not listed as part of the IEEE process such as database reorganization, data related problems, configuration management related activities, etc. About 5% of the total effort was spent on conference calls between the clients and the Syntel team and also for onsite - offshore communication.


•

•

•

• •

•

•

It was also interesting to note that some maintenance requests do not result in any change of code but clarifications and training to the users. This is reflected by close to 1000 hours being spent on providing clarifications on system usage. Maintenance of mission critical systems involves strict adherence to check-in and checkout procedures to make sure that the right versions of software components are in production. About 3.3% of the efforts of the team were spent in configuration management activities. The archival of the software and data was not the responsibility of the corrective maintenance team and hence no efforts were spent by the team on these activities. This is probably due to the fact that most of the systems ran on the mainframe platform and the data center staff at the customer location was responsible for this activity. 412 hours were spent by the team on acceptance support while the client did the acceptance. Close to 30% of the total time is spent on problem analysis. This is not really surprising, as the team that maintains the systems in an outsourcing deal is usually not the team that developed it. The efforts are spent in analyzing the nature of the problem, conducting impact analysis and identifying the software components that need to be modified. The analysis some times may lead to correction of data errors from the feeder systems. The team spent 7.5% of the effort in correcting the data errors. After the analysis function, the maximum effort of 16.5% was spent on the testing activity. There is no data available in the public domain on the distribution of efforts for various maintenance activities in large outsourcing projects. Hence the effort distribution could not be compared with any available benchmarks. The effort spent on activities impact analysis, coding, and testing, which constituted close to 47% of the overall effort, was in line with what the project managers expected. However, the effort on activities of database reorganization, correction of data errors varied from one system to another and was dependent on factors such as system stability, user training and efficiency of job scheduling. These factors were not analyzed in detail as part of this study.

6. Conclusions After a detailed study of industrial reports, we concluded that software maintenance outsourcing will be a growing business area of future. Due to a growing demand for better service levels and increased pressures

on rates, there is a need to arrive at better processes to ensure cost-effective delivery. We conducted empirical studies to analyze the maintenance mix and effort distribution on 46 software maintenance projects that support various lines of business on the IBM mainframe platform. Taking the IEEE software maintenance process as the standard, we measured the effort and duration for various corrective maintenance tasks and also the efforts spent on various activities. Our analysis of the effort distribution shows that the processes for each type of maintenance are different and there is a need to fine-tune them especially in the context of outsourcing. While problem analysis and testing form a significant part of the corrective maintenance effort, teams spend efforts on other activities such as database reorganization and configuration management that are not defined by the IEEE maintenance process. Good knowledge management tools that capture information about various software components and facilitate enquiry could result in reduction of the efforts spent on analysis. Similarly, testing tools could help the teams in reducing the effort spent on conducting the unit/system testing. We also found that some maintenance requests do not result in any change of code but clarifications and training to the users. Finally, effective communication between the customer and the outsourcing team and also within the outsourcing team could significantly improve the turn around times for maintenance requests.

7. Future Research The authors are currently working on the following areas associated with outsourcing: • Defining the value metrics of the outsourcing engagements based on customer objectives. • Defining the knowledge management and resource management models based on customer objectives. The current focus of the authors is the software maintenance outsourcing of customized commercial systems and also this study was limited to IEEE definition of maintenance types and the IEEE maintenance process. An alternative definition of maintenance types has been presented by Ned Chapin and others [10]. The authors propose to examine the applicability and impact of this classification of maintenance types on corrective maintenance outsourcing in their future research. The issues involved with maintenance of real time systems and the COTS systems (which traditionally have wide user distribution) are significantly different and can be examined. The impact of cultural and time zone differences on the productivity in outsourcing engagements is an interesting area for all the types of


engagements, maintenance.

namely

development,

migration

and

8. Acknowledgement One of the authors, Baru. S. Rao would like to thank the management and employees of his employer, Syntel Inc., for providing access to the metrics data and also for their valuable feedback.

9. References 1. 2. 3. 4. 5.

NASSCOM Report, “The IT Software and Services Industry in India Strategic Review 2001” Information on Indian software industry from NASSCOM available at www.nasscom.org Thomas M.Pigoski, “Practical Software Maintenance” John Wiley & Sons, Inc, 1997 IEEE Standard for Software Maintenance, IEEE Standard 1219-1998, The Institute of Electrical and Electronics Engineers, Inc, 1998 Mira Kajko-Mattson, “Corrective Maintenance Maturity Model (CM3): Maintainer’s education and training”, Proceedings of the 23rd International Conference on Software Engineering”, pp 610-619, May 12-19, 2001, Toronoto, Canada.

6.

Mira Kajko-Mattson, “Taxonomy of Problem Management Activities”, Proceedings of the International Conference on Software Engineering”,http://Kompetenswebben.artisan. se/metoder/ProbManTax.pdf 7. Edward Yourdon, “When Good Enough Software is Best”, IEEE Transactions Software, pp 79-81 May 1995 8. K.H.Bennett, V.T.Rajlich, “Software Maintenance and Evolution: a Roadmap”, Proceedings of the International Conference on Future of Software Engineering 2000, Limerick, Ireland, June 4-11, 2000 9. Quality Management System and related documents of Syntel Inc. 10. Ned Chapin et al “ Types of Software Evolution and Software Maintenance”, Journal of Software Maintenance and Evolution: Research and Practice, Volume 13, Issue 1, 2001 pp 3-30. 11. “Outsourcing Services Market Prospers in Asia

/Pacific Region” research notes of IDC available at www.idcindia.com 12. Presentation of Mike Dodd of GIGA on India Outsourcing available at www.gigaweb.com
