Software Testing in Latvia: Lessons Learned

3 downloads 40180 Views 108KB Size Report
1999 - Latvia's two largest software producers - DATI and SWH. Technology - exported products worth a total of 7 million DEM. 2000 - Up to 100 local companies ...
Software Testing in Latvia: Lessons Learned Dr.math. Anda Adamsone, SWH Technology Dr.hab.sc.comp. Juris Borzovs, RITI M.Sc.comp. Martins Gills, RITI M.Sc.comp. Sanda Linde, IT Alise M.Sc.comp. Janis Plume, RITI 1

Testing in Latvia: a look into history Early research - Automatic construction of test cases - theoretical and practical approach. Courses on software testing are included into undergraduate computer curricula of Latvian Universities for more than a decade. Recent and current research in testing: • • • •

software test tools universal symbolic interpretation software process improvement; testing issues practical manual methods of software testing

ICSTEST 2000

Software Testing in Latvia: Lessons Learned

2

Testing in Latvia: major publications 1. J.M.Barzdins, J.J.Bicevskis, A.A.Kalnins. Construction of Complete Sample Systems for Correctness Testing. 1975 2. J.M.Barzdin, J.J.Bicevskis, A.A.Kalninsh. Automatic Construction of Complete Sample Systems for Program Testing. 1977 3. A.Auzins, J.Barzdins, J.Bicevskis, K.Cerans, A.Kalnins. Automatic Construction of the Test Sets: Theoretical Approach.1991 4. J.Bicevskis, J.Borzovs, U.Straujums, A.Zarins, E.F.Miller,jr. SMOTL - A System to Construct Samples for Data Processing Program Debugging. 1979 5. J.Borzovs, A.Kalnins, I.Medvedis. Automatic Construction of Test Sets: Practical Approach. 1991 6. Z.Bicevska, J.Bicevskis, J.Borzovs. Regression Testing of Software System Specifications and Computer Programs. 1995 ICSTEST 2000

Software Testing in Latvia: Lessons Learned

3

IT in Latvia: a snapshot 1991 - First international development project in by Latvian software designers to set up an information system for the social insurance of artists in the German state of Bremen. The successful result gave the basis for the local software industry. 1999 - Latvia's two largest software producers - DATI and SWH Technology - exported products worth a total of 7 million DEM. 2000 - Up to 100 local companies are engaged in SW development. Three major of them, DATI, SWH Technology and IT Alise, employ more than 650 employees, IT professionals. 2000 - 5 computer manufacturers and 2 software companies in Latvia have obtained the ISO 9001 certificate.

ICSTEST 2000

Software Testing in Latvia: Lessons Learned

4

The quest for quality Initial stage - any computer program is more or less OK Current move - software as a company’s business process backbone. The outmost priority is quality. Customer Satisfaction

Efficient Work

Product Quality

Profit

Competitiveness ICSTEST 2000

Software Testing in Latvia: Lessons Learned

5

Awareness of testing Customers require quality

Quality related activities within the software development life-cycle

• • • • • • • •

identification, documentation, tractability, verification, validation, reviews, audits, testing, problem resolution ...

What is customer going to pay for? They are interested in the least expensive, but at the same for time the most evident and efficient activities. Historically, the most widespread and the most acceptable is testing. Although developers understand its importance, only a minor part of customers are aware. ICSTEST 2000

Software Testing in Latvia: Lessons Learned

6

IT standards: origin 1993-1995 Leading IT company DATI (formerly SIS) launches the initiative to examine internationally used software engineering standards. Goals: • Searching for world-known and the most informative standards • Assess the experience from IT projects • Guidelines, standards, templates Output: • List of industry standards • IT terminology in Latvian • Company standards, guidelines; mostly based on IEEE • Changes in development process management ICSTEST 2000

Software Testing in Latvia: Lessons Learned

7

National IT standards Most of Latvian National IT standards were approved in 1996, adopting DATI internal standards based on IEEE standard family. Areas covered: • • • • • • •

Software Quality Assurance, configuration management, reviews and audits Testing, verification and validation Software documentation SW requirements specification, Operational concept description Guidelines for Software design descriptions Project management plans User Documentation ICSTEST 2000

Software Testing in Latvia: Lessons Learned

8

IT standards: industry practice For couple of years in Latvia, large business software development, especially documentation, is based on IEEE J-STD-016-1995 Software development: Acquirer-supplier agreement standard. Main features: • • • •

All software development project activities specified Major document templates given Includes guidance for tailoring to a certain project Built-in quality assurance mechanism

ICSTEST 2000

Software Testing in Latvia: Lessons Learned

9

Software development processes ISO/IEC 12207 defines 5 primary, 8 supporting and 4 organizational life cycle processes. Industry uses a reduced version of ISO/IEC 12207. For example, DATI has adopted the standard identifying the following processes: 3 primary processes: Development, Reengineering and Maintenance. 3 supporting processes: Documentation, Configuration management, Quality Assurance 2 organizational processes: Management and Training

ICSTEST 2000

Software Testing in Latvia: Lessons Learned

10

Who performs testing? (1) Testing laboratory / department

Project A Testing function is provided by a separate testing laboratory

Project B ...

Advantages: United methodology, pool of testing professionals, independence.

Project Z

Disadvantages: Testing mostly impossible at unit or integration level. ICSTEST 2000

Software Testing in Latvia: Lessons Learned

11

Who performs testing? (2) Each project has a small testing group Advantages: Testing at all levels, high knowledge about software under test. Disadvantages: Reduced independence, may not be testing professionals but rather programmers, designers or analysts.

ICSTEST 2000

Project A Testing group

...

Project B Testing group

Project Z Testing group Software Testing in Latvia: Lessons Learned

12

Demand for testing professionals What qualifies a testing professional?

Typical project mistake - testing is given to person who is bad at programming, designing or system analysis. Testing professional is a software engineer specialized in testing. His/her main task is to have a different look at development activities. How to become a testing professional? Currently a course on testing is included as a part in the curriculum of Latvian SWEPE (Software Engineering Professional Education) project.

ICSTEST 2000

Software Testing in Latvia: Lessons Learned

13

Tester’s body of knowledge Initiated by SWEPE project, a certain analysis has started to define a knowledge area for the testing professional. Key areas: • Basic concepts and definitions of testing • Analysis of test levels • Test techniques • Test automation • Test related measures

ICSTEST 2000

Software Testing in Latvia: Lessons Learned

14

Qualification levels of the tester Level 1

2

3

4

Tasks Execution of previously specified tests and other tasks that do not require special knowledge or training Tasks of 1st level + Test description according to system documentation. Tasks of 2nd level + Test planning and managing of test team Tasks of 3rd level + Training of other testers and development of testing methods

ICSTEST 2000

Required skills Computer usage skills - advanced used level and ability to understand specified tests. Skills of 1st level + Ability to analyze system documentation and to write a testing documentation. Skills of 2nd level + Ability to identify required tests, skills in team and resource management. Skills of 3rd level + Tuition skills and expert knowledge in testing theory

Software Testing in Latvia: Lessons Learned

15

Tester’s role Test engineer is not just a resource! Test engineer =

Functional tests Non-functional tests Test tool development

This is a person specialized on testing in a categories of development platforms, test types and system application areas. ICSTEST 2000

Software Testing in Latvia: Lessons Learned

16

Collaboration among the developers

System analyst / designer

Requirements

Problem reports Requirements

Tester

Code

Programmer Problem reports

ICSTEST 2000

Software Testing in Latvia: Lessons Learned

17

Test levels and performers

Test level

Structural tests

Unit testing Integration testing System testing Qualification testing Acceptance testing

+ +

ICSTEST 2000

Functional tests

+ + + +

Performed by developers developers developers, independent testers developers, independent testers customer, independent testers

Software Testing in Latvia: Lessons Learned

18

Tasks of tester Type of software development activity System analysis, specification of requirements Design Coding

Testing

Prior to delivery (preparation for the acceptance)

ICSTEST 2000

Tasks for tester Verify whether requirements specification and other documents are suitable for testing. Plan the testing: its tasks and schedules. Plan in detail the tests: type, coverage. Make test descriptions. Refine plans and test descriptions. Execute tests. Make automation and regression tests. Thoroughly implement the system, installation and documentation testing. Do regression testing. Check-up the corrections. Run the regression tests.

Software Testing in Latvia: Lessons Learned

19

Well defined testing process Test planning

Test identification

Test execution Review of tests and their results ICSTEST 2000

Software Testing in Latvia: Lessons Learned

20

The absolute minimum of testing What to do when no testing is planned? Problem: customers refuse to pay for testing activities (assuming that product will be perfect). When the original contract agreement does not provide enough resources for testing, DATI practices the allocation of the extra staff to the project to keep the expected quality of the product.

Project A

ICSTEST 2000

Software Testing in Latvia: Lessons Learned

21

Educating the customer Commonly asked customer questions are: What is testing? Why is it necessary for the particular project? Why it costs so much?, etc. How does the developer benefit from a well informed and competent customer? Main results: • It is possible to develop an appropriate testing strategy for the project • Better collaboration between acquirer and supplier in the reporting and solving the problems • Customer can do the acceptance testing ICSTEST 2000

Software Testing in Latvia: Lessons Learned

22

A case study (1): Testing process for porting project Task: to port software from 16bit to 32bit environment

Test cases

Test coverage extension

Test operation

Initial state for testing: test cases cover 25% of functionality. Recordplayback regression testing is done.

Test results review Test coverage maintenance

ICSTEST 2000

Problem registration

Problem reports

Software Testing in Latvia: Lessons Learned

23

A case study (2) measurements 6 4

Testing

3

Testing makes up 40% of the total project effort.

Coding

2 1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Months

After 15 months there was no evidence found that 20% source code (modules) contains 80% problems. The defects were very evenly disseminated across the source code. ICSTEST 2000

Lines of code in module

Persons

5

100000 90000 80000 70000 60000 50000 40000 30000 20000 10000 0 0

50

100

150

200

250

Updates in module

Software Testing in Latvia: Lessons Learned

24

Future vision The goal for the next 10-20 years The Baltic States - a "second IBM" - a unified "concern" with 120,000 highly qualified specialists. • Baltic States - exporter of software services • Design and maintenance of information systems and software products - a "trademark" for the Baltic States

ICSTEST 2000

Software Testing in Latvia: Lessons Learned

25

Latvian software export income mill. Ls 5,000 4,073 4,000 3,000 2,000

1,142 7

1,000 7 1999

153 25 2005

78 2010

181 2020

year

Income from Latvia’s exports in the sector if it is not developed Income from Latvia’s exports in the sector if it is developed

ICSTEST 2000

Software Testing in Latvia: Lessons Learned

26

Employment in IT sector People employed in the export-oriented software sector

Number of people

25,000 22,760 20,000 15,000 8,743

10,000 5,000

225

2,460 398

1,012

597

0 1999

2005

If the sector is not developed If the sector is developed

2010

2020

years

Students in software design

No. of people

6,000 4,850

5,000

4,850

4,850

4,000 3,000 2,000 1,000

1,400

1,400

1,400

1,400

1999

2005

2010

2020

0 years

If the sector is not developed If the sector is developed

ICSTEST 2000

Software Testing in Latvia: Lessons Learned

27

Expected role of testing (1) In a context of a planned increase of remote development projects in the Baltic region, DATI has analyzed its current experience. Activities that suit best the remote development are: • • • • •

design coding testing maintenance reengineering and porting

Testing is among the top ones, mainly because of its task to be an independent verifying activity.

ICSTEST 2000

Software Testing in Latvia: Lessons Learned

28

Expected role of testing (2) Main factors that define the increasing role of testing: • • • •

Increasing complexity of the software Dependence from the software in almost every area Increasing number of interfaces between various programs A large number of reengineering and porting tasks

Software testing - Latvian IT industry sector to boom

ICSTEST 2000

Software Testing in Latvia: Lessons Learned

29

Questions ? Contact information: Riga Information Technology Institute Kuldigas iela 45 LV 1083 Riga Latvia E-mail: [email protected] http://www.riti.lv tel.: + 371 761 15 22 fax: + 371 761 95 73

ICSTEST 2000

Software Testing in Latvia: Lessons Learned

30