Teaching global software engineering: experiences and lessons learned

45 downloads 16231 Views 309KB Size Report
Information and Computer Science Department, King Fahd University of Petroleum and ... classes [10], the use of new technology [11, 12] and designing courses ...
IET Software Research Article

Teaching global software engineering: experiences and lessons learned

ISSN 1751-8806 Received on 27th June 2013 Revised on 17th September 2014 Accepted on 27th February 2015 doi: 10.1049/iet-sen.2014.0042 www.ietdl.org

Mahmood Niazi ✉ Information and Computer Science Department, King Fahd University of Petroleum and Minerals (KFUPM), Dhahran, Saudi Arabia ✉ E-mail: [email protected]

Abstract: In this paper the author demonstrates how the GSE course was executed related to each level of the Bloom's taxonomy. It was observed that Bloom's taxonomy can play a vital role in order to effectively design and deliver a GSE course. Overall, the six levels of Bloom's cognitive domain were helpful in transforming students learning from the knowledge level to the evaluation level. However, it was observed that the knowledge level is not appropriate for higher calibre students because the content used in the knowledge level is often referred to as simple. The comprehension level seemed appropriate for all levels of students as they were able to produce different solutions based on their understanding of the problem domain. Many international students had problems with the application level as they were unable to apply the concepts and theories in the laboratories. For the analysis level, some students had difficulties in breaking down the bigger problem into smaller components. Students had mixed responses to the synthesis level. Students had some minor problems in the evaluation level. However, it was observed that if Bloom's taxonomy is used with other available teaching guidelines, the overall design and delivery of different courses can be improved.

1

Introduction

In the past two decades, there has been much concern in the UK about the need for higher education to develop the personal competencies of its students in particular academic disciplines [1]. This concern has motivated the redesign of undergraduate and postgraduate degree courses, with a stronger emphasis on the needs of employers [2]. In addition, there have been a few studies on improving teachers’ learning process so that a quality curriculum can be designed [3–9]. Moreover, there has been a trend towards distinguishing between teaching small and large classes [10], the use of new technology [11, 12] and designing courses required by the software industry, such as global software engineering [13, 14]. Over the past decade, many firms have started to engage in global software engineering to reduce software development costs. Previous work suggests that half of the companies that have tried global software engineering have failed to realise the anticipated outcomes which has resulted in poor global relationships, misunderstanding of the projects’ requirements, high costs and poor services [15]. There are many reasons for these failures [16], the most common being ‘language and cultural barriers’, ‘country instability’, ‘lack of project management’, ‘lack of protection for intellectual property rights’ and ‘lack of technical capability’ [16]. Other reasons were complex communication and coordination problems in conditions of time and cultural separation [17], difficulties developing software development outsourcing practices, creating confidence and trust among the outsourcing companies and managing the expectations of what can and what cannot be done in a distributed setting [18, 19]. Most of these are because of the lack of effective training programmes which can provide sufficient knowledge and skills relating to global software engineering [16]. In addition, a small number of academic institutions provide different courses on global software engineering which are not sufficient to satisfy the requirements of different employers [20]. Moreover, advances in the development of systems have not been matched by equal advances in the development of new subjects in academia which have resulted in a

IET Softw., 2015, Vol. 9, Iss. 4, pp. 95–102 & The Institution of Engineering and Technology 2015

gap between the software industry and academia. An understanding of theoretical issues and the development of practical skills in the modern age are the key success criteria for any system development in the 21st century. Some work has been done to design courses in the domain of global software engineering [13, 14, 20, 21]. Crnkovic´ et al. [13] identified ten tips to be successful in global software engineering education. These tips are based on the authors’ experience of nine years in which they considered the most important issues to increase the probability of success in teaching a global software engineering course. Fortaleza et al. [20] conducted a literature review to consolidate the body of knowledge on the topic of global software engineering. In addition, they proposed the creation of a collaborative repository to store and report the experiences of teaching global software engineering. Most recently, Paasivaara et al. [14] described distributed scrum with the best practices in global software engineering in order to teach critical competencies in global software engineering. They found that ‘the scrum method, along with supporting collaboration practices and tools, supports the learning of important global software engineering (GSE) competencies, such as distributed communication and teamwork, building and maintaining trust, using appropriate collaboration tools and inter-cultural collaboration’ [14]. The work in this paper complements the work previously done in these studies. However, to the best of my knowledge, Bloom’s taxonomy has not been used in global software engineering courses. The objective of this paper is to share experiences of teaching a global software engineering course, based on Bloom’s taxonomy. In this paper, I demonstrate how the global software engineering course was executed related to each level of the Bloom’s taxonomy. Different challenges relating to the six levels of Bloom’s cognitive domain are also discussed in this paper. Owing to the space restriction, the planning part of this work was previously published as a short paper [22]. This paper is a revised and substantially extended version in which the experience of teaching a global software engineering course, based on Bloom’s taxonomy, is discussed. The long-term research goal of this

95

research is to provide academics with a body of knowledge that can help them to plan, design and deliver better courses. This paper is organised as follows. Section 2 describes Bloom’s taxonomy. Sections 3–6 describe the planning, preparation, delivery, design and assessment of global software engineering courses, based on Bloom’s taxonomy. Section 7 provides a discussion provided and Section 8 presents the conclusion.

2 2.1

Background Introduction to Bloom’s taxonomy

There are a number of ways that a person can learn and these are referred to as learning styles. A learning style is the preferred way (s) in which individuals interact or process new information across the three domains of learning identified in Bloom’s taxonomy of educational objectives: cognitive (intellectual capability, i.e. knowledge or ‘think’), psychomotor (manual and physical skills, i.e. skills or ‘do’) and affective (feelings, emotions and behaviour, i.e. attitudes or ‘feel’) [23]. Academics often refer to these three domains as knowledge (cognitive), skills (psychomotor) and attitude (affective). This taxonomy can be thought of as the goals of the learning process. That is, after the learning session, the learner should have acquired new knowledge, skills and/or attitudes. The three domains of learning are organised as a series of levels or subdivisions, starting from the simplest behaviour to the most complex. These levels can be thought of as degrees of difficulties. It is suggested that one cannot effectively address higher levels until those below them have been covered and provides a basic sequential model for dealing with topics in the curriculum. These divisions are not absolutes and some revision has been suggested by other academics [24]. However, Bloom’s taxonomy is easily understood and is probably the most widely applied today. 2.1.1 Cognitive: The cognitive domain is the most commonly used domain of Bloom’s taxonomy. This domain involves knowledge and the development of scholarly skills. The focus in the cognitive domain revolves around knowledge, comprehension and thinking through a particular topic. There are six levels in this domain, moving through the lowest-order processes to the highest: knowledge, comprehension, application, analysis, synthesis and evaluation. 2.1.2 Psychomotor: The skills associated with the psychomotor domain describe the ability to manipulate a tool or instrument or to perform a manual or physical task. It also covers modern trends such as communications and the operation of IT equipment, for example, telephone and keyboard skills or public speaking. The psychomotor domain usually focuses on change and/or development in behaviour and/or skills. 2.1.3 Affective: This domain provides the framework in which we deal with things emotionally, such as feelings, values, motivations and attitudes. The five major categories are: receiving, responding, valuing, organising and internalising values. 2.2

Research design method

The case study method was used because this method is said to be powerful for gaining a deep understanding of a phenomenon in the real-world environment [25]. The case study also provides valuable insights for problem solving, evaluation and strategy. The objective of this paper is to demonstrate how the global software engineering course can be executed using each level of the Bloom’s taxonomy. Since the global software engineering is more applicable to a real-world environment, the case study research method is believed to be a more appropriate method for this situation. The sample population consists of software engineering postgraduate students. All the participants have four year bachelor degree in software engineering. All of the participants have at least

2 years experience in software engineering. The sample is not random but a convenience sample, because responses sought from those people who were available and willing to take part. In the beginning of this course, I talked to each participant face-to-face, explained what the case study was about and handed out a hard copy of the course material. A combination of different methods was used for data collection, that is, observation, face-to-face discussion, document analysis and questionnaires. These methods were used during different stages of the course, that is, planning, delivery, assessment and teaching evaluation. For data analysis, simple qualitative analysis was used to conclude the findings.

3 Planning and preparation of a global software engineering course For the design of a global software engineering course, I adapted Bloom’s cognitive domain as it provided me with opportunities to teach students in different stages, that is, from knowledge to evaluation. In addition, in my teaching area of global software engineering, the focus is on understanding the theories first and then applying these theories to practice. When I was asked to design and deliver a postgraduate course on global software engineering, the real challenge for me was in designing a piece of work that would motivate students. Another challenge was to design a course in a distributed environment for a diverse range of students. There were many international students in this course; however, no disabled students were enrolled in the course. I reviewed the literature on student motivation [7, 26, 27], teaching a global software engineering course [28–30] and existing learning theories [23]. It was clear to me that in order to effectively manage a diverse class and to motivate students, I needed to create a welcoming and engaging environment in the classroom. It was also clear to me that I needed to provide equality of opportunities in terms of access, treatment and outcomes. For this, I used the following guidelines for the design of the course: † Increase the difficulty of the course as the semester progressed (i. e. from knowledge to evaluation [23]). † Use a variety of teaching methods, that is, small group work and case studies (during tutorials) and discussion (during lectures and tutorials) in order to improve students’ motivation and to create a welcoming and engaging environment [31]. † Prepare handouts in advance for the class and make these handouts available on the course web site so that students have an opportunity to read them before they come to the lecture. † Instead of going over all the material provided in the handout, go over the difficult points and then provide students with additional examples in order for them to better understand the difficult parts of the topics. † Publish all the assessment dates at the beginning of the semester on the course web site and also in the handouts so that students can better plan their time during the semester. In addition to these pedagogical guidelines, I also created the following five key competencies required in a global software engineering course [21]: 1. Understanding of the difference between in-house development and global software development (GSD). 2. Awareness of the impact of software quality and different standards and models for software quality on GSD. 3. Understanding of the management of virtual teams and cross-cultural collaboration. 4. Understanding of global software project management. 5. Measuring project progress, productivity and other aspects of GSD. Previously, a textbook had been approved for the course. However, I realised that the recommended textbook was not a

IET Softw., 2015, Vol. 9, Iss. 4, pp. 95–102

96

& The Institution of Engineering and Technology 2015

good match for the course as it was too vague for the particular purposes of the course. In consultation with senior colleagues, I chose a new textbook for this course. 3.1 Reflection on planning and preparation of the global software engineering course based on Bloom’s taxonomy I adapted the concept of Bloom’s cognitive domain as it provided me with a useful framework for transforming theories into practice and application. The following section illustrates the application of the six levels of Bloom’s cognitive domain in the design and planning of the global software engineering course. ‘Knowledge’ (learners are expected to store or recall in their mind data or information of previously-learned materials): I designed review questions after each lecture to encourage the students to recall the contents of the lectures. The five competencies required in the GSD course described in Section 3 were used in these review questions. For example, I used the following questions in the knowledge level of Bloom’s cognitive domain: † What is the difference between in-house development and GSD? † What are the different models of cross-cultural collaboration? † List four criteria for GSD project success. The knowledge level of Bloom’s cognitive domain helped the students to recall the contents of previously delivered lectures. However, this level did not seem appropriate for higher calibre students because the content in the knowledge level was tiring and even tedious for these students, and in the teaching evaluation survey, the students referred to it as ‘simple’. This is because the high calibre students were more mature and it was expected that they would like to prove their academic skills at a higher level, rather than being restricted to the lowest level of the cognitive skills. When designing the global software engineering course, I should have ensured that, in the first few lectures, the students were presented with course content which was adequate for their intellectual skills. This issue should be taken into consideration for new course design in the future, to provide students with adequate intellectual skills relevant to their calibre. However, the use of the

knowledge level seemed more appropriate for those students who appreciated a discussion of the basic concepts and principles of global software engineering. I observed this appropriateness during conversations with my students and in the teaching evaluation survey. ‘Comprehension’ (state a problem in one’s own words using one’s own understanding or interpretation of facts): I covered comprehension by providing students, in different workshop sessions, with a better understanding of the five key competencies required in the GSD course. The comprehension level of Bloom’s taxonomy seemed appropriate for all levels of students as I noted that students were able to produce different solutions based on their understanding of the problem domain. This helped students to move from a ‘recalling’ state to an ‘understanding’ state [23] in which they were able to state a problem in their own words, based on their understanding. For example, in the workshops, I asked students to describe the pros and cons of the GSD model described in Fig. 1 and most of the students were able to display this level of comprehension. ‘Application’ (application of new knowledge, learned in the classroom, into novel situations in the workplace): For this level, I designed different laboratory sessions in which the students have to apply the concepts and theories learned in the lectures to practical situations. Students also have to present the outcome of the projects in the class for comparison and discussion with other students. I used key competency number 3 (i.e. understanding of the management of virtual teams and cross-cultural collaboration) for this project in which students had to improve the communication and coordination models used in GSD. For example, I gave the centralised communication and coordination model in Fig. 2 to students and asked them to suggest ways to reduce the number of communication channels. Students reviewed the relevant literature to address this problem. Different alternative models were proposed by students. This was one of the true GSD activities which are commonly used in global projects. For the application level, I observed that some of the international students (from Asia) had difficulties in applying the concepts and theories, learned in the lectures, in the laboratories. They needed more support and supervision in creating different models or diagrams in the laboratories. After discussion with these international students, I realised that the education systems in their

Fig. 1 Local programmes of global projects [30]

IET Softw., 2015, Vol. 9, Iss. 4, pp. 95–102 & The Institution of Engineering and Technology 2015

97

Fig. 2 Centralised communication and coordination model [30]

home countries were designed in such a way that they were given less opportunities to apply the concepts and theories learned in the lectures to practical situations. This led me to address the diversity of students’ issues [32]. Hence, I developed a new practice which attempted to address the individual needs of students. I created specialist support for such students in the form of providing them with extra time and resources. Special attention was given to these students where I gave them easy application questions first and then moved to slightly harder questions. This practice worked well, as I noted that the learning of these international students about the application of knowledge was improved, as these students were able to answer questions and also they were able to complete tasks in the laboratories. In addition, the demonstrators also provided me with positive feedback about the learning improvement of these international students. ‘Analysis’ (examine and break information into parts so that its causes can be understood): For the analysis level, I gave students different problems for analysis in the laboratories as well as in their assignments. For analysis, key competency number 4 was used (i.e. understanding of global software project management). The concept of work breakdown structure was used in which students had to break down a bigger problem into smaller components in order to better understand it. After analysis, they are required to provide solutions for each component. In other cases, I asked students to identify different parts of the systems, analyse the relationship between these parts and provide solutions for each part. For the analysis level of Bloom’s cognitive domain, I observed that some students faced difficulties in breaking down the bigger problem into smaller components in order to better understand it. To address this problem, I provided students with simple problems first and then moved them to harder problems. I discussed with the demonstrators if there were any improvements in students’ learning after using this technique. I received positive feedback from the demonstrators that students had learned that large and complex problems cannot be managed and controlled as a whole, and that they need to be broken down into smaller components in order to achieve the required control and ability to monitor progress. After a few weeks, I observed through their marks in the assessed practical sessions that most students were able to analyse given problems and they achieved good marks in their laboratories.

‘Synthesis’ (put parts together to form a whole, a new pattern or alternative solutions): I encouraged students to create something new by making predictions and solving problems. For different assignments and final project reports, I asked students to synthesise different solutions identified in the analysis section to form a new portfolio. Students had mixed responses to the synthesis level of Bloom’s cognitive domain. Some students were able, without any help, to synthesise different solutions to form a new portfolio and this was depicted in their final marks. However, some students displayed inconsistency problems, that is, they synthesised irrelevant solutions to a single portfolio. In addition to my own observations, the demonstrators also reported the same problem. I overcame this problem by providing students with more examples of synthesis. However, I had a feeling that these types of problems needed more attention and that students would be able to better understand synthesis issues if they were given scenarios or examples from real-world problems. I also think that such preliminary exercises should be done at a high school level so that when students come to university, they will be able to understand and complete more complex synthesis problems. ‘Evaluation’ (make judgements about the value of ideas or materials based on a set of criteria): For the evaluation level, I designed a project in which students had to make a decision as to whether in-house development or GSD is appropriate, based on a given scenario. I encouraged students to use their acquired knowledge and relevant research to provide a solution to a problem in the project. The project was about a health clinic which has decided to computerise its office systems. Information about number of technical staff members, budget, technical expertise of staff members, available equipment and the timeframe for the new system was provided. Students were asked to justify: † What conclusions do you make from your critical assessment of this problem? † Can you trust recommendations of the literature? † What are your recommendations for the health clinic as a result of your research and acquired knowledge – for instance, should the health clinic consider these recommendation suitable for its office

IET Softw., 2015, Vol. 9, Iss. 4, pp. 95–102

98

& The Institution of Engineering and Technology 2015

systems development or should the health clinic be cautious about these recommendation? † Should the health clinic go for in-house development or GSD? Students were asked to provide their recommendation with appropriate supporting evidence as to the type and forms of GSD (or in-house development) they would recommend, in addition to an outline plan and typical list of risks. Students had some minor problems in the evaluation level of Bloom’s cognitive domain. Some students did the evaluation based on limited knowledge, limited research and using limited factors. This problem was reduced with the passage of time and by requiring the students to complete more evaluation practice. This was evidenced by the demonstrators’ feedback to me and my own observations of the students’ work in the past few weeks of the semester.

4

the lectures. These mixed groups helped them to interact and learn from each other. This was depicted in the assessment and in chatting with the students. The integration of lectures and practical classes also helped students to better understand the module and the application of theories learned in the lectures to practical situations. 4.1 Reflection on delivery of the global software engineering course based on Bloom’s taxonomy During the delivery of the module, I realised that, in addition to Bloom’s cognitive domain, some of the categories of Bloom’s affective domain might be useful for students’ motivation, if used in the future delivery of modules. This is because this domain describes the manner in which we deal with things emotionally, such as enthusiasm and motivation. In future, I will adapt the following guidelines from this domain in order to effectively deliver modules and improve students’ motivation (as shown in Table 1).

Delivery of global software engineering course

The global software engineering subject was presented as a set of lectures and weekly computer laboratory sessions. Delivery of this module was challenging because of the nature of the course. Other challenges were managing demonstrators responsible for practical work and establishing a good relationship with students. I used the following guidelines for the successful delivery of the module [33]. I chose the O’Brien guidelines because these guidelines were designed for the effective management of a diverse class. † Provide optimal course information using different resources, that is, web site, university course handbook etc. † Explain course outline to students in the first lecture. † Create a welcoming and engaging environment. † Integrate lectures and practical classes. † Use a variety of presentation strategies, that is, text, models, diagrams and video (how to conduct interviews). † Establish small working groups.

5 Assessment and feedback of global software engineering course To comply with the school’s policy, it was necessary to have the following three types of assessments for the global software engineering course. Each assessment was peer reviewed for quality assurance purposes: † Workshop/practical activity worth 10%. † Assignments (two) worth 30%. † Final exam worth 60%. There were two assignments worth 15% each. The students were given three weeks to complete each assignment. It was the Table 1 Applying Bloom’s affective domain in the delivery of future modules

The O’Brien guidelines helped me in the smooth delivery, coordination and management of a diverse class. At the start of the semester, I created a module web site in order to effectively communicate with students and to provide them with all the module documentation (i.e. lecture notes, information about practical classes and reading material). The module web site helped ensure a smooth start to the module as there were limited queries from students, as all the relevant information was available on this web site. The objective of establishing small groups in the laboratories was to provide the students (especially the international students from Asia) with opportunities to learn from each other’s experience and to improve their communication skills. However, there was a problem in the laboratories during group work, as discussed below: ‘Incident in the laboratory: In the laboratories, I created three groups of students so that students could have an opportunity to interact and learn from each other. Another objective was to improve the international students’ communication skills. Before the group work, I asked my students to form three groups of their own choice. I noted that the international students always formed one group and the local students formed the others. This was not helping me to achieve the laboratories’ objectives. Therefore I changed this practice and informed the students that, in future, I would decide which students would be allocated to the various groups to ensure there was a mix of local and international students in each. This new practice helped the international students improve their communication skills (based on observations from the beginning of the semester to the end of the semester and I noted an improvement in their communication skills, i.e. their speaking improved and they were able to write good essays).’ As mentioned earlier, some of the international students (from Asia) had difficulties in applying the concepts and theories learned in

IET Softw., 2015, Vol. 9, Iss. 4, pp. 95–102 & The Institution of Engineering and Technology 2015

Generic definition

Applying Bloom’s affective domain in the delivery of future modules

Receive: Willing to hear and learn with respect

† be willing to hear and learn with respect anything that can improve delivery † try to remember the names of a few students † ask students if they are happy with teaching methods † hold a class discussion on a different topic every week so that students can actively participate

Respond: React and participate actively

† encourage students to present new ideas to address different problems

Value: Attach values and express personal opinions

† create an atmosphere in the class in which students can express their personal opinions † explain the role of a systematic problem solving approach in the class

Organisation: Organise internal conflicts

† organise equal time for learning theories and application of theories † create more group work in the laboratory in order to improve understanding of teamwork

Internalising values: Behave consistently with personal value set

† create tasks in such a way that students can improve their judgement and change their behaviour towards problem solving, based on new evidence

99

responsibility of the demonstrators to mark these assignments and submit the marks to the course lecturer (me). I was already aware of problems of inconsistency in marking. These problems resulted in an incident: ‘Incident global software engineering assignment 1 marking: I asked the demonstrators to initially mark 2 to 3 assignments and submit these marks to me. I noted many inconsistencies in marking, that is, a few demonstrators were strict in their marking while others were relaxed in their marking. For a diverse class, it was a real challenge to reduce this inconsistency in marking.’ I also wanted to use the students’ assessments to explicitly guide them in their study so that they did not repeat the same mistakes in future assessments. To address this problem of inconsistency in marking, I implemented the following guidelines: † I designed a marking scheme for each question. † I asked all the demonstrators to initially mark 2 to 3 assignments. Then, in a formal meeting, each demonstrator’s marks were discussed in order to remove inconsistencies with other demonstrators. † I tried to make sure that everybody followed the same marking scheme and that there were no inconsistencies in marking. † Each demonstrator was required to provide feedback to the students on what was missing in their solution.

knowledge, are usually easier to answer and these types of questions can dominate the assessment if lecturers are not careful. This was true in my case, as this was one of the first exams I had designed. Furthermore, the validity of an assessment may be questioned if it only assesses the cognitive skills of knowledge. I decided to change the exam questions for the following year to enable students to apply their theories in a practical way. From this experience, I learned that there should be a balance between the academic skills being assessed. I observed that the validity of an assessment method shows that appropriate abilities should be assessed and these assessments should not only involve the knowledge and comprehension levels of Bloom’s taxonomy, but other higher levels as well. 5.1 Reflection on assessment and feedback of the global software engineering course, based on Bloom’s taxonomy I also learned about Bloom’s psychomotor and affective domains. During the student assessment, I noted that some categories of these domains might be useful for the further refinement of future assessments. The guidelines from Bloom’s affective and psychomotor domains, provided in Table 2, will be adopted to effectively assess different modules.

6 I cannot provide any evidence to show whether this practice greatly reduced the inconsistency problem, however, I noted that many incidents of inconsistency were identified and resolved. A similar pattern was followed for the marking of the final exam. I tried to base all assessments on Bloom’s cognitive domain (i.e. to test the students learning from the knowledge level to the evaluation level). For different assignments, the students had to apply the theories learnt in the lectures to generate a practical solution in the form of a model/diagram or a design for some artefacts. However, the global software engineering course exam questions related only to the knowledge and comprehension levels of Bloom’s taxonomy: ‘Incident global software engineering exam: After the exam, I had an informal chat with a few of the students about the exam. The students informed me that the exam questions were not really challenging.’ When I mapped my exam questions with Bloom’s cognitive domain, I also noted this problem. For example, in the exam I asked: † Differentiate between system owners and system users’ roles. † What important question is addressed during the problem analysis phase? These questions were easy to answer and were only based on the knowledge and comprehension levels of Bloom’s taxonomy. Objective questions, which assess the cognitive ability of Table 2 Applying Bloom’s affective and psychomotor domains in the assessment of future modules Generic definition

Receive: Willing to hear and learn with respect

Applying Bloom’s affective and psychomotor domains in the assessment of future modules † be willing to hear and learn with respect anything that can improve assessments

Precision: Refining and improving

† refine assessments using students’ and staff members’ feedback

Organisation: Organise internal conflicts

† In diverse classes, develop new and comprehensive marking schemes in order to address inconsistencies in marking

Evaluation of teaching

I used the following three methods for the evaluation of my teaching. 6.1

Evaluation questionnaires

The students completed the teaching evaluation questionnaire at the end of the course. This gave me a general overview of the design and the delivery of the course. The students’ comments helped me to adapt the module contents and the course delivery to accommodate the different suggestions. On the whole, I felt the course was successful, as evidenced by the results of the teaching evaluation questionnaires, shown in Appendix. The overall average of the students’ responses was 8.73 out of 10. The other evidence to indicate the success of the course was the high number of students who passed the course. I believe the main reason for its success was the appropriate use of Bloom taxonomy. An issue that I believe the students had with the course was because of the theoretical nature of the global software engineering topic and the lectures, in particular, which focused on the theory. The students seemed to see the lectures as too theoretical and the workshops/laboratories as the place where they did most of their learning and understanding. One recommendation for the course would be to change the focus of the lectures, although some theoretical aspects will still need to be covered to ensure the students have a basic understanding of the topics. However, examples of the implementation of the particular topic would become the major part of the lectures. An idea would be to introduce one main illustration with which all students are familiar, to be used throughout the semester – for example, the university’s admission system. Then, other ‘outside world’ examples could also be added, both in the lectures and tutorials, so as to show their relevance in industry. 6.2

Conversation with students

I used direct conversation with the students as a second mode of my teaching evaluation. I informally asked students (first in week 3 and then in week 6) what they felt about the module delivery, content, the difficulty level and the workload. This helped me to become aware of any problems the students faced and, based on this information, I made minor changes to reflect the students’ comments. For example, a few of the global software engineering students told me that I used too many animations in the PowerPoint slides while other students indicated they were having issues with the dim

IET Softw., 2015, Vol. 9, Iss. 4, pp. 95–102

100

& The Institution of Engineering and Technology 2015

lighting in the lectures. A few of the students also asked me for a catch-up practical. Conversations with the students helped me to identify these types of problems in the middle of the semester and allowed me to address them. This technique helped me to resolve any issues that needed urgent attention. I observed that an informal assessment technique (i.e. conversation with students) can easily be incorporated at any time during the semester. The informal assessment technique should not be considered a single event, but rather an ongoing series of assessment opportunities [34]. 6.3

Peer observation

The third method I used for my teaching evaluation was peer observation. Two of my lectures were observed by two different colleagues. I received feedback from my colleagues on my presentation, slide quality, module content and other general areas that could be improved.

7

Discussion

Although Bloom’s taxonomy has been criticised for its simplicity [35], it provided a useful framework for my teaching as it helped me to transform theories into practice and application. Overall, the six levels of Bloom’s cognitive domain were helpful in my teaching, in which I transformed students learning from the knowledge level to the evaluation level. However, the knowledge level of Bloom’s cognitive domain is not appropriate for higher calibre students because the content used in the knowledge level is often referred to as simple. The comprehension level seemed appropriate for all levels of students as I noted in the laboratories that students were able to produce different models and diagrams based on their understanding of the problem domain. Many international students were having problems with the application level as they were unable to apply the concepts and theories in the laboratories. In the analysis level, some students faced difficulties in breaking down the bigger problem into smaller components in order to better understand it. Students had mixed responses in relation to the synthesis level of Bloom’s cognitive domain. Students had some minor problems in the evaluation level of Bloom’s cognitive domain. However, if we use Bloom’s taxonomy and other available teaching guidelines, we can improve the overall design and delivery of these modules. After delivering the global software engineering course, I has a better understanding of the role of Bloom’s taxonomy in the design and delivery of any course. I observed that Bloom’s cognitive domain provides opportunities to teach students in different stages, that is, from knowledge to evaluation. In addition, Bloom’s cognitive domain helped me to transform global software engineering theories into practice and application. However, in addition to Bloom’s cognitive domain, the following guidelines should be considered in order to further improve the design and delivery of different courses [32]: † Positively respond to diversity and ensure equality of opportunities in terms of access, treatment and outcomes. † Take into account individual needs related to disability, race and gender. † Make deadlines flexible so they fit different students’ constraints and circumstance. † Create specialist support for disabled students in the laboratories. † Attend staff development training on awareness of disability practices. † Engage with students at their level of understanding. † Design tasks to allow as much autonomy and activity as possible. I used three methods for my teaching evaluation, that is, evaluation questionnaire, peer observation and conversation with students. The school designed the first method (evaluation questionnaire) as it is a general requirement to use this type of assessment. I was not given any opportunity to alter the evaluation questionnaire. This type of evaluation method is a summative type of assessment where a

IET Softw., 2015, Vol. 9, Iss. 4, pp. 95–102 & The Institution of Engineering and Technology 2015

judgement of teaching effectiveness usually occurs at the end of the teaching process. I think, based on the students’ feedback, that summative assessments are useful to identify where changes are needed to the course structure and/or teaching content for future delivery. However, in addition to summative assessments, I think formative assessments should also be used early in the teaching process in order to address any issues that need urgent attention, that is, changes to assessment tasks and changes to students’ work load. Therefore, in addition to the evaluation questionnaire (summative assessment), one should also use peer observation and conversations with students (formative assessments) in order to find out which aspects of teaching needs urgent attention. The students’ assessments showed that after completing this course, students were able to: † Understand the difference between in-house development and GSD. † Know the impact of software quality and different standards and models for software quality on GSD. † Apply software project management techniques in GSD. † Measure project progress, productivity and other aspects of GSD. However, one key competency required in the global software engineering course (i.e. managing virtual teams and cross-cultural collaboration) was hard to implement in the classroom environment. This was because I could not provide a distributed environment for students [13, 14] because of limited resources. To address this issue, different scenarios were created to develop this competency which is required in the global software engineering course. Finally, as an academic, it is important to conduct research in order to provide students with up-to-date knowledge in their respective fields. I think, in general, for students who pursue their undergraduate degrees and in particular, for students who pursue their postgraduate degrees, it is extremely important that their lecturers undertake research in their respective fields. This is because the advances in software engineering have not been matched by equal advances in the development of new modules in academia which has resulted in a gap between the software industry and academia. Up-to-date research in this area will help to fill this gap.

8

Conclusion and limitations

I has discussed the design, delivery, assessment and evaluation of a global software engineering course. I believe courses should be designed to provide students with sufficient knowledge and skills that are required in the current software industry so that they become real software engineers rather than simply graduate programmers. Thus, I am interested in the quality of teaching and designing new industry-based courses at undergraduate and postgraduate levels. Bloom’s taxonomy helped in the design, delivery and assessment of five competencies required in the global software engineering course. It was fairly easy to teach four competencies using Bloom’s cognitive domain. However, one key competency (i.e. managing virtual teams and cross-cultural collaboration) was hard to teach. This is not because of Bloom’s cognitive domain, but because one needs a real-world environment to teach such a competency. In addition, ‘the most effective setting for training in global software engineering is to provide a distributed environment for students’ [13] which could not be done because of limited resources. Moreover, a combination of different methods was used for data collection. However, because of space restrictions in this paper no specific data are presented. After delivering the global software engineering subject, I have a better understanding of the role of Bloom’s taxonomy in the design, delivery, assessment and evaluation of different modules. I have observed that Bloom’s cognitive domain provides opportunities to teach students in different stages, that is, from knowledge to evaluation. In addition, Bloom’s cognitive domain helped in transforming software engineering theories into practice and application. Hence, Bloom’s cognitive domain is appropriate for teaching any software engineering course.

101

I observed that summative assessments are useful to identify where changes are needed to the course structure and/or teaching content for future delivery. However, in addition to summative assessments, formative assessments should also be used in order to address any issues that need urgent attention. In the future, this course can be better taught if two or more global universities collaborate in the design and delivery of this course. This will provide an opportunity to teach in an environment required for the global software engineering course.

Islam, S., Alam, M.M.J., Hilde, S.H.: ‘Goal and risk factors in offshore outsourced software development from vendor’s viewpoint’. Fourth IEEE Int. Conf. on Global Software Engineering, 2009 Khan, S.U., Niazi, M., Rashid, A.: ‘Barriers in the selection of offshore software development outsourcing vendors: an exploratory study using a systematic literature review’, J. Inf. Softw. Technol., 2011, 53, (7), pp. 693–706 Damian, D., Izquierdo, L., Singer, J., Kwan, I.: ‘Awareness in the wild: why communication breakdowns occur’. Int. Conf. on Global Software Engineering, 2007 Nguyen, P., Ali-baber, M., Verner, J.: ‘Trust in software outsourcing relationships: an analysis of Vietnamese practitioners’ views’. EASE, UK, 2006 Cataldo, M., Bass, M., Herbsleb, J.D., Bass, L.: ‘On coordination mechanisms in global software development’. Int. Conf. on Global Software Engineering, 2007 Fortaleza, L.L., Conte, T., Marczak, S., Prikladnicki, R.: ‘Towards a GSE international teaching network: mapping global software engineering courses’. Collaborative Teaching of Globally Distributed Software Development Workshop (CTGDSD), IEEE, 2002, pp. 1–5 Saldaña-Ramos, J., Sanz-Esteban, A., García, J., Amescua, A.: ‘Skills and abilities for working in a global software development team: a competence model’, J. Softw. Evol. Process, 2013, 26, (3), pp. 329–338 Niazi, M.: ‘Teaching global software engineering: planning and preparation using a Bloom’s taxonomy’. World Congress on Engineering, The 2013 Int. Conf. of Information Engineering, London, 2013 Bloom, B.S., Englehart, M.B, Furst, E.J., Hill, W.H., Krathwohl, D.R. ‘Taxonomy of Educational Objectives, the classification of educational goals’, Handbook I: Cognitive Domain. (David McKay, New York, 1956) Anderson, L.W., Krathwohl, D.R.: ‘A taxonomy for learning, teaching, and assessing: a revision of Bloom’s taxonomy of educational objectives’ (Longman, New York, 2001) Yin, R.K.: ‘Case study research: design and methods’ (Sage Publications, London, 2009) Lucas, A.F.: ‘Using psychological models to understand student motivation’, in Svinicki, M.D. (Ed.): ‘The changing face of college teaching. new directions for teaching and learning, no. 42’ (Jossey-Bass, San Francisco, 1990) Heather, F., Steve, K., Stephanie, M.: ‘A handbook for teaching and learning in higher education’ (Kogan Page, London, 1999) Khan, S.U., Niazi, M., Rashid, A.: ‘Critical barriers for offshore software development outsourcing vendors: a systematic literature review’. 16th IEEE Asia-Pacific Software Engineering Conf., APSEC09, Penang, Malaysia, 2009 Khan, S.U., Niazi, M., Ahmad, R.: ‘Factors influencing clients in the selection of offshore software outsourcing vendors: an exploratory study using a systematic literature review’, J. Syst. Softw., 2011, 84, (4), pp. 686–699 Binder, J.: ‘Global project management – communication, collaboration and management across borders’ (Gower, Hampshire, 2010) Paul, R.: ‘Learning to teach in higher education’ (Routledge Falmer, Oxon, 2003), vol. 2 Stephen, B.: ‘A simple audit of inclusiveness’, 2006 O’Brien, M.: ‘The large class teaching guidelines, teaching and educational development institute’ (University of Queensland, Brisbane 4072 Australia, 2002) Beth-Critchley, C., Beth, C.: ‘Informal assessment strategies’ (Pembroke Publishers Limited, Ontario, 2005) Pritchett, N.: ‘Effective question design’, in Brown, S., Race, P., Bull, J.: ‘Computer-assisted assessment in higher education’ (Kogan Page, London, 1999)

15

16

17 18 19 20

9

Acknowledgment

I am thankful to the Deanship of Academic Research, King Fahd University of Petroleum and Minerals, Saudi Arabia for supporting this research project via a project number IN111030.

21

10

23

22

References

1 Toohey, S.: ‘Designing courses for higher education’ (Society for Research into Higher Education/Open University Press, Buckingham, 1999) 2 Shah, A., Pell, K., Brooke, P.: ‘Beyond first destinations; graduate employability survey’, Act. Learn. High. Educ., 2004, 5, (1), pp. 9–26 3 Voogt, J., Westbroek, H., Handelzalts, A., et al.: ‘Teacher learning in collaborative curriculum design’, Teach. Teach. Educ., 2011, 27, (2011), pp. 1235–1244 4 Clarke, D., Hollingsworth, H.: ‘Elaborating a model of teacher professional growth’, Teach. Teach. Educ., 2002, 18, pp. 947–967 5 Borko, H.: ‘Professional development and teacher learning: mapping the terrain’, Educ. Res., 2004, 33, pp. 3–15 6 Lange, D.J., Burroughs-Lange, G.S.: ‘Professional uncertainty and professional growth: a case study of experienced teachers’, Teach. Teach. Educ., 1994, 10, (6), pp. 617–631 7 Zeple, N., Leach, L.: ‘Improving student engagement: ten proposals for action’, Act. Learn. High. Educ., 2010, 11, (2), pp. 167–177 8 Jessop, T., McNab, N.: ‘Mind the gap: an analysis of how quality assurance processes influence programme assessment patterns’, Act. Learn. High. Educ., 2012, 13, (2), pp. 143–154 9 D’Artrey, M., Row, D.-A.: ‘Peer observation of teaching: a decoupled process’, Act. Learn. High. Educ., 2011, 12, (3), pp. 189–201 10 Finn, J.D., Pannozzo, G.M., Achilles, C.M.: ‘The ‘why’s’ of class size: student behavior in small classes’, Rev. Educ. Res., 2003, 73, (3), p. 321 11 Kassens-Noor, E.: ‘Twitter as a teaching practice to enhance active and informal learning in higher education: the case of sustainable tweets’, Act. Learn. High. Educ., 2012, 13, (1), pp. 9–21 12 Sølvberg, A.M., Rismark, M.: ‘Learning spaces in mobile learning environments’, Act. Learn. High. Educ., 2012, 13, (1), pp. 23–33 13 Crnkovic,´ I., Bosnic,´ I., Žagar, M.: ‘Ten tips to succeed in global software engineering education’. Proc. of the 2012 Int. Conf. on Software Engineering, , 2012, pp. 1225–1234 14 Paasivaara, M., Lassenius, C., Damian, D., Räty, P., Schröter, A.: ‘Teaching students global software engineering skills using distributed scrum’. Proc. 2013 Int. Conf. on Software Engineering, 2013, pp. 1128–1137

24

25 26

27 28

29

30 31 32 33 34 35

11

Appendix

See Table 3.

Table 3 Teaching evaluation questionnaire Question description

A

B

C

D

E

Mean

Weight

comes to class on scheduled time ends lecture on scheduled time comes prepared for the class states objectives of each class session speaks clearly assigns homework regularly explains concepts clearly with examples encourages student participation in class explains the subject matter clearly uses possible educational aids for demonstrations uses computer to communicate with students is concerned about the student’s understanding speaks with an accent that is hard to understand responds to questions in class motivates students in all works related to the course is available during scheduled office hours presents class material clearly gives tests that focus on understanding of the subject does not assign homework at all grades and returns tests without delay overall rating of the instructor overall average

8 8 5 7 7 2 5 7 6 6 6 6 0 7 4 7 5 4 0 7 6

3 3 6 4 4 4 5 4 5 5 4 5 0 4 6 3 6 5 2 4 4

0 0 0 0 0 2 1 0 0 0 1 0 3 0 1 0 0 1 4 0 1

0 0 0 0 0 0 0 0 0 0 0 0 8 0 0 0 0 1 3 0 0

0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 1 0 0 2 0 0

3.73 3.73 3.45 3.64 3.64 3 3.36 3.64 3.55 3.55 3.45 3.55 1.27 3.64 3.27 3.7 3.45 3.09 1.89 3.64 3.45

8 7 10 6 8 6 8 5 10 8 6 8 0 10 8 8 10 10 0 4 10

Weighted mean 29.82 26.09 34.55 21.82 29.09 18 26.91 18.18 35.45 28.36 20.73 28.36 0 36.36 26.18 29.6 34.55 30.91 0 14.55 34.55 8.73/10

Standard deviation

Variance

0.47 0.47 0.52 0.5 0.5 0.76 0.67 0.5 0.52 0.52 0.69 0.52 0.47 0.5 0.65 0.48 0.52 0.94 0.78 0.5 0.69

0.22 0.22 0.27 0.25 0.25 0.57 0.45 0.25 0.27 0.27 0.47 0.27 0.22 0.25 0.42 0.23 0.27 0.89 0.61 0.25 0.47

A: strongly agree, B: agree, C: neutral, D: disagree and E: strongly disagree.

IET Softw., 2015, Vol. 9, Iss. 4, pp. 95–102

102

& The Institution of Engineering and Technology 2015