Educ Inf Technol DOI 10.1007/s10639-015-9393-5
Performance and perception in the flipped classroom Erik Blair 1 & Chris Maharaj 2 & Simone Primus 3
# Springer Science+Business Media New York 2015
Abstract Changes in the conceptualisation of higher education have led to instructional methods that embrace technology as a teaching medium. These changes have led to the flipped classroom phenomenon - where content is delivered outside class, through media such as video and podcast, and engagement with the content, through problem-solving and/or group work, occurs in class. Studies investigating the impact of the flipped classroom have mainly looked at the student experience with little focus on whether exam outcomes are enhanced by flipping. An undergraduate Material Technology course at The University of the West Indies was taught in two formats over two successive years. The course was taught during the 2012/13 academic year in a ‘traditional’ format but, after reflecting on student feedback and personal pedagogy, the lecturer restructured the class and taught it in a flipped format during the 2013/14 academic year. This research examines whether the flipped format improved the learning experience in relation to exam performance and student perception. Data was gathered through analysis of course grades and student evaluation questionnaires. The lecturer’s reflective comments were also reviewed before and after the study. Analysis of the qualitative data shows that the flipped format led to a slight improvement in how students perceived the course and the lecturer’s reflection shows that they are keen to continue with the flipped format as it allowed more time for them to work with students at an individual level. While no significant change in relation to average cohort exam performance was found, fewer students in the flipped classroom achieved marks at the highest level. It is therefore recommended that practitioners who intend to flip their classroom pay as much attention to student performance as they do to student perception.
* Erik Blair
[email protected] 1
The Centre for Excellence in Teaching and Learning, The University of the West Indies, St Augustine, Trinidad and Tobago
2
Department of Mechanical & Manufacturing Engineering, The University of the West Indies, St Augustine, Trinidad and Tobago
3
The School of Education, University of the West Indies, St Augustine, Trinidad and Tobago
Educ Inf Technol
Keywords Screencasts . Higher education . Flipped classroom . Student perception
1 Introduction Instructional design in higher education varies depending on context (Lea and Callaghan 2008), societal norms, cultural beliefs, local histories and the available systems (Cervero and Wilson 2001). Recent changes in the format and conception of higher education have led to a more student-centred approach that attempts to embrace technology as a teaching medium. There is now recognition that how educational materials are perceived and interacted with, as well as the instructional methods employed by educators, can affect the quality of the learning experience in both face-to-face and online modalities (Bransford et al. 2000; Ward et al. 2013). Therefore educators are tasked to consider how they construct learning environments, and how they perceive students’ approaches to learning, in their pedagogical decision-making. Information Communication Technology (ICT) has brought a raft of new tools into the higher education arena. As well as the ubiquity of PowerPoint, lecturers have access to ICT in the form of, inter alia, social networking sites, virtual learning environments and massive open online courses (MOOCs). Courses are also taught in various formats: face-to-face, online, distance and blended - leading to a situation where teaching staff are expected Bnot only to master the new technologies, but also to understand and capitalise on the pedagogical implications^ (Rennie and Morrison 2013, p.16). In examining such pedagogical implications we find that, through the application of ICT, the focus of learning has moved from the ‘traditional’ teacher-centred to a student-centred paradigm. In moving beyond conceptualising the learning experience as the transmission of knowledge, the student-centred conceptualisation of the student experience Bplaces responsibility on educators to create learning environments oriented towards deep approaches to learning^ (Case and Marton 2009, p.18). This study reports on an attempt to transform a ‘traditional’ learning environment into one that embraces technology to develop a learner-orientated format. Here the introduction of a flipped classroom format into an engineering course is examined in order to establish if the flipped format improved the teaching and learning experience in relation to exam performance and student perception.
2 The flipped classroom Recent efforts to improve student engagement in higher education classrooms have resulted in the flipped classroom phenomenon. Ojalvo and Doyne (2011) describe the flipped classroom as an Binverted teaching structure in which instructional content is delivered outside class, and engagement with the content – skill development and practice, projects and the like – is done in class, under teacher guidance and in collaboration with peers^ (para. 1). Video lectures viewed at home along with problem-solving group work completed in-class are hallmarks of the phenomenon (Bishop and Verleger 2013; Du et al. 2014). The theoretical strength of this new
Educ Inf Technol
pedagogy seems to be the combination of constructivist and behaviourist learning theories working in tandem (Bishop and Verleger 2013). Research studies investigating the flipped classroom strategy suggest several benchmarks for best practice and successful implementation. Milman (2012) notes the importance of Bformative and summative assessments, as well as, meaningful face-to-face learning activities^ (p. 85) and Enfield (2013) points to the collaborative nature of the flipped classroom providing students with various skills and abilities to learn from and support each other. Miller (2012) advocates the importance of motivating students to want to view video lectures by using strategies such as project-based and game-based learning while Enfield (2013) found frequent quizzes a motivation. Advocates of the flipped classroom argue several advantages to both students and faculty who participated in the strategy. Enfield (2013) found that while video preparation was time consuming, there was a decrease in individual class preparation time, as well as, time spent in correcting student misconceptions; additionally, absent students could be directed to the videos thereby reducing time spent on repeat instruction. Video lectures give students a measure of control over their learning (Du et al. 2014) and students appreciate the ability to view videos at their convenience and pace (Enfield 2013). Equally important is the need to include post-viewing reflective activities so students make connections between learned content (Miller 2012). Finally, some thought should be given to students’ access to technology to view videos, which should be in manageable chunks (Miller 2012). Challenges to implementing the flipped classroom have also been identified. Enfield (2013) notes the significant amount of time needed to prepare videos and the need to ensure they meet standards required for students with disabilities. Additionally, planning for effective in-class activities is vital but may prove challenging for faculty comfortable with a traditional style of lecture. The need for access to technology to view videos may be a challenge for economically disadvantaged students (Du et al. 2014; Enfield 2013; Milman 2012). Further, video quality may be poor, and students may not understand the video content and may be unprepared for learning activities (Milman 2012). Studies investigating the impact of the flipped classroom phenomenon have looked primarily at students’ perceptions. Mason et al. (2013a) found that students in flipped classrooms like the use of video and feel the inversion is a better use of class time. Others found that students prefer the active approach to learning that the flipped format often brings and believe that having more in-class activity helps with learning content (Enfield 2013; Mason et al. 2013b; Phillips and Trainor 2014). In a meta-review of 24 studies on the flipped classroom, Bishop and Verleger (2013) found that generally student perceptions Btended to be positive, but there were invariably a few students who strongly disliked the change^ (p. 10). They also noted that studies tended to focus on perceptions of flipping rather than outcomes - with only two studies reporting on performance. Bishop and Verleger noted that the two studies that did report on student performance were atypical in that one (Day and Foley 2006) used a participant group drawn from students on a computer interaction course where high levels of at-home online activity would be expected, and the other (Moravec et al. 2010) used a semiflipped format which included in-class lectures. It is clear from this, that while there is a generally positive perception of the flipped classroom, there is a gap in the research on
Educ Inf Technol
performance in the flipped classroom and that there is Bnot strong enough evidence to support the claim that student learning was enhanced by this format^ (Chin 2014, p.1).
3 The research context This paper examines the implementation of a flipped classroom in an undergraduate Material Technology class (MENG 3015) in the Department of Mechanical and Manufacturing Engineering in the Faculty of Engineering at The University of the West Indies, St. Augustine, Trinidad and Tobago. Students in the course were majoring in either Industrial Engineering or Mechanical and Manufacturing Engineering and this course underpinned key components of both. The course was taught during the 2012/13 academic year in a traditional format but, after reflecting on student feedback and personal pedagogy, the lecturer looked for specific means to improve the learning experience. Reflective practice is Bthe process of learning through and from experience towards gaining new insights of self and/or practice^ (Finlay 2008, p.1). Kane et al. (2004) established that reflective practice is a common characteristic of excellent university teachers and that purposeful reflection leads to developing excellence. The excerpt below shows that lecturer’s rationale for introducing changes into the course. From this excerpt we can see that the initial intention was not to flip the classroom but to introduce a more effective method for teaching course content, so that more time could be spent on practical engagement with the topics learnt. This reflection offers a rationale for this research project and sets an informal baseline for analysing later outcomes: Over the last 2 years, I have been on a quest to make my teaching methods more student-centred. This commitment arose from reflection on my previous teaching methods (traditional lecture with open class discussions and tutorials, for example). In my attempt to make the student experience more enjoyable and beneficial, I developed the screencasts to minimize the time spent on class lectures in order to get more participation from the students through in-class group activities. I also considered the students who missed classes due to unforeseen circumstances and did not want them to be severely disadvantaged. These screencasts are digital recordings of computer screen output with an audio narration. The screencasts essentially capture the lectures that I would have traditionally delivered in the classroom and now optimize the material, making it freely available to the students for viewing outside the classroom, at any time. Essentially, the screencasts facilitate a classroom whereby students cover the course content in their own time (at home, in-transit, at the university etc.). This allows the time in class to focus on difficult concepts and student queries. The majority of the class time can then be spent on group activities. I subsequently found that the screencasts facilitated a Bflipped classroom^ where the majority of the class time could be spent on group activities. In this flipped classroom, I aimed to provide direct and purposeful learning experiences for the students. Research in pedagogy, such as Dale’s Cone of Experience, has shown that learners retain more information by doing as opposed to more passively
Educ Inf Technol
hearing, reading, or observing. Specifically, the students learn more effectively when they work on the in-class activities, having covered the content beforehand. They also gain appreciation of how to answer a question, ensuring that the response is cogent. Finally, by engaging in cooperative learning, the in-class group activities prepare students for the real world of work where teamwork is essential. This project started with the course lecturer reflecting on the experience of teaching this course and from a number of comments raised in student feedback questionnaires. Through pre-teaching course content using screencasts and then using class time to examine these concepts, a flipped classroom format was developed for the 2013/14 MENG 3015 cohort. This research examines whether the introduction of this flipped format improved the learning experience in relation to exam performance and student perception. Two classroom formats are examined here: the traditional and the flipped classroom. The traditional class format was used in the 2012/13 academic year and the flipped format was used in the 2013/14 academic year. During 2012/13 there were 71 students registered on MENG 3015 and during 2013/14 there were 42 registered students. The course is a third year undergraduate engineering course. The traditional classroom approach consisted of two classes per week, with one being 2 h and the other 1 h in duration. The approach taken was to firstly present information in the form of a lecture using PowerPoint slides then use half the class time to through the slides, followed by questions and group work. No assignments were given to the students to perform outside of class with the exception of the formally assessed coursework project. The flipped classroom approach consisted of one 2 h class each week. In advance of this class time, screencasts were uploaded onto the University’s virtual learning environment for students to review in their own time. These screencasts covered the course content. The first 15 min of the class involved skimming through the screencast slides while addressing student queries on their content and any concepts they had difficulty with. The remainder of the class time was spent on group work activities. As in the traditional classroom setting, no assignments were given to the students to perform outside of class with the exception of the formally assessed coursework project.
4 Methodology Three ‘types’ of data were gathered: course performance data, student perception data and the lecturer’s reflections. The performance data shows the results of the MENG 3015 course in relation to the percentage of students who passed the courses in both the traditional and flipped format. This data is also examined in relation to all other courses taught at the University and the average pass marks achieved in the other courses the students were studying for. This data was analysed in relation to other University courses as the two cohorts do not allow for like-for-like analysis. By analysing both sets of MENG 3015 data in relation to other University courses, a comparison of the traditional and flipped format in relation to each year’s ‘norms’ can be achieved. This means that the cohorts are not only examined in relation to each other but in
Educ Inf Technol
relation to their standing within a particular year. Performance is also examined through an analysis of the spread of marks achieved by each cohort in their final examination. Finally, the performance data is shown in relation to the attendance data in an effort to establish if there is a link between attending class (in traditional and flipped formats) and exam success. Attendance was measured as it is an important aspect of both formats. Students attend the traditional classroom to gain content knowledge and students attend the flipped classroom to manipulate the content they learned outside of the classroom. In both cases, the classroom offers a valuable learning opportunity. The impact of attendance was used as a rather crude measure as the differences between the two cohorts would not allow for nuanced analysis of this factor. However, it was important to examine the performance data against the attendance data in case any major differences emerged between the two that might offer insight into future versions of either format. The perception data is drawn from the student course evaluation results. Each course taught at the University is evaluated each term—this evaluation involves a quantitative evaluation of the course and lecturer using 36 Likert-type questions followed by openended questions that allow for qualitative feedback. Students rate statements about the course and lecturer 1–5: where 1 is ‘Strongly disagree’ and 5 is ‘Strongly agree’. They also rate the lecturer’s overall performance 1–5: where 1 is ‘Poor’ and 5 is ‘Excellent’. Student evaluation questionnaires (SEQs) are seen as a means to improve future practice by reviewing what has come before (Arreola 1995). Students place value on SEQs (Arreola 1995) and they offer an insider understanding of the student experience (Greenwald 2002; Theall and Franklin 2001). The completion of the SEQ was voluntary. In 2012/13 49 of the 71 students completed the SEQ and in 2013/14 15 of the 42 students completed it. Since the two cohorts were made up of a different number of students, t-tests were applied to all the data. All results were done with a 95 % confidence level. The SEQ closes with three open-ended questions that allowed students to offer qualitative responses: 1. What did you like most about the class? 2. What did you like least about the course? 3. How could the class be improved? Greenwald argues that SEQs highlight the perspectives of students with regard to the effectiveness of their lecturers and that assessing lecturers’ practice through SEQs allows for a valid gauge of quality and effectiveness. Further, SEQs offer valuable insight into the learner’s perspective on course development (Campbell et al. 2007). Finally, SEQs offer statistical and methodological validity (Hellman 1998); are reliable (Piccinin et al. 1999); and are valid instruments for measuring the relationship between learning and student satisfaction (Clayson 2009). Both sets of data (quantitative and qualitative) were examined in order to establish if there were any areas that students perceived had changed as a result of flipping the classroom. The lecturer data here takes the form of two verbatim accounts. At the start of the project, the lecturer (who is also one of the research team) was asked by the other two researchers why he had decided to change the format of the course. His account is shown above and sets the tone for the project. Then, once the performance and perception data had been scrutinised, the lecturer gave a final reflective account
Educ Inf Technol
regarding his perception of the project as a whole. The two transcriptions are shown without analysis as they speak for themselves; offer a valuable insight, and remind us that such interventions are not research for the sake of research but educational interventions that aim to improve the teaching and learning experience. It is recognised that such analysis is not like-for-like. In analysing the impact of MENG 3015 in relation to a change in teaching format it was not possible to involve the same cohort of students and like-for-like analysis would not be possible without constructing artificial models. Synthetic cohorts are used here as it was not possible to control certain factors and any differences in the make-up of the two cohorts has to be accepted as a function of the actual situation. Since these were the actual students on the actual course, their perspectives are valuable in themselves and are presented here as valid commentary. Both cohorts would have followed similar paths of study and would share many ‘averaging’ factors such as culture and educational experiences but they will always be different. Further, although the course was taught by the same lecturer in both iterations, maturation, experience and reflection are likely to have impacted on their pedagogy. It is proposed that the triangulation of the three types of data, alongside key messages from the literature, increases the reliability of such data.
5 Data and data analysis 5.1 Performance data One of the key measures as to the success or otherwise of the flipped classroom is the exam results. If there were significant changes in the course results from previous years or if the standing of the course results in relation to average University results changed significantly then there might be evidence of impact. Likewise in examining the results students achieved in the course in relation to the results they achieved in their other courses and in relation to their attendance in class it might be possible to identify evidence of impact. Table 1 shows the relationship of results of the 2012/13 (traditional) cohort against those of the 2013/14 (flipped) cohort: We can see from Table 1 that the pass rate for the flipped classroom version of MENG 3015 decreased by 1 % from the baseline set in 2012/13. In order to more fully understand the cohorts’ pass rates they need to be matched against the University average for each year. While both iterations of MENG 3015 clearly outperform the average pass rate across the campus, the average pass rate improved by 2 % in 2013/14 - this means that the adjusted pass rate for the flipped classroom is actually marginally, but not significantly, lower than 95 %. From this we can see that there was a slight dip in overall performance in the flipped classroom version in relation to students passing Table 1 Gauging the success of MENG 3015 against campus average pass rate 2012/13
2013/14
MENG 3015 pass rate
96 %
95 %
Average pass rate across all courses
84 %
86 %
Educ Inf Technol
the course. However the actual course pass rate remains high in relation to other courses and Table 1 shows that the course produced a high pass rate in both formats and that it outperformed other courses. Using a confidence level of 95 %, there is no evidence here of any significant impact of the flipped format on student performance. Not only did the pass rate of MENG 3015 outperform the average across the entire University over the 2 years but the average mark, expressed as a percentage, was higher than the average achieved in the other engineering subjects that the students on MENG 3015 were studying for. In 2012/13 the average mark for MENG 3015 was 68 % and in 2013/14 it was 56 %. However, the average mark for the entire faculty also dropped during this time: 57 % in 2012/13 and 49 % in 2013/14. Therefore, chi-squared tests were applied to the average mark achieved by the students studying for MENG 3015 and the average students achieved in other engineering courses across the faculty (see Table 2) and show that there was no significant difference. Although there was a dip in the average percentage mark achieved in the flipped cohort, this was mirrored across the average achieved in all other courses. Since this study involves synthetic cohorts there were a number of factors at play that were beyond control and may have affected all students in the faculty. This suggests that the decrease in the flipped cohort’s average mark was a symptom of whatever caused the dip in the group average across the entire faculty rather than an effect of the flipping itself, and that the flipping led to no significant change in MENG 3015’s marks in relation to all other engineering courses that year. Through examining the spread of marks in both MENG 3015 formats (see Fig. 1) we can see that, while the average mark achieved in 2013/14 is close to that achieved in 2012/13, this is mainly due to an increase in those achieving low pass marks. The University sets 40 % as the minimum pass mark and in 2013/14 the percentage of students achieving 40–59 % in MENG 3015 is higher than that of the previous year. Conversely, the percentage of students achieving over 60 % is lower in the flipped format that it is in the traditional format. Here we can see that the flipped classroom has led to an increase in the percentage of students achieving marks between 40 and 59 % from 23.9 % in 2012/13 to 42.8 % in 2013/14. We can also see a decrease in the percentage of students achieving marks over 60 % - from 71.9 % in 2012/13 to 52.4 % in 2013/4. Both these changes are significant at p
