Relationship Between Use of Online Support Materials and Student Performance in an Introductory Finance Course
Ernest N. Biktimirov* and Kenneth J. Klassen
Department of Finance, Operations and Information Systems Goodman School of Business, Brock University 500 Glenridge Ave, St Catharines, Ontario, Canada, L2S 3A1 * corresponding author: E. Biktimirov: (905) 688-5550 ext. 3843
[email protected]
Published in the Journal of Education for Business 83 (3), January/ February, 2008, 153-158. The final publication is available at http://www.tandfonline.com/doi/abs/10.3200/JOEB.83.3.153-158#.Up1j0uJIOSo
Citation information: Biktimirov E. N. and K. J. Klassen. (2008). “Relationship between use of online support materials and student performance in an introductory finance course,” Journal of Education for Business 83 (3), January/February, 153-158. DOI: 10.3200/JOEB.83.3.153-158
Relationship between the use of Online Support Materials and Performance in the Introductory Finance Course ABSTRACT The relationship between student online activity (including access to specific course materials) and student performance is examined in a traditional face-to-face introductory finance course supported with a class web site. Six measures are used: total hits, hit consistency, number of unique files accessed, and accesses to homework solutions, PowerPoint slides, and exam solutions. Results indicate that access to homework solutions and, to lesser extent, hit consistency, are both positively related to student performance. In addition, results suggest that access to specific files, rather than access to online course materials in general, is associated with better student performance.
Technology is becoming an integral part of teaching and learning in many business courses. It is expected that this trend will continue in the future, partly due to the high percentage of junior faculty using more advanced classroom technology (Cudd, Tanner & Lipscomb, 2004). Also, today it is relatively straightforward for instructors to place course materials online due to the ease of creating one’s own website, or by using pre-established software (i.e., a course management system such as WebCT (2005) or Blackboard (2005)). The underlying assumption behind this rise in educational use of technology is that it benefits student learning. However, the empirical examination of this assumption in education literature has been limited. Moreover, while a growing body of literature examines the effectiveness of completely online courses, there is a lack of research into the more common “hybrid” courses that use online materials to supplement traditional face-to-face teaching (Baugher, Varanelli & Weisbord, 2003).
2
This paper extends the earlier research in three areas. First, this paper examines the value of online support materials in a hybrid course, rather than a completely online course. Second, this paper specifically analyzes hits to different course materials, such as solutions to homework problems, PowerPoint slides, and solutions to exams. Finally, this paper uses a more accurate measure of student total hits by measuring access to specific course materials rather than hits to the course homepage.
LITERATURE REVIEW The increasing use of technology in finance instruction has inspired a growing body of research in this area, covering various aspects. Van Ness, Van Ness & Adkins (2000) examine student performance in the introductory finance course and find that students who take the course online are more likely not to complete the course than students who take the course in the traditional classroom setting. They also find that there is no difference in student grades between the two types of settings. In contrast, Anstine & Skidmore (2005) find that the online environment produces inferior learning outcomes than the traditional environment in statistics and economics courses.
The Value of Course Web Sites The use of course management systems (e.g., WebCT (2005), Blackboard (2005)) is becoming more common as a means to supplement traditional classroom courses. This type of software permits faculty to distribute course materials and provide communication abilities among instructor and students. In a recent study, Cudd et al., (2004) report that 40 percent of finance faculty are currently using course management systems.
3
Wilson (2003) finds that total hits to the site supported by the course management system are associated with a higher student performance in the introductory finance course. In contrast, Baugher et al. (2003) do not find a significant relation between total hits and student performance in the introductory management course. These authors also offer a new measure of online course activity; hit consistency. They suggest that if a student consistently uses the website throughout the course, it has a significant positive impact on the students’ course grade. In addition, researchers have examined student use of discussion boards, which are a common element of course management systems. These boards permit students and instructors to post messages to a common area for reading and responding. Wilson (2003) examines the number of hits to a discussion board and concludes that the boards are significant contributors to student performance, although less significant than total hits to the course web site. While course management systems permit instructors to efficiently distribute different course related materials (such as homework solutions, PowerPoint slides, and exam solutions), to the best of our knowledge, no studies have examined the relationship between student access to these specific online course materials and their performance. However, as reviewed below, there have been some studies on the value of these materials in the traditional environment where the Internet is not used.
The Value of Specific Course Materials Mixed results have been obtained in the education literature regarding homework assignments, PowerPoint presentations, and after-exam teaching strategies. For example, although Rayburn & Rayburn (1999) find a positive relation between homework completion and student performance, Peters, Kethley & Bullington (2002) show that requiring graded homework
4
has a negative effect on students’ performance on exams. Lefcort and Eiger (2003) find no difference in performance between students given preparatory (before class) versus practice (after class) homework. However, student performance increased as compared to that of students who were not assigned any homework. This current study contributes to this literature by examining the relationship between making solutions to homework problems available to students and student performance. A growing body of literature regarding the effect of PowerPoint presentations on student learning also produces mixed results. For example, while Bartsch & Cobern (2003) and Lowry (1999) conclude that PowerPoint presentations can improve student performance compared to overhead transparencies, Szabo & Hastings (2000) do not find that using PowerPoint presentations helps student learning. Moreover, Rankin & Hoaas (2001) find that the use of PowerPoint presentations does not affect student performance, student attitudes towards the course, or student evaluations of the instructor. Despite these mixed results, PowerPoint slides now accompany almost all finance textbooks. With spreadsheets, they are the most widely used teaching tools of finance faculty (Cudd et al., 2004). The question arises whether instructors should make PowerPoint slides available to students before lectures. As DenBeste (2003) suggests, if students have PowerPoint slides available outside of class, they might decide not to attend the lectures. This study contributes to this literature by examining the relationship between PowerPoint slide availability and student performance in the introductory finance course. Finally, despite the in-class test being the primarily assessment tool in the introductory finance course (Farooqi & Saunders, 2004), very limited research exists on post-exam pedagogical strategies. A rare exception is Chan and Shum’s (2004) study of the cumulative/re-
5
work testing strategy, which allows students to earn partial credit for correcting their mistakes in initial tests. The authors suggest that the effect of this testing strategy may be different across disciplines or across different academic standings and students may not study as hard as they should since they know that they can rework the missed problems. This current study considers the value of making solutions to exam problems available to students.
SAMPLE AND COURSE DESCRIPTION Participants in the study were 85 undergraduate students in two sections of an introductory finance course that was offered in the Fall 2003 semester at a medium-size AACSBaccredited Canadian university. Five of these students did not have any courses at that university prior to Fall 2003, and thus did not have a GPA (grade point average) at the university. Since GPA was important to include in the model, these five were removed from the analysis, for a final sample of 80 students. The same instructor taught both sections using the same content and pedagogical methods. Introductory finance is a required course for undergraduate business students. It is usually scheduled to be taken in the students’ sophomore year, and these two sections included 1 freshman, 66 sophomores, 9 juniors, and 4 seniors. The WebCT course management system (2005) was used to support traditional classroom instruction and to collect data about student online course activity. For files placed in “Content Modules,” WebCT recorded the date and time every student accessed each file. Therefore, for this course all files were set up on the site in Content Modules. Students from both sections of the course accessed the same WebCT site. It contained the course syllabus, PowerPoint slides of class lectures, solutions for homework problems, and other
6
supporting files (e.g., solutions for current exams, solutions for sample exams, financial calculators’ settings, and math review). None of the above were handed out in class except the syllabus. The PowerPoint slides were in outline format – students could print and use them to take notes during the lectures. Based on the instructor’s observation during the term, most students did print these out, although some still attempted to take all notes in class. Solutions for homework problems and some other files (e.g., financial calculators’ settings, and math review) were posted to the site at the beginning of the term, while some other files were posted later. Specifically, the PowerPoint slides were posted several days before the material was discussed in class, and solutions to the sample and current exams were posted several days before and after the exam day, respectively. All files on the site were “optional” for the students, so the students could attend class and do all required assignments and exams without using the files offered. While using a single instructor for both sections avoided potential confounding effects of different instructors or different teaching methods, this may limit the generalizability of the study. However, this is a required course in many business programs and it is taught with a common textbook, suggesting that there may be broader applicability. This is discussed further in the “Discussion” section.
VARIABLES Course Grade, the dependent variable, is composed of the following: two quizzes – 10% each, mid-term exam – 30%, and final exam - 50%. All regression analyses in this study use the Course Grade as the dependent variable. Content Hits is the total number of times the student accessed any file in the Content Modules on the WebCT site. This is similar to the variable “Total Hits” used in prior studies
7
(e.g., Baugher et al., 2003; Wilson, 2003), except that Content Hits does not include hits to the home page of the site. Students must use the homepage to access the content pages, but there is nothing on the home page that can be accessed or downloaded. Hit Consistency measures how consistently students use the website. During the 13 week semester, we can identify 13 periods for purposes of analysis. The first period starts at the first lecture and ends one week later at the second lecture (both sections had lectures once a week for three hours), the second period starts at the second lecture and lasts a week, etc. until week 13. This results in 12 periods. The 13th period, the time between the last lecture and the final exam, is 2 weeks long. As in Baugher et al., (2003), this variable is calculated by assigning a zero if the student did not access the site during a period, and a one if they did access it. The sum of these resulted in a variable that ranges from 0 to 13. Unique Files Accessed is the number of different files accessed by a student. A student could have a high number of Content Hits, but only access very few files through the term, while another student could access every file once and download it immediately, having low number of Content Hits. There were 37 unique files on the course WebCT site. Homework Solutions measures the number of the end-of-chapter textbook problem solutions accessed on the course WebCT site. There were 12 solution sets (one for each chapter) to textbook problems on the website; these were optional for students. PowerPoint Slides measures the number of PowerPoint presentation files accessed. There were 12 files (one for each chapter) on the course WebCT site. Exam Solutions measures the number of students’ accesses to solutions for the two quizzes and the mid-term exam: three files that were posted shortly after each test date.
8
Finally, GPA is the student’s grade point average for all university courses taken prior to the fall 2003 term. Finance instructors consistently find GPA as a positive and significant variable in predicting students’ performance in finance courses (e.g., Didia & Hasnat, 1998; Wilson, 2003), and therefore it is used as a control variable in all regressions. At this university, GPA is measured as a percentage out of 100.
ANALYSIS Table 1 presents descriptive statistics for all variables in our analysis. The mean Course Grade is 62.28%, or low “C,” which is considerably less than the mean GPA of 73.68%. This difference is not surprising given that students overwhelmingly regard the introductory finance course as a difficult one, both before and after they take it (Krishnan, Bathala, Bhattacharya & Ritchey, 1999).
[Table 1 about here]
Students had on average 45.95 Content Hits during the semester. The mean and standard deviation of Hit Consistency, 7.74 and 2.84, respectively, are relatively close to the same statistics of hit consistency, 9.05 and 2.65, reported for the introductory management class by Baugher et al., (2003). The mean for Unique Files Accessed indicates students accessed on average 23 out of 37 available content files. The use of web-based course materials varied considerably among students, as evidenced by the range of Hit Consistency (0 to its’ maximum possible 13) and the range of Unique Files Accessed (range from 0 almost to its’ maximum).
9
Students accessed fewer Homework Solutions than PowerPoint Slides, with both mean and median lower for Homework Solutions. At the same time, access to both Homework solutions and PowerPoint slides ranged from 0 to the maximum of 12, again demonstrating a wide range of use of the materials by various students. The relatively low mean number of 1.53 of accesses to Exam Solutions can be explained by the fact that the solutions to each exam were discussed in class. Therefore, students who performed well on the exam or understood the solution in the class did not see the need to access the detailed solutions from the course WebCT site. Table 2 displays the correlations between the variables. GPA has the highest correlation with the Course Grade, 0.654, which is significant at the 1% level. Among five variables which measure online course activity, only Hit Consistency and Homework Solutions have significant positive correlations with Course Grade (0.282 and 0.273, respectively). While these low correlations suggest their marginal practical significance compared to the GPA, they are comparable with other, similar correlations in prior studies. For example, Baugher et al. (2003) report a correlation of 0.37 between hit consistency and course average.
[Table 2 about here]
All measures of online course activity have positive significant correlations with each other, except Exam Solutions that is not significantly correlated with Homework Solutions and PowerPoint Slides.
10
Table 3 presents the ordinary-least-squares regression results for the dependant variable, the Course Grade. The parameter estimate for each variable is followed with its’ p-value in brackets. [Table 3 about here]
Having the highest correlation with the Course Grade, a control variable, GPA, yields an R2 of .428 in Regression I, and it is the most significant predictor of course grade in the other six regressions. Examining regressions II through VII as a whole, only Homework Solutions has a significant positive relation with the Course Grade. The F-test is used to estimate incremental gain in the predictive power of regressions when compared to Regression I. Thus, only Homework Solutions provides additional predictive power over that provided by GPA alone. In contrast, additions of other measures of online activity do not significantly improve the predictive power of regressions compared to regression I. It should be noted that although Hit Consistency is not a significant factor in the regression (as it was in the correlation analysis), its’ p-value is closer to being significant than any other factor (at the10.8% level).
DISCUSSION The results of the correlation and regression analyses show that students’ access to homework solutions is associated with higher course grades. This evidence suggests making homework solutions available to students. Moreover, the homework problems do not have to be submitted for grading to provide learning benefits. This result is consistent with the findings of
11
Peters et al. (2002), who show that requiring graded homework (as opposed to optional homeworks), has a negative effect on exam performance in the introductory operations management course. While using a single instructor and a single course avoided potential confounding results due to an instructor or course factor, we do not make generalizing conclusions based just on two sections of one course. However, this study may have broader applicability in the sense that a typical introductory finance course is analyzed, and because the textbook and teaching methods used are those of the modal Canadian finance professor, as described in Farooqi & Saunders (2004). Although this study was conducted in a Canadian university, the modal Canadian finance professor is similar to the U.S. counterpart as described in Saunders (2001). Similar to Baugher et al. (2003), we do not find a significant relationship between Content Hits and the Course Grade. This result differs from finding of Wilson (2003), who shows that total hits on the course site is a significant and positive predictor of student performance. A possible factor that could contribute to this difference is that Wilson awarded points to students for discussion board hits. The results for Hit Consistency are in agreement with those of Baugher et al. (2003), although in our study the relationship is not as strong. Based on the correlation analysis, we find a positive relationship between Hit Consistency and the Course Grade. However, the regression analysis does not support this; in the presence of GPA, Hit Consistency is not a significant variable in predicting the Course Grade. Unique Files Accessed, PowerPoint Slides, and Exam Solutions accessed do not show significant relations with the Course Grade. These results suggest that access to materials other than homework solutions may not affect students’ performance.
12
It is instructive to consider why results in different studies vary. For instance, in a course where all materials are posted at the beginning of the term, it may be expected that hit consistency is less important than in a course where new material is posted every week or a discussion board is used. Similarly, if an instructor goes over all the material on the website, there is less need for students to access it. Indeed, in the current study the PowerPoint slides and exam solutions were reviewed in class, but the optional homework problems were not. This may help explain the significant results for homework solutions. Thus, although this is still a very new area of research, we may find that it is impossible to come up with general relationships that will apply broadly to all introductory finance courses. It may depend more on how the instructor uses the posted material in class. One benefit of posting material online that is not considered in any study to date is simply the convenience for the students. They can access all course material at any time, and if they lose a document they can re-download it. Indeed, based on student evaluations, students seem to appreciate this feature in a course even though it may not have a significant effect on their grade.
CONCLUSION While the number of completely web-based courses keeps increasing, the great majority of courses use a traditional face-to-face format with supporting online materials. However, the research on the learning benefits of the students’ online activity and access to specific course materials has been limited. We examine student online activity in general and students’ access to specific online course materials in a traditional face-to-face introductory finance course supported with a WebCT site. We use two measures of online course activity that have been used in the literature,
13
total hits and hit consistency, and three more specific measures that capture access to specific course materials: access to homework solutions, access to PowerPoint slides, and access to exam solutions. After controlling for students prior performance (GPA), we find that access to homework solutions, and to a lesser extent, hit consistency, are significantly related to student’s performance. Although causal relations cannot be demonstrated by this kind of study, the results suggest the benefits of: making homework solutions available to students, and encouraging students to use the site regularly throughout the term. While anecdotal evidence exists on benefits of making PowerPoint slides and exam solutions available to students, we do not find empirical support for this. As can be discerned from the mixed results found in educational literature and the limited generalizability of this study, much research remains in the area of student online activity and the value of course support materials. We suggest several avenues for future research. First, to assess the generalizability of earlier results, student online activity and the value of the course support materials in different business courses and in different types of educational institutions could be examined. Second, the study of various ways of using course support materials, trying to identify factors that contribute to their educational benefits could be continued. For example, instructors could analyze the relationship between different versions of PowerPoint slides distributed to students before each lecture (e.g., slides are not available to students, slides are available but have missing information that needs to be filled, slides have full information) and student performance. Finally, researchers could address the causality issue by examining whether student accesses to the course website and specific course materials result in higher grades. It is possible that student online activity and higher grades relate to some third common cause, for example, student effort, which causes both online activity and grades to move together.
14
REFERENCES
Anstine, J. & Skidmore, M. (2005). A small sample study of traditional and on-line courses with sample selection adjustment. Journal of Economic Education, 36(2), 107-128. Bartsch, R. A., & Cobern, K. M. (2003). Effectiveness of PowerPoint presentations in lectures. Computers & Education, 41, 77-86. Baugher, D., Varanelli, A., & Weisbord, E. (2003). Student hits in an internet-supported course: How can instructors use them and what do they mean? Decision Sciences Journal of Innovative Education, 1(2), 159-179. Blackboard (2005). Retrieved August 2, 2005, from http://www.blackboard.com/us/. Chan, K. C. & Shum C. (2004). An empirical analysis of a cumulative/re-work testing strategy: Its effect on student performance in principles of finance. Journal of Financial Education, 30(Winter), 16-31. Cudd, M., Tanner, J. & Lipscomb, T. (2004). A profile of classroom technology usage in finance instruction. Journal of Financial Education, 30(Spring), 28-40. DenBeste, M. (2003). Power Point, technology and the web: More than just an overhead projector for the new century? The History Teacher, 36(2), 491-504. Didia, D. & Hasnat, B. (1998). The determinants of performance in the university introductory finance course. Financial Practice and Education, 8(1), 102-107. Farooqi, N. & Saunders, K. T. (2004). Teaching methods and assessment techniques for the undergraduate finance course: A Canadian survey. Advances in Financial Education, 2(Spring), 52-56.
15
Krishnan, V. S., Bathala, C. T., Bhattacharya, T. K. & Ritchey, R. (1999). Teaching the introductory finance course: What can we learn from student perceptions and expectations? Financial Practice and Education, 9(1), 70-82. Lefcort, H. & Eiger, S. M. (2003). Preparatory versus practice homework. Journal of College Science Teaching, 33(1), 16-18. Lowry, R.B. (1999). Electronic presentation of lectures – Effect upon student performance. University Chemistry Education, 3(1), 18-21. Peters, M., Kethley, B. & Bullington, K. (2002). The relationship between homework and performance in an introductory operations management course. Journal of Education for Business, 77(6), 340-344. Rankin, E. L. & Hoaas, D. J. (2001). Teaching note: Does the use of computer-generated slide presentations in the classroom affect student performance and interest? Eastern Economic Journal, 27(3), 355-366. Rayburn, L. G. & Rayburn, J. M. (1999). Impact of course length and homework assignments on student performance. Journal of Education for Business, 74(6), 325-331. Saunders, K. T. (2001). Teaching methods and assessment techniques for the undergraduate introductory finance course: A national survey. Journal of Applied Finance, 11(2), 110112. Szabo, A. & Hastings, N. (2000). Using IT in the undergraduate classroom: Should we replace the blackboard with PowerPoint? Computers & Education, 35, 175-187. Van Ness, B. F., Van Ness, R. A., & Adkins, R. L. (2000). Student performance in principles of finance: Differences between traditional and internet settings. Financial Practice and Education, 10(2), 160-166.
16
WebCT (2005) WebCT Learning without limits. Retrieved August 2, 2005, from http://www.webct.com. Wilson, A.H. (2003). Evidence of the effectiveness of course management software and asynchronous communication in a first finance course. Journal of Financial Education, 29(Fall), 40-54.
17
TABLE 1. Descriptive Statistics
Mean
Median
Minimum
Maximum
Standard Deviation
Course Grade
62.28
63.36
12.46
98.80
15.96
Content Hits
45.95
42.50
0.00
156.00
26.83
7.74
8.00
0.00
13.00
2.84
23.00
25.00
0.00
34.00
7.84
Homework Solutions Access
7.59
9.00
0.00
12.00
4.30
PowerPoint Slides Access
8.79
10.00
0.00
12.00
3.67
Exam Solutions Access
1.53
2.00
0.00
3.00
0.93
73.68
74.15
50.63
95.00
7.18
Variable
Hit Consistency Unique Files Access
GPA
18
TABLE 2. Correlations between the Variables Variable Course Grade
Content Hits 0.182
Content Hits
Hit Consistency
Hit Unique Files Homework PowerPoint Consistency Accessed Solutions Slides
Exam Solutions
GPA
0.282*
0.212
0.273*
0.068
-0.125
0.654**
0.593**
0.756**
0.666**
0.535**
0.293**
0.124
0.748**
0.494**
0.724**
0.255*
0.225*
0.837**
0.742**
0.367**
0.147
0.336**
0.179
0.139
0.160
0.078
Unique Files Accessed Homework Solutions PowerPoint Slides
Exam Solutions
-0.005
* Significant at the .05 level. ** Significant at the .01 level.
19
TABLE 3. Regression Results Variable
Regressions I
II
III
IV
V
VI
Constant
-44.855** (0.00)
-45.574** (0.00)
-45.790** (0.00)
-47.543** (0.00)
-45.838** (0.00)
-45.288** (0.00)
VII -41.56** (0.00)
GPA
1.454** (0.00)
1.426** (0.00)
1.383** (0.00)
1.416** (0.00)
1.396** (0.00)
1.451** (0.00)
1.453** (0.00)
0.061 (0.24)
Content Hits Hit Consistency
0.795 (0.108)
Unique Files Accessed
0.241 (0.17)
Homework Solutions
0.688* (0.03) 0.074 (0.84)
PowerPoint Slides
-2.096 (0.16)
Exam Solutions R2
0.428
0.438
0.447
0.442
0.461
0.428
0.443
Adj R2
0.420
0.423
0.432
0.427
0.447
0.413
0.428
R2 Change
0.010
0.019
0.014
0.034
0.000
0.015
F-test for R2 Change
1.411
2.641
1.897
4.827*
0.039
2.051
* Significant at the .05 level. ** Significant at the .01 level.
20
