Predicting Academic Success Using Admission Profiles - Eric

5 downloads 0 Views 707KB Size Report
connection between admission policies and students' subsequent academic .... University, Bar Ilan University, Weizmann Institute, Ben Gurion University, and Haifa ..... Changes in access to higher education in the United States, 1820-1992, ...
Journal of International Education Research – Third Quarter 2015

Volume 11, Number 3

Predicting Academic Success Using Admission Profiles Nitza Davidovitch, Ariel University of Samaria, Israel Dan Soen, Ariel University of Samaria, Israel ABSTRACT This study, conducted at a tertiary education institution in Israel, following two previous studies, was designed to deal again with a question that is a topic of debate in Israel and worldwide: Is there justification for the approach that considers restrictive university admission policies an efficient tool for predicting students’ success at the end of their first year of studies and at the conclusion of their requirements for an undergraduate degree. This study reviews the spread of higher education in and outside Israel in recent years, and discusses the institution of admission policies as a response to the gap between the high demand for studies and the limited supply. This study discusses a diverse list of admission policies that offered admission based on students’ success at the end of their first year of studies and at the fulfillment of their requirements for an undergraduate degree. This study also reviews the debate in and outside Israel on whether restrictive admission policies have fulfilled the hopes pinned on them. Finally, the study conducted a detailed study of the effectiveness of different admission policies in the various faculties of the institution in question, and found no systematic connection between admission policies and students’ actual achievements (measured by the grade average at the end of their first year and their grade average for their degree). Keywords: Justification; University Admission Policies; Students’ Success; Undergraduate Degree; Higher Education INTRODUCTION

O

ur study offers an additional layer in a series of studies on the connection between admission policies and academic achievements. The most important finding of our previous studies (Davidovitch & Soen, 2006, 2008), which focused on graduates of a specific academic institution, was that there was no connection between admission policies and students’ subsequent academic achievements. Our studies revealed the surprising finding: Only 23% - less than one quarter - of the graduates had been admitted on the basis of the formal admission requirements of Matriculation and psychometric exam scores, while the vast majority of graduates had been admitted on the basis of informal criteria. What we found interesting is that the students who persevered in their studies and graduated - the majority - managed to overcome this seeming hurdle in the course of their studies. The present study also focuses on a case study and tracks predictors of success in higher education in the same academic institution several years after the original studies were conducted. Our aim was to reexamine whether Matriculation and psychometric score currently constitute reliable means that predict academic success (Kimmerling, 2000). We posited that currently admission policies continue to define a hurdle of sorts that has no proven systematic connection to students’ actual academic achievements. This case study addresses students’ fields of study (faculties) and achievements: their grade averages at the end of their first year in their chosen program, and their final degree grade averages. Embarking on this study, we accepted the assumption, already mentioned by Ayalon (2000), that different programs have different admission requirements and policies. We assumed that the admission requirements are a function of the study program selected. We sought to reexamine whether these requirements affect students’ achievements (Zadok, 1996). A careful analysis of the findings confirmed our hypothesis: This time - as in the two previous case studies - we found no consistent connection between admission profiles and students’ achievements at Copyright by author(s); CC-BY

125

The Clute Institute

Journal of International Education Research – Third Quarter 2015

Volume 11, Number 3

the end of their first year in their studies and at the end of studies toward a degree. This finding joins additional findings that emerge from numerous studies, including studies conducted outside Israel, and once again highlights the debate over the role of academic admission requirements and policies (Soen, 2004). Changes in the Higher Education System and Admission Policies in Israel and Worldwide Before WWII, the higher education system reflected the existing social order and function as a hothouse for cultivating the elite class (Havigarst, 1989; Morrison, 1998). Since the second half of the twentieth century, however, the higher education system has expanded significantly (OECD, 2003) and its target population has changed. This was the beginning of the era of “higher education for the masses” (Arun, Gamoran, & Shavit, 2007; Trow, 1970). Higher education began to be considered a basic right of citizens. For example, in the 1950s, the participation rate in higher education of the relevant age groups ranged from 3 percent and 5 percent, and in the mid1990s, this rate reached 20% in England, 35% in France and Germany, and 55% in the USA (Guri-Rosenblit, 1994). At the beginning of this era, a high school matriculation certificate or its equivalent was the admission slip into institutions of higher education. Demand for higher education continued to grow, on the one hand, while, on the other hand, the expansion of universities failed to meet this demand. As a result, in recent decades, changes have occurred in the admission policies of higher education worldwide, creating increasingly meritocratic policies. Needless to say, the dramatic expansion of higher education also affected Israel. In the last two decades, higher education in Israel has shown a general increasing trend. For example, in 1989/90, 21 institutions in Israel awarded academic degrees (compared to seven institutions in the 1960s). These institutions had 88.8 thousand students. In 2012/12, 70 (!) academic institutions in Israel were attended by 306.6 thousand students. The student population grew at an average annual rate of 5.8% (Central Bureau of Statistics, 2013, Press release 276/2012). In Israel, too, the growing demand for higher education had an impact on admission requirements, as we discuss below. The selection systems that “screen” candidates for admission into higher education institutions around the world are very different. Clearly, when we compare their various aspects, we find that most countries use high school grades or matriculation certificates as their admission requirement. Moreover, almost all OECD countries use additional tests, but unfortunately only in some fields of study. Such tests include national achievement tests (in 35% of the countries), uniform aptitude tests (in 14% of the countries), and specific admission tests that are administered by each education institution (in 25% of the countries). Additional criteria are used in 88% of the countries (Goldtzweig, 2014). Academic institutions worldwide use three main admission requirement policies for registered students (OECD, 2012): a combination of high school grades or matriculation grades; university entrance exams; unrestricted admission based on minimum requirements. Some institutions add to these three methods criteria such as recommendation letters, extra-curricular activities (“life experience”), community service, and socio-economic status. The custom of combining high school grades and one or another kind of entrance exam is the most prevalent practice. Needless to say, different conventions are used in different countries. For example, admission to universities in Australia is based on the candidate’s rank according to her or his matriculation certificate grades, which are weighted according to each graduating class. In Germany, admission is based on matriculation certificate grades or graduation certificate grades of vocational schools. Alongside this basis requirement, different programs pose additional, specific conditions. In the USA, admission to most universities is based on a weighting of high school course grades, the student’s academic ranking in his class, and the student’s score on the SAT or ACT test; additionally, more flexible parameters also enter into consideration. In Japan, public and private universities require candidates to pass a test administrated by the NCUEE, a national center for university admission tests, which covers 33 topics in six fields of study. This is only a small example of the large variance between countries. In addition, many universities require candidates for specific programs to pass uniform assessment tests. Table 1 presents these programs and the countries in which such exams are conventionally used. This table indicates that most programs in the health sciences have a general exam that is part of the admission requirements. Notably, general tests such as the SAT in the USA are not listed in this table, although this test is a requirement for various university programs. Copyright by author(s); CC-BY

126

The Clute Institute

Journal of International Education Research – Third Quarter 2015 Program Health sciences and medicine Law Psychology Engineering Mathematics History Education Source: Edwards et al., 2012

Volume 11, Number 3

Table 1. Programs that require specific exams, by country Country Required in all institutions? Australia, Belgium, Canada, Ireland, Yes South Korea, New Zealand England, USA Some institutions Australia, Britain, USA Some institutions South Korea Yes Finland Yes Australia, Belgium Some institutions Britain Some institutions Britain Some institutions Finland Yes

In summary, the admission requirements of academic institutions vary worldwide — In most countries, at least in several programs, candidates must pass an exam that is not administered directly by high schools. The exam is not necessarily a uniform assessment exam. Information on the tests is presented in Table 2. Table 3 presents data on other components of admission policies and their relative significance in the admission process. This table shows that the average matriculation grade is used in 21 of the 35 countries whose data appear in this table, and in one half of these countries, the matriculation grades are considered the most influential element in the admission process. Thus, with the exception of university entrance exams in the countries where they are used, matriculation exams are undoubtedly the most dominant parameter that is considered in a student’s admission process. Table 2. Exams and admission policies for higher education, by country (OECD, 2012). Country Admission policy Chile, Greece, Japan, South Korea, Mexico, Switzerland, An exam that is not administered by the high schools is Turkey, Brazil, Indonesia required for all programs. Czech Republic, Estonia, Finland, Israel, Italy, Spain, USA An exam that is not administered by the high schools is required for most programs. An exam that is not administered by the high schools is required for specific programs. Russia, Holland, Iceland No entrance exam is required Table 3. Effect of other elements on university admission (OECD, 2012) Impact on the admission process No. of countries that use this Element element (of 35 countries) High Moderate Low N/A Average matriculation grade 21 11 3 2 5 Previous work experience 14 0 2 5 7 Candidate’s letter of intent 11 2 3 3 3 Volunteering or community service 10 0 0 6 4 Family income 8 1 2 2 3 Recommendations 6 1 1 2 2 Ethnic origin 6 1 1 1 3

Establishment of the National Institute for Testing and Evaluation (NITE) and the Introduction of Psychometric Exams as a Screening and Prediction Tool in Israel Universities in Israel set their admission policies independently, based on the Council of Higher Education Law 1958. In Israel, as in most other developed countries, changes in university admission policies were introduced in response to adoption of the approach designed to transform higher education into a mass system. Until the 1960s, there were seven academic institutions in Israel: the Hebrew University of Jerusalem, the Technion, Tel Aviv University, Bar Ilan University, Weizmann Institute, Ben Gurion University, and Haifa University. In view of the number of students who passed their matriculation exams in each year until the 1960s, there was no need to define complicated admission policies. The demand was less than the supply. Just as was the case in the British Mandate period before independence, the institutions made do with candidates’ matriculation certificates or their equivalents, Copyright by author(s); CC-BY

127

The Clute Institute

Journal of International Education Research – Third Quarter 2015

Volume 11, Number 3

with the exception of specific programs, where admission was restricted due to limitations, such as laboratories (Davidovitch & Soen, 2006). In 1981, the Committee of University Heads decided to establish the National Institute for Testing and Evaluation, in order to create and operate a series of tests that predict success in academic studies. This was done to create a useful tool that would facilitate the selection process of candidates for admission into institutions of higher education in Israel (Vininger & Teshler, 2014). The universities were no longer able to admit everyone who sought to enter their gates, as demand outpaced supply. Since the 1980s, the main selection tool for admitting students into undergraduate programs at higher education institutions has been a weighted combination of their achievements, reflected in their matriculation exam scores, on the one hand, and the psychometric entrance examination (Machpal), on the other (Vininger & Teshler, 2014). In certain cases, an especially high grade on either matriculation exam or the psychometric exam might exempt the candidate from submitting her or his grades on the second exam. In January 2014, the Minister of Education gave notice of his attempt to institute a reform in the admission policies in Israel’s higher education institutions, so that admission would be based solely on students’ achievements on their matriculation exams (Skop, 2014). In the years that elapsed since the psychometric exam was introduced in Israel, a public debate has developed on the question of whether to continue or cancel its use or replace it with another selection system. Concurrently with this public debate, the NITE published a series of studies that supported the predictive validity and reliability of the psychometric exam. The findings of these studies indicated that the test has a high predictive value, which means that whoever received a high score on the psychometric exam generally succeeded more in their academic studies than those who received a lower score on the psychometric exam. Their grades at the end of their first year of academic studies, and at the conclusion of their undergraduate studies were higher than others who had less success on the psychometric exams (Kenet-Cohen et al., 1999; Oren et al., 2007). The NITE’s final conclusion, based on the studies that it had conducted over the years, was that the psychometric exam’s average predictive ability was greater than the predictive ability of the matriculation certificate, and that the combination of both grades generally offers a better prediction than each of the exams independently (NITE, 2013). PART II A Field Study in a Single Higher Education Institution In view of the above, the authors decided to conduct a case study and examine, in a single higher education institution, whether a significant statistical association exists between students’ admission profiles and their average grades at the end of their first year of studies and at the end of their studies in their respective undergraduate programs. The research question, therefore, was whether if the official admission requirements constitute a reliable tool for predicting academic success. The admission parameters that we examined included: psychometric score, matriculation grade, academic preparatory course (Mechina) grade, grades in associate engineering programs, and grades in previous undergraduate programs. The measures of academic success we used included: year 1 grades, and final undergraduate degree grades. We defined the following detailed research questions: 1.! 2.! 3.!

Is there an association between students’ psychometric scores and their grade average at the end of their first year of academic studies in the different faculties, and if so, what is the magnitude of this association? Is there an association between students’ Mechina grades and their grade average at the end of their first year of academic studies, and if so, what is the magnitude of this association? Is there an association between students’ grade average in their associate engineering programs and their grade average at the end of their first year of academic studies in the Faculty of Engineering, and if so, what is the magnitude of this association?

Copyright by author(s); CC-BY

128

The Clute Institute

Journal of International Education Research – Third Quarter 2015 4.!

Volume 11, Number 3

Is there any association between students’ grade average at the end of their first year of academic studies in the different faculties and their final grade average in their degree, by admission criteria (matriculation average, psychometric score, and Mechina grade average) and faculty, and if so, what is the magnitude of this association? In summary, can we point to a reliable predictor of success in the first year of academic studies and in undergraduate programs in total?

5.!

PART III Findings of the Study The association between students' psychometric scores and their grade average at the end of their first year of academic studies in the different faculties. We examined the association between students’ psychometric scores and their grade average at the end of their first year of academic studies. The following are the Pearson correlations of these two variables. Figure 1 presents first year grade averages by faculty and the distribution of grades for each faculty. The figure is followed by an analysis of the findings. Figure 1. First year grade averages and psychometric scores by faculty

Engineering 90

First&year

85 80

76.5

75.8

73.9

73.0

75

75.7

76.8

78.3

70 65 60 66)75

Copyright by author(s); CC-BY

76)80

81)85

86)90

129

91)95

96+or+higher

The Clute Institute

Journal of International Education Research – Third Quarter 2015

Volume 11, Number 3

(Figure 1 continued)

Health&Sciences 90

First&year

85

81.0

83.3

81.5

83.3

83.9

84.7

86)90

91)95

96+or+higher

80 75 70 65 60 66)75

76)80

81)85

Natural&Sciences

90 First&year

85 80 75

81.3

78.8 74.9 68.7

70

74.5

71.7

65 60 66)75

76)80

81)85

86)90

85 First&year

96+or+higher

Social&Sciences&and&Humanities

90

80

91)95

78.2

77.6

77.3

or+lower+65

66)75

76)80

78.6

81.2

82.4

85.1

75 70 65 60

Copyright by author(s); CC-BY

81)85

130

86)90

91)95

96+or+higher

The Clute Institute

Journal of International Education Research – Third Quarter 2015

Volume 11, Number 3

Faculty of Natural Sciences: Our analysis shows that candidates who were admitted with a psychometric score of 600 or higher achieved the highest grade averages in their first year of academic studies. Students whose psychometric scores ranged from 601 to 650 achieved an annual grade average of 82.1 in their first year, students whose psychometric scores were 650 or higher achieved an annual grade average of 81.2 in their first year, and students whose psychometric scores were lower than 450 failed and their annual grade average in the first year was 42.3. Students whose psychometric scores ranged from 451 to 500 achieved an annual grade average of 71. Students whose psychometric scores ranged from 501 to 600 achieved an annual grade average of between 78 and 80 in their first year. Our conclusion is that in the Faculty of Natural Sciences (N = 257), the association between psychometric scores and final grade averages at the end of the first year is positive and moderate (rp = .44). Higher psychometric scores predict, with relatively high probability, higher grade averages at the end of the first year. Faculty of Health Sciences: Our analysis indicates that the distribution of grades in the first year of the program is similar for all psychometric score groups. Students whose psychometric scores were lower than 450 achieved an average grade average of 83.6 (!). Students, whose psychometric scores exceeded 650, achieved an annual grade average of 85.5. The difference between grade averages in the first year is not consistent and is relatively small (range is between 81 and 85.5), compared to the range of students’ psychometric scores. Our conclusion is that in the Faculty of Health Sciences (N = 98), there is a relatively weak association between psychometric scores and grade averages at the end of the first year in the program (rp = .17). Psychometric scores have a limited ability to predict success in the first year of academic studies. Faculty of Engineering: Our analysis indicates that students whose psychometric scores are lower than 450 are the students with the lowest grade average at the end of the first year in the program (M = 72.3). The highest grade average at the end of the first year in the program was achieved by students whose psychometric scores were between 601 and 650. This group was followed by students, whose psychometric scores were between 551 and 600, achieving a grade average of 77.7 on average. We conclude that in the Faculty of Engineering (N = 912), the association between psychometric scores and grade averages in the first year of the program is very weak (rp = .12) and that psychometric scores have little predictive ability for first year success. Faculty of Social Sciences and Humanities: Our analysis indicates a correlation between psychometric scores and grade averages in the first year of the program. Students whose psychometric scores were very low (450 or lower) had the lowest average annual grade average (M = 77.6), while students whose psychometric scores were the highest (650 or higher) had the highest average annual grade average at the end of the first year (M = 89.6). Our conclusion is that in the Faculty of Social Sciences and Humanities, the association between psychometric scores and first year grade averages is moderate (rp = .28). We can say that higher psychometric scores frequently predict a higher first year grade average. Furthermore, in the Faculty of Social Sciences and Humanities (N = 1571), first year grade averages are rather strongly associated with students’ final degree grade average (rp = .47), such that a higher first year grade average predicts a higher final degree grade average (see Figure 2). Figure 2 indicates that in the Faculty of Social Sciences and Humanities, the higher the first year grade average, the higher the final degree grade average (rp = .47). For example, a first year grade average of between 76 and 80 predicts a final degree grade average of 76. A first year grade average of more than 91 predicts a final degree grade average of 88.2.

Copyright by author(s); CC-BY

131

The Clute Institute

Journal of International Education Research – Third Quarter 2015

Volume 11, Number 3

Figure 2. The Association between Mechina Grades and First Year Grade Averages, by Faculty

Social&Sciences&and&Humanities 90 84.1

First&year

85 80

86.2

88.2

76.0

75 70 65 60 76)80

81)85

86)90

91+or+higher

In the institution in question, many students were admitted into their respective programs on the basis of their grades on a preparatory academic course known in Hebrew as a Mechina. Students who attended the Mechina were students whose matriculation grades were insufficient for admission, as well as students who were permitted to take the Mechina even though they had not matriculated. The study examined the association between their final Mechina grade and their first year grade average. Figure 3 presents the distribution of these two grades by faculty. Figure 3. Average Mechina grades and first year grades by faculty

Engineering 90 First&year

85

80.5

80 75

71.7

72.3

76)80

81)85

74.6

70 65 60

Copyright by author(s); CC-BY

86)90

132

91+or+higher

The Clute Institute

Journal of International Education Research – Third Quarter 2015

Volume 11, Number 3

First&year

(Figure 3 continued)

90 85 80 75 70 65 60 55 50

Natural&Sciences 79.3

79.6

86)90

91+or+higher

67.4 57.3

76)80

81)85

Social&Sciences&and&Humanities

90 First&year

85 80

75.3

74.0

76)80

81)85

75

77.2

80.2

70 65 60 86)90

91+or+higher

Faculty of Natural Sciences: Figure 3 indicates a high correlation between Mechina and first year grades, where students with the highest Mechina grades (M > 91) also achieved the highest first year grade average (M = 79.6), while students who achieved the lowest Mechina grades also had the lowest first year grade averages (M = 57.3). We conclude that in the Faculty of Natural Sciences (N = 42), a moderate positive association was found between Mechina grades and first year grade averages (rp = .43). Faculty of Engineering: Similar to the case of the Faculty of Natural Sciences, this graph shows a correlation between Mechina and first year grades. Students with the highest Mechina grades (M > 91) are also the students with the highest first year grade average (M = 80.5). Furthermore, the students with the lowest Mechina grades (76-80) achieved the lowest first year grade averages (M = 71.7). We conclude that in engineering studies (N = 184), students’ Mechina grades have a relatively moderate association with first year grade averages (rp = .27). As seen in Figure 4, grade averages in an associate engineering program (N = 72) are weakly related to first year grade averages (rp = .15). On this point, we elaborate further below. Faculty of Social Sciences and Humanities. The figure indicates that when one examines the students who were admitted according to their final Mechina grades, one finds that students with the highest Mechina grades (91 or higher) also have the highest first year grade averages (M = 80.2). Students with the lowest Mechina grades also have low but not the lowest first year grade averages (M = 75.3). The lowest first year grade averages were Copyright by author(s); CC-BY

133

The Clute Institute

Journal of International Education Research – Third Quarter 2015

Volume 11, Number 3

achieved by students whose Mechina grades were between 81 and 85. Therefore, the association between the two sets of grades is inconsistent. We conclude that Mechina grades have a moderate association (rp = .47) with first year grade averages in the Faculty of Social Sciences and Humanities (N = 316). The association between Final Grades in an Associate Engineering Program and First Year Grade Averages in the Faculty of Engineering Program Candidates may be admitted into one of the programs offered by the Faculty of Engineering on the basis of an associate engineering certificate. In this study we examined the association between the final grade in students’ associate engineering program and their first year grade average. Figure 4 presents the distribution of first year grade averages in the Faculty of Engineering by students’ final grades in their previous associate engineering program. Figure 4. Associate engineering grades and first year grade averages in the Faculty of Engineering

Engineering 90 First&year

85 80 75

75.9 71.8

74.4

70.5

70

66.3

65

64.8

60 70+or+lower

71)75

76)80

81)85

86)90

91+or+higher

Figure 4 indicates that the association between these two sets of grades is far from consistent! The two lowest associate engineering grades do not predict first year grade averages in the Faculty of Engineering at all. Instead, the next two groups of grades predict much lower first year grade averages. Furthermore, the highest associate engineering grades (above 91) predict lower first year grade averages than do lower associate engineering grades (86-90). We conclude that the findings above show no correlation between grades in an associate engineering program and subsequent first year grade averages in the Faculty of Engineering (rp = .15). Final Degree Grades by Admission Requirements and Faculty In this section we examined the question, what admission requirements did the students meet when they were accepted, and what is the distribution of their grades upon fulfilling the requirements for their degree? The findings are presented separately by faculty, beginning with the Faculty of Health Sciences.

Copyright by author(s); CC-BY

134

The Clute Institute

Journal of International Education Research – Third Quarter 2015

Volume 11, Number 3

Figure 5. Health Sciences – Final degree grade average by admission profile

90 85

83.3

83.0

83.0

81.7 79.4

80 75 70 65 105

26

8

10

28

60

Note: Numbers inside the columns represent the size of the group.

This figure indicates that the largest group of students who earned an undergraduate degree from the School of Health Sciences had been accepted into the program on the basis of a weighted combination of their psychometric scores and their matriculation grades. This group also earned the highest final grades (M = 83.3). Very similar grades were also achieved by the students who were accepted into the program on the basis of their matriculation grades (M = 83.0) or on the basis of their matriculation grades and a previous academic degree (M = 83.0). Students who were accepted without meeting any requirements (there were 28 students in this group) earned the lowest final degree grades (M = 79.4). Students who were accepted on the basis of a combination of their psychometric scores, matriculation grades, and Mechina grades, earned a final grade that was only slightly higher (M = 81.7). In the Faculty of Engineering, a different picture emerges.

Copyright by author(s); CC-BY

135

The Clute Institute

Journal of International Education Research – Third Quarter 2015

Volume 11, Number 3

Figure 6. Faculty of Engineering – Final degree grade average by admission profile. 90 85.1 85

84.7 82.6

81.8 79.2

80

77.9

77.3 75.2

75

74.9

74.0

73.5

73.2 69.4

70

65 30

29

730

58

18

833

104

419

100

30

None

Mechina

48

24

9

60 Degree

Psychometric7Psychometric7Matriculation7Psychometric7MatriculationPsychometric7Matriculation7 +7 +7 +7degree +7 +7 +7Mechina Matriculation7Matriculation Matriculation7 Matriculation7 +7degree +7ass.7 +7Mechina Engineering

Copyright by author(s); CC-BY

136

Matriculation7Psychometric Ass.7 +7ass.7 Engineering Engineering

The Clute Institute

Journal of International Education Research – Third Quarter 2015

Volume 11, Number 3

Figure 6 indicates that the highest final grade average (M = 85.1) was earned by students who had been accepted into the program on the basis of a previous academic degree. A similar average (M = 84.7) was earned by students who had been accepted on the basis of a combination of a previous academic degree, psychometric score, and matriculation grades. Students who had been accepted on the basis of an associate engineering certificate earned the lowest grade average (M = 69.4), while students who had been accepted on the basis of their psychometric scores earned a higher grade average (73.3). Students who were accepted without meeting any admission requirements earned a final grade average of 74.9 in the engineering program. Findings regarding the Faculty of Natural Sciences show their own variations and are presented in Figure 7. Figure 7. Faculty of Natural Sciences - Final degree grade average by admission profile

90 85

83.3

83.0

83.0

81.7

80

79.4

75 70 65

105

26

8

10

28

60

This figure shows that the most effective predictor of success in a natural sciences program is a combination of a previous academic degree (in any field), psychometric scores, and matriculation grades (M = 83.2), although the grade averages of this group are slightly lower than their counterparts in the Faculty of Engineering. In contrast, students who had been accepted on the basis of a combination of psychometric scores, matriculation grades, and Mechina grades had a lower grade average at the end of the program in Natural Sciences (M = 68.4). Students who had been accepted without meeting any admission requirement earned a final grade average of 74.4. Predictors that Arise from the Association between Admission Requirements and First Year Success and from the Association between First Year Success and final Success in the Degree Program In view of everything stated above, one may state the following: With regard to grade averages at the end of the first year of academic studies, psychometric scores are a predictive tool of moderate success, and even so, only in two faculties: the Faculty of Natural Sciences and the Faculty of Social Sciences and Humanities. In the case of the Faculty of Social Sciences and Humanities, psychometric scores also manage to predict final degree grades with good success. Copyright by author(s); CC-BY

137

The Clute Institute

Journal of International Education Research – Third Quarter 2015

Volume 11, Number 3

Mechina grades also predict rather well first year academic success in the natural sciences, social sciences and humanities, and engineering. In contrast, grades in associate engineering programs do not make a significant contribution to the prediction of success at the end of the first year in an engineering program. With regard to general success at the end of the degree program, the best predictors vary from one faculty to another. In the Health Sciences, psychometric scores and/or matriculation grades are the best predictive measures, while in the Natural Sciences a combination of psychometric scores, matriculation grades, and previous academic degree generates the highest grade average. In engineering programs, a previous academic degree or psychometric scores in combination with matriculation grades and grades in a previous academic degree stand out among students who achieve the highest grades. This information is summarized in Table 4. Table 4. Association between admission parameters and academic success at the end of the first year and final year Admission parameters Success measures Summary of findings Psychometric score Moderately, in Natural Sciences and Social Sciences and Humanities Matriculation grades Mechina grades Predict first year grades In Natural Sciences, in Social Sciences and Humanities, and in Engineering Associate engineering grades Make no significant contribution to predictions of first year grades Final degree grade average Psychometric score Relatively well in Health Sciences Matriculation grades In Health Sciences Combination of psychometric score and In Natural Sciences and in Engineering matriculation grade Predict final degree grade average Mechina grades Associate engineering grades Grades in a previous academic degree

In general it appears that there are no absolute predictors of academic success. This implies that traditional measures such as psychometric scores or matriculation grades on their own are unable to predict academic success in all faculties. DISCUSSION In most countries in the western world today, students are admitted into universities on a meritocratic basis (Klitgraad, 1986), rather than a universal basis: not all candidates are admitted, only those whom the universities consider to be deserving. This approach is a result of the fact that the number of individuals who wish to study in institutions of higher education exceeds the number of available places. To screen candidates, universities set admission requirements that are supposed to predict candidates’ success in their studies. As we approach a summary of the effectiveness of admission requirements as predictors of academic success in undergraduate programs, we should take into consideration the fact that this issue has occupied the attention of researchers outside Israel as well. For example, Birch and Miller (2005) found that students’ success in undergraduate programs is actually a product of a long series of factors. A not inconsiderable portion of their success should certainly be attributed to their fulfillment of the relevant admission requirements. Another study conducted in Australia (McKenzie & Schweitzer, 2001) found that students’ previous academic achievements (their final high school grades) are the most significant predictors of their achievements in university. A team of researchers who examined this issue in New Zealand (Shulruf et al., 2008) also concluded that success on the NCEA exams (New Zealand National Certificate of Educational Achievement – a system that is analogous to the matriculation exams in Israel) is the best predictor of students’ success in their first year in an academic program. A similar conclusion was also reached by a study conducted in the UK by Smith and Naylor (2001) who found that matriculation exam grades are the most important predictor of students’ undergraduate grade average in university. Studies in Israel generate disputes on the connection between admission requirements and students’ success in their academic studies. For example, a study by Ayalon and Yogev (2000) argued that in certain faculties, there is no significantly statistical association between candidates’ performance on the psychometric exam, used as an Copyright by author(s); CC-BY

138

The Clute Institute

Journal of International Education Research – Third Quarter 2015

Volume 11, Number 3

admission requirement, and their success at the end of their first academic year. Another study, conducted in 2012 on students in the social sciences at the Yizreel Valley College (Ben David & Shichor, 2012) also found that the association between psychometric scores and undergraduate degree grades was doubtful, whereas the impact of matriculation grades was much more significant. Repeated studies by Davidovitch and Soen in one institution of higher education (2006, 2008) concluded that there is no connection between students’ admission profiles and their achievements at the end of their first year in an academic program. These studies revealed what was a surprising finding at the time: Only 23% - slightly less than one quarter - of all graduates were admitted on the basis of fulfilling formal admission requirements such as minimum matriculation grades or psychometric scores. The vast majority of graduates were admitted on the basis of informal admission requirements. What is surprising is that the students who remained in the system - and they are the majority - overcame this hurdle in the course of their studies. In contrast, a study conducted at Oranim College of Education (Zaslevsky & Lev-Ari, 2009) concluded that matriculation grades and psychometric scores predict students’ academic achievements in their first year of study at Oranim. Findings of the study, described in detail above, once again indicate that the admission requirements set by the institution’s various faculties constitute a type of hurdle that has no proven systematic connection to students’ subsequent achievements, both in terms of their grade average at the end of their first year in the academic program, and their final grade average for their degree. Although differences were found in the admission requirements defined by various programs, we did not find any consistent relationship between the score required by the admission requirements and students’ subsequent measured achievements. Admission requirements turned out to be a mixed bag in terms of their effectiveness. With regards to the probability of success at the end of the first academic year, psychometric scores managed to predict such success in only a moderate degree, and only in the Faculty of Natural Sciences and the Faculty of Social Sciences and Humanities. In the case of the Faculty of Social Sciences and Humanities, psychometric scores also predict success in the entire degree, rather well. Mechina grades also predict first year success in the Faculty of National Sciences, the Faculty of Social Studies and Humanities, and the Faculty of Engineering, reasonably well. In contrast, grades in an associate engineering program do not make a significant contribution to the prediction of first year success. With regards to overall success in the degree, the best predictors varied from one faculty to another. In Health Sciences, psychometric scores and/or matriculation grades are the dominant predictors, while in Natural Sciences, the combination of psychometric scores, matriculation grades and grades in a previous academic degree gives the highest grade average. In Engineering, a previous degree or psychometric score combined with matriculation grades and previous degrees are the dominant predictors among highest achieving students. This finding joins additional findings from numerous studies worldwide (Bolotin-Chachashvilli, Shavit, & Ayalon, 2002; Karen, 2002; Davidovitch & Soen, 2006, 2008) and brings up once again the question of the effectiveness of admission requirements as predictors of academic success at the end of students’ first year in the program and at the conclusion of their undergraduate studies. AUTHOR INFORMATION Prof. Nitza Davidovitch Prof. Nitza Davidovich, head of quality assessment and academic instruction at the Ariel University. Her areas of research interest include academic curriculum development, development of academic instruction, Holocaust awareness and Jewish identity, student exchange programs with Germany and Poland, preservation of the heritage of Jewish sects, and moral education Prof. Dan Soen Full Professor, Dept. of Sociology & Anthropology, Ariel University, Israel, and Emeritus Professor, Kibbutzim College of Education. Taught for years at the Tel-Aviv University, Ben Gurion University, Bar-Ulan University, Haifa University, the Technion and several universities abroad. His areas of interest include the Israeli society, urban and regional development, higher education, and Holocaust awareness. Copyright by author(s); CC-BY

139

The Clute Institute

Journal of International Education Research – Third Quarter 2015

Volume 11, Number 3

REFERENCES Arun, R., Gamoran, A., & Shavit, Y. (2007). More inclusion and diversion: Expansion, differentiation, and market structure in higher education. In Y. Shavit, R. Arun & A. Gamoran (eds.): Stratification in higher education: A comparison study (pp. 1-35). Stanford, CAL: Stanford University Press. Ayalon, H. (2000). Differences between women and men in selecting fields of study in university. Sotziologia Yisraelit, B(2), 523–544. [Hebrew] Ayalon, H., & Yogev, A. (2000). A window to the academic dream: Stratification and diversity in the expansion of Israeli higher education. Paper presented at the Calgary meeting of ISA-RC28. Calgary, Canada. Ben David, N., & Shichor, T. (2012). Does raising the minimum psychometric score for admission increase success in undergraduate studies in social studies. Harivon Lacalcala, 59(1-2), 51–79. [Hebrew] Birch, E. R., & Miller, P. W. (2005). The determinants of students' tertiary academic success. Perth, Australia: Business School, University of Western Australia. Bolotin-Chachashvilli, S., Shavit, Y., & Ayalon, H. (2002). Expansion of higher education and its implications on stratification in Israel 1980-1996, Sotziologia Yisraelit, D, 317–346. [Hebrew] Central Bureau of Statistics (2013, March 2). Higher education in Israel – Press release on the occasion of the new 2013 academic year. No. 276/2012. Jerusalem. [Hebrew] Davidovitch, N., & Soen, D. (2006). Enhancing access in theory and practice: A study of graduates of a large Israeli public college in 1995-2003. International Education Journal, 7(3), 284–304. Retrieved from http://iej.cjb.net/ Davidovitch, N., & Soen, D. (2008). A college as a lever for graduates settling in the region: A case study of the College of Judea and Samaria. Israel Affairs, 15(1), 66–80. Edwards, D., Hamish Coates, H. & Friedman, T. A survey of international practice in university admissions testing. Higher Education Management and Policy 24/1, 2012. Goldtzweig, A. (2014). A review of higher education selection systems worldwide. Paper presented at the Tenth Conference of the Israel Psychometric Association, January 12, Wahl Center, Ramat Gan. [Hebrew] Guri-Rozenblit, S. (1994). The university: For elites or for everyone? Psifas, 29–31. [Hebrew] Havigarst, H. V. (1989). Education and social mobility in four societies. In R. Shapira & R. Peleg (Eds.), Sociology of education – An anthology (pp. 187–203). Tel Aviv: Am Oved University Library. [Hebrew] Karen, D. (2002). Changes in access to higher education in the United States, 1820-1992, Sociology of Education, 75, 191–210. Kenet-Cohen, T., Bruner, S., & Oren, K. (1999). Meta-analysis of the predictive validity of the components of the university screening system for success in studies, Megamot, 40, 54–71. [Hebrew] Kimmerling, B. (2000, May 19). Basic law – the right to learn. Haaretz. [Hebrew] Klitgraad, R. (1986). Elitism and meritocracy in developing countries: Selection policies for higher education. Baltimore: The Johns Hopkins University. McKenzie, K., & Schweitzer, R. (2001). Who succeeds at university? Factors predicting academic performance in first year Australian university students. Higher Education Research & Development, 20(1), 21–33. Morrison, T. (1998). Welcome to the world: Quality, higher education and change. Mauritius: Mauritius Institute of Education. NITE (2013). Use of the university psychometric entrance exam (Machpal) – A status report and summary of data for the years 2010-2012. Jerusalem. [Hebrew] OECD (2003). Education at a glance. Table A2.4. www.oecd.org/document. Accessed: 2.3.2011. OECD (2011). Factors, criteria or special circumstances used by tertiary institutions to determine access to the first stage of tertiary education. In Education at a glance. http://www.oecd-ilibrary.org/education/education-ata-glance-2012/factors-criteria-or-special-circumstances-used-by-tertiary-institutions-to-determine-accessto-the-first-stage-of-tertiary-education-2011_eag-2012-table279-en Oren, K., Kenet-Cohen, T., & Bruner, S. (2007). Collected data on the validity of university selection system in predicting first year success (2003-2008). NITE [Hebrew] Shulruf, B., Hattie, J., & Tumen, S. (2008). The predictability of enrolment and first- year university results from secondary school performance: The New Zealand National Certificate of educational achievement. Studies in Higher Education, 23(6), 685–698. Skop, Y. (2014, September 1). Study: Psychometric score is better predictor of academic success than matriculation grades. Haaretz. Retrieved from http://www.haaretz.co.il/news/education/.premium-1.2421288 [Hebrew] Copyright by author(s); CC-BY

140

The Clute Institute

Journal of International Education Research – Third Quarter 2015

Volume 11, Number 3

Smith, J., & Naylor, R. A. (2001). Determinants of degree performance in UK universities: A statistical analysis of the 1993 student cohort. Oxford Bulletin of Economics and Statistics, 63, 29–60. Soen, D. (2004). Center and periphery in higher education in Israel – Second change for whom? Hahinich Visvivato, 26, 153–166. [Hebrew] Trow, M. (1970). Reflections on the transition from mass to universal higher education. Daedalus, 99, 1–42. Vininger, A., & Teshler, N. (2014). Selection in admission to higher education institutions and the psychometric exam. Jerusalem: Knesset, Research and Information Center. [Hebrew] Zadok, M. (1996). Post-secondary education in Israel in the year 2000. In A. Peled (Ed.), Master plan for Israel in the twenty-first century – Thematic planning policy: The education system (pp. 281-291). Ministry of Education, Culture, and Sport. [Hebrew] Zaslevsky, T., & Lev-Ari, L. (2009). A three-year longitudinal study: The connection between admission criteria at Oranim (matriculation grades and psychometric scores) and students’ college achievements (A follow-up initiative). In M. Velitzker-Pollack (Ed.), Pedagogical initiatives and social intervention initiatives (pp. 146). Oranim: The Academic College of Education. [Hebrew]

Copyright by author(s); CC-BY

141

The Clute Institute

Journal of International Education Research – Third Quarter 2015

Volume 11, Number 3

NOTES

Copyright by author(s); CC-BY

142

The Clute Institute