LONGTERM RETENTIONAS A MEASURE OF SCHOOL EDUCATION: A NOVEL METRIC PALLAVI VETURY-IYER and VETURY SITARAMAM* "Vetury",# 18, S.No. 245,AnantCoop.Hsg. Society, Aundh, Pune - 411 067, India,*Corresponding author,
[email protected].
A major rethink on the structuring of pedagogy in terms of teaching and evaluation practicesare the need of the hour in the growing aspiration for an effective STEM education1.The student, teacher and curriculum form a triad in the context of learning, requiring interdependent methods of evaluation, necessarily quantitative, befitting STEM education itself2. The purpose ofeducation continues to be two-fold: to ‘perform’ in the society and to ‘belong’ to that society in terms of its value systems,the former being of the current concern, the latter, of no less importance3-5.Evaluation in education reduces the individual performance to a set of scores or grades, much debated for its comprehensiveness6.The crux of the problem is that while rote learning is considered the bane of education in policy statements7, the actual testing practices (time-bound, content-bound and agebound),actively ensure it. In this study, we modified the nature of the examination and measured the extent of the ‘failure’ of the system, in terms of retention by students, based on trends rather than on the conventional headcounts based on scores. We arrived at a metric that shows a consistent fall in retention not attributable to rote learning or the performance level of students by usual yearend grades, but varies with the content of the subject/test. It helps identify among the lower scorers, those who have better retention/comprehension, though less able to cram for tests. Since the students do vary in their ability to learn as much the teachers in their ability to teach, the new metric of the fall in retention described here focuses uniquely on the need to determine and distinguish quantitatively the limits to learning and limitations in imparting learning, a prerequisite for an effective STEM education8. The isolationist policy of cherry-picking students after school, based on such conventional evaluations, into extremely selective institutes (like IITs etc., in India, prevalent in rest of the world as well) cannot substitute for a national strategy for STEM education9 since it needs to be mass-based, aiming at the creators as well as users of that knowledge. At the same time, the income-dependent10-12, high dropout rates (> 30% in India) and grossly inadequate learning at the mass level do not indicate any success of public policies (cf. studies by Pratham13 and elsewhere14, followed by preliminary reports of baseline tests by the several Indian states15-17 as with the rest of the world18). Insights that facilitate causally relevant intervention for students are few. Evaluation of teachers is equally dismal19. Percentage of teachers that pass the 1
National teacher eligibility test (TET) at the middle school level varies from 0.6% (2012) to 12% (2015)20. The third critical component, the curriculum design itself, is with hardly any metric or feedback21,22. Since the conventional examinationsin schools actually incentivize cramming, we tested the 9th class studentsfor their retention from class 1 to even class 10 (as a control for the class they were yet to attend) without prior notice.The multiple choice tests were based on what they were actually taught and the questions set by the teachers who taught these subjects.Fig. 1 summarizes the possible outcomes…of the ability of the 9th class students to recall what they learnt in the previous years, i.e., retention. Had the learning been perfect, the ideal representation would be invariant over the years as in AB (~100% as in Model I, actually seen with students selected for higher education in national tests23).Had the learning been purely by rote, it would be increasingly more difficult to recall from the previous years and the results would approximate rote learning and memory measured empirically as with Ebbinghaus24,25, i.e.,Model II. Though his model was based on time dependence of the limiting case of discrete, disconnected words without meaning, various reinforcements for better memorizing such as repetition would tend to enhance recall, i.e., the angle β to 900approximating Model Icf.24. At the population level, precise relationships such as exponential decay etc., would be irrelevant beyond just trends, since the data would be strongly influenced by distributions.The third alternative, in principle, would represent a disturbing element in assessing education: the actual learning declines(α tending to zero) over years relentlessly (Model III). The anticipation was that the students should not be able to answer the 10th class questions, set as a logical control since they are yet to begin to attend the class. A
B α
90- β
90-α
β
Score →
C
-x
9
1
10
+x
Class → Fig.1. Possible outcomes of retrospective testing of class 9 students tested up to class 10 after completion of their class 9 year-end examinations as a linear approximation. A-B represents a perfect score of ~100 for all the years up to 9 and goes down to 0 at class 10 which the students are yet to begin (Model I). If the education were entirely by disconnected rote learning, the Ebbinghaus model would
2
predict B-(-x), where β follows the memorized content which disappears progressively towards earlier classes (Model II). At best, variants of Ebbinghaus model on memory/retention would only predict that β tends to zero at no retention or a right angle at full retention (Model I). However, if retention/learning is compromised in education, the performance would deteriorate (α tending to zero) with successive years (A-C) (Model III). Note that one would expect B or C to drop to near zero for Class 10 component of the test. If it were to merely follow the regression line of A-C, i.e. C-(+x)), it would represent a major anomaly that requires an explanation.
A major problem in designing these studies was that the anticipated normal distribution of the performance of the students would tend to confound the statistical relationships due to grouping around the mean. We selected students based on equidistant performance in the latest available 8th class school scores to yield a near uniform (platykurtic) distribution, kurtosis being the major concern(Fig. 2).
th
Fig.2.Frequency distribution of the 8 grade marks (filled circles, 67.8 ±10, n= 105, kurtosis, -0.165), students for the new test selected to achieve a uniform distribution (unfilled circles, 68.8 ± 14, n= 28, kurtosis, -0.846) and the final result of the new test (filled triangles, 68.8 ± 8.5, n=28, kurtosis, 0.648). Note that the 3 sets of results on average did not differ.
Fig. 3,Left shows the aggregate 8th class scores plotted against the current test aggregate scores. The relationship is significant even though 4 students among the 32 selected did not take the test. The frequency distributions of the new test scores were considerably more leptokurtic than the sampled uniform distribution based on 8th class scores (Fig. 2), showing that the new test measures something different than the conventional school tests26. The critical observation was that the new test score showed an increase in variation at lower scores i.e., marked heteroscedasticity (Fig.3,Left). This is important because at least 20% of the children that would be labelled poor performers in the class tests emerged with better scores indicating comparable retentive ability as with the 3
better performers, suggesting an unjust misclassification! Fig.3, Right reveals what was not anticipated. The score for 10th class was not low and the Model III regression, i.e., past performance, determined even the future 10th class score, raising certain distinct possibilities. Firstly, if the fall in scores shown by model III was due to lesser learning, a. it reflects a general tendency in education that the ability to learn per se decreases, grave, if true; b: they are taught little in 10th class, largely spent in cramming, preparative to Board exams, the merit of which itself questionable. Secondly, acquiring scores higher than anticipated for 10th class not yet taught could mean that i. they attend coaching classes (which is true in majority of students), or, ii. this test per se was very lenient as they were not anyway taught much. Reality could well be a combination of all these.
1 Std 10 test scores, Recorded
Current test score,%
100 80 60 40 20
0.8 0.6 0.4 0.2 0
0 30
50 70 90 8th grades scores,total,%
0.1
0.3 0.5 0.7 Std 10 test scores, Predicted
0.9
Fig. 3. (Left):A comparison of performance by school tests and the current test (normalized to 100) for retention. Linear regression of the 28 students who attended the test was by the method of least squares, y = 0.224x + 52.30, r=0.374, p
