It is very often said that when Z score scaling process was introduced in 2001 there was no problem and the Supreme Court agreed with the scheme proposed by the Educational Authorities but now a problem has arisen due to the pooling of populations advocated by a panel of “experts” in Statistics. In 2001 Z scores of individual candidates were added and then taken the mean dividing by four or three depending on the number of subjects that the candidate had sat.
As has been mentioned previously Z scores in different subjects can be compared but that does not make them equivalent and a Z score of x in a subject A is not equivalent to the same Z score of x in subject B. Nevertheless the University admissions are based on Z score and we will stick to the Z score scaling process for the time being.
Scaling of marks is usually done when the populations are different. The differences in populations could arise due to various reasons. The students sitting for two different subjects belong to two different populations and similarly students sitting for either the old syllabus or the new syllabus in the same subject have to be considered as belonging to two different populations.
Under normal circumstances (say from 2004 to 2010) when in each year the subjects were the same and the syllabi had not been changed the differences in the populations arose mainly as a result of degree of hardness of the question papers in each subject, though strictly speaking the degree of hardness in each paper of the same subject also had to be taken into account, assuming of course that there was no difference between say the quality of the students sitting the examination in different subjects. In such years assuming that the Z scores are equivalent the individual Z scores of a candidate in each subject may be added and then the mean can be calculated for the purpose of ranking of students.
However, when the syllabi are changed there is another difference in the populations that has to be taken into consideration. It is due to the fact that in each subject there are two different populations sitting the examination, one in the new syllabus and the other in the old syllabus. The first group consists of students sitting for the first time and the second group sitting for the second or the third time. The quality of the students of the two groups is not the same and some remedial scaling process has to be adopted when such situations arise. The “expert” panel has come out with a procedure called pooling and they have suggested a formula to be adopted. It is not a formula of the UGC or the Examinations Department or the Ministries of Education and Higher Education but one that has been proposed by a panel of “experts” in Statistics. I myself have a difference of opinion on the formula though I am not an “expert” in Statistics by any stretch of imagination, but in principle I agree with the suggestion of a formula for pooling by the panel.
As has been suggested already, marks can be pooled and scaled first and then such scaled marks can be rescaled after de-pooling or regrouping, using the Z scores, if it is Statistically sound, thus taking into consideration both the differences in populations due to the quality of students and due to the degree of hardness of the subjects. In 2001, 2002 and 2003, as I understand, there was no way of first pooling and scaling taking into consideration the differences in quality of students as not only the number of subjects had been reduced from four to three but in certain cases subjects such as Pure Mathematics, Applied Mathematics, Botany and Zoology had been dropped altogether adding new subjects such as Combined Mathematics and Biology. In 2011 (and in 2012 and 2013) the subjects remain the same though the syllabi have been changed.
Copyright Prof. Nalin De Silva