Z Score then and now

It is very often said that when Z score scaling process was introduced in 2001 there was no problem and the Supreme Court agreed with the scheme proposed by the Educational Authorities but now a problem has arisen due to the pooling of populations advocated by a panel of “experts” in Statistics. In 2001 Z scores of individual candidates were added and then taken the mean dividing by four or three depending on the number of subjects that the candidate had sat.

As has been mentioned previously Z scores in different subjects can be compared but that does not make them equivalent and a Z score of x in a subject A is not equivalent to the same Z score of x in subject B. Nevertheless the University admissions are based on Z score and we will stick to the Z score scaling process for the time being.

Scaling of marks is usually done when the populations are different. The differences in populations could arise due to various reasons. The students sitting for two different subjects belong to two different populations and similarly students sitting for either the old syllabus or the new syllabus in the same subject have to be considered as belonging to two different populations.

Under normal circumstances (say from 2004 to 2010) when in each year the subjects were the same and the syllabi had not been changed the differences in the populations arose mainly as a result of degree of hardness of the question papers in each subject, though strictly speaking the degree of hardness in each paper of the same subject also had to be taken into account, assuming of course that there was no difference between say the quality of the students sitting the examination in different subjects. In such years assuming that the Z scores are equivalent the individual Z scores of a candidate in each subject may be added and then the mean can be calculated for the purpose of ranking of students.

However, when the syllabi are changed there is another difference in the populations that has to be taken into consideration. It is due to the fact that in each subject there are two different populations sitting the examination, one in the new syllabus and the other in the old syllabus. The first group consists of students sitting for the first time and the second group sitting for the second or the third time. The quality of the students of the two groups is not the same and some remedial scaling process has to be adopted when such situations arise. The “expert” panel has come out with a procedure called pooling and they have suggested a formula to be adopted. It is not a formula of the UGC or the Examinations Department or the Ministries of Education and Higher Education but one that has been proposed by a panel of “experts” in Statistics. I myself have a difference of opinion on the formula though I am not an “expert” in Statistics by any stretch of imagination, but in principle I agree with the suggestion of a formula for pooling by the panel.

As has been suggested already, marks can be pooled and scaled first and then such scaled marks can be rescaled after de-pooling or regrouping, using the Z scores, if it is Statistically sound, thus taking into consideration both the differences in populations due to the quality of students and due to the degree of hardness of the subjects. In 2001, 2002 and 2003, as I understand, there was no way of first pooling and scaling taking into consideration the differences in quality of students as not only the number of subjects had been reduced from four to three but in certain cases subjects such as Pure Mathematics, Applied Mathematics, Botany and Zoology had been dropped altogether adding new subjects such as Combined Mathematics and Biology. In 2011 (and in 2012 and 2013) the subjects remain the same though the syllabi have been changed.

Copyright Prof. Nalin De Silva

Z score and populations

I would not have written this if not for the explanations given by Prof. Thatil on the Z score. He has referred to the formula used by the panel of experts on pooling of two populations and said that it is not correct to pool them and then find the Z score of the pooled populations. Instead he suggests that the Z scores of different subjects of the two populations should be taken as they are and then add the Z scores of the three subjects each student has sat and then find the average of the three Z scores. The panel on the other hand pools the populations that sat the old syllabi and new syllabi examinations, find a Z score in each subject for the pooled populations and then take the average of the three Z scores to determine what is called final Z score of a student.

However it is the addition of Z scores in separate subjects that is questionable rather than the pooling of two populations. What is forgotten in the whole exercise is that what we are interested in selecting the students who performed better at the GCE (A/L) examination. Even if we have a group of students who sat for the same three subjects in the new syllabi how are we going to select the better students? Consider the following two cases. Sandun obtains 90 marks for Combined Mathematics 80 for Physics and 40 for Chemistry, while Sanduni obtains 60 marks for Combined Mathematics 70 for Physics and 85 for Chemistry. If we go by the aggregates of marks Sanduni is the better student. If the Z scores are considered this order may be reversed. However we cannot select the better student even by adding the respective Z scores. How does one compare the Z score in Chemistry with that in Physics? In other words how many marks in Chemistry are equivalent to say 70 marks in Physics?

If Sandun has 90 US Dollars, 80 Pounds and 4000 SLRupees  and Sanduni has 60 US Dollars, 70 Pounds and 8500  SLRupees we can find out who has more money by converting each currency to a common currency using the current exchange rates and then adding the money in the common currency. My question is what the exchange rates are as far as marks obtained in different subjects are concerned. I do not think that the Z scores give an "exchange rate" in respect of marks obtained in different subjects.

This problem arises not only at selection of students to the Universities but even at term tests in schools. We may say Sanduni came first and Sandun came second by just adding the raw marks. However it may not be so and I suppose in order to overcome this problem some schools adopted a method called scaling of marks in the fifties. The idea was again, I am assuming as I had not discussed this problem with any of the teachers, to find exchange rates between marks in different subjects. It may not be a bad idea for the experts in Statistics to find these exchange rates at least from a literature survey, if they are available, to be used at future examinations.

Though Prof. Thatil may not have realized it, when we calculate the average mark in a subject that is examined using more than one question paper we pool two or more populations and obtain the combined average. For example, though the same students sit for Combined Mathematics I ("Pure Mathematics") and Combined Mathematics II ("Applied Mathematics") they cannot be considered as one population. They are two populations but we consider them as one population for the purpose of calculation of the average mark in Combined Mathematics and then calculating the mean and variance in order to find the Z scores in Combined Mathematics. It would have been better if we have an exchange rate for conversion of marks from Combined Mathematics I to Combined Mathematics II and vice versa but until such exchange rates are found we may have to pool populations whenever pooling is better than any other method available. 

Copyright Prof. Nalin De Silva