Scotch College

From the Principal

Top 100 ranking flawed

Rank order unreliable and misleading

The publication of comparative school performances in the 1996 VCE was opposed by Principals from all parts of the schools systems. In spite of this opposition, the Board of Studies insisted that school specific data would be released for publication.

The clear intent was to suggest that inter-school comparisons should be made from the published material.

With headlines referring to 'Top Schools', the Herald Sun published tables of limited information that they had manipulated to produce a rank-order of their top 100 schools.

The clever element of the Board of Studies data was the introduction of the idea of an 'achievement index', purporting to indicate the value the school had added during the VCE year. This removed any notion of comparing schools purely on VCE results, from which position schools with selective intake policies, or which draw students from educationally advantaged backgrounds, would clearly benefit. The concept of value-adding is fine, but only if the method used to measure it is sound.

VCE index

Regrettably, the Board of Studies/Herald Sun index is demonstrably very seriously flawed, and the subsequent rank-order of schools is unreliable and misleading.

The index compared each schools' VCE results with its students' performances in the General Achievement Test (GAT).

This process confers on the GAT a status for which it was never designed, its purpose only being to indicate if a school's internal marking of Common Assessment Tasks (CATs) should be re-assessed by the Board of Studies.

To suggest that the GAT measures 'raw material' is quite absurd, since the skills it tests are taught, almost always, by the school in which the student attempts the VCE. A school that does an excellent job developing those basic skills will find it impossible to excel in the index.

The patent absurdity of the index is demonstrated by the following comparison. Scotch, placed 33rd by the Herald Sun, had 25% of its CATs scoring 40 or above (out of the maximum of 50). The school listed 6th in the table, however, had 12% of its scores over 40. Clearly, our GAT results must have been better than the other school's, yet they ranked higher - with inferior VCE results and more inferior GAT scores. There is something very seriously amiss with a school comparison system that rewards poor performance in one of its measures!

Other serious concerns exist about the index. It was calculated using 'raw' scores and ignored the scaling process which takes into account relative difficulties of subjects. Scaling means, for example, that a score of 40 in Further Mathematics counts as 36 towards the final Tertiary Entrance Ranking (TER) whereas 40 in Specialist Mathematics counts as 50. Yet the index treated both as equally meritorious achievements. The use of raw and not scaled scores renders the index invalid. The misuse of the GAT adds to the absurdity of the process.

Scotch reaction to the rank-ordering

How will Scotch react to the advent of this rank-ordering process? If we believe it is important to improve our position then there are three steps to take:

1. Improve VCE results. Given the excellence of our performance in recent years, and particularly in 1996, it is not realistic to anticipate a significant improvement - although staff and boys will certainly maintain their outstanding endeavours.

2. De-emphasise the GAT. There can be no educational ground for suggesting to students that students should enter a test to do poorly.

3. Discourage students from attempting the more difficult (and more highly rewarded) subjects.

The latter strategies would certainly boost Scotch's index, but would be detrimental to our students' best interests. This, of course, is completely contrary to Scotch's educational philosophy, and will not be pursued.

Council has no confidence in the rank-ordering

The Scotch College Council recently considered the matter and declared that they have no confidence in any numerical inter-school comparison, which does take into consideration the complex elements that are involved in a full educational programme. The Council is united in its support of the Principal in deciding not to be influenced by the introduction of the ranking of schools.

Rather, Scotch will continue to encourage our students to tackle demanding subjects and be involved in the school's whole programme in the firm belief that this will be, for them, the best preparation for their futures.

Dr F G Donaldson, Principal