League tables exposed
Professor Tan Sri Dato' Dzulkifli Abd Razak
Learning Curve : Perspective
New Sunday Times - 04/11/2010
WHEN the criteria for "world-class" universities was proposed for Malaysian institutions, I recalled cautioning against the use of ranking.
And the rankings have now come back to haunt us!
From past experiences, we are acutely aware that such an exercise is far from perfect or desirable from an academic point of view. The Asiaweek ranking folded with the magazine. And the much touted The Times Higher Education Supplement (THES)-QS World University Rankings are forced down our throats!
So some six years after the Malaysian proposal, at the recent Going Global 4 Conference organised by the British Council in London, I felt somewhat vindicated after attending an illuminating session on League Tables: Flawed Ranking or Key Benchmarks?
The Times Higher Education deputy editor Phil Baty (also The Times Higher Education World University Rankings editor) and United Kingdom's Nottingham University Professor Rebecca Hughes were among the speakers.
Baty confessed to "distancing" himself from the THES-QS rankings.
He candidly spoke about the "flaws and imperfections" in the "old" THES-QS ranking system. He presented convincing data on " some of the tricks and games played to manipulate results". While this is not entirely unknown, hearing it from the horse's mouth is another matter altogether. Baty says: "QS failed to take into account dramatically different citation volumes between disciplines."
Ditto the hyped up "internationalisation" criterion. Baty questioned the relevance of the number of international students or staff as indicators of quality.
Baty also questioned "Peer review" which carries a 40 per cent weight. Some reviewers are not the best to make judgment, never mind their subjectivity. For those who insist that "ranking" is here to stay, Hughes gives a different viewpoint.
She argued that "academic communities already have their own systems for international evaluation of research (most notably peer review), and the aspect of collaborations which affects most students - teaching and learning - is less well served by the current ranking systems".
She brought up much neglected dimensions by asking questions such as: "How feasible are culturally neutral definitions of teaching quality?"
Unless properly calibrated, rankings can go against the trend to internationalise.
While there are pros and cons, striking a fine and meaningful balance seems difficult especially when ranking is so overwhelmingly driven by commercial interest!
There have been efforts to do a much more in-depth comparison using a broad based information capture without sacrificing "the missions and autonomy of higher education institutions in the subsequent effort to improve quality". It should be good for all cultures and languages, and the diversity of higher education institutions.
Such is the platform planned by the Organisation for Economic Co-operation and Development's Assessment of Higher Education Learning Outcomes project aimed at international teaching collaboration which has been undervalued by the other ranking systems.
It emphasises the multi-dimensional nature of quality in tertiary education, instead of a one-size-fits-all model!
Another ranking in the offing is the Alternative University Appraisal System pioneered by a group of Asian-based universities, including Universiti Sains Malaysia (USM), which takes a similar multi-dimensional approach.
It is now at a pilot stage and is expected to be launched next year in the hope of providing a much needed mindset shift to bring back universities into the realms of serious scholarship rather than a preoccupation with numbers.
* The writer is the Vice-Chancellor of Universiti Sains Malaysia. He can be contacted at vc@usm.my
Learning Curve : Perspective
New Sunday Times - 04/11/2010
WHEN the criteria for "world-class" universities was proposed for Malaysian institutions, I recalled cautioning against the use of ranking.
And the rankings have now come back to haunt us!
From past experiences, we are acutely aware that such an exercise is far from perfect or desirable from an academic point of view. The Asiaweek ranking folded with the magazine. And the much touted The Times Higher Education Supplement (THES)-QS World University Rankings are forced down our throats!
So some six years after the Malaysian proposal, at the recent Going Global 4 Conference organised by the British Council in London, I felt somewhat vindicated after attending an illuminating session on League Tables: Flawed Ranking or Key Benchmarks?
The Times Higher Education deputy editor Phil Baty (also The Times Higher Education World University Rankings editor) and United Kingdom's Nottingham University Professor Rebecca Hughes were among the speakers.
Baty confessed to "distancing" himself from the THES-QS rankings.
He candidly spoke about the "flaws and imperfections" in the "old" THES-QS ranking system. He presented convincing data on " some of the tricks and games played to manipulate results". While this is not entirely unknown, hearing it from the horse's mouth is another matter altogether. Baty says: "QS failed to take into account dramatically different citation volumes between disciplines."
Ditto the hyped up "internationalisation" criterion. Baty questioned the relevance of the number of international students or staff as indicators of quality.
Baty also questioned "Peer review" which carries a 40 per cent weight. Some reviewers are not the best to make judgment, never mind their subjectivity. For those who insist that "ranking" is here to stay, Hughes gives a different viewpoint.
She argued that "academic communities already have their own systems for international evaluation of research (most notably peer review), and the aspect of collaborations which affects most students - teaching and learning - is less well served by the current ranking systems".
She brought up much neglected dimensions by asking questions such as: "How feasible are culturally neutral definitions of teaching quality?"
Unless properly calibrated, rankings can go against the trend to internationalise.
While there are pros and cons, striking a fine and meaningful balance seems difficult especially when ranking is so overwhelmingly driven by commercial interest!
There have been efforts to do a much more in-depth comparison using a broad based information capture without sacrificing "the missions and autonomy of higher education institutions in the subsequent effort to improve quality". It should be good for all cultures and languages, and the diversity of higher education institutions.
Such is the platform planned by the Organisation for Economic Co-operation and Development's Assessment of Higher Education Learning Outcomes project aimed at international teaching collaboration which has been undervalued by the other ranking systems.
It emphasises the multi-dimensional nature of quality in tertiary education, instead of a one-size-fits-all model!
Another ranking in the offing is the Alternative University Appraisal System pioneered by a group of Asian-based universities, including Universiti Sains Malaysia (USM), which takes a similar multi-dimensional approach.
It is now at a pilot stage and is expected to be launched next year in the hope of providing a much needed mindset shift to bring back universities into the realms of serious scholarship rather than a preoccupation with numbers.
* The writer is the Vice-Chancellor of Universiti Sains Malaysia. He can be contacted at vc@usm.my