As was announced by the Government a while ago, the schedule has slipped back a year from the original plans - it was going to be REF 2013. On the new plans, institutions will be invited to make their submissions to REF 2014 in January 2013, with submissions closing on November 29th 2013. The results will be published in December 2014, so a whole year has been allowed for the review of submissions across the 36 units of assessment (UoA).
For each UoA, the evaluation of research will be based on three elements. These are output, impact and environment. Research output will account for 65% of the overall rating of each UoA, research impact 20%, and the research environment the remaining 15%.
As with the previous Research Assessment Exercise (RAE), each researcher included in a given UoA will be able to submit up to four publications for assessment, their quality being judged in relation to relevant international standards, and for their 'originality, significance and rigour' (p6). Apparently some panels might choose to use citation data to support their judgements, though when this idea was first proposed - for universal use - it was shot down in flames. There are huge differences between disciplines in citation practices, for some disciplines there is hardly any published data, and citations are subject to well known distortions - such as self-citation, citation of one's friends, etc. It's not as objective, consistent and reliable as people initially assumed.
Research impact has been the most controversial measure, and is still widely debated - and criticized. The aim, though, will be for the impact of each UoA on the economy, society, culture to be judged. To me, this sounds an absolute nightmare, with assessments highly vulnerable to subjectivity, prevailing fashions in research, and the like. But hopefully my skepticism will prove to be unfounded.
Judging the research environment is a bit less problematic, I think, except that there seems to me to be a built in bias towards large size, in that a larger UoA with lots of staff and PhD students will have an easier time showing that it possesses the required attributes of 'vitality and sustainability' (p6). Yet these days, with collaboration across department in different institutions, and much easier research networking through the internet, I would have thought that the scale of a Unit should matter far less than it used to. One almost gets the feeling that those who have designed the REF haven't quite caught up with the huge benefits we get from IT and electronic communication to support our research. But you never know, the assessors might find ways to take all this into account.
REF 2014 will be a big and costly exercise, so it's worth asking what the results will be used for - apart from being highlighted on institutional websites when they come out well!
Three aims are featured in recent HEFCE publications about REF 2014. These are:
- To provide the basis for the selective allocation of research money to individual higher education institutions (in the past, this was the QR stream of funding; I don't know if it will continue to be denoted this way);
- To give the sector a means of benchmarking its research standards and providing a measurable basis for research reputation;
- Given that much research is supported by public funding, either from the Funding Councils of the Research Councils, there is a perceived need for public accountability and the results of REF 2014 are expected to show both how public money has been used, and to demonstrate value for money.
It's probably quite handy to collect just one set of data to fulfill these three rather different aims, but it's hard not to wonder whether the data we shall have can really serve such distinct purposes. I need to think about this a bit more, and discuss the area again in later posts.
No comments:
Post a Comment