REF - Transparency and Feedback

The latest issue of Times Higher Education includes an article by Mark Burnley about the REF. He complains that when the results of REF2014 are eventually published, while individual units of assessment will receive their grading in the form of a distribution (or profile) rather as occurred with RAE2008, individual academics will not get any direct feedback telling them how well or otherwise they are judged to have performed. Mark Burnley is clearly unhappy about this situation, and thinks it should be changed.

One of the reasons he gives is that if a given unit is assessed as having, say, 10% of its research of outstanding, world class quality, how can it know which 10% and who delivered that spectacularly good work? His point, I think, is that if the unit doesn't know that, it can't learn from the outcome of REF2014 and do even better next time. More generally, he claims that academics need individualised feedback from the REF to enable them to understand where their weaknesses are and hence, what they can do about them (for next time). However, I think this way of using the REF results is inappropriate, it's not what the system has been designed for, and there ought to be other ways for academics to get the feedback they need. Let me explain.

(a) The REF has been designed to evaluate units of assessment, not individuals, and its fundamental purpose is to provide performance indicators that will be used by the Funding Councils to allocate the research component of their grants to institutions (hitherto, the QR stream) for some years following REF2014. Thus its function is to guide the allocation of financial resources, not to evaluate individuals.

(b) Next, if a unit of assessment does indeed have 10% of its research adjudged 'outstanding' then it seems to me quite bizarre to imagine that members of that unit, or managers in the relevant institution, might find it difficult to identify which parts of their work are so extremely good. If they can't figure out for themselves which elements of their research are world class, then surely something is badly wrong......... After all, we would surely like to think that the REF panels, etc., will do their work as fairly and objectively as reasonably possible, so the designation 'outstanding' is hardly going to be conferred pretty much randomly. One hopes not, anyway!

(c) Last, quite regardless of the RAE, and now the REF, most academic institutions that are serious about developing and improving their research already have systems in place for assessing individual staff - setting targets periodically, monitoring performance, providing training, and so on. Not always, but very often, promotion is the reward for successful delivery of quality research output over a period, and for most of us, promotion is a rather strong incentive to perform well. Of course, acknowledging all this doesn't stop many of us from griping about the iniquities of our internal evaluation systems, and I'm sure many of us don't much like this sort of thing. But it's bound to be more finely tuned to suit a given institution than the rough and ready REF could ever be, so whatever we do internally is likely to give most academics a fairer hearing.

Overall, therefore, the REF has an important job to do, so let's simply allow it to do that job well, without trying to give it additional tasks for which it was never designed. And in case the reader might wonder, I'm not a huge fan of the REF, but it's there, and is not about to go away. We just have to live with it - it's part of our university world nowadays!