In the past few months I've been part of a group at Heriot-Watt University overseeing a small project - involving some outside consultants - to produce a report estimating the economic impact of the University. Where possible, this impact is being assessed for the local area, for Scotland, for the wider UK; and since we also have a campus in Dubai, some attempt is also made to estimate the economic impact over there.
Our report isn't quite finalised, but it's almost done, and the draft final report should not need much further change. And I think it tells a pretty good story about the direct and indirect impact of the University on local and national incomes and employment, these impacts coming through various channels. What the University does with the report once it's completed remains to be seen, and I'm not sure whether anything much will be placed on the University website. But I imagine it will be useful for the University to have some reasonably firm estimates to quantify our impact when it engages in discussions with funders, the government, and so on. So overall, probably a useful exercise, and certainly an interesting one to have been (a small) part of.
It seems that reports like ours have recently become quite the thing for universities to do, as several Scottish universities and universities elsewhere have commissioned such reports in the last two-three years. I imagine that like us, institutions are thinking that at a time when budgets are under pressure, we need all the arguments we can muster to justify and protect our spending, and a well written report with some good solid numbers in it can surely only help.
However, it struck me a day or so ago, that all these reports - and not at all through any fault of the various consultants - lack one rather important dimension of what universities are all about. This has to do with their research and contribution to knowledge. True, our report, and those of some other universities that I have read, do include research, but the impact of research is typically estimated via what we spend on it, supported by research grants and other diverse funding. What this means is that we're measuring research inputs, which is not too hard, but we're not measuring the outputs.
The problem is that measuring research output, especially across an entire institution containing numerous departments in different academic disciplines, is an absolute nightmare. Most research output takes the form of research papers published in academic journals or as book chapters, and since different disciplines have different styles of doing their research and reporting the results, the typical pattern of output is not at all the same in different departments. This is why the REF, and previously the RAE, effectively assess research across all UK universities by department (termed a Unit of Assessment). These exercises count publications, assess their quality, look at citations and other measures of impact, review PhD training and assess the research environment in each Unit. This is all quite complicated and time consuming so it is not done very often - hence the last Research Assessment Exercise was RAE2008, and the first run of the new Research Excellence Framework will be REF2014.
In terms of measuring research output, these exercises are a step in the right direction, since they are mostly measuring outputs in various ways, or attempting to do so. But how does this carefully measured research output all add up to a contribution to knowledge? And where is the time and space for creative thinking by academic staff in our universities? It seems to me that universities need to find a way of highlighting this aspect of their activities in a really positive way. For research is one of their core functions in society, and just measuring how much we spend on it, and how many jobs are associated with our research activity, seems fairly inadequate.
Perhaps the way forward would be for universities, periodically, to produce reports about these more intangible aspects of their research, their contribution to knowledge. If so, we need to think hard to figure out good ways of doing it! Such a report would have to show just what makes universities such special places, what makes them totally different from any other kind of organisation.
Our report isn't quite finalised, but it's almost done, and the draft final report should not need much further change. And I think it tells a pretty good story about the direct and indirect impact of the University on local and national incomes and employment, these impacts coming through various channels. What the University does with the report once it's completed remains to be seen, and I'm not sure whether anything much will be placed on the University website. But I imagine it will be useful for the University to have some reasonably firm estimates to quantify our impact when it engages in discussions with funders, the government, and so on. So overall, probably a useful exercise, and certainly an interesting one to have been (a small) part of.
It seems that reports like ours have recently become quite the thing for universities to do, as several Scottish universities and universities elsewhere have commissioned such reports in the last two-three years. I imagine that like us, institutions are thinking that at a time when budgets are under pressure, we need all the arguments we can muster to justify and protect our spending, and a well written report with some good solid numbers in it can surely only help.
However, it struck me a day or so ago, that all these reports - and not at all through any fault of the various consultants - lack one rather important dimension of what universities are all about. This has to do with their research and contribution to knowledge. True, our report, and those of some other universities that I have read, do include research, but the impact of research is typically estimated via what we spend on it, supported by research grants and other diverse funding. What this means is that we're measuring research inputs, which is not too hard, but we're not measuring the outputs.
The problem is that measuring research output, especially across an entire institution containing numerous departments in different academic disciplines, is an absolute nightmare. Most research output takes the form of research papers published in academic journals or as book chapters, and since different disciplines have different styles of doing their research and reporting the results, the typical pattern of output is not at all the same in different departments. This is why the REF, and previously the RAE, effectively assess research across all UK universities by department (termed a Unit of Assessment). These exercises count publications, assess their quality, look at citations and other measures of impact, review PhD training and assess the research environment in each Unit. This is all quite complicated and time consuming so it is not done very often - hence the last Research Assessment Exercise was RAE2008, and the first run of the new Research Excellence Framework will be REF2014.
In terms of measuring research output, these exercises are a step in the right direction, since they are mostly measuring outputs in various ways, or attempting to do so. But how does this carefully measured research output all add up to a contribution to knowledge? And where is the time and space for creative thinking by academic staff in our universities? It seems to me that universities need to find a way of highlighting this aspect of their activities in a really positive way. For research is one of their core functions in society, and just measuring how much we spend on it, and how many jobs are associated with our research activity, seems fairly inadequate.
Perhaps the way forward would be for universities, periodically, to produce reports about these more intangible aspects of their research, their contribution to knowledge. If so, we need to think hard to figure out good ways of doing it! Such a report would have to show just what makes universities such special places, what makes them totally different from any other kind of organisation.
No comments:
Post a Comment