Research ethics

From time to time we can read scandalous stories about research.

Someone has falsified their data, or their survey findings, to claim a result that is not really supported by the evidence. Quite properly, we are shocked when such stories crop up. They clearly do not represent the ethical, or honourable way to do research. On the other hand, given the pressures that exist in universities and research centres to get results and to deliver highly rated publications, it's easy to see why and how these things can happen. That's not at all to defend unethical research, of course, since that doesn't contribute to the generation of new knowledge and understanding of the world - which is what good research is surely all about.

But it does mean that institutions that deliberately put a great deal of pressure on their researchers to 'perform' and 'deliver' must always be on their guard to watch out for this sort of distortion. Moreover, undue pressure on researchers can sometimes lead not just to deliberate falsification, but also to honest mistakes committed in the rush to deliver results.

Such concerns about the quality and reliability of research are probably among the factors behind the increasing tendency for universities to insist that all research must be subject to some sort of ethical oversight, and for funding bodies to require a section on research ethics in their funding application forms. However, most of the new research ethics 'machinery' is actually not so much concerned with the aspect of research referred to above, namely its basic honesty and integrity, since this is still largely taken for granted. And anyway, how could the honesty of research be checked? I suppose the system still relies on peer pressure and the replication of experiments to highlight areas where some falsification or other distortion has occurred, though this is inevitably rather hit-and-miss. It also relies on occasional heavy penalties, like the loss of one's job, to provide negative incentives!

More mundanely, though still importantly, most of the new ethical machinery, at least in the social sciences, is more designed to ensure that where research might involve human subjects, the research is done in a way that involves voluntary and informed participation, without any pressure being placed on potential 'subjects' to take part. Within my own School, for instance, the ethical guidelines for research were recently updated, and it's clear that how we do research involving human subjects was a paramount concern.

This makes good sense, though it did seem to me that we were running a risk of being overly heavy handed in our approach. For the guidelines state that all the research we do must be subject to official ethical oversight, not only that where human subjects are or might be involved. This means that our ethics committee should, in principle, get to know about every single bit of research that goes on in the School. I would be surprised if the system works that well in practice, for I suspect that through a mix of ignorance and common sense, many of us will just get on with our perfectly respectable research without worrying about the ethical dimension - if we make a judgement at all, it will be that the research does not give rise to ethical issues. Now, I know that according to the rules we are not supposed to be the judges in our own cases, but I bet we often are.

For instance, a recent paper I wrote on North Korea (now published) was based on diverse secondary sources plus some of my earlier work on institutions related to transition to a market-type economy in Central and Eastern Europe. I didn't perform a survey, nor did I interview anyone, so although I didn't seek ethical approval for the work (and to be truthful, the idea never crossed my mind) it's hard to see what would have been done differently had I done so. And a good deal of research is like this.

Hence bringing ethical considerations into our research can be highly desirable if it helps to make us more careful and honest in what we do, and if it promotes good practice (as with the treatment of research involving human subjects), but we probably shouldn't push it too far. Nor should we make it overly bureaucratic.

What makes universities special places?

In the past few months I've been part of a group at Heriot-Watt University overseeing a small project - involving some outside consultants - to produce a report estimating the economic impact of the University. Where possible, this impact is being assessed for the local area, for Scotland, for the wider UK; and since we also have a campus in Dubai, some attempt is also made to estimate the economic impact over there.

Our report isn't quite finalised, but it's almost done, and the draft final report should not need much further change. And I think it tells a pretty good story about the direct and indirect impact of the University on local and national incomes and employment, these impacts coming through various channels. What the University does with the report once it's completed remains to be seen, and I'm not sure whether anything much will be placed on the University website. But I imagine it will be useful for the University to have some reasonably firm estimates to quantify our impact when it engages in discussions with funders, the government, and so on. So overall, probably a useful exercise, and certainly an interesting one to have been (a small) part of.

It seems that reports like ours have recently become quite the thing for universities to do, as several Scottish universities and universities elsewhere have commissioned such reports in the last two-three years. I imagine that like us, institutions are thinking that at a time when budgets are under pressure, we need all the arguments we can muster to justify and protect our spending, and a well written report with some good solid numbers in it can surely only help.

However, it struck me a day or so ago, that all these reports - and not at all through any fault of the various consultants - lack one rather important dimension of what universities are all about. This has to do with their research and contribution to knowledge.  True, our report, and those of some other universities that I have read, do include research, but the impact of research is typically estimated via what we spend on it, supported by research grants and other diverse funding. What this means is that we're measuring research inputs, which is not too hard, but we're not measuring the outputs.

The problem is that measuring research output, especially across an entire institution containing numerous departments in different academic disciplines, is an absolute nightmare. Most research output takes the form of research papers published in academic journals or as book chapters, and since different disciplines have different styles of doing their research and reporting the results, the typical pattern of output is not at all the same in different departments. This is why the REF, and previously the RAE, effectively assess research across all UK universities by department (termed a Unit of Assessment). These exercises count publications, assess their quality, look at citations and other measures of impact, review PhD training and assess the research environment in each Unit. This is all quite complicated and time consuming so it is not done very often - hence the last Research Assessment Exercise was RAE2008, and the first run of the new Research Excellence Framework will be REF2014.

In terms of measuring research output, these exercises are a step in the right direction, since they are mostly measuring outputs in various ways, or attempting to do so. But how does this carefully measured research output all add up to a contribution to knowledge? And where is the time and space for creative thinking by academic staff in our universities? It seems to me that universities need to find a way of highlighting this aspect of their activities in a really positive way. For research is one of their core functions in society, and just measuring how much we spend on it, and how many jobs are associated with our research activity, seems fairly inadequate.

Perhaps the way forward would be for universities, periodically, to produce reports about these more intangible aspects of their research, their contribution to knowledge. If so, we need to think hard to figure out good ways of doing it! Such a report would have to show just what makes universities such special places, what makes them totally different from any other kind of organisation.

How do we measure an individual's research impact?

This is a subject that I thought I already understood fairly well, having expressed on this blog and elsewhere various sceptical views about the whole notion of impact, including some of the commentary about it that has come out in connection with REF discussions and the like.

Some of the debate about impact has certainly been pitched at a fairly low level, and has often not taken us a great deal further than the well worn territory of either assessing 'research quality' - of an individual or a department (or, more formally, a unit of assessment) - in quite subjective terms; or making the assessment on the basis of publication numbers, journal rankings and citation counts. Not only has this not advanced matters much beyond the old RAE system, but it is open to all the familiar arguments: how many articles to count; how far back to go; how to judge the quality of an individual article, etc. Like a lot of important things in life, research quality seems to be one of these things where 'we know it when we see it'.

However, a few days ago I was intrigued to find that I didn't really understand this topic of measuring research impact as well as I thought. It happened like this. I had been approached by a couple of academic bodies in Italy to help them with two tasks: (i) assessing the research quality of individual academics, by reading and ranking a selection of their papers; (ii) assisting Italian universities to make new appointments at professorial level. I presume my name was put forward by a colleague in Italy, but in order to take things forward the Italian academic bodies naturally asked me to provide my CV and to complete a couple of simple forms. Whether, in the end, the Italians ask me to do these academic tasks, remains to be seen, but completing their forms was unexpectedly challenging.

For one of the forms asked me for the h-index and g-index of my research, something that left me totally baffled as I've never heard of them. Just to deal with the forms I did a quick Google search using the string 'Paul Hare h-index'. To my surprise this came back with h = 5 and g = 7. I still had no idea what they meant so I just put them on the forms and sent them off as requested.

Once that was done, though, I had to investigate further.

It then turned out that the h-index originated in a paper by Hirsch published in a physics journal in 2005. His idea is a really simple one. If a researcher has an h-index of 22, say, it means they have 22 papers with 22 or more citations. The h-index is the maximum number of a researcher's papers where this property holds. Thus someone might have published 80 or so papers, but most will only have a handful of, or even no citations. With 80 papers, the h-index could turn out to be 25, if that researcher - among all his/her output - has 25 papers with at least 25 citations (and not 26 papers with at least 26 citations).

The g-index, by the way, is a modification of the h-index to weight more heavily the articles that gain more citations.

What you realise when you think about all this is that anyone's measure of h (or g) will depend on the specific database that is examined - what papers are included in it, what citations are counted, etc. This means it's not easy to get a wholly reliable measure of h, because available research publication and citation databases are nowhere near good enough. Whatever database is used, for any given researcher their h value should rise over time, implying that it's probably not a great indicator for early-career researchers. Moreover, if different databases are used to investigate a given individual, one can expect to find different h values. All one can say, then, is that the person's true h-value is at least as large as the largest value one manages to find.

To illustrate the effect of using an alternative database, I downloaded an amusing bit of free software called Publish or Perish (Harzing, A.W., 2007, Publish or Perish, available at this link). This searches through the Google Scholar database, and using this approach I found the following values for my research: h = 20 and g = 33. I still don't really know whether these values are good, bad or indifferent, as I haven't checked the corresponding values of any colleagues and haven't seen these research performance indicators reported before.

Should we try to use indicators like this in the forthcoming REF? It seems to me worth thinking about, despite the practical difficulties I have mentioned above.

More on publishing

Academics, even retired ones, are supposed to keep up a steady pace of publishing their research output, though in my case it can't really matter very much as I don't expect to be part of the forthcoming REF (Research Excellence Framework), for which the data are due to be collected during 2013, and the results published in late 2014.

Anyway, regardless of the REF, I guess I've been doing my bit recently. At the weekend I was proof reading a paper on North Korea that I thought was due to be published in the International Economic Journal early next year. It now seems the paper will come out in September this year, in just two months' time.

And a few days ago, my co-editor (Gerard Turley) and I finally submitted to Routledge our edited volume, Handbook of the Economics and Political Economy of Transition, which should be published in March 2013; see the publisher's web-pages for this book. We've been working on this volume for just over two years. It has involved us working out a structure for the volume, then commissioning a large number of contributions from experts in the field, mostly academics, but a few who have also served as ministers in or advisors to various Eastern European governments. This process went surprisingly well, with the submitted volume comprising our editors' introduction plus 37 contributions. Clearly it'll be a large book, so anyone wishing to buy it should get saving up right away.

The submission process itself was quite novel for me, as it was the first time I've submitted a book entirely electronically. In fact as co-editor I've never even printed out the whole thing, so I guess we've saved a tree or two. To make their life easier, the publisher wanted the text of each chapter in a separate file, with each table and figure also in a separate file. Hence our submission amounted to 186 files in all, bundled up in a single, large zipped file. Needless to say, for this to work, we had to be really careful with our file names.

The next step, I imagine, will be a steady stream of queries from Routledge as they work through the files and prepare everything for publication, but our main work as editors is done, and these final stages should not be too hard to fit in with whatever else we are doing. It's been an amazingly interesting project to work on, and I'm really impressed that so many contributors delivered their contributions to a very high standard, and more or less on time. I can say that now, though I must admit that the final weeks were marked by a certain amount of anxiety as we awaited the last few contributions.

Research Excellence Framework 2014

After the publication last week of the Guidelines for the operation of the Research Excellence Framework (REF), we now know a lot more about how REF 2014 is expected to work, including the detailed schedule. The Guidelines have been published by HEFCE, acting on behalf of all four HE funding bodies operating across the UK, and can be found in full at this link.

As was announced by the Government a while ago, the schedule has slipped back a year from the original plans - it was going to be REF 2013. On the new plans, institutions will be invited to make their submissions to REF 2014 in January 2013, with submissions closing on November 29th 2013. The results will be published in December 2014, so a whole year has been allowed for the review of submissions across the 36 units of assessment (UoA).

For each UoA, the evaluation of research will be based on three elements. These are output, impact and environment. Research output will account for 65% of the overall rating of each UoA, research impact 20%, and the research environment the remaining 15%.

As with the previous Research Assessment Exercise (RAE), each researcher included in a given UoA will be able to submit up to four publications for assessment, their quality being judged in relation to relevant international standards, and for their 'originality, significance and rigour' (p6). Apparently some panels might choose to use citation data to support their judgements, though when this idea was first proposed - for universal use - it was shot down in flames. There are huge differences between disciplines in citation practices, for some disciplines there is hardly any published data, and citations are subject to well known distortions - such as self-citation, citation of one's friends, etc. It's not as objective, consistent and reliable as people initially assumed.

Research impact has been the most controversial measure, and is still widely debated - and criticized. The aim, though, will be for the impact of each UoA on the economy, society, culture to be judged. To me, this sounds an absolute nightmare, with assessments highly vulnerable to subjectivity, prevailing fashions in research, and the like. But hopefully my skepticism will prove to be unfounded.

Judging the research environment is a bit less problematic, I think, except that there seems to me to be a built in bias towards large size, in that a larger UoA with lots of staff and PhD students will have an easier time showing that it possesses the required attributes of 'vitality and sustainability' (p6). Yet these days, with collaboration across department in different institutions, and much easier research networking through the internet, I would have thought that the scale of a Unit should matter far less than it used to. One almost gets the feeling that those who have designed the REF haven't quite caught up with the huge benefits we get from IT and electronic communication to support our research. But you never know, the assessors might find ways to take all this into account.

REF 2014 will be a big and costly exercise, so it's worth asking what the results will be used for - apart from being highlighted on institutional websites when they come out well!

Three aims are featured in recent HEFCE publications about REF 2014. These are:

• To provide the basis for the selective allocation of research money to individual higher education institutions (in the past, this was the QR stream of funding; I don't know if it will continue to be denoted this way);
• To give the sector a means of benchmarking its research standards and providing a measurable basis for research reputation;
• Given that much research is supported by public funding, either from the Funding Councils of the Research Councils, there is a perceived need for public accountability and the results of REF 2014 are expected to show both how public money has been used, and to demonstrate value for money.

It's probably quite handy to collect just one set of data to fulfill these three rather different aims, but it's hard not to wonder whether the data we shall have can really serve such distinct purposes. I need to think about this a bit more, and discuss the area again in later posts.

A New EU Package - Buying Time!

Well, rather to my surprise - and relief - European leaders managed yesterday in Brussels to cobble together a new financial deal that provides further support to the ailing Greek economy. The new package comprises Euros 109 billion of official financing, with private sector involvement expected to add a further Euros 39 billion (this part is dependent on bondholder agreement, not yet assured). The interest rates payable on much of Greece's debt are being reduced and maturities extended, all of which takes a bit of pressure off the Greek 'sovereign'. This is a big step forward and the result will help to bring down total Greek debt and provide the country with a breathing space to put through various reforms.

However, the new deal leaves some important issues and questions for later consideration. These include the following:

• Will the deal restore Greece to solvency? Probably not, so there will be further rounds of discussion further down the line. Rating agencies might already treat the present deal as a 'selective default', but it's probably not the end of the story.
• Some funds are expected to be raised through a wide-ranging privatisation of Greek assets, with figures as high as Euros 50 billion being mentioned. But it would be unwise to put much weight on this as privatisation takes time and should not be rushed for the sake of quick sales (probably at fire-sale prices).
• The current deal does nothing to stimulate Greek economic growth, though in due course that remains the key to improving Greece's situation.
• Greece remains in the Eurozone,which constrains the policy actions it can take to improve its competitiveness. It remains to be seen whether this will be viable for long. On the other hand, exit would clearly be messy and difficult. It is not an easy option.

For the Eurozone more widely, the leaders agreed to make the use of the EFSF (European Financial Stability Facility), a fund of Euros 450 billion, rather more flexible, but there has not (yet) been an agreement to increase the size of the fund - some economists argue that it needs to be of the order of Euros 2 trillion to be really credible, and I agree.

Moreover, a declaration by the Eurozone leaders, released at the time of the summit, has some worrying features, in particular:

'All euro area Member States will adhere strictly to the agreed fiscal targets, improve competitiveness and address macro-economic imbalances. Deficits in all countries except those under a programme will be brought below 3% by 2013 at the latest.'

The history of member states adhering to fiscal targets is not a happy one, and there are no penalties for failure to comply. More importantly, in a European economy still struggling to recover from recession, it's not obvious that these fiscal targets are very sensible ones, especially in a context where the European Central Bank has already started to push up interest rates. Sure, deficits and debts need to be managed, but it's surely not urgent to achieve stringent targets quite so rapidly.

So, some welcome progress in Europe, which buys some time. But still no sign of an agreement on the debt limit in the US, so that remains a worry.

Modelling university behaviour and choices

Earlier this week I was reading a very interesting theoretical economics paper on the incentives and constraints that influence universities' choices between teaching and research. The paper is by John Beath, Joanna Poyago-Theotoky and David Ulph and it is entitled: 'University Funding Systems: Impact on Research and Teaching'. The paper is available as a Discussion Paper at the Economics e-Journal, and you can download it using this link.

You might imagine that theory has no place in thinking about our universities, or that if it does it should not be economics, but perhaps sociology, political science, anthropology, etc. But it seems to me that this is a big mistake, not because I believe economic theory has all the answers, but because I find that building an economic model, even quite a simple one as in this paper, often helps to illuminate important questions, and sometimes in quite unexpected ways. It is always easy to criticise a model by claiming that this or that assumption is unrealistic, and in a strict formal sense such criticism is nearly always right. But in my view, it is also nearly always irrelevant, because it misunderstands how models work, why we build them, and what we think we can learn from them.

The present paper illustrates these points quite nicely. The model makes a variety of assumptions, some of which clearly relate to core aspects of what universities do and how they function, while other assumptions are more technical and are made to ensure analytical tractability. After all, there isn't great value in a highly complicated model that is considered to be 'correct' in some sense if no one can solve it or derive any interesting conclusions from it. The first type of assumption includes the idea that universities engage in teaching and research, and they have preferences over these two 'outputs' or 'products'. Where the preferences come from remains unexamined, but different universities in the system can place different weights on research as against teaching. The ideas that universities need to produce teaching of at least some minimum quality level, and that quality is costly in terms of academic staff time, are other assumptions built into the model. Achieving research quality is also costly, institutions face a budget constraint implying that there are real tradeoffs between teaching and research, and individual academics have a time budget - if they spend more time on research, less is available for teaching, and conversely.

Most of this seems to me highly reasonable, at least as a useful starting point for analysing universities. Of course, one might want to extend it by modelling aspects of system-wide competition, or by having multiple disciplines (let's say, at least two, to start with the simplest case) in each institution competing for resources. But even the present paper, with reasonable values for the main parameters, reaches some intriguing conclusions. Most notably, it finds that universities are likely to divide into two groups, one of which opts for a very research intensive strategy (the research elite), the other consists of institutions placing greater weight on teaching - they may still do some research but it is not such a central part of their portfolios. Since this is not a terrible approximation to UK reality, I thought it was interesting to see it drop out of a model with such a nice, intuitively appealing and simple structure.

Building a Smarter Future (1)

The heading of this post comes from the Scottish Government's Green - consultation - Paper on higher education in Scotland, published in December 2010 and inviting comments in the next couple of months. The whole process has been set up with a view to arriving at specific proposals about Scottish higher education for whatever government we have in Scotland after the May 2011 elections to consider. If all goes according to plan, any new arrangements are supposed to come into effect from academic year 2012-13, so there isn't a great deal of time to reach agreement, pass the needed legislation, and prepare for implementation.

In this post I want to comment specifically on the section of the Green Paper dealing with research, while in the next one I shall comment on proposed funding arrangements for Scottish universities. There are lots of other topics covered in the Green Paper, and I would encourage all readers to get their own copy by using this link. In addition, the Scottish Government has set up a website to facilitate discussion about the issues raised in the Green Paper; again, just go to this link.

General remarks

There are three general points I would make before talking about my main topic of research.

(1) Given the amount of discussion about funding research and funding students across the UK that has already taken place, it seems a little surprising for Scotland to be embarking on this consultation exercise with a virtually clean slate, apparently open to any ideas that might come forward - except for the taboo topic of tuition fees. The whole process has the feel of re-inventing the wheel, and I only hope we don't come up with a square one just for the sake of being different from England! It's hard not to think that at least in part, what is going on in Scotland is a delaying tactic to ensure that no controversial decisions about the Scottish universities are taken prior to the May election.

(2) A term used widely in the Green Paper is 'learner journey', referring to the diverse pathways taken by students from application to a university through to getting a degree. To me, the term has a somewhat patronising feel to it, though I dare say that was never intended.

(3) The Paper is clearly aiming to come up with what it calls a 'Scottish Solution' to the conundrum of financing and organising Scottish higher education in these times of falling budgets and limited resources. Personally, I'm quite sceptical that there is anything uniquely Scottish to be discovered, so I await the outcome of this whole exercise with enormous interest.

Research

Research is one of the core activities of a good university, along with the teaching of undergraduate and postgraduate students. And in the best cases, the more advanced teaching ought to be able to draw on the research findings of academic staff, a process that often benefits both teaching and research.

Research in UK universities is funded through several channels - research grants for specific projects (from diverse sources, including the Research Councils); from the relevant funding council for research infrastructure; and also from the funding council as a result of the most recent research assessment exercise - the last of which was RAE2008. In the future, the RAE is to be replaced by the Research Excellence Framework (REF), but as presently envisaged, it too will be used to allocate part of the public money for universities on the basis of research performance.

For Scotland, the Green Paper (GP) asks some important questions about how research should be supported and funded. One quite fundamental one (GP, p17) asks whether we need to retain research in all of our universities. It seems to me that the Scottish Funding Council (SFC) might reasonably decide not to allocate any public research money to a given university, let us say, but this would in no sense prevent it from doing research. Individual academics would still be free to pursue their research interests, and could support them by winning research grants. So I would certainly want to see in Scotland a situation where all universities were free to undertake research, without necessarily insisting that all should receive public funding.

The GP (p17) also asks whether research funding should be more concentrated or focused on larger research groups. If our concern is to achieve the highest possible good quality research output from limited resources, this seems a poor way to achieve that, as there is very limited evidence that larger research groups are more effective, or more productive than smaller ones in most fields.

More controversially, GP (p18) raises the idea of focusing research funding on Government priorities, supposedly to 'have a greater impact on increasing sustainable economic growth'. This idea seems to me complete nonsense, not because I am against growth, but because I am absolutely certain that the Government has no idea what particular research would stimulate faster growth. No doubt there are lots of connections between research and growth, but they are terribly poorly understood. This relates a little to the whole 'impact' agenda that has become fashionable these days. Again, most researchers have no idea about the possible or likely impact of their work, though we can all make up stories if our research administrators tell us to do so. But really, it doesn't mean a great deal.

An aspect of GP that seems to me quite weak in the research chapter has to do with international aspects of research. True, this does get a mention on p18, but there ought to be far more to it. After all, Scotland is a small country, competing and operating internationally in numerous areas of research, and this international dimension must have a lot to do with Scottish universities' remarkable success in research. Our natural research partners are not necessarily other Scottish universities, or Scottish firms, but they will often lie much further afield - in the UK, Europe, or anywhere else in the world. Finding and nurturing these international links seems absolutely critical to Scotland's future as a nation that values the research that universities do, and I would hope that whatever Scottish Solution finally emerges later this year will include some very strong acknowledgement of, and support for all this. The last thing we want is a set of institutional and funding models that foster more parochial thinking in our universities. The whole world is out there for us to study and explore, and it's an exciting place.

Research degrees and supervision

This summer marks the end of a significant aspect of my academic career, as both my last research students have recently submitted their theses and await their oral examinations (more formally known as the viva voce, or viva).

One of these students,a Ukrainian, has been doing full-time research for her PhD in economics at my university for over three years and I have been her principal supervisor. We have met often, or exchanged e-mails, and I have read several drafts of most of her work; other colleagues have made big contributions to the supervision, especially for the more technical econometric part of the thesis which has never been my forte. The whole process has been hugely interesting, and I keep my fingers firmly crossed that she will manage the oral successfully - in fact my last stage in advising her will take place next week, when we meet to enable me to advise her what sorts of question to expect in the viva. Other than that, I am not of course involved at all in the examination process, in keeping with standard UK practice (this would be different in the US, for instance).

My other student is a Norwegian whom I have not yet met - with luck we shall finally meet on the day that he comes over to Edinburgh for his viva, and I look forward to that. He is a distance learning DBA (Doctor of Business Administration) student of the Edinburgh Business School (EBS), where I was working part-time until end-April this year. Despite formally ending my employment at EBS, however, I agreed with the School that I would continue supervising this particular student through to completion, since anything else would have been quite disruptive to the student concerned. He is already employed and works full-time at a Norwegian consulting firm, so for the most part his research has had to be fitted into spare time, including holidays, though his employer is also very supportive. The research is not in economics, but in risk management, a subject quite new to me, and about which I have learned a great deal in the past couple of years.

Supervision 'at a distance' is very different from what I have been used to, and has taken the form of a mix of e-mails and telephone conversations, the latter almost weekly at critical points of the research. Some of the correspondence has involved detailed commentary on draft chapters or longer chunks of the thesis. What is missing from this sort of supervision is the quick informal chat resulting from a chance meeting in the corridor, the sort of link that has been very useful with my on-campus students. But I think we have nevertheless managed to establish a good working relationship and mostly communicate quite well.

Another complication, though, is that because the DBA is quite a new 'product' of the Edinburgh Business School, some of the supervision is - in effect - conducted by committee, something I haven't encountered before and don't like very much. What happens is that a draft of all or part of the thesis is circulated to some or all of the DBA research committee, with comments coming back to the committee for discussion, then being cast as advice to go back to the student. With the best will in the world this is a problematic approach to supervision, since it's extremely hard to avoid mixed and confusing messages going back to the student concerned, though I've done my best to act as a 'filter' where I thought this would be useful. Officially, though, I'm probably not supposed to do that as this could be viewed as 'diluting' the committee's views. That said, my guess is that this fairly inefficient approach to supervision will be largely dropped once the DBA is more established and the School feels more confident that good standards of research and scholarship are being maintained.

So, my last two research students, and two very different styles of supervision. This makes me wonder how I learned to be a supervisor in the first place. But I started supervising PhD students back in the mid-1970s, only shortly after I finished my own PhD, and I have no recollection of receiving any formal training at all. As far as I remember, my first few students always had someone more senior as principal supervisor, so I guess I learned quite a lot from observing their practice - this sort of on-the-job learning was standard in those days, as there was very little proper training for anything then. Before long, I was encouraged to take on students as a principal supervisor in my own right, and also saw the other end of the process as I was increasingly invited to serve as a PhD examiner by various universities. Gradually, therefore, I picked up the elements of what supervision was all about.

Nowadays, the same learning process takes place but it is more formally structured, with more systematic training of new supervisors (to which I have contributed on occasion); clearer codes of practice for research students and supervisors so everyone knows what to expect and how to operate; and regular reviews of progress to identify emerging problems in good time (I don't recall any reviews taking place in my early years as a supervisor). All this makes the supervision of research students rather more professional than it sometimes was in the past, though I doubt whether good practice has really changed all that much. Supervising bright and imaginative research students has always been fun, and long may it remain so!

University funding - Research

UK universities all receive public funding from the various funding councils to support their teaching and research, as well as to promote a variety of other activities (such as measures to improve access; knowledge transfer; etc.). The teaching grant largely depends on approved student numbers and their subject mix, while the research grant (generally referred to as the QR funding stream) depends on the results of the most recent Research Assessment Exercise (RAE), specifically RAE2008 at the moment. In addition, of course, individuals and groups seek project-based funding from a wide variety of sources to support their research, so total funding for research is much larger than the QR stream alone.

In Scotland, the Scottish Funding Council (SFC) distributes QR funding to all the institutions that it funds, but there is also another channel for distributing research funds in Scotland; this is called research pooling. The idea is to encourage researchers in different Scottish universities to work together to form research partnerships or alliances, and such schemes have been developed in several subject areas - physics, chemistry, mathematics, economics, and a few others - with each involving a different grouping of institutional partners. Once SFC approves a particular subject-based scheme, it provides funding - usually over five years - to support new academic posts, network co-ordination and management, conferences and workshops, and so on. The aim is to give Scottish research a big boost by achieving critical mass in key research areas, bringing in new highly productive researchers, and where relevant, by building links with policy-makers and the business community. Put like this, it sounds a great idea. But is it really so wonderful?

A few years ago, when the idea of research pooling was first advanced, SFC issued a consultation paper on the subject. Strictly in my personal capacity (i.e. not as a spokesman for my university), I wrote to argue against the proposed policy. I made several points, of which the two key ones were these: (a) there is not much evidence to suggest that large research groups are more productive than small ones (except in a few cases where big equipment needs arise); and (b) even if partnerships were a great idea, it wasn't obvious to me that the best partners would necessarily be other Scottish universities. However, I clearly lost the argument, since the research pooling policy was adopted. At that stage, I naturally supported my own university's efforts to form research partnerships and hence do well from this funding opportunity. After all, once it has been decided what 'game' is to be played, it makes sense for each institution to play it as well as it can.

However, there is another important point here, based on the natural question, where does the money for research pooling come from? To my knowledge, the Scottish government did not increase its allocation to SFC to promote research pooling, so the money must have come from elsewhere in the existing budget. I presume, therefore, that the QR funding stream was top-sliced in some way to provide the money. What this means, though, is that research pooling does not represent any additional research money going into the Scottish universities, but rather a redistribution of funds that would have been paid out anyway. So to convince ourselves that research pooling is a good way of allocating a chunk of research money to Scottish universities, we must have a strong suspicion that merely allocating more through the established QR mechanism would have been less productive in terms of the likely research outcomes. But what reason do we have for believing that? Personally, I'm still struggling to make sense of all this.

Examining a PhD - Collegiate versus commercial models

Having examined dozens of PhD theses in many different universities, I had a surprising experience late last year. I was asked to examine a fairly technical economics PhD at another UK university - which will remain nameless to avoid any additional embarrassment. The thesis itself was pretty good, and the internal examiner was extremely thorough. In the end, after the formal viva, we agreed a few minor changes and corrections, and quite soon I heard that the student had done what was required and the thesis was approved. So far so good.

However, the overall examination process was not quite what I had expected, in that the department made no effort to look after me in the way departments normally do, e.g. by providing opportunities to meet other staff, renewing old acquaintances, and the like. My wife thought I was just complaining about the absence of a nice lunch - partly true but not wholly fair. So why was I making such a big deal out of this?

Well, as many of you will understand, examining a PhD is one of the many things we do that both consumes a lot of time (easily 2-3 days per PhD) and is hardly rewarded at all - in this case I was offered the minimal fee of just over £100. Thus such examining is part of what I have described before as the collegial aspect of university life; and it seems to me that when you're doing a favour for another institution, they should look after you properly. As I see it, this is part of the 'implicit contract'. The alternative is to examine a PhD as a commercial activity, where you simply do a piece of work for a fee with no expectation of any personal contact with the department concerned. But in that case, a more reasonable fee might have been around £1000. Thus the institution referred to above seemed to me to be operating a commercial model of the examination process, but they were doing so very much 'on the cheap'.

As soon as the viva was over, therefore, I wrote to the relevant Vice-Principal at the university in question to complain about the casual manner in which I had been treated, and sketching out some of the above points. After a short investigation, I was pleased to receive a comprehensive apology, and an assurance that external examiners would be much better treated in future. I have since also met some members of the department who took the opportunity to apologise individually. So all ended well.

This experience does, though, highlight an important issue. Much of what academics do, both for their own institution and for others, relies on goodwill and a continuing support for collegiate ideals of how universities ought to function. But this goodwill can be undermined in many ways - both by individual incentives and performance indicators that might oblige academics to prioritise their time differently from in the past; and, as in this example, by a basic lack of courtesy, kindness and appreciation of what we do. Yet I rather doubt whether many of us really want move to a purely commercial model for the provision of extra-institutional services, and I also doubt whether our institutions would like to pay the resulting high fees!