Consultancy projects - Points to watch out for! (2)

Continuing the theme of the previous post, here I comment on two other issues that arise in most reasonably challenging consultancy projects, namely information and judgement. As before, I treat my recent work in the Caribbean as a relevant case study.

For any consultancy assignment, good information is a critical success factor. When we started our work in St Kitts and Nevis (SKN), for instance, we were assisted by the provision of copies of various other consultancy reports and background papers. That was very useful. However, when we asked for more, our official contact person lacked the authority (and possibly the inclination) simply to say ‘Yes’ and had to consult more senior officers. On occasion, our requests resulted in a ‘No’ (as with the 2010 IMF Article IV consultation, which we were never permitted to see), or with a claim that the reports concerned still awaited ‘cabinet approval’; under further pressure, we were allowed to see executive summaries of the reports concerned. This is, frankly, an absurd way of treating highly experienced consultants, and I must say it took us by surprise. Hence the third lesson for successful consultancy:

Lesson 3. Make sure that all relevant reports, papers and other material needed to carry out the consultancy project to the required standard will indeed be made available (and preferably with the absolute minimum of hassle).

Later, when it appeared that the first version of our final report was insufficiently technical, and when we agreed to produce a more technical and analytical supplementary report, we asked for copies of various data-files to help us. We were advised that the files were ‘too big’ to send, and that some of what we asked for required approval from other ministries (presumably the Ministry of Finance or the Prime Minister’s Office). I’m afraid these days, such arguments look quite silly, for I have no doubt that everything we asked for could easily have been sent to us within a day, since none of it was especially complicated or problematic. Hence we had to do the best we could with the information we already had. Thus the fourth lesson:

Lesson 4. Ensure that all relevant (economic and financial) data - and these days that usually means, primarily, the relevant Excel files - is freely available to the consultancy project, without restriction.

An important aspect of most consultancy projects, where information is often assembled from a variety of sources of uneven quality, and where information is frequently incomplete or even unavailable, is that the consultants have to make judgements about various matters, and form conclusions on the basis of imperfect and fragmentary information. This is normal practice, an important part of consultants’ professional responsibility, and our project in SKN was no exception.

However, it turned out that on almost every occasion when we offered a judgement, the official SKN reaction was not ‘interesting point, we need to think about that’, but more along the lines of ‘how could you possibly know that, delete it from your report’. In other words, there was an extremely negative reaction to almost anything from us that sounded like a judgement. Aside from undermining our professional integrity, such an attitude implies that our SKN interlocutors were determined not to learn much, if anything, from all our work. To put it mildly, this was quite disappointing!

Let me briefly give some examples. Back in October when we first got GDP data on the country (1990 base), we already knew that a major rebasing exercise was going on, though we were not allowed to see any of the new figures until the exercise was published in mid-December. We found official estimates of the investment ratio at around 40% of GDP or even more, and I immediately commented that that could not be right. I discussed my view with ministry officials and with the Eastern Caribbean Central Bank, and all defended the existing figures. I said the figure should be 20-25%, and continued to question the official data in our draft reporting on SKN. My judgement, based on very elementary economic common sense, turned out to be correct.

Strangely, no one seemed remotely interested in how I had formed my judgement, whereas I saw a potentially useful part of our assignment as being to instil some ‘economic common sense’ into SKN officials. Surely we were there in part to teach people how to think about economic issues, but we weren’t allowed to.

We also commented on other contentious areas such as the government debt and the public enterprises (the latter including government departments producing utilities, public enterprises proper, and statutory bodies). In both of these areas we made many judgements and offered concrete policy advice, but in most cases where we made judgements the official reaction was as I indicated above, i.e. ‘how could we know; delete’. For the most part, I think what we had to say was correct and to the point, and would have benefited from open discussion and debate rather than simple rejection.

These remarks leads to my final, and in some ways most problematic lesson for consultancy projects:

Lesson 5. Establish right at the start that the consultancy project will involve making controversial and difficult judgements based on imperfect and incomplete information, and do everything possible to prepare the ground so that such judgements will not simply be dismissed out of hand.

Consultancy projects - Points to watch out for! (1)

As I mentioned in some previous posts, during the Autumn of last year I spent nearly three months out in the Caribbean working on an EU-funded technical assistance project. The project was based in the lovely islands of St Kitts and Nevis (SKN), ran from early October until just before Christmas, and drew on funds that formed part of the SKN Sugar Adaptation Fund. This fund was set up a few years ago, and it represents compensation from the EU for changes to the Sugar Protocol that made it less economic for SKN and other small states to produce sugar for export to the EU. In fact SKN took the decision to end sugar production back in 2005.

Working with a colleague who had long experience in the Caribbean, our task had nothing to do with sugar, however, and that's just as well as I have never pretended to be an agricultural economist. Rather, our remit was to study and advise on a macroeconomic framework for the islands, and deliver some training on that theme to officials from the relevant ministries. So for this project I had to pretend to be a macro-economist!

The picture shows the view from the house that we rented for the project, and where we did most of our work - yes, really.

While out in SKN, we wrote drafts of most of what we needed to do, but the final report was not completed and submitted to the St Kitts and Nevis government (through the consulting firm that hired us) until January this year. Then for a short time we ran into difficulties when the SKN authorities announced that our initial final report was unacceptable, and reported to the EU office in Barbados (which oversees the whole Eastern Caribbean sub-region) that they had been expecting something far more technical than what we had done.

This was rather strange as we (thought we) had made very clear to senior SKN officials how we proposed to approach the work, and why, and they seemed perfectly happy with what we planned to do. Our view was that in such a small country, with tiny technically qualified staffs in the key ministries, it would not be useful to set up a very technical macroeconomic framework - we thought no one would be able to use it effectively. So our first report was largely descriptive, underpinned by some economic analysis and supported by proposals about the organisational/ institutional changes that we thought would help in implementing the framework. But as noted above, we evidently misunderstood exactly what was wanted; this leads to:

Lesson 1. Make sure (really sure) you understand what the client wants.

Luckily, when these difficulties blew up we were already scheduled to spend a few days in Barbados (the picture shows a beach just north of Bridgetown), as a paper we had written in parallel to this project, on crisis and recovery in the Eastern Caribbean, had been accepted for an international conference in Bridgetown in late January (sponsored by the Central Bank of Barbados, the IMF and University of the West Indies). Our paper went down well, and we had time to meet the EU delegation in Bridgetown to discuss how to finalise our St Kitts and Nevis project. We agreed to write a more technical supplementary report, and this was delivered in February, finally accepted in March. So we got there in the end.

Part of the problem, I think - with the advantage of hindsight - is that we didn't manage to establish working relationships with key officials that were as trusting and positive as we would have liked. I don't fully understand what went wrong, though one thing that would have helped at the start would have been meetings with senior politicians - the Prime Minister and leading ministers relevant for our work. We failed to meet anyone at that level, and indeed were kept at arms length to a large extent by the middle level officials who were overseeing our work and (supposedly) helping us. This leads to the second lesson for consultancy projects:

Lesson 2. Establish positive and trusting working relationships at the most senior levels.

That's it for today; in my next post I shall offer some more lessons on the same theme.

What do Students Need to Learn?

This is a question I think about and wonder about quite often, partly because of the huge emphasis nowadays on the workplace skills that our students are supposed to need to help them succeed in the world of work (what we sometimes call, 'the real world'!) once they've graduated. The required attributes include well developed literacy, report-writing, and presentation skills, as well as softer skills to do with communication, team work, and the like.

Besides these undoubtedly useful practical skills - to which no one paid any attention whatsoever in the distant past when I was a student at university myself - I'm also quite keen to see our students learning something about the discipline I have always taught, namely economics. I like to think that when students leave us, they take with them both an interest in and a fairly extensive knowledge of economics, and even better, the ability to think like an economist. Probably I'm being over-optimistic to imagine that everyone we teach manages all that, but I think our better students do so.

However, a book I've been reading in the past couple of weeks has set me thinking afresh about this whole area. The book in question is Academically Adrift: Limited Learning on College Campuses (by Richard Arum and Josipa Roksa, University of Chicago Press, 2011). Through surveying a large sample of US college students, Arum and Roksa find that for many of them, distressingly little learning seems to take place, despite the fact that modules are mostly passed and normal academic progress is maintained. The survey showed that students prefer to get lots of pre-digested course handouts, and that they don't much like modules where the instructor expects them to do a lot of reading (yes, really!). And as soon as a module is done and passed, it's best forgotten so that attention can be focused on the next ones. Overall, the widespread student goal seems to be to secure the paper qualifications needed by the job market, without great concern to learn much along the way. Such an instrumental approach to university education is a bit worrying though not, I must admit, wholly unfamiliar on this side of the Atlantic either; and institutions apparently collude in their students' chosen approach because they are unwilling to devote more academic staff time and effort to the teaching function.

All this has led me to re-visit the modular structure of our courses. For students, this way of structuring our teaching offers a lot of flexibility, in that credits for modules already passed can often be transferred when a student wishes to shift to a different degree programme, or even to a different university.

But I do wonder how well the modular approach facilitates and supports the sort of learning I would like to see. Think of my own subject, economics, for instance (I could also comment on mathematics, but leave that for another occasion). It seems to me that we want our students to learn three things: (a) a body of knowledge about the subject; (b) the ability to apply that knowledge to new issues and problems as they arise, in other words problem solving; and (c) the ability to extend and develop their knowledge into new fields. Our modules do a decent job as far as (a) is concerned, are mixed as regards (b), and quite poor with (c) - though sometimes the experience of writing a dissertation can push a student into new areas of study and hence develop (c).

The trouble is that our modules are supposed to be both self-contained and, more importantly, complete in the sense that nothing can be examined that is not somewhere in the course material for that module. On the face of it that sounds totally reasonable, and fair to the students, but it's actually quite restrictive in my view. For example, in a highly practical subject like economics I think it is perfectly reasonable to tell the students that some questions in the exam will be drawn from topical issues (related to the theme of the given module) that have come up in the media - and there are usually lots of such issues. Doing that forces the students, especially the better ones, to read around the subject far more than they might otherwise do, and that is surely highly desirable. But these days we don't do that, since the general view seems to be that we cannot set an exam question on a topic that has not been covered in the lectures. Absurd!

And what if we want to set a question that draws on parts of micro- and macroeconomics in the same problem? That too is quite difficult under current arrangements and module structures, though it would be fine if we simply had final year modules in advanced economics, rather than treating micro and macro as separate. The point here is that many practical problems out there in the 'real world' do require bits and pieces of knowledge from several areas to be brought together, so it would be good if our students developed the facility to handle this (a mix of my (b) and (c) above). At present, our course structures tend to divide knowledge up into too many distinct pieces, and aren't too good at fostering integration across modules.

So when it comes to student learning, we still have some way to go before we get it right, assuming there is a 'right way' out there, waiting to be discovered!

After the Election, What Next?

So now we know. The election results up here in Scotland could hardly have delivered a clearer verdict. For not only does the SNP remain the largest party in terms of the number of MSPs it has in the Parliament, but it pulled in enough extra votes this time around - contrary to what most political pundits were expecting - to give it an overall majority. Hence we shall continue to have an SNP government in Scotland, but now there will be far less need for the governing party to reach accommodations with other parties in order to get its legislative programme implemented. The SNP government will be able to do pretty much what it wishes, subject only to the unpleasant and regrettable realities of budget constraints and the like............

Quite evidently, at some point during the new parliament there will be a referendum on Scottish independence. Strictly speaking, however, this can only be advisory/consultative, since constitutional matters remain among the powers reserved to Westminster - and Scottish independence is a constitutional matter par excellence. At this point, it's hard to say how such a referendum might go. Opinion polls are not currently favourable to independence, but there's plenty of time for that to change, and one assumes the SNP government will start to explain to the Scottish people what independence would mean, in terms of lots of nitty-gritty practical details not currently widely understood.

Leaving that aside, the new SNP government will soon need to turn its attention to the thorny question of funding the Scottish universities. In the run up to the election, the question was largely ducked, with the major parties (other than the Scottish Conservatives) coming out firmly against any form of student fees, whether payable up front or taking the form of a graduate contribution (paid once the student has graduated and earning more than some threshold). The claim was that somehow, the Scottish government would keep the universities adequately funded without the need for fees (except for English students, and non-EU foreign students).

Well, we'll soon see about that!

For one thing, commentators who have looked in detail at the figures find that as compared to English universities, the budgets of the Scottish ones will fall short by at least £93 million per year, with more recent estimates - based on the latest information on likely fee levels in England - being as high as £300 million per year. That's a lot of money, and at a time of public spending cuts and competing priorities, it's quite hard to see where it might come from.

So what is likely to happen? Well, I think there are several possibilities, which I simply list:

• An early U-turn by the SNP government, with a decision to introduce fees (I think this is not very likely);
• Muddling along, with no clear decision, slow erosion of funding to the universities, slow decline in system quality (I think this is the most probable outcome);
• Major restructuring to get system-wide costs down, with departmental closures in many institutions, job losses, possibly some mergers and perhaps even an institutional closure or two (this would all be politically quite uncomfortable, I imagine);
• An alternative form of restructuring that would end the traditional four-year degree in Scotland, shifting the first year either back to secondary schools or to FE colleges - both of which are cheaper. Such a reform would reduce the undergraduate population in Scottish universities by around a quarter, in the new steady state, making possible substantial staff savings and diverse other economies.

We do indeed live in interesting times - watch this space!