Consultancy projects - Points to watch out for! (2)

Continuing the theme of the previous post, here I comment on two other issues that arise in most reasonably challenging consultancy projects, namely information and judgement. As before, I treat my recent work in the Caribbean as a relevant case study.

For any consultancy assignment, good information is a critical success factor. When we started our work in St Kitts and Nevis (SKN), for instance, we were assisted by the provision of copies of various other consultancy reports and background papers. That was very useful. However, when we asked for more, our official contact person lacked the authority (and possibly the inclination) simply to say ‘Yes’ and had to consult more senior officers. On occasion, our requests resulted in a ‘No’ (as with the 2010 IMF Article IV consultation, which we were never permitted to see), or with a claim that the reports concerned still awaited ‘cabinet approval’; under further pressure, we were allowed to see executive summaries of the reports concerned. This is, frankly, an absurd way of treating highly experienced consultants, and I must say it took us by surprise. Hence the third lesson for successful consultancy:

Lesson 3. Make sure that all relevant reports, papers and other material needed to carry out the consultancy project to the required standard will indeed be made available (and preferably with the absolute minimum of hassle).

Later, when it appeared that the first version of our final report was insufficiently technical, and when we agreed to produce a more technical and analytical supplementary report, we asked for copies of various data-files to help us. We were advised that the files were ‘too big’ to send, and that some of what we asked for required approval from other ministries (presumably the Ministry of Finance or the Prime Minister’s Office). I’m afraid these days, such arguments look quite silly, for I have no doubt that everything we asked for could easily have been sent to us within a day, since none of it was especially complicated or problematic. Hence we had to do the best we could with the information we already had. Thus the fourth lesson:

Lesson 4. Ensure that all relevant (economic and financial) data - and these days that usually means, primarily, the relevant Excel files - is freely available to the consultancy project, without restriction.

An important aspect of most consultancy projects, where information is often assembled from a variety of sources of uneven quality, and where information is frequently incomplete or even unavailable, is that the consultants have to make judgements about various matters, and form conclusions on the basis of imperfect and fragmentary information. This is normal practice, an important part of consultants’ professional responsibility, and our project in SKN was no exception.

However, it turned out that on almost every occasion when we offered a judgement, the official SKN reaction was not ‘interesting point, we need to think about that’, but more along the lines of ‘how could you possibly know that, delete it from your report’. In other words, there was an extremely negative reaction to almost anything from us that sounded like a judgement. Aside from undermining our professional integrity, such an attitude implies that our SKN interlocutors were determined not to learn much, if anything, from all our work. To put it mildly, this was quite disappointing!

Let me briefly give some examples. Back in October when we first got GDP data on the country (1990 base), we already knew that a major rebasing exercise was going on, though we were not allowed to see any of the new figures until the exercise was published in mid-December. We found official estimates of the investment ratio at around 40% of GDP or even more, and I immediately commented that that could not be right. I discussed my view with ministry officials and with the Eastern Caribbean Central Bank, and all defended the existing figures. I said the figure should be 20-25%, and continued to question the official data in our draft reporting on SKN. My judgement, based on very elementary economic common sense, turned out to be correct.

Strangely, no one seemed remotely interested in how I had formed my judgement, whereas I saw a potentially useful part of our assignment as being to instil some ‘economic common sense’ into SKN officials. Surely we were there in part to teach people how to think about economic issues, but we weren’t allowed to.

We also commented on other contentious areas such as the government debt and the public enterprises (the latter including government departments producing utilities, public enterprises proper, and statutory bodies). In both of these areas we made many judgements and offered concrete policy advice, but in most cases where we made judgements the official reaction was as I indicated above, i.e. ‘how could we know; delete’. For the most part, I think what we had to say was correct and to the point, and would have benefited from open discussion and debate rather than simple rejection.

These remarks leads to my final, and in some ways most problematic lesson for consultancy projects:

Lesson 5. Establish right at the start that the consultancy project will involve making controversial and difficult judgements based on imperfect and incomplete information, and do everything possible to prepare the ground so that such judgements will not simply be dismissed out of hand.