Regulaton & Policy
Procurement Challenge in Bristol Brings Evaluation Methodology Issues Into Focus

A recent legal case in the UK (a challenge to a procurement run by Bristol Council) has a number of interesting implications for contracting authorities potentially across Europe.

Bristol Missing Link Ltd (“BMLL”), the incumbent supplier, challenged the procurement of a significant contract for domestic violence and abuse support services in Bristol. The defendant, Bristol City Council wants to award the contract to another tenderer, Refuge. Because of the challenge, the Council had been stopped from entering into the proposed contract, but as often happens, had applied to the court to lift the ban and be allowed to place the contract. In many cases, the courts have allowed contract award to go ahead even if there might be further legal action in terms of the procurement process.

However, the judge refused and in this case said the matter must go on to a full hearing, and in the meantime the suspension of contract award would not be lifted. One interesting point was that as BMLL is a not for profit organisation, damages would not be an adequate remedy, even if the subsequent case went in their favour. There were other factors too which made him inclined to retain the suspension of contract award, such as complex TUPE issues if BMML lost the contract.

There was much comment by the judge on that matter of suspension, but we are more interested here about the fundamental procurement issues which appear to hinge around the tender evaluation, in particular the process by which the scoring of the tender was moderated and whether it was fair and transparent.

But before we get onto that, issues of disclosure are worth noting. The judge was not impressed in general with the amount of information that the Council had provided to BMLL or indeed to the court. Certainly, it seems odd that the contracting authority made no attempt to explain how that moderation process had worked.

There are other anomalies – what appeared to be a document showing the evaluation of the successful bid was made available, apparently by accident, on the Council’s website! But then the Council disclosed to the court a totally different and fuller score sheet to that one. The Council then made various assertions to the Court about how wonderful the winning bid was, without making the actual tender document available, which again did not impress the judge.

So one learning from this is that if you are challenged, you are not likely to get away with being highly selective in what you disclose and what you don’t. Hiding the documents that don’t support your argument may well be exposed or seen as the court as a negative.

But coming back to the scoring process. One key element within that process was the scoring, which was done by a panel of six people, then moderated to arrive at a single score. And that is where the problems seem to have started. As the judge’s decision says:

Thus, for example, we know that BMLL’s tender on sub-criteria 2.5 was the subject of scores from the individual evaluators of 4, 4, 4, 4, 4 and 3. These were subsequently moderated to an overall score of 3, but no explanation for that result has been provided. Similarly, BMLL’s tender for sub-criteria 2.7, with individual evaluation scores of 4, 4, 5, 4 and 3, was also given an overall score of 3, but again no explanation for that reduction has been provided”.

That, not surprisingly, worried the judge.

There were five evaluators, and a sixth person who was named and described as a procurement specialist, who was giving advice and guidance to the Council. Surprisingly, there is no evidence from her. The names of four of the five evaluators have been redacted … there is no evidence from them either. There is evidence from Ms Griffiths, the fifth evaluator, but her statement was short, and was only provided in reply to the evidence from Ms Metters. There has been no explanation as to why those actually involved in the process have not given evidence”.

I wonder who the “procurement specialist” was? I have played that role in public procurement projects over the years and it really is not good when it ends up in Court … I would say your number one objective as an adviser is to avoid challenge!

The judge stated that the principle of moderation was accepted as a legitimate element of public procurement. However, it was also legitimate for BMLL to question the way in which the moderation exercise was performed. which is where he came back to the question of documentation – “there was a vital moderation meeting about which the court has no evidence at all”.

From my personal experience, there are times when the individual markers do get it wrong. They may misinterpret the scoring methodology, or miss something in a particular response. That is why I prefer a moderation process to a simple averaging of the individual scores. But if you have evidence of those individual scores, and then a final score that is different, it needs to be documented to explain what happened. Where it leads to big changes in terms of the final marking versus individual markers’ scores, that needs to be explained and handled very carefully. It is not good enough to simply assert without reason that the final mark is “x”, if that is quite different from the mean or the median of the individual evaluators’ marks.

(I think we will write a separate article shortly with our advice on how to score and assess tenders when you have multiple evaluators).

BMLL also complained about the individual scores they were awarded for their responses to certain cases. The Council has replied to some of those points in correspondence, but this had not been pleaded in this court – so as the judge said, “Whilst, in order to succeed, BMLL will need to show that the errors were clear, that does not mean that the court can decide these points of detail without hearing at least some evidence”.

So that is another issue that will be heard in the subsequent court case, unless a settlement is reached in the meantime. And the fundamental question for that further hearing is this. Was Bristol basically fair but somewhat incompetent in the way it ran the procurement, and the lack of documentation of the evaluation process in particular? Or was it unfair, in that there was a desire to ensure BMLL did not win the contract, even though they may well have done so if the bids had been marked fairly? Was there some “reverse engineering” of the scoring to ensure another winner? That is the big question that the next hearing will have to unpick.

First Voice

  1. Peter Stuttard says:

    We should ask ourselves what is unusual here, the fact that this happened or that the victim of the allegedly unfair competition had the courage (or was desperate enough) to challenge it?