Good Practice
Evaluating and Scoring Price – Questionable Advice from Crown Commercial Service?

We wrote here about the new guidance produced by the UK’s Cabinet Office around the use of the new Digital Outcomes and Specialists procurement framework. We admired much of the guidance material, although have some concerns about the practicality of running “mini-competitions” to choose the supplier when there may be a large number of capable suppliers who should be part of that final selection process – there are no less than 1200 suppliers on the list!

The guidance is very extensive covering the whole process, with an extensive section around evaluation. But it is in this area that we have another concern about the guidance. When it comes to evaluation criteria and evaluating price, the guidance recommends a route that many public sector organisations do follow (certainly in the UK) when it comes to converting a cost number into a “score”. That number can then be combined with “quality” type scores to arrive at a total mark for each supplier, which informs the selection decision.

The guidance says this (for fixed quotes).

Fixed quotes

To score fixed price quotes, you must divide the cheapest quote by each supplier’s quote.

Example:

  • supplier A’s quote is £15,000
  • supplier B’s quote is £10,000
  • supplier C’s quote is £30,000

To calculate a score for supplier A, divide 10,000 by 15,000. Supplier A scores 0.667.

To calculate a score for supplier B, divide 10,000 by 10,000. Supplier B scores 1.

To calculate a score for supplier C, divide 10,000 by 30,000. Supplier A scores 0.333.

The problem with this is that the methodology is severely flawed, to the point that we believe a legal challenge could well succeed – if a bidder went that far of course. We have written about this problem several times before.

The method is non-linear in the relationship between cost and the score awarded which is basically illogical form a value perspective. Indeed, the simple example given in the guidance above highlights this illogicality perfectly. Supplier A loses 0.333 points by being £5000 more expensive than the cheapest bid (supplier B). But supplier C only loses an additional 0.333 points for being a further £15,000 more expensive. So actually the method favours really expensive bids over slightly expensive! This all is certainly open to serious questioning if it ever came to court.

On a similar note, here is the clever example given by Peter Marshall of Qinetiq Commerce Decisions on our Spend Matters UK/Europe website last year. He shows how a selection decision between two firms can be changed if a third enters the evaluation – even if that third firm does not win, its mere presence changes the winner in his example from one supplier to another from supplier A to supplier B, which is clearly crazy. It is also illogical that the mark given to one bidder is dependent on that given to another – it means that we are not assessing each bid in its own right.  A “low-ball” bid can distort scoring for everyone else, for instance.

The problem is that this method is simple and easy to implement – other options are more complex or require a bit more effort. We won’t go into those alternatives today, but there is a problem here. We heard at a conference last year that this methodology is actually illegal (or specifically excluded in the national regulations) in Portugal. That may be going too far – but it is surprising to see such a dodgy methodology recommended by Crown Commercial Service. We’ve sent a copy of this article to the CCS policy head – we’ll let you know if we get any response.

Voices (2)

  1. hp chavaz says:

    That is a well-known problem that can arise anytime an offer(1) is used to define a scoring function.

    If a price scoring function is needed, a linear scoring function in which the slope is predetermined solves the problem.

    Another solution is to use the “corrected price” method in which the value taken into account for the ordering of the offers is in the form :
    price + VTV (1 – TVscore/TVMaxscore)
    with
    VTV = Valorization of the Technical Value
    TVscore = score for the Technical Value
    TVMaxscore = upper bound of the scoring range for the scores for the Technical Value

    By the way, at this time, the “corrected price” method is the only one which I find practical in case of use of the possibility opened n the article 46 of the directive of combining several lots.

    1) an offer or a set of offers, as the use of the mean or the median could but in much fewer cases introduce the same problem.

  2. Maurissa says:

    Good Day,

    Need a formula for tender price scoring when there is an engineer’s estimate.

    Some bids are much lower than the estimate and some are much higher in comparison to the estimate.

    Need urgent help

    I found a formula online – Maximum bib – bid price / maximum bid – optimal price

    the weighing is 20% but when I use this formula, the result if higher than 20.

Discuss this:

Your email address will not be published. Required fields are marked *