DecisionOS — Decide · module

Vendors

Which of these options to choose — including building it ourselves and doing nothing — and what the choice actually rests on.

When you reach for it

Shortlist stage, when three vendors have presented and everyone has an opinion. Or when procurement demands a scoring matrix and the honest situation is that a preference already exists.

What it asks you

5 blocks, in this order. Each one exists for a stated reason, and the run tells you that reason as you answer it.

01
The options
Two to five named options. Two more are added automatically — building or extending what you already have, and doing nothing for now — because their absence is the most common defect in vendor selection and a module that asked whether to include them would be told no by exactly the runs that need them.
02
What matters, before you look at the options again
Allocate 100 points across the eight criteria. Allocation rather than sliders, because sliders let every criterion be important and a weight set in which nothing was traded off is not a weight set. This block comes before any option is scored, and before the options are shown again — weights set after seeing the options get fitted to the preferred option, and no amount of good faith prevents it.
03
Absolute requirements
Which of these are absolute requirements rather than preferences? Knockouts eliminate — they are not traded off. Separating gates from weights is the second most common defect in vendor scoring: a fatal flaw that is scored gets a low mark on one criterion and is then outweighed by seven others.
04
The evidence, option by option
Not a 1-to-5 rating. Each criterion offers three statements about what you have actually seen, and a fourth that says nobody has asked. The fourth is a real answer with a real consequence — a criterion nobody asked about for one option is withdrawn from the comparison for all of them, because scoring only where somebody asked rewards not asking.
05
One question about the exercise itself
Recorded, not judged. Most evaluations begin with a preference and a good many of them confirm a sound instinct. It is written into the method section because a reader in six months should know what the exercise started from — and because the stability check above means considerably more when they do.

The method

How the result is produced

Absolute requirements are applied first, as gates: an option that fails one is eliminated and is never scored, so a fatal flaw cannot be outweighed by seven other criteria. The survivors are then scored by weighted addition using your own allocation, elicited before any option was scored. Each criterion contributes weight × band ÷ 3, where the three bands are statements about what you have seen rather than a 1-to-5 rating. A criterion answered "we have not asked" for any option is withdrawn from the comparison for every option and its weight is redistributed across the rest, which is stated rather than done silently. Stability is then tested by moving each criterion's weight in turn to half, three-quarters, one and a quarter and one and a half of what you gave it, renormalising the rest, and recomputing the leader — 32 recomputations for eight criteria, always the same 32, with no sampling and no randomness. The differentiators are the criteria carrying the leader's advantage over the runner-up, smallest set covering 70% of it.

What this is not

This is not a vendor assessment. Nothing here was checked with the vendor, and no claim any vendor made was verified. Every band is a statement about what you have seen.

This is not a benchmark. There is no comparison to what other organisations chose, or to a market view of these vendors, and none is implied.

This is not a due-diligence report. It tells you which questions have not been answered; it does not answer them.

This does not tell you whether the leading option is any good. It tells you which of the options in front of you best fits weights you supplied — a comparison, on a shortlist somebody else assembled.

Where it stops

Everything here is what you told us. Nothing was read, inspected, profiled or connected to.

There is no peer comparison behind any statement in this report, and none is implied. We will not show one until there is enough real data to state an n.

This run reflects the answers of whoever completed it. A second respondent answering the same questions frequently disagrees, and that disagreement is usually the finding.

The weights are yours and the result is only as good as they are. That is the point of eliciting them before the options were scored, and it is not a defence against having allocated them carelessly.

The shortlist is whatever was brought to this run. Whether the right options are on it cannot be established from inside it, and the two the module adds are a partial correction rather than a complete one.

Stability is tested against movement in the weights, not against error in the bands. An option scored a band too generously on one criterion is not detected here.

Withdrawing a criterion is symmetric — it leaves every option's score — but it still helps whichever option was weakest on it, so declining to answer where you are weak shifts the comparison in your favour. There is no arrangement of this arithmetic that removes the effect: assigning "we have not asked" a score invents one, and scoring only the options somebody asked about penalises the option that was actually examined. What the module does instead is refuse to let it happen quietly — every withdrawal is reported with the option and the weight involved, and it lowers the confidence grade.

Where a preferred option was declared, the report says so. It does not attempt to correct for it, and no scoring method can.