The scorecard in one page
A useful vendor scorecard does two different jobs. First, it removes options that cannot meet a genuine obligation. Then it compares the remaining options against outcomes that matter to users, operations, security, finance and support. Mixing those jobs into one percentage lets a polished demonstration compensate for a failed requirement.
Begin with four to eight hard gates, such as an approved data location, required accessibility level, an essential integration, a viable exit path or evidence of a secure development practice. Score only vendors that pass those gates. Keep the weighted section short enough that reviewers can explain every criterion and show the evidence behind every mark.
The result is not a machine that chooses a supplier. It is a controlled record of why one option fits the agreed operating need better than another, what remains uncertain and which questions must be tested in a pilot. The score can guide a decision, but accountable people still own it.
| Decision area | Weight | Evidence to review |
|---|---|---|
| User and workflow fit | 25 | Task scenarios, accessibility evidence, role coverage |
| Technical and data fit | 20 | Architecture, APIs, data flows, portability |
| Security and resilience | 20 | Control evidence, incident process, recovery tests |
| Operating model | 15 | Administration, support, change and reporting |
| Commercial and exit fit | 15 | Comparable cost model, terms, export and transition |
| Supplier evidence | 5 | Ownership, provenance and supply-chain transparency |
Write the decision before the criteria
A scorecard built from vendor feature lists rewards whichever supplier publishes the longest catalogue. Write a decision statement first: the people affected, the tasks that must improve, the constraints that cannot move and the evidence that would justify change. This gives the team a stable reference when an impressive feature appears halfway through procurement.
Turn desired outcomes into observable scenarios. “Easy to use” is not a criterion. “A new team member can complete the priority task with the approved identity method and without local administrator help” can be demonstrated. “Good reporting” is vague. “The service owner can produce the agreed exception report without repairing exports in a spreadsheet” is testable.
The GOV.UK Technology Code of Practice is useful as a prompt because it connects user needs, accessibility, security, data, open standards, commercial choices and service operation. It is written for the UK public sector, so use the questions as a lens rather than treating it as a universal purchasing template.
Separate gates, scores and risks
A gate answers yes, no or not yet proven. A weighted criterion compares the degree of fit among viable options. A risk records exposure, likelihood, impact, owner and treatment. These three controls should appear separately even when the same subject touches all of them. Data export might be a gate, export effort a scored criterion and an undocumented legacy format a migration risk.
Use a small evidence scale instead of pretending that every reviewer can distinguish 72 from 74. A four-level scale works well: no usable evidence, claim only, demonstrated in a controlled scenario, or verified through an artefact or reference the team can inspect. Multiply that evidence-backed score by the agreed weight only after reviewers record their reasoning.
Do not convert unknowns into zero without explanation. Zero may wrongly imply failure, while a neutral midpoint may reward missing evidence. Mark the item as unproven, assign an owner and decide whether it blocks the shortlist, requires clarification or belongs in the pilot. The treatment of uncertainty matters more than decimal precision.
- Hard gate: a non-negotiable condition supported by an explicit reason.
- Weighted score: a comparable degree of fit among options that passed the gates.
- Risk: an exposure that needs an owner, treatment and review date.
- Unknown: missing evidence that receives a defined next action, not an invented assumption.
Ask for evidence that matches the risk
Supplier due diligence should be proportionate. A low-impact team tool does not need the same file as a platform that handles sensitive records or controls a critical workflow. Start by classifying business impact, data sensitivity, integration privilege, recoverability and dependence on the supplier. Increase scrutiny where failure would be hard to detect, contain or reverse.
Published in July 2026, NIST SP 1326 frames ICT supplier due diligence around ownership or influence, provenance, resilience, foundational cyber practices and supply-chain tiers. It is US guidance, but the categories expose questions that a feature comparison misses: who controls the supplier, what it depends on and how it continues through disruption.
For a higher-risk software purchase, ask how the producer builds, protects and responds to problems in the product. NIST SP 800-218, the Secure Software Development Framework provides a common set of practices. A statement of alignment is a starting point; the buyer still needs evidence appropriate to the decision and contract.
Make demonstrations comparable
Give every shortlisted vendor the same scenario pack and time boundary. Include a normal journey, an exception, an administration change, a reporting task and a recovery or exit question. Let suppliers explain a better route, but do not replace the buyer’s difficult scenarios with a rehearsed product tour. Record whether the result was shown, described or deferred.
Use realistic but non-sensitive data. The people who will operate and support the service should perform important tasks themselves. Observing a supplier click through an administrator account does not prove that a real employee, approver or service-desk analyst can complete the job with the permissions, accessibility tools and policy constraints that will apply.
Score immediately after each scenario, then calibrate as a group. Ask reviewers to cite the observation that changed their mark. If two people use the same criterion differently, improve the definition before the next demonstration. A repeatable scorecard is produced through calibration, not merely through a spreadsheet formula.
Compare full cost and exit, not licence price
Build one cost scenario for every vendor: the same users, locations, modules, environments, storage, integrations, support window and contract period. Separate subscription, implementation, migration, identity work, hardware, training, administration, support and expected change. Mark optional modules clearly so a low entry price cannot hide the cost of the required operating model.
Test exit before commitment. Ask for export formats, API limits, deletion evidence, transition assistance, retention after termination and the treatment of configuration, logs and derived data. A credible exit path is both a commercial control and an architectural requirement. If the organisation cannot explain how it would leave, it does not yet understand the dependency it is buying.
Do not force cost into a single score when the estimates have different confidence. Show a range, assumptions and excluded items beside the weighted fit result. The lowest calculated total may not be the lowest-risk choice, while a stronger operational fit does not justify an unlimited premium. Keep the trade-off visible for the accountable decision maker.
Use the scorecard through the pilot and contract
The shortlist is not the end of evaluation. Move the highest-risk claims into a bounded enterprise software pilot. Convert each important criterion into a scenario, measure and stop rule. Evidence from the pilot may change weights, expose a failed gate or show that a lower-ranked option works better in the real environment.
Carry the chosen criteria into acceptance and service review. If resilience, administration effort, accessibility or export quality mattered during selection, the contract and operating dashboard should make those expectations inspectable. Otherwise the evaluation becomes a procurement ceremony disconnected from the service people receive.
For a wider operating view, compare the site’s enterprise application pathway and four-stage decision process. The scorecard is strongest when it sits inside a clear decision, evidence and ownership system rather than acting as a standalone purchasing file.
The approval gate
Before approving a preferred vendor, confirm that every hard gate has current evidence, every material unknown has an owner, score differences are supported by observations and cost assumptions are comparable. Review conflicts of interest and ensure that no supplier received a different route through the process without a recorded reason.
Write a one-page decision record that states the chosen option, viable alternative, decisive evidence, unresolved risks, pilot conditions, exit assumptions and the date the evidence should be revisited. A future reviewer should be able to understand the decision without reconstructing it from meeting notes and private messages.
A defensible scorecard does not promise certainty. It makes trade-offs, evidence and uncertainty visible early enough to change the choice. That is the standard to use: not whether the sheet looks objective, but whether another responsible person can inspect the reasoning and reach a coherent conclusion.
