Skip to content

Procurement

Test every agentbefore you trust it.

Your vendor, your team and your adviser each recommend a different AI, and new frontier models keep arriving. Butterfly will test each one on about a hundred of your own held-back cases, and score what comes back.

Check an agent · £20kHow the score works
An example taskMatching a supplier invoice to its purchase orderPass mark 100 of 100 clean runs
The queue5 waiting

Next inYour finance system’s agent

The benchFive agents, one task

0 clean 0 failed

The podium
Each agent runs the same task 100 times.Example

01 · The choice

Everyone has an agent for you.Your own work decides.

  • Your vendors76%4

    of enterprise AI use cases are now purchased from vendors.

    A survey of 495 US enterprises in 2025. A year before, the share was 53%.

    Source: Menlo Ventures, 9 Dec 2025 4

  • Your own people78%25

    of people who use AI at work bring their own AI tools.

    31,000 people in 31 markets, asked from February to March 2024.

    Source: Microsoft / LinkedIn Work Trend Index, 8 May 2024 25

  • Frontier models

    The leaderboard winner lost on another test.

    A frontier model$9,235Its predecessor$11,18217

    A frontier model tops the Remote Labor Index, yet earned less than its predecessor running Vending-Bench 2’s simulated business: an average over 6 runs each.

    Source: Andon Labs, 24 Sep 2026 17

  • Token economics21,730 runs18

    In a majority of them, more reasoning effort meant less accuracy.

    Princeton’s study of 9 models from 2025. Paying for more thinking is not paying for more right answers.

    Source: HAL, arXiv, 13 Oct 2025 18

One side: yours. Butterfly has no agents of its own and takes no fees from agent vendors, so a score is about your work and nothing else.

Harder work needs more proof.

Agents take on harder, higher-stakes work every quarter, and that market keeps growing. Soon they may be able to do a company’s hardest work. But how many businesses would trust an AI with their sensitive processes? Readiness, tested on your own work, gives you real confidence and a return from AI you can plan on.

1 in 5

firms has mature governance for autonomous agents.

Source: Deloitte AI Institute, 2026 edition 3

02 · The score

How the score works.Six steps, every agent.

  1. 01

    Your task, as a working copy

    The job, its rules and its tools, rebuilt in your own cloud with synthetic records in place of real ones.

  2. 20 runs, 4 different answers
    02

    A hundred held-back cases, not one demo

    Ask a model the same thing twice and the answer can change: one test got 80 different answers from 1,000 identical requests.19

  3. Up to 31% could still fail61
    03

    Why the cases are held back

    Kept aside from the start, never used to build or tune the agent, so its score shows how it does on work it has not seen.

  4. 04

    A pass mark set with the task’s owner

    The task owner sets the bar. Every agent then gets one of three verdicts.

  5. 05

    The cost of a right answer

    The model bills for each run, plus the cost of fixing each failure. A cheap run that often fails is the dearest result.

  6. 06

    Tested again every week

    On the same tasks, the top score rose from 62.9% to 76.0% in four months. So each verdict would be tested again every week, and fully each quarter.14

03 · Vendor Agent Readiness Check

A £500k AI decision.£20k to check it first.

One agent, tested on your own task before you commit to it.

The AI decision
4%£20k to check a £500k decision before it goes out The decision The checkIllustrative
Vendor Agent Readiness CheckDraft for review · subject to contract

£20kper agent, fixed

Included in the Enterprise and Group plans.

  • Your task, rebuilt in your own cloud
  • About a hundred held-back cases, on each model and effort level
  • The score, the verdict and the cost of a right answer
  • Its access, and where a person stays in the loop
Check an agentEvery price

04 · Why it matters

Harder work needsmore proof.

Agents take on harder, higher-stakes work every quarter, and that market keeps growing. Soon they may be able to do a company’s hardest work. But how many businesses would trust an AI with their sensitive processes? Readiness, tested on your own work, gives you real confidence and a return from AI you can plan on.

The higher the stakes, the longer the testExample
  1. Drafting a reply a person checksfails less than 1 in 1029 clean runs
  2. Answering a customer directlyfails less than 1 in 50149 clean runs
  3. Paying out a claimfails less than 1 in 200598 clean runs
  4. Approving creditfails less than 1 in 1,0002,995 clean runs

Clean runs in a row to show it at 95% confidence.61

Your proposal

Get your proposal.Then register your interest.

A readiness report made from your company’s public website: what your tasks cost, and the first 25 to test. If it reads right, register your interest: a person from Butterfly will call you to talk it through.

We start reading your public website as you type.

Or read on:Agent benchmarksHow it works