This is the page you get back. It uses our own test jobs, not a customer's.
Switch four jobs. Keep one where it is.
| Job | Runs a month | Today | Cheapest model that passed | After | Test | Answer |
|---|---|---|---|---|---|---|
| Monitoring every 15 min | 2,880 | $119.52 | gpt-oss-120b | $1.52 | 10 of 10 | Switch |
| Ticket tagging 50 batches a day | 1,500 | $63.14 | gpt-oss-120b | $0.69 | 10 of 10 | Switch |
| Status check every 30 min | 1,440 | $57.57 | DeepSeek V4 Pro | $2.65 | 10 of 10 | Switch |
| Daily briefing once a day | 300 | $12.24 | gpt-oss-120b | $0.12 | 10 of 10 | Switch |
| Inbox digest once a day | 150 | $6.06 | None passed every run | $6.06 | 9 of 10 and 7 of 10 | Keep |
| A month | $258.53 | $11.04 | You save $247.49 | |||
This job decides which emails need action. The two cheaper models passed 9 of 10 and 7 of 10 runs. A job only switches when a model passes every run, so this one stays on Claude Sonnet 5.5.
The cheaper models were slower: 6 to 8 seconds an answer, against about 2.4. That is fine for background jobs. It is not fine for anything a person waits on.
Each job ran 10 times on each cheaper model, across two hosts, with the same instructions and inputs. Code graded every answer. No model graded another model. In your report, the answer key is what your current model already said. Read the full test.
You change one setting per job. If the report finds less than $50 a month to save, it says so and tells you not to bother.
About this sample: five test jobs written by Bitdepth, short and without tool calls. It is a small test. Real jobs can pass at different rates, which is why your report runs on your own.