Endpoint leaderboard

Who actually serves the model.

Every OpenRouter endpoint for each open-weight model, sent the same prompts at temperature zero every night, graded against a published reference, and ranked by what a correct answer costs.

Last run Nov 19, 2026
23 nights measured
185
endpoints graded
7
open-weight models
10
endpoints outside the reference
What the board found

Same model name, different answers.

DeepSeek V4.1 Flash: outside the reference

On DeepSeek V4.1 Flash, 5 of 61 endpoints sit outside the reference: baidu/fp8, open-inference/fp4, near-ai, ionstream/flex, reka. Ranked endpoints run 79 to 88% on exact match.

baidu/fp876.0%still listed
open-inference/fp473.2%still listed
near-ai74.9%still listed
ionstream/flex76.5%still listed
reka76.1%not listed since Nov 15

Exact match, mean over every night listed.

Cite: Bitdepth · bitdepth.dev/m/deepseek-v4.1-flash-1017 · as of Nov 19, 2026

venice/int4, night by night

On gpt-oss-120b, venice/int4 matched the reference on 95% of prompts over its first three nights and 81% over its last three. It now sits outside the reference.

Exact match with the reference each night: venice/int4 against the median of the other endpoints serving gpt-oss-120b.60708090100Oct 27Nov 8Nov 19median of the other 29 endpointsmedian of the other 29 endpointsmedian of the other 29 endpointsmedian of the other 29 endpointsvenice/int4venice/int4venice/int4venice/int4

Cite: Bitdepth · bitdepth.dev/charts · as of Nov 19, 2026

Cheapest real endpoint

Per model, the best price for the right answer.

DeepSeek V4.1 Flash

61 endpoints

On DeepSeek V4.1 Flash, the cheapest endpoint that matches the reference is particle-ai/us/flex at $0.25 per million output tokens. 5 of 61 endpoints sit outside the reference.

  1. 1particle-ai/us/flex$0.098
  2. 2deepinfra/fp8/flex$0.14
  3. 3deepinfra/fp8$0.17
Full table and charts →

DeepSeek V4 Pro

18 endpoints

On DeepSeek V4 Pro, the cheapest endpoint that matches the reference is relace/fp4 at $1.50 per million output tokens. 4 of 18 endpoints sit outside the reference.

  1. 1relace/fp4$1.61
  2. 2streamlake/fp8$1.71
  3. 3siliconflow/fp8$1.82
Full table and charts →

gpt-oss-120b

30 endpoints

On gpt-oss-120b, the cheapest endpoint that matches the reference is near-ai/fp4 at $0.15 per million output tokens. 1 of 30 endpoints sit outside the reference.

  1. 1near-ai/fp4$0.062
  2. 2coreweave/fp4$0.068
  3. 3dekallm/bf16$0.070
Full table and charts →

Mistral Large 4

2 endpoints

On Mistral Large 4, the cheapest endpoint that matches the reference is mistral/zdr at $2.10 per million output tokens. All 2 graded endpoints match it.

  1. 1mistral/zdr$1.05
  2. 2mistral/eu$1.16
Full table and charts →

GLM 5.3 Flash

48 endpoints

First look at GLM 5.3 Flash: 48 endpoints on OpenRouter, 6 nights measured. Grades start at 7 nights.

  1. First look, 6 nights. Ranked from 7.
Full table and charts →

Gemma 4 31B

21 endpoints

First look at Gemma 4 31B: 21 endpoints on OpenRouter, 6 nights measured. Grades start at 7 nights.

  1. First look, 6 nights. Ranked from 7.
Full table and charts →

MiMo V2.6 Pro

5 endpoints

First look at MiMo V2.6 Pro: 5 endpoints on OpenRouter, 6 nights measured. Grades start at 7 nights.

  1. First look, 6 nights. Ranked from 7.
Full table and charts →
The board

Every endpoint, named.

On DeepSeek V4.1 Flash, the cheapest endpoint that matches the reference is particle-ai/us/flex at $0.25 per million output tokens. 5 of 61 endpoints sit outside the reference.

#EndpointDeclaredExact matchSelf-agreeNightsDrift$ in / out per MQuality-adjustedState
1particle-ai/us/flexunknown86.3%83.5%15stable$0.030 / $0.25$0.098Matches reference
2deepinfra/fp8/flexfp884.9%83.6%15stable$0.078 / $0.24$0.14Matches reference
3deepinfra/fp8fp885.4%83.5%15stable$0.098 / $0.29$0.17Matches reference
4wafer/usmxfp486.6%80.9%15stable$0.036 / $0.50$0.18Matches reference
5alibabaunknown85.7%82.0%15stable$0.090 / $0.36$0.18Matches reference
6particle-ai/usunknown86.4%86.3%15stable$0.060 / $0.50$0.20Matches reference
7morph/fp8fp886.5%85.3%15stable$0.059 / $0.52$0.20Matches reference
8wafer/mxfp4mxfp485.5%80.4%15stable$0.066 / $0.50$0.20Matches reference
9stepfun/fp8fp885.5%94.1%15stable$0.10 / $0.42$0.21Matches reference
10streamlake/fp8fp882.5%85.8%15stable$0.11 / $0.44$0.24Matches reference
11primeintellectunknown86.3%90.3%15stable$0.044 / $0.70$0.24Matches reference
12relace/fp4fp485.8%92.4%15stable$0.12 / $0.50$0.25Matches reference
13inference-net/fp8fp885.6%81.9%15stable$0.080 / $0.64$0.26Matches reference
14cloudflareunknown87.7%83.5%15stable$0.10 / $0.60$0.26Matches reference
15inference-net/usfp885.2%81.3%15stable$0.080 / $0.64$0.26Matches reference
16deepinfra/fp8/priorityfp885.8%83.9%15stable$0.15 / $0.44$0.26Matches reference
17riverunknown80.7%93.6%8stable$0.12 / $0.48$0.26Matches reference
18dekallmunknown85.1%81.9%15stable$0.10 / $0.60$0.26Matches reference
19cohere/fp8fp887.8%87.5%12stable$0.15 / $0.50$0.27Matches reference
20siliconflow/fp8fp885.9%89.6%15stable$0.15 / $0.60$0.31Matches reference
21novita/fp8fp885.8%84.5%15stable$0.17 / $0.66$0.34Matches reference
22modelrun/fp4fp484.9%91.5%9stable$0.17 / $0.67$0.35Matches reference
23makora/fp8fp886.2%84.4%15stable$0.15 / $0.80$0.36Matches reference
24gmicloud/fp8fp883.4%91.4%15stable$0.18 / $0.72$0.38Matches reference
25ionstreamunknown79.3%85.5%15stable$0.21 / $0.62$0.39Matches reference
26digitaloceanunknown79.0%74.4%15improving$0.18 / $0.72$0.40Matches reference
27atlas-cloud/fp8fp885.3%84.3%15stable$0.20 / $0.78$0.40Matches reference
28fireworks/spotunknown85.6%81.8%15stable$0.21 / $0.84$0.43Matches reference
29primalabsunknown85.3%89.6%14stable$0.24 / $0.96$0.49Matches reference
30mistral/fp8fp885.1%88.5%15stable$0.24 / $0.96$0.49Matches reference
31modalunknown86.8%86.9%15stable$0.25 / $1.00$0.50Matches reference
32venice/fp8fp886.9%87.6%15stable$0.30 / $1.20$0.60Matches reference
33fireworksunknown86.6%85.1%15stable$0.30 / $1.20$0.61Matches reference
34coreweave/fp8fp886.1%87.3%14stable$0.30 / $1.20$0.61Matches reference
35basetenunknown85.9%88.3%15stable$0.30 / $1.20$0.61Matches reference
36crusoe/fp8fp884.2%83.2%15stable$0.29 / $1.20$0.61Matches reference
37parasail/fp8fp885.3%85.3%15stable$0.30 / $1.20$0.62Matches reference
38togetherunknown83.9%78.5%15stable$0.30 / $1.20$0.63Matches reference
39fireworks/usunknown85.1%88.1%15stable$0.45 / $1.80$0.93Matches reference
40baseten/fastunknown87.5%96.9%14stable$0.60 / $2.40$1.20Matches reference
41sail-research/fp4/usfp486.2%85.2%14stablen/a / n/an/aMatches reference
42nebius/fp8fp885.4%83.6%13stablen/a / n/an/aMatches reference
43sail-research/fp4fp484.1%83.9%14stablen/a / n/an/aMatches reference
First look: fewer than 7 nights, not ranked
pearl-research/fp8fp888.1%n/a1n/a$0.050 / $0.30$0.13First look, 1 night
ambient/fp4fp489.6%85.9%3stable$0.080 / $0.52$0.21First look, 3 nights
antdigital/fp8fp885.3%85.0%2stable$0.12 / $0.48$0.25First look, 2 nights
inceptron/fp4fp484.6%90.0%2stable$0.10 / $0.60$0.27First look, 2 nights
nebius/fp4fp478.3%85.8%2stable$0.18 / $0.72$0.40First look, 2 nights
morph/fastfp885.8%83.2%5stable$0.60 / $2.40$1.22First look, 5 nights
fireworks/ultrafastunknown89.4%n/a1n/an/a / n/an/aFirst look, 1 night, not listed since Nov 8
cohereunknown87.7%88.3%3stablen/a / n/an/aFirst look, 3 nights, not listed since Nov 6
relace/fastfp485.6%86.9%5stablen/a / n/an/aFirst look, 5 nights, not listed since Nov 8
decart/fp8fp885.2%86.6%5stablen/a / n/an/aFirst look, 5 nights, not listed since Nov 10
phalaunknown84.3%82.3%6stablen/a / n/an/aFirst look, 6 nights, not listed since Nov 10
decart/fp4fp476.4%n/a1n/an/a / n/an/aFirst look, 1 night, not listed since Nov 4
deepseekunknownn/an/a15n/a$0.15 / $0.60n/aFirst look, 15 nights
Outside the reference
baidu/fp8fp876.0%83.1%15stable$0.15 / $0.60$0.35Outside reference
open-inference/fp4fp473.2%86.6%15stable$0.060 / $0.90$0.37Outside reference
near-aiunknown74.9%67.7%15stable$0.30 / $1.20$0.70Outside reference
ionstream/flexunknown76.5%85.6%8stablen/a / n/an/aOutside reference
rekaunknown76.1%80.4%11stablen/a / n/an/aOutside reference, not listed since Nov 15

Cite: Bitdepth · bitdepth.dev/m/deepseek-v4.1-flash-1017 · as of Nov 19, 2026

Reference: plurality across all endpoints (fewer than two declare bf16 or fp16). Open the DeepSeek V4.1 Flash page.

Method

How this board is measured.

The reference

For each model, the answer most endpoints declaring full precision (bf16 or fp16) give at temperature zero. Where fewer than two declare it, the plurality answer across all endpoints. Method.

Agreement and self-agreement

Agreement is how often an endpoint gives the reference answer. Self-agreement is how often it gives its own answer twice. An endpoint that agrees with itself far more than with the reference is serving something different.

How rank works

Endpoints that match the reference are ranked by quality-adjusted price: the blended price, (3 × input + output) ÷ 4 per million tokens, divided by agreement. A cheap endpoint that misses the reference is never shown as the cheapest.

First look

An endpoint needs 7 nights before it is ranked. Until then it shows its night count and its numbers, with no rank.

Drift

The mean of the last three nights against the first three. A move of more than 10 points reads as improving or worsening; anything smaller is stable.

How to cite

Every chart and table carries a cite line with the date. Copy it with the figure. Wrong model name or missing endpoint? Tell us.

Get the data

The same figures as JSON, one file per model, rewritten every night.

https://bitdepth.dev/data/v1/index.json

Serve one of these models?

Endpoints that match the reference can carry the Graded by Bitdepth badge, linked to their row. To add an endpoint, correct a listing or ask about grades and data for your routes, write to us.

Numbers against a stated reference. Not a claim about intent.Listed prices from OpenRouter, read nightly; grades are Bitdepth's.