| Model | Quality | p99 latency | Cost per request | vs today | Result |
|---|---|---|---|---|---|
| Claude Sonnet 5.5 Today's modelopenrouter/anthropic/claude-sonnet-5.5 | 94.3% replayed on 159 gold rows | 1,636 ms benchmark replies on the same messages | ₹0.4583 benchmark replies on the same messages | — | Today |
| openrouter/openai/gpt-6-luna | 93.7% replayed on 159 gold rows | 2,161 ms replayed on 159 gold rows | ₹0.0195 replayed on 159 gold rows | −96% | Fails 2 checks |
openai/gpt-6-luna is at worst −1.9 pt against today's model, beyond the −1 point margin; falls below the minimum on Singapore receipts: 0.84 vs 0.85 (64 gold rows). Every other gated topic is at or above its minimum. Weakest topic: Singapore receipts 0.84 | |||||
| openai_compatible/dvar:glm-5.3-flash | 93.7% replayed on 159 gold rows | 9,444 ms replayed on 159 gold rows | ₹0.034 replayed on 159 gold rows | −93% | Fails 2 checks |
dvar:glm-5.3-flash is at worst −1.9 pt against today's model, beyond the −1 point margin; falls below the minimum on United States receipts: 0.93 vs 0.99 (14 gold rows). Every other gated topic is at or above its minimum. Weakest topic: United States receipts 0.93 | |||||
| Fine-tuned open modelnext step | Measured after tuning on your data. No numbers yet. | Talk to us | |||
Fine-tuned on your masked gold set. Masked rows only. Needs your data-sharing opt-in. | |||||
No candidate passes every check. Open a model's row for its reasons.
Prices are per request, at each provider's list price.
Judge: 0 of 7 checks trusted.
| Topic | Rows | Today · Claude Sonnet 5.5 | Candidate · GPT-6 Luna | Candidate · dvar:glm-5.3-flash | ||
|---|---|---|---|---|---|---|
| Singapore receipts | 64 | 0.86 | 0.84 | −0.02 | 0.86 | 0.00 |
| United States receipts | 14 | 1.00 | 1.00 | 0.00 | 0.93 | −0.07 |
| Australia receipts | 32 | 1.00 | 1.00 | 0.00 | 1.00 | 0.00 |
| Hong Kong receipts | 30 | 1.00 | 1.00 | 0.00 | 1.00 | 0.00 |
| Netherlands receipts | 19 | 1.00 | 1.00 | 0.00 | 1.00 | 0.00 |
| All topics, weighted by traffic | 159 | 0.94 | 0.94 | 0.00 | 0.94 | 0.00 |