BenchSift

Decision Index

Artificial Analysis leaderboard of decision models, which pick among options and return their probabilities, against language models answering in JSON: classification, tool calling, knowledge, hallucination and reasoning.

View on Artificial Analysis
Configurations
26
Best index
79.0
Claude Opus 5.5 (Max, Default Fallback)
Best decision model
57.5
Jev 1.13
Fastest
0.063 s
Celeris-1 Decision · 52.3

Index by cost or time

One point per measured configuration. The horizontal axis is reversed: towards the top right, a model scores higher for less money or less time.

Decision Indexmost efficient ↗Cost per 1M tasks

Models shown26 of 26

Select a model to show or hide it; hovering it brings it forward on the chart. The axes fit the models shown.

26 / 26 configurations

  1. #1
    Claude Opus 5.5 (Max, Default Fallback)LLM

    Anthropic

    79.0
    Accuracy
    93.4
    Non-halluc.
    30.3
    HLE
    70.2
    When2Call
    90.5
    WildGuard
    93.5
    Cost / 1M tasks
    $51,294.90
    Time / task
    3.57 s
  2. #2
    Claude Opus 5.5 (Low, Default Fallback)LLM

    Anthropic

    75.4
    Accuracy
    93.4
    Non-halluc.
    27.3
    HLE
    59.8
    When2Call
    88.1
    WildGuard
    93.5
    Cost / 1M tasks
    $6,688.16
    Time / task
    1.87 s
  3. #3
    GPT-6.1 Sol (Low)LLM

    OpenAI

    71.5
    Accuracy
    93.0
    Non-halluc.
    22.9
    HLE
    55.4
    When2Call
    80.6
    WildGuard
    92.0
    Cost / 1M tasks
    $2,048.35
    Time / task
    2.89 s
  4. #4
    GPT-6.1 Sol (Max)LLM

    OpenAI

    70.9
    Accuracy
    91.0
    Non-halluc.
    11.1
    HLE
    58.9
    When2Call
    81.2
    WildGuard
    92.5
    Cost / 1M tasks
    $6,322.89
    Time / task
    4.43 s
  5. #5
    Claude Haiku 5.5 (Max)LLM

    Anthropic

    68.4
    Accuracy
    57.6
    Non-halluc.
    69.3
    HLE
    37.4
    When2Call
    79.4
    WildGuard
    93.2
    Cost / 1M tasks
    $2,073.24
    Time / task
    2.66 s
  6. #6
    Claude Haiku 5.5 (Xhigh)LLM

    Anthropic

    64.4
    Accuracy
    58.6
    Non-halluc.
    55.1
    HLE
    30.2
    When2Call
    77.6
    WildGuard
    93.1
    Cost / 1M tasks
    $493.09
    Time / task
    1.55 s
  7. #7
    Claude Haiku 5.5 (High)LLM

    Anthropic

    62.6
    Accuracy
    57.2
    Non-halluc.
    48.6
    HLE
    26.7
    When2Call
    76.9
    WildGuard
    93.8
    Cost / 1M tasks
    $292.17
    Time / task
    1.48 s
  8. #8
    Claude Haiku 5.5 (Medium)LLM

    Anthropic

    61.4
    Accuracy
    58.6
    Non-halluc.
    48.8
    HLE
    23.2
    When2Call
    75.4
    WildGuard
    93.3
    Cost / 1M tasks
    $232.41
    Time / task
    1.53 s
  9. #9
    GPT-6 Luna (Max)LLM

    OpenAI

    60.8
    Accuracy
    73.6
    Non-halluc.
    3.0
    HLE
    35.3
    When2Call
    79.0
    WildGuard
    90.5
    Cost / 1M tasks
    $591.63
    Time / task
    2.25 s
  10. #10
    Claude Haiku 5.5 (Low)LLM

    Anthropic

    60.6
    Accuracy
    52.4
    Non-halluc.
    54.6
    HLE
    23.6
    When2Call
    72.6
    WildGuard
    92.6
    Cost / 1M tasks
    $184.25
    Time / task
    1.43 s
  11. #11
    GPT-6 Luna (High)LLM

    OpenAI

    59.1
    Accuracy
    69.6
    Non-halluc.
    3.3
    HLE
    32.4
    When2Call
    78.6
    WildGuard
    88.9
    Cost / 1M tasks
    $234.32
    Time / task
    1.8 s
  12. #12
    GPT-6 Luna (Medium)LLM

    OpenAI

    58.2
    Accuracy
    71.2
    Non-halluc.
    0.0
    HLE
    29.0
    When2Call
    78.2
    WildGuard
    89.8
    Cost / 1M tasks
    $181.79
    Time / task
    1.73 s
  13. #13
    Jev 1.13Decision model

    Typesafe

    57.5
    Accuracy
    34.6
    Non-halluc.
    35.2
    HLE
    20.9
    When2Call
    80.7
    WildGuard
    93.6
    Cost / 1M tasks
    $41.53
    Time / task
    0.127 s
  14. #14
    Gemma 4 31B (Reasoning)LLM

    Google · DeepInfra

    55.5
    Accuracy
    47.6
    Non-halluc.
    4.2
    HLE
    25.7
    When2Call
    77.6
    WildGuard
    92.8
    Cost / 1M tasks
    $624.44
    Time / task
    14.4 s
  15. #15
    GPT-6 Luna (Low)LLM

    OpenAI

    55.2
    Accuracy
    66.0
    Non-halluc.
    0.0
    HLE
    22.2
    When2Call
    77.0
    WildGuard
    88.5
    Cost / 1M tasks
    $131.05
    Time / task
    1.5 s
  16. #16
    GLiDEDecision model

    Fastino

    53.9
    Accuracy
    36.4
    Non-halluc.
    16.4
    HLE
    10.5
    When2Call
    83.7
    WildGuard
    95.1
    Cost / 1M tasks
    $119.02
    Time / task
    0.217 s
  17. #17
    Microsoft-Decision-1Decision model

    Microsoft · OpenRouter

    53.8
    Accuracy
    28.2
    Non-halluc.
    39.8
    HLE
    10.9
    When2Call
    81.9
    WildGuard
    88.2
    Cost / 1M tasks
    $30.71
    Time / task
    0.207 s
  18. #18
    Decider v1.1 27BDecision model

    Perplexity

    53.1
    Accuracy
    39.8
    Non-halluc.
    13.3
    HLE
    15.4
    When2Call
    76.5
    WildGuard
    93.8
    Cost / 1M tasks
    $16.53
    Time / task
    0.17 s
  19. #19
    Mercury DecideDecision model

    Inception

    52.7
    Accuracy
    35.0
    Non-halluc.
    32.0
    HLE
    10.1
    When2Call
    79.3
    WildGuard
    88.0
    Cost / 1M tasks
    $30.41
    Time / task
    0.5 s
  20. #20
    Celeris-1 DecisionDecision model

    Celeris

    52.3
    Accuracy
    37.8
    Non-halluc.
    0.0
    HLE
    13.3
    When2Call
    83.7
    WildGuard
    93.3
    Cost / 1M tasks
    $31.57
    Time / task
    0.063 s
  21. #21
    d1Decision model

    Liquid

    50.3
    Accuracy
    41.0
    Non-halluc.
    1.7
    HLE
    11.5
    When2Call
    77.4
    WildGuard
    90.9
    Cost / 1M tasks
    $30.29
    Time / task
    0.299 s
  22. #22
    GPT-6 Luna (Non-reasoning)LLM

    OpenAI

    49.6
    Accuracy
    45.4
    Non-halluc.
    0.4
    HLE
    12.3
    When2Call
    76.1
    WildGuard
    87.1
    Cost / 1M tasks
    $88.44
    Time / task
    1.14 s
  23. #23
    ClefDecision model

    Cloudflare

    48.6
    Accuracy
    39.0
    Non-halluc.
    4.3
    HLE
    11.1
    When2Call
    71.0
    WildGuard
    90.7
    Cost / 1M tasks
    $199.41
    Time / task
    0.309 s
  24. #24
    GPT-6 Luna (Decisions)Decision model

    OpenAI

    48.5
    Accuracy
    40.6
    Non-halluc.
    10.1
    HLE
    11.7
    When2Call
    74.1
    WildGuard
    82.8
    Cost / 1M tasks
    $78.87
    Time / task
    0.168 s
  25. #25
    Clef-FlashDecision model

    Cloudflare

    45.8
    Accuracy
    32.6
    Non-halluc.
    2.7
    HLE
    10.3
    When2Call
    64.4
    WildGuard
    90.9
    Cost / 1M tasks
    $31.57
    Time / task
    0.298 s
  26. #26
    Clef-OmniDecision model

    Cloudflare

    44.3
    Accuracy
    36.0
    Non-halluc.
    1.6
    HLE
    9.0
    When2Call
    62.1
    WildGuard
    87.2
    Cost / 1M tasks
    $123.08
    Time / task
    0.172 s
Method and versions

The Artificial Analysis Decision Index combines four evaluations. Each task poses a state, the text to decide about, and one or more questions with named options or a yes/no answer; the model answers all of a task's questions in a single call.

  • The index is the equal-weight average (25% each) of AA-Omniscience-Decision, HLE-MultiChoice, When2Call and WildGuardTest; AA-Omniscience-Decision counts its accuracy and its non-hallucination rate at half each.
  • Decision models receive the state and questions through their decision API and return the option they pick with a probability for every option; a yes/no answer is yes from a probability of 0.5.
  • Language models receive a fixed system prompt and the state and questions as text, and answer in JSON enforced with structured outputs.
  • AA-Omniscience-Decision: 500 private AA-Omniscience questions, each with four answers, one correct, and "I don't know".
  • HLE-MultiChoice: the 513 text-only multiple-choice questions of Humanity's Last Exam, out of 2,500 in all.
  • When2Call: 3,423 of the 3,652 tasks in NVIDIA's test set; AA drops 229 ambiguous "ask for more information" tasks.
  • WildGuardTest: a safety classification evaluation, scored by average F1.
  • Cost and time are the per-task averages Artificial Analysis publishes; cost is shown per 1 million tasks, as on AA.

Data via Artificial Analysis, Decision Index.

Preferences and analytics

Rybbit: visits, performance, clicks, copies, forms and session replay (masked inputs), with your permission. Refusing does not limit the site.

Privacy

Allow navigation analytics?

With your permission, Rybbit measures visits, performance and interactions (clicks, copies, forms), including session replay, to improve BenchSift. Input values are masked in replays. Refusing does not limit the site.

Your choice is stored for 180 days. Change it at any time using “Privacy preferences” in the footer. Withdrawal reloads the page.

No choice stored: Rybbit is disabled.

Privacy