Top of the leaderboard
in financial research on Fin50.

A financial research benchmark where Clartha leads in fully correct answers, with zero marked wrong.

Try ClarthaExplore the dataset
  1. 1
    Clartha20 fully correct answers out of 50
  2. 2
    Claude18 fully correct answers out of 50
  3. 3
    Perplexity Finance11 fully correct answers out of 50
  4. 4
    ChatGPT10 fully correct answers out of 50
  5. 5
    Google Finance7 fully correct answers out of 50

Tested across real-world financial research tasks.

  • Financial facts
  • Calculations
  • Comparisons
  • Recent events
  • Forecasts
  • Investment analysis

The complete picture.

All 50 questions. Every outcome included.

  • Fully correct
  • Partly correct
  • Wrong
  • Abstained

20 correct · 24 partial · 0 wrong · 6 abstained

18 correct · 16 partial · 2 wrong · 14 abstained

11 correct · 19 partial · 12 wrong · 8 abstained

10 correct · 26 partial · 9 wrong · 5 abstained

7 correct · 19 partial · 23 wrong · 1 abstained

Ranked by fully correct answers, rather than a weighted overall score.

What we look for in a well-supported answer.

Question-specific criteria inform the verdict; these are not separate weighted scores.

A useful financial answer needs more than the right number. It needs the right company, context and evidence.

Factual accuracyReporting contextClear reasoningSource supportUncertainty
Factual correctness
Required facts match the recorded reference or reviewed evidence.
Completeness
Every requested part is addressed, including all companies in a comparison.
Company and reporting basis
The right entity, with standalone and consolidated figures clearly distinguished.
Period and cutoff
The right fiscal period and date window, without information published after the cutoff.
Calculations and units
Matching inputs, correct formulas and arithmetic, with consistent currency, units and conversions.
Reasoning
Conclusions follow from evidence; facts, management guidance and inference are separated.
Forecast quality
A relevant baseline, explicit assumptions, scenarios and risks. Future outcome accuracy is not assessed.
Uncertainty
Missing evidence is acknowledged, with unsupported conclusions avoided.
Source support
Citations are considered; independent page-by-page verification remains incomplete.

How Fin50 was evaluated.

50 questions spanning financial facts, calculations, comparisons, recent developments, forecasts and investment reasoning.

Reference answers

Cross-checked across AI platforms and reviewed by the Clartha team. Independent source verification remains incomplete.

Response assessment

Provisional AI-assisted grading with question-level review notes. These verdicts are not from a single frozen, blind LLM-judge run.

Recorded comparison

Five products. All 50 questions. Four separate verdict categories, with the saved answers open to inspection.

Explore the research behind the results.

Every question, recorded answer, reference and grading note is available in the dataset on Hugging Face.

View Fin50 dataset

Your next research question starts here.

Ask about a company, compare peers or explore a financial question.

Try Clartha