ScoreTrace AI - Forecast Track Record Auditor
ScoreTrace AI: Forecast Track Record Auditor
Is that stock caller, football tipster or market analyst actually any good?
ScoreTrace AI answers with numbers. Give it a forecaster's dated public calls and the outcomes, and it returns what the record really shows: hit rate with a confidence interval, how it compares with chance, whether stated probabilities were calibrated, and flags for everything that makes a record look better than it is.
Most "tracker" tools tell you what someone said. ScoreTrace tells you whether they were right.
What you get
- Hit rate and verdict: per call type, with a 95% interval, an exact statistical test against a baseline you choose, and a plain-language verdict
- No verdict on thin data: under 30 resolved calls, it says the sample is too small instead of guessing
- Calibration: Brier score and probability bins for calls that state a probability ("70% chance Team A wins")
- Returns for stock-style calls: directional return, excess return versus a benchmark, win/loss size, and how much of the result comes from the three best calls
- Bias flags: record not confirmed complete, missing sources, calls never scored, outlier dependence, overconfidence, rejected rows
- Category breakdown and a per-call appendix with source links
- Machine-readable JSON report
How to use it
Give the agent:
- The forecaster's name or handle and the area (stocks, football, anything with checkable calls)
- Their public calls: paste posts, attach a CSV or JSON, or share public links
- Outcomes (prices or results), or ask the agent to look them up from public sources
- Optional: a baseline hit rate, a benchmark, and whether your list is their complete record
Example output
From the bundled sample audit of a fictional forecaster, not a real person.
| Check | Result |
|---|---|
| Direction calls scored | 40, with 24 hits (60%, 95% interval 44.6% to 73.7%) |
| Verdict | Not distinguishable from a 50% baseline (p = 0.13) |
| Mean directional return | 1.35%, but the 3 best calls are 111% of the total |
| Football calls with stated 70% | 12 scored, 8 hit, Brier score 0.223 |
| Flags | Too few football calls, record not confirmed complete, 23% of scored calls lack a source, 8 calls never scored, 3 rows rejected |
Why you can trust the numbers
- Deterministic: a script with no network access does every calculation, so the same input always gives the same result.
- Nothing invented: the agent never makes up a call, date, probability or outcome, and it lists every call it could not verify.
- Honest limits: results describe the sample supplied and are labeled that way.
- Neutral by design: statistics only. No accusations, no guesses about intent, no buy, sell or bet advice.
Limits
- It cannot see calls that were deleted, never supplied or never public. If the list is not the complete record, figures may be overstated, and the report says so.
- Returns are gross of costs, slippage and fees, and use the figures supplied.
- A good record is not proof of skill, and a poor one is not proof of the opposite. Small samples are mostly noise.
- It does not predict outcomes and is not financial, investment or betting advice.
- Public figures and public statements only. It will not compile a record about a private individual.
Data handling
Only material you supply and public pages are used, through the runtime's fetch or search tool, and the report names the tools used. No passwords, API keys, cookies or account access are needed. The analysis script runs offline.
Version
1.0.0
