Free
Free
Open-source evaluation codebase and datasets are published via GitHub and Hugging Face without a listed SaaS fee.
- Access
- Open-source evaluation codebase and datasets are published via GitHub and Hugging Face without a listed SaaS fee.
LiveBench
● Identity checkedResearch · Vendor site 2026-10-04
LiveBench is a contamination-aware LLM benchmark that releases fresh questions monthly with verifiable ground-truth answers. It spans 18 tasks across six categories and provides open-source tooling to generate answers, score models, and publish leaderboard results.
- Category
- Research
- Free access
- Open-source evaluation codebase and datasets are published via GitHub and Hugging Face without a listed SaaS fee.
- API access
- Evaluation uses Python CLI scripts and optional OpenAI-compatible APIs for model inference; local model paths are unmaintained per README.
Is LiveBench right for you?
A good fit for
ML researchers and model vendors who need objective, automatically scored benchmarks that refresh to reduce training-data contamination.
Before you choose
Agentic coding tasks require Docker.
Challenging, contamination-free LLM benchmark
LiveBench combines frequently updated tasks, objective graders, and an public leaderboard for comparing frontier models across diverse categories.
What it can do
Features & capabilities
Unknown is different from unavailable. Each fact carries its own evidence.
| Capability | Value | Evidence | Checked |
|---|---|---|---|
| Monthly refresh | Project README states new questions release monthly to limit benchmark contamination. | Facts sourced | 2026-10-04 |
| Objective scoring | README emphasizes verifiable ground-truth answers scored automatically without an LLM judge. | Facts sourced | 2026-10-04 |
Understand the total cost
LiveBench pricing & plans
Alternatives to LiveBench
View all ↗
AllyhubAllyhub provides AI agents that collect, monitor, and act on web data from social platforms, marketplaces, and the open web. Users describe what to track, and agents gather fresh data, watch for meaningful changes, and deliver scheduled insights.
Explore tool
I IMAGINEIIMAGINE helps decision-makers test LLMs on their own work, route tasks to the best model, and ground advice in a SCOPED context framework. Agents and connections ingest business data so recommendations reflect objectives, constraints, and deadlines.
Explore tool
HumataHumata is a PDF and document AI that lets teams upload files, ask questions across their knowledge base, summarize long papers, compare documents, and embed answers on webpages. It targets researchers and professionals who need cited Q&A over private files with team permissions on higher tiers.
Explore toolThe practical questions
Frequently asked questions
What is LiveBench used for?
LiveBench is a contamination-aware LLM benchmark that releases fresh questions monthly with verifiable ground-truth answers. It spans 18 tasks across six categories and provides open-source tooling to generate answers, score models, and publish leaderboard results.
Does LiveBench have a free plan?
Open-source evaluation codebase and datasets are published via GitHub and Hugging Face without a listed SaaS fee.. This record lists ongoing free access; check the plan limits before starting.
How much does LiveBench cost?
No paid monthly price is listed; this record treats the product as free to start. See the plan cards for entitlements, billing commitments and seat minimums.
Can I use LiveBench through an API?
Evaluation uses Python CLI scripts and optional OpenAI-compatible APIs for model inference; local model paths are unmaintained per README.. API access and subscription access may have different terms; consult the linked sources.
What should I check before choosing it?
Agentic coding tasks require Docker.
Price history
No retained pricing changes yet. A current price alone does not establish a historical trend.
How this profile is supported
Identity and profile
Reviewed vendor source
Read original source ↗Facts apply to the named version and check date. Send a sourced correction if something changed.