Free
Free
Leaderboards and benchmark descriptions are published on the public site without a paid subscription.
- Access
- Leaderboards and benchmark descriptions are published on the public site without a paid subscription.
Research · Vendor site 2026-10-04
SWE-bench is a family of open software-engineering benchmarks built from real GitHub issues, with public leaderboards comparing model and agent performance. The site tracks variants such as Verified, Lite, Multilingual, and Multimodal tasks and links related tools like mini-SWE-agent and SWE-agent.
Researchers and teams evaluating coding agents and LLMs on realistic repository repair and software engineering tasks.
Verified Bash Only evaluation uses 500 curated instances rather than the full 2,294-instance SWE-bench set.
SWE-bench hosts official leaderboards and benchmark variants for measuring how well models and agents resolve real open-source issues.
What it can do
Unknown is different from unavailable. Each fact carries its own evidence.
| Capability | Value | Evidence | Checked |
|---|---|---|---|
| Benchmark scope | Site describes SWE-bench as real GitHub issues from Python repositories with multiple subset benchmarks. | Facts sourced | 2026-10-04 |
| Leaderboards | Public leaderboards report percent resolved across models with comparison and analysis views. | Facts sourced | 2026-10-04 |
Understand the total cost
Free
Free
Leaderboards and benchmark descriptions are published on the public site without a paid subscription.
RunInfra provides hosted open models through OpenAI- and Anthropic-compatible APIs. It offers prepaid token billing and workspace coding plans, with session-aware caching and separately reported cached-token usage.
Explore toolBuild Or Not is a startup research platform that combines AI tool traffic trends, backlink datasets, AI model listings, and startup revenue signals so builders validate ideas before investing engineering time.
Explore toolWindsurf, renamed Devin Desktop on 2 June 2026, is Cognition's AI code editor with an Agent Command Center for running local and cloud (Devin) coding agents, plus Tab completions and inline edits. Paid plans use shared usage quotas.
Explore toolThe practical questions
SWE-bench is a family of open software-engineering benchmarks built from real GitHub issues, with public leaderboards comparing model and agent performance. The site tracks variants such as Verified, Lite, Multilingual, and Multimodal tasks and links related tools like mini-SWE-agent and SWE-agent.
Leaderboards and benchmark descriptions are published on the public site without a paid subscription.. This record lists ongoing free access; check the plan limits before starting.
No paid monthly price is listed; this record treats the product as free to start. See the plan cards for entitlements, billing commitments and seat minimums.
Not confirmed. API access and subscription access may have different terms; consult the linked sources.
Verified Bash Only evaluation uses 500 curated instances rather than the full 2,294-instance SWE-bench set.
No retained pricing changes yet. A current price alone does not establish a historical trend.
Reviewed vendor source
Read original source ↗Facts apply to the named version and check date. Send a sourced correction if something changed.