Loading ToolBench
Methodology
How ToolBench scores MCP servers
Each server is scored on a few independent dimensions, combined into a weighted overall score, and mapped to a letter grade. Scoring is baselined to the current MCP spec revision (2026-07-28). Every run is kept in the server's score history with its rubric version, so a change in methodology is never presented as a change in quality.
Dimensions and weights
| Dimension | Local (GitHub) | Hosted endpoint | What it measures |
|---|---|---|---|
| Definition Quality | 50% | 30% | Tool names, descriptions, input schemas, and how well tools compose |
| Protocol Readiness | 20% | 30% | Transport and spec posture against the 2026-07-28 baseline |
| Security | n/a | 20% | Live checks of authorization, PKCE, and request validation |
| Supportability | 30% | 20% | Maintenance signals such as license, documentation, releases, and recent activity |
Hosted endpoints are tested live. When a hosted server's tool list can't be read, the remaining dimensions are reweighted (Protocol Readiness 40%, Security 30%, Supportability 30%). A hosted endpoint has no public repository to read, so it receives a neutral Supportability value.
Spec posture
A hosted server declares its protocol version during the live MCP handshake. A GitHub repository has no runtime declaration, so ToolBench reports an inferred effective spec from source evidence.
- Negotiated spec: the exact protocol version observed from a hosted server.
- Inferred effective spec: the earliest spec generation supported by version-defining capabilities found in source.
- Capability floor: a value such as 2025-06-18+ means “at least this generation,” not an exact declared version.
- Unresolved: the available files contain no reliable version-defining signal.
Inferred posture is shown with medium confidence. Missing a feature doesn't make a repository outdated, because most MCP capabilities are optional and SDKs can hide protocol details.
How the spec affects Protocol Readiness
- Credited: stateless requests, Streamable HTTP, structured tool output, tool annotations, multi round-trip requests, and per-request log levels.
- Small deduction, with a migration note: relying on patterns the 2026-07-28 spec deprecates, such as server-initiated Sampling, Roots, Logging, the stateful initialize and Mcp-Session-Id handshake, Dynamic Client Registration, or HTTP+SSE-only transport. Deductions total at most 15 points.
- Neutral: not implementing an optional or deprecated capability. A server that only exposes tools is fully compliant.
- Small bonus, never required: official extensions such as Tasks and Apps, up to 5 points.
Evidence rules
- A local STDIO server can't be tested over the network, so its Protocol Readiness is scored from source and tops out at 50. Its Definition Quality, the largest share of a local score, is scored the same way as any other server.
- A tool without a visible input schema scores 0 for its schema.
- A tool that can't be matched to a registration in source is scored at up to 50 until it can be verified.
- Every finding links to a public pattern guide that explains the fix.
Grade bands
| Grade | Score |
|---|---|
| A+ | 90 to 100 |
| A | 80 to 89 |
| B | 70 to 79 |
| C | 60 to 69 |
| D | 50 to 59 |
| F | below 50 |
What ToolBench does not measure
Scores describe how well a server's tools are defined and how closely it follows the spec. They don't test whether a tool returns correct results, how fast it responds, or how it behaves in a real agent workflow. Treat a grade as a strong signal about the interface, not a guarantee of production behavior.