Loading ToolBench
eipmcp demonstrates solid tool design with 20 well-named, verb-prefixed tools covering a coherent domain (EIP/spec discovery, diffing, cross-referencing). All tools have descriptions (avg ~150 chars, within baseline 34-392 range). Input schemas are present with type declarations and parameter descriptions. However, output schemas are not explicitly documented in the source, responses are inferred from code logic rather than declared. Error handling is minimal (e.g., 'EIP-{number} not found' but no recovery guidance). No tool annotations (readOnlyHint/destructiveHint) despite clear risk distinctions (18 READ_ONLY, 2 WRITE). Parameter descriptions are adequate but lack format constraints (e.g., 'number' lacks min/max bounds). Composition is strong, tools chain well (list_eips → get_eip, search_eips → get_eip, eip_requires → eip_required_by). Naming is consistent and clear (verb_noun pattern throughout).
Unified git diff for one EIP between two revisions. Use to see how an EIP shifted since the last sync, or between any two refs. since: 'previous_sync' (default — the sync before the latest), 'last_sync' (latest sync vs current HEAD), or any git rev (SHA, tag, branch). until: any git rev; defaults to current HEAD.
Unified git diff for one spec path between two revisions. Same `since` / `until` semantics as `diff_eip`. Use to track what changed in a spec file between syncs or releases.
Files (specs, tests, other EIPs) that mention `EIP-<number>` in body or path. Catches prose references the formal `requires:` graph misses — useful for tracing where an EIP is actually implemented, specified, or tested. Filter by source repo (e.g. 'consensus-specs', 'execution-specs', 'eips').
Reverse of `eip_requires`: every EIP that lists this one in `requires:`. Use for impact analysis ('what depends on EIP-N?').
EIPs this EIP explicitly depends on (from its `requires:` frontmatter). Use when planning what to read before tackling a new EIP.
Output schemas not documented. Tool descriptions state what is returned (e.g., 'returns relevance-ranked matches with highlighted snippets') but no formal schema is visible in code. LLMs cannot plan downstream tool calls without knowing response field names and types.
No tool annotations (readOnlyHint/destructiveHint/idempotentHint). 18 tools are READ_ONLY and 2 are WRITE (sync_repo, sync_all), but this is not declared in tool metadata. Agents cannot distinguish safe from destructive operations without explicit hints.
Minimal error recovery guidance. Error responses like 'EIP-{number} not found in repo. Run `sync` first?' are present but inconsistent. No structured error classification (retryable vs user-fixable vs fatal). Agents cannot reliably decide next steps on failure.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2026-07-28+ | v2 |
Fetch one EIP/ERC with full body, frontmatter, and parsed `requires` list. Use when you need the actual spec text. For summary-only listings call `list_eips`. repo: 'eips' (default) or 'ercs'.
Resolve a fork codename to its Meta EIP(s) and the EIPs it bundles. Accepts aliases: pectra↔prague↔electra, fusaka↔fulu↔osaka, glamsterdam↔amsterdam, cancun↔deneb, shanghai↔capella, merge↔paris↔bellatrix. Use to answer 'what's in fork X?' in one call.
Fetch one spec file's full contents. Pair with `list_specs` to discover paths. execution-apis stores per-method JSON-RPC / Engine API docs; paths carry a '#<method>' fragment, e.g. 'src/eth/block.yaml#eth_getBlockByHash'. Search by method name via search_specs to find the exact path first.
List EIP summaries (number, title, description, status, type, category). Cheap discovery call — filter by repo / status / type / category to scope. Use before `get_eip` when you don't already know the number.
All indexed Meta EIPs — the universe of forks `get_hardfork` can resolve.
List spec file paths in `repo`. `glob` accepts `*` as a wildcard. Examples: `*prague*`, `specs/electra/*`, `*beacon-chain.md`. Use for discovery before `get_spec`.
Open pull requests against `repo` that mention EIP-<number>. Live GitHub query (not from the local index). Use to spot in-flight changes before relying on the indexed version of an EIP.
Open pull requests against `repo` matching the basename of `path`. Approximate file-touched filter (GitHub search has no exact one). Live query.
Files changed in each tracked repo since `days` ago, anchored to the last sync recorded before that point. EIP rows get enriched with current status and (when changed) the previous status.
The EIPs mentioned inside one file (reverse of `eip_referenced_in`). Pass the same (repo, path) you'd give `get_spec`. Use to summarise what an unfamiliar spec or test file relates to.
Ranked full-text search across EIP title, description, and body. Returns relevance-ranked matches with highlighted snippets. Use when you don't know the EIP number — e.g. 'EIPs about blob mempool', 'withdrawals queue'. Multi-word queries match all tokens.
Ranked full-text search across spec file bodies. Use for concept-level questions in consensus-specs / execution-specs that aren't tied to an EIP number — e.g. 'where is RANDAO mixed?', 'how is the blob commitment computed?'. Filter by repo to narrow CL vs EL. execution-apis holds per-method JSON-RPC / Engine API docs (one doc per method); search by method name, e.g. 'eth_getBlockByHash' or 'engine newPayload blob'. Those spec paths carry a '#<method>' fragment, e.g. 'src/eth/block.yaml#eth_getBlockByHash'.
Pull and reindex all tracked repos. Returns per-repo stats.
Pull one repo and reindex. Returns counts of upserted/unchanged/deleted/skipped rows plus interesting deltas (status changes, newly added, body churn).
Test files under execution-specs/tests/ that exercise or reference an EIP. Matches both `tests/<fork>/eip<n>_*/` directories and in-body mentions — catches cross-fork interactions (e.g. EIP-7702 referenced by Amsterdam's block-access-list tests). Use to find conformance tests when implementing.
Numeric parameters lack bounds. 'number' (EIP number), 'limit' (max results), 'snippet_words', 'context', 'days' have no min/max constraints in schema. LLMs can pass absurd values (limit=999999, days=-1) that break logic or cause timeouts.
Enum constraints missing for string parameters. 'repo' parameter accepts 'eips', 'ercs', 'consensus-specs', 'execution-specs', 'execution-apis' but is declared as free-form string. LLMs may hallucinate invalid repo names. Should be enum.