Loading ToolBench
Read-only MCP server exposing the AGmind Systems Lab claim registry: measured local-LLM benchmarks with citations and raw-run provenance.
Three well-named, read-only tools with clear descriptions (194 - 250 chars, within baseline 34 - 392). All tools have explicit input schemas with type and description. Tool names follow verb_noun pattern (search_claims, get_claim, list_measured). Descriptions explain WHAT, WHEN, and WHAT-TO-RETURN. Output is structured JSON with attribution and helpful hints. No parameter descriptions are missing. Error handling includes recovery guidance ('Try fewer terms', 'did you mean' suggestions). Main gaps: output schemas not formally documented in code; no pagination support for search_claims (could return many results); no explicit per-tool error classification.
Fetch a single claim from the AGmind registry by id: full statement, measured value, unit, scope, limitations, evidence level, raw run ids and links, permalink, and the ready-made citation string. Ids look like "strix.qwen36.interactive2.c1.ttfa-nothink".
List the distinct system × model × runtime combinations that have published, measured claims in the AGmind registry, with claim counts and example claim ids (with permalinks). Start here to see what hardware and models the lab has qualified.
Keyword search over the AGmind claim registry of measured local-LLM benchmarks (hardware, runtime, model, metric, workload scope). Case-insensitive; every whitespace-separated term must match. Returns claim summaries with value, unit, evidence level, permalink, and a ready-made citation string. Use get_claim for the full record including limitations and raw run ids.
Output schemas not formally documented. Tool descriptions explain what fields are returned (id, headline, value, unit, evidence_level, permalink, cite, attribution), but no JSON Schema definition is visible in code. LLMs cannot plan downstream operations without knowing return structure.
search_claims lacks pagination. Description does not mention result limits or pagination parameters. If registry grows large, search could return hundreds of claims, exhausting context window. No limit, offset, or cursor parameters visible.
Error handling is implicit in code (closeIds fallback, cache fallback) but not documented in tool descriptions. LLMs cannot know whether a 'not found' error is retryable, user-fixable, or fatal.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 66 | 2026-07-28+ | v2 |
Tool annotations present (readOnlyHint: true) but openWorldHint: false is overly restrictive. These are read-only tools with no side effects, openWorldHint should be true to signal safety.