FastMCP 3.2: Fleet grade aggregator (ToolBench, Glama, LobeHub) + ToolBench guide & Playwright archiver. Replaces toolbench-mcp.
Critical gaps across all dimensions. Tool definitions lack proper schemas, no visible input parameter type definitions, only enum/description hints. Descriptions are present but generic (avg ~80 chars, well below 194-char baseline). No output schemas documented. Tools appear inferred from file paths rather than explicit registration. No error handling guidance, no parameter validation rules, no security annotations. Naming is inconsistent: 'scraper_help', 'scraper_matrix', 'scraper_refresh' use prefixes but lack clear verb-noun structure. No evidence of idempotency, pagination, or composition patterns.
Multi-level help for scraper-mcp. Provides tiered documentation: - basic: Quick commands, grade meaning - advanced: All MCP tools, fleet exceptions, improvement workflow - platform: Per-platform grading methodology, rescore links, dimensions
Concrete Python code snippets for each issue
Prioritized fix list from ToolBench criticisms
Coverage matrix: repos x platforms with grades
List/info/add grading platforms
Request rescoring on platforms
Refresh all (or one) repo grades from platforms
Single-repo detail with history, TDQS dims, risk scores
No visible input schemas with type definitions. Tools show only enum constraints and descriptions; parameter types (string, number, boolean, object) are not explicitly declared in JSON Schema format.
No output schemas documented. LLMs cannot infer what fields to expect from tool responses, preventing downstream tool chaining and forcing agents to guess at response structure.
Tool definitions appear inferred from file paths rather than explicitly registered with full schemas. Source code does not show actual tool registration calls with complete parameter definitions.
Descriptions are generic and under 100 characters for most tools (e.g., 'Coverage matrix: repos x platforms with grades'). Lack context on WHEN to use, WHAT it returns, and prerequisites. Baseline is 194 chars.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 37 | 2026-07-28+ | v2 |
Server health, last fetch times, grade distribution
ToolBench-specific context and links
No error handling guidance. Tools do not document what errors can occur, whether they are retryable, or what the LLM should do next (e.g., 'If repo not found, call scraper_matrix first').