Loading ToolBench
A monorepo containing multiple Model Context Protocol servers for FerrLabs platform services: FerrFleet (agents, runs, skills), FerrGrowth (sites, pages, forms, audits), FerrTrack (issues, labels, comments), FerrVault (vaults, secrets, rotation), and FerrLabs core platform (orgs, projects, subscriptions, docs, tokens, stats)
FerrFleet MCP provides 6 tools with consistent naming (verb_noun pattern: list_agents, get_agent, trigger_agent_run, list_runs, get_run, get_run_transcript). All tools have descriptions (avg ~100 chars, within baseline 34-392 range). Input schemas are present with typed parameters. However, parameter descriptions are minimal or absent in several tools (e.g., 'input' in trigger_agent_run lacks detail on expected structure), and output schemas are not documented. Error handling guidance is missing, tools do not indicate what to do on failure or how to recover. No tool annotations (readOnlyHint/destructiveHint) despite clear risk levels (WRITE vs READ_ONLY). Composition is sound: tools chain logically (trigger_agent_run → get_run → get_run_transcript). Naming is clear and unambiguous.
Get full details of a FerrFleet agent (manifest, scopes, last run).
Get the metadata of a single FerrFleet run (status, timing, agent, trigger). Use get_run_transcript for the captured output.
Get the full captured transcript of a FerrFleet run — every event the runner emitted, in order. Heavy for long runs; consider get_run first to confirm the run is interesting before pulling the transcript.
List FerrFleet agents available to the authenticated user (marketplace + custom).
List recent FerrFleet runs for the authenticated org (most recent first).
Trigger a new run of a FerrFleet agent. Returns the freshly-created run id — poll get_run for status or pull get_run_transcript when finished.
Output schemas not documented. LLMs cannot plan downstream calls or extract required fields (e.g., what fields does get_agent return? Is there a manifest field?). Baseline: 100% of A+ tools document return types.
Parameter descriptions are sparse or missing. 'input' in trigger_agent_run is described as 'Optional structured input' but does not explain what structure is expected or provide constraints. 'reason' lacks guidance on format or length. Baseline: 100% of A+ tool params have descriptions.
No error handling guidance. Tools do not indicate what to do on failure (e.g., if agent_id is invalid, should the LLM call list_agents first?). No recovery hints or error classification (retryable vs user-fixable).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 63 | 2026-07-28+ | v2 |
No tool annotations despite clear risk levels. trigger_agent_run is marked WRITE but lacks destructiveHint annotation. list_* and get_* tools are READ_ONLY but lack readOnlyHint. Annotations help LLMs reason about side effects and retry safety.
list_runs accepts 'limit' parameter but no pagination cursor or offset. Large result sets may exceed context window. Baseline: tools returning lists should support pagination with total count or next_cursor.