Loading ToolBench
Monitor how AI systems perceive and recommend software products
GeoStorm provides 4 read-only monitoring tools with reasonable descriptions and input schemas. All tools follow verb_noun naming (list_, get_, get_trajectory). Descriptions range 46 - 220 chars, within acceptable bounds. Input parameters have type declarations and brief descriptions. However, output schemas are not explicitly documented in the source, we can infer structure from parameter names and descriptions, but no formal return type documentation is visible. Error handling guidance is absent. Parameters lack detailed constraints (e.g., date format, aggregation period enum values). The fuzzy-matching feature mentioned in descriptions is not formalized as a schema constraint. Overall solid for a monitoring/read-only domain, but gaps in output documentation and error recovery patterns prevent a higher score.
Get a full summary for a project: detail, perception scores, breakdown, recent runs, and alerts.
Get details for a single monitoring run, including perception score and competitors detected.
Get historical trajectory data showing recommendation share, position, and competitor delta over time.
List all monitored projects with their latest perception score, run count, and active alert count. Use this first to discover projects, then pass a project ID or name to other tools.
Output schemas not documented. Tools return structured data but no formal return type definitions are visible in source. LLMs cannot reliably parse responses or plan downstream tool calls.
Parameter constraints missing. get_trajectory accepts 'period' with no enum definition (docs say 'day', 'week', 'month' but schema does not enforce). Dates lack format specification (YYYY-MM-DD stated in description but not as JSON Schema pattern).
Error handling guidance absent. No documentation of what errors each tool can return, when to retry, or what to do if a project/run_id is not found. LLMs will not know how to recover from failures.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 67 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 52 | - | v1 |
Fuzzy matching feature mentioned in descriptions but not formalized. 'Fuzzy matching supported' is vague, does it match on name, ID, or both? Partial names? Hamming distance threshold? Without specification, LLMs will guess and may misidentify projects.
Discovery tool (list_projects) lacks guidance on when to use it. Description does not explicitly state 'Call this first to discover available projects before calling get_project_summary', agents may miss the dependency.