Loading ToolBench
Frost MCP has well-structured tool definitions with complete JSON schemas and consistent naming patterns. All 7 tools follow verb_noun convention (check_*, get_*, dry_run_*, watch_*). Descriptions are present and contextual (avg ~80 chars), explaining WHAT each tool does. However, parameter descriptions lack depth, many omit format constraints, valid ranges, and dependency hints. Output schemas are partially documented in code but not fully exposed in tool registration. Error handling and recovery guidance are absent. No tool annotations (readOnlyHint, destructiveHint) despite all tools being READ_ONLY. Missing idempotency guarantees and per-item error reporting for batch-like operations.
Summarize health across all tables in a catalog
Compute fleet-level signals across all tables in a catalog
Run health checks on an Iceberg table
Dry-run a fix: return the scope (what will change) without exposing the executable command
Estimate cost waste for a table
Generate a fix command for a specific finding
Query watch mode state from the state database
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite all tools being READ_ONLY. LLMs cannot infer safety properties without explicit hints.
Parameter descriptions lack format constraints and valid ranges. E.g., 'dormant_days' has no min/max bounds; 'checks' array has no size limits; 'trend_days' has no range. LLMs will pass invalid values.
No error handling or recovery guidance in tool descriptions. If a table is not found or a check fails, LLMs have no guidance on next steps (retry, search, fallback).
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 54 | <=2025-11-25 | v2 |
Output schemas (CatalogSummary, TableBrief, WatchStatusResponse) are defined in code but not documented in tool descriptions. LLMs cannot plan downstream calls without knowing response structure.
No pagination or result limits documented. check_catalog and check_fleet could return hundreds of tables; no guidance on max results or cursor-based pagination.