Loading ToolBench
Connect to your Jellyfish Software Engineering Intelligence platform for comprehensive team insights, effort allocation, and delivery metrics.
The server has 9 well-named tools with consistent verb_noun patterns (get_*, ai_*). Descriptions are present and mostly adequate (100-200 chars range), meeting baseline expectations. Input schemas are explicitly defined with types and enums. However, there are systematic gaps: output schemas are completely undocumented (no indication of what fields each tool returns), parameter descriptions lack depth (format constraints, ranges, examples are sparse), and error handling guidance is absent. The allocation_by_person and allocations_by_team tools have sophisticated parameter logic (max_n_allocation_card_keys with a CRITICAL note) but this complexity is not well-integrated into the schema. Per-tool response validation is minimal, the code validates enums but provides no structured error recovery guidance.
Returns per-tool AI adoption analytics aggregated across the entire company, including cohort counts and average usage percentage.
Returns per-tool AI impact analytics aggregated across the entire company, including median issue cycle time, median PR cycle time, total PR throughput, and total AI adoption lines.
Returns per-tool AI adoption usage dates for the specified people, with a breakdown by each configured tool.
Returns per-tool AI adoption analytics for the specified people, including cohort and usage percentage.
Returns per-tool AI adoption analytics aggregated by team, including cohort counts and average usage percentage.
Output schemas are completely undocumented. No tool definition includes a 'returns' or 'outputSchema' field describing what fields agents can expect. This forces LLMs to guess at response structure and plan downstream calls blindly.
Parameter descriptions lack format constraints. Date parameters (start_date, end_date) are described as 'Start date (YYYY-MM-DD)' but do not state: required vs optional, whether times are supported, timezone handling, or validation bounds. The enum constraint on 'unit' is present but similar constraints for other params are missing.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 66 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 35 | - | v1 |
Returns per-tool AI impact analytics for the specified teams, including median issue cycle time, median PR cycle time, and total PR throughput.
Returns allocation data for the whole company, aggregated by person.
Returns allocation data for the whole company, aggregated by team at the specified hierarchy level.
Returns today's date in the specified timezone (default UTC). Call this BEFORE computing any relative date like "yesterday", "last week", "this quarter", "last Monday". Today's date is not in your context — guessing it from training data leads to wrong-year errors that downstream tools (e.g. search_deliverables) will reject. After calling, use the returned `date` to compute ISO bounds for timeframe_start / timeframe_end. The `day_of_week` field helps with weekday-relative queries.
Complex parameter logic in allocations_by_person and allocations_by_team (max_n_allocation_card_keys with CRITICAL note in description) is not reflected in the schema. The description mentions 'Default (omitted) returns ALL card keys which is rarely advised' but there is no minProperties, maxProperties, or conditional schema to guide the LLM. The critical note is in prose, not enforced programmatically.
Error handling provides no recovery guidance. The ApiTool._validateEnums() method returns 'Invalid parameters: <enum errors>' but does not suggest what the LLM should do next (e.g. 'Valid values are: quarter, month, week. Retry with one of these.'). api_generic() responses are opaque, no indication of whether an error is retryable, requires user action, or is fatal.
The person_id and team_id parameters accept arrays but do not document: minimum/maximum array length, whether duplicates are allowed, max number of IDs to prevent bloated responses. Large arrays could cause runaway API calls or massive response sizes.
The get_current_date tool includes a long, explanatory description but lacks a clear 'WHEN TO USE' signal. For an agent with many tools, the prose explanation may not be scannable enough to trigger tool selection at the right moment. Consider: 'Call this FIRST if computing relative dates (yesterday, last week). Returns ISO date and day_of_week to prevent LLM year-guessing errors.'