Loading ToolBench
FastMCP server exposing tools for analyzing the Bitext Customer Service dataset. Provides data retrieval, filtering, and analysis capabilities for customer support interactions.
Server provides 7 well-named, read-only tools with complete input schemas and reasonable descriptions. All tools follow verb_noun naming (list_, get_, count_, search_). Descriptions are present and actionable (avg ~120 chars), exceeding the 10-char minimum. However, parameter descriptions are sparse or missing context about constraints and use cases. Output schemas are inferred from code but not formally documented. Error handling is minimal, tools return generic error dicts rather than actionable recovery guidance. No tool annotations (readOnlyHint, idempotentHint) despite all being read-only. Missing pagination metadata (total_count, next_cursor) in paginated results.
Count rows in the dataset matching optional category and/or intent filters. Returns total count when no filters given.
Return up to n example rows (category, intent, instruction, response). Filter by category and/or intent as needed. Row order is stable, so offset pages through the matches: after fetching 3 rows with offset=0, call again with offset=3 to get the next 3 distinct rows.
Return count of rows per intent within the specified category. Useful for distribution/breakdown questions.
Return all unique category names in the Bitext Customer Service dataset.
Return unique intent names, optionally filtered by category (case-insensitive).
Search customer instructions for a keyword or phrase (case-insensitive). Returns up to n matching rows.
Output schemas not formally documented. Tools return dicts/lists inferred from code, but LLMs cannot see field types, required fields, or nested structure. Agents must guess what fields exist.
Error handling returns generic dicts like {"error": "No matching rows"} without recovery guidance. Should suggest next steps: 'No matching rows. Try list_categories() to see available options.'
Parameter descriptions lack constraint details. E.g., 'n' param has min/max in schema but description does not mention the 1-20 range. LLMs cannot read JSON Schema, they rely on description text.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2026-07-28+ | v2 |
Return sampled instructions and responses for open-ended summarization. Provides raw data for the caller to synthesize a summary.
Pagination metadata missing. get_examples() accepts offset but does not return total_count or next_cursor. Agents cannot determine if more results exist without calling count_records() separately.
No tool annotations. All 7 tools are read-only and idempotent, but lack readOnlyHint and idempotentHint annotations. Agents cannot infer safety properties without explicit hints.