Loading ToolBench
An enterprise AI agent platform that orchestrates document retrieval, database queries, and report generation through an MCP server with LangGraph-based agentic workflows
AgentPlatform has 4 tools with basic descriptions but significant gaps in schema completeness, parameter documentation, and error handling. Tool names follow verb_noun convention (search_, summarize_, query_, generate_), which is good. However, input schemas lack proper type definitions for several parameters, descriptions are generic and under-optimized for LLM selection, and output schemas are entirely undocumented. No tool annotations (readOnlyHint/destructiveHint) despite clear risk classifications. Error handling is absent, no recovery guidance or actionable error messages. The server registers tools via fastmcp but provides minimal metadata for agent reasoning.
Generates a comprehensive report on a given topic by pulling data from Qdrant, writing the report with Gemini, and logging the report in Postgres.
Query the structured Postgres database safely. question_type must be one of: - 'document_count': Returns how many documents have been uploaded. - 'recent_logs': Returns the last 5 tool usages.
Search the document knowledge base using vector similarity. Use this tool when you need to find information from the uploaded documents.
Summarizes a large piece of text. Use this when you have retrieved a long document chunk and need a concise summary.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields (e.g., does search_documents return document_id, content, score, metadata?). Agents must guess structure.
query_structured_data accepts 'question_type' as free-form string with enum constraint only in description text, not in schema. LLMs cannot parse the constraint from description alone and may hallucinate invalid values like 'user_count' or 'total_documents'.
No tool annotations (readOnlyHint/destructiveHint/idempotentHint) despite clear risk classifications. generate_report is marked WRITE but agents have no machine-readable signal that it modifies state and may not be safely retried.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 49 | <=2025-11-25 | v2 |
Descriptions are generic and under-optimized for LLM selection. 'Search the document knowledge base using vector similarity' (50 chars) lacks WHEN to use it vs other tools, what it returns, or prerequisites. Baseline is 194 chars for A+ tools.
No error handling or recovery guidance. If search_documents returns no results, or query_structured_data fails, agents have no actionable next step. No categorization of errors as retryable, user-fixable, or fatal.