Loading ToolBench
Collection of example integrations with PlanVault including MCP servers, webhook handlers, and frontend applications
This is an examples repository, not a production MCP server. Tool definitions are scattered across multiple example implementations (Node.js, Python, Java) with inconsistent quality. Only 5 tools identified; 3 have minimal descriptions (10-30 chars), 2 lack visible input schemas in source. No error handling guidance, no output schemas documented, no parameter descriptions for most inputs. Naming is reasonable (verb_noun pattern) but descriptions fail the 20-char minimum for actionable LLM guidance. The HITL approval gate tool (send-message) is the most complete but still lacks structured output schema and error recovery guidance.
Return a record by ID. Use list_records first if you do not know the ID.
SELECT * FROM users WHERE email = ? (read-only).
List all available record IDs with their names.
Return PRAGMA table_info for the users table (read-only).
PlanVault calls this endpoint as an HTTP tool. The call blocks until a human approves (or rejects) via POST /approve/{id}.
Descriptions too short to guide LLM selection. 'List all available record IDs with their names' (9 words) and 'SELECT * FROM users WHERE email = ?' (7 words) lack context on when/why to call these tools vs alternatives.
No input parameter descriptions visible in source. 'id' parameter in get_record has description 'Record ID (e.g. rec_001)' but example values in descriptions violate best practice, LLMs reuse examples literally. Other tools lack parameter docs entirely.
No output schemas documented. Agents cannot plan downstream calls or extract required fields. send-message returns dict with 'status', 'request_id', 'recipient', 'subject' but this is inferred from code, not declared in tool metadata.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 45 | 2026-07-28+ | v2 |
Error handling lacks recovery guidance. send-message raises HTTPException(408) on timeout and HTTPException(403) on rejection, but responses do not suggest next steps (retry, escalate, ask user). Bare HTTP status codes give LLM no actionable path.
Tool naming inconsistency: 'send-message' uses kebab-case while others use snake_case. Inconsistent naming increases LLM confusion when many tools are available. Standardize to snake_case per MCP convention.