Loading ToolBench
MCP server for embedded systems documentation - search registers, memory maps, and datasheets
Server has 5 tools with clear verb-noun naming and present descriptions. However, critical gaps exist: (1) NO input schemas are explicitly visible in the source code, parameter definitions exist in function signatures but are not registered with JSON Schema in the FastMCP decorator, meaning LLMs cannot parse constraints or types; (2) output schemas are completely undocumented, tools return strings but what those strings contain (JSON? Free text? Structured format?) is not specified; (3) error handling is absent, no recovery guidance, error categorization, or actionable messages documented; (4) parameter descriptions exist in docstrings but are minimal and lack constraint details (ranges, enums, format requirements); (5) two tools (ingest_docs, remove_docs) perform destructive/write operations but have no confirmation, dry-run, or destructive hints. Per-tool analysis shows naming is consistently strong (verb_noun pattern, 10-18 chars), but schema registration and output documentation are the primary blockers.
Find a specific hardware register by name. Returns detailed register definition including address, bit fields, and descriptions.
Ingest a documentation file into the search index. Extracts text, detects register tables, creates embeddings, and makes the document searchable. Currently supports PDF files. This operation may take several minutes for large documents.
List all documentation files with their status. Shows indexed documents with statistics (chunks, tables) and available PDF files ready for ingestion (pages, size).
Remove a document from the search index by its document ID.
Search documentation using hybrid keyword and semantic search. Returns relevant sections and register definitions from indexed embedded systems documentation.
No input schemas visible in source code. FastMCP decorators show function signatures but no explicit JSON Schema registration with type/description pairs. LLMs cannot parse parameter constraints, enums, or formats, they must infer from descriptions alone.
Output schemas completely undocumented. All 5 tools return 'str' but the content structure (JSON object? Array? Free text? Error format?) is not specified. LLMs cannot reliably parse responses or extract chaining IDs.
No error handling documentation. Tools have no recovery guidance, error categorization (retryable vs fatal), or actionable error messages. A failed search returns an unspecified error with no guidance on what the LLM should do next.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 46 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 55 | - | v1 |
Destructive operations lack confirmation or dry-run support. remove_docs irreversibly deletes indexed documents; ingest_docs overwrites existing embeddings. No confirmation step documented, inviting catastrophic agent errors.
Parameter descriptions lack constraint details. 'doc_path' has no format (absolute? relative? glob patterns allowed?). 'top_k' has no range bounds (1 - 1000? 1 - 100?). 'doc_filter' omits whether it is a doc_id or filename. Insufficient for LLM validation.
No tool annotations (readOnlyHint, destructiveHint). Per spec, tools should declare safety properties. search_docs/find_register/list_docs should have readOnlyHint:true; ingest_docs and remove_docs should have destructiveHint:true.
remove_docs description is extremely terse (52 chars). No guidance on impact, recovery, or whether a removed document can be re-ingested. Below 100-char guideline for destructive operations.
list_docs has no pagination documented. If many PDFs exist, the response could be huge. No mention of how results are structured, whether counts are included, or how to handle large document libraries.