Loading ToolBench
MCP server for querying Markdown frontmatter with DuckDB SQL
Server has 8 tools with complete input schemas and descriptions. Naming follows verb_noun convention (query, update, batch_update). Descriptions are present (avg ~120 chars) but lack actionable context for LLM selection. Parameters are typed but lack constraints (enums, ranges). Output schemas are documented in docstrings but not formally in tool registration. Error handling is minimal, no recovery guidance or categorization. Security: path traversal validation present in _resolve_path(), but glob patterns are not sanitized against directory escape. Composition is reasonable: separate tools for single vs batch operations, but batch_array_add is overly specific.
Add a value to an array property in multiple files.
Update frontmatter properties in multiple files matching glob pattern.
Refresh the semantic search index (differential update).
Get the status of the semantic search index.
Wait for semantic search indexing to complete.
Query frontmatter with DuckDB SQL.
Get frontmatter schema from files matching glob pattern.
Descriptions lack actionable context for LLM tool selection. E.g., 'Get frontmatter schema from files matching glob pattern' does not explain WHEN to call query_inspect vs query, or what the schema is used for downstream.
Parameters lack constraints. 'glob' accepts any string; no validation that it is a valid glob pattern or safe from directory traversal. 'timeout' in index_wait has no min/max bounds (could be negative or 999999). 'sql' in query is free-form with no injection guidance.
Error handling is absent. No recovery guidance, categorization, or actionable messages. E.g., if a glob matches no files, does the tool return empty results or an error? If SQL is invalid, what does the LLM see?
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 67 | 2026-07-28+ | v2 |
Update frontmatter properties in a single file.
batch_array_add is overly specific and combines two concerns: adding to arrays AND managing duplicates. Consider splitting into add_to_array (simple append) and merge_array_values (dedup logic) for better composability.
Output schemas are documented in docstrings but not formally declared in tool registration. LLMs cannot parse docstring examples to infer response structure. Formal schema declarations (via tool metadata or response type hints) are needed.