Loading ToolBench
Single tool with basic registration but significant quality gaps. Tool name is clear and action-oriented ('get_vault_contents'), but description is minimal (Japanese language only, ~30 chars in English equivalent). Input schema present but incomplete: the 'path' parameter has a description and default value, but no explicit 'required' field declaration. No output schema documented, the tool returns JSON stringified vault contents, but LLMs have no formal documentation of what fields to expect. Error handling exists but is reactive rather than preventive (handles 404 after the fact, but no input validation guidance). Security concern: HTTPS client configured with rejectUnauthorized=false, creating a production risk. The tool is read-only and does one thing, which is good, but lacks the polish needed for agent-driven workflows.
Obsidian Vaultの内容を取得
No output schema documented. LLMs cannot infer what fields vault contents will contain (e.g., files vs folders, properties of each entry, metadata structure). Tool returns JSON.stringify(response.data), an opaque string to the LLM.
Description is minimal and in Japanese. English description translates to ~30 characters ('Get Obsidian Vault contents'). Lacks WHEN to use, WHY, and any prerequisite info. Should explain: is this for reading files, listing directories, or both? What data structure is returned? What happens with large vaults?
Input schema incomplete. 'path' parameter has description and default (''), but schema does not declare 'required: []' or 'required: ["path"]'. Ambiguous whether 'path' is optional or required. LLMs cannot determine if omitting 'path' will list root or error.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 51 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 36 | - | v1 |
No parameter format or constraint documentation. 'path' accepts a string but tool description does not explain: must it be relative or absolute? What characters are invalid? Should it use forward or backslashes? Tool code normalizes Windows paths, but LLM has no guidance.
HTTPS client misconfiguration: rejectUnauthorized=false in src/api-client.ts. This disables SSL certificate validation, creating a man-in-the-middle vulnerability in production. Should validate certificates or document this as dev-only.
No error recovery guidance. Tool returns 'Directory not found: {path}' on 404, but does not suggest next steps (e.g., 'Call with path="" to list root', or 'Use search tool to find similar paths'). Error should guide the LLM toward recovery.