Loading ToolBench
A Model Context Protocol server for accessing GitHub repositories containing Obsidian vaults
This server has 9 tools with READ-ONLY risk profiles, all accessing GitHub-hosted Obsidian vaults. Tool naming follows verb_noun convention well (searchVault, getFileContent, listVaultFiles, getCommitHistory, analyzeVaultStructure, diagnoseSearch, getRepositoryInfo, getFileHistory, searchIssues). Descriptions are present for all tools and range from adequate to good (60-150 chars typically). However, critical gaps emerge: (1) Parameter descriptions are minimal or missing in several tools, maxResults, searchIn, maxCommits lack detail on valid ranges or constraints. (2) No output schemas are documented anywhere in the source code; LLMs cannot infer what fields to expect from tool responses. (3) Input schemas visible in the tool list are incomplete (missing type definitions for some parameters, e.g., no explicit type for 'path' in listVaultFiles). (4) Error handling is minimal, no recovery guidance, no error classification, no actionable error messages. (5) No tool annotations (readOnlyHint, idempotentHint) despite all tools being read-only. Tools are well-scoped (one concern each) and follow good naming practices, but lack the robustness needed for production. The server does properly declare all tools read-only via risk metadata, which is good for security communication.
Analyze the structure and statistics of your Obsidian vault
Run diagnostics on the GitHub search functionality for your repository
Get commit history for a specific file or the entire repository
Get the content of a specific file from your Obsidian vault
Get detailed history and changes for a specific file
Get metadata about your Obsidian vault repository
List all markdown files in your Obsidian vault repository
Output schemas are completely undocumented. No tool in the source code defines what fields LLMs should expect from responses. This forces LLMs to guess at return structure and breaks tool chaining (they don't know what IDs/fields are available for follow-up calls).
Parameter constraints and ranges are missing or vague. 'maxResults' and 'maxCommits' have no explicit min/max bounds, allowing LLMs to pass arbitrary values (e.g., 999999). 'searchIn' enum is declared but lacks description of what each value does. No validation error messages shown.
Many parameter descriptions are missing or trivial. 'path' in listVaultFiles has no description explaining what path format is expected (absolute? relative? root=vault root?). 'query' in diagnoseSearch and searchIssues lack detail on supported syntax and examples.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 48 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Search for issues in your Obsidian vault repository
Search for files and content in your Obsidian vault using GitHub's code search
No error handling or recovery guidance. Tools lack descriptions of what errors are possible, when they're retryable, and what the LLM should do next. The GitHub API can fail in many ways (rate limit, auth, not found, large repo search timeout), none are communicated.
No tool annotations. All 9 tools are read-only and idempotent, they should declare readOnlyHint=true to help clients optimize caching and access policies. Currently no annotations present.
Configuration validation is weak. The server starts without GitHub credentials (githubToken, owner, repo optional at startup), logging an 'info' message. Any tool call with missing config throws an error. LLMs will encounter cryptic failures with no guidance on how to provide credentials. Should validate at startup or fail with clear instructions.
No pagination support documented. searchVault and searchIssues accept 'maxResults' but don't explain pagination behavior (e.g., how to get the next page if results exceed maxResults). Large result sets could hit context window limits.