Loading ToolBench
Personal system knowledge API & MCP server
Server exposes 11 tools via FastMCP with HTTP transport. All tools have descriptions (10-200 chars, baseline 194) and input schemas with typed parameters. Naming follows verb_noun convention (get_, search_, list_, impact). However, descriptions lack LLM-optimized guidance on WHEN to use each tool vs. alternatives (e.g., search_docs vs search_code vs search_commits distinction unclear). Parameter descriptions are present but minimal (e.g., 'Search query text' lacks format/length constraints). Output schemas are not explicitly documented in the source, responses inferred from route handlers. Error handling and recovery guidance absent. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite all tools being READ_ONLY. Missing pagination guidance for search tools that return lists.
Account-level facts: age, bio, and public/private repo and follower counts. `found=False` means no sync has populated it yet, not that the account is empty.
What a project declares it depends on - the forward direction of the graph.
Metadata and tech stack for a single project by name.
Canonical name, repo link, and generic per-repo stats (age, stars, language, activity) - for citing an exact source and its basic facts, not recalling them from prose. `projects` narrows to those names; omitted, returns all.
Most recent commits by date, not relevance - no query text, no embedding call.
Health check endpoint
Output schemas not documented. Tool descriptions state what is returned (e.g., 'returns all' for get_project_links) but response structure, field types, and pagination behavior are not formally specified. LLMs cannot plan downstream tool calls without knowing output shape.
Search tools (search_docs, search_code, search_commits) lack pagination guidance. No mention of limit defaults, max results, or next_cursor/offset behavior. Baseline pattern requires pagination for list-returning tools.
Parameter descriptions are minimal and lack format/constraint guidance. E.g., 'Search query text' does not specify min/max length, character restrictions, or semantic requirements. 'Optional ISO datetime start bound' lacks timezone clarification. LLMs cannot validate inputs without explicit constraints.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 67 | 2026-07-28+ | v2 |
Analyze impact of changes to a project interface on dependent projects
List projects, optionally filtered by technology (e.g. 'postgres', 'fastapi').
Semantic search over code chunks (L2), optionally scoped to one project.
Semantic search over commit history (L4), optionally time- and project-scoped.
Semantic search over README/docs content (L1), optionally scoped to one project.
No tool annotations despite all tools being READ_ONLY. FastMCP supports readOnlyHint, destructiveHint, and idempotentHint annotations. Adding readOnlyHint=true to all tools signals safety to agents and enables optimizations.
Descriptions lack WHEN-to-use guidance. Three search tools (search_docs, search_code, search_commits) have similar names and purposes. Descriptions do not clarify which layer (L1 docs vs L2 code vs L4 commits) to query for different intents, forcing LLMs to guess or try all three.