Loading ToolBench
Agent-first MCP server: AI coding agents find, compare, verify, and stack-build the right open-source tools across 35+ registries (npm, PyPI, Cargo, Maven, Go, RubyGems, NuGet, Hex, Composer, and more). Graph-aware ranking + version-aware compatibility in 16 tools.
ToolCairn MCP has 16 tools with complete input schemas and descriptions, but quality is uneven. Most tools have adequate descriptions (100-250 chars), but several lack parameter-level detail. Naming is generally verb-first and clear (search_tools, verify_suggestion, check_compatibility). However, parameter descriptions are sparse or generic in many tools (e.g., 'data' in update_project_config is just 'Action-specific data payload' with no structure hint). Output schemas are not documented in the provided source. Error handling guidance is absent, tools do not indicate what to do on failure or how to recover. Composition is reasonable: tools chain logically (init → read_config → update_config), but some tools conflate multiple concerns (toolcairn_init does parsing, classification, AND atomic writes in one call).
Check if tool A is compatible with tool B: version constraints, peer dependencies, known conflicts, and graph edges.
Check if a tool has known issues or bugs related to a specific problem.
Classify a developer prompt to determine if ToolCairn tool search is needed. Returns a structured classification prompt for the agent to evaluate.
Compare two or more tools side-by-side: features, versions, compatibility, maturity, license, and graph relationships.
Submit general feedback about ToolCairn search results or recommendations.
Get a recommended stack of tools for a given use case or requirement.
Output schemas not documented. Tools like search_tools, compare_tools, and get_stack return complex results (ranked candidates, compatibility info, stacks) but no schema is provided in source. LLMs cannot plan downstream calls or extract fields without knowing the response structure.
Parameter 'data' in update_project_config is vague ('Action-specific data payload'). No schema or structure hint provided. LLMs cannot infer what fields to pass for add_tool vs remove_tool vs add_evaluation actions.
No error handling guidance. Tools do not document failure modes, recovery steps, or actionable error messages. E.g., search_tools does not explain what happens if the query is malformed or if no results match.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 62 | <=2025-11-25 | v2 |
Read .toolcairn/config.json from disk and return the structured project snapshot: project metadata, confirmed tools, stale tools, pending evaluations, and last audit entry. Auto-migrates v1.0 configs to v1.1 on first read.
Refine a developer requirement into a structured search query for ToolCairn.
Report the outcome of a tool recommendation: whether it worked, was replaced, or failed.
Search the ToolCairn graph for tools matching a requirement. Returns ranked candidates with compatibility info.
Respond to a search_tools result with user feedback or selection.
Suggest a new tool or relationship to add to the ToolCairn graph. Entries are staged for admin review.
Sign in to ToolCairn. Opens a browser for authentication — required before any other tools are available. Call with action="login" to start.
Bootstrap ToolCairn for the current project. Walks every workspace, parses manifests across 12 ecosystems, classifies tools against the ToolCairn graph, and writes .toolcairn/config.json + audit-log.jsonl atomically. Returns setup_steps for CLAUDE.md / .mcp.json / .gitignore (agent applies those).
Apply a mutation to .toolcairn/config.json: add_tool, remove_tool, update_tool, or add_evaluation. Atomically updates config and appends audit-log entry.
Verify a tool suggestion: check if it exists in the graph, is maintained, and matches the requirement.
toolcairn_init conflates multiple responsibilities: workspace walking, manifest parsing, tool classification, and atomic config writes. Should split into separate tools (parse_manifests, classify_tools, write_config) so agents can compose them independently.
toolcairn_auth requires browser-based login but no guidance on when/how agents should call it. Description says 'required before any other tools are available' but does not explain the flow: should agents call status first? What happens if login fails?