Loading ToolBench

Reading the current corpus…

herbie MCP server · F · ToolBench

herbie

A Go-based chatbot server with user-configured HTTP API tools, OAuth integration, and MCP support. Provides chat functionality with tool execution, approval workflows, and credential management.

2026-07-28+STREAMABLE_HTTP1 tools

29/100

Definition Quality17

Protocol Readiness45

Supportability38

Spec posture

Inferred effective spec 2026-07-28+·current baseline 2026-07-28

Static source inference · medium confidence · detected: Dynamic Client Registration

Deprecated protocol patterns detected

Summary

Single tool 'user_http_tool' has critical definition gaps. Input schema visible but severely incomplete: only declares a generic 'parameters' object property with no type constraints, no parameter descriptions, no enum constraints, and no output schema documented. Description is present (67 chars) but lacks actionable guidance on when/how to use the tool, what it returns, or error recovery. Tool name is generic and does not follow verb_noun convention. No evidence of parameter validation rules, format constraints, or security guidance in the schema. This server exposes a dynamic HTTP tool without the structured definitions LLMs need to use it safely and effectively.

Tools (1)

user_http_toolwrite21/100

Dynamically configured HTTP API tool that executes user-authored HTTP requests with parameter substitution, body templating, and optional approval gating

Issues (5)

HIGHtool ↗

Tool name 'user_http_tool' is generic and does not follow verb_noun convention. LLMs cannot infer the action from the name alone. Should be 'execute_http_request' or similar action-oriented name.

user_http_tool

CRITICALconstrained-input ↗

Input schema declares only a generic 'parameters' object with no type definitions, no parameter descriptions, no enums, and no constraints. LLMs cannot determine what parameters are valid, required, or what format they expect.

user_http_tool

CRITICALtool ↗

No output schema documented. LLMs cannot plan downstream operations or extract relevant fields from HTTP responses. Tool description does not specify what fields are returned or their types.

user_http_tool

HIGHtool-description ↗

Tool description (67 chars) lacks WHEN to use it, WHAT it returns, and error recovery guidance. Does not explain how parameter substitution works, what body templating syntax is supported, or how approval gating behaves.

Recommendations

Rename tool to 'execute_http_request' or 'call_http_api' to follow verb_noun convention and signal the action clearly to LLMs.
Expand input schema to explicitly define each parameter: add 'type', 'description', 'required', 'enum' (if applicable), and 'pattern' (for format constraints). Example: 'method' should be an enum of [GET, POST, PUT, PATCH, DELETE]; 'url_template' should have a maxLength and pattern; 'body_template' should document the templating syntax (e.g., {{param_name}}).
Document the output schema in the tool description or as a separate schema field. Specify: HTTP status code, response body structure (JSON object with typed fields), headers returned, and any pagination fields.
Rewrite description to 100-150 chars covering: (1) What it does: 'Executes user-configured HTTP requests with parameter substitution and optional approval gating'; (2) When to use: 'Call this to invoke external APIs defined by the user'; (3) What it returns: 'HTTP response status, headers, and body'; (4) Prerequisites: 'Requires a valid HTTP method, URL template, and parameters matching the tool configuration'.
Add error handling guidance: document which errors are retryable (network timeouts, 5xx), which are user-fixable (4xx with invalid params), and which are fatal (permission denied). Provide recovery hints: 'If approval is required, the tool will return a pending status, ask the user to approve before retrying.'

Score history

Overall score trend

First recorded score · v2 rubric

29/100

Scored	Grade	Overall	Spec posture	Rubric
2026-09-23	F	29	2026-07-28+	v2