Loading ToolBench

Reading the current corpus…

DevoChat MCP server · D · ToolBench

DevoChat

A FastAPI-based chat application with MCP (Model Context Protocol) server integration for tool execution, supporting multiple AI providers (OpenAI, Anthropic, Google, Grok, OpenRouter) and image generation services.

<=2025-11-25STREAMABLE_HTTP1 tools

55/100

Definition Quality58

Protocol Readiness62

Supportability45

Spec posture

Inferred effective spec <=2025-11-25·current baseline 2026-07-28

Static source inference · medium confidence · detected: Logging

Deprecated protocol patterns detected

Summary

DevoChat exposes a single tool 'visit_url' with a basic but incomplete definition. The tool has a verb-starting name and a reasonable description (120+ chars), but the schema is minimal, parameter descriptions lack detail, and there is no documented output schema. The tool lacks error handling guidance, security annotations, and clarity about what the response structure will be. With only one tool visible and significant gaps in schema documentation and error guidance, this falls squarely into the 'Fair to Poor' range.

Tools (1)

visit_urlread onlysource verified58/100

Visits a URL, validates it for security (checks for localhost and private IPs), follows redirects, parses HTML content using BeautifulSoup, removes script/style tags, and returns cleaned text content.

Issues (5)

CRITICALtool ↗

No output schema documented. Tool description states it 'returns cleaned text content' but does not specify the structure of the response (string? object with fields?). LLMs cannot plan downstream calls or extract fields without knowing the output shape.

visit_url

HIGHconstrained-input ↗

Parameter 'url' description lacks actionable constraints. States 'Max 20000 characters' but does not explain: what happens if URL exceeds limit? What format is required beyond 'HTTP or HTTPS'? Are query parameters allowed? Does it follow redirects (which the description mentions in the tool summary but not the param)?

visit_url

HIGHrecovery-guide ↗

No error handling guidance. Tool description mentions security validation (localhost/private IP checks) but does not document what happens if validation fails, what error message is returned, or what the LLM should do next (retry? ask user for a different URL?). This violates recovery-guide pattern.

visit_url

Recommendations

Add explicit output schema: { 'type': 'object', 'properties': { 'content': { 'type': 'string', 'description': 'Cleaned text content from the URL' }, 'status_code': { 'type': 'integer' }, 'final_url': { 'type': 'string', 'description': 'Final URL after redirects' } } }
Expand url parameter description to include format constraints, max size behavior, and examples: 'The URL to visit. Must be HTTP or HTTPS. Max 20000 characters. If exceeded, returns error. Supports redirects (max 5 hops). Localhost and private IP ranges (10.x, 172.16 - 31.x, 192.168.x) are blocked for security.'
Document error conditions with recovery guidance: 'If URL is blocked (localhost/private IP), the agent should ask the user for a different URL. If parsing fails, the response will include error_message field; the agent should inform the user of malformed content.'
Add timeout constraint to description: 'Requests that do not complete within 30 seconds return a timeout error. Retry after requesting a smaller or faster URL.'
Add size limit to response: 'Content exceeding 100KB is truncated. If truncation occurs, the response includes truncated: true flag.'
Clarify script/style tag removal behavior: 'JavaScript and CSS blocks are removed; only text content is returned. If content is image-only or binary, an error is returned.'
Consider adding tool annotations (readOnlyHint: true) to indicate this is a read-only tool that does not modify state.

Spec posture evidence

Inferred effective spec: <=2025-11-25.

Relies on Logging (deprecated) - log to stderr or use OpenTelemetry

Score history

Overall score trend

↑ 26 points across a rubric change (v1 → v2)

55/100

Scored	Grade	Overall	Spec posture	Rubric
2026-09-22	D	55	<=2025-11-25	v2
2026-03-09	F	29	-	v1