Loading ToolBench

Reading the current corpus…

agent-smithers MCP server · F · ToolBench

← Browse

agent-smithers

github.com/h1ddenpr0cess20/agent-smithers

Modular Matrix bot with multi-provider LLM support, tool calling, MCP server integration, and end-to-end encryption

<=2025-11-257 tools

42/100

Definition Quality52

Protocol Readiness22

Supportability40

Spec posture

Inferred effective spec <=2025-11-25·current baseline 2026-07-28

Static source inference · medium confidence · detected: Logging

Deprecated protocol patterns detected

Summary

This server exhibits significant definition quality gaps. Tool descriptions are present but minimal (10-50 chars, well below the 194-char baseline). Input schemas are visible but lack depth: parameters have types and basic descriptions, but no enums, ranges, or format constraints. The code_interpreter tool exposes dangerous functionality without safety guardrails or dry-run capability. The mcp tool accepts arbitrary server URLs without validation. No output schemas are documented. Error handling is absent, no recovery guidance, categorization, or actionable messages. Tool naming is reasonable (verb_noun pattern), but composition issues exist: grok_edit_image and grok_generate_image should clarify their distinct purposes more explicitly. Security concerns are severe: code_interpreter allows arbitrary Python execution; no mention of sandboxing, timeouts, or permission gates.

Tools (7)

code_interpreterwriteauthsource verified45/100

Code execution tool for running Python code

grok_edit_imagewriteauthsource verified52/100

Edit existing images using Grok image editing

grok_generate_imagewriteauthsource verified57/100

Generate images using Grok image generation

grok_generate_videowriteauthsource verified58/100

Generate videos using Grok video generation

mcpwritesource verified45/100

MCP server tool for calling external MCP servers

web_searchread onlyauthsource verified55/100

Web search tool for querying the internet

x_searchread onlyauthsource verified52/100

X (Twitter) search tool for querying posts and trends

Issues (5)

CRITICALconfirmation-request, permission-gate, secret-injection ↗

code_interpreter tool allows arbitrary Python execution with no sandboxing, timeout, or permission gate. Descriptions lack safety warnings. No dry-run or confirmation step before execution.

code_interpreter

CRITICALtool, response-shaper ↗

No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields. Violates pattern:tool and pattern:response-shaper.

web_searchx_searchcode_interpretergrok_generate_imagegrok_edit_imagegrok_generate_videomcp

HIGHtool-description ↗

Tool descriptions are 10-50 characters (well below 194-char baseline). Descriptions lack WHAT/WHEN/WHY context. LLMs cannot determine when to select each tool over alternatives.

x_searchcode_interpretergrok_edit_imagemcp

Recommendations

Document output schemas for all tools. Specify field names, types, and whether results are paginated. Example: web_search should return {results: [{title, url, snippet}], total_count, next_cursor}.
Expand tool descriptions to 100-200 characters. Include WHAT (action), WHEN (use case), and any prerequisites. Example: 'Search the web for current information. Use when you need real-time data, news, or facts not in your training data. Returns top 10 results with snippets.'
Add parameter constraints: enums for aspect_ratio/resolution (already defined in code constants), format patterns for URLs, and ranges for numeric limits. Replace free-form strings with validated enums.
Implement safety guardrails for code_interpreter: add a dry_run parameter, enforce execution timeouts (e.g., 30s), sandbox output, and require explicit user confirmation before running untrusted code.
Add error handling guidance to tool descriptions. Example: 'If search fails, try a simpler query. If the MCP server is unreachable, check the server_url and try again in 30 seconds.'

Spec posture evidence

Inferred effective spec: <=2025-11-25.

Relies on Logging (deprecated) - log to stderr or use OpenTelemetry

Score history

Overall score trend

First recorded score · v2 rubric

42/100

Scored	Grade	Overall	Spec posture	Rubric
2026-09-23	F	42	<=2025-11-25	v2