Loading ToolBench
High-level framework for building MCP (Model Context Protocol) servers in Go with minimal boilerplate. Auto-generates JSON schemas, handles transport, validation, and middleware.
hamr provides 30 tools across multiple examples (basic, devtools, k8s, postgres, middleware, multi-tenant). While many tools have basic descriptions and visible schemas, critical quality gaps prevent a higher score. Tool naming is generally verb-first (search, greet, reverse, run_command, read_file, write_file, list_dir, etc.), which is good. However, descriptions are inconsistent: most are under 100 characters (below the 194-char baseline), several lack context on when/why to use them, and parameter documentation is sparse. Output schemas are not visible in the provided code, we can infer input schemas from the tool definitions, but return types are undocumented. Error handling is minimal; there is no guidance in tool descriptions on how to recover from failures. Security concerns exist: run_command accepts arbitrary shell commands without obvious input validation, and there is no visible secret injection pattern for tools like http_fetch or postgres tools. The large number of tools (30) and diversity of domains (filesystem, git, k8s, postgres, http) suggests breadth but increases the burden on getting each tool right, most appear to be 'good enough' examples rather than production-ready.
Describe a Kubernetes resource in detail
Show columns, types, and defaults for a table
Get recent cluster events in a namespace
Get logs from a pod
List pods in a namespace (with optional label selector)
List any resource type (deployments, services, etc.)
Show git diff (staged or unstaged)
STDIO-only transport: not remotely accessible, cannot be tested with hosted MCP clients or production agent orchestrators.
Output schemas not documented: return types are inferred from tool names but never formally specified. LLMs cannot plan downstream tool calls or extract needed fields without seeing output structure.
run_command accepts arbitrary shell commands with minimal apparent input validation. No visible input sanitization against command injection. Agents could be tricked into executing dangerous commands.
Duplicate tool name 'greet': appears twice (tools #2 and #24) with identical signatures but different descriptions. LLMs will be confused about which to select.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 53 | <=2025-11-25 | v2 |
Show recent git commits
Show git repository status
Greet a user with a personalised message.
Greet a person by name
Fetch a URL and return its content
List files and directories
List all tables in the public schema with sizes
Look up a value by key. Results are cached for 1 minute.
Run a read-only SQL query (SELECT/WITH/EXPLAIN only)
Read the contents of a file
Generate a report (pro plan only).
Reverse a string
Trigger a deliberate panic when trigger=true to demonstrate the Recovery middleware.
Execute a shell command and return output
Search for information
Search for a regex pattern in code files (grep)
Search a table column for a value (partial match)
Find files matching a glob pattern
Sleep for the specified number of seconds. The global 10-second timeout will interrupt long sleeps.
Get row count and size for a table
Show resource usage (CPU/memory) for pods or nodes
Count words in text
Write content to a file (creates directories if needed)
Many tool descriptions are under 50 characters and lack context on when/why to use them. Examples: 'Greet a person by name' (28 chars), 'Reverse a string' (16 chars), 'Count words in text' (19 chars). This violates the 50 - 200 char LLM-optimized guideline.
No error handling guidance in tool descriptions. Tools like run_command, query, and http_fetch can fail but offer no recovery hints. Descriptions should say 'If command fails, check syntax' or 'Invalid SQL: check your SELECT statement'.
No visible secret injection pattern. Tools like http_fetch and postgres query tools likely need authentication but parameters are visible. Secrets must be injected server-side, not passed as parameters.
File system tools (read_file, write_file, search_files) lack visible path traversal and symlink escape protections in descriptions. The test file shows these are implemented, but tool descriptions do not mention the sandbox guarantee.
Parameter descriptions lack format constraints. Examples: run_command 'dir' parameter has no default or example path; search_code 'glob' is optional but no guidance on what happens if omitted; query has no comment on SELECT-only restriction (mentioned in description but not enforced in schema).
Pagination not visible in tool definitions. Tools like list_dir, search_files, get_pods, and query likely return unbounded lists but no limit or offset parameters are defined. Large result sets will exhaust context windows.