Loading ToolBench
The server demonstrates solid foundational structure with 8 well-named tools (all verb-prefixed), documented schemas for all tools, and parameter descriptions present. However, multiple critical gaps prevent a higher score: (1) Tool descriptions are extremely terse (8-20 chars for most tools, well below the 34-392 baseline), failing to explain WHEN to use tools or WHAT they return; (2) Parameter descriptions are similarly minimal and lack actionable context (no examples of constraints, formats, or valid ranges); (3) Output schemas are completely undocumented, no indication of response structure, field names, or what data flows between tools; (4) Error handling is absent from tool definitions; (5) No tool annotations (readOnlyHint/destructiveHint/idempotentHint) despite clear risk classifications in metadata. The fastmcp framework provides a solid base, but tool definitions themselves need substantial enrichment to meet production agentic standards.
Generate a SWAG configuration from a Jinja2 template
Create a new SWAG reverse proxy configuration
Get details of a specific SWAG reverse proxy configuration
Display help information about available SWAG actions
List SWAG reverse proxy configurations
Remove a SWAG reverse proxy configuration
Update an existing SWAG reverse proxy configuration
Validate SWAG NGINX configuration syntax
Tool descriptions critically under-threshold. 'Create a new SWAG reverse proxy configuration' (8 words, ~55 chars) lacks context for LLM selection. Production baseline is 34-392 chars with explicit WHEN, WHY, and WHAT-YOU-GET guidance. No tool exceeds 15 words.
Output schemas completely undocumented. No indication of response field names, types, or structure for any tool. LLMs cannot plan downstream calls or extract required data. swag_create should document: does it return config_id? config_name? status? validation_result? Required for tool chaining and context efficiency.
Inferred effective spec: 2025-06-18+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 51 | 2025-06-18+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Parameter descriptions lack actionable constraint information. Example: 'Name of the configuration file (without extension)', no mention of allowed characters, length limits, or validation rules. 'Port number for the upstream service', no range (1-65535?). LLMs cannot validate inputs without explicit constraints in descriptions.
No tool annotations (readOnlyHint/destructiveHint/idempotentHint) despite clear risk classifications in metadata. swag_create, swag_update are WRITE; swag_remove is DESTRUCTIVE. These should emit proper tool annotations so agents understand safety implications without reading metadata external to tool definition.
Error handling guidance absent. Tools like swag_remove and swag_update don't indicate what errors are possible (file not found? permission denied? validation failed?) or what the agent should do next. 'Create a backup before updating' is a feature description, not error recovery guidance.
swag_help tool purpose is vague. 'Display help information about available SWAG actions', what does this return? Is it for discovery? It should state: 'Returns list of available SWAG configurations and supported operations. Call before create/update if unsure of valid config names.'
swag_list filter enum values ('all', 'active', 'samples') lack documentation. What distinguishes 'active' from 'all'? What are 'samples'? Are these pre-installed configs? LLMs cannot reason about when to use each filter without explanation.
generate_config_from_template 'template_vars' parameter is type 'object' with minimal description. No guidance on required vs optional keys, expected structure, or validation. LLMs will guess at structure and pass invalid payloads. Should enumerate required keys or link to template documentation.
No idempotency declarations. swag_create with same config_name, does it fail, update, or skip? swag_update, is it idempotent? Agents retry on ambiguous failures; non-idempotent tools risk duplicate configs or overwrites without warning.
Tool composition assumes agent has config_name available, but no discovery tool returns a canonical config_name format for reuse in downstream calls. swag_list returns configs but undocumented field names. Does it return config_name or just config? If agents must parse output to extract the right field, tool chains break.