Loading ToolBench
The superargs server exposes a single tool 'update_args' with critically deficient definitions. The tool has an empty input schema (no properties, empty required array) despite being intended to accept dynamic arguments. The description is minimalist (12 chars: 'Updates args (tokens, etc).') and provides no guidance on what parameters are accepted, how the tool modifies server state, or what the output contains. The tool is registered programmatically with no visible output schema documentation. The code shows the tool receives 'args' from regex parsing of the stdio command, but the schema is not constructed to match, it shows 'properties: {}' even though the tool should accept those parsed argument names as parameters. This is a clear case of incomplete/mismatched schema and insufficient description.
Updates args (tokens, etc.).
Empty input schema: update_args declares 'properties: {}' and 'required: []' despite the code parsing named arguments (args) from the command template. The schema should list each parsed arg as a parameter with type 'string' and description. Currently, LLMs cannot determine what parameters this tool accepts.
Insufficient description: 'Updates args (tokens, etc).' is 12 characters and does not explain what the tool does, when to use it, or what happens when called. No mention that it restarts the stdio child process, no explanation of parameters, no return value documentation. Baseline for tool descriptions is 34 - 392 characters; this is critically short.
No output schema documented: The code shows the tool returns a result via 'CallToolResult', but the response structure is not visible. LLMs cannot predict what fields to expect or how to chain subsequent calls. The code snippet is truncated at the tool handler, so the complete return format is unknown.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 35 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 22 | - | v1 |
Schema does not match code intent: The code shows 'args.reduce((acc, v) => { acc[v] = { type: 'string', description: `...` } })' but the displayed schema shows 'properties: {}'. Either the schema construction failed or the code snippet is stale. Either way, the schema is inaccurate.
No error recovery guidance: If an arg update fails (e.g. invalid value, child process crash), the tool provides no guidance on what the LLM should do next. The code shows error rejection but no categorization (retryable vs user-fixable).