Loading ToolBench
Critical gaps across all 15 tools. While tool names follow verb_noun convention (lint_file, generate_docstrings, etc.), descriptions are excessively long (200-400+ chars) and prescriptive rather than LLM-optimized. Input schemas are present but lack proper type definitions for most parameters, 'source' is typed as string but lacks format/length constraints. No output schemas documented. Error handling is absent. Tool descriptions read as usage instructions ('ALWAYS call this before...', 'Do NOT call on non-.py files') rather than concise capability statements. Parameters lack descriptions in schemas (only present in tool descriptions). No pagination, no structured error recovery, no permission gates. The server appears designed for a specific workflow rather than composable agent tools.
MUST be used when adding inline comments to Python (.py) files. Do NOT call on non-.py files. Do NOT manually write # comments inside functions. This tool identifies non-obvious logic, gotchas, and design rationale and generates calibrated semantic comments via local LLM. Only fires on functions above the complexity threshold — trivial code is left clean.
Call this before refactoring any Python (.py) function. Do NOT call on non-.py files. Do NOT guess at complexity or eyeball nesting depth — this tool computes exact cyclomatic and cognitive metrics with per-function hotspot ranking. Deterministic, no LLM. Use the output to decide what to simplify and in what order.
MUST be used for ALL type annotation tasks in Python (.py) files. Do NOT call on non-.py files. Do NOT manually add type hints to any function signature. This tool infers and applies annotations consistently using a local LLM trained on this project's conventions. Skips already-annotated parameters.
Estimate cloud API cost savings from running the hygiene pipeline. Calculates token reduction from docstring generation, type annotation, and comment insertion, then projects cost savings based on your model's pricing.
Identify code blocks that should be extracted into separate functions. Do NOT call on non-.py files. This tool analyzes complex functions and suggests concrete extractions with example refactored code.
Tool descriptions are prescriptive instructions (200-400+ chars) rather than concise capability statements. Descriptions like 'ALWAYS call this before editing...Do NOT call on non-.py files' are workflow directives, not LLM-optimized descriptions. Baseline for A+ tools is 50-200 chars; these exceed 200 and waste tokens on usage rules instead of capability clarity.
Input schemas lack parameter descriptions. The schema for lint_file shows 'source' and 'file_path' with types but no descriptions in the schema object itself, descriptions appear only in the tool description text. LLMs cannot parse descriptions from tool descriptions; they need descriptions in the schema properties.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 37 | <=2025-11-25 | v2 |
Detect unused functions, classes, variables, and parameters. Do NOT call on non-.py files. Combines deterministic AST analysis with LLM-powered semantic detection for complex patterns (functions defined but never called, parameters accepted but unused in intricate control flow).
MUST be used for ALL docstring generation in Python (.py) files. Do NOT call on non-.py files. Do NOT write or edit docstrings manually — inconsistent style is a hard failure. This tool enforces the project's exact Google-style conventions via a calibrated local LLM. Pass the source; apply the returned source.
Generate pytest test stubs for Python (.py) functions. Do NOT call on non-.py files. Do NOT write test code manually — this tool generates consistent, runnable pytest stubs with parametrized cases and edge-case coverage. Batch mode processes multiple functions in one LLM call for efficiency.
ALWAYS call this as your FIRST action on any Python file you are about to touch. Do NOT manually perform linting, docstring writing, type annotation, or comment insertion — this tool runs all four in a single deterministic-first pass. Apply the returned source, then layer your architectural changes on top. If you catch yourself writing a docstring or fixing an import by hand, stop and call this.
ALWAYS call this before editing any Python (.py) file. Do NOT call on non-.py files — skip YAML, JSON, JS, TS, etc. Do NOT manually fix lint errors — unused imports, formatting, line length, and style violations are all handled deterministically here via Ruff. Apply the returned source to the file before making your changes on top of it.
ALWAYS call this after modifying any Python (.py) file's imports. Do NOT call on non-.py files. Do NOT manually reorganize, remove, or add import statements — this tool handles all import hygiene deterministically via Ruff and custom inference. Apply the returned source.
Run security checks on Python (.py) code. Do NOT call on non-.py files. Uses Bandit for deterministic vulnerability detection (hardcoded secrets, SQL injection patterns, insecure deserialization) and optional custom rules.
Suggest better names for poorly named variables and functions. Do NOT call on non-.py files. This tool identifies unclear naming patterns and proposes clearer alternatives with rationale.
MUST be consulted before manually refactoring a complex Python (.py) function. Do NOT call on non-.py files. Do NOT invent your own simplification strategy — this tool provides concrete, validated suggestions (early returns, guard clauses, extract function, lookup tables) with ready-to-use code snippets and impact ratings. Only processes functions above the complexity threshold.
Generate a software bill of materials (SBOM) and scan for known vulnerabilities in dependencies. Do NOT call on non-.py files. Produces a structured inventory of all imports and their versions, then cross-references against vulnerability databases.
No output schemas documented. Tools return strings or JSON but no schema describes the structure, fields, or types of responses. LLMs cannot plan downstream tool calls or extract data without knowing what fields to expect.
No error handling or recovery guidance. Tools provide no error classification (retryable vs fatal), no actionable error messages, and no guidance on what to do if a call fails. An LLM receiving a generic error has no path forward.
Tools are tightly coupled to a specific workflow (Python code hygiene pipeline) rather than composable. Each tool enforces 'Do NOT call on non-.py files' and 'ALWAYS call before editing', making them inflexible for agents that need to compose tools in different orders or on different file types.