Loading ToolBench
MCP server for agile team with model wrapper tools
This server defines 9 tools with explicit schemas and descriptions. However, the implementation has significant issues: (1) Tool names are inconsistent, 'prompt_tool', 'prompt_from_file_tool', 'prompt_from_file2file_tool' lack the standard verb_noun clarity pattern. Names like 'prompt_from_file2file' are non-standard (should be 'prompt_file_to_file' or similar). (2) Parameter descriptions are mostly present but generic, many lack actionable format guidance or constraints. For example, 'models_prefixed_by_provider' says 'in format "provider:model"' but does not validate or list allowed providers upfront. (3) No output schema documentation visible, tools return List[str], Dict[str, List[str]], or bare str without describing the structure of returned data (e.g., what fields are in persona documents?). (4) Error handling is minimal, no recovery guidance, no categorization, no actionable error messages visible in source. (5) Security concern: the tools accept file paths and LLM API credentials without visible input sanitization or secret injection pattern. (6) The persona tools (persona_dm, persona_ba, persona_pm, persona_sw) are complex with many optional parameters and interdependencies not clearly documented. Overall: solid structure, but descriptions need LLM-specific refinement, output schemas need documentation, and error handling is absent.
List all available models for a specific provider.
List all supported LLM providers.
Generate business analysis using a specialized Business Analyst persona, with optional decision making.
Generate responses from multiple LLM models and use a decision maker model to choose the best direction.
Generate product management plans using a specialized Product Manager persona, with optional decision making.
Generate software specifications using a specialized Software Engineer persona, with optional decision making.
Tool naming does not follow verb_noun pattern consistently. 'prompt_from_file2file_tool' uses non-standard underscore/numeral style; 'prompt_tool' and similar are generic. Names should be 'send_prompt', 'prompt_file_to_file', 'list_providers' (without '_tool' suffix in MCP conventions). LLMs struggle to parse intent from unclear names.
Output schemas are not documented. Tools return List[str], Dict[str, List[str]], or str without describing what the returned data contains. For example, persona_dm_tool returns a file path string, but the description does not document the structure of the persona output file (is it markdown? JSON? What fields?). This forces LLMs to guess at downstream processing.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 45 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 31 | - | v1 |
Read a prompt from a file, send it to multiple LLM models, and write responses to files.
Read a prompt from a file and send it to multiple LLM models.
Send a text prompt to multiple LLM models and return their responses.
Parameter 'models_prefixed_by_provider' lacks actionable format constraints. Description states format is 'provider:model (e.g., "openai:gpt-4")' but does not enumerate valid providers (openai, anthropic, gemini, etc.) or validate that the LLM will not hallucinate invalid provider names. Should include enum constraint or list of accepted providers in description.
No error handling or recovery guidance visible. Tools do not document what happens if a model call fails, a file is not found, an API key is missing, or the LLM provider is unavailable. Error responses should categorize errors as retryable, user-fixable, or fatal, and guide the LLM on next steps.
Complex persona tools (persona_ba_tool, persona_pm_tool, persona_sw_tool) have 10+ optional parameters with interdependencies (e.g., 'decision_maker_models' only applies if 'use_decision_maker' is true). These relationships are not explicitly documented, forcing LLMs to reason about valid parameter combinations.
File path parameters ('file_path', 'output_dir', 'output_path') are exposed without visible input sanitization. No path traversal validation, no restrictions on write locations. Tools accept file paths directly from LLM input, malicious or confused LLMs could be tricked into reading/writing arbitrary files or exposing secrets.
Default values for optional array parameters ('models_prefixed_by_provider' defaults to ['openai:gpt-4o-mini']) are hardcoded, not configurable. If an LLM omits this parameter, all calls silently use the hardcoded default. This is not necessarily wrong, but the description should emphasize the default behavior to avoid silent misuse.