Loading ToolBench
Multi-language AI agent framework with LLM integration, tools, skills, event bus, MCP support, and ReAct reasoning engine
BrainOS exposes 10 tools across example files with significant quality gaps. Tool definitions are scattered across JavaScript and Python examples rather than centralized in the MCP server implementation. Schemas are present but descriptions are minimal (avg 45 chars, well below 194-char baseline). Multiple tools lack parameter descriptions entirely. Duplicate tool names (calculator, weather appear twice) indicate poor composition. No evidence of error handling guidance, output schema documentation, or security patterns. Tools are read-only with no destructive operations, but lack the structured metadata needed for production LLM integration.
Math calculator
Evaluate a mathematical expression
Evaluate a mathematical expression and return the result.
Get current UTC time
Duplicate test tool
Greet someone
Look up a user profile by their user ID. Simulates a database query.
Send a notification message to a recipient. Simulates an email/push notification.
Duplicate tool names (calculator, weather appear twice) violate single-responsibility principle and force LLM to disambiguate between identical-looking tools. Consolidate or rename.
Descriptions are critically short (avg 45 chars vs 194-char baseline). 'Math calculator', 'Greet someone', 'Get current UTC time' lack context for LLM selection. Expand to 50-200 chars explaining WHAT, WHEN, and WHY.
Parameter descriptions missing or minimal. 'city' in weather tool has description but 'expression' in calculator lacks detail on format/constraints. 'value' in dup_tool has no description. 100% of params need descriptions.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 48 | <=2025-11-25 | v2 |
Get current weather for a city
Get current weather information for a given city. Simulates an API call.
No output schema documentation. Tools return results but LLMs cannot plan downstream calls without knowing response structure. Document return types for all tools.
Tools defined in example files (agent_tool_boundary_demo.js, async_tool_demo.py) rather than centralized MCP server. No evidence of tool registration via MCP protocol. Cannot verify if this is a functional MCP server or just example code.