Loading ToolBench
safe-mcp is a security framework for MCP servers, not a production tool server. The 6 example tools in examples/example.py are proof-of-concept demonstrations of decorator patterns (@safe, @unsafe, @sanitize, @validate_inputs), not a real MCP service. Tool descriptions are minimal (20-65 chars), parameters lack detailed descriptions and constraints, and output schemas are undocumented. The tools exist only in the example file and are not formally registered in a production server context. Framework quality is reasonable for a library, but tool definitions fall below production standards. Per the hard scoring rule: 'If you cannot see the actual tool definition in the source (only inferred)', these tools are defined in an example file, not a live server, capping their practical score.
Add two positive numbers with input validation
Fetch data from an external API
Fetch data from a trusted internal source
Handle user input with chained sanitization and validation decorators
Process text with custom sanitizer using encoding filter
Simulate user-generated content search with sanitization
Tool descriptions are too brief (20-65 characters) and lack action context. Examples: 'Fetch data from a trusted internal source' (47 chars) does not explain WHEN to use it vs fetch_external_api, what it returns, or its prerequisites. Baseline for A+ tools is 50-200 chars with clear WHAT/WHEN/WHY.
Parameter descriptions are minimal or missing. 'user_id: User identifier' (19 chars) and 'query: Query parameter' (21 chars) are generic. No indication of format, length, valid values, or examples. Baselines: 94% of A+ tools have full param descriptions of ~72 chars.
Output schemas are not documented. Examples return SecuredResponse (wrapping data with trust_level enum) but the schema is not visible in the tool definition, only in the decorator. LLMs cannot know to expect {data, trust_level, warnings}. Baseline: 100% of A+ tools document return types.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 46 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
No input validation or error handling guidance in tool descriptions. 'add_positive_numbers' enforces is_positive constraint via decorator but does not document what error message the LLM receives if it passes negative values. Baseline: errors must tell the LLM what to do next.
Parameter 'x' in handle_user_input has inline constraint '(max 100 characters)' in description but no formal JSON Schema maxLength or pattern. LLMs cannot reliably read inline constraints, they need machine-parseable schema properties.
Tools are example-only, defined in examples/example.py, not in a production server entry point. No evidence of a running FastMCP server, actual tool registration, or deployment context.