Loading ToolBench
Reference-based caching for FastMCP servers with namespace isolation, access control, and private computation support
mcp-refcache presents a reference-caching library with 19 tools across multiple example servers. Strengths: most tools have descriptions and input parameters with type hints; the codebase is well-structured with proper Pydantic models. Critical weaknesses: (1) Schema documentation is incomplete, while type hints exist in Python code, the actual JSON Schema outputs are not visible in the provided source, making it impossible to verify schema completeness for LLM consumption. (2) Output schemas are almost entirely undocumented, no tool shows what fields are returned or how to chain results. (3) Descriptions range from good (e.g., analyze_data, compute_with_secret) to vague (e.g., 'Retrieve or paginate through cached results' without explaining what 'cached results' structure looks like). (4) Multiple tools are duplicated or near-duplicates (calculate appears twice, generate_sequence twice, get_cached_result twice) with inconsistent descriptions, confusing LLM tool selection. (5) Error handling guidance is absent, no tool description explains what happens on failure or how to recover. (6) Parameter relationships are undocumented (e.g., analyze_data accepts 'data' as 'list[float] | str' but doesn't explain the ref_id resolution mechanism clearly to an LLM). (7) No tool declares idempotence, permissions, or scope. The library itself is solid for backend caching, but the tool interface design does not follow LLM-optimization patterns.
Get detailed cache statistics including memory usage, hit rates, and namespace breakdown. Restricted to admin users.
List cached references with optional filtering. Restricted to admin users.
Compute aggregates (sum/mean/min/max/count/product) from lists or ref_ids.
Perform statistical analysis on numeric data. Accepts either raw data or a ref_id from another MCP tool (e.g., langfuse-calculator). The ref_id is automatically resolved from the shared SQLite cache.
Analyze a document with simulated processing time. This tool simulates a long-running document analysis. If processing exceeds 3 seconds, it returns a processing status immediately and continues in the background. Use get_task_status to poll for completion.
Evaluate mathematical expressions with safe evaluation.
Output schemas are almost entirely undocumented. No tool description explains the structure of returned data, forcing LLMs to reason about return types blindly. This violates the pattern:tool-description and pattern:response-shaper rules.
Duplicate and near-duplicate tools (calculate appears twice, generate_sequence appears twice, get_cached_result appears twice) with inconsistent parameter sets and descriptions. This forces LLMs to reason about which variant to use, violating pattern:tool (single responsibility) and increasing error likelihood.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 42 | - | v1 |
Evaluate a mathematical expression. Supports: +, -, *, /, **, (), sqrt, sin, cos, tan, log, exp, pi, e, factorial
Use a secret in computation without revealing it. The secret value never leaves the server.
Generate Fibonacci sequence and cache the result.
Generate prime numbers and cache the result.
Generate mathematical sequences (Fibonacci, primes, arithmetic, geometric, triangular, factorial) with caching.
Generate a mathematical sequence and cache the result. Sequence types: fibonacci, prime, arithmetic, geometric, triangular, factorial
Retrieve or paginate through cached results. Accepts a ref_id from any tool and returns the cached value with optional pagination.
Retrieve cached results with pagination. Accepts a ref_id from any cached tool result.
Check the status of an async task. Call this repeatedly with the ref_id from analyze_document until the status is 'complete'.
List all cached keys from this server.
Perform matrix operations (multiply, transpose, determinant, inverse, add, scalar_multiply, trace, eigenvalues).
A quick task that completes within the timeout. This demonstrates that fast operations return results directly, not a processing status.
Store a secret value for private computation without revealing it to the client.
The analyze_data tool accepts 'data' as 'list[float] | str' where str represents a ref_id, but the description does not clearly explain the resolution mechanism or format of ref_id strings. An LLM reading 'list[float] | str' has no guidance on what string values are valid or how they map to cached results.
No tool description indicates whether the tool is idempotent, destructive, or read-only. While toolAnnotations feature is not enabled, descriptions should still clarify side effects so LLMs know if retries are safe.
Error handling guidance is absent from all tool descriptions. No tool explains what happens on failure, what errors are retryable, or how to recover. This violates pattern:recovery-guide and pattern:error-classification.
Parameter constraints are partially documented. While some tools (e.g., calculate with minLength/maxLength, generate_sequence with minimum/maximum) include constraints, many others lack them (e.g., list_cached_keys has no parameters, aggregate lacks min/max on 'count' if applicable). Inconsistency forces LLMs to guess at valid ranges.
Some parameter descriptions are too brief. E.g., 'Type of sequence to generate' (generate_sequence) does not list valid options in the description text. While a SequenceType enum may be present in code, the description lacks clarity for an LLM reading only text.
analyze_document and get_task_status form a polling pattern but lack explicit guidance in descriptions. The analyze_document description mentions 'Use get_task_status to poll for completion' but does not explain the polling interval, timeout, or max retries an LLM should use.