Loading ToolBench
Calculates retention rate drops per monthly cohorts to detect early customer dropouts.
This MCP server has critical definition quality gaps. The single tool 'calculate_cohort' lacks essential documentation and has an undescriptive name that doesn't follow action-verb conventions. The tool description is minimal (82 chars, well below the 194-char baseline for production tools). No output schema is documented in the provided source code. Parameter descriptions exist but are sparse and generic. The tool appears to be a pure calculation utility with no error handling, validation guidance, or recovery hints. Overall, this reads as a prototype or internal utility rather than a production-grade agent tool.
Calculates retention rate drops per monthly cohorts to detect early customer dropouts.
Tool name 'calculate_cohort' is non-standard and lacks a clear action verb. LLMs expect names like 'get_cohort_retention', 'analyze_cohort_decay', or 'detect_churn_risk'. The current name is a noun/verb hybrid that does not clearly signal intent or outcome to an LLM.
Tool description is 82 characters, below the 194-char production baseline and lacking context. It does not explain WHEN to use this tool, what insights it provides, or how to interpret the output (what makes a cohort 'high-risk' vs 'stable'?). LLMs cannot determine appropriate selection.
No output schema is documented in the source code. The function returns a dict with keys 'month1_retention_pct', 'month3_retention_pct', 'decay_tier', but LLMs have no declared schema to understand these fields' types, constraints, or meaning. This prevents proper downstream tool chaining and planning.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 24 | 2026-07-28+ | v2 |
Parameter descriptions are minimal and lack context. 'Number of active users in month 1' does not clarify: Is this a count of distinct users? Daily average? Total logins? What if the number exceeds the initial registration count? No validation rules, ranges, or error guidance provided.
No error handling or validation guidance. If active_users_month1 > cohort_registrations, the tool silently produces a retention rate > 100%. No recovery hints for invalid inputs. LLMs cannot self-correct without explicit guidance.
Hard-coded decay tier thresholds (60%, 30%) are undocumented and not configurable. The tool description does not explain these business logic boundaries. An agent calling this tool has no way to know why month3_retention_pct=35% triggers 'decaying' vs 'high-risk'.