Financial metrics analysis server for customers and projects with performance comparison capabilities
Two tools with complete input schemas and descriptions, but descriptions are generic and lack LLM-optimized guidance. Tool names follow verb_noun convention (get_data, compare_performance). Schemas are properly typed with parameter descriptions. However, descriptions lack actionable context (when to use, what to do next, dependencies). Output schemas are not documented, LLMs cannot plan downstream calls. No error handling guidance. No parameter constraints (enums, ranges). Fuzzy matching logic (thefuzz library) is undocumented in tool descriptions, creating hidden behavior. Overall: functional but below production baseline.
Compare performance across different customers or projects for specified metrics. Parameters: - entity_type: Either 'customer' or 'project' - metrics: List of metrics to compare, defaults to ['revenue', 'gross_margin', 'ebitda'] if None - top_n: If provided, returns only the top N entities by revenue Returns: - Dictionary with entity names as keys, each containing metric values and percentages
Get financial metrics at customer or project level. If none is provided, return the overall metric. Available metrics: 'gross_margin', 'revenue', 'ebitda'
Output schemas not documented. LLMs cannot infer return structure (Tuple[pd.Series, pd.Series] for get_data; Dict for compare_performance). Agents cannot plan downstream calls or extract fields reliably.
Descriptions lack actionable context. No guidance on when to use get_data vs compare_performance, what to do if a customer/project is not found, or how fuzzy matching (80% confidence threshold) affects results. LLMs cannot reason about tool selection.
No parameter constraints. 'metric' accepts free-form strings; should be enum ['gross_margin', 'revenue', 'ebitda']. 'entity_type' should be enum ['customer', 'project']. Free-form strings invite hallucinated values.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 54 | 2026-07-28+ | v2 |
Error handling does not guide recovery. ValueError exceptions (e.g., 'No close match found for customer') are raised but not documented in tool descriptions. LLMs cannot anticipate or handle failures gracefully.
Fuzzy matching behavior (thefuzz library, 80% confidence threshold) is hidden from LLM. Tool description does not mention that 'customer' and 'project' parameters accept partial/misspelled names. LLMs may not understand why 'Acme Corp' matches 'ACME Corporation'.