AI agent cost intelligence tools for LLM agents. Track spend, optimize models, manage budgets, detect waste, and prove ROI.
The Metrx MCP server provides 24 tools covering cost intelligence and agent optimization. Tools follow a consistent verb-noun naming pattern (get_, list_, create_, update_, apply_, configure_, attribute_, generate_, run_) which is strong. All tools have descriptions ranging from ~60-180 chars, meeting the 10-1024 char baseline. However, several critical quality gaps reduce the score: (1) Parameter schemas are present but lack formal constraint documentation in descriptions (enums, ranges, formats are not always explicit). (2) Output schemas are not documented in the code provided, LLMs cannot infer return structures. (3) Some parameter descriptions are minimal (e.g., 'Optional: agent UUID' with no guidance on resolution). (4) Error handling is not visible in the tool definitions, no recovery guidance documented. (5) Two tools appear to be duplicates (configure_alert_threshold at #13 and #23 with identical names and slightly different inputs, creating ambiguity). (6) Idempotency and side-effect documentation is weak, the distinction between WRITE/READ operations is labeled but not explained in tool descriptions. The server demonstrates good foundational structure and naming discipline but lacks the depth of documentation expected for production-grade agent tools.
Mark an alert as read/acknowledged
Apply a recommended optimization to an agent
Attribute the outcome value of a task to the AI agents that contributed to it
Compare cost and quality metrics across different AI models for a use case
Configure alert threshold for a specific alert type
Configure alert thresholds for cost anomalies and budget warnings
Create an A/B experiment to test different models for an agent
Duplicate tool name: 'configure_alert_threshold' appears twice (#13 and #23) with slightly different input schemas. LLMs cannot disambiguate between them. One must be renamed or merged.
Output schemas are not documented. The provided code shows input parameter schemas but no documentation of return structures. LLMs cannot infer what fields are returned, which blocks downstream tool chaining and planning.
Error handling and recovery guidance are not visible in tool definitions. No indication of what errors are retryable, what causes require user input, or how to resolve failures. This violates recovery-guide pattern.
Parameter constraints (enums, ranges, formats) are not consistently documented in descriptions. For example, 'budget_preference' accepts 'lowest_cost', 'balanced', or 'highest_quality' but the enum is not stated in the description or parameter definition. LLMs may hallucinate invalid values.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 52 | <=2025-11-25 | v2 |
| 2026-03-09 | D | 55 | - | v1 |
Generate a comprehensive ROI audit report for all agents showing cost vs. attributed outcomes
Get detailed information about a specific AI agent including cost breakdown and performance metrics
Get recent alerts about cost anomalies, budget warnings, and error spikes
Get an attribution report showing how outcomes are distributed across agents
Get the current status of all budgets including usage and remaining balance
Get a comprehensive cost summary for your AI agents over a specified period
Get results and statistical significance of a running or completed model experiment
Get ML-powered predictions of agent failures including budget breaches and error rate spikes
Get AI-generated cost optimization recommendations for your agents including model switches, caching opportunities, and routing improvements
Get the ROI for a specific task including cost and attributed outcome value
Get a business justification for upgrading an agent to a better model based on ROI and use case impact
List all AI agents with their current status, cost, and ROI information
Route a request to an optimal model based on cost and quality tradeoffs
Scan for waste patterns in your AI agent operations including retry storms, oversized contexts, model mismatch, and missing caching
Create or update a budget for an agent or organization
Stop a running model experiment and optionally apply the winning model
Update the enforcement mode of a budget (alert_only, soft_block, or hard_block)
Several parameters marked 'Optional' lack guidance on resolution logic. E.g., 'agent_id' is optional in get_optimization_recommendations and get_failure_predictions, but the description does not explain what happens if omitted (all agents? current agent? error?).
WRITE-class tools (apply_optimization, set_budget, update_budget_mode, acknowledge_alert, configure_alert_threshold, create_model_experiment, stop_experiment, attribute_task) lack idempotency documentation. No indication of whether repeated calls with the same inputs produce the same result or trigger side effects twice.
Pagination support is not documented for list_agents. No mention of page/limit behavior or total count in response. Tools that return potentially large result sets must support pagination to avoid context window exhaustion.
Parameter descriptions for UUIDs/IDs do not indicate whether human-readable names (agent names, budget labels) are accepted as alternatives. E.g., does 'agent_id' accept only UUID or also agent display name? This forces unnecessary lookup calls.
Tool descriptions do not clearly state side effects or data mutability. The Risk field (READ_ONLY vs WRITE) is present in metadata but NOT reflected in the description text. LLMs may not realize a tool is destructive if they only read the description.
No confirmation or dry-run mechanism documented for irreversible operations. apply_optimization and stop_experiment(apply_winner=true) could have unintended consequences. Suggest adding a 'dry_run' parameter to preview changes before committing.