Loading ToolBench
Command-line interface for Trifle metrics and data management system. Provides authentication, source management, metrics querying, and MCP server capabilities.
This server exposes 20 tools via MCP, all with basic descriptions and input schemas. However, critical quality issues prevent a higher score: (1) Parameter descriptions are generic and lack actionable constraints (e.g., 'Database host' without format/validation hints); (2) No documented output schemas, LLMs cannot infer what fields to expect from responses; (3) Parameter naming inconsistency ('user_token' vs 'token' vs 'user_token' across tools); (4) No error handling guidance, errors are not classified or paired with recovery steps; (5) Secrets (passwords, tokens) exposed as tool parameters instead of server-side injection; (6) Missing enums for constrained fields (e.g., driver='sqlite|postgres|mysql|redis|mongo' should be an enum); (7) Tools like source_create_database have 14 optional parameters with no guidance on dependencies or mutual exclusivity; (8) No confirmation/dry-run pattern for destructive operations (transponders_delete); (9) Description lengths vary wildly (40 - 300 chars) and many lack WHEN-to-use context. The server is functionally complete but does not meet production quality standards for agent integration.
Authenticate user with email and password and create an organization API token
Get current authenticated user information
Create a new user account and organization with email and password
Aggregate metrics data across a time range
Get metric values grouped by category
Retrieve metric values for a key within a time range
List all metric keys available in a time range
Push metric data to Trifle
Secrets exposed as tool parameters. Passwords, API tokens, and user_token are passed as input parameters, which means they appear in execution logs, traces, and potentially in agent prompt history. This violates the secret-injection pattern and creates credential leak risk.
No output schemas documented. Tools return responses, but the server does not publish what fields or structure the agent should expect. LLMs cannot plan downstream calls or extract required IDs without documented output schemas.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 39 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Setup metrics collection for a source
Get timeline of metric data points
Create a new database source with connection parameters
Create a new project source
List all data sources in the organization
Setup a database source with automatic configuration
Manage source-specific API tokens
Set the active source in configuration
Create a new transponder for data transformation
Delete a transponder
List all transponders for a source
Update an existing transponder
Missing constrained input enums. Parameters like 'driver' accept a fixed set (sqlite|postgres|mysql|redis|mongo) but are defined as free-form strings. LLMs will hallucinate invalid values. Also, 'format' in metrics_keys should be an enum (json|table|csv), and 'aggregation' in metrics_aggregate should enumerate valid aggregation methods.
Parameter naming inconsistency. Some tools use 'token', others use 'user_token', and some use both. This confuses LLMs trying to correlate outputs from one tool to inputs of another. Standardize across all tools.
No error handling guidance. Tools do not document what errors they may return, which ones are retryable, or what the LLM should do next. For example, if auth_login fails with 'invalid credentials', the agent does not know whether to ask the user or retry.
Destructive operations lack confirmation/dry-run pattern. transponders_delete is destructive but has no dry-run, confirmation, or undo mechanism. Agents can permanently delete transponders without safeguards.
Parameter descriptions are generic and lack validation hints. E.g., 'Database host' does not specify format (hostname, IP, FQDN), 'port' does not state valid range (1 - 65535), 'timezone' does not list valid values (IANA TZ identifiers). Generic descriptions force LLMs to guess.
Undocumented parameter dependencies. source_create_database has 14 parameters with unclear dependencies: which are required together? (e.g., host+port+user+password for postgres, but file_path for sqlite). No mutual exclusivity guidance.
No pagination or result limits documented. Tools like metrics_keys and transponders_list may return large result sets. No indication of pagination support, max result limits, or how to fetch subsequent pages.
Missing permission/scope declarations. No tool documents what permissions are required (e.g., 'read:metrics', 'write:sources'). Agents cannot be configured with least-privilege access.