Loading ToolBench
MCP schema linter & quality grader — validate, audit, optimize, grade (A+ to F). Also: @tool exports to OpenAI, Claude, Gemini, MCP.
35 tools with complete input schemas and descriptions. Naming follows verb_noun convention (alert_*, audit_*, batch_*, config_*, datetime_*). Descriptions are present but often generic (avg ~120 chars, below 194-char baseline). Parameters have types and descriptions, but lack enums for constrained values (e.g., condition, severity, outcome). Output schemas are documented in descriptions but not formally specified. Error handling guidance is minimal. No tool annotations (readOnlyHint, destructiveHint). Security: no secrets in params, but audit_log and config_set are WRITE operations without confirmation patterns.
Add (or subtract with negative values) a duration to a datetime. Returns ISO 8601 string.
Clear alert history for *rule* (or all if omitted). Returns {cleared, rule}.
Define a named alert rule. *condition*: gt | gte | lt | lte | eq | ne | between | outside | contains | not_contains | is_empty | is_truthy. *threshold*: primary threshold value (required for most conditions). *threshold_high*: upper bound for 'between' and 'outside'. *severity*: info | warning | error | critical. *cooldown_s*: minimum seconds between repeated firings (0 = no limit). Returns {name, condition, severity}.
Delete an alert rule by name. Returns {deleted, name}.
Evaluate *value* against rule *name*. Returns {rule, fired, value, severity, message, threshold, timestamp}. If cooldown is active, *fired* is False even if condition matches.
Get full definition of a rule. Returns full rule dict including threshold, cooldown, fire_count.
Missing enum constraints for categorical parameters. 'condition' (gt|gte|lt|lte|eq|ne|between|outside|contains|not_contains|is_empty|is_truthy), 'severity' (info|warning|error|critical), 'outcome' (success|failure|denied|unknown) are documented in descriptions but not declared as JSON Schema enums. LLMs cannot parse free-form text constraints and will hallucinate invalid values.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Destructive tools (alert_delete, alert_clear, audit_clear, config_delete, config_drop) lack destructiveHint annotation. This prevents clients from warning users or requiring confirmation before execution.
Output schemas not formally documented. Tool descriptions state return values (e.g., '{name, condition, severity}') in prose, but no structured output schema is provided. LLMs cannot reliably parse prose descriptions of complex nested objects.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | 2026-07-28+ | v2 |
Retrieve fired alert history. *rule*: filter to a specific rule name. *severity*: filter to a severity level. *limit*: max events to return (default 50). Returns {events: [...], count, total}.
List all defined alert rules. Returns {rules: [{name, condition, severity, fire_count, last_fired}], count}.
Clear events from the log. *before*: if provided, only clear events older than this timestamp. Otherwise clears all events. Returns {cleared, remaining}.
Export matching events as a JSON lines string. Returns {lines, count} where *lines* is newline-separated JSON.
Retrieve a single event by ID. Returns full event dict or {error}.
Record a structured audit event. *event_type*: dot-notation string, e.g. "user.login", "file.delete". *actor*: who performed the action (user, agent, service name). *resource*: what was acted upon (file path, DB table, API endpoint). *metadata*: arbitrary dict of additional context. *severity*: info | warning | error | critical. *outcome*: success | failure | denied | unknown. Returns {id, type, actor, resource, timestamp}.
Search and filter the audit log. All filters are ANDed. *text* performs a substring search across type, actor, resource, and JSON-serialized metadata. Returns {events, count, total, offset}.
Aggregate stats: counts by type, actor, resource, severity, outcome. Returns {total, by_type, by_actor, by_resource, by_severity, by_outcome}.
Keep items where *fn(item)* is truthy. Returns {results, total, kept, rejected, elapsed_ms}.
Apply *fn* to each item. Returns {results, ok, errors, elapsed_ms}. *on_error*: what to put in results for failed items: "null" (None), "skip" (omit from results), "raise" (fail fast).
Fold items with *fn(acc, item)*. *initial*: starting accumulator (None uses the first item). Returns {result, total, elapsed_ms}.
Set multiple default values (only sets if key not already present). Returns {set: N} — the number of keys that were actually set.
Delete a specific key. Returns {deleted: true/false, key: "..."}
Delete a named config store.
Export all config as a JSON object.
Get a configuration value. If the key does not exist, *default* is returned. *as_type* coerces the value: int, float, bool, str, json. Returns {key: "...", value: ..., found: bool}
List all keys in a config, optionally filtered by *prefix*. Returns a JSON array of keys (sorted).
List all named config stores with their key counts.
Populate config from environment variables. Only variables starting with *prefix* are imported. If *strip_prefix* is true the prefix is removed from the key name. If *lowercase* is true keys are lowercased (and __ → . for dot-notation). Returns {loaded: N, keys: [...]}
Assert that required keys are all present and non-None. Returns {ok: true} if all present, else {ok: false, missing: [...]}
Set a configuration key. *key* may use dot-notation (e.g. "db.host"). Returns {ok: true, key: "...", value: ...}
Convert a datetime from one timezone to another.
Compute the absolute difference between two datetime strings.
Register a named function from Python source. The source must define exactly one function: For map/filter/partition: def fn(item): ... For reduce: def fn(acc, item): ... (set is_reducer=True). Returns {name, registered, is_reducer}.
Format a datetime string using a strftime format.
Convert a Unix timestamp to an ISO 8601 datetime string.
Return the current date and time as ISO 8601.
Parse a date/time string and return ISO 8601.
Convert a datetime string to a Unix timestamp (seconds since epoch). Returns a string representation of the integer timestamp.
fn_define accepts arbitrary Python source code as a parameter. This is a code injection risk if the server is exposed to untrusted agents. No validation, sandboxing, or AST parsing is visible in the provided code.
Batch tools (batch_map, batch_filter, batch_reduce) accept function names as strings without validation. If a function name does not exist, error handling is unclear. No guidance on recovery or available functions.