Loading ToolBench
MCP server for Chalk feature platform integration, providing tools to manage Chalk configurations, features, environments, deployments, queries, logs, and linting
Server defines 7 tools with consistent naming (verb_noun pattern: chalk_config, chalk_features, chalk_environment, chalk_apply, chalk_query, chalk_logs, chalk_lint). All tools have descriptions (avg 120 chars, within 10-1024 range). Input schemas are present with JSON types and descriptions. However, output schemas are NOT documented, tools return text/structured results but LLMs cannot predict field structure. Parameter descriptions are adequate but lack constraints (enums, ranges, format specs). No error handling guidance visible in tool definitions. Missing tool annotations (readOnlyHint, destructiveHint) despite clear risk levels (READ_ONLY vs WRITE). Tests exist but do not validate schema completeness.
Apply Chalk configurations with branch and JSON output. Recommended workflow is to use the chalk_environment tool to set the environment, then use this tool to apply the configuration.
Get the chalk config from a chalk project
Get or set the chalk environment for the current project
Get the list of features from a chalk project
Check for errors in Chalk feature pipelines
Search through Chalk logs using powerful filtering capabilities
Query Chalk features using fully qualified feature names (FQNs). Supports input features, output features, and branch specification.
Output schemas not documented. Tools return text or structured data but LLMs cannot predict field names, types, or nesting. Breaks downstream tool chaining and forces LLMs to parse unstructured output.
Missing tool annotations. chalk_apply and chalk_environment are WRITE operations but lack destructiveHint/idempotentHint annotations. LLMs cannot distinguish safe-to-retry from irreversible calls.
Parameter constraints missing. chalk_environment 'operation' param accepts free-form string with no enum. chalk_query 'input_features' is object type but no schema for nested structure. Invites hallucinated invalid values.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 51 | 2026-07-28+ | v2 |
No error handling guidance in tool definitions. Errors return raw text but do not tell LLM what to do next (retry? ask user? fatal?). No recovery hints or actionable error messages.
chalk_query 'input_features' parameter is object type but no description of expected structure (key-value pairs, nested fields, etc.). LLMs cannot construct valid input without explicit schema.