Loading ToolBench
MCP server for building, auditing, and managing data connectors with tools for connector assembly, tool/skill management, auditing, and preview functionality
Elliot MCP Plugin demonstrates solid definition quality with consistent naming patterns (all tools start with 'elliot_' verb prefix), comprehensive descriptions (avg 180+ chars), and explicit input schemas with types. Tool names clearly convey intent (e.g., elliot_generate_audit_seeds, elliot_submit_audit_transcript). However, output schemas are not documented in the provided source, descriptions mention return types narratively but lack formal JSON Schema definitions. Parameter descriptions are present but some lack constraint details (e.g., 'count' has min/max but 'scope' lacks enum documentation). Error handling guidance is minimal; tools don't describe recovery paths or categorize failures. Security considerations (audit trails, permission gates) are implicit in the domain but not explicitly declared in tool definitions.
Assemble a ConnectorConfig from selected (or all) tools and skills. ``instructions`` is connector-level guidance the agent authors for this connector — the same role Elliot's own ``instructions`` play for the platform. The connector runtime surfaces it to any MCP client on the ``initialize`` handshake (and it is shown to the connector's owner in the Studio / cloud UI), so use it to tell downstream agents how to use these tools: auth quirks, pagination defaults, which tool to reach for first. Leave empty to fall back to an auto-generated description. Build = SELECTION. Later edits to selected tools/skills refresh the build automatically (a stale definition can never ship), but tools created AFTER this build join only on the next explicit build. Lint and quality-scan always analyze the full current session either way.
Discard all submitted audit transcripts (start a fresh audit run).
Lint the built ConnectorConfig, then write it to disk as JSON. A connector is a contract, so export is GATED on the linter — it will not ship a connector that fails it. Lint ERRORS always block. Lint WARNINGS also block by default (the workflow expects zero warnings before export); pass ``allow_warnings=true`` to ship with warnings present. Errors can never be overridden. Run ``elliot_lint_connector`` to see the issues.
Generate realistic agent-task seeds for auditing the built connector. Returns up to ``count`` seeds plus the rubric and the instructions for spawning the audit sub-agents. Each seed should be handed to one sub-agent that exercises the connector's tools (via elliot_preview_tool) AND its skills (via elliot_preview_skill) and reports a transcript.
Output schemas not documented. Tool descriptions mention return types (e.g., 'AuditTranscript', 'ConnectorConfig') but no formal JSON Schema definitions are visible. LLMs cannot plan downstream tool calls or extract fields without documented output structure.
Enum constraints not formally declared. 'scope' parameter in elliot_judge_audit accepts 'current' or 'all' but is typed as string without enum constraint. LLMs may hallucinate invalid values.
Error handling guidance absent. Tools do not describe what errors can occur, whether they are retryable, or what the LLM should do next. E.g., elliot_export_connector mentions lint failures but provides no recovery path.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 68 | 2026-07-28+ | v2 |
Return the connector-level branding (accent colors + logo) for this session.
Return the current product context for this session.
Return a summary of the current session: sources, tools, skills, connector status.
Judge submitted transcripts and return a scored audit report. Scores graded 1-10 dimensions and emits findings that cite the exact failing call. Fix the error-severity findings, rebuild, then re-run the audit until it passes. ``scope`` defaults to ``"current"`` — only transcripts recorded against the connector's CURRENT build are scored, so a re-judge after fixing tools isn't dragged down by stale prior-build runs. Pass ``"all"`` to score every submitted transcript regardless of build.
List the audit transcripts submitted so far this session.
Set connector-level branding for MCP Apps views: accent color and logo. The accent (hex, e.g. ``#c02434``) drives highlights/selection/focus in every tool view; ``accent_dark`` optionally overrides it on dark host themes. ``logo`` is a ``data:image/...`` URI (recommended) or an ``https://`` URL shown in each view's header. Text and background always follow the host theme, so views stay legible in any client. Fields you pass are merged over the current branding; pass ``clear=True`` to reset everything first (``clear=True`` alone removes branding entirely). Branding is applied on the next ``elliot_build_connector`` and immediately in ``elliot_preview_tool_ui``.
Set the product context (name, description, base URL) for the current session.
Submit one audit sub-agent's transcript. ``transcript_json`` is an AuditTranscript: pass it as an object (preferred) or a JSON string. Fields: seed_id, task, agent_label, task_completed, summary, and a `calls` list where each call has tool_id, arguments, ok, error_code, error_message, result_row_count, result_token_estimate, note.
Minimal descriptions for simple tools. elliot_list_audit_transcripts, elliot_get_context, elliot_get_branding, elliot_get_session_state have descriptions under 70 chars. Insufficient context for LLM selection when multiple similar tools exist.
Parameter constraints underspecified. 'count' in elliot_generate_audit_seeds has min/max but no description of what seeds represent or why the 1-20 range exists. 'path' in elliot_export_connector lacks format guidance (absolute vs relative, file extension requirements).