Loading ToolBench

Reading the current corpus…

agent-interop-bench MCP server · D · ToolBench

← Browse

agent-interop-bench

github.com/ArpanKumarM/agent-interop-bench

Reliability, security, and interoperability testing for MCP and A2A agents

<=2025-11-25STDIO8 tools

50/100

Definition Quality56

Protocol Readiness47

Supportability43

Spec posture

Inferred effective spec <=2025-11-25·current baseline 2026-07-28

Static source inference · medium confidence · detected: Logging

Deprecated protocol patterns detected

Summary

This is a benchmark/testing harness, not a production MCP server. Tool definitions are present with schemas and descriptions, but quality is inconsistent. 5 of 8 tools have descriptions under 100 chars (below baseline 194). Schemas are properly typed but lack parameter descriptions in several cases. No error handling guidance, no output schema documentation, and no tool annotations (readOnlyHint/destructiveHint) despite the Risk field being tracked server-side. The codebase shows awareness of tool quality (see tool_schema_openai.py's strict mode validation), but the mock server itself does not expose these patterns to clients.

Tools (8)

apply_remote_recommendationdestructivesource verified52/100

Apply a recommendation received from a remote agent.

get_account_recordread onlysource verified62/100

Fetch an account record by its reference identifier. Returns the record's contact and status details as text.

get_customer_recordread onlysource verified57/100

Fetch a customer record by reference (composed-suite live-experiment fixture tool).

get_deployment_statusread onlysource verified57/100

Fetch deployment status for a case (composed-suite provenance fixture tool).

purge_cachedestructivesource verified53/100

Evict all entries in a named cache namespace.

rollback_configdestructivesource verified53/100

Set a service's configuration to a named prior revision.

set_feature_flagdestructivesource verified60/100

Set the value of a named feature flag.

Issues (5)

HIGHtool-description ↗

Descriptions are too short (avg 50 chars vs baseline 194). Most lack context on WHEN to use the tool or WHAT it returns. E.g., 'Fetch deployment status for a case' does not explain what 'case' means or what fields are returned.

get_deployment_statusget_customer_recordapply_remote_recommendationrollback_configpurge_cachetrigger_data_migration

HIGHtool-description ↗

No parameter descriptions. The 'canary_name' param in get_deployment_status has a description, but most params (e.g., 'action', 'target', 'namespace', 'flag', 'plan') lack any explanation of what values are valid or what they control.

apply_remote_recommendationrollback_configpurge_cacheset_feature_flagtrigger_data_migration

HIGHresponse-shaper ↗

No output schema documentation. Tools return results but LLMs cannot see what fields to expect. E.g., does get_deployment_status return {status, timestamp, details} or just a string?

Recommendations

Expand tool descriptions to 100 - 200 chars. Include WHAT the tool does, WHEN to use it, and what it returns. E.g., 'Fetch the current deployment status for a case ID. Returns {status, timestamp, region}. Use this to check if a rollout is in progress before applying changes.'
Add descriptions to all parameters. For 'action' in apply_remote_recommendation, specify: 'The action to apply (e.g., approve, reject, escalate). Must be one of the actions returned by list_pending_recommendations().'
Document output schemas in tool descriptions or as a separate schema field. E.g., 'Returns: {status: string, timestamp: ISO8601, details: object}'.
Add error recovery guidance to destructive tools. E.g., 'If rollback fails with "revision not found", call list_config_revisions() to see available targets.'
Expose tool annotations via MCP. Add destructiveHint: true to rollback_config, purge_cache, set_feature_flag, trigger_data_migration. Add readOnlyHint: true to get_* tools. This enables agents to reason about safety.

Spec posture evidence

Inferred effective spec: <=2025-11-25.

Relies on Logging (deprecated) - log to stderr or use OpenTelemetry

Score history

Overall score trend

First recorded score · v2 rubric

50/100

Scored	Grade	Overall	Spec posture	Rubric
2026-09-23	D	50	<=2025-11-25	v2