Loading ToolBench

Reading the current corpus…

ai-engine MCP server · D · ToolBench

← Browse

ai-engine

github.com/oscarcitoz/ai-engine

FastAPI-based AI orchestration service with tool management, MCP server integration, and agent execution

<=2025-11-25STREAMABLE_HTTP12 tools

57/100

Definition Quality74

Protocol Readiness69

Supportability21

Spec posture

Inferred effective spec <=2025-11-25·current baseline 2026-07-28

Static source inference · medium confidence · detected: Logging

Deprecated protocol patterns detected

Summary

AI Engine exposes 12 tools for managing custom tools and MCP server configurations. Tool naming follows verb_noun convention (list_, create_, get_, update_, delete_, test_, ping_) which is strong. Descriptions are present and moderately detailed (avg ~80-120 chars), meeting the 10-1024 char baseline. However, parameter descriptions are sparse or generic in several tools. Input schemas are visible and typed (string, integer, object, boolean), but lack granularity, e.g., 'config' parameters accept generic objects without field documentation. Output schemas are not documented in the source. Error handling is minimal; no recovery guidance or actionable error messages visible. The server manages tool/MCP-server CRUD operations but lacks composition patterns (e.g., no batch operations, limited chaining support). Security: API key auth is enforced via middleware, but no per-tool permission scoping or audit trail visible.

Tools (12)

create_mcp_serverwriteauthsource verified68/100

Create a new MCP server configuration

create_toolwriteauthsource verified68/100

Create a new tool with HTTP or agent configuration

delete_mcp_serverdestructiveauthsource verified77/100

Delete an MCP server (soft delete via is_active flag)

delete_tooldestructiveauthsource verified77/100

Delete a tool (soft delete via is_active flag)

get_mcp_serverread onlyauthsource verified82/100

Retrieve a specific MCP server by ID

get_toolread onlyauthsource verified82/100

Retrieve a specific tool by ID

list_mcp_serversread onlyauthsource verified78/100

List all MCP servers with optional filtering by project

Issues (5)

HIGHconstrained-input ↗

Generic 'config' parameter lacks field documentation. create_tool and create_mcp_server accept config as untyped object; LLMs cannot infer required/optional fields, structure, or valid values.

create_toolupdate_toolcreate_mcp_serverupdate_mcp_server

HIGHtool ↗

No output schema documentation. Tools return ToolResponse, MCP ServerResponse objects, but response field structure is not visible in source. LLMs cannot plan downstream calls or extract required IDs.

list_toolscreate_toolget_toolupdate_toollist_mcp_serverscreate_mcp_serverget_mcp_serverupdate_mcp_server

HIGHconfirmation-request ↗

Destructive operations (delete_tool, delete_mcp_server) lack confirmation or dry-run support. No recovery guidance in error responses. Agents can permanently delete resources without safeguards.

delete_tooldelete_mcp_server

Recommendations

Document the 'config' parameter schema explicitly. For create_tool, specify: config = {api: string (URL), method: enum(GET|POST|PUT|DELETE), headers?: object, body?: object, query_params?: object, properties?: object}. For create_mcp_server, specify transport-specific config structure (stdio: {command, args}, http: {url, auth}, sse: {url, auth}).
Add output schema documentation to all response-returning tools. Include field names, types, and which fields are safe to pass to downstream tools (e.g., tool_id from create_tool can be passed to update_tool).
Implement confirmation pattern for delete_tool and delete_mcp_server. Add a 'confirm' boolean parameter (default false) or return a confirmation token that must be passed in a second call. Document: 'This operation is irreversible.'
Enhance error messages with recovery guidance. E.g., 'Tool not found (ID: {tool_id}). Call list_tools() to see available tools.' or 'Config validation failed: missing required field 'api' in config object.'
Add per-tool permission scoping. Document which tools require 'admin', 'write:tools', 'read:tools' permissions. Implement permission checks in middleware and return 403 Forbidden with required scope in error message.

Spec posture evidence

Inferred effective spec: <=2025-11-25.

Relies on Logging (deprecated) - log to stderr or use OpenTelemetry

Score history

Overall score trend

First recorded score · v2 rubric

57/100

Scored	Grade	Overall	Spec posture	Rubric
2026-09-23	D	57	<=2025-11-25	v2