Loading ToolBench
MCP server for SerenAI - AI assistant integration with Seren database management and agent deployment
Seren MCP has 8 well-named tools with consistent verb-noun patterns (list_, get_, submit_). All tools have descriptions (194 char baseline met). Input schemas are present and properly typed with descriptions for all parameters. However, output schemas are NOT documented, critical for LLM planning. Error handling is generic; no recovery guidance. No tool annotations (readOnlyHint/destructiveHint). Idempotency story is unclear for submit_managed_mutation (WRITE tool). Overall above average but missing output schema documentation and structured error responses.
Get a specific audit log entry by organization ID and log ID
Get status of a managed deployment mutation by deployment ID and request ID
Get generated skill.md guidance for a publisher
Get the generated skill.md guidance for the core Seren API
Get details about a specific publisher by slug or UUID
List audit logs for an organization with optional filtering
List all active publishers in the store with optional filtering by category, verification status, and search
No output schemas documented. Tools return data but LLMs cannot plan downstream calls without knowing field names and types. This violates pattern:tool which requires documenting return types.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). LLMs cannot distinguish safe retry patterns from destructive operations. submit_managed_mutation is marked WRITE but lacks destructiveHint annotation.
Error handling lacks recovery guidance. No evidence of actionable error messages (e.g. 'User not found. Try search_users() with a partial name.'). Generic errors prevent LLM self-correction.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | B | 71 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 32 | - | v1 |
Submit a managed mutation request (deployment, runtime policy reconciliation, etc.) and poll until completion or failure
submit_managed_mutation (WRITE tool) lacks idempotency clarity. Parameter 'request_id' suggests idempotency intent, but tool description does not explicitly state 'This operation is idempotent: calling multiple times with the same request_id produces the same result.'
list_store_publishers and list_audit_logs accept pagination (limit, offset) but no documentation of max allowed limit or default behavior. Should specify: 'Maximum 100, default 20' to prevent LLM from requesting absurd batch sizes.
get_store_publisher accepts both 'slug or UUID' but parameter name is just 'publisher', ambiguous. Should be 'publisher_id_or_slug' or provide separate parameters (publisher_slug, publisher_uuid) to reduce LLM confusion.