Static source inference · medium confidence · detected: stateful session
Deprecated protocol patterns detected
Summary
Concierge has 14 tools spanning two distinct domains (MySQL migration + pizza discovery), but definitions are inconsistent and lack crucial details. MySQL tools have basic descriptions but no parameter descriptions, insufficient error guidance, and missing output schema documentation. Pizza tools are stubs with minimal descriptions. No tools declare output schemas. Parameter validation constraints are absent. Security considerations (destructive operations) lack confirmation patterns. The server mixes production-critical tools (apply_migration, drain_connections) with toy examples, creating maintenance and clarity issues.
Tools (14)
apply_migrationdestructivesource verified62/100
Execute the SQL migration script against the database.
create_backupwritesource verified67/100
Create a full logical backup (mysqldump) of the target database.
drain_connectionswritesource verified63/100
Gracefully drain all active MySQL connections before migration.
finalize_migrationwritesource verified60/100
Mark the migration as complete and clean up temporary artifacts.
notify_stakeholderswritesource verified65/100
Send a notification to the team about migration status.
No parameter descriptions on any tool. LLMs cannot infer intent from parameter names alone (e.g., 'backup_id', 'migration_file', 'pizzaTopping' lack guidance on format, valid range, or dependency on other parameters).
Output schemas not documented. Tools return dicts but LLMs have no formal specification of response structure (e.g., preflight_check returns {status, replication_lag_ms, disk_free_gb, ...} but no schema provided). This forces LLMs to infer field names and types from examples.
Add descriptions to every parameter. Example: For create_backup, add 'database (string, required): The name of the MySQL database to back up (e.g., my_app_db). Must be an existing database; invalid names will cause the tool to fail.'
Document output schemas formally. For each tool, add a Returns section specifying the structure: e.g., preflight_check returns {status: string (healthy|degraded), replication_lag_ms: integer, disk_free_gb: float, active_connections: integer, version: string}.
Add confirmation pattern to destructive operations. Before executing apply_migration, require the agent to pass a confirm=true parameter, and document: 'This tool executes irreversible SQL changes. Always call with confirm=false first to validate the migration script, review the response, then call again with confirm=true.'
Expand pizza tool descriptions to 50+ characters explaining the purpose and use case. Example: pizza_map → 'Display a map of pizza restaurants matching the specified topping. Use this when the user wants to see locations visually.'
Add recovery hints to error responses. Example: apply_migration should return 'error: database is not drained. Call drain_connections() first, then retry apply_migration().'
Separate MySQL migration and pizza discovery into two independent MCP servers. Mixing domains increases cognitive load and complicates testing.
Return state changes explicitly in responses rather than relying on server.set_state(). Example: drain_connections should return the list of terminated connections and their IDs, so the agent sees what changed.
Spec posture evidence
Inferred effective spec: 2026-07-28+.
Relies on Stateful initialize / Mcp-Session-Id (removed; protocol is stateless) - make each request self-contained
Score history
Overall score trend
↓ 15 points across a rubric change (v1 → v2)
56/100
Scored
Grade
Overall
Spec posture
Rubric
2026-09-22
D
56
2026-07-28+
v2
2026-03-09
B
71
-
v1
read only
source verified
50/100
Show a list of pizza spots
pizza_mapread onlysource verified50/100
Show a map of pizza spots for a given topping
pizza_shopwritesource verified48/100
Open the Pizzaz shop
preflight_checkread onlysource verified63/100
Check MySQL cluster health, replication lag, and disk space.
run_smoke_testsread onlysource verified63/100
Run post-migration smoke tests against key tables and queries.
undrain_connectionswritesource verified63/100
Re-enable new client connections to the MySQL cluster.
validate_backupread onlysource verified63/100
Validate the integrity of a MySQL backup by restoring to a scratch instance.
Pizza tools (pizza_map, pizza_carousel, pizza_albums, pizza_list, pizza_shop) have minimal descriptions (25 - 30 chars, all under the 50-char minimum for actionable LLM guidance). These descriptions do not explain when to use each tool or how they differ functionally.
Mixed domain focus (MySQL migration + pizza discovery). These are unrelated use cases bundled into one server. Maintenance, testing, and clarity suffer. Each should be a separate, focused server.
Error handling lacks recovery guidance. Examples: apply_migration returns {status: error, reason: 'database is not drained'} but does not guide the LLM to call drain_connections. validate_backup returns 'unknown backup' but does not suggest calling create_backup. No actionable next steps.
Pizza tools use generic names (pizza_map, pizza_carousel, pizza_list) without clear distinction. The descriptions do not explain when to use 'carousel' vs 'list' vs 'albums'. LLMs will struggle to choose the right tool.
Stateful server design (using server.set_state/get_state) creates issues: state is not returned to the caller in responses, making it difficult for the agent to understand what changed; state persists across unrelated calls, risking side effects; and state management is hidden inside tool implementations rather than explicit in schemas.
Add enum constraints for parameters with fixed valid values. Example: notify_stakeholders channel parameter should be {'type': 'string', 'enum': ['slack', 'email', 'pagerduty']} with description 'Notification channel: slack, email, or pagerduty.'
Add validation error examples. Example: 'If the backup_id does not match a prior create_backup call, the tool returns {status: error, reason: unknown backup, suggestion: Call create_backup() with the target database name first}.'
Document parameter dependencies. Example: 'apply_migration requires that drain_connections has been called first (check server state) and that validate_backup has succeeded for at least one backup.'
Add idempotent operation pattern. Document which tools can safely be retried (preflight_check, run_smoke_tests) vs which have side effects (drain_connections, apply_migration, create_backup).