Loading ToolBench
MCP server that integrates with Skyflow Detect API for de-identification and re-identification of PII across multiple data sources (CRM, Hospital, Insurance, Claims). Demonstrates end-to-end data privacy workflow with tokenization and selective re-identification.
Server has 8 tools with basic schemas and descriptions, but significant gaps limit production readiness. All tools have descriptions (10-100 chars, mostly adequate), and input schemas are present with type definitions. However, output schemas are undocumented, LLMs cannot predict return structure. Parameter descriptions are minimal (1-2 words). Error handling is absent: no guidance on retryability, user-fixable vs fatal errors, or recovery paths. No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Tool naming is inconsistent: verb_noun pattern mostly followed (crm_lookupMember, hospital_getBilling) but some tools use underscores awkwardly (detect_deidentify_string, skyflow_reidentify_string). No pagination support despite tools returning structured data. Security: credentials injected via environment (correct), but no audit logging or permission gates visible.
Check claim status and de-identify via Skyflow
Look up member information from CRM database and de-identify via Skyflow
Direct pass-through to Skyflow deidentify/string.
Retrieve hospital billing information and de-identify via Skyflow
Retrieve insurance policy information and de-identify via Skyflow
Runs Steps 1–9: De-id CRM/Insurance/Claims/Hospital (via Skyflow), draft answer with a real NAME token, re-id final text (names plaintext, emails redacted)
Output schemas completely undocumented. Tools return Dict[str, Any] or str with no field documentation. LLMs cannot predict structure, plan downstream calls, or extract required data.
Parameter descriptions are minimal (1-2 words: 'The member ID to look up', 'Text to de-identify'). Missing format constraints, ranges, and dependency hints. LLMs cannot validate inputs or understand when to call each tool.
No error handling or recovery guidance. skyflow_deidentify_safe() catches exceptions and returns fallback, but LLM sees no error classification (retryable vs fatal). No guidance on what to do if Skyflow API fails.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 56 | 2026-07-28+ | v2 |
Verbose end-to-end run with intermediate Skyflow outputs for demo/inspector. Returns: { question, steps: [...], draft, final_text }
Direct pass-through to Skyflow reidentify/string, with format controls.
Tool naming inconsistency: detect_deidentify_string and skyflow_reidentify_string use redundant prefixes and unclear verbs. 'detect_deidentify' is confusing, does it detect or deidentify? Should be 'deidentify_string' and 'reidentify_string'.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All tools are read-only but LLM has no explicit signal. Prevents proper agent planning and safety checks.