Loading ToolBench
MCP Server for insurance carrier accounting — ingest carrier PDFs/Excel, reconcile against Applied Epic, post entries. Built for US insurance agencies.
Server has 21 tools with consistent naming (verb_noun pattern) and descriptions present for all. However, parameter descriptions are sparse or missing for many tools, and output schemas are not formally documented. Tool annotations are present (readOnlyHint, destructiveHint, idempotentHint, openWorldHint), which is good. Error handling is minimal, most tools lack recovery guidance. Ingestion and posting tools are high-risk (WRITE/DESTRUCTIVE) but lack confirmation patterns or dry-run safeguards. Parameter validation rules are not visible in descriptions (e.g., 'mode' accepts 'trial' or 'live' but this is not enforced via enum). Overall: solid naming and basic structure, but parameter rigor and error guidance are weak.
Approve a single transaction for posting to Epic. Moves from review queue to auto-post queue.
Approve all transactions in a run's review queue at once. Bulk operation.
Automate login and download from carrier portals using Playwright. Returns downloaded file path.
Get accuracy metrics for a specific carrier over N days. Post rate, rejection rate, error rate.
Get the daily accuracy scorecard. Counts by status, average confidence, total amount processed.
Generate a CSV file for bulk import into Epic. Returns file path and preview.
Parameter descriptions missing or incomplete. Many parameters (e.g., 'mode', 'carrier', 'run_id') lack detailed descriptions explaining valid values, constraints, or format. LLMs cannot infer that 'mode' must be 'trial' or 'live' without explicit enum or description text.
Output schemas not formally documented. Tool descriptions state what is returned (e.g., 'run_summary with counts by status') but do not provide structured JSON Schema definitions. LLMs cannot reliably extract fields from unstructured responses.
Inferred effective spec: 2026-07-28+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 64 | 2026-07-28+ | v2 |
Get all transactions in the review queue for today. Paginated, sortable by confidence score.
Full pipeline: ingest a carrier PDF or Excel statement, normalize, validate against BigQuery, and stage for review or posting.
Ingest a single carrier Excel bordereaux. Runs the full pipeline.
Ingest a single carrier PDF statement. Runs the full pipeline. Alias for ingest_carrier_statement for explicit PDF files.
List all 48+ supported carriers with their field mappings and portal URLs.
Parse and normalize a carrier document WITHOUT validation or staging. Returns the raw normalized transactions for inspection. Useful for debugging normalization quality before running full pipeline.
Post approved transactions to Applied Epic via REST API. Requires Epic SDK credentials.
Post transactions to Epic via Playwright browser automation. Useful when REST API is unavailable.
Generate a reconciliation report comparing carrier statements to Epic postings. Identifies discrepancies.
Reject a transaction. Moves to rejected queue with reason recorded.
Rollback a posted run. Reverses all Epic entries created by this run.
Get recent ingestion run history. Paginated, sortable by date.
Score a single transaction's confidence without running the full pipeline. Looks up the policy in BigQuery, checks for duplicates, and returns the confidence score breakdown. Useful for testing and debugging.
Compare trial mode results to live mode results. Shows what would have been posted.
Parse, normalize, AND validate a carrier document against BigQuery — but do NOT stage or post. Returns the full validation report including confidence scores, policy matches, and queue classifications.
Destructive operations lack confirmation or dry-run safeguards. Tools like 'rollback', 'post_to_epic' (live mode), and 'approve_run' can irreversibly modify Epic records, but no confirmation pattern or dry-run preview is enforced before execution.
Error handling lacks recovery guidance. Tools do not return actionable error messages. E.g., if 'carrier' slug is invalid, the tool should suggest available carriers; if 'run_id' is not found, it should offer alternatives or suggest how to find the correct ID.
Credentials exposed in tool parameters. 'browse_carrier_portal' accepts 'username' and 'password' as parameters. These should never appear in tool signatures, use server-side secret injection via environment variables or a vault.