Loading ToolBench
Local-first, AI-native personal finance data platform built on DuckDB, SQLMesh, and MCP. Provides financial data aggregation, analysis, import/export, and AI assistant integration through MCP tools.
MoneyBin exposes 6 import tools with mixed quality. Tool naming follows verb-noun convention (import_*), which is good, but descriptions are inconsistent in depth and clarity. Input schemas are present and use proper JSON Schema types, but critical issues emerge: (1) Several parameter descriptions are generic or lack actionable guidance for LLMs. (2) Output schemas are not documented, callers cannot see what import_confirm, import_preview, or import_status return. (3) Error handling is minimal, no guidance on recovery paths or retryable vs. fatal failures. (4) Destructive operations (import_revert) lack clear confirmation semantics and error classification. (5) Parameter relationships (e.g., confirmation_token only required for destructive ops) are underdocumented. This is a domain-specific tool set, but falls short of production-grade quality due to missing output documentation and weak error-handling patterns.
Ratify one unchanged staged preview and load the file
Import files (critical when a PDF bridge proposal is returned)
List available tabular import formats (low sensitivity)
Stage a preview, optionally overriding the column mapping
Undo an import batch by import_id (low sensitivity)
List past import batches (low sensitivity)
Output schemas are not documented. Tools like import_confirm, import_preview, import_status, and import_formats return data structures, but the response format is invisible to callers. LLMs cannot plan downstream steps or extract required fields without documented return types.
import_revert combines two different operations (revert_import and delete_saved_format) with different semantics under a single 'operation' enum. This violates single-responsibility, the tool should split into revert_import and delete_saved_format, each with clear error paths. Also, confirmation_token requirement is conditional on operation type but underdocumented.
Error handling and recovery paths are absent or minimal. There is no guidance on what errors are retryable (transient network failures) vs. fatal (invalid format), nor clear error messages that tell LLMs what to do next. E.g., 'File format not recognized' is better as 'File format not recognized. Try calling import_formats() to see supported formats, or import_preview() to debug the format.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | 2025-06-18+ | v2 |
Parameter descriptions lack actionable detail. E.g., 'Optional format name override' (import_preview) does not explain when or why to override, what happens if the override is invalid, or what format names are valid. 'Optional account metadata for routing' (import_files) leaves LLMs guessing about required fields and structure.
confirmation_token parameter semantics are unclear across tools (import_confirm, import_revert). Is this required always, or only for certain operations? What happens if omitted? No validation rules or error guidance.
import_preview and import_confirm accept optional parameters (format_name, column_mapping, bridge_response) that have conditional dependencies, but these are not documented. An LLM cannot know whether passing all three is valid, required, or mutually exclusive.
import_status and import_formats are discovery/read-only tools that return lists, but there is no pagination support documented (limit and offset exist on import_status, but max and default are not specified). A large import history could be returned unpaginated, exhausting context.
Tool descriptions do not distinguish when to use import_preview vs. import_confirm. What is the purpose of staging? When should an LLM call preview before confirm? The workflow is implied but not explicit.