A Spring Boot banking assistant application that uses Spring AI for tool calling, LLMs for chat, and Langfuse for evaluation. Provides banking tools for card management, transaction retrieval, and dispute handling with observability via OpenTelemetry.
Spring Boot AI server with 7 banking tools. All tools have descriptions (10-50 chars, mostly adequate) and visible input schemas with typed parameters. Naming follows verb_noun convention (freezeCard, unfreezeCard, openDispute, etc.). However, descriptions are terse and lack context about when to use each tool, prerequisites, or side effects. Parameter descriptions are minimal (e.g., 'The account identifier' repeated across tools). No output schemas documented. Error handling is absent, no guidance on what happens on failure or how to recover. No pagination support for listDisputes or getTransactions despite returning lists. Missing idempotency hints for write operations.
Freeze the card associated with an account
Check the status of a dispute
Retrieve transactions from an account
Check if the card associated with an account is frozen
List all dispute ids for an account
Open a dispute for a transaction
Unfreeze the card associated with an account
No output schemas documented. LLMs cannot predict return types or plan downstream tool calls. E.g., freezeCard, openDispute, getDisputeStatus return objects but structure is not declared.
Descriptions are terse (10-50 chars) and lack context. Missing WHEN to use each tool, prerequisites, and side effects. E.g., 'Freeze the card associated with an account' does not explain that this is a destructive operation or what happens to pending transactions.
No error handling guidance. Tools do not document what happens on failure (e.g., invalid accountId, non-existent dispute). No recovery hints for LLM.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 49 | 2026-07-28+ | v2 |
listDisputes and getTransactions return lists but lack pagination support (no limit, offset, or cursor parameters). Large result sets will blow context window.
Write operations (freezeCard, unfreezeCard, openDispute) lack idempotency hints or confirmation steps. No dry-run support. Agents cannot safely retry on ambiguous failures.