Loading ToolBench
MCP server for managing utility bills and payments (komunalne račune)
The server has 5 tools with mixed definition quality. All tools are explicitly registered with @mcp.tool() decorators in main.py, with visible descriptions and parameter schemas. However, critical issues reduce the score: (1) tool names lack action verbs, 'dodaj_racun_tool', 'plati_racun_tool', 'izvjestaj_tool', 'dohvati_neplacene_tool', 'posalji_mail_duplikat_tool' all append '_tool' suffix, which is redundant and violates verb_noun convention; (2) descriptions are present but inconsistent in quality, some are adequate (100-200 chars) but others are verbose with formatting issues ('sting' typo in dodaj_racun_tool description); (3) parameter schemas use basic JSON Schema with types (string, float, integer) and descriptions, but enum constraints are documented only in descriptions, not as JSON Schema enums, forcing LLMs to parse text rather than selecting from machine-readable options; (4) no output schemas are documented for any tool, LLMs cannot infer what fields to expect in responses; (5) error handling is minimal, no recovery guidance visible in tool definitions; (6) one tool (dohvati_neplacene_tool) takes zero parameters, which is unusual but acceptable; (7) two resources are present (placanje://ogranicenja, placanje://savjeti) with descriptions, but these are supplementary, not core tools. The dodaj_racun_tool and plati_racun_tool appear to perform writes (risk: WRITE) but have no dry-run, confirmation, or idempotency hints. Overall: definitions are functional but lack production-grade polish.
Tool dodaje novi racun za korisnika u bazu podataka. Koristi ovaj tool kada korisnik zatraži dodavanje novog računa, kada agent dobije informacije o računu koji nije u bazi. Parametri: - tip_racuna: (string) mora biti jedan od [struja, voda, plin, komunalije, telefon, internet] - iznos: (float) iznos računa - rok_uplate: (string), datum u formatu YYYY-MM-DD - mjesec: (sting) -godina (integer)
Dohvaća listu neplaćenih računa za korisnika. Koristi kada korinsik želi vidjeti koje račune ima neplaćene. Tools ne prima parametre jer uvijek vraća listu aktivnih neplaćenih računa
Generiše mjesečni izvještaj za korisnika. Koristi kada korisnik zatraži izvještaj za određeni mjesec i godinu, ili kada agent želi ponuditi generisanje izvještaja nakon što korisnik plati račun. Parametri: mjesec (string) i godina (integer)
Tool placa racun za korisnika, Koristi ovaj tool kada korisnik želi platiti račun i kada korisnik potvrdi da želi platiti račun nakon što agent prikaže informacije o računu i upozori na rok uplate. Parametri: tip racuna (string) moze biti neki od: struja, voda, plin, komunalije, telefon, internet mjesec (string) i godina (integer)
All tool names append redundant '_tool' suffix and lack English action verbs (dodaj, plati, dohvati, posalji). Violates verb_noun naming convention. LLMs struggle with non-English verbs and redundant suffixes add noise.
Enum constraints (e.g., tip_racuna must be one of [struja, voda, plin, komunalije, telefon, internet]) are documented only in description text, not declared as JSON Schema enums. LLMs cannot programmatically select valid values and must parse text, risking hallucinated invalid values like 'struja1' or 'electricity'.
No output schemas documented for any tool. LLMs cannot infer response structure (e.g., does dodaj_racun_tool return {id, status, error}? Is izvjestaj_tool output a string or an object?). Forces LLMs to guess and risks misinterpretation of results.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 45 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 34 | - | v1 |
Tool šalje mail pružaocu usluga kada je detektovan duplikat računa. Koristi ovaj tool kada dodavanje računa vrati grešku duplikata i korisnik potvrdi slanje maila. Parametri: - tip_racuna: (string) tip računa za koji je detektovan duplikat - mjesec: (string) mjesec računa - godina: (integer) godina računa
WRITE operations (dodaj_racun_tool, plati_racun_tool, posalji_mail_duplikat_tool) have no confirmation step, dry-run option, or idempotency declaration. Agents can accidentally duplicate records or send multiple emails on retry. Prompt instructs 'Šalji mail samo ako korisnik eksplicitno potvrdi', but this is enforced in prompt logic, not tool design.
Tool description for dodaj_racun_tool contains typo ('sting' instead of 'string') and inconsistent formatting (embedded parameter list duplicates schema). Reduces clarity and suggests incomplete review.
Error handling is not visible in tool definitions. database/crud.py shows custom exceptions (ValidationError, NotFoundError, AlreadyPaidError) but tool wrappers do not document how these map to LLM-actionable recovery steps. E.g., if ValidationError('Račun već postoji'), what should LLM do, suggest sending mail? Ask user for different month?
Parameter descriptions use human language (e.g., 'Datum u formatu YYYY-MM-DD') instead of machine-readable format declarations. LLMs frequently misformat dates even with text hints. Should declare format: 'date-time' in JSON Schema.
Tool 'posalji_mail_duplikat_tool' hardcodes recipient as GMAIL_ADDRESS from env var (tools_mail.py line 26: msg['To'] = GMAIL_ADDRESS). If GMAIL_ADDRESS is the service provider address, this is correct. But description does not clarify recipient, LLM may believe it is sending to user's email, creating confusion.