Loading ToolBench
Local MCP server for Polish company registries — Biała Lista VAT, KRS, REGON lookups by NIP.
Two tools with clear, domain-specific descriptions (194 - 250 chars, within baseline 34 - 392). Input schemas present with proper JSON Schema structure and type declarations. Parameters have descriptions. However, output schemas are completely undocumented, LLMs cannot plan downstream calls or extract structured data. No error handling guidance (e.g., 'If NIP not found, try searching by company name'). No parameter constraints (e.g., date format validation, NIP format hints). Tool names are descriptive (lookup_by_nip, lookup_by_krs) but lack action verbs (get_, search_). Validation logic exists in code but is not reflected in schema constraints (minLength, pattern, format). No mention of rate limits, timeouts, or API failure modes.
Look up a Polish organization by KRS number in the Krajowy Rejestr Sądowy. Returns the full current registry entry: name, legal form, address, share capital, board members, PKD codes, shareholders (for companies), registration/change dates.
Look up a Polish company by NIP on the Biała Lista (official Ministry of Finance VAT payer list). Returns: legal name, address, VAT status (active/exempt/removed), REGON, KRS, list of confirmed bank accounts. Include a date to check historical status (default: today) — this matters for tax deductibility of costs paid to that vendor.
Output schemas completely undocumented. LLMs cannot infer what fields to expect from lookup_by_nip (legal name, address, VAT status, REGON, KRS, bank accounts) or lookup_by_krs (name, legal form, address, share capital, board members, PKD codes, shareholders). This forces LLMs to guess field names and risks extraction errors.
No error recovery guidance. If NIP is invalid or not found, the tool returns raw API JSON with an 'error' field, but the description does not explain what the LLM should do next (e.g., 'Try a different NIP format' or 'Search by company name instead'). Agents cannot self-correct.
Tool names lack action verbs. 'lookup_by_nip' and 'lookup_by_krs' are descriptive but do not start with a standard verb (get_, search_, find_). LLMs rely on verb prefixes to infer intent. Rename to 'search_company_by_nip' and 'search_organization_by_krs' for clarity.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
Input schema constraints are incomplete. The 'nip' parameter accepts 'with or without dashes/spaces' per description, but the schema has no minLength, maxLength, or pattern constraint. The 'date' parameter has no format constraint (should be 'date' format in JSON Schema). Validation happens in code (_validate_nip, _validate_krs) but is not exposed to the LLM.
No timeout or rate-limit documentation. The code sets timeout=30 for HTTP requests, but the tool descriptions do not mention this. If the Polish government API is slow or rate-limited, the LLM has no guidance on retry strategy or expected latency.