Loading ToolBench
MCP server wrapping SWI-Prolog for symbolic reasoning in coding agents
Prolog-MCP has clear, action-oriented tool names (prolog_query, prolog_assert, prolog_retract, prolog_write_file, prolog_load_file, prolog_list_facts, prolog_reset_layer) and detailed descriptions explaining what each tool does and when to use it. However, parameter descriptions are inconsistent, some are thorough (e.g., prolog_assert's 'layer' param explains agent:main vs session:<id>), while others are minimal (e.g., prolog_list_facts 'limit' lacks min/max bounds). Input schemas are present and typed (string, number, optional flags), but output schemas are not documented in the tool definitions, LLMs cannot predict what prolog_query returns beyond 'JSON'. Error handling is implicit (timeouts, query errors mentioned in code but not in tool descriptions). No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels (READ_ONLY, WRITE, DESTRUCTIVE). Security is reasonable (no secrets in params), but parameter validation rules are not stated in descriptions (e.g., 'layer' must match pattern 'agent:<id>' or 'session:<id>', but this is only visible in code comments, not in the tool description).
Assert a fact or rule into the KB. Persists to disk and survives daemon restarts. Use agent:main (default) for permanent knowledge; use session:<id> for ephemeral facts tied to the current session.
List facts in the KB, optionally filtered by functor
Hot-reload an existing .pl file
Execute a Prolog goal and return all solutions as JSON
Clear a session or scratch layer (core and agent layers are permanent)
Retract matching facts or rules from a layer. Removes from disk and reloads — retraction survives daemon restarts.
Write a .pl file to disk and hot-reload it. WARNING: replaces the entire file — not an append. Use for authoring multi-clause rule files (core.pl, scratch/). For individual facts use prolog_assert instead. On syntax error the file is rolled back and the server keeps running.
Output schemas not documented. Tool descriptions do not specify what fields prolog_query, prolog_list_facts, or other tools return. LLMs cannot plan downstream calls or extract required data without knowing the response structure.
Parameter constraints not stated in descriptions. 'layer' parameter requires format 'agent:<id>' or 'session:<id>', but this is only visible in code comments. 'limit' and 'offset' in prolog_list_facts lack min/max bounds. LLMs cannot validate inputs without explicit constraints in descriptions.
No tool annotations despite clear risk levels. Tools are marked READ_ONLY, WRITE, REVERSIBLE, or DESTRUCTIVE in metadata, but these are not exposed as readOnlyHint, destructiveHint, or idempotentHint in the tool schema. Agents cannot determine which tools are safe to retry or have side effects.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 67 | 2026-07-28+ | v2 |
Error handling guidance missing from tool descriptions. prolog_query mentions timeout behavior in code but not in the description. No guidance on what to do if a query fails, times out, or returns no solutions. Descriptions should state 'Returns empty list if no solutions found' or 'Timeout returns error with partial results'.
Parameter descriptions lack actionable detail. prolog_list_facts 'limit' description is '(default 100)', does not state min/max or why 100 is the default. prolog_query 'timeout_ms' lacks guidance on when to use it or what happens on timeout. Descriptions should be 50 - 200 chars and answer WHAT, WHEN, and WHY.