Loading ToolBench
A Model Context Protocol server that provides symbolic reasoning capabilities using Logic-LM framework and Answer Set Programming (ASP)
Logic-LM MCP server has 11 tools with significant definition quality gaps. Most tools lack comprehensive input parameter descriptions and proper schema documentation. While tool names follow verb-noun convention (good), descriptions vary widely in quality, some are clear (check_solver_health, solve_asp_program) but many are vague or assume domain knowledge without explanation (translate_natural_language_to_asp, refine_program_with_error). Critical issue: complex parameters like 'trace' (array), 'result' (object), and 'bindings' (object) have minimal descriptions. No output schema documentation is visible. No enum constraints on string parameters. No error handling guidance. This server would struggle to guide an LLM toward correct tool selection and usage.
Check Logic-LM server and Clingo solver health status.
Get comprehensive ASP translation guidelines (cached for efficiency). Returns the complete ASP Logic Translation Guidelines document that provides comprehensive instructions for translating natural language into Answer Set Programming format. This is cached to avoid repeated context consumption.
Get lightweight instructions for translating a specific natural language problem to ASP. This tool provides concise, problem-specific instructions for LLM-based ASP translation. For full guidelines, call get_asp_guidelines() first (cached for efficiency).
Directly verify and solve an ASP program using the Clingo solver. This tool allows direct input of Answer Set Programming code for verification and solution finding. Useful for testing ASP programs or working with pre-formulated logical constraints.
Object and array parameters lack schema definition and meaningful descriptions. 'trace' is described as 'List of trace messages' with no explanation of structure, format, or purpose. 'result' parameter is 'LogicResult object with answer sets' with no documented fields. 'bindings' is 'Dictionary mapping variable names to their values' with no type info for values or examples. LLMs cannot correctly structure these inputs without detailed schema.
No output schema documentation visible for any tool. Users and LLMs cannot determine what fields are returned, their types, or how to chain results into downstream tools. For example, solve_asp_program likely returns a LogicResult but the structure is undocumented. This breaks tool composition and forces LLMs to guess.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 0 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 49 | - | v1 |
Descriptions for domain-specific tools assume heavy prior knowledge of ASP, Logic-LM, and Clingo. 'translate_natural_language_to_asp' says 'Generate LLM instructions... referencing comprehensive guidelines' without explaining what this tool actually does or when to call it. An LLM unfamiliar with ASP cannot determine if this is a solver, a translator, or a guide generator. Descriptions must state WHAT, WHEN, and WHY.
No error handling guidance. If solve_asp_program encounters an unsolvable program, times out, or returns no solutions, there is no documented recovery path. If instantiate_template fails due to invalid bindings, no error message hints at what went wrong or what to try next. LLMs need explicit error descriptions and recovery suggestions.
'trace' parameter appears in multiple tools but is never defined. Is it a list of strings? A list of objects with timestamps and messages? Required or optional? An LLM cannot generate valid input without this clarity. The parameter description must specify structure, format, and whether it's mandatory.
No pagination or result-limiting documented. 'list_templates' and 'get_all_templates_info' do not specify whether results are capped, sorted, or paginated. If there are hundreds of templates, returning all of them wastes tokens and risks context overflow. Tools returning collections must declare limits and offer pagination.
Program validation and format constraints not documented. 'solve_asp_program' accepts an ASP program string but does not specify syntax requirements, encoding, or constraints. Can it be empty? What is the maximum size? Are there reserved keywords? Without constraints, LLMs generate invalid input.
Template instantiation behavior undefined. 'instantiate_template' does not document whether it validates bindings, whether missing bindings error or are treated as optional, or whether it returns the rendered program or a LogicResult. LLMs cannot plan multi-step workflows without this detail.