Loading ToolBench
web-algebra exposes 7 tools via MCP, but critical definition quality issues prevent a higher score. Tool names lack action verbs (GET, PATCH, Bindings, Current, Execute, Filter, ForEach are mostly nouns or generic). Several tools have descriptions that exceed 1024 characters (PATCH is 402 chars, ForEach is 358 chars, which approach but don't violate the limit). Input schemas are present for all tools and use proper JSON Schema with type definitions. However, parameter descriptions are sparse or missing, many parameters lack descriptions explaining their purpose. Error handling is absent: no tools provide recovery guidance or error classification. No tool declares permissions, and some tools (Execute, ForEach) appear to accept arbitrary nested operations, creating security surface without validation guidance. The architecture suggests this is a composition/DSL engine rather than a traditional tool set, which trades clarity for flexibility but hurts LLM discoverability and safety.
Extracts the sequence of binding dictionaries from SPARQL Result. Converts Result table format to sequence for use with ForEach.
Returns the current ForEach sequence item. Similar to XSLT's current() function or select=".", this returns the current item from a ForEach sequence operation. Used to access the current sequence item within ForEach operation processing. Returns the current sequence item directly.
Executes a (potentially nested) operation from its JSON representation. The operation is expected to be an instance of the Operation class.
Filters SPARQL results using filter expressions, similar to XSLT predicates. Currently supports positional filtering (e.g., [1], [2]) with future extensibility. Currently supports positional access: Numeric values select by position (1-based, following XSLT convention) and Returns the filtered SPARQL results with matching bindings. Future versions may support field-based and complex filter expressions.
Applies operations to each item in sequences or SPARQL results, similar to XSLT's xsl:for-each. Similar to XSLT's <xsl:for-each select="...">, this operation can iterate over: Sequences: Each item becomes the context for the operation and Result (SPARQL results): Each result row (ResultRow) becomes the context. Returns a sequence of operation results.
Tool names do not start with action verbs. GET, PATCH, Bindings, Current, Execute, Filter, ForEach are mostly nouns or generic verbs. LLMs cannot infer intent from names alone. Should be: get_rdf, update_rdf, extract_bindings, get_current_item, execute_operation, filter_results, iterate_sequence.
Multiple parameters lack descriptions. Current tool has no parameter descriptions (empty properties object). Execute's 'operation' parameter has minimal guidance ('An instance of an Operation to execute'). Bindings, Filter, ForEach parameter descriptions are brief or missing context on expected structure and purpose.
No error handling or recovery guidance in any tool. Descriptions do not explain what errors might occur, how to retry, or what the LLM should do next. E.g., GET does not explain HTTP failure modes; PATCH does not clarify why an update might fail (malformed SPARQL, permission denied, graph not found).
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 51 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 46 | - | v1 |
Retrieves RDF data from a named graph using HTTP GET. The URL serves as both the resource identifier and the named graph address in systems with direct graph identification. Returns the RDF graph describing the resource at that URL.
Updates RDF data in a named graph using HTTP PATCH with SPARQL Update. The URL serves as both the resource identifier and the named graph address in systems with direct graph identification. Updates the RDF graph at that URL using the SPARQL update payload provided in the `update` argument. Only INSERT/WHERE and DELETE WHERE forms of SPARQL Update are supported by PATCH! Only a single update operation is allowed. Returns True if the operation was successful, False otherwise. Note: This operation does not return the updated graph, it only confirms the success of the operation.
No permission declarations. Tools that modify state (PATCH, Execute, ForEach) do not declare what permissions they require or validate caller authority. This violates the principle of least privilege and prevents audit trails.
Execute tool accepts arbitrary 'operation' objects with only 'type' as a required field. No validation of operation payload structure, no guidance on valid operation types, no error handling for malformed operations. This creates a large security surface and makes it impossible for LLMs to validate inputs.
Filter tool's 'expression' parameter accepts any type with minimal description. Currently supports positional integers, but description hints at future expression types without constraining what the LLM can pass. This invites hallucinated filter syntax.
Output schemas are not documented. No tool description explains what the return value structure is, what fields to expect, or what data types are returned. LLMs cannot plan downstream calls without knowing what GET returns or what Execute produces.
Composition is opaque. ForEach and Execute accept 'operation' and 'select' parameters that are expected to be nested Operation objects, but the schema does not document the Operation class structure, valid operation types, or how nested operations compose. This makes it impossible for an LLM to construct valid nested calls without trial and error.