Loading ToolBench
A Retrieval-Augmented Classification system that uses local embedding models via YZMA/llama.cpp and stores query-topic mappings with their vector embeddings in PostgreSQL using pgvector. Exposes classification as a Model Context Protocol server with stdio transport for integration with AI assistants.
Four tools with basic descriptions and visible input schemas, but significant gaps in parameter documentation and output schema clarity. Tool names follow verb_noun convention (add_entry, delete_entry, list_entries, get_topic), which is good. However, parameter descriptions are minimal (e.g., 'The topic name to classify into' lacks format/constraint details), and output schemas are not formally documented in the tool definitions. Error handling is present but generic. No tool annotations (readOnlyHint, destructiveHint) despite clear risk levels. Schemas use jsonschema tags but lack comprehensive type information for all fields.
Add a new query-topic mapping to the classifier.
Delete a classification entry by its hash.
Classify a user query text into a topic name.
List all classification entries currently in the database.
Output schemas not formally documented. get_topic returns {topic, score} and list_entries returns entries array, but these structures are not declared in tool definitions. LLMs cannot plan downstream calls without knowing return field names and types.
Parameter descriptions lack format/constraint details. 'The topic name to classify into' does not specify length limits, allowed characters, or examples. 'The user query text to classify' lacks guidance on expected length or format.
No tool annotations despite clear risk levels. delete_entry is DESTRUCTIVE but lacks destructiveHint annotation. add_entry is WRITE but lacks annotation. list_entries and get_topic are READ_ONLY but lack readOnlyHint. Annotations guide agent safety and retry logic.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | 2026-07-28+ | v2 |
Error handling is generic. Functions return fmt.Errorf wrapping but do not categorize errors as retryable, user-fixable, or fatal. No guidance on recovery steps (e.g., 'If hash not found, call list_entries() to find valid hashes').
list_entries lacks pagination support. No limit, offset, or cursor parameters. If the database grows large, returning all entries will blow context windows and degrade LLM reasoning.