Loading ToolBench
A full-stack boilerplate combining LangGraph agent, RAG (Retrieval-Augmented Generation) with vector search, memory management, and MCP tool servers for web search and SQLite access. Includes FastAPI backend with streaming, React frontend, and multiple tool implementations.
This boilerplate MCP server presents a collection of 10 tools with highly variable quality. While tool names follow reasonable verb-noun conventions (get_contact_info, save_memory, recall_memories, delete_memory, brave_web_search, web_search, web_news, query_memories, get_memory_stats, search_memories), the definitions suffer from several critical gaps. Most tools have descriptions (positive), but many descriptions are generic or lack actionable guidance for LLM selection. Input schemas are present and properly typed for most tools, but parameter descriptions vary from detailed to minimal. The most significant issue is that error handling is either absent or minimal across all tools, there is no guidance for agents on what to do when operations fail. Additionally, output schemas are documented for some tools but absent or vague for others. The server demonstrates basic competency in naming and structural organization but falls short of production-grade quality.
Search the web using Brave Search. Use this tool when: - The user asks about current events or news - The RAG context doesn't have relevant information - The user explicitly asks to search online - Questions about topics not in the codebase
Delete a memory containing a specific keyword. Use this tool when the user asks to: - Forget something - Delete a memory - Remove something from memory
Get the contact information of the application owner/support team. Use this tool when the user asks for: - Email address of the owner/admin/support - Phone number to contact - How to reach support or the team - Contact details or contact information - Ways to get in touch
Get statistics about saved memories. Returns: - Total count - Categories breakdown - Oldest and newest memory dates - Recent memories preview
Execute a READ-ONLY SQL query on the memories database. Use this for: - Counting memories: SELECT COUNT(*) FROM memories - Filtering: SELECT * FROM memories WHERE memory LIKE '%keyword%' - Analytics: SELECT category, COUNT(*) FROM memories GROUP BY category - Time queries: SELECT * FROM memories WHERE created_at > date('now', '-7 days') Table schema: - id: INTEGER (primary key) - user_id: TEXT (default 'default') - memory: TEXT (the saved memory content) - category: TEXT (personal, preference, fact, general) - created_at: TIMESTAMP - updated_at: TIMESTAMP ONLY SELECT queries allowed for safety.
Missing or insufficient error handling across all tools. No guidance for agents on retryability, user-fixable errors, or fatal failures. Tools like delete_memory and save_memory have try-catch blocks but return generic error messages.
Output schemas not documented for any tool. Callers cannot predict what fields will be returned, forcing LLMs to guess at response structure and preventing reliable chaining of tool calls.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 57 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 54 | - | v1 |
Retrieve saved memories for a user. Use this tool when the user asks: - What do you remember about me? - What did I tell you to save? - What are my saved preferences? - Recall my memories
Save something to the user's long-term memory. Use this tool when the user asks you to: - Remember something about them (name, preferences, etc.) - Save information for later - Store something in memory - Don't forget something - Keep track of something Common triggers: "remember", "save", "store", "don't forget", "keep in mind"
Search memories by keyword. Args: keyword: Word or phrase to search for limit: Max results (default 10)
Search for news articles using DuckDuckGo News (FREE). Use this tool when: - User asks about "latest news", "recent news", "current events" - User wants to know what's happening in the world
Search the web using DuckDuckGo (FREE, no API key). Use this tool when: - The user asks about current events, news, or recent information - The RAG context doesn't have relevant information - The user explicitly asks to search online - Questions about topics not in the codebase - User says "search", "look up", "find online"
query_memories exposes raw SQL interface to LLMs, creating security and usability risks. No input validation shown; LLMs could craft malicious queries. Tool name 'query_memories' is too generic and does not clearly indicate it accepts arbitrary SQL SELECT statements.
get_memory_stats has no input schema (empty required array, no properties). Per hard scoring rules, schema score must be 0 for tools lacking input schemas. Parameter descriptions are also missing.
Web search tools (brave_web_search, web_search, web_news) lack output field documentation and do not specify result limits or pagination. Tool descriptions do not clarify how many results will be returned or the structure of the response.
Memory tools use generic user_id with default 'default'. No guidance on whether multi-user scenarios are supported or how to disambiguate users in a shared agent context.
delete_memory uses LIKE pattern matching on memory_keyword, which could match unintended records. No confirmation step for this irreversible operation. Agents could accidentally delete multiple memories with a single overlapping keyword.
Redundant tools: save_memory, recall_memories, delete_memory, search_memories, and query_memories all operate on the same memories database. This fragmentation increases cognitive load on LLM tool selection and creates multiple paths to the same data.
brave_web_search and web_search are nearly identical but use different backends. Tool descriptions do not explain when to use one vs the other, forcing LLMs to guess or try both.