Loading ToolBench
A FastMCP-based todo list server with MongoDB backend, MQTT integration, and task management capabilities
This server has 16 tools with moderately good naming (action verbs present) and acceptable descriptions (most in the 100-200 character range), but critical gaps in schema definition, parameter documentation, and error handling. Most tools have input schemas visible, but many lack detail on constraints, ranges, and type specificity for complex parameters (e.g., 'filter' and 'updates' fields are typed as 'object' with minimal guidance). Parameter descriptions are present but often vague, e.g., 'Optional context for logging' appears on 8 tools without explaining what context structure is expected. Output schemas are entirely undocumented; the rubric requires LLMs to know what fields to expect, but no tool specifies return structure. Error handling is absent, no tool provides recovery guidance or actionable error messages. The server lacks error categorization (retryable vs. fatal) and confirmation patterns for destructive operations (delete_todo_tool, delete_lesson). Security is not explicit: no documented permission gates, audit trails, or scope declarations. Composition is reasonable, tools are single-responsibility, but the interface does not match chat data model well (e.g., todos require 'project' and 'target_agent' strings with no discovery mechanism or validation). Overall, this is a C-grade server: functional but requiring significant hardening for production agent use.
Add a lesson learned entry to the lessons collection.
Creates a task in the specified project with the given priority and target agent. Returns a compact representation of the created todo with an ID for reference.
Delete a lesson learned entry by ID.
Delete a todo by ID. Permanently removes a todo item from the database. This action cannot be undone.
Retrieve a specific lesson learned by ID.
Get a specific todo by ID. Retrieves detailed information about a todo item including its description, status, priority, and creation/completion information.
List all lessons learned with optional filtering and pagination.
No output schemas documented. LLMs cannot predict what fields (id, title, status, created_at, etc.) are returned by any tool. This breaks response-field-naming consistency and forces agents to hallucinate downstream field names.
Complex parameters ('filter', 'updates', 'metadata', 'projection') typed as 'object' with minimal guidance. 'filter' claims to be MongoDB-style but no schema shows allowed fields (status, priority, project, created_at, etc.). LLMs will guess and pass invalid filters.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 53 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 35 | - | v1 |
List todos filtered by status. Returns todos matching the specified status with essential fields.
Mark a todo as complete by ID.
Get the latest MQTT message from a specified topic.
Publish MQTT message to a specified topic.
Query todos with flexible filtering options. Searches the todo database using MongoDB-style query filters and projections. Returns a collection of todos matching the filter criteria.
Search lessons learned using text search or filters.
Search todos using text search or filters.
Update an existing lesson learned by ID.
Update an existing todo by ID. Modifies specified fields of a todo item. Common fields to update include status, priority, description, and project.
No error handling or recovery guidance. Tools lack actionable error messages. If update_todo_tool fails because todo_id doesn't exist, the agent gets nothing to act on. No categorization of errors (retryable vs. fatal). No suggestion of alternative tools (e.g., 'User not found; try search_todos() first').
Destructive operations (delete_todo_tool, delete_lesson) lack confirmation or dry-run capability. No mention of whether they can be undone. Agent could delete critical data without warning.
'ctx' parameter (context for logging) appears on 8 tools with no definition of expected structure. Is it a dict of {user_id, agent_name, session_id}? A string? LLMs will pass garbage.
'project' parameter in add_todo_tool lists 7 hardcoded enum options but no other tool documents valid project values. If a user says 'create a todo in my custom project', the tool fails. No discovery mechanism (e.g., list_projects()) to let agents find valid projects.
Pagination not fully specified. query_todos_tool and list_lessons accept 'limit' but don't document whether they return total_count, next_cursor, or next_offset. If results exceed limit, agents don't know how to fetch more.
No permission gates or scope declarations. No mention of who can delete todos, whether read/write separation exists, or audit trails. No tool describes required permissions (read:todo, write:todo, admin:todo).
MQTT tools (mqtt_publish_tool, mqtt_get_tool) are loosely specified. mqtt_publish_tool takes 'topic' and 'message' as free strings with no format guidance. What topics exist? What message structure is expected? No schema for retained messages.
search_todos and search_lessons accept 'query' as a free string with no format hints. Full-text search? Regex? MongoDB query syntax? LLMs will guess and pass invalid queries.