Loading ToolBench
Model Context Protocol server providing information retrieval and Teams messaging capabilities with content indexing, search, and embedding generation
This MCP server has fundamental definition quality issues. Tool definitions are partially visible across multiple files (ir/Dockerfile, ir/mcp/server.py, mcp_server/server.py), creating ambiguity about actual registration and schemas. While 11 tools are listed, only the search tool has a complete, visible schema in the code sample. Descriptions range from adequate to generic. Several tools lack proper parameter descriptions, and error handling guidance is absent. The fastmcp framework is used but actual tool registration code is not shown in the provided source excerpts, making it impossible to verify that schemas are properly bound to tool definitions in the framework.
Delete indexed content.
Get count of indexed content.
Get recent messages from a specified chat, store in DuckDB with embeddings.
Index a piece of content.
Index messages from Teams.
Perform information retrieval search.
List all Teams chats the agent is a member of and store in DuckDB.
Tool registration code not visible. Only partial schema definitions shown in code sample. Cannot verify that fastmcp actually registers tools with complete input schemas and return types as declared.
Output schemas not documented. No return type specifications visible for any tool. The Dockerfile shows simulated FastAPI responses, but actual MCP server output formats are not declared in the visible source.
Destructive operations lack confirmation or dry-run guidance. delete_content and send_message are write/communication operations with no documented safeguards, error recovery, or confirmation-request patterns.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 44 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 44 | - | v1 |
Search indexed content.
Hybrid search over messages: mode can be 'bm25', 'vector', or 'hybrid'.
Advanced search with LLM reranking: first retrieves candidates with hybrid search, then optionally uses LLM to rerank results based on query relevance.
Send a message to a specified chat.
Parameter descriptions are generic or missing context. 'Optional specific source ID to count' and 'Optional timestamp to index messages after' lack actionable detail about format, range, or usage triggers.
Multiple search tools with overlapping names cause LLM ambiguity. search, search_messages, search_messages_llm_rerank, and ir_search all perform search. The differentiation is unclear, when should the LLM choose ir_search over search?
No error handling guidance. Descriptions do not explain failure modes, retryability, or recovery actions. LLMs cannot determine whether a tool failure is transient or requires user intervention.
Pagination support unclear or missing. search accepts limit/offset parameters, but other list-like tools (list_chats, get_messages) do not advertise pagination. No total_count or next_cursor patterns visible.
Tool composition unclear. get_messages and send_message accept chat_id, but list_chats does not document whether its output includes the chat_id field needed to call the other two tools.