Loading ToolBench
MCP server + Claude Code skills for semantic search over Turkish legal contracts and emails
Legal RAG server has clear, domain-specific tool names and reasonable descriptions (avg 180 chars), but lacks structured output schemas, parameter validation guidance, and error recovery hints. All 6 tools are explicitly registered with @mcp.tool() decorators and have docstrings. However, parameter descriptions are minimal (e.g., 'file_type' lacks enum constraint; 'sender' lacks format guidance), and return types are undocumented, LLMs cannot plan downstream calls or validate results. Error handling returns plain strings without actionable recovery paths. No tool annotations (readOnlyHint/destructiveHint) despite clear risk profiles.
Retrieve all indexed chunks for a specific document. Returns the full text of all chunks from a given source file, reassembled in order. Useful for reading a complete contract or email after finding it via search.
Ingest PDF, DOCX, and UDF documents into the contracts search index. Parses documents, splits into chunks, generates embeddings with BGE-M3, and stores in the vector database for semantic search.
Ingest .eml email files into the emails search index. Parses email bodies and metadata, generates embeddings with BGE-M3, and stores in the vector database for semantic search.
List all indexed documents and emails with statistics. Shows the number of chunks, file types, and indexed file names for both the contracts and emails indices.
Search across indexed legal contracts using semantic similarity. Finds contract clauses, provisions, and sections that are semantically related to the query. Works with Turkish and English queries.
No output schemas documented. Tools return plain strings; LLMs cannot parse structured results or chain calls. E.g., search_contracts returns formatted text, not JSON with fields like [{ source_file, chunk_id, score, text }].
Parameter constraints missing. 'file_type' accepts 'pdf', 'docx', 'udf' but is not declared as an enum, LLMs may hallucinate invalid values. 'limit' is unbounded (no min/max). 'sender' lacks format guidance (email vs name).
Error responses are non-actionable. E.g., 'No results found. The contracts index may be empty -- run /ingest-docs first.' uses slash notation (not MCP tool names) and does not classify error as retryable or user-fixable.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | 2026-07-28+ | v2 |
Search across indexed lawyer-client emails using semantic similarity. Finds emails related to the query by meaning, including email body text, subject lines, and metadata.
No tool annotations. ingest_docs and ingest_email are destructive (WRITE risk) but lack destructiveHint. search_* and list_* are read-only but lack readOnlyHint. Agents cannot distinguish safe from risky operations.
No pagination support. search_contracts and search_emails accept 'limit' but do not return total_count or next_cursor. Large result sets could exhaust context; no way to iterate safely.