Loading ToolBench
The Overleaf MCP server exposes 2 tools with basic functionality but suffers from critical definition quality gaps. Tool naming is weak (list_of_files uses underscore+plural inconsistently, read_file is generic). Descriptions exist but are sparse (140-180 chars, below the 194-char baseline for A+ tools). Parameter descriptions are present for read_file but minimal. No output schemas are documented, no pagination support for list_of_files, and error handling is missing actionable recovery guidance. The server reads files from an Overleaf project via git clone, a reasonable use case, but the tool interface does not follow production patterns for file operations.
Returns a list of files in the Overleaf project. Returns: list: A list of file paths in the Overleaf project.
Reads the content of a specific file in the Overleaf project. Args: filename (str): The name (path) of the file to read. The name is relative to the project root. For example, "main.tex" or "sections/chapter1.tex". Returns: str: The content of the specified file.
Missing output schemas for both tools. LLMs cannot plan downstream calls or structure reasoning without knowing what fields are returned. list_of_files returns a list of strings, but this is undocumented in the tool definition.
list_of_files has no pagination parameters (limit, offset) and no stated limit on returned items. Returning all files in a large Overleaf project could blow the context window. The description does not specify max results or suggest pagination.
Weak naming: 'list_of_files' uses underscore+plural ('of' + 'files') inconsistently. Production tools use 'list_files' or 'get_files'. The underscore-heavy name also does not follow verb_noun convention clearly.
Error handling is minimal. read_file raises FileNotFoundError, but the tool definition does not document error cases or recovery guidance. The LLM receives a Python exception message, which is not actionable. Missing: 'Try calling list_of_files() first to confirm the file exists.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 32 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 30 | - | v1 |
list_of_files description is vague (140 chars, below baseline 194). It states 'Returns a list of files in the Overleaf project' but does not explain that it lists all files recursively, whether it includes directories, or how to use it in sequence with read_file.
No documentation of the filename parameter format. The description for read_file says 'relative to the project root' and gives examples ('main.tex', 'sections/chapter1.tex'), but does not state whether paths with '..' are rejected, or whether leading/trailing slashes are allowed. This invites path traversal attacks.