Loading ToolBench
MCP server that lets AI assistants read Overleaf projects, parse LaTeX document structure, and push section-level edits back via Git.
The server has 4 tools with explicit definitions in overleaf-mcp-server.js. Tool names follow verb_noun convention (list_files, read_file, parse_sections, push_section_edit), which is good. However, descriptions are very brief (11-83 chars), falling below the 50-200 char production baseline. Most critically, parameter descriptions lack detail on constraints, formats, and usage context. Input schemas are present and use proper types (string), but lack enum constraints where applicable (e.g., projectKey could validate against known keys). Output schemas are not documented at all, callers have no visibility into what these tools return. Error handling is minimal with no recovery guidance. The server reads/modifies LaTeX files via Git, but lacks confirmation patterns for destructive operations and does not document permission requirements or security boundaries around token handling.
List all TeX files in the Overleaf project
Parse LaTeX sectioning structure of a file (extracts \section, \subsection, etc.)
Push a section-level edit back to Overleaf via Git
Read the contents of a file in the Overleaf project
Tool descriptions are too brief (11 - 83 chars). Production baseline is 50 - 200 chars. Descriptions do not explain WHAT the tool does, WHEN to use it, or what it returns. Examples: 'List all TeX files in the Overleaf project' (51 chars) omits details on structure returned; 'Parse LaTeX sectioning structure' (35 chars, below 20-char floor) is vague about output format and when to call it instead of read_file.
No output schemas documented. Callers have no visibility into what these tools return (data types, fields, structure). This forces LLMs to guess at downstream tool chaining and data extraction. Production baseline: 100% of A+ tools have documented return types.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 51 | 2026-07-28+ | v2 |
Parameter descriptions lack constraint and format details. 'filePath' is described as 'Relative path to the file within the project' but does not specify: allowed path separators, character restrictions, max length, or examples of valid paths. 'projectKey' lacks explanation of valid formats or how to discover available keys. This invites invalid input from LLMs.
push_section_edit is destructive (writes to Git, modifies source) but has no confirmation step, dry-run option, or rollback guidance. If an agent calls this with wrong content, the edit is permanent. No error handling describes how to undo or recover.
No error classification or recovery guidance. If a tool fails (e.g., file not found, Git auth fails), there is no actionable error message telling the LLM whether to retry, ask the user, or abort. Error responses would likely be raw exceptions or empty failures.
Git token is passed via environment variables (OVERLEAF_GIT_TOKEN / OVERLEAF_GIT_TOKEN_FILE), which is correct for server-side injection. However, token masking (maskToken function) only partially sanitizes logs, stack traces or error messages could still leak tokens if exceptions occur during Git operations. No mention of permission scoping (e.g., which branches/commits the token can access).
projectKey parameter is required for all tools. Description says 'Project key from projects config (or \'default\' for single-project mode)' but does not explain how to discover available keys or what happens if an invalid key is passed. This forces agents to guess or trial-and-error.
No idempotency guarantees documented. push_section_edit modifies state (commits to Git). If an agent retries due to ambiguous failure, will the same content be committed again, creating duplicate commits? No deduplication or versioning guidance.