Loading ToolBench
MCP server for LaTeX file creation, editing, validation, and compilation
The MCP LaTeX Server demonstrates solid tool coverage with 8 well-structured tools. All tools are explicitly registered using FastMCP decorators with complete input schemas. Tool names follow verb_noun conventions (create_, edit_, read_, list_, validate_, extract_, compile_). Descriptions are present for all tools and range from 67 - 140 characters, meeting the minimum threshold. Input schemas use Pydantic Field annotations with type hints and parameter descriptions. However, there are meaningful gaps: (1) Missing per-parameter descriptions for some optional fields (e.g., 'packages' array items have no description; 'geometry' description is brief); (2) No output schema documentation beyond Pydantic model definitions, LLMs cannot easily infer what fields are returned without reading code; (3) Error handling is basic (ValueError, FileNotFoundError raised but no recovery guidance); (4) No idempotency or atomicity guarantees documented for write operations; (5) edit_latex_file has a truncated error handler (incomplete except clause at end of source). The structured output models (FileResult, ValidationResult, StructureInfo, CompileResult) are well-defined but not exposed in tool docstrings, forcing LLMs to infer return structures.
Compile a LaTeX file to PDF using pdflatex.
Create a LaTeX document from a bundled template file.
Create a new LaTeX document with specified content and structure.
Edit an existing LaTeX file by replacing, inserting, or appending content.
Extract document structure and metadata from a LaTeX file.
List all .tex files in a directory.
Read and return the contents of a LaTeX file.
Output schema documentation missing from tool descriptions. Pydantic models (FileResult, ValidationResult, StructureInfo, CompileResult, FileListResult) are defined but never referenced in @mcp.tool() docstrings. LLMs must infer return structure by reading code or receiving actual responses.
Incomplete error handling in edit_latex_file: truncated except clause at line end ('raise ValueEr' is cut off, likely 'ValueError'). This is a syntax error that will prevent the tool from running.
No recovery guidance in error messages. When search_text is not found or a file doesn't exist, the tool raises ValueError or FileNotFoundError with minimal context. LLMs receive no hint about what to do next (e.g., 'Try list_latex_files first' or 'check file_path spelling').
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-21 | D | 57 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 16 | - | v1 |
Perform LaTeX syntax validation (structure, braces, environments, references).
Parameter descriptions for array/nested items are missing or generic. 'packages' (array of strings) has description 'Extra LaTeX packages to include' but no guidance on format (\pkg{foo} vs foo vs foo.sty). Geometry description is minimal ('Geometry settings, e.g. margin=1in'), no mention of required LaTeX syntax or valid keys.
No documented constraints on edit_latex_file's 'search_text' replacement. If search_text appears multiple times, all occurrences are replaced. This can cause unintended side effects. Behavior should be documented: 'Replaces ALL occurrences of search_text' or 'Replaces only the first occurrence'.
compile_latex has no timeout or resource limit documented. pdflatex can hang indefinitely on complex or malformed documents. An agent could block waiting for compilation. No guidance on handling slow compilations.
Idempotency not documented. create_latex_file overwrites existing files without warning. edit_latex_file with search_text replacement is not idempotent if search_text appears multiple times. Agents may not know if retrying a failed call will corrupt existing work.