Loading ToolBench
AI agent for civil engineering research paper generation with pipeline orchestration, quality checking, AI review simulation, and submission preparation
PaperFactory exhibits pervasive definition quality gaps across 43 tools. Most tool descriptions are present but generic (avg ~80 chars, well below the 194-char baseline for A+ tools). Parameter schemas are visible but lack depth: many parameters have minimal type info and no descriptions. No output schemas documented. Error handling is absent, tools provide no recovery guidance. Tool naming is reasonable (verb-noun pattern mostly followed) but composition is poor: many tools do overlapping work (e.g., generate_abstract, improve_abstract, analyze_paper all touch paper content). No tool annotations (readOnlyHint, destructiveHint, idempotentHint). The server is a Streamlit UI wrapper around utility modules, not a proper MCP server, no MCP protocol implementation visible in source.
Orchestrate the 5-step paper generation pipeline with state persistence
Address a reviewer comment and track the response
Analyze paper readability and statistics
Check for duplicate references in a reference list
Run quality check on a paper
Mark current pipeline step as complete and store outputs
Convert reference citation style
Create a new revision of a paper with tracked changes
No MCP protocol implementation detected. Source shows Streamlit UI (app.py) wrapping utility modules, not an MCP server. No tool registration via MCP, no JSON-RPC handlers, no transport layer.
Output schemas completely undocumented. Tools return results but no schema is specified for what fields/types agents should expect. Violates pattern:tool and pattern:response-shaper.
Parameter descriptions are minimal or absent. Most params have type but no description of what they control, valid ranges, or format constraints. E.g., 'journal' param lacks enum of valid journal keys.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 32 | 2026-07-28+ | v2 |
Create a publication-quality table figure
Detect the citation style of a reference list
Convert equations to plain text representation
Extract mathematical equations from paper content
Format mean and standard deviation for publication
Format a number for publication with appropriate precision
Generate an abstract from paper body sections
Generate a cover letter for paper submission
Generate LaTeX source from paper content
Generate a PDF document from paper content
Generate a response letter to reviewer comments
Generate paper skeleton/template from a template type
Generate a Word document from paper content
Get a publication-quality color palette
Get recommended figure size for publication
Import references from BibTeX file
Analyze and suggest improvements for an existing abstract
List research data sources, optionally filtered by field
List available paper templates
Load journal submission guidelines
Add equation numbering to paper content
Recommend appropriate figure types for research content
Recommend appropriate statistical tests for data analysis
Reformat paper to match journal guidelines
Render a LaTeX equation to an image
Resume a pipeline from a saved state file
Run AI reviewer simulation for pre-submission peer review
Save a rendered equation image to file
Save a matplotlib figure to file with publication-quality settings
Save table data to CSV file
Setup matplotlib style for publication-quality figures
Check paper submission readiness against journal requirements
Suggest research data sources for a given topic
Track changes made to a paper during revision
Validate paper references for completeness and format
No error handling or recovery guidance. Tools provide no error messages, validation feedback, or hints for LLM retry logic. Violates pattern:recovery-guide.
Overlapping tool responsibilities. generate_abstract, improve_abstract, and analyze_paper all operate on paper content; generate_word, generate_pdf, generate_latex all export. Violates pattern:tool (single responsibility).