Research quality assurance tools: citation verification, overclaim detection, parameter audit, reproducibility checks, statistical rigor, adversarial review, claim-evidence mapping, and full integrity checks.
Rigorously has 7 tools with explicit schemas and descriptions, but significant gaps limit production readiness. Tool names lack action verbs (check_paper, verify_citation are acceptable; audit_parameters, audit_time_units are weak). Descriptions are present but generic (avg ~80 chars, below 194-char baseline). Parameter descriptions exist but lack constraints (no enums, ranges, or format specs). Output schemas are undocumented, callers cannot predict response structure. Error handling is absent from visible code. One tool (verify_numbers) has WRITE risk but no confirmation pattern. Tool composition is reasonable (single concerns), but missing pagination, result limits, and chaining IDs in responses.
Check ODE parameter consistency in a Python file (comments vs code values).
Audit time unit consistency across ODE model files. Detects mismatched time units (hours vs minutes) and verifies coupling/solver code has proper conversions.
Scan a manuscript file for overclaimed results and language issues.
Run all integrity checks on a manuscript (.tex or .md) and optional .bib file.
Generate a full Markdown integrity report for a manuscript.
Verify a single DOI against the CrossRef API.
Verify that numbers in LaTeX tables match Python script output. Extracts numbers from tabular environments, runs the script, and flags discrepancies >5% (warning) or >20% (critical).
Output schemas undocumented. Tools return JSON but callers cannot predict field names, types, or structure. LLMs cannot plan downstream calls or extract data reliably.
No error handling guidance. Code shows no try-catch, validation, or recovery hints. If a file is missing or API fails, LLM receives raw exception or silent failure with no actionable next step.
verify_numbers has WRITE risk (executes arbitrary Python scripts) but no confirmation or dry-run pattern. Agents could accidentally run malicious code without safeguards.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 54 | 2026-07-28+ | v2 |
Parameter descriptions lack constraints. 'timeout' has no min/max (default 120s but unbounded). 'warning_threshold' and 'critical_threshold' lack range specs. LLMs may pass invalid values.
Tool names use weak verbs. 'audit_parameters' and 'audit_time_units' are vague, do they check, validate, report, or fix? Stronger names: 'validate_ode_parameters', 'check_time_unit_consistency'.