Loading ToolBench
MCP server for FreeTaxUSA tax filing automation via Playwright browser automation
Server has 15 tools with explicit schemas and descriptions. Phase 1 tools (8) are well-defined with clear names and moderate descriptions (100-200 chars). Phase 2-3 tools (7) are stubbed with '[Phase X - Not yet implemented]' prefixes, reducing practical value. Naming follows verb_noun convention (authenticate, get_session_status, fill_taxpayer_info). Schemas use Zod with type definitions. However, descriptions lack actionable context (no 'when to use', no prerequisites, no error recovery guidance). Parameter descriptions are minimal (10-40 chars). Output schemas are not documented, LLMs cannot predict response structure. Error handling wraps all tools but returns generic 'internal_error' without recovery hints. Security concern: authenticate tool accepts email/password as parameters despite mentioning Hermes brokering, credentials should never be params.
Log in to FreeTaxUSA. When Hermes is configured (HERMES_URL/HERMES_CLIENT_TOKEN), the session is brokered by Hermes and no email/password is needed. Otherwise, pass email and password for the embedded browser login (credentials are used once and never stored).
[Phase 3 - Not yet implemented] File Form 4868 for an automatic extension.
[Phase 2 - Not yet implemented] Enter 1099 income data.
[Phase 3 - Not yet implemented] Enter deduction information (standard or itemized).
Set the filing status (single, married_joint, married_separate, head_of_household, qualifying_widow).
Fill the taxpayer personal information section (name, SSN, DOB, address, occupation).
Credentials exposed as tool parameters: authenticate() accepts email/password as string parameters. Credentials must never appear in tool params, they are logged in traces and prompt history. Use server-side secret injection via environment variables.
7 of 15 tools marked '[Phase X - Not yet implemented]' with stub implementations. These tools have schemas but return placeholder errors, reducing server utility. Either remove unimplemented tools or document expected availability timeline.
Output schemas not documented. Tools return JSON via filterPII() but LLMs cannot predict response structure (fields, types, nesting). Document what each tool returns: e.g., get_tax_summary returns {refund_amount: number, agi: number, filing_status: string, completed_sections: string[]}.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 57 | 2026-07-28+ | v2 |
[Phase 2 - Not yet implemented] Enter W-2 wage data.
[Phase 3 - Not yet implemented] Get which sections are complete, incomplete, or have errors.
Get the current calculated federal and state refund or amount owed.
Check if the FreeTaxUSA session is active and which tax year/section is loaded.
Get the tax return overview: refund/owed amount, AGI, filing status, completed sections.
Jump to a specific tax section by name (e.g., "income", "deductions") or SID number.
Read all form fields and their values on the current FreeTaxUSA page.
[Phase 3 - Not yet implemented] Run error check and get review results before filing.
Submit the current FreeTaxUSA page and advance to the next page.
Error handling returns generic 'internal_error' with message only. No recovery guidance, no error classification (retryable vs fatal), no invalid-value feedback. E.g., if navigate_section fails, tell LLM: 'Section not found. Available sections: income, deductions, personal_info. Try navigate_section(section="income").'
Parameter descriptions are minimal (10-40 chars). E.g., navigate_section.section: 'Section name (e.g., "income", "deductions", "personal info")' lacks format constraints, valid enum values, and when to use SID vs name. Expand to: 'Section name (string) or SID number. Valid sections: income, deductions, personal_info, credits, filing_status. Use name for discovery; use SID for direct navigation.'