Loading ToolBench
An MCP server over the SAM.gov Contract Opportunities API. Every result carries the exact URL it came from, and missing fields come back missing instead of guessed.
Strong tool definitions with excellent naming, clear descriptions, and well-structured schemas. All 5 tools follow verb_noun convention (search_, get_, list_, fetch_, server_). Descriptions are detailed (100-250 chars) and explain WHEN to use each tool. Input schemas use Zod with proper types, patterns, and constraints. However, output schemas are not formally documented in the tool registration, they exist in code but are not visible to the LLM. Error handling is present but could be more granular. No tool annotations (readOnlyHint, etc.) despite all tools being read-only.
Downloads a single attachment by its resource URL, as returned by list_attachments. Returns UTF-8 text when the document is text, and otherwise returns the content type and byte length without guessing at the contents. Binary formats such as PDF are reported, not silently decoded into noise.
Retrieves a single notice by its solicitation number, within a posted-date window. Abstains explicitly when the number is not found in that window, because 'not in this window' and 'does not exist' are different answers.
Returns the downloadable attachment URLs for one notice. An empty list means the notice carries no attachments, which is a real and common state and is reported as zero rather than as an error.
Searches federal contract opportunities in a posted-date window. postedFrom and postedTo are required, MM/dd/yyyy, at most one year apart. Returns normalised notices, each with its attachment links. The returned count is the length of the list actually returned, which is not the same as totalRecords across all pages.
Reports the server's operational mode (live or fixtures) and configuration.
Output schemas not formally documented in tool registration. LLMs cannot see what fields search_solicitations, get_solicitation, list_attachments, and fetch_attachment return. The grounding/provenance structure exists in code but is invisible to the protocol.
No tool annotations despite all tools being read-only. Adding readOnlyHint=true to all 5 tools would signal to agents that these are safe to call without side effects and can be retried freely.
Error handling returns SamError messages as plain text. While readable, errors lack structured classification (retryable vs user-fixable vs fatal) and recovery guidance. E.g., 'No notice numbered exactly X was posted between Y and Z' could suggest 'Try widening the date range' as a recovery hint.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2026-07-28+ | v2 |
fetch_attachment returns binary content as 'content type and byte length without guessing'. This is correct, but the response structure is not documented in the schema. LLMs cannot plan downstream handling of text vs binary responses.
search_solicitations returns 'returned' (actual count) and 'total_records_upstream' (SAM.gov total). The distinction is explained in the description but not in the output schema. LLMs may conflate these or misuse pagination.