Loading ToolBench
TrackMage MCP server has 10 tools with explicitly defined schemas and descriptions. However, quality is inconsistent and mediocre. All tools have names starting with action verbs (create_, list_, update_, get_, detect_, retrack_) which is good. Descriptions exist for all tools but are brevity-focused and lack context on WHEN to use each tool or what the LLM should expect back. Parameter descriptions are minimal (1 - 3 words each). No output schemas are documented, the rubric requires tools to document what they return so LLMs can plan chaining. Error handling is generic (CallToolRequestSchema returns isError flag but no recovery guidance). No tool annotations (readOnlyHint, destructiveHint) despite having 6 WRITE operations and 4 READ_ONLY operations, LLMs cannot distinguish safe from destructive calls. Resources are implemented but tools do not leverage them. No input validation examples visible. Pagination is present on list_* tools but capped at default 30 items per page with no guidance on max limits. Overall, definitions are functional but fall short of production quality, typical of a community server that works but lacks refinement.
Create a new order
Create a new shipment
Detect possible carriers for a tracking number
Get tracking checkpoints for a shipment
List available carriers
List orders from a workspace
List shipments from a workspace
Tool descriptions are uniformly 8 - 25 characters and provide no guidance on WHEN to use the tool, what it returns, or how it differs from similar tools. Baseline for production tools is 194 chars avg. (p10=34, p90=392). LLMs cannot disambiguate tools with vague descriptions.
No output schemas documented for any tool. The rubric requires tools to state what fields they return so LLMs can plan downstream calls and field extraction. For example, create_shipment returns 'Shipment created: {id}' but LLMs do not know the structure of the shipment object or what fields downstream tools (like get_shipment_checkpoints) expect.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 50 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 42 | - | v1 |
Retrack multiple shipments by tracking numbers
Update an existing order
Update an existing shipment
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite having 6 WRITE operations (create_shipment, create_order, update_order, update_shipment, retrack_shipments) and 4 READ_ONLY operations. LLMs cannot distinguish safe idempotent reads from destructive writes and will not know which calls are safe to retry.
Parameter descriptions are minimal (1 - 3 words). For example, workspaceId is described as 'Workspace ID (optional, uses default from .env)' but does not explain what a workspace is, how to obtain one, or how to list available workspaces. Baseline is 72 chars avg per param. Example: 'trackingNumber' lacks format guidance (numeric only? alphanumeric? max length?).
No input validation guidance or error recovery hints. Error responses use generic 'isError: true' with no actionable recovery message. E.g., if create_shipment fails with 'duplicate tracking number', the LLM has no guidance on whether to retry, modify the input, or check for an existing shipment first.
create_shipment, create_order, update_order, update_shipment, and retrack_shipments modify state but have no confirmation step or dry-run mode. Agents making mistakes could trigger duplicate shipments or orders. No mention of idempotency keys or deduplication.
Parameter naming lacks specificity in some cases. 'email' parameter is used in create_shipment, create_order, update_order, and update_shipment but never described as 'customer_email' or 'recipient_email', creating ambiguity about whose email is being set. Baseline suggests suffixing with type: email_address, user_email, etc.
Pagination parameters (page, itemsPerPage) lack constraints. No min/max bounds are stated. Baseline requires min - max for numeric params (e.g. page_size 1 - 100). An LLM could pass itemsPerPage: 100000 causing timeout or memory issues.
No enum constraints on categorical parameters. 'status' parameter in update_order and update_shipment accepts any string with no guidance on valid values (e.g., 'pending', 'delivered', 'failed'). Baseline requires enums for known sets of values to prevent hallucinated invalid states.
list_orders and list_shipments return only default pagination (page 1, 30 items). No nextCursor or total_count guidance in tool descriptions. When agents need to iterate through large datasets, they lack clarity on whether they should loop with page++, use a cursor, or if a max limit exists.