Loading ToolBench
Orbit Planner MCP Server with SMB Document Access - provides tools for time queries, execution attestation, and document management from a shared NASA SMB share
Server has 7 tools with basic descriptions and parameter schemas. However, descriptions are minimal (10-50 chars), parameter descriptions lack detail on constraints/formats, and output schemas are not formally documented. Error handling returns generic dicts with 'error' fields but no recovery guidance. No tool annotations (readOnlyHint/destructiveHint). Tool names are reasonable (verb_noun style) but descriptions do not explain WHEN to use each tool or dependencies between them. Schemas are present but sparse, most parameters lack format/constraint details. This is a typical community server in the C-D range.
Add two numbers and return the sum.
Return a fresh, unguessable proof-of-execution token.
Return the same message back.
List the documents available in a folder of the shared NASA SMB share.
Read the content of a document from the shared NASA SMB share (max 5000 chars).
Search the shared NASA SMB share for documents matching a keyword, by filename or text content.
Return the server's current UTC time in ISO-8601 format.
Descriptions are too brief (10-50 chars). Rubric baseline is 194 chars. Descriptions lack WHEN to use, dependencies, and prerequisites. E.g., 'Return the same message back' does not explain the echo tool's purpose in an agent context.
Parameter descriptions are missing or minimal. E.g., 'folder' in list_documents has description 'The folder path to list documents from' but does not specify format (absolute vs relative), constraints (max length), or examples. Rubric requires format/range/allowed values in descriptions.
Output schemas are not formally documented. Tools return dicts with fields like 'documents', 'results', 'content', 'error' but no schema definition visible. LLMs cannot plan downstream calls without knowing return structure. Rubric requires documented return types for all tools.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 48 | 2026-07-28+ | v2 |
Error handling is generic. All tools return {'error': str(e), ...} with no recovery guidance. Rubric requires errors to tell LLM what to do next. E.g., 'Path escapes share root' should suggest 'Use a path within /mnt/share' or 'Call list_documents() to see available folders'.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Rubric rewards these for spec alignment. attest_execution is marked WRITE but has no destructiveHint annotation. This prevents agents from understanding side effects.