Loading ToolBench
Convert REST API endpoints to Postman collections
Two tools with complete input schemas and descriptions, but significant gaps in parameter-level documentation and output schema clarity. Tool names follow verb_noun convention (rest_to_postman_*). Descriptions are adequate (150-200 chars) but lack actionable guidance on when to use each tool or what happens on retry. The complex nested schema for rest_to_postman_collection is difficult for LLMs to navigate without field-level descriptions. No error handling guidance or recovery paths documented. Missing idempotency guarantees for write operations.
Creates or updates a Postman collection with the provided collection configuration. This tool helps synchronize your REST API endpoints with Postman. When updating an existing collection, it intelligently merges the new endpoints with existing ones, avoiding duplicates while preserving custom modifications made in Postman.
Creates or updates a Postman environment with the provided environment variables. This tool helps synchronize your REST application's environment configuration with Postman. It supports both creating new environments and updating existing ones in your Postman workspace. Environment variables related to sensitive data (containing 'token' in their names) are automatically marked as secrets.
Output schemas not documented. Tools return success/failure but LLMs cannot plan downstream actions without knowing response structure (e.g., does rest_to_postman_collection return collection_id, collection_uid, or both?).
Nested schema for rest_to_postman_collection lacks field-level descriptions. The collectionRequest object contains deeply nested auth, item, and event structures with minimal guidance on required vs optional fields or valid value ranges.
No error handling guidance. Tools do not document what happens on failure (e.g., 'collection already exists', 'invalid auth type', 'Postman API rate limit exceeded'). LLMs cannot self-correct without actionable error messages.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 50 | 2026-07-28+ | v2 |
Idempotency not declared. Both tools perform write operations (create or update) but do not state whether repeated calls with identical input are safe or produce duplicates/overwrites.
Missing Postman API credentials handling. Tools require Postman API key but no documentation on how credentials are injected or validated. Credentials must never be tool parameters.