Loading ToolBench
Server defines 6 tools with complete input schemas and descriptions in Japanese. However, critical gaps limit production readiness: (1) All descriptions are in Japanese, not English, LLMs default to English and may misinterpret intent. (2) No output schemas documented, LLMs cannot plan downstream calls or validate responses. (3) No error handling guidance, tools return raw JSON without recovery hints. (4) Parameters lack format constraints (e.g., project_code format, cases array size limits). (5) No tool annotations (readOnlyHint, destructiveHint) despite clear read/write distinction. (6) Bulk operation (create_test_cases_in_bulk) lacks per-item error reporting. Average tool score: 52/100.
テストスイートを作成します
テストケースを作成します
複数のテストケースを一括で作成します
テスト実行を作成します
プロジェクト一覧を取得します
指定したプロジェクトのテストケース一覧を取得します
All tool and parameter descriptions are in Japanese. LLMs default to English and may misinterpret intent or fail to select tools correctly. Descriptions must be in English for cross-language LLM compatibility.
No output schemas documented. LLMs cannot infer response structure, plan chained calls, or validate results. Each tool must document its return type (e.g., 'Returns object with fields: id (number), name (string), created_at (ISO 8601 string)').
No error handling guidance. Tools return raw JSON or errors without recovery hints. Implement pattern: 'If project_code not found, call get_projects() first to list valid codes.' Include error classification (retryable vs user-fixable).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 47 | 2026-07-28+ | v2 |
Missing tool annotations. Destructive tools (create_*, write operations) lack destructiveHint=true. Read-only tools lack readOnlyHint=true. Annotations guide agent safety and retry logic.
create_test_cases_in_bulk lacks per-item error reporting. If 1 of 50 cases fails, the entire operation fails with no indication which case(s) succeeded. Return {success: [], failed: [{case, error}]} to enable partial recovery.