Loading ToolBench
Single tool 'count_occurence' has critical definition gaps. Tool name lacks action verb (should be 'count_letter_occurrences' or similar). Description is adequate (58 chars) but parameters lack any descriptions, both 'word' and 'letter' have null descriptions, violating the requirement that every parameter must have a description. Input schema is present with proper types (string), but parameter documentation is completely absent. No output schema documented. Error handling returns a bare 0 on exception with no guidance to the LLM. Tool is READ_ONLY (good), but overall definition quality is well below production baseline.
Counts the number of occurrences of a given letter in a word.
Tool name lacks action verb. 'count_occurence' should start with a verb like 'count_' or 'get_count_'. Current name is ambiguous and does not follow verb_noun convention.
All input parameters lack descriptions. Both 'word' and 'letter' have null descriptions. LLMs cannot infer parameter meaning from names alone, 'letter' could mean a single character or a full word.
No output schema documented. Tool returns an integer, but LLMs need explicit documentation of return type and meaning (e.g., 'Returns: integer count of occurrences, 0 if not found').
Error handling returns bare 0 with no guidance. Exception caught and silently converted to 0, LLM cannot distinguish between 'no occurrences' and 'error'. Should return structured error with recovery hint.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 40 | 2026-07-28+ | v2 |
No input validation or constraints. Tool accepts any string for 'letter', should validate that letter is a single character and provide actionable error if not.