Loading ToolBench
Backend for Clouisle application - a FastAPI-based system supporting tool management, MCP client integration, chat execution, and custom tool execution
Clouisle Backend exposes 13 tools via FastAPI HTTP transport. Critical assessment reveals severe definition quality gaps across the entire toolkit. Most tools have descriptions present but extremely minimal (under 50 chars). The remaining 7 tools have partial schemas but many lack parameter descriptions. Naming conventions are inconsistent and sometimes ambiguous. No tool includes structured output documentation. Error handling guidance is absent from all tool definitions. Security patterns are not evident (no permission gates, scope declarations, or audit trail guidance visible in definitions).
Ask user for input during tool execution.
Create a memory entity in the knowledge graph.
Create a memory relation between entities.
Built-in tool: Get current time.
Get memory subgraph for an entity.
Built-in tool: Get weather (mock implementation).
Inspect asset metadata and capabilities within conversation scope.
Six tools (create_memory_entity, create_memory_relation, update_memory_entity, search_memory, get_memory_subgraph, ask_user) have completely empty input schemas with no parameters defined. Per hard scoring rule, schema score must be 0 for these tools. This makes them unusable by LLMs, the agent cannot know what inputs to provide.
All tool descriptions are extremely short (under 50 characters) and lack critical context. 'Create a memory entity in the knowledge graph' does not explain WHAT fields are required, WHEN to call this vs alternatives, or WHAT it returns. LLMs cannot properly select these tools with vague descriptions.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 56 | 2026-07-28+ | v2 |
| 2026-03-09 | C | 61 | - | v1 |
Search knowledge base for relevant contexts.
Materialize asset to sandbox workspace.
Parse asset content with truncation support.
Read asset content with max character limit.
Search memory graph for entities and relations.
Update a memory entity.
No output schemas or return type documentation visible for any tool. LLMs have no way to know what fields a tool returns, which breaks downstream tool chaining. Per pattern, tools must document their output schema so agents can extract the right data for follow-up calls.
Parameter descriptions are missing or generic across most tools. For example, 'read_asset' has 'max_chars' with description 'Maximum characters to read', does not explain valid range (min/max bounds), whether 0 means unlimited, or what happens if exceeded.
No error handling guidance in any tool definition. If 'read_asset' fails because the asset is not found, or 'knowledge_search' times out, LLMs have no instruction on what to do next, retry? call a different tool? ask the user? Per pattern:recovery-guide, error responses must tell the LLM the recovery action.
Naming inconsistency and ambiguity: 'parse_asset', 'read_asset', and 'inspect_asset' all operate on assets but names do not clearly distinguish what each does. 'inspect_asset' could mean introspect metadata, read content, or validate structure, unclear.
Memory tools (create_memory_entity, create_memory_relation, update_memory_entity, search_memory, get_memory_subgraph) lack any explanation of the knowledge graph schema. What are valid entity types? What relations exist? What fields are searchable? Without schema documentation, LLMs cannot use these tools effectively.
No tool annotations visible (readOnlyHint, destructiveHint, idempotentHint). Per current MCP spec, tools should be annotated to help clients understand side effects. 'materialize_asset' and all memory-write tools should be marked destructiveHint or explicitly state they modify state.
'ask_user' tool has empty input schema. This violates elicitation pattern, if the tool is meant to prompt for user input, the schema must specify what prompts/questions are valid, and what response types it expects.