Persistent model chats with built-in host shell and file tools enabled, without approval prompts or a sandbox.
ConferLLM defines 4 tools with mixed quality. Tool names follow verb_noun conventions (create_chat, continue_chat, list_sessions) and descriptions exist for all tools. However, critical gaps undermine overall quality: (1) Parameter descriptions are sparse or absent in key tools, create_chat and continue_chat lack descriptions for 'images' and 'image_output_dir' parameters; (2) No output schemas are documented for any tool, making it impossible for LLMs to plan downstream operations; (3) Error handling is not visible in the tool definitions, no guidance on retryability, user-fixable errors, or recovery paths; (4) Security concerns: the 'include_raw_response' parameter suggests raw model output is returned, which may contain tokens or internal details; (5) The 'chat' tool is marked as deprecated in its description but still actively exposed, creating ambiguity; (6) No input validation, enums, or constraints documented (e.g., 'images' array has no min/max length, model names have no constraints). Tool descriptions are present (100+ chars each) and address WHAT and WHEN, but lack specific guidance on WHICH parameters are required in edge cases (e.g., model vs session_id mutual exclusivity in 'chat' is documented but not enforced via schema constraints). Schemas visible in source show basic types (string, array, boolean) but lack required field declarations, pattern constraints, and detailed field descriptions.
Chat with a configured model and enabled host shell and file tools. Supply ``model`` for the first message or ``session_id`` to continue. New integrations should use ``create_chat`` or ``continue_chat``. ``image_path`` is a deprecated alias for ``image_output_dir``.
Continue with stored history and enabled host shell/file tools.
Create a persistent chat with host shell and file tools enabled. ``images`` preserves input order and accepts server-local absolute paths or image data URLs. The result includes a reusable session ID. Model tool calls execute on the server host without approval prompts.
List sessions matching optional query filter.
No output schemas documented for any tool. LLMs cannot infer what fields are returned, breaking tool composition and forcing ad-hoc parsing of responses.
Parameter descriptions missing or incomplete. 'images' in create_chat/continue_chat lacks description of format (absolute paths vs data URLs) in the schema itself, only in the tool description. Same for 'image_output_dir', no clarity on whether it must exist or be created.
No input validation or constraints visible. 'model' parameter accepts any string with no enum or pattern. 'images' array has no minItems/maxItems. 'message' parameter has no min/max length. This invites invalid calls.
Inferred effective spec: 2025-06-18+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 53 | 2025-06-18+ | v2 |
'chat' tool description states 'New integrations should use create_chat or continue_chat' but tool is still exposed and callable. Deprecated tools should be marked with a deprecation notice or removed. This creates ambiguity for LLM tool selection.
Parameter 'include_raw_response' (boolean, optional) may leak internal LLM tokens or debug info. No documentation of what 'raw response' means or security implications. Consider sanitizing responses before returning to agent.
'model' and 'session_id' in 'chat' tool are documented as mutually exclusive (one required), but schema doesn't enforce this via 'required' array or 'oneOf'. LLM may pass both or neither, causing ambiguity.
No error handling guidance visible. Tools return 'WRITE' risk designation but no documentation of what errors are retryable, what LLM should do on failure, or recovery paths. Example: if model API rate-limits, should agent retry? Wait? Give up?