Loading ToolBench
A multi-agent AI platform that integrates with various LLM providers, manages agents, flows, and supports MCP (Model Context Protocol) servers. Includes admin and voice UIs for managing agents, conversations, and AI workflows.
Magec exposes 7 tools across two functional groups (artifacts and flow state). Tool definitions are present with descriptions and basic input schemas, but lack depth in parameter documentation and output schema specification. Naming conventions are generally clear (verb_noun pattern), but descriptions are superficial and do not fully explain when/why to use each tool or what errors might occur. Parameters have types but many lack detailed constraints or format guidance. No evidence of output schemas, pagination support, or error recovery guidance. The artifact system is the core offering, but lacks critical details around expiration, size limits, and error cases.
Export an artifact's content to a file in the system temporary directory and return its absolute path on disk. Use this when you need to hand an artifact off to another tool that reads from the filesystem (e.g. to run a parser, converter or text extractor). The destination file is created with a unique name derived from the artifact name; the directory is chosen for you and cannot be overridden. Returns the absolute path, byte size and MIME type so the caller can pass them to the next tool.
Issue a short-lived, signed download URL for an artifact. Use this when you need to hand an artifact off to another tool that fetches files over HTTP (e.g. when the consumer runs in a different process or container and cannot reach the local filesystem). The URL serves the artifact's raw bytes with the correct MIME type, requires no authentication header, and stops working after a server-defined expiration. Returns the URL, the absolute expiration timestamp, and the MIME type.
Read a value previously stored in the shared flow state by another agent (or by an earlier turn of this agent). Returns {found: true, value: ...} when the key exists and {found: false} when it does not. Use this to check signals, decisions or intermediate results produced by upstream agents.
List all artifacts saved in the current session. Returns the filenames of all available artifacts.
Output schemas are not formally documented for any tool. Descriptions mention return fields (e.g., 'absolute path, byte size and MIME type' for export_artifact; 'URL, expiration, MIME type' for get_artifact_url), but no JSON Schema is provided. LLMs cannot reliably parse or chain tools without explicit output schemas.
Input parameter constraints are documented in descriptions but not enforced in JSON Schema. For example, set_state's 'key' parameter is described as 'letters, digits, underscores' but the schema lacks a 'pattern' field. Descriptions alone are insufficient, LLMs cannot reliably validate constraints and may pass invalid values.
Critical operational details are missing: artifact size limits, temp directory persistence (export_artifact), URL expiration TTL (get_artifact_url), and filename collision handling (save_artifact). Without these, LLMs cannot reason about tool applicability or handle edge cases.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 65 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 35 | - | v1 |
Load an artifact from the session and deliver it to the LLM in its native multimodal format (text or inlineData), bypassing base64 noise and context burn.
Save a file artifact (code, documents, data, images, etc.) that will be delivered to the user as a downloadable file. Use this instead of pasting long code blocks or file contents directly in chat. For text content, provide it directly in the 'content' field. For binary content (images, PDFs, etc.), set 'is_base64' to true and provide base64-encoded data in 'content'. The artifact persists across the current session.
Record a value in the shared flow state so other agents in the same workflow can read it. Keys must be simple identifiers (letters, digits, underscores). Values can be strings, numbers, booleans, lists or objects. State persists for the duration of the conversation and is visible to every other agent in the same flow.
No error handling guidance. Tools do not document what happens on failure (e.g., artifact not found, disk full, URL expired, network error). Descriptions lack recovery hints ('Try list_artifacts() first if unsure of the name'). Without actionable error messages, LLMs cannot self-correct.
load_artifact and get_artifact_url lack clear input parameter documentation. Neither tool's source code is visible, so it cannot be verified whether they accept 'name' (matching save_artifact/export_artifact) or use a different identifier. Parameter naming inconsistency forces LLMs to guess.
Artifact lifecycle is unclear. Do artifacts persist only during the session? When do temp files from export_artifact get cleaned up? What happens to signed URLs after expiration? Without explicit lifecycle documentation, LLMs cannot reason about tool sequencing or data safety.
No pagination or result limiting for list_artifacts. If many artifacts exist, the response could overwhelm the context window. Best practice: return a count, support offset/limit parameters, and document max results.