Loading ToolBench
Orxhestra provides 25 tools with mixed quality. Core artifact/memory/task tools have reasonable descriptions (50-150 chars) and basic schemas, but lack critical details: no output schemas documented, minimal parameter constraints, no error handling guidance. Example tools (search_web, get_weather) are stubs with trivial implementations. Tool names follow verb_noun convention well (save_artifact, load_skill, task_create). However, 8 tools lack descriptions entirely or are under 20 chars. Destructive tools (delete_memory, drop_table, delete_records) have no confirmation/dry-run patterns. No tool annotations (readOnlyHint, destructiveHint). Schemas present but incomplete: most lack type constraints, enums, or ranges. Composition is reasonable (artifact, memory, task tools are separate concerns), but chaining IDs are not always returned. This is a C-grade server, functional but with significant gaps in LLM-optimization and error guidance.
Check billing status for an account.
Delete a saved memory by name.
Delete records from a database table matching a condition.
Drop an entire database table.
Get the current weather for a city.
List all available artifact filenames.
List all saved persistent memories.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields. E.g., list_artifacts returns a list but structure is undefined; task_create returns a task_id but no schema shows what fields the task object contains.
Destructive tools (delete_memory, delete_records, drop_table) lack confirmation/dry-run patterns and error guidance. No indication of what happens on failure or how to recover. Agents may accidentally destroy data.
Example tools (search_web, get_weather, lookup_order, check_billing) are stubs with trivial mock implementations. Schemas are minimal (single string param, no constraints). These appear to be placeholder examples, not production tools.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 53 | 2026-07-28+ | v2 |
List all available skills with their names and descriptions.
Load an artifact from the store by filename.
Read the full content of a saved memory by name. Use list_memories first to see available names.
Load a skill's instructions by name. Use this when a skill is relevant to the current task.
Load a specific resource file from a skill by its relative path. Use after loading a skill to access its scripts, references, or assets.
Look up an order by ID.
Save a file or blob to the artifact store.
Save a persistent memory that survives across sessions. Use for user preferences, feedback, project context, or references to external systems. Types: user, feedback, project, reference.
Search the database for records matching a query.
Search the web for information.
Pause execution for a specified number of seconds. Use when you need to wait before retrying an operation or polling for a result. Max 300 seconds.
Delegate a subtask to a fresh agent with isolated context. The sub-agent has the same tools but a clean conversation. Blocks until the sub-agent finishes and returns the result.
Create and start a background task. Returns a task ID you can use to check status and read output later.
Get details of a specific task.
List all background tasks with their status.
Read the output of a completed or running task.
Stop a running background task.
Update the status of a task.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) present. LLMs cannot infer which tools are safe to retry, which modify state, or which are read-only without explicit hints.
Parameter descriptions lack constraints: no enums for status fields (task_update accepts 'status' but valid values are not enumerated), no ranges for numeric params (sleep accepts 'seconds' but max 300 is only in description, not schema), no format specs for identifiers.