Loading ToolBench
Slack-driven agent sandboxes. Mention a bot in Slack; it forks a cloud sandbox for that thread and runs a coding agent inside it.
Sark provides 5 well-defined Slack integration tools with complete JSON schemas and clear descriptions. Tool names follow verb_noun convention (slack_post_message, slack_update_message, slack_add_reaction, slack_upload_file, slack_get_thread). All tools have descriptions (avg 180 chars, within 10-1024 baseline). Input schemas are properly typed with required fields and parameter descriptions. However, output schemas are not documented, callers cannot predict response structure. Error handling is minimal (only basic validation in callTool). No tool annotations (readOnlyHint/destructiveHint) despite clear read/write semantics. Parameter descriptions lack constraint details (e.g., emoji name format, file size limits, thread limit bounds).
Add an emoji reaction to the message that triggered this run.
Read the recent messages in this thread, oldest first. Use it when you need context the prompt did not include.
Post a message into the Slack thread that asked for this work. This is the ONLY way to talk to the user - stdout is not shown to them. Returns the message ts, which you can pass to slack_update_message later.
Edit a message you posted earlier. Use this to turn a progress note into the final answer instead of posting many messages.
Upload a file into the thread — code, logs, diffs, reports, or binary files like images and PDFs. Prefer this over a huge message for anything longer than ~3000 characters. IMPORTANT: for binary files (png, jpg, pdf, zip, …) base64-encode the bytes and set encoding to "base64" — otherwise the file arrives corrupted. Text files can be sent as-is.
Output schemas not documented. Callers cannot predict response structure (e.g., what fields are in thread messages, what slack_post_message returns beyond 'ts'). LLMs need documented return types to plan downstream calls.
No tool annotations (readOnlyHint/destructiveHint/idempotentHint). slack_post_message, slack_update_message, slack_add_reaction, and slack_upload_file are destructive writes; slack_get_thread is read-only. Annotations help agents reason about safety and retry semantics.
Parameter descriptions lack constraint details. slack_add_reaction 'name' field has no format spec (emoji name without colons, but what length? what characters?). slack_upload_file 'encoding' enum is present but no guidance on when to use base64 vs utf8 beyond 'binary files'. slack_get_thread 'limit' has no bounds (default 50, but min/max?).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 69 | 2026-07-28+ | v2 |
Error handling is minimal. callTool returns generic success messages ('Posted. ts=...', 'Updated.', 'Reaction added.') with no guidance on failure modes. No recovery hints (e.g., 'If emoji not found, try a different name'). No categorization of retryable vs fatal errors.
slack_post_message and slack_update_message lack idempotency guarantees. Agents retry on ambiguous failures, if a message posts but the response is lost, a retry could create a duplicate. No idempotency key or deduplication strategy documented.