Loading ToolBench
AI Secretary - Execution support system for technical professionals. Provides task triage, daily planning, task decomposition, and plugin integration (Slack) for JIRA-based workflow management.
TrIAge exhibits significant definition quality gaps. Tool descriptions are present but often vague or incomplete (e.g., 'Health check endpoint' lacks context on when to call it). Parameter schemas are partially defined but lack consistency: some tools have detailed parameter descriptions (e.g., generate_plan with date/closure_rate), while others expose raw request bodies (handle_slack_webhook with 'body' as string). No output schemas are documented anywhere. Error handling is absent, no guidance on retryability, user-fixable errors, or recovery paths. Security concerns: OAuth tokens and Slack signatures are passed as parameters (exchange_code_for_token, handle_slack_webhook), violating secret-injection patterns. Tool composition is weak, present_plan and present_decomposition appear to be UI/approval handlers, not stateless tools suitable for agent composition. Most tools lack idempotency guarantees or confirmation patterns for destructive operations (approve_plan modifies state without dry-run support).
Approve or reject a plan. POST /api/v1/plan/{date}/approve with approval status and optional feedback.
Decompose a long-running task into subtasks. POST /api/v1/task/{taskId}/decompose with optional target_days parameter.
Exchange Slack authorization code for access token. Calls Slack's oauth.v2.access endpoint to exchange authorization code for access token and bot information.
Generate daily plan. POST /api/v1/plan with optional date and closure_rate parameters.
Generate OAuth authorization URL for Slack workspace installation. Returns complete authorization URL to redirect user to.
Get existing plan for a date. GET /api/v1/plan/{date}
Secrets exposed as tool parameters: OAuth codes, tokens, and Slack signatures passed directly in handle_slack_webhook, exchange_code_for_token, store_tokens, and refresh_token. These will be logged in agent traces and prompt history.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields (e.g., what does generate_plan return? Does it include plan_id for approval?). Violates pattern:tool and mxe:response-field-naming.
Vague or missing descriptions on 6+ tools. 'Health check endpoint' (health_check), 'Health check endpoint for plugin handler' (handle_plugin_health), and 'Present daily plan to user' (present_plan) lack context on when/why to call them. Descriptions under 50 chars cannot guide LLM selection.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | <=2025-11-25 | v2 |
Health check endpoint for plugin handler. Returns status of plugin registry and loaded plugins.
Handle Slack OAuth 2.0 authorization callback. Exchanges authorization code for access token and stores installation.
Handle Slack webhook events including slash commands, interactive components, event API events, and URL verification challenges. Validates Slack signatures and routes to plugin registry.
Health check endpoint.
Present task decomposition proposal to user and collect approval. Displays proposed subtasks and collects user feedback with optional timeout.
Present daily plan to user and wait for approval with optional timeout. Collects approval status, feedback, and modifications.
Refresh expired access token using refresh token. Calls Slack's oauth.v2.access endpoint with refresh token to obtain new access token.
Store OAuth tokens in database with encryption. Creates a new plugin installation record with encrypted tokens.
Raw request body parameters (handle_slack_webhook accepts 'body' as string, headers as object) bypass schema validation. LLMs cannot infer structure. Should decompose into typed fields (event_type, user_id, command, etc.).
No error handling guidance. Tools lack recovery hints (e.g., 'If plan not found, call generate_plan first'). No categorization of retryable vs user-fixable errors. Agents cannot self-correct on failures.