Loading ToolBench
FastAPI backend for Sentinel - a three-engine system for employee wellbeing monitoring (Safety Valve, Talent Scout, Culture Thermometer). Provides REST API for external tool integrations via Composio, MCP Tool Router sessions, and orchestration of multiple AI agents.
Sentinel Backend exposes 11 tools with mixed quality. Naming is generally verb-first and clear (get_*, execute_*, analyze_*, orchestrate). Descriptions are present for all tools and range 50-250 chars, meeting baseline minimums. However, parameter descriptions are inconsistent: some tools (e.g., execute_tool, orchestrate) have detailed param docs, while others (e.g., list_agents, orchestrator_health) have empty input objects with no descriptions. Input schemas are visible and typed (string, integer, object, array), but output schemas are not documented in the source. Error handling is minimal, no recovery guidance, no categorization of retryable vs fatal errors. Security concerns: entity_id parameter allows cross-user data access with only admin check mentioned but not enforced in visible code. No tool annotations (readOnlyHint, destructiveHint) despite clear risk classifications in metadata.
Check health status of an individual agent. Returns health status (healthy/degraded/unhealthy), uptime, and timestamp.
Analyze calendar meeting load and detect burnout signals. Returns total meeting hours, average hours per day, back-to-back meeting count, risk assessment (LOW/MODERATE/HIGH), and comparison to healthy baselines.
Execute an external tool action via Composio. Allows AI agent to fetch real-time data from integrated tools (Calendar, Slack, GitHub, Jira). Security: Requires authentication. Entity ID defaults to current user if not specified. Admin users can query other users' data.
Get detailed information about a specific agent including capabilities, status, and configuration.
Fetch calendar events for a user. Returns events for the specified number of days ahead.
Output schemas not documented. Tools return responses (e.g., ToolExecuteResponse, IntegrationStatusResponse) but LLMs cannot see field types or structure in tool definitions. Agents cannot plan downstream calls or extract required IDs.
Empty input schemas for discovery tools. list_agents and orchestrator_health have {} input with no parameter descriptions. Violates pattern:tool-description requirement that every parameter needs a description.
No error handling guidance. Tools lack recovery hints (e.g., 'If tool not found, call list_agents() first'). No categorization of retryable vs fatal errors. Agents cannot self-correct on failures.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | <=2025-11-25 | v2 |
Get status of external tool integrations. Returns which tools are configured and available actions.
Get Slack activity metrics for a user. Useful for detecting communication overload or isolation patterns.
List all available agents and their status.
Execute multiple agents in parallel and aggregate results. Supports task dependencies and multiple aggregation strategies (hierarchical, weighted_average, majority_vote, ensemble).
Overall orchestrator health check. Returns status of all agents and orchestrator health.
Set the status of an agent (online/offline/busy). Requires admin authentication.
Cross-user data access via entity_id parameter. execute_tool, analyze_calendar_load, get_calendar_events, get_slack_activity all accept entity_id to query other users' data. Code comment mentions 'Admin users can query other users' data' but no visible permission enforcement in provided source. Violates pattern:permission-gate.
No tool annotations. Tools marked with Risk: READ_ONLY or WRITE in metadata but no readOnlyHint or destructiveHint in schema. LLMs cannot infer safety properties from definitions alone.