Loading ToolBench

Reading the current corpus…

agentos MCP server · F · ToolBench

agentos

An agent orchestration platform with tool registry, capability management, and streaming tool execution for multi-connector workflows (HubSpot, Google Calendar, Instagram, QuickBooks, Stripe, Slack, Gmail, Google Drive, Twilio)

2025-06-18+15 tools

31/100

Definition Quality38

Protocol Readiness22

Supportability25

Spec posture

Inferred effective spec 2025-06-18+·current baseline 2026-07-28

Static source inference · medium confidence · detected: Logging, Dynamic Client Registration

Deprecated protocol patterns detected

Summary

AgentOS exhibits severe definition quality gaps across most tools. Only 2 of 15 tools (drive.read, drive.write) have complete input schemas with property descriptions and type definitions. The remaining 13 tools (gmail.read, gmail.send, hubspot.read, hubspot.write, web.search, llm, stripe.chargeCustomer, stripe.refundPayment, shell.execute, zapier.webhook.trigger, calendar.events.list, calendar.events.get, calendar.availability.get) have empty or missing input schemas, just `{"type":"object","properties":{}}`. Descriptions exist but are minimal (10-50 chars for most), lacking WHEN to use, prerequisites, or consequences. No output schemas documented for any tool. No error handling guidance. High-risk tools (stripe.chargeCustomer, shell.execute) lack permission gates, dry-run patterns, or confirmation steps. Tool naming is inconsistent: some use verb_noun (drive.read, drive.write), others use noun.verb (calendar.events.list, gmail.read). No tool annotations (readOnlyHint, destructiveHint, idempotentHint) present.

Tools (15)

calendar.availability.getread onlyauthsource verified58/100

Check free/busy status for a set of email addresses within a date range — used to find meeting slots.

calendar.events.getread onlyauthsource verified57/100

Get a single calendar event by ID from Google Calendar.

calendar.events.listread onlyauthsource verified60/100

List calendar events from Google Calendar — returns events within a date range (default: next 7 days).

drive.readread onlyauthsource verified75/100

Read files from Google Drive — lists files and reads file content (CSV parsing for Monday loop)

drive.writewriteauthsource verified75/100

Write files to Google Drive — uploads CSV or text files

gmail.readread onlyauthsource verified23/100

Read messages from Gmail

gmail.sendwrite

Issues (5)

CRITICALtool-description, constrained-input ↗

13 of 15 tools have empty input schemas (only `{"type":"object","properties":{}}`). No parameter types, descriptions, or constraints visible.

gmail.readgmail.sendhubspot.readhubspot.writeweb.searchllmstripe.chargeCustomerstripe.refundPaymentshell.executezapier.webhook.trigger

CRITICALresponse-shaper ↗

No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields. Violates pattern:tool and pattern:response-shaper.

gmail.readgmail.sendhubspot.readhubspot.writeweb.searchllmstripe.chargeCustomerstripe.refundPaymentshell.executezapier.webhook.triggercalendar.events.listcalendar.events.getcalendar.availability.get

Recommendations

Add complete input schemas to all 13 tools with empty schemas. Include parameter types (string, number, array, object), descriptions (50-100 chars), and constraints (enum, pattern, min/max). Example: gmail.read should accept query (string), maxResults (number, 1-100), and labels (array of strings).
Document output schemas for all tools. Specify returned fields, types, and pagination info. Example: calendar.events.list should return {events: [{id, summary, start, end, attendees}], nextToken}.
Expand tool descriptions to 100-200 chars. Include WHAT it does, WHEN to use it, and key prerequisites. Example: 'Send an email via Gmail. Use this after composing message content. Returns message ID and thread ID for follow-up replies.'
Add permission gates and audit logging to stripe.chargeCustomer, stripe.refundPayment, and shell.execute. Implement dry-run or confirmation patterns for irreversible operations. Document required scopes (e.g., 'payment:write', 'shell:execute').
Add error handling guidance to all tools. Specify error codes, retryability, and recovery steps. Example: 'User not found (404). Try search_users() with partial name. Retryable: no.'

Spec posture evidence

Inferred effective spec: 2025-06-18+.

Relies on Logging (deprecated) - log to stderr or use OpenTelemetry

Score history

Overall score trend

First recorded score · v2 rubric

31/100

Scored	Grade	Overall	Spec posture	Rubric
2026-09-23	F	31	2025-06-18+	v2