Loading ToolBench
Agent-gated community platform for people with AI psychosis. Provides chat interface with founder's AI agent, application processing, and email subscription management.
This HTTP-based Next.js server exposes 5 tools with significant definition gaps. Tool schemas are present but descriptions lack LLM-optimization guidance. The 'chat' tool has a verbose description (200+ chars) but omits critical details about streaming behavior and error modes. 'apply' and 'subscribe' have good input validation logic visible in code but descriptions don't explain rate limiting, email validation, or honeypot mechanics to the LLM. 'apply_status' and 'api_catalog' are READ_ONLY but lack descriptions entirely. Parameter descriptions exist but are generic ('Optional metadata including email and social links' doesn't explain when/why to use them). No output schemas documented. Error handling returns JSON status codes but lacks recovery guidance ('Try again in 3 months' is present for rate limit, but most errors are bare 400/429 responses). Security: email validation and disposable domain checks are implemented server-side (good), but no tool-level permission gates or audit logging visible. Composition: tools are single-purpose (good), but 'chat' conflates message streaming, PII extraction, and history storage in one description.
GET/HEAD endpoint serving RFC 9727 linkset+json API catalog. Advertises service descriptions and documentation for chat, apply, and subscribe endpoints.
POST endpoint to submit an application to join psychopats.ai. Validates application data (name, email, what_you_build, why_you_want_in, social_links), checks for disposable emails, enforces rate limiting (1 per email per 3 months), stores application in database, and sends email notification.
GET endpoint to check application status by email. Returns application status and timestamp without exposing personal details.
Stream text responses from the founder's AI agent using Google Gemini 2.5 Flash. Processes user messages, extracts PII, sanitizes content, applies system prompt with runtime state, and stores conversation history and unknown questions.
POST endpoint to subscribe email to newsletter. Validates email format, checks for duplicates, enforces unique constraint, and stores subscriber in database.
apply_status and api_catalog have NO descriptions. LLMs cannot determine when to call these tools or what they return.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields (e.g., sessionId from chat, application_id from apply).
Error responses lack recovery guidance. Rate limit returns 'Try again in 3 months' but most errors (400, 429) are bare status codes with generic messages. LLMs cannot self-correct.
Parameter descriptions are generic and lack constraints. 'Optional metadata including email and social links' doesn't explain format, max length, or when to include. 'Array of message objects' doesn't specify role enum values.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 41 | 2026-07-28+ | v2 |
chat tool description conflates three concerns: streaming, PII extraction, and history storage. Unclear what the LLM should expect or when to call it vs. other endpoints.