A tenant assistance agent for rental property management that helps tenants pay rent, submit maintenance requests, view payment/maintenance history, and answer questions about lease agreements and maintenance policies using an LLM-powered chatbot.
Five tools with basic schemas and descriptions, but significant gaps in parameter documentation and error handling. All tools have names starting with action verbs (pay_, get_, submit_, search_) and descriptions present. However, descriptions are minimal (10-50 chars), parameter descriptions lack detail on constraints/formats, and no output schemas are documented. No error handling guidance. Tool composition is reasonable (separate read/write operations), but parameter design could be more user-friendly (e.g., accepting usernames alongside user_id).
Get maintenance request history.
Get rent payment history.
Submit a rent payment.
Search lease and maintenance policy documents for relevant information.
Submit a maintenance request.
Descriptions are too brief (10-50 chars). Rubric baseline is 194 chars. LLMs cannot determine when/why to select tools without context on prerequisites, return values, and use cases.
No output schemas documented. LLMs cannot plan downstream calls or extract required fields (e.g., payment_id, maintenance_request_id) without knowing response structure.
Parameter descriptions lack constraint details. 'month' accepts free-form strings (e.g., 'January 2026') with no format guidance. 'amount' has no min/max bounds. 'query' has no length limits.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 50 | 2026-07-28+ | v2 |
No error handling guidance. Tools do not indicate what errors are retryable, user-fixable, or fatal. No recovery suggestions (e.g., 'If user not found, try search_users').
pay_rent and submit_maintenance_request are destructive (WRITE) but lack confirmation/dry-run patterns. Agents cannot preview consequences before executing irreversible operations.