AI-powered Smart Home Hub with MCP integration for device control, automations, and chat-based home management
Artemis has 16 tools with basic schemas and descriptions, but significant gaps prevent higher scores. All tools have descriptions (10-50 chars, mostly short), and input schemas are present with type definitions. However, descriptions are often too brief to guide LLM selection (e.g., 'Return the authenticated user's profile' lacks context on when to call it vs. update_me). Parameter descriptions are minimal or missing context. Output schemas are not documented. Error handling is absent, no guidance on retryability, user-fixable errors, or recovery paths. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels. Composition is reasonable (single responsibility per tool), but chaining support is weak, responses likely omit IDs needed for downstream calls. Security concerns: no visible secret injection pattern, no audit logging in tool definitions, no permission gates documented.
Create a new automation
Create a new chat message
Delete an automation
Return the authenticated user's profile
Get the next pending command for ESP32 hardware bridge
List all automations for the current user, optionally filtered by type
List execution logs (audit trail) for the current user
Descriptions are too brief (10-50 chars) to guide LLM tool selection. Baseline is 194 chars. Missing WHEN to use, prerequisites, and dependencies. E.g., 'Return the authenticated user's profile' doesn't explain when to call get_me vs. update_me.
No output schemas documented. LLMs cannot plan downstream calls or extract required fields. E.g., does register return a JWT token? Does list_automations return automation_id for use in update_automation?
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels. delete_automation is DESTRUCTIVE but unmarked. Agents cannot distinguish safe retries from irreversible operations.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 48 | 2026-07-28+ | v2 |
List chat messages for the current user
Authenticate and return a JWT
Parse natural language into structured AAL JSON using the user's actual devices/functions
Post the result of a command execution from ESP32 bridge
Register a new user and return a JWT
Store the user's Expo push token for proactive notifications
Enable or disable an automation
Update an existing automation
Update profile information
No error handling guidance. Tools lack recovery hints, retryability classification, or actionable error messages. E.g., if parse_aal_text fails, should the agent retry, ask the user, or try a different approach?
Parameter descriptions lack format/constraint details. E.g., automation_type accepts 'aal or python' but no enum constraint. trigger/condition/action parameters lack examples or format guidance. LLMs will hallucinate invalid values.