A FastAPI-based backend system for managing tools, including local Python tools, OpenAPI tool servers, and MCP (Model Context Protocol) tool servers with access control and authentication
Critical failures across all dimensions. Tool definitions exist in backend/ruvie/tools/builtin.py but source code is not provided for inspection. All 47 tools have only 1-line descriptions (10-30 chars), far below the 50-200 char baseline for LLM-optimized descriptions. No input schemas, parameter types, or output schemas are visible in the provided code. Tool names are inconsistent: some use verb_noun (add_memory, create_automation), others use noun_verb (grep_knowledge_files, kb_exec). No evidence of parameter descriptions, enums, or validation. Error handling patterns not visible. This server cannot be reliably used by agents without complete schema documentation.
Add a memory entry
Calculate a timestamp
Create an automation workflow
Create a calendar event
Create tasks
Delete an automation workflow
Delete a calendar event
Delete a memory entry
No input schemas visible for any of 47 tools. Cannot verify parameter types, required fields, enums, or constraints.
All 47 tool descriptions are 10-30 characters (e.g., 'Add a memory entry', 'Create a calendar event'). Rubric baseline: 50-200 chars for LLM-optimized descriptions.
No parameter descriptions visible. Rubric requires: 'Add a non-empty description to every input parameter.' Without parameter docs, LLMs cannot infer what 'status', 'type', 'query', or other params mean.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 29 | 2026-07-28+ | v2 |
Edit an image
Execute code
Fetch content from a URL
Generate an image
Get the current timestamp
Search knowledge files using grep
Execute knowledge base operations
List automations
List knowledge entries
List knowledge bases
List memories
List memory paths
Query knowledge bases
Query knowledge files
Read a memory path
Replace memory content
Replace note content
Search calendar events
Search channel messages
Search channels
Search chats
Search knowledge bases
Search knowledge files
Search memories
Search notes
Search the web
Toggle an automation on or off
Update an automation
Update a calendar event
Update a memory entry
Update a task
View a channel message
View a channel thread
View a chat
View a file
View a knowledge file
View a note
View a skill
Write a note
No output schemas documented. Rubric requires: 'Document the output schema. LLMs need to know what fields to expect so they can plan downstream tool calls.' Agents cannot chain tools without knowing return types.
Ambiguous tool names: 'kb_exec' (what does it execute?), 'grep_knowledge_files' (why grep instead of search?). Rubric: 'The tool name alone should convey what happens when called.' These names require reading descriptions to understand intent.