Loading ToolBench
Multi-modal AI Memory Infrastructure - a FastAPI-based backend for storing, processing, and retrieving text, image, PDF, and audio memories with vector embeddings, OCR, and advanced search capabilities
SARVAI exposes 22 tools with reasonable naming (verb_noun pattern) and descriptions, but critical gaps undermine production readiness. All tools have descriptions (10 - 200 chars, within baseline), but parameter schemas are incomplete: many lack type constraints, enums, or validation guidance. File upload tools (remember_image, remember_pdf, remember_audio) accept 'file' type without format/size constraints. Search and retrieval tools lack pagination (limit present but no offset/cursor or total_count). Output schemas are not documented, LLMs cannot infer response structure. Error handling is absent from tool definitions; no recovery guidance. Destructive tool (delete_memory) has no confirmation pattern. Namespace-based multi-tenancy is pervasive but not validated in descriptions. Average tool description ~120 chars (within baseline p10 - p90 range), but parameter descriptions are sparse or generic.
Add a message to a conversation
Advanced hybrid search combining semantic, BM25, and re-ranking
Ask a question about stored memories with RAG (Retrieval-Augmented Generation)
Create a new conversation
Delete a memory by ID
Get all messages in a conversation
Get all conversations for a user
File upload tools (remember_image, remember_pdf, remember_audio) lack format and size constraints. Parameter 'file' has no maxSize, mimeType, or format specification. LLMs cannot validate uploads before sending.
No output schemas documented for any tool. LLMs cannot infer response structure (fields, types, pagination). Downstream tool chaining is blind, agents cannot extract IDs or references needed for follow-up calls.
Search and retrieval tools (search_memories, advanced_search, get_conversations) accept 'limit' but lack offset/cursor and total_count. No pagination guidance. Large result sets will blow context windows.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 51 | <=2025-11-25 | v2 |
Get a specific memory by ID
Get timeline of memories grouped by date
Get popular search queries for a user
Get user by namespace
Get user preferences
Get user statistics and analytics
Login with namespace only
Register a new user with namespace only
Store audio memory. This endpoint: 1. Validates the user and audio file 2. Transcribes audio using Whisper 3. Chunks and generates embeddings 4. Uploads audio to MinIO 5. Stores everything in the database
Store image memory. This endpoint: 1. Validates the user and image file 2. Extracts text via OCR 3. Generates CLIP embeddings 4. Uploads image to MinIO 5. Stores everything in the database
Store PDF memory. This endpoint: 1. Validates the user and PDF file 2. Extracts text from all pages (with OCR fallback) 3. Chunks and generates embeddings 4. Uploads PDF to MinIO 5. Stores everything in the database
Store text memory. This endpoint: 1. Validates the user exists 2. Processes and chunks the text 3. Generates embeddings for each chunk 4. Stores everything in the database
Search memories using semantic search with vector embeddings
Update user preferences
Search the web and store results
Destructive tool delete_memory has no confirmation or dry-run pattern. No error recovery guidance. Agents can permanently delete memories without safeguards.
Parameter descriptions are generic or missing validation rules. E.g., 'namespace' appears in all tools but is never described as required, format, or length constraints. 'role' in add_message_to_conversation lacks enum (user|assistant).