Loading ToolBench
MCP server for Alma — a mental health support system with memory layers, crisis risk evaluation, and identity linking backed by PostgreSQL + pgvector
Server has 6 tools with complete input schemas and descriptions. Naming follows verb_noun convention (get_, search_, upsert_, build_, evaluate_, link_). Descriptions are present and substantive (100-200 chars typical). However, output schemas are not documented in the source code, LLMs cannot predict return structure. Parameter descriptions are adequate but lack constraint details (e.g., layer enum values, k bounds). Error handling is minimal, no recovery guidance or actionable error messages visible. The link_anonymous_to_account tool is particularly complex (DESTRUCTIVE, multi-table transaction) but lacks confirmation/dry-run pattern. Crisis evaluation tool uses hardcoded keywords without explaining scoring logic or thresholds to the LLM.
Build a human-readable context string from all memory layers, ready to inject into the LLM system prompt.
Heuristic crisis risk evaluation based on message content. Returns score (0-1), level (low/medium/high), and matched keywords. No LLM call — deterministic and fast.
Get all 4 memory layers (mood_history, mentioned_events, habits, interaction_prefs) for a user.
WS-D.2 — Link an anonymous identity to a Google identity, or wipe it. action='keep' → copy 4 layers + conversations + messages + proactive_log to google_sub via INSERT ... ON CONFLICT DO NOTHING, then delete the anon rows. Single transaction. action='reset' → delete anon rows only. No copy. Both actions write to alma_consent_log for GDPR audit. Both are idempotent.
Semantic search over all memory entries for a user. Returns top-k matches with similarity scores.
Output schemas not documented. LLMs cannot predict return structure for any tool. get_memory_tool returns 4 layers but structure is opaque; search_memories_tool returns 'top-k matches with similarity scores' but field names unknown; build_context_tool returns 'human-readable context string' but format undefined.
upsert_memory_tool accepts 'layer' as free-form string but valid values (mood_history, mentioned_events, habits, interaction_prefs) are only mentioned in description text, not as enum constraint. LLMs may hallucinate invalid layer names.
link_anonymous_to_account_tool is DESTRUCTIVE (deletes user data) but lacks confirmation/dry-run pattern. No recovery guidance if agent makes a mistake. Idempotency is mentioned in docstring but not in tool description visible to LLM.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 63 | 2026-07-28+ | v2 |
Add or update a memory entry in the specified layer (mood_history, mentioned_events, habits, interaction_prefs). Include 'entry_key' in data to enable upsert by key.
search_memories_tool parameter 'k' lacks bounds. Description says 'Number of top results to return' with default=5 but no min/max specified. LLM could request k=10000, causing performance issues.
evaluate_crisis_risk_tool returns 'score (0-1), level (low/medium/high), and matched keywords' but output schema undefined. LLM cannot reliably extract score value or parse matched keywords array.