Loading ToolBench
Ambient discovery and research engine — scans arXiv, HN, GitHub, Anthropic docs for new capabilities
Interject has explicit tool definitions with schemas and descriptions visible in server.py. However, quality is uneven. Most tool descriptions are concise but lack context on WHEN to use them and prerequisite information. Parameter descriptions are present but often generic or incomplete. Output schemas are not documented, the LLM cannot predict what fields to expect from results, forcing it to discover structure by trial. Several tools have ambiguous names (e.g., 'interject_profile' conflates view, add, remove, remove actions into one tool). No error handling guidance is visible, if a scan fails or a discovery is not found, the LLM gets no hint on recovery. Security considerations are absent (no audit logging documented, no permission gates visible). The tool set is well-scoped but composition is loose, tools operate on overlapping concerns (discoveries, promotions, dismissals, profiles) without clear data flow documentation.
Get full details on a specific discovery.
Dismiss a discovery as irrelevant. Negative signal for the recommendation model.
Get discoveries above threshold since last review. Use for session-start checks.
View or edit the interest profile (topics, weights, learned preferences).
Promote a discovery to a bead with briefing and optional plan. Updates the recommendation model.
Record a query for cross-session pattern detection and topic boosting.
Trigger a full or per-source scan for new discoveries. Scans arXiv, HN, GitHub, Anthropic docs.
No output schemas documented for any tool. LLMs cannot predict response structure and must discover fields by trial, wasting tokens and risking context loss.
interject_profile combines four distinct operations (view, add_topic, remove_topic, reset) into one tool. Should be split into separate tools (view_profile, add_profile_topic, remove_profile_topic, reset_profile) per pattern:tool single-responsibility principle.
Parameter 'source' appears in multiple tools (scan, inbox, search) as an optional unvalidated string. Schema does not constrain it to the documented enum values (arxiv, hackernews, github, anthropic). LLMs will hallucinate invalid sources.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 54 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Semantic search across all stored discoveries.
Get discoveries relevant to the current session topic via embedding similarity.
Health check — last scan times, queue depth, profile stats, adapter health.
Descriptions lack recovery guidance. No tool description tells the LLM what to do if a discovery is not found, a scan fails, or a promotion fails. Per pattern:recovery-guide, every error-prone tool should hint at remediation.
interject_profile parameter 'topic' is optional even for add_topic and remove_topic actions, but is logically required in those cases. Schema does not enforce this dependency. LLMs may call add_topic without a topic value.
Tools that modify state (scan, promote, dismiss, record_query) have no explicit confirmation or dry-run mechanism. Per pattern:confirmation-request, agents should confirm before irreversible actions.
Tool descriptions do not clarify WHEN to use related tools. E.g., inbox vs. search, when should the LLM pick each? scan vs. inbox, what's the difference in result timeliness? Dependencies and distinctions are undocumented.
No permission gates or audit logging documented. Tools like promote, dismiss, and record_query modify state, but no indication of access control, who called it, or when. Per pattern:audit-trail, sensitive operations should be traceable.
Tools assume LLM knows domain concepts like 'discovery', 'bead', 'session topic', and 'embedding similarity' without explanation. Descriptions should be self-contained for agents unfamiliar with the discovery domain.
interject_session_context does not document how session state is established. Parameter 'topic' is optional, but if the server tracks session state, how does the LLM or server know the session ID? No mechanism is visible.