Loading ToolBench
An edge-profile event ingestion and semantic search system with MCP (Model Context Protocol) integration for querying sensor data, current state, time-range windows, and vector-based semantic search.
Percept demonstrates solid tool design with clear verb-based naming (describe_, get_, search_) and comprehensive parameter schemas. All 4 tools have documented input schemas with type definitions and parameter descriptions. Tool descriptions are substantive (100-200 chars), explaining what each tool does and when to use it. However, output schemas are not explicitly documented in the source, and error handling guidance is minimal. The semantic search tool (search_events) lacks required parameter validation hints. Overall, this is a B-grade server with good fundamentals but missing production-grade error recovery patterns.
What sensors are available, what they mean, and which are currently misbehaving. Returns merged Source+Kind descriptors plus a recent_errors digest per source.
Latest event per (source, kind) from the hot ring. Each entry carries age_ms and a stale flag derived from the descriptor's freshness_ttl_ms. Falls back to the cold store on a hot-ring miss; those rows are tagged from_cold = true.
Time-range scan from the cold store. Paginated via an opaque cursor. Ordered by (ts_ms_utc, event_id) ascending. Per-call hard limit 10,000 events.
Semantic search via the vector index, with optional time / source / kind filters. Returns the top-k hits with similarity scores and the matching events. Requires the embedder to be configured (see [storage].embed_default and per-kind / per-source `embed`).
Output schemas not documented. Tool descriptions explain what is returned (e.g., 'merged Source+Kind descriptors', 'top-k hits with similarity scores'), but formal output schema definitions are not visible in source code. LLMs cannot reliably extract fields from undocumented responses.
Error handling lacks recovery guidance. No evidence of error categorization (retryable vs. user-fixable vs. fatal) or actionable error messages. E.g., if search_events fails because embedder is not configured, the error should suggest checking [storage].embed_default configuration.
search_events 'limit' parameter lacks bounds. Description says 'Maximum number of results to return' but does not specify min/max (e.g., 1 - 1000). Unbounded numeric parameters invite LLMs to pass absurd values.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 65 | 2026-07-28+ | v2 |
get_window 'limit' parameter capped at 10,000 but description does not state the hard cap. LLMs may request 50,000 events expecting pagination; the tool silently truncates. Explicit cap in description prevents confusion.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All 4 tools are read-only and idempotent, but this is not declared in the tool definitions. Agents cannot infer safety properties without explicit hints.