Loading ToolBench
cross-venue prediction-market landscape scanner. For a given topic, shows how 5 venues price it side by side and highlights divergence: Polymarket, Kalshi, Manifold, PredictIt, Futuur.
Two tools with complete input schemas and detailed descriptions. Naming follows verb_noun pattern (compare_topic, venue_quotes). Descriptions are comprehensive (194 - 280 chars) and include use-case guidance and caveats. However, output schemas are NOT documented, LLMs cannot infer response structure. Parameter descriptions are present but lack format constraints (e.g., per_venue lacks min/max bounds; venue lacks enum constraint despite listing 5 valid options). Error handling is implicit (venue errors reported as {reachable: false}) but not explicitly documented in tool descriptions. No tool annotations (readOnlyHint, idempotentHint) despite both being READ_ONLY.
Cross-venue prediction-market LANDSCAPE scanner: for one topic, show how 5 venues (Polymarket, Kalshi, Manifold, PredictIt, Futuur) are pricing it side by side and flag divergence. Each venue is queried in parallel; a venue that errors or times out is reported as { reachable: false } and never fails the call. Returns each venue's top matches normalized to { venue, title, implied_probability (0-1 for the main/YES outcome, or null), url, n_outcomes }, plus a divergence summary across venues' top matches. HONESTY: this surfaces WHERE TO LOOK for mispricing — it is NOT an executable-arb tool. A high spread_pp usually reflects different bet shapes, resolution criteria, dates, or thin liquidity, not a real arbitrage. Divergence is only computed when ≥2 venues' top titles share strong token overlap (Jaccard); otherwise divergence is null with a low-match-quality note telling you to compare manually. Verify the questions resolve identically before trading. For executable poly↔kalshi arbitrage (needs order-book depth) use the `polymarket_arbitrage` pack.
Single-venue drill-down: fetch one venue's current top 5 matches for a query, each as { title, implied_probability (0-1 or null), url }. Useful after compare_topic to look closer at one venue. Implied probability is the main/YES outcome where available; multi-outcome markets report the top outcome and note it. Keyless.
Output schemas not documented. LLMs cannot infer response structure (NormMarket fields, divergence summary format, error shape). Responses must be documented so agents can plan downstream calls and extract data reliably.
Parameter 'venue' in venue_quotes lacks enum constraint. Description lists 5 valid values but does not enforce them. LLMs may hallucinate invalid venue names. Add enum: ['polymarket', 'kalshi', 'manifold', 'predictit', 'futuur'].
Parameter 'per_venue' lacks bounds. Description says 'default 3, max 5' but schema has no minimum/maximum constraints. LLMs may pass 0, negative, or >5 values. Add minValue: 1, maxValue: 5.
Tool annotations missing. Both tools are READ_ONLY and idempotent (no side effects), but inputSchema lacks readOnlyHint and idempotentHint annotations. Agents cannot infer safety properties without explicit hints.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 57 | 2026-07-28+ | v2 |
Error handling not documented in tool descriptions. Code shows venues report {reachable: false} on timeout/error, but this behavior is not mentioned in descriptions. Agents need explicit guidance on error shapes and recovery steps.