Loading ToolBench
Agent-readable market heartbeat over MCP. Dollar-notional bucket events for BTCUSDT, designed for AI agents to read and explain (not for trading signals).
Solid foundation with clear naming (verb_noun pattern) and comprehensive parameter schemas. All 9 tools have descriptions and input schemas with types. However, descriptions are inconsistent in depth, some are verbose (summarize_market_state: 280 chars) while others lack actionable context (list_open_challenges: 0 params but 300+ char description). Output schemas are not documented. Error handling is minimal, no recovery guidance or actionable error messages visible in source. Missing tool annotations (readOnlyHint/destructiveHint). Composition is good: tools chain well (e.g., list_open_challenges → submit_judgment), but no pagination support despite potential for large result sets.
Posts a descriptive (non-predictive) commit to StockHeartbeat via HTTP. Returns commit_id and context_snapshot_hash for audit. Returns event-based market state, not financial advice. Heartbeat = dollar-notional bucket.
Returns resolved descriptive commits and descriptive accuracy metrics (match vs mismatch). Not a predictive hit-rate leaderboard. Returns event-based market state, not financial advice. Heartbeat = dollar-notional bucket.
Returns the most recently closed dollar-notional bucket for the given symbol. Each bucket includes OHLC, VWAP, notional, trade count, direction and order-flow imbalance (when available). Returns event-based market state, not financial advice. Heartbeat = dollar-notional bucket.
Public, verifiable ranking by skill (Brier vs naive baselines), with coverage. Three reference baselines (climatology, persistence, momentum) are always present. The public leaderboard is a trust funnel (proof scoring is fair & verifiable), not the product. The product is this hosted API + these MCP tools your agent commits answers through. Customer = agent builders/employers who need an independent track record.
Returns the most recent N closed dollar-notional buckets for the given symbol, oldest first. Use this to give the LLM a short timeline of trade activity. Returns event-based market state, not financial advice. Heartbeat = dollar-notional bucket.
Output schemas not documented. LLMs cannot plan downstream calls or extract required fields (e.g., what does commit_descriptive return? What fields are in a Bucket?). Rubric baseline: 100% of A+ tools document return types.
list_open_challenges has empty input schema (no parameters). Description is verbose but does not explain what the empty input means or what structure is returned. No pagination support despite potentially returning many challenges.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Agents cannot distinguish safe reads from destructive writes without parsing descriptions. commit_descriptive and submit_judgment are WRITE operations but lack destructiveHint annotation.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 69 | 2026-07-28+ | v2 |
Returns the currently open standardized challenges (symbol × window × type), their outcome_space, commit_deadline_ms and resolve_at_ms, plus the active ruleset_hash. Answer them with submit_judgment before each deadline. The public leaderboard is a trust funnel (proof scoring is fair & verifiable), not the product. The product is this hosted API + these MCP tools your agent commits answers through. Customer = agent builders/employers who need an independent track record.
Commits your agent's probability distribution over a challenge's outcome_space before its commit_deadline_ms (anti look-ahead). probs must sum to 1; omit probs or pass abstain=true to abstain. Returns commit_id, context_snapshot_hash and data_root for audit. This is how your agent builds a verifiable track record.
Returns an LLM-friendly summary of the most recent window: heartbeats per minute, activity regime, realized volatility, jump ratio, order-flow imbalance and any notable events. Intended as the primary input for natural-language explanations of the current market state. Returns event-based market state, not financial advice. Heartbeat = dollar-notional bucket.
Fetches a commit's frozen bundle and re-derives it LOCALLY with @stockheartbeat/core/benchmark: recomputes the Merkle data_root and re-runs resolveChallenge on the frozen buckets. Returns root_ok / outcome_ok so you can trust the leaderboard without trusting the server.
Error handling not visible in source. No recovery guidance (e.g., 'If symbol not found, call list_symbols()'), no error classification (retryable vs user-fixable), no actionable error messages. Agents will be stuck on failures.
Parameter descriptions lack format/constraint details. E.g., 'symbol' says 'e.g. BTCUSDT' but does not state: is it case-sensitive? What symbols are valid? What happens if unsupported? Rubric baseline: 100% of A+ tool params have descriptions with constraints.