Loading ToolBench
Serverless podcast transcription and semantic search on Modal with MCP integration
Two tools with strong, LLM-optimized descriptions (194 - 250 chars) that clearly state WHAT, WHEN, and WHY. Both have complete input schemas with typed parameters and descriptions. search_podcasts includes helpful filtering (date range, show name) and explains result ordering. latest_on_topic is concise and purpose-driven. No output schemas documented in source. No error handling guidance visible. No tool annotations (readOnlyHint, etc.). Naming is clear and verb-driven (search_, latest_). Parameters are well-described with sensible defaults. Missing: output structure documentation, error recovery paths, and per-tool risk classification in schema.
Find the most recent things said about a topic across all podcasts. Use this when the user wants the current or latest view on something, or when asking whether a position has changed recently.
Search podcast transcripts semantically. Returns relevant chunks with speaker attribution, episode metadata, and publication date. Use this whenever the user asks about topics, opinions, or statements from the podcasts. Results are sorted most-recent first so recency is always visible — factor in publication date when reasoning about whether views may have changed.
Output schemas not documented. LLMs cannot infer what fields to expect (e.g., speaker, episode_id, publication_date, transcript_chunk). This forces agents to guess at response structure and risks failed downstream reasoning.
No error handling guidance. If a search returns zero results, or if a date range is invalid, the tool provides no recovery hint. LLMs cannot self-correct without explicit error messages like 'No results found. Try broadening the date range or removing the show filter.'
No tool annotations. Both tools are read-only (Risk: READ_ONLY noted in metadata), but this is not declared in the schema via readOnlyHint. Agents cannot determine which tools are safe to call speculatively vs. which require confirmation.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | B | 79 | 2026-07-28+ | v2 |
Parameter n_results lacks explicit bounds. Unbounded integers invite LLMs to pass absurd values (e.g., 10000) that could timeout or exhaust memory. Should specify min=1, max=100 (or appropriate cap).
after_date and before_date parameters accept free-form strings with only a format hint (YYYY-MM-DD). No regex pattern or validation rule visible in schema. LLMs may pass invalid dates like '2024-13-45' without catching the error server-side.