Loading ToolBench
Feedpilot demonstrates solid naming conventions (verb_noun pattern: list_sources, headlines, digest, search, mark_seen, add_source, remove_source, export_digest) and comprehensive parameter schemas with types and descriptions. All 10 tools have descriptions and input schemas. However, output schemas are undocumented, responses are returned as plain dicts without formal schema declarations. Parameter descriptions are adequate but lack constraint details (e.g., no min/max for limit, no regex for URLs, no enum for format in export_digest). Error handling is minimal, tools return error strings in responses but provide no recovery guidance. Security concerns: add_source accepts raw URLs without validation; no permission gates on destructive operations (remove_source, mark_seen). Composition is good, tools are single-responsibility and chainable (e.g., list_sources → headlines). Baseline: avg param count ~4, descriptions ~100-150 chars (within 10-1024 range), but missing output schema documentation and constraint specificity.
Add a custom feed source to your personal source list. Args: name: Display name for the source (must be unique). url: RSS/Atom feed URL (must start with http:// or https://). tags: Optional list of tags for filtering (e.g. ['tech', 'linux']).
Fetch headlines from all default sources and return a combined digest. Args: limit_per_source: Default number of items to fetch per source. tags: Optional list of tags to filter sources (e.g. ['linux', 'hardware']). If empty, all sources are included. exclude: Optional list of source names to skip (e.g. ['LKML']). overrides: Optional per-source item limits (e.g. {'LKML': 3, 'Phoronix': 10}). Unspecified sources use limit_per_source. force_refresh: Bypass the cache and fetch live data. fetch_summaries: Follow each article URL and include a short body preview. unseen_only: Return only articles not yet marked as seen.
Export today's digest to a Markdown or Org-mode file.
Fetch recent headlines from a specific source. Args: source: Name of a default source (e.g. 'Phoronix') or a raw feed URL. limit: Maximum number of items to return. force_refresh: Bypass the cache and fetch live data. fetch_summaries: Follow each article URL and include a short body preview. unseen_only: Return only articles not yet marked as seen.
Output schemas undocumented. Tools return plain dicts (e.g., {items, last_updated, cache_age_seconds, errors}) but no formal schema is declared. LLMs cannot reliably extract fields or plan downstream calls without documented return types.
Parameter constraints missing. 'limit' has no min/max bounds (unbounded integers risk API abuse). 'url' in add_source has no regex validation. 'format' in export_digest should be an enum, not free-form string. Descriptions lack constraint details.
Error handling lacks recovery guidance. Tools return error strings (e.g., 'error': result.error) but do not categorize errors as retryable/user-fixable/fatal or suggest next steps. LLMs cannot self-correct without actionable error messages.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
Return all user-added custom feed sources.
Return all configured default feed sources with their tags.
Mark article URLs as read/seen so they can be excluded from future digests. Args: urls: List of article URLs to mark as seen.
Fetch recent headlines from several sources in parallel. Args: sources: List of source names (e.g. ['Phoronix', 'LKML']) or raw feed URLs. limit: Maximum number of items to return per source. force_refresh: Bypass the cache and fetch live data. fetch_summaries: Follow each article URL and include a short body preview. unseen_only: Return only articles not yet marked as seen.
Remove a custom feed source from your personal source list. Args: name: Name of the custom source to remove.
Search headlines across all sources by keyword. Fetches all sources (optionally pre-filtered by tag), then returns only entries whose title or summary contains the query string (case-insensitive). Args: query: Keyword or phrase to search for. tags: Optional list of source tags to restrict which sources are searched. limit_per_source: Maximum items to fetch per source before filtering. force_refresh: Bypass the cache and fetch live data.
No permission gates on destructive operations. remove_source and mark_seen have no authorization checks. An untrusted agent could delete all custom sources or mark all articles as seen without validation.
URL validation absent in add_source. Tool accepts 'url' parameter with description 'must start with http:// or https://' but no regex pattern or validation logic visible. LLMs may pass invalid URLs; tool should validate and return clear error.