Loading ToolBench
A repository of curated tutorial indexes and discoverability assets for code documentation, featuring validation scripts, market signal tracking, and tutorial manifest generation
This MCP server exposes 15 tools for documentation generation and GitHub API interactions. While tool names follow verb_noun conventions appropriately (check_index_format_v2, collect_broken_links, refresh_market_signals), the implementation reveals critical gaps in parameter descriptions, input schema completeness, and output documentation. Most tools lack descriptions for their input parameters, only 'description' fields are present at the tool level, not for individual parameters. Several tools accept string/Path parameters with minimal guidance on expected format or constraints. Error handling is largely absent, tools appear to raise exceptions without recovery guidance. Output schemas are not documented anywhere in the visible code.
Build a machine-readable tutorial inventory manifest containing metadata about tutorial structure, chapter counts, and file organization.
Validate required section structure for tutorial indexes that opt into format v2. Checks for presence of mandatory sections: Why, Current Snapshot, Mental Model, Chapter Map, Learning Outcomes, and Source References.
Classify tutorial directory structures as root_only, docs_only, index_only, or mixed based on numbered markdown file patterns and verify README.md presence.
Collect all broken local markdown links in the repository by scanning markdown files, extracting links, normalizing paths, and checking file existence.
Audit and collect stale release/activity date claims in tutorial files by parsing ISO and long-format dates and checking age against threshold.
Identify tutorial index files containing the AI-generated placeholder summary text.
Input parameter descriptions are missing for nearly all tools. Tools accept 'root', 'output_dir', 'token', 'files', etc. with only high-level context at tool level, not per-parameter guidance. For example, 'token' in fetch_repo is described as 'GitHub API token (optional, for higher rate limits)' but no guidance on token format, minimum length, or failure modes if invalid.
Output schemas are not documented. Tools like 'collect_broken_links', 'classify_tutorial_structure', and 'generate_discoverability_assets' return complex structures (lists of BrokenLink objects, dicts with structure_counts, JSON files) but the MCP schema for return values is absent. LLMs cannot plan downstream tool calls or extract data correctly without knowing the response structure.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Identify core abstractions (classes, functions, interfaces) from source code for use in tutorial chapter examples.
Fetch repository metadata from GitHub API including stars, forks, issues, and push activity.
Fetch GitHub repository data including metadata and latest release information for snapshot generation.
Fetch key source code files from a GitHub repository for tutorial chapter regeneration, respecting file size and extension filters.
Generate machine-readable discoverability assets for SEO and LLM retrieval including tutorial-index.json, tutorial-directory.md, llms.txt, and llms-full.txt with query hub classification, keyword clustering, and market signal integration.
Refresh live GitHub market signals for tracked coding-agent and vibe-coding repositories, update trending README section, and generate market signals JSON snapshot.
Refresh Current Snapshot sections in tutorial README.md files with live GitHub repository metadata and release information.
Regenerate tutorial chapter content with real source code examples, mermaid diagrams, and source-grounded explanations from GitHub repositories.
Audit dated release/activity claims in tutorial index files to identify and report stale or outdated date references.
Tool definitions are inferred from argparse patterns rather than explicitly registered via MCP schema. The source code shows Python scripts with argparse.ArgumentParser, but no explicit MCP ServerSchema or tool registration is visible.
No error handling or recovery guidance. Tools like 'fetch_repo' and 'refresh_tutorial_snapshots' call external GitHub APIs but provide no error messages guiding LLMs what to do on rate-limit, auth failure, or network timeout. Per pattern:recovery-guide, error responses must tell the LLM what to do next.
Multiple tools that write to disk or GitHub (generate_discoverability_assets, refresh_market_signals, refresh_tutorial_snapshots, regenerate_tutorial_chapters) lack confirmation or dry-run patterns. Per pattern:confirmation-request, irreversible operations should support dry-run or confirmation. Only refresh_tutorial_snapshots accepts a dry_run parameter; others do not.
GitHub API token is exposed as a tool parameter. Per pattern:secret-injection, credentials must never appear as tool parameters, they should be injected server-side via environment variables or vault. Agents log all parameters; secrets in params leak into traces and prompt history.
Parameter constraints are missing or implicit. 'max_age_days' in collect_claims and release_claims_audit lacks min/max bounds. 'batch' in regenerate_tutorial_chapters lacks guidance on valid range. 'workers' in refresh_tutorial_snapshots lacks constraints.
Pagination is absent from tools that return lists. 'collect_broken_links', 'classify_tutorial_structure', and others return all results without limit or cursor support. Per pattern:paginated-result, tools returning lists should accept page/offset and limit parameters to avoid context window explosion.