A template repository for building AI agents with MCP server integrations (filesystem, git, memory). Contains agent skills, workflows, and benchmarking infrastructure.
This server has 4 tools with significant definition gaps. Tool names lack clear action verbs (download_sharepoint_stream vs get_sharepoint_video). Descriptions are present but generic (194-char baseline; these average ~150 chars, acceptable length but lack WHEN/WHY context). Input schemas ARE visible with types and descriptions, which is positive. However, NO output schemas are documented, LLMs cannot plan downstream calls or extract results. Parameters lack enums where appropriate (e.g., 'out_dir' should validate path safety). Error handling is absent, no guidance on retryability, user-fixable vs fatal errors, or recovery paths. Security concern: cookies passed as file paths (good) but no validation that paths don't traverse outside intended directories. Overall: definitions exist but lack production rigor.
Deterministic context and manual-usage benchmark for the agent template - measures token usage, correctness, and clarity across scenarios
Validate the AI-agent project template structure - verifies required files, context budgets, gitignore patterns, and security scanning
Automated SharePoint Stream Video Downloader - downloads video from SharePoint Stream URLs using yt-dlp or Playwright MediaRecorder fallback at 1.0x native playback speed
Harvest 100% full-length transcript and SRT subtitles from SharePoint Stream videos using Playwright browser automation
No output schemas documented for any tool. LLMs cannot infer result structure, forcing them to guess at field names and types. This breaks tool chaining and wastes tokens on clarification.
Tool names lack clear action verbs. 'download_sharepoint_stream' is acceptable, but 'benchmark_context' and 'check_template' are vague. LLMs cannot infer intent from names alone. Should be 'measure_agent_context' and 'validate_template_structure'.
No error handling guidance. Tools that write files (download, harvest) or validate structure (check_template) provide no recovery hints. E.g., 'If download fails, check cookies file exists and is valid Netscape format.' Missing error classification (retryable vs fatal).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 42 | 2026-07-28+ | v2 |
Path parameters ('cookies', 'out_dir', 'root', 'manifest', 'usage') lack validation constraints. No mention of path traversal protection, absolute vs relative paths, or directory existence checks. LLMs could pass '../../../etc/passwd' without guidance.
Descriptions lack WHEN/WHY context. E.g., 'Harvest 100% full-length transcript...' tells WHAT but not WHEN to call it vs download_sharepoint_stream, or what prerequisites exist (must video be downloaded first?). Baseline for A+ tools is 50-200 chars with clear intent.