Loading ToolBench
JARVIS exhibits severe definition quality gaps across all dimensions. Tool descriptions are present but generic (avg ~80 chars, well below the 194-char baseline). Input schemas are visible but lack rigor: many parameters missing type constraints, enums underutilized, and descriptions are sparse or trivial. No output schemas documented. Duplicate tool names (rust_file_search appears twice). Critical security issue: playwright_tool accepts file_path and profile_path parameters without sanitization, inviting path traversal. Error handling is absent, no recovery guidance, no categorization. Composition is poor: 'high_perf_data_processor' and 'high_perf_file_analyzer' are vague; 'rust_data_processor' and 'rust_text_analyzer' lack clear use-case differentiation. No tool returns structured output with chaining IDs. Overall, this server reads as a prototype with minimal production hardening.
Get the current date and time, optionally for a specific timezone
Process large datasets with high-performance Rust backend
Analyze files and directories with high-performance Rust backend
Automate web browsers using Playwright for scraping, testing, and interaction. Supports Chromium, Firefox, and WebKit with optional persistent profile.
Extract structured data from files using high-performance Rust implementation
Process large datasets using high-performance Rust implementation
Search for patterns in files using high-performance Rust implementation
Duplicate tool name 'rust_file_search' registered twice with identical schemas. LLM cannot disambiguate; will cause routing errors and confusion.
playwright_browser_automation accepts 'file_path' and 'profile_path' parameters without path sanitization or validation. Invites path traversal attacks (e.g., '../../../etc/passwd'). No input validation documented.
Generic tool names 'high_perf_data_processor' and 'high_perf_file_analyzer' lack action verbs and specificity. LLM cannot infer intent from name alone. Should be 'filter_dataset', 'aggregate_data', 'search_files', etc.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields. Responses are unstructured; no pagination, no chaining IDs, no error categorization.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 41 | 2026-07-28+ | v2 |
Search for patterns in files using high-performance Rust implementation
Count lines in files using high-performance Rust implementation
Analyze text complexity and extract insights using high-performance Rust implementation
Set a timer for a specified duration
Parameter descriptions are trivial or missing. E.g., 'criteria' in high_perf_data_processor is 'Operation-specific criteria', tells LLM nothing about structure, required fields, or format. Violates pattern:tool-description baseline (100% of A+ tools have param descriptions).