Loading ToolBench
A collection of MCP (Model Context Protocol) servers for various purposes including financial data aggregation (GuruFocus), test reporting (SEM-4AI Reporter), and text pseudonymization
This multi-server repository exhibits significant inconsistencies across three distinct MCP server implementations. Average tool score is 42/100. Critical issues: 11 tools (44%) have no visible input schemas in the provided source code, forcing schema scores to 0. Descriptions vary wildly, some are detailed (e.g., pseudonymize_text at 185 chars), others are generic or missing context. Parameter descriptions are inconsistently applied. The three sub-servers (gurufocus, textpseudomizer, sem-4aireporter) were developed independently, leading to fragmented quality. Naming conventions are generally sound (verb-noun pattern, action-oriented), but schema depth and error handling are weak across the board. No tool annotations (readOnlyHint, destructiveHint) are present, limiting agent reasoning about side effects.
Clear all mappings for a session.
Create a new test case in Zephyr
Detect the language of input text.
Export entity mappings from a session as JSON.
Get estimate for diverse figures for the next 3 years including revenue, ebit, ebitda, net income, eps, dividend, etc.
Get concise, processed analyst estimates for a given symbol.
Get concise, processed news headlines for a given symbol.
No input schemas visible for JIRA tools (get_requirements, get_test_cases, get_test_executions). Per HARD SCORING RULES, these tools receive schema score 0. Source files list shell scripts and REST API wrappers but actual schema definitions are not provided in the submitted code.
Tool descriptions for JIRA integration tools are extremely brief (<50 chars) and lack context on what data is returned, when to use each tool, or how they relate to one another. Example: 'Get JIRA requirements/epics for a project' (36 chars) does not explain whether this returns ALL requirements, how pagination works, or what fields are included.
Missing parameter descriptions for all JIRA tools. The project_id parameter in get_requirements, get_test_cases, and get_test_executions has no description explaining the expected format (e.g., 'JIRA project key in uppercase, 2-10 characters').
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 47 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 43 | - | v1 |
Get concise, processed segment data for a given symbol.
Get concise, processed financial data for a given symbol.
Get concise, processed stock summary information for a given symbol.
Get entity mappings and statistics for a session.
Get the news headlines for a given symbol.
Get JIRA requirements/epics for a project
Get segment data for a given symbol. Segments may contain business segments as well as geographical segments.
Get overall service statistics and status.
Get financial data including REIT status, annual and quarterly balance sheet, income statement, cashflow statement, per share data, general ratios, valuation ratios, and valuation and quality metrics.
Get basic stock information for a given symbol.
Get Zephyr test cases for a project
Get test executions for a project
Import entity mappings from JSON data.
Link a test case to a JIRA requirement
List all active pseudonymization sessions.
Get information about supported languages and their NER models.
Pseudonymize text by replacing sensitive entities with placeholders. Automatically detects and replaces named entities (persons, locations, organizations), email addresses, phone numbers, dates, ID numbers, IBANs, license numbers.
Update a test execution result
No tool annotations present (readOnlyHint, destructiveHint, idempotentHint). Tools like clear_session and update_test_execution are destructive but not marked, preventing agents from reasoning about retry safety and side effects. Pattern: WRITE tools should include destructiveHint: true.
Generic parameter descriptions lacking constraints. Example: detect_language has 'Text to analyze for language detection' but does not specify encoding (UTF-8), length limits, or what languages are supported. Agents cannot validate input before calling.
Output schemas are not documented in descriptions. Tools like get_stock_financials, get_analyst_estimates, and create_test_case do not describe what fields are in the response, forcing LLMs to guess what data is available for downstream use.
No pagination guidance in tool descriptions. Tools returning lists (get_requirements, get_test_cases, get_concise_news_headlines) do not document whether results are paginated, what the default limit is, or how to retrieve additional pages. Agents cannot safely handle large result sets.
Error handling is not described. Tools do not document what happens if a stock symbol is invalid, a JIRA project does not exist, or a pseudonymization session expires. Agents receive errors with no recovery guidance.
Inconsistent naming conventions across related tools. Stock data tools use both 'get_stock_X' and 'get_concise_stock_X' patterns (10 tools), creating cognitive load for LLM tool selection. Clarify when to use concise vs. detailed variants in descriptions.