Loading ToolBench
Provides tools to retrieve information about SonarQube projects.
SonarQube MCP server has 7 tools with mixed quality. Naming is generally good (verb-noun patterns: get_*, create_*, delete_, list_). Descriptions are present but brief (average ~80 chars, baseline 194 chars). Schemas are visible for 2 tools (get_status has empty input {}, get_sonarqube_metrics has project_key string). Output schemas are not documented anywhere. Error handling is absent, no recovery guidance, no retryability classification, no actionable error messages visible in code. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite having READ_ONLY and DESTRUCTIVE risk levels that should be declared.
Creates a new SonarQube project. Requires administrator privileges.
Deletes a SonarQube project. Requires administrator privileges. USE WITH CAUTION!
Retrieves component tree metrics for a SonarQube project, including metrics for files, directories, and other components.
Retrieves the latest metrics for a specific SonarQube project.
Retrieves the historical metrics for a specific SonarQube project within a date range.
Performs a health check on the configured SonarQube instance using /api/system/status. Returns a readable status message.
Tool definitions inferred from client_test.py rather than visible in server.py. Cannot verify actual registration, schema compliance, or implementation for 5 of 7 tools (create_sonarqube_project, delete_sonarqube_project, list_projects, get_sonarqube_metrics_history, get_sonarqube_component_tree_metrics).
No output schemas documented. LLMs cannot plan downstream tool calls or extract required data. E.g., list_projects returns unknown structure; cannot determine if it includes project_key needed by other tools.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-21 | D | 52 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 48 | - | v1 |
Lists all SonarQube projects available in the configured SonarQube instance.
Descriptions are too brief (average 60 - 70 chars vs baseline 194 chars). Missing context for tool selection and when to use one vs another. E.g., delete_sonarqube_project says 'Deletes a SonarQube project' but does not explain consequences, required permissions, or recovery steps.
Destructive and write-heavy tools lack error handling guidance. delete_sonarqube_project and create_sonarqube_project have no recovery paths, confirmation steps, or actionable error messages for LLM retry.
Tool annotations missing. Risk levels (READ_ONLY, WRITE, DESTRUCTIVE) are in metadata but not declared as tool annotations (readOnlyHint, destructiveHint, idempotentHint) per MCP spec. LLMs cannot distinguish safe vs irreversible operations from tool definition alone.
No pagination declared. get_sonarqube_component_tree_metrics accepts page_size but response schema unknown, no total count, next_cursor, or limit documented. list_projects returns unknown structure, could return thousands of projects without warning.
Parameter descriptions lack format and constraint details. E.g., from_date and to_date accept ISO 8601 per description, but no validation hint for LLM (e.g., 'must be YYYY-MM-DD'). metric_keys has no guidance on valid keys.
Credentials and secrets likely exposed. Code loads .env with SONARQUBE_* and LLM API keys; no evidence that SonarQube auth credentials are injected server-side vs passed as tool parameters. If auth is parameterized, it will appear in logs and traces.