Input schemas lack comprehensive type definitions and parameter descriptions. Across all 60 tools, parameter descriptions are minimal or absent. Example: 'alm_integrations_set_pat' describes the tool but does not enumerate what parameters it accepts (e.g., alm_key, token format, required vs optional). LLMs cannot infer valid parameter values from sparse schemas.
Output schemas are not documented. Tools return complex SonarQube objects (issues, projects, components) but the server does not publish what fields are in the response, their types, or whether pagination is available. LLMs cannot plan multi-step calls or extract needed IDs when response structure is opaque.
Destructive tools (delete, update, bulk operations) lack confirmation or dry-run support. Example: 'issues_delete_comment', 'alm_settings_delete', 'issues_bulk_change' can irreversibly modify SonarQube state but offer no confirmation_before_execute or dry_run option. Agents risk data loss.
Descriptions are generic and lack actionable guidance. Many tools (e.g., 'alm_integrations_search_bitbucketserver_repos', 'issues_assign') repeat SonarQube API documentation verbatim without explaining WHEN to use them, WHAT they return, or prerequisites. Baseline for A+ tools is 194 chars; most here are <80 chars with minimal context.
No pagination documentation. Tools that return lists (issues_search, components_search, hotspots_search, measures_component_tree) do not document page/offset parameters, result limits, or whether a total_count or next_cursor is returned. Without pagination guidance, LLMs may fetch incomplete or excessive results.
No error handling guidance. Tools do not document what errors can occur, whether they are retryable, or what the LLM should do if a call fails. Example: 'authentication_login' does not explain what error is returned for invalid credentials, whether to retry, or how to guide the user to fix the input.
Permission requirements mentioned in descriptions but not formalized. Many tools state 'Requires Administer System permission' in the description text, but this is not exposed as structured permission metadata. Agents cannot apply least-privilege config or audit permissions declaratively.
Natural-language identifiers not supported. Tools likely require system IDs (alm_setting_key, project_key, component_id) rather than accepting user-friendly names (project name, component path). This forces extra lookup calls and increases agent reasoning burden.
For all 60 tools: Expand descriptions to 100-200 chars (baseline = 194 chars). Include WHAT the tool does, WHEN to use it (vs similar tools), and WHAT it returns. Example for 'issues_search': 'Search SonarQube issues by status, severity, and assignee. Returns up to 50 issues per page with IDs, type, severity, and assignee. Use this to find issues for a project before bulk-updating them. Requires Browse permission on the project.'
Document input schemas comprehensively. For each tool parameter, add: type (string, integer, enum, array), description (20+ chars), examples (not in description, in schema), constraints (min/max, regex, enum values), and required vs optional flag. Example for 'alm_integrations_set_pat': { alm_key: { type: 'string', description: 'DevOps Platform setting key (e.g., github, azure-devops). Find via alm_settings_list().', required: true }, token: { type: 'string', description: 'Personal Access Token with minimum required scopes for the platform. Tokens are stored securely server-side.', required: true } }
Document output schemas. For each tool, publish a sample response with field names, types, and descriptions. Example for 'issues_search': 'Returns {issues: [{id, key, type, status, severity, assignee, created_date}], total: number, page: number, page_size: number}'. Include references to fields needed for follow-up calls (e.g., issue.id for issues_add_comment).
Add pagination parameters and guidance. Tools returning lists (issues_search, components_search, etc.) should document: page (default 1), page_size (default 20, max 100), and response structure including total_count. Example: 'Returns up to 100 issues per page. Set page=2 to fetch the next 100. total_count tells you how many issues exist overall.'
Score history
Overall score trend
↓ 26 points across a rubric change (v1 → v2)
0/100
Scored
Grade
Overall
Spec posture
Rubric
2026-09-22
F
0
2026-07-28+
v2
2026-03-09
F
26
-
v1
Implement confirmation patterns for destructive tools. Add a 'dry_run' parameter to issues_bulk_change, alm_settings_delete, and similar tools. Or add a separate confirm_bulk_change(request_id) step. Example: issues_bulk_change(issues=[...], changes={...}, dry_run=true) returns 'Would update 5 issues. Call with dry_run=false to confirm.'
Add error handling guidance. Document error conditions and recovery steps in each tool description. Example for authentication_login: 'Returns 401 Unauthorized if credentials are invalid. Verify the username/password and retry. If the user is disabled, contact a SonarQube admin.' For alm_settings_delete: 'Returns 409 Conflict if projects are bound to this setting. Use alm_settings_count_binding(key) to check, then unbind projects before deleting.'
Formalize permission requirements. Add a 'required_permissions' field to tool metadata or include as structured JSON in tool description. Example: required_permissions: ['admin:system'] or ['browse:project']. Enable agents to pre-flight permission checks via get_current_user_permissions().
Support natural-language identifiers for lookup tools. For tools like alm_settings_list, ce_component, measures_component, add alternate parameters that accept names or partial matches. Example: ce_component(component_id='...' OR component_key='...' OR component_path='...'). Document internally: 'If agent only has component name from chat, resolve via components_search() first.'
Add idempotency markers. For non-destructive reads (issues_search, components_search, etc.), add idempotent: true to tool metadata. For potentially-destructive writes, document idempotency strategy (e.g., 'Issues are identified by ID; calling issues_assign twice with the same issue_id and assignee is safe; idempotent: true'). Agents need to know when retries are safe.
Strip verbose response fields. If SonarQube API returns audit metadata, timestamps, or internal fields not needed for chat, filter them in the tool implementation. Keep responses under 500 tokens for discovery tools and under 1000 for detailed tools. Example: issues_search should return {id, key, type, severity, status, assignee}, not raw SonarQube API response with analyzed_date, uuid, author, update_date, etc.