Loading ToolBench
A CLI and MCP server for analyzing iOS code quality using AI suggestions and test results.
This server has significant quality gaps. Tools are clearly named with action verbs (test, lint, list-tests), but descriptions lack depth and context for LLM selection. Parameter schemas are present with type information, but descriptions are minimal and many parameters lack guidance on expected formats, constraints, or dependencies. Output schemas are not documented, critical for agents planning downstream operations. Error handling is present in code but not described in tool definitions. The server attempts legitimate iOS testing/linting automation, but the tool interface is underdeveloped for agent reasoning.
Run SwiftLint and report issues with comprehensive analysis
Discover and list all available tests in the iOS project
Run iOS tests and report failures with comprehensive error handling. Supports running specific tests and optional target parameter.
Tool descriptions are too brief and lack context for LLM selection. 'Run iOS tests and report failures with comprehensive error handling' (test) and 'Run SwiftLint and report issues with comprehensive analysis' (lint) do not explain WHEN to use each tool, what prerequisites exist, or what structure is returned. Descriptions should be 50-200 characters and answer: what does it do, when should the LLM call it, what does it return?
Parameter descriptions are minimal or missing context. For example, 'destination' in test tool is described as 'Test destination (e.g., 'platform=iOS Simulator,name=iPhone 15')', this includes a sample value (an anti-pattern; LLMs reuse examples literally) but lacks explanation of valid format, required structure, or when it is optional vs required. 'scheme' parameter lacks any guidance on scheme discovery or relationship to xcodeproj/xcworkspace.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 50 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 48 | - | v1 |
Output schemas are not documented. LLMs cannot determine what fields to expect from test, lint, or list-tests results. Without documented response structures (success flag, data payload, error fields, build errors, warnings, test results), agents cannot plan downstream operations or extract the right data. Code shows formatTestResultResponse and other formatters exist, but response schema is not exposed in tool definitions.
Parameter dependencies and mutual exclusivity are undocumented. The test tool accepts both xcodeproj and xcworkspace, but it is unclear whether both can be provided, only one is allowed, or the tool resolves the ambiguity internally. Similarly, the lint tool description says 'path' is required, but the schema shows 'required: false'. This inconsistency violates the schema contract and confuses agents.
No pagination or result limiting guidance. The list-tests tool will discover and return all tests in a project, but there is no mention of pagination, limit, offset, or a result cap. Large iOS projects with hundreds of tests could blow the context window. Tool descriptions should state: 'Returns up to N tests; call with offset to fetch more.' Baseline pattern expects paginated_result with limit/offset and total_count.
Error handling is present in code (buildErrors, success flag) but not described in tool definitions. Agents need recovery guidance: if test fails, should they retry, ask the user for a different destination, or is it a fatal schema mismatch? Tool descriptions must explicitly state: 'On failure, returns buildErrors array with actionable error messages. Try a different destination if iOS Simulator is unavailable.'
Sample values in descriptions violate LLM prompt-engineering best practices. The destination parameter includes 'e.g., platform=iOS Simulator,name=iPhone 15', LLMs tend to reuse example values literally rather than adapting to actual context. Replace with formal constraints: 'Format: platform=<iOS|macOS>,name=<device-name>. Example valid values: iPhone 15, iPhone SE (3rd generation), iPad Pro (6th generation).' Better yet, document supported devices and use validation to reject invalid names.
Tool composition and idempotency are unclear. Can test be called multiple times with the same parameters and produce identical results (idempotent)? If tests have side effects (e.g., modifying simulator state), agents should know they might fail on retry. Descriptions should state: 'Idempotent: can be safely retried.' or 'Destructive: modifies simulator state; do not retry without verification.'
The 'path' parameter in lint tool is marked required=false in schema but described as required in the tool description. This is a contract violation. Schemas must match descriptions. Either make path required in the schema and clarify that it is the root to analyze, or make it optional and document what happens if omitted (e.g., 'Defaults to project root').