Loading ToolBench
The Conduit MCP server has 19 tools covering Phabricator/Phorge operations. Most tools have descriptions and input schemas with parameter types. However, there are critical gaps: (1) descriptions are often terse (10-50 chars) when the rubric baseline recommends 34-392 chars; (2) parameter descriptions are inconsistent, some parameters lack detail about constraints, formats, or valid values; (3) output schemas are completely undocumented, no tool specifies what fields it returns; (4) error handling guidance is absent; (5) no per-tool security declarations or permission requirements; (6) parameter naming has some inconsistencies (e.g., 'repository' vs 'repository_identifier' used interchangeably). These gaps prevent LLMs from confidently selecting tools and composing multi-step workflows. The server lands in the C (Fair) range, above-average parameter coverage but insufficient for production confidence.
Add a comment to a code review.
Create a new code review (Differential revision).
Create a diff from raw diff content.
Get detailed information about a specific code review.
Get a commit message template for a code review.
Get the raw content of a diff.
Search for code reviews (Differential revisions) with token optimization.
NO OUTPUT SCHEMAS DOCUMENTED. All 19 tools lack documented return types. LLMs cannot plan downstream tool calls or extract required data (task_id, revision_id, repository_phid). This forces agents into failure-retry loops when they need context from prior results.
PARAMETER DESCRIPTIONS LACK CONSTRAINT DETAILS. Most parameters (e.g. 'query_key', 'status', 'priority', 'action') have no enum constraints or valid-value lists in descriptions. LLMs cannot self-correct invalid inputs, they hallucinate values. E.g., pha_user_search query_key accepts 'active|admin|all|approval' but description does not list these.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-21 | D | 55 | 2026-07-28+ | v2 |
| 2026-03-09 | C | 60 | - | v1 |
Update an existing code review with new diff or metadata.
Browse files and directories in a repository with token optimization.
Create a new repository in Phabricator.
Get the content of a specific file from a repository.
Get commit history for a repository or specific path with token optimization.
Get detailed information about a specific repository.
Search for repositories in Phabricator with token optimization.
Create a new Phabricator task.
Get details of a specific Phabricator task
Update an existing Phabricator task
Search for users with advanced filtering capabilities.
Get the current user's information.
TOOL DESCRIPTIONS TOO BRIEF. Most descriptions are 30-50 chars (below the rubric baseline of 34-392). They lack context on WHEN to use the tool, dependencies on other tools, or what makes this tool distinct from similar ones. E.g., pha_task_create is just 'Create a new Phabricator task', does not explain prerequisites, return value, or error recovery.
NO ERROR HANDLING GUIDANCE. No tool describes what errors can occur, whether they are retryable, or what the LLM should do next. E.g., if pha_user_search finds no matches, should the LLM fuzzy-search, or abandon? If pha_diff_create fails due to invalid reviewer, should it list valid reviewers?
AMBIGUOUS PARAMETER FORMATS. Parameters like task_id, diff_id, revision_id, repository_identifier accept multiple formats (ID, PHID, name, callsign) but descriptions do not specify which are preferred or what format conversions happen internally. LLMs may pass wrong format.
NO PERMISSION DECLARATIONS OR SCOPE HINTS. Tools like pha_task_update, pha_diff_create, pha_repository_create are destructive/sensitive but lack permission scope declarations (e.g., 'requires: write:task, admin:repository'). No audit trail pattern visible. Agents cannot configure least-privilege access.
INCONSISTENT PARAMETER NAMING. Some tools use 'repository' (pha_diff_create), others 'repository_identifier' (pha_repository_info). This variation confuses LLMs and breaks tool composition patterns where output from one tool feeds into another.
MISSING PAGINATION & LIMITS IN OUTPUT. Tools like pha_user_search, pha_diff_search, pha_repository_search accept limit params but do not document pagination strategy (offset, cursor) or total count in response. Large result sets risk blowing context window without guidance.
NO TOOL ANNOTATIONS FOR READ/WRITE DISTINCTION. Tools are labeled with Risk: READ_ONLY or WRITE in metadata, but MCP tool annotations (readOnlyHint, destructiveHint, idempotentHint) are not visible in schema. This prevents agents from reasoning about side effects and retry safety.