Loading ToolBench
A candidate search and evaluation system that preprocesses profiles, builds search indices, and retrieves candidates based on query plans using embeddings and hard constraints
This server has 30 tools with significant structural gaps. Most tools lack proper input schema definitions, parameters are declared as generic 'object' types with minimal constraints. Descriptions are present but often generic (e.g., 'Runtime configuration', 'Build workspace'). No enum constraints, no parameter-level type specificity, and no documented output schemas. Tools like search_candidates and build_index accept complex nested objects (config, workspace, model_client) without field-level documentation. Error handling is absent from tool definitions. The server exposes internal implementation details (event_writer, event_stream) as parameters rather than handling them server-side. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear WRITE vs READ_ONLY risk classifications.
Build embedding index for candidate profiles
Build a complete test sample (preprocess and index)
Build embedding index for a test sample
Create a new test sample with specified size and seed
Create a new user prompt set for query generation
Get the current status of the preprocessing and indexing pipeline
List all generation runs for an owner
All 30 tools accept generic 'object' type parameters (config, workspace, model_client, event_writer, event_stream) with no field-level schema. LLMs cannot infer what fields these objects require or what types they contain.
No output schemas documented for any tool. Agents cannot plan downstream calls or extract required fields (e.g., what does search_candidates return? Does it include candidate_id for follow-up lookups?).
Tool definitions expose internal implementation details (event_writer, event_stream, model_client) as required parameters. These should be injected server-side, not passed by the agent.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 36 | 2026-07-28+ | v2 |
List all retrieval trials
List all test samples with their metadata and status
List all user prompt sets
Generate query plans from user prompts using LLM
Preprocess raw candidate profiles using LLM to extract structured fields and generate embeddings search texts
Preprocess profiles in a test sample
Read details of a generation attempt
Read metadata of a generation run
Read events from a generation run with pagination
Read metadata of a retrieval trial
Read errors from a retrieval trial with pagination
Read search results from a retrieval trial with pagination
Read metadata and status of a test sample
Read preprocessing and indexing errors from a test sample with pagination
Read preprocessed profiles from a test sample with pagination
Read raw profiles from a test sample with pagination
Read metadata of a user prompt set
Read errors from a user prompt set with pagination
Read query plan mappings from a user prompt set with pagination
Read prompts from a user prompt set with pagination
Resample an existing test sample with new size and seed
Run a retrieval trial using a test sample and user prompt set
Search for candidates based on a query plan with hard constraints and soft preferences using embeddings
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear WRITE vs READ_ONLY risk classifications in metadata. Agents cannot determine which tools are safe to retry.
Parameter descriptions are generic placeholders ('Runtime configuration', 'Build workspace', 'Model client for embeddings'). No guidance on what fields are required, valid ranges, or dependencies between parameters.