Loading ToolBench
A FastAPI-based server that queries ClinicalTrials.gov API using natural language queries parsed by OpenAI's GPT API
Single tool 'query_trials' has a reasonable description (194 chars, within baseline 34-392 range) but critical structural deficiencies. Input schema is present with 'query' parameter typed as string with description, meeting minimum requirements. However, no output schema is documented, LLMs cannot plan downstream calls or extract structured data. Tool uses eval() on untrusted LLM output (critical security flaw). Error handling returns generic error strings without recovery guidance. No parameter constraints (enums, ranges, patterns). Tool description lacks WHEN to use it, prerequisites, or state-modification clarity. No pagination support despite potentially returning many trials. Response includes raw API fields without stripping irrelevant metadata. Tool is read-only but lacks explicit idempotentHint annotation.
Query clinical trials from ClinicalTrials.gov using natural language. Converts natural language queries to API parameters using OpenAI's GPT API, then retrieves matching trials and returns a summary of each study including NCT ID, title, status, and phase.
No output schema documented. LLMs cannot infer response structure (fields, types, nesting). Prevents downstream tool chaining and forces LLM to parse unstructured text.
eval() on untrusted LLM-generated output is a code injection vulnerability. Malicious or malformed GPT responses can execute arbitrary Python. Use json.loads() with schema validation instead.
Error responses are generic strings ('OpenAI parsing query failed: {e}') without recovery guidance. LLM cannot determine if error is retryable, user-fixable, or fatal.
No pagination support. API may return hundreds of trials; tool caps at pageSize=5 default but does not expose limit/offset parameters or document result limits in description.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 42 | 2026-07-28+ | v2 |
Tool description does not state it modifies no state (read-only) or explain when to use it vs. other trial discovery methods. Missing WHEN/WHY context for LLM selection.