Loading ToolBench
A controlled stimulus generation framework for AI news authenticity research. Exposes fragment ingestion and retrieval as MCP tools.
RogueGPT defines 2 tools with schemas and descriptions present, but quality is uneven. Both tools have descriptions (ingest_fragment: 63 chars, retrieve_fragments: 66 chars) that are below the production baseline of 194 chars. Schemas are present and typed, but lack enum constraints for origin filtering and parameter descriptions are embedded in docstrings rather than in schema. No error handling guidance, no recovery patterns, and parameters lack LLM-friendly constraints. The server uses STDIO transport (hard cap 50), but definition quality alone would score mid-50s due to missing description depth, incomplete schema annotations, and lack of error classification.
Ingest a news fragment into the RogueGPT dataset.
Retrieve random fragments from the RogueGPT dataset.
Tool descriptions are below production baseline (45-48 chars vs 194 char average). ingest_fragment (63 chars): 'Ingest a news fragment into the RogueGPT dataset.' lacks WHEN to use, prerequisites, or side effects. retrieve_fragments (66 chars): 'Retrieve random fragments from the RogueGPT dataset.' omits pagination semantics.
origin parameter accepts free-form string ('Human' or 'Machine') without enum constraint. LLMs may hallucinate other values. Both tools allow this ambiguity.
Error handling is minimal. ingest_fragment catches ValidationError but returns generic error dict without recovery guidance. LLM cannot determine if error is retryable, user-fixable, or fatal. No error classification pattern (pattern:error-classification).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 57 | 2026-07-28+ | v2 |
Parameter descriptions are in docstring only, not in schema. JSON Schema input object has no descriptions on individual parameters (content, origin, is_fake, etc.). LLMs cannot access docstring text from schema introspection.
ingest_fragment combines validation, database persistence, and warning collection in one call. Response includes 'status', 'fragment_id', 'warnings', unclear which fields are always present or optional. No documented output schema structure.
machine_model parameter lacks enum constraint. It accepts strings like 'openai_gpt-4o_2024-08-06' from prompt_engine.json, but no enum list is exposed to the LLM. LLM cannot know valid values without calling retrieve_models resource.
strict_model parameter defaults to True but has no description of the difference between strict and non-strict behavior. Parameter docs say 'If True, reject unknown model names. If False, warn but allow.' but LLM cannot see this from schema.
retrieve_fragments returns list of fragments with serialized datetimes (isoformat()) but does not document the schema of each fragment object. LLM cannot know what fields to expect per fragment (e.g., Content, Origin, IsFake, etc.).
ingest_fragment has 8 parameters but only 3 are strictly required (content, origin, is_fake). Parameter relationships are not documented, e.g., 'human_outlet is required if origin=Human', 'machine_model recommended for origin=Machine'. Undocumented dependencies cause misuse.