Local web app for generating Unsloth fine-tuning datasets via OpenRouter
Dataset Genie exposes 6 tools via fastmcp/HTTP with minimal documentation. Tool names follow verb_noun convention (health, secrets_status, set_secret, get_settings, update_settings, list_models), which is good. However, descriptions are extremely brief (10-50 chars), parameter descriptions are absent or minimal, and output schemas are not documented. No error handling guidance, no parameter constraints (enums, ranges), and no indication of which operations are destructive. The set_secret and update_settings tools modify state but lack confirmation patterns or clear side-effect documentation. Input schemas exist but are sparse, most parameters lack descriptions. This is a functional but poorly documented tool set that would confuse LLMs about when to use each tool and what to expect.
Retrieves current application settings
Returns health status with version and MCP flag
Lists available models from OpenRouter catalogue with optional search and refresh
Returns status of all configured secrets (set or missing)
Sets a secret by name and value
Updates application settings with a patch object
Parameter descriptions missing or minimal. 'name' in set_secret, 'patch' in update_settings, 'search' and 'refresh' in list_models lack context about format, constraints, or valid values. LLMs cannot infer whether 'patch' is a JSON object, a string, or a structured dict.
Output schemas not documented. Tools return dicts but LLMs do not know the field names, types, or structure. E.g., list_models returns {'models': items, 'warning': warning}, is 'warning' always present? What is the structure of each item in 'models'?
Destructive operations (set_secret, update_settings) lack confirmation or dry-run patterns. No indication that these modify state. LLMs may invoke them without understanding consequences.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 56 | 2026-07-28+ | v2 |
Tool descriptions are too brief (10-50 chars). 'Returns health status with version and MCP flag' (health), 'Returns status of all configured secrets' (secrets_status), 'Sets a secret by name and value' (set_secret) lack context about WHEN to use each tool, what prerequisites exist, or what the return value contains.
No error handling guidance. If set_secret fails (invalid name), update_settings fails (bad patch), or list_models fails (API error), the tool returns a generic error. LLMs have no recovery path, should they retry? Ask the user? Call a different tool?