MCP server (harness) for the TypeSafe AI System One API (Jev): evaluate, batch, rank, taxonomy walk, question lint
Tenbin demonstrates solid tool design with clear naming (verb_noun pattern), comprehensive parameter schemas, and detailed descriptions. All 7 tools have explicit schemas with typed properties and descriptions. Tool names are action-oriented (lint, evaluate, rank, walk). However, output schemas are not documented in the visible code, only input schemas are defined. Error handling guidance is minimal. Descriptions are well-crafted (100-250 chars) but lack explicit recovery hints for failure cases. The server uses tool annotations (readOnlyHint, idempotentHint) correctly, signaling read-only semantics.
POST /v1/systemone: evaluate a state against a map of Choice/Score/Noul questions in one call. Put every question that uses this state here, including speculative ones. Questions are linted first; errors block, warnings are returned alongside answers. Returns answers, usage, cost_usd, request_id, latency_ms.
Runs tenbin_evaluate over a list of states concurrently (map-reduce, feature extraction, threshold calibration). repeat>1 measures self-consistency. Returns per-row answers, per-question statistics (mean/std, choice histogram, confidence), and failures.
Static checks on a questions map without calling the API: level/option counts, numeric-only or degree-only levels, compound or counting/date instructions, inverted Noul criteria, missing state paths, state fields no question uses, forbidden state paths, token budget. Run before tenbin_evaluate.
GET /v1/models plus the current pricing table and the server's default model.
Scores each candidate against a query (rerank, semantic find, dedupe). noul mode asks one independent Noul per candidate in a fan-out; choice mode asks one Choice over candidate ids. Batches automatically when candidates exceed the token budget or the 255-option cap. Optional existence_check Noul says whether any candidate fits.
Output schemas not documented. Tool descriptions state what is returned (e.g., 'Returns answers, usage, cost_usd, request_id, latency_ms') but no formal JSON Schema for responses is visible in the code. LLMs cannot plan downstream tool calls or extract fields reliably without documented output structure.
Error handling lacks recovery guidance. Descriptions mention 'errors block' and 'warnings are returned' but do not explain what the LLM should do on failure (retry, ask user, call alternative tool). No actionable error messages visible in implementation.
Parameter 'mode' in tenbin_rank uses enum ['noul', 'choice'] but lacks explanation of when to use each. Description does not clarify the semantic difference or guide LLM selection between modes.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2026-07-28+ | v2 |
Calls, failures, tokens, cost in USD, and remaining session token budget for this server process.
Walks a nested tree level by level, asking a Choice whose option descriptions are the subtrees and keeping the top beam_width paths by geometric-mean edge probability. Levels with more than 255 options or oversized descriptions are split into chunks whose winners meet in a final round; beam paths share requests where the limits allow.
tenbin_list_models and tenbin_session_stats accept empty input objects (no required properties). Descriptions do not explain why these are parameterless or what context they operate on (current session? global state?).
No pagination or result-limiting guidance. tenbin_evaluate_many accepts arbitrary-length 'states' array; no documented limit or guidance on batch size. Large batches could exhaust token budgets or API limits without explicit constraints.