Loading ToolBench
Janusly exposes 10 tools with significant definition gaps. Most tools lack descriptions under the 10-1024 char baseline (e.g., 'saluda' is 7 chars). Input schemas are present but sparse: many parameters lack type definitions (e.g., 'brief' and 'workflow' in workflows.propose are typed 'object' with no nested schema). Parameter descriptions are minimal or missing context. Tools like 'recovery.cases.apply' and 'recovery.cases.diagnose' have domain-specific semantics (expectedRevision, immutable candidates) that are under-explained. No output schemas documented. Error handling is absent, no recovery guidance, no categorization of retryable vs fatal errors. Tool naming uses dot notation (recovery.cases.inspect) which is non-standard; verb_noun convention (inspect_recovery_case) would be clearer. The 'saluda' tool appears to be a test artifact left in production.
Make an HTTP request to an external API.
Return Janusly's deterministic top-three tenant priorities. This is the exact read model used by Home; AI cannot rerank it.
Apply an immutable validated candidate only after an independent active human approval. MCP never creates approvals.
Persist a deterministic or bounded AI-enriched diagnosis and one to three engine-owned immutable typed recovery candidates. Provider absence never blocks this operation.
Inspect bounded recovery state, transition and artifact metadata without raw evidence, outputs, workflow DAGs, or approval grants.
Validate one immutable candidate against the exact workflow snapshot and persist a content-bound validation artifact.
Descriptions critically short or missing. 'recovery.cases.apply' (25 chars) lacks context on approval workflow, immutability, or when to call it. Baseline is 194 chars avg; these are 10-15% of that.
Input schemas use generic 'object' type for complex params (brief, workflow, manualReplacement) with no nested schema. LLMs cannot infer structure. Rubric requires 'proper JSON Schema with types and descriptions', these violate that.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields. Rubric baseline: 100% of A+ tools have documented return types.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 46 | 2026-07-28+ | v2 |
greets
Search stored vector embeddings by semantic similarity.
Store or update a vector embedding in memory with optional metadata.
Compile an intent brief and bind a provider-free or caller-supplied workflow draft to exact tenant capabilities. Returns a bounded summary, never a full DAG.
No error handling or recovery guidance. Tools like 'recovery.cases.apply' (IRREVERSIBLE risk) lack confirmation/dry-run patterns. No categorization of retryable vs fatal errors. Rubric requires: 'Error responses must tell the LLM what to do next.'
Tool naming uses dot notation (recovery.cases.inspect) instead of verb_noun convention (inspect_recovery_case). Rubric baseline: 90% of A+ tools start with action verb. Dot notation is non-standard for MCP and harder for LLMs to parse.