Loading ToolBench
Treats normalized scan findings as immune signals: discovers co-occurring fleet threat patterns, synthesizes non-executing defensive recipes, retains successful outcomes as recallable memory, and targets adjacent cohorts likely to show the same variant.
Fleet Immune System defines 6 tools with complete JSON schemas and descriptions. Naming follows verb_noun pattern (immune_*) but lacks clarity on distinctions between similar operations. All tools have descriptions (avg 120 chars, within baseline 194), but descriptions are domain-specific and lack actionable context for LLM selection. Schemas are well-formed with type constraints, enums, and minLength/minItems validation. However, parameter descriptions are entirely absent, the schema defines structure but not semantics. No error handling guidance, no output schema documentation, and no indication of which tools are idempotent or safe to retry. Domain terminology (antibody, cohort, crystallize) is specialized but unexplained.
Breed two non-executing antibody recipes into a deterministic next-generation candidate without escalating destructive action.
Persist a successful pattern/antibody outcome into the server-local immune memory for later recall.
Plan one fleet immune cycle with variants, memory recall and adjacent cohort targeting.
Discover co-occurring emergent fleet threat patterns from normalized defensive scan signals.
Recall the closest retained immune-memory crystal for a threat pattern.
Synthesize a non-executing defensive antibody recipe from a threat pattern.
Parameter descriptions entirely absent. Schema defines structure (type, minLength, enum) but LLMs cannot infer semantic meaning of 'cohort', 'signature', 'novelty', 'fitness', 'floor', 'threshold', or 'mutation' from names alone. This violates the critical requirement that every parameter must have a description explaining what it controls.
No output schema documentation. Tools return structured results (patterns, antibodies, outcomes) but LLMs have no specification of return fields, types, or what to expect. This prevents downstream tool chaining and forces LLMs to guess field names.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | 2025-06-18+ | v2 |
No error handling guidance. Tools throw generic errors (e.g., 'Invalid Request', 'Method not found', 'unknown tool') with no recovery hints. LLMs cannot determine if errors are retryable, user-fixable, or fatal.
Tool descriptions lack actionable context. 'Discover co-occurring emergent fleet threat patterns' is domain-specific but does not explain WHEN to call this vs immune_cycle, WHAT prerequisites exist, or WHAT the output enables. Descriptions should answer: what does it do, when to use it, what does it return?
No idempotency or destructiveness hints. immune_crystallize is marked WRITE but no tool declares readOnlyHint or destructiveHint. LLMs cannot determine which tools are safe to retry or which have irreversible consequences.