Loading ToolBench
Grounded visual evaluator MCP server — honest image judgment via SigLIP2. Measures visual similarity with calibrated sigmoid scores, not prose. Returns model revision, truncation flags, and refusals when scores are unreliable.
Strong vision-specific tool design with excellent descriptions (avg 180 chars, well above baseline 72 chars minimum). All 8 tools have clear, LLM-optimized descriptions explaining WHAT, WHEN, and HOW. Input schemas present and typed for all tools. However, output schemas are not explicitly documented in the source code, only inferred from docstrings. Tool names follow verb_noun pattern (image_*, eyes_*). No parameter descriptions are missing, but some lack constraint details (e.g., threshold range, baseline pair validation rules). Error handling is present but generic in places. No tool annotations (readOnlyHint, destructiveHint) despite all tools being read-only operations.
Run known orderings on bundled reference images and report the revision used. Proves the model is loaded correctly and the cache is not corrupted. Returns pass/fail and the model revision.
Health check. Returns the model ID, revision, device, dtype, loaded status, and VRAM usage. Does not load the model if it is not already loaded.
Score an image against multiple candidate labels. Returns independent sigmoid scores for EACH label — these are NOT softmax. Multiple labels can score high simultaneously (e.g. an image with both a cat and a dog). best is the highest of the labels you supplied, not a presence verdict. Absolute scores are query-phrasing sensitive: a low number can mean a weak phrase rather than absence. Compare labels to each other. For a relative yes/no against caller-supplied contrasts, prefer image_verify.
Cosine similarity between two images. Returns a raw similarity score (0-1) and a separation verdict if baseline pairs are supplied. Without baselines, returns the raw score only. With baselines, compares the similarity to the maximum similarity measured between the baseline pairs and returns separated (true/false), margin, and a confidence band.
Check if an image contains something described by the query. Returns an independent sigmoid score (0-1) — higher means stronger visual match. The score is NOT relative to other queries. Each query is evaluated on its own. Uses SigLIP2 discriminative vision — measures visual similarity, does not generate text or hallucinate. If the image doesn't clearly match, the score will be low. NOTE — sigmoid scores are query-phrasing sensitive: absolute scores swing widely with wording, so a fixed threshold needs query engineering per use case. For robust yes/no decisions across varied inputs, prefer image_classify (relative ranking of candidate labels), which is insensitive to absolute score magnitude.
Output schemas not explicitly documented in source code. Descriptions mention return types (e.g., 'Returns independent sigmoid scores') but formal JSON Schema output definitions are absent, forcing LLMs to infer structure.
Tool annotations missing. All 8 tools are read-only (no state modification), but readOnlyHint is not declared in tool definitions. This prevents clients from optimizing caching and retry logic.
Baseline pair validation is complex and error-prone. The _parse_baseline_pairs() function validates type and length, but the constraint is not formally expressed in the schema (baselines is just 'array of arrays'). LLMs cannot infer the [path_a, path_b] structure from schema alone.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 67 | <=2025-11-25 | v2 |
Rank candidate images by similarity to a reference image. Returns matches (images that clear the baseline floor) in descending order of similarity, or an empty list if none qualify. Requires baseline pairs to establish a floor — without them, returns incomplete.
Score N images against one query. Returns independent sigmoid scores for each image. Faster than calling image_contains N times because the query is encoded once.
Verify a claim about an image against alternatives. Returns a decision (true/false), margin, and a confidence band. Compares the score of the target query against the best score of the alternative queries and returns separated (true/false), margin, and confidence.
Threshold parameter in image_contains lacks explicit range constraints in schema. Description mentions 'default 0.02' and notes query-phrasing sensitivity, but min/max bounds are not declared. LLMs may pass invalid values (e.g., threshold=5.0).
Error messages are actionable but generic across tools. _tool_error() sanitizes exceptions well (prevents path/stack leaks), but does not distinguish between retryable (OOM, transient GPU failure) and non-retryable (corrupt image, invalid path) errors. LLMs cannot infer retry strategy.