Loading ToolBench
Dependency-free detectors for deceptive signals in AI-agent marketplaces. Exposes integrity indicators as MCP tools for autonomous agents to verify marketplace listings before committing time or resources.
Five tools with clear, domain-specific descriptions (avg 180 chars, within baseline 194). All tools have input schemas with typed parameters. Naming is verb-forward and unambiguous (scan_listings, check_listing, list_indicators, make_observation, generate_audit_certificate). However, parameter descriptions are sparse, most lack detail on format, constraints, or valid ranges. Output schemas are not documented in the code. Error handling is absent; no guidance on recovery or retryability. Security posture is good (READ_ONLY risk, no secrets exposed), but composition could be tighter, scan_listings and check_listing overlap in intent.
Check a SINGLE listing for red flags before bidding on or accepting it: application/view inversion (fabricated interest) and unpaid-work risk (priced with no escrow or payment evidence). Returns findings for that one listing.
Generate a machine-readable Boardcheck Audit Certificate and embeddable verification badge for a marketplace based on its current listings batch.
List the integrity indicators this tool checks: stable AMS-* IDs, names, maximum severity, and a plain-language summary of each. Lets an agent discover what can be checked and cite findings precisely by ID.
OPT-IN: produce a privacy-preserving, non-reversible summary of a scan that MAY be contributed to improve the shared thresholds over time. It contains no platform name, no listings, and no identity — only coarse buckets and boolean flags. Returns null if the sample is too small to share safely. Nothing is sent anywhere by this call; sharing is a separate, manual step.
Scan a batch of AI-agent marketplace listings for deceptive integrity signals: fabricated engagement, seeded/astroturfed listings, supply-glut boards, unpaid-work risk, and high-budget bait. Returns findings, a severity summary, and a data-coverage report. Use this to verify a marketplace before trusting its numbers or committing time to it — e.g. before an agent decides where to look for paid work. Each listing is a dict; only ``id`` and ``created_at`` are required.
Output schemas not documented. Tools return dicts but LLMs cannot infer field names, types, or structure. Downstream tool chaining and result extraction are error-prone.
Parameter descriptions lack format/constraint details. 'listings' param says 'List of listing dictionaries' but does not specify required fields beyond id/created_at, expected data types, or size limits. LLMs cannot validate input.
No error handling or recovery guidance. Tools do not document what happens on invalid input (e.g., missing id/created_at, malformed dates). LLMs receive no actionable error messages.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2026-07-28+ | v2 |
Semantic overlap: scan_listings and check_listing both detect unpaid-work risk and application/view inversion. Tool selection ambiguity forces LLM to reason about batch vs. single-item intent.
make_observation returns Optional[dict] but does not document the structure of the observation object or the conditions under which null is returned (min_sample threshold). LLMs cannot plan around null responses.