Loading ToolBench
FireKey MCP has 14 well-named tools with consistent verb_noun patterns (browser_start, profile_create, group_list). All tools have descriptions (avg ~80 chars, within baseline 34-392). Input schemas are present and typed for all tools, with enums used appropriately for constrained values (e.g., headless: ["0","1"], os: ["win","macArm","macIntel","android"]). However, parameter descriptions are sparse, many lack detail on format, range, or dependencies. Output schemas are not documented in the source. Error handling is basic (HTTP status checks, generic error messages). No tool annotations (readOnlyHint, destructiveHint) despite clear risk classifications (READ_ONLY, WRITE, DESTRUCTIVE). Parameter relationships (e.g., timezone_mode=manual requires timezone) are undocumented. Descriptions are functional but not LLM-optimized, they state WHAT but rarely WHEN or WHY.
List currently running browser profiles with their WebSocket endpoints.
Start a browser profile. Returns WebSocket endpoints for Puppeteer/Selenium automation.
Stop a running browser profile.
Create a new profile group.
List profile groups with optional name search.
Update an existing group.
Create a new browser profile with fingerprint and proxy settings.
Output schemas not documented. LLMs cannot infer what fields browser_start returns (WebSocket endpoint structure, connection details, etc.) or what profile_list yields (pagination, field names). Forces agents to guess downstream field names.
Parameter descriptions lack format/range constraints. profile_create has 30+ params but most descriptions omit valid ranges (e.g., cpu_threads: '2,4,6,8,12,16' mentioned in description but not enforced as enum), timezone format, or dependencies (timezone_mode=manual requires timezone param). LLMs cannot validate inputs.
No tool annotations despite clear risk classifications. Tools marked DESTRUCTIVE (profile_delete, profile_delete_cache) and WRITE (profile_create, browser_start) lack readOnlyHint/destructiveHint/idempotentHint annotations. Agents cannot distinguish safe from dangerous operations.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2026-07-28+ | v2 |
Delete browser profiles (soft delete to trash).
Delete cached data for browser profiles (cookies, history, localStorage, etc.).
List browser profiles with pagination and filtering.
Move profiles to a different group.
Update an existing browser profile.
Create a new proxy configuration.
List saved proxy configurations.
Error handling is generic. client.ts returns 'HTTP 401: Unauthorized' or 'Invalid JSON response' with no recovery guidance. No distinction between retryable (timeout) vs user-fixable (invalid proxy_host) vs fatal errors. Agents cannot self-correct.
Undocumented parameter dependencies. profile_create: timezone_mode=manual requires timezone; language_mode=manual requires languages; proxy_method=saved requires proxy_saved_id. No cross-parameter validation or dependency hints in descriptions.