MCP server for parsing screen UI elements, detecting interactive components, bounding boxes, and icons from screenshots
This MCP server has critical structural issues preventing production use. The server claims to parse UI bounding boxes but provides only a stub implementation with hardcoded mock responses. The single tool 'execute_skill_action' has a schema (screenshot_width, screenshot_height, confidence_threshold) but the actual implementation in mcp_server.py does not register or validate this tool, it only prints a JSON status message. The client.py returns hardcoded mock data unrelated to input parameters. The tool is never actually invoked or wired to real processing logic. Parameter descriptions exist but are minimal. No error handling, no input validation, no output schema documentation, and the core MCP protocol handshake is incomplete (mcp_server.py does not respond to protocol requests beyond a one-time status print).
Parse screen UI bounding boxes and interactive elements from screenshots
Tool not registered with MCP server. mcp_server.py prints a one-time status message but does not implement the MCP protocol's tool_list endpoint or handle tool calls. The tool 'execute_skill_action' is never invoked.
No input schema validation. Parameters (screenshot_width, screenshot_height, confidence_threshold) lack type constraints, min/max bounds, and enum values. screenshot_width and screenshot_height default to 1920x1080 but the code never validates ranges or uses them.
Tool implementation is a hardcoded mock stub. client.py returns fixed JSON with mock URLs and counts (screen_parse_id='omn_prs_5519', interactive_elements_detected_count=28) regardless of input parameters. No actual bounding box parsing or vision processing.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 30 | 2026-07-28+ | v2 |
No output schema documentation. The return object structure (screen_parse_id, interactive_elements_detected_count, clickable_icons_count, text_input_fields_count, bounding_boxes_normalized_ast_url, annotated_som_screenshot_url) is inferred from client.py but not formally documented for LLM consumption.
Incomplete protocol implementation. mcp_server.py does not handle requests/responses according to MCP spec. It prints a one-time JSON init message and exits. No request handling loop, no resource management, no error responses.
Parameter descriptions are generic and uninformative. 'Width of the screenshot in pixels' (for screenshot_width) does not explain accepted ranges, aspect ratios, or typical values. 'Confidence threshold for element detection (0-1)' lacks guidance on how this affects results or when to adjust it.
Tool description lacks WHEN to use it and WHY. 'Parse screen UI bounding boxes and interactive elements from screenshots' does not explain when an agent should call this vs. alternative tools, what downstream actions require the output, or what constitutes success/failure.
No error handling or recovery guidance. If confidence_threshold is invalid (e.g., 1.5), no error is returned. If screenshot dimensions exceed API limits, no guidance is given. No per-item success/failure reporting.
Tool name 'execute_skill_action' is vague and does not follow verb_noun convention. 'Parse_screen_ui' would be clearer and self-documenting about the action.
No idempotency hints or destructive operation markers. Tool annotations (readOnlyHint, idempotentHint) are absent, so LLMs cannot reason about retry safety or state changes.