Windows App UI Automation Test Framework and MCP connector for ChatGPT integration
This is a mixed-language SDK with 17 tools across Node.js and C#/Python components. Critical issues: (1) Tool definitions are inferred from file paths and brief descriptions rather than explicitly registered with full schemas in a single MCP server implementation. (2) Most tools lack detailed parameter descriptions, many parameters have only 1-3 word descriptions ('Agent identifier', 'Job ID'). (3) No visible input validation, error handling guidance, or output schema documentation. (4) Tool names are inconsistent in style (camelCase vs PascalCase). (5) No evidence of pagination, rate limiting, or idempotency patterns. (6) Security-critical tools (Store, Cleanup, AppBotAppendFile) lack permission gates or audit trail documentation. The codebase appears to be a collection of utilities rather than a cohesive, production-grade MCP server.
Append a single line to a canonical log file. If the target file was created on a previous day, rotate it into .old-append/YYYYMMDD/ and start a fresh file for today.
Trim old outputs for one tool — keeps only the most recent keepLast log files
Serialize a batch of ToolOutputRecords into MCP-style tool_use / tool_result XML blocks so the fallback AI receives them in the same shape its native tool-use protocol would deliver
Read by a combined <tool>/<slug> address
Read full captured output by tool + slug
Store a tool's output and return the assigned record. Writes verbatim output to tool-out/<tool>/<slug>.log and appends a trimmed record to tool-out/<tool>/index.jsonl. Slug collision within a tool is resolved by appending -2, -3, ... .
Append data to an existing session
Parameter descriptions are minimal (1-3 words). LLMs cannot infer meaning from 'Agent identifier' or 'Job ID' alone. Descriptions must explain WHAT the parameter controls, WHEN it's needed, and any constraints (format, range, enum values).
No output schemas documented. Tools return data but LLMs have no way to know what fields to expect, preventing downstream tool chaining and forcing agents to guess field names.
Destructive tools (Cleanup, AppBotAppendFile, Store) lack permission gates, audit trail documentation, or confirmation patterns. No evidence of access control or logging for compliance.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 39 | 2026-07-28+ | v2 |
Mark a job as complete with result
Create a new job for execution
Create a new session
Retrieve a job by ID
Retrieve a session by ID
Send agent heartbeat
List all registered agents
List all jobs
List all sessions
Get the next job for an agent
List tools (listJobs, listAgents, listSessions) have no pagination parameters (limit, offset, cursor). Unbounded results will exhaust context windows and degrade LLM reasoning.
Tool definitions appear inferred from file paths rather than explicitly registered in a single MCP server. No visible tool registration code, capability negotiation, or protocol handshake in provided source.