Static source inference · medium confidence · evidence: Streamable HTTP
Current-spec patterns detected
Summary
Knotwork exhibits severe definition quality issues. The server exposes 11 tools, but source inspection reveals fragmentary tool definitions scattered across test files and implementation repositories. Most tools lack complete input schemas, descriptions are generic or missing entirely, and parameter documentation is minimal. The evidence shows tool definitions inferred from test mocks and repository classes rather than explicit, well-structured registrations. Only 2 of 11 tools have both a description AND a visible input schema with typed properties; the remaining 9 tools either lack schemas entirely or have descriptions under 30 characters. No output schemas are documented. Error handling guidance is absent. Security concerns are acute: the `http_request` and `delegate_task` tools accept unbounded external input without validation or rate limiting guidance. The `schedule_task` and `delegate_task` tools lack enum constraints for their model/interval parameters, inviting hallucinated values.
Tools (11)
append_filereversible33/100
Append content to file in workspace
delegate_taskwriteauthsource verified60/100
Delegates a complex or specialized task to an external LLM and saves the result to memory. ONLY use this tool if you need cloud reasoning.
9 of 11 tools completely lack visible input schemas. read_file, list_files, find_files, write_file, edit_file, append_file, delete_file, and http_request have zero schema documentation in the source code.
9 of 11 tools have descriptions shorter than 30 characters (e.g., 'Read file from workspace', 'List files in workspace directory'). Violates pattern:tool-description.
Create explicit tool registrations in a canonical location (e.g., a ToolRegistry class or manifest file). Every tool must declare its name, description (50-200 chars), input schema with typed parameters and descriptions, and output schema. Do NOT scatter definitions across test files.
For file operation tools (read_file, list_files, find_files, write_file, edit_file, append_file, delete_file): expand descriptions to 100-150 characters explaining WHEN to use each tool, WHAT constraints apply (e.g., 'Operates within the workspace directory only'), and HOW results are returned. Example: 'Read a text file from the workspace directory. Fails if file does not exist or is outside workspace bounds. Returns file contents as plain text or UTF-8 string depending on file type.'
Add comprehensive input schemas for all tools. At minimum: (1) explicitly type all parameters as string|number|boolean|object|array; (2) add descriptions for each parameter explaining constraints, format, and examples; (3) mark parameters as required/optional; (4) add enum or pattern constraints where values are bounded. For file paths, add pattern: '^[^/][^\0]*$' to reject absolute paths and null bytes.
For schedule_task: change intervalHours description to include 'Positive integer: 0 for one-time, >0 for periodic (max 8760 hours = 1 year)' and add minValue/maxValue constraints. Add enum for delayMinutes or clarify range (0-43200 = 30 days).
For delegate_task: change targetModel to an enum property with values ['anthropic', 'openai', 'google', 'deepseek', 'ollama'] in the schema. Add description explaining: 'Which external LLM provider to route the task to. The result is saved to memory for future reference. Consider token cost and latency trade-offs. Default: google.'
Score history
Overall score trend
First recorded score · v2 rubric
42/100
Scored
Grade
Overall
Spec posture
Rubric
2026-09-23
F
42
2026-07-28+
v2
33/100
Read file from workspace
schedule_taskwritesource verified53/100
Schedules a task to be executed by the agent in the background. intervalHours: >0 for periodic, 0 for one-time. delayMinutes: >0 for delayed execution.
search_toolread onlysource verified72/100
Searches Wikipedia for up-to-date information about a topic. Use this when you need facts or data to answer a user's question.
schedule_task and delegate_task lack enum constraints for critical model-selection parameters (intervalHours, targetModel). These parameters accept free-form values, inviting LLM hallucination. schedule_task allows any integer for intervalHours; delegate_task allows any string for targetModel (description says 'MUST be one of: anthropic, openai, google, deepseek, ollama' but no schema enum). Violates pattern:constrained-input.
http_request tool lacks any visible input schema, description boundary, rate limiting guidance, or security model. Tool description states 'Make HTTP requests to configured domains' but does not explain which domains are allowed, whether credentials are injected server-side, request/response size limits, or timeout behavior. LLMs could abuse this tool to make arbitrary requests, trigger SSRF attacks, or leak sensitive data. Violates pattern:secret-injection and pattern:permission-gate.
No output schemas documented for any tool. Pattern:tool and mxe:strip-api-responses require documented return types so LLMs can plan downstream calls and extract relevant data. Users cannot understand what fields are returned, what data types to expect, or how to chain tools together.
Destructive tools (delete_file, write_file, edit_file) and sensitive operations (delegate_task, http_request) lack any confirmation, dry-run, or permission-gate mechanism. Pattern:confirmation-request and pattern:permission-gate require reversible or gated approaches for irreversible operations. An agent could accidentally delete entire workspaces or trigger arbitrary HTTP requests without safeguards.
File operation tools (read_file, write_file, edit_file, append_file, delete_file, list_files, find_files) lack path validation, sanitization, or traversal guards. LLMs can be tricked via prompt injection into passing paths like '../../../etc/passwd' or absolute paths outside the workspace. Pattern:tool-gateway requires input sanitization against path traversal.
No error handling guidance provided. When tools fail (file not found, network timeout, permission denied, HTTP error), the source code does not show how errors are communicated back to the LLM or what the LLM should do next. Pattern:recovery-guide requires error responses that tell the LLM what to do: 'File not found. Call list_files() to see available files.' Raw error codes or stack traces offer zero guidance.
Tool definitions are inferred from test mocks and repository implementations rather than explicit server registration. Evidence: tool schemas and descriptions appear in ToolsScreenLocalToolsTest.kt (a Compose UI test file) and scattered across data layer classes, NOT in a central tool registry or manifest.
delegate_task tool accepts a targetModel parameter with description 'The external model to use. MUST be one of: anthropic, openai, google, deepseek, ollama. Default is google.' However, no enum constraint exists in the schema, and no explanation of HOW the delegation works, whether credentials are injected, whether the task result is stored, or whether external costs are incurred. This violates pattern:constrained-input and pattern:secret-injection.
delegate_task
For http_request: add explicit input schema documenting (1) allowed domains/URL patterns (whitelist); (2) HTTP methods allowed (GET, POST, PUT, DELETE, etc.); (3) request body size limits; (4) response size limits; (5) timeout (e.g., 30s); (6) whether cookies/auth headers are auto-injected. Add description: 'Make authenticated HTTP requests to pre-configured domains only. Timeouts after 30 seconds. Max response size 10 MB. Returns {status, headers, body}. Credentials are injected server-side; never pass API keys in the URL.'
For search_tool: expand description to clarify format of returned results (is it a list of articles? JSON objects with title/url/snippet?). Document output schema showing what fields are returned.
Document output schemas for ALL tools. Example for read_file: '{"type": "object", "properties": {"content": {"type": "string", "description": "File contents"}, "encoding": {"type": "string", "enum": ["utf-8", "binary"], "description": "File encoding"}}, "required": ["content"]}'. Output schemas enable LLMs to chain tools and extract data reliably.
Add error guidance to tool descriptions. Example for read_file: 'If file not found, call list_files() to see available files in the directory. If permission denied, check workspace access. If file is too large (>100MB), use find_files() to search for a smaller alternative.'
Implement path validation: reject paths containing '..' (directory traversal), leading '/' (absolute paths), null bytes, or unusual characters. Return clear errors: 'Invalid file path: cannot traverse parent directories (../) or use absolute paths (/). Workspace is confined to the current directory.'
Add rate limiting guidance and cost warnings. For delegate_task, add: 'Each delegation incurs cloud API charges. Use sparingly. Typical cost: ~$0.01 per call depending on model and task complexity.' For http_request, add: 'Requests are rate-limited to 10 per minute per agent. Exceeded limits will be rejected with 429 status.'
For destructive tools (delete_file): add a confirmation flow. Pattern:confirmation-request suggests a two-step process: (1) delete_file(path) returns 'Ready to delete path/to/file.txt (42 bytes). Call confirm_delete_file(transaction_id) to proceed'; (2) agent calls confirm_delete_file(transaction_id). This prevents accidental data loss.
Add audit logging. Every tool call should log: who (agent ID), what (tool name, parameters), when (timestamp), and result (success/error). Include this in the description: 'All file operations are logged for compliance and debugging.'
For search_tool and delegate_task: add dependency hints in descriptions. Example for search_tool: 'Returns up to 20 results. If you need more context, chain multiple searches with refined keywords or different languages.' Example for delegate_task: 'Use this when local reasoning is insufficient. Consider cost and latency. Results are cached in memory for the session.'
Validate numeric parameter ranges upfront. For schedule_task (intervalHours, delayMinutes), add minValue/maxValue in schema and return clear errors: 'Invalid intervalHours: -5. Must be 0 (one-time) or a positive integer (hours). Max 8760 (1 year).'
Never expose secrets in tool outputs. Audit http_request response to ensure auth headers, tokens, and credentials are stripped before returning to the LLM. Document this in the tool description: 'Authorization headers and sensitive response fields are scrubbed before returning to the agent.'
Consolidate similar tools. read_file, list_files, and find_files overlap in purpose, consider merging into a single filesystem discovery tool or adding clear distinguishing descriptions for when to use each. Pattern:tool requires each tool to do exactly one thing; if three tools query files, clarify their distinct purposes.