An agent system built in Go with MCP (Model Context Protocol) support, providing tools for code analysis, execution, file management, and integration with language models
Buckley exposes 8 tools with dangerous capabilities (shell execution, file writes, patches) but lacks production-grade definition quality. Tool definitions are inferred from test files (cmd/buckley/acp_toolmap_test.go, cmd/buckley/oneshot_tool_failure_recovery_test.go, cmd/buckley/acp_tool_offer_test.go) rather than from explicit registration code. Descriptions are present but minimal (5-30 chars). Input schemas are sparse and incomplete, only 3 of 8 tools have documented parameters, and none have full JSON Schema with type validation. No output schemas are documented. No error handling guidance. No idempotent hints, destructive hints, or readonly hints. Parameter descriptions are missing for 5 tools. The server exposes 'run_shell' (DESTRUCTIVE) and 'apply_patch' (WRITE) without any permission gates, rate limiting, or audit hooks, critical security gaps.
Destructive tool 'run_shell' has zero input schema, no permission gates, no rate limiting, and no audit trail. Allows arbitrary shell command execution.
Make tool definitions explicit in the server's tool registry (not inferred from tests). Each tool must have explicit name, description, inputSchema, and outputSchema properties.
Expand tool descriptions to 100-250 characters. Current baseline is 194 chars average. Include WHAT the tool does, WHEN to use it, and WHAT it returns. Example: 'Execute arbitrary shell commands on the system. WARNING: destructive tool, verify commands before execution. Returns command exit code, stdout, stderr, and execution time.'
Add comprehensive input schemas with JSON Schema type definitions and descriptions for ALL parameters. Example for run_shell: { 'command': { 'type': 'string', 'description': 'Shell command to execute. Sanitized against injection; alphanumeric, pipes, redirects allowed. Max 2048 chars.' }, 'timeout_seconds': { 'type': 'integer', 'minimum': 1, 'maximum': 300, 'default': 30, 'description': 'Command execution timeout.' } }
Add explicit output schema for each tool. Example for read_file: { 'content': { 'type': 'string' }, 'line_count': { 'type': 'integer' }, 'file_size_bytes': { 'type': 'integer' } }
Add tool annotations (destructiveHint, readOnlyHint, idempotentHint) to every tool. run_shell and write_file must have destructiveHint=true. read_file must have readOnlyHint=true.
Implement permission gates. Every destructive or sensitive tool (run_shell, write_file, apply_patch, edit_file_terminal, mark_conflict_resolved) must verify caller permissions before execution. Return clear permission denied errors.
Score history
Overall score trend
First recorded score · v2 rubric
34/100
Scored
Grade
Overall
Spec posture
Rubric
2026-09-23
F
34
2026-07-28+
v2
5 of 8 tools have no documented input parameters or schemas. LLM cannot determine what arguments are valid or required.
Tool names contain ambiguous verbs. 'edit_file_terminal' does not clearly indicate what editing mode or limitations apply. 'apply_patch' lacks context on what patch format or validation rules exist.
Parameter descriptions sparse or missing. 'command' param in run_shell has minimal guidance on format, validation, or safety constraints.
run_shellapply_patchwrite_filesubmit_artifact
Add error handling guidance to all tools. Include recovery hints: 'File not found. Use list_files() to find available files.' Categorize as retryable, user-fixable, or fatal.
Add parameter validation with actionable error messages. Example: 'Invalid status: got "pending", must be one of: open, in_progress, resolved, closed'
For file-writing tools, implement path traversal protection. Validate that paths do not escape allowed directories.
Add rate limiting to prevent runaway agents from spawning unlimited shell commands or file writes.
Add audit logging. Log who called what tool, with which parameters (sanitized for secrets), at what time, and what happened.
Document parameter dependencies and constraints in descriptions. Example: 'line_numbers boolean param affects output format; when true, each line is prefixed with line number.'
For tools that return large results (e.g., read_file on large files), add pagination/truncation guidance in the description and output schema.