ProbeCode defines 11 file operation and code parsing tools via fastmcp. Tool naming follows verb-noun convention and is generally clear (create_folder, list_directory, delete_item, read_file, write_file). However, the definitions suffer from multiple critical gaps: (1) Most tool descriptions are very brief (10-25 characters), far below the 50-200 char baseline for LLM-optimized descriptions; (2) Parameter descriptions are minimal and lack detail about formats, constraints, or validation rules; (3) Output schemas are not visible in the provided source code, no structured response definitions are shown; (4) Error handling guidance is absent, no recovery hints, classification, or actionable messages; (5) No evidence of input validation, permission gating, or security controls. The tools appear to be defined but not richly annotated for agent use. Tool names are solid, but descriptions and schemas are severely underdeveloped.
Tool descriptions are critically short (10 - 25 characters), far below the 50 - 200 character baseline required for LLM reasoning. Descriptions like 'Create a new folder at the specified path.' lack context for when to use the tool and what it returns.
Parameter descriptions are missing or trivial. The 'path' parameter appears in multiple tools but lacks details about format (absolute vs relative), allowed characters, max length, or what happens if the path is invalid. LLMs cannot infer these constraints.
Expand every tool description to 50 - 200 characters, explaining: (1) what the tool does, (2) when to use it vs similar tools, (3) what it returns, (4) key prerequisites or side effects. Example: 'Delete a file or directory. Removes the item permanently; use with caution as this is irreversible. Fails if path does not exist or is not accessible. Returns success status and error details if applicable.'
Add detailed descriptions to every parameter. Include format constraints (absolute/relative path), acceptable characters, min/max lengths, and validation rules. Example for 'path': 'Absolute or relative file/directory path (Unix-style separators allowed). Max 4096 chars. Must not contain null bytes or symlink cycles. Path resolution is relative to current working directory.'
Document output schemas for all tools. Specify field names, types, and meanings. Example for list_directory: 'Returns {"items": [{"name": string, "type": "file"|"dir", "size": int (bytes), "modified": ISO 8601 timestamp}], "total_count": int, "truncated": bool}. If truncated=true, use list_directory with offset/limit to fetch remaining items.'
Add error classification and recovery guidance to destructive operations. Example for delete_item: 'On FileNotFoundError, suggest checking the path or listing the parent directory. On PermissionError, explain that the file is protected and ask the user for admin access. On DirectoryNotEmptyError, offer to recursively delete with explicit confirmation.'
Implement input validation on all path parameters. Reject paths containing '..', leading '/', or suspicious patterns. Return detailed errors: 'Path "/etc/passwd" is outside the allowed root directory. Paths must be relative and within /home/user/project.'
Score history
Overall score trend
↓ 1 points across a rubric change (v1 → v2)
40/100
Scored
Grade
Overall
Spec posture
Rubric
2026-09-22
F
40
2026-07-28+
v2
2026-03-09
F
41
-
v1
33/100
Read all file content (implementation incomplete).
No output schemas are documented in the source code. LLMs cannot plan downstream tool calls or extract required fields (e.g., what does list_directory return, file names only, or full stat info?) without knowing the response structure.
Destructive operations (delete_item, write_file) have no confirmation or dry-run capability, no error classification, and no recovery guidance. If delete_item fails, the LLM has no idea whether to retry, ask the user, or abandon the operation.
read_all_file_content has description '(implementation incomplete)' and empty input schema. This tool is non-functional and should be removed or properly implemented before deployment.
No evidence of input validation or security controls. Tools accepting path parameters are vulnerable to path traversal attacks (e.g., '../../../etc/passwd'). No sanitization hints in descriptions or error handling guidance.
Tool descriptions do not indicate what each tool returns. E.g., read_file says 'Read the contents of a file' but does not specify the format (string, bytes, chunks) or what happens if the file is binary. LLMs will guess incorrectly.
Ambiguous parameter naming: rename_item and move_file both take 'src' and 'dest' parameters. The distinction is unclear, rename operates on the same path level, move operates across directories. Descriptions should clarify this to prevent misuse.
No pagination or result limiting on list_directory. If a directory contains thousands of files, the response could blow the context window. The tool should return a subset and offer pagination via offset/limit parameters.
list_directory
Add a dry-run or confirmation mode to destructive tools (delete_item, write_file). Use a 'confirm' boolean parameter: set to false for dry-run (returns what would happen), true to execute. This prevents accidental data loss from agent mistakes.
Add pagination to list_directory. Accept optional 'offset' (default 0) and 'limit' (default 50, max 1000) parameters. Return total_count and truncated flag so agents know when to fetch more.
Split rename_item and move_file into separate operations with clearer descriptions. rename_item: 'Rename a file or directory in place (same parent directory).' move_file: 'Move a file or directory to a new location (may change parent directory).'
Remove or complete read_all_file_content. The current stub with description '(implementation incomplete)' signals unreliability. Either implement it fully or delete it.
Add audit logging to all tools. Log caller identity, timestamp, parameters (sanitized), and outcome. Include in tool descriptions: 'This operation is logged for security and audit purposes.'
Document timeout and size limits. Example: 'Reads up to 10 MB. Timeouts at 30 seconds. Larger files require streaming via separate tool or batched reads.'
For write_file, clarify overwrite behavior. Add 'overwrite' parameter (default false). Describe: 'If false, fails if file exists (safe default). If true, replaces entire file content.'