Loading ToolBench
An AI agent framework with built-in tools for file operations, web access, code execution, database queries, and external integrations via MCP servers
Pepe exhibits severe definition quality gaps across nearly all 55 tools. Most tools (>80%) have empty or near-empty input schemas with no properties defined, violating the critical rule that parameters must have type definitions and descriptions. Descriptions are present but generic and often under 50 chars, providing minimal LLM guidance. Tool names lack consistent verb-noun structure (e.g., 'skill', 'docs', 'board' are nouns, not actions). No output schemas are documented. Error handling is absent from visible code. Only 5 tools (read_file, write_file, edit_file, move_file, fetch_url, web_search, db_query, send_file, send_to_agent, switch_agent, rename_agent, config_get, config_set, enable_tool, session_search, memory_search) have any parameter structure; the remaining 40 tools expose empty schemas, making them unusable by LLMs without guessing. This is a foundational failure in tool design.
Ask the user a question and wait for their response
Execute a bash command on the local machine
Access or manage a board
Control a headless browser to navigate and interact with web pages
Create or manage a commitment
Get a configuration value
Set a configuration value
Execute a query against a database
Delegate a task to another agent
40 of 55 tools (73%) have empty input schemas with no properties defined. LLMs cannot infer parameter names, types, or constraints, making these tools unusable without hallucination.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 37 | 2026-07-28+ | v2 |
Access documentation
Run diagnostics on the system
Edit a file by replacing a section of text
Add a tool to your own allowlist so you can start using it - e.g. right after installing a plugin under plugins/. Pass the tool's `name` (it must already exist as a built-in or a plugin). Takes effect on your next message.
End the current conversation session
Fetch the contents of a URL
Set or manage a goal for the agent
Generate insights from data
Make predictions using insights
Inspect the results of a graph execution
Create or manage an invoice
List the contents of a directory
Manage agent configuration and settings
Manage a communication channel
Manage database connections and configuration
Manage a workflow graph
Manage MCP (Model Context Protocol) servers
Manage Pepe system configuration
Manage plugins
Manage skills
Manage authentication tokens
Search through agent memory
Move or rename a file
Create or manage a plan
Read the contents of a file
Rename an agent
Review content or decisions
Execute code in a specified language
Execute a workflow graph
Run a script file with a specified interpreter
Scan for available skills
Schedule a task to run at a specific time
Send a file to a user or channel
Send a presentation or slideshow
Send a message to another agent
Search through session history
Set a routing rule for messages
Access or manage a skill
Curate and organize skills
Manage skill configuration
Switch to a different agent
Access Telegram integration
Create or manage a Telegram poll
Monitor a file or resource for changes
Search the web for information
Write content to a file
Tool names lack consistent verb-noun structure. 'skill', 'docs', 'board', 'goal', 'plan', 'review', 'watch', 'commitment' are nouns, not action verbs. LLMs cannot infer intent from names alone.
No output schemas documented for any tool. LLMs cannot plan downstream tool calls or extract required fields (e.g., IDs for chaining). Violates response-shaper pattern.
Destructive tools (bash, run_script, run_code, write_file, edit_file, move_file, db_query, send_file, invoice, end_session, manage_* tools) lack dry-run or confirmation patterns. No recovery guidance in error handling.
No visible error handling, recovery guidance, or actionable error messages in tool definitions. LLMs cannot self-correct or know whether to retry.