Loading ToolBench
Local AI analyst for SQLite database queries with MCP integration. Provides SQL query execution against a payments database through an MCP server, with Streamlit UI for interactive data analysis.
PocketAnalyst has a single tool (query_database) with a basic but incomplete implementation. The tool name follows verb_noun convention, but the description lacks depth. The input schema is visible and typed, but lacks parameter-level description detail. No output schema is documented. The tool performs a critical database operation (SQL execution) but error handling is minimal and not designed to guide LLM recovery. The server provides a resource (sqlite://schema) which is good for context, but the tool itself is underspecified for production use. This is a simple analytical tool, not a full-featured integration.
Executes a read-only SQL query against the database.
Missing output schema documentation. The tool returns JSON but no schema is documented for the LLM to parse results. LLMs cannot infer the structure of result_data array or handle pagination/truncation signals.
Parameter 'sql' lacks a detailed description in the MCP schema. While the docstring says 'The SELECT statement to execute', the MCP tool definition shows only a bare type:string with description 'The SELECT statement to execute.', insufficient for LLM safety and correctness. No mention of constraints (SELECT-only enforcement is code-side), format expectations, or failure modes.
Error handling is minimal and not LLM-recoverable. SQL errors are returned as plain strings ('SQL Error: ...') without guidance on what to do next. Security enforcement (SELECT-only check) returns a generic error. No classification of retryable vs user-fixable errors.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 45 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | 2024-11-05+ | v1 |
Result truncation logic (limit to 100 rows) is not idempotent or predictable. An LLM will not know in advance whether results will be truncated, and repeated calls with the same query may hit truncation at different boundaries if results grow. No pagination support (limit/offset parameters).
Tool does not return query execution context needed for chaining. If the LLM generates a query and gets back only a JSON array, it has no way to know the query that was executed (for logging/debugging) or the execution time. Useful metadata is stripped.
The security guard ('SELECT-only' check) is implemented in the tool, not declared as a constraint in the schema. LLMs cannot see this restriction in the parameter definition and may waste reasoning cycles trying non-SELECT queries or craft queries with deceptive syntax.