A persistent IPython workspace for data analysis and log processing via MCP with OAuth integration and external MCP server support.
Sherlog MCP exhibits significant definition quality gaps across most dimensions. While 7 tools are explicitly registered with basic descriptions and input schemas, the descriptions are generic and often lack actionable detail. Tool naming follows conventions (verb-first) but some names are vague. Parameter schemas exist but descriptions are sparse or missing constraint information. Critical issues: (1) No error handling guidance documented in any tool, no recovery paths, retryable vs fatal classification, or actionable error messages. (2) Descriptions average ~150 chars but lack WHEN/WHY context that LLMs need for proper tool selection. (3) No output schemas documented, LLMs cannot predict response structure. (4) Several tools expose mutable state (execute_python_code, call_cli) without confirmation patterns or dry-run support. (5) The 'save_as' parameter in 3 tools lacks validation/constraint documentation, LLMs could pass invalid Python identifiers. Most tools hover in the 35 - 55 range; none exceed 60 due to missing output documentation and sparse error guidance.
Execute a CLI command using IPython's ! syntax and save the result. Args: command: The CLI command to execute save_as: Variable name to save the result in IPython namespace Returns: dict: Response stating that if the command was successful and saved to the IPython namespace. You can inspect the result `save_as` and work with it as you would with any other variable. For --help commands, the actual output is also included in the response for immediate viewing.
Executes a given string of Python code in the underlying IPython interactive shell. Executes Python code in a persistent IPython session where all variables from previous tool calls are available. Use list_dataframes() to see available data or list_shell_variables() for all variables. This tool allows for direct execution of arbitrary Python code, including defining variables, calling functions, or running any valid Python statements. The code is run in the same IPython shell instance used by other tools, allowing for state sharing (variables defined in one call can be used in subsequent calls).
Retrieves IPython shell execution history within a specified range or for all history.
Provides detailed information about an object in the IPython shell by its name. Uses IPython's object inspector.
No output schemas documented for any tool. LLMs cannot predict response structure, making downstream chaining unreliable. Tools like search_pypi and execute_python_code return complex objects (lists, dicts) with no field documentation.
No error handling guidance for any tool. Descriptions omit failure modes, recovery paths, and error categorization (retryable vs user-fixable vs fatal). execute_python_code could raise SyntaxError, NameError, or timeout, no guidance on how agent should respond.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 52 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 40 | - | v1 |
Lists variable names in the current IPython user namespace. Tries to exclude common IPython internal variables (e.g., 'In', 'Out', 'exit', 'quit', 'get_ipython') and variables starting with an underscore unless they are common history accessors. Special underscore variables like '_', '__', '___' (output history) and '_i', '_ii', '_iii' (input history) are included if present.
Query the status of an APT package (installed, upgradable, available). Args: package: Name of the APT package to query save_as: Variable name to save results in IPython shell Returns: dict: Response with package status information
Search PyPI for Python packages. Args: query: Search query for package names or descriptions save_as: Variable name to save results in IPython shell Returns: dict: Response with search results
Destructive operations (call_cli, execute_python_code) lack confirmation or dry-run support. An LLM could irreversibly delete files or corrupt data without confirmation. No confirmation request pattern implemented.
'save_as' parameter (used in 3 tools: call_cli, search_pypi, query_apt_package_status) lacks validation/constraint documentation. No mention of Python identifier rules, length limits, or forbidden names. LLMs could pass 'import', '123var', or other invalid identifiers.
Descriptions lack actionable WHEN/WHY context. Example: 'Search PyPI for Python packages' does not explain when to use this vs execute_python_code's import capability, what search scope is, or how results are ranked. Generic descriptions force LLMs to guess.
No pagination or result limiting documented. get_shell_history could return megabytes of history with no indication of max results, pagination cursors, or streaming. execute_python_code output could be arbitrarily large.
Stateful execution without isolation. execute_python_code modifies a shared IPython namespace, and all tools see the same state. No indication of scope isolation, transaction rollback, or multi-agent conflict handling. One agent's pollution affects another.
Parameter descriptions are sparse. Example: 'detail_level' in inspect_shell_object has a description but no examples of what 0, 1, 2 actually show. 'range_str' in get_shell_history says 'e.g. 1-10' but does not specify format, inclusivity, or max range.
No security/permission documentation. Tools access filesystem, execute code, and query package managers, no mention of what permissions are required, whether tools can be restricted per agent, or what audit trails exist.