The server defines 16 tools with varying quality. All tools have names starting with action verbs (get_, search_, execute_, trigger_, clear_) which is positive. Descriptions are present but generic, most lack depth about when to use the tool, what it modifies, or dependencies. Parameter schemas are well-structured with type declarations and descriptions, but several tools lack output schema documentation and error recovery guidance. The tool set spans read-only operations (search_products, get_product, etc.) and destructive operations (execute_groovy, import_impex, trigger_cronjob) without clear separation or confirmation patterns. Security-sensitive tools like execute_groovy and import_impex accept raw code/data without validation hints or permission checks. Overall, this is a functional foundation that falls into the 'Fair' range, present but incomplete.
Destructive and irreversible tools (execute_groovy, import_impex, trigger_catalog_sync) lack confirmation or dry-run patterns. No tool description warns the LLM about side effects.
No error handling or recovery guidance. Tools like execute_groovy and flexible_search can fail silently or return verbose error stacks that LLMs cannot act on.
Recommendations
Add output schemas to all tool definitions. Example: search_products should document that it returns {results: [{code, name, price}], totalResults: number, nextPage: number|null}. This enables LLMs to chain calls and extract required IDs.
Add tool annotations (readOnlyHint, destructiveHint, idempotentHint) to each tool definition. For example, mark execute_groovy and import_impex as destructive, and search_products as readOnly. This lets clients and LLMs understand tool risk without parsing descriptions.
Implement dry-run or confirmation patterns for execute_groovy and import_impex. Example: add a 'dryRun' boolean parameter to execute_groovy that executes the script in a sandbox and returns the result without committing. Or split into execute_groovy_preview and execute_groovy_commit.
Enhance descriptions for execute_groovy, flexible_search, and import_impex. State the language/syntax expected (Groovy, FlexibleSearch DSL, ImpEx format), the maximum query complexity, example usage, and common failure modes. Example: 'Execute a Groovy script in the Hybris scripting console. Syntax: standard Groovy with access to Spring beans. Max 30KB per script. Use dryRun=true to preview changes before committing. Common errors: NullPointerException (missing bean), ParseException (invalid syntax).'
Add result limits and pagination guidance. For search_products, set a max pageSize (e.g., 100) and always return totalResults and a next_cursor if more results exist. For flexible_search, cap maxCount at 1000 and warn if results are truncated.
Score history
Overall score trend
↑ 8 points across a rubric change (v1 → v2)
51/100
Scored
Grade
Overall
Spec posture
Rubric
2026-09-22
D
51
2026-07-28+
v2
2026-03-09
F
43
-
v1
List all cron jobs and their status
get_orderread onlyauthsource verified80/100
Get details of a specific order
get_ordersread onlyauthsource verified77/100
Get orders for a specific user
get_productread onlyauthsource verified82/100
Get detailed information about a specific product by its code
get_system_inforead onlyauthsource verified73/100
Get Hybris system information and health status
health_checkread onlyauthsource verified78/100
Check if the Hybris instance is healthy and reachable
import_impexwriteauthsource verified65/100
Import data using ImpEx format
search_productsread onlyauthsource verified82/100
Search for products in the Hybris catalog using a query string
Tool descriptions lack context about when to use each tool and dependencies. 'Execute a FlexibleSearch query' does not explain the syntax, whether it's read-only, or how results map to other operations.
execute_groovy and import_impex accept boolean 'commit' and raw 'impexContent' without defaults that prevent accidental data loss. Default commit=true or validate before execution.
flexible_search and export_impex accept raw query strings without injection prevention. No mention of SQL escaping, query syntax validation, or result limits.
search_products accepts pageSize and currentPage but no guidance on max page size or total result count. LLMs can request 1000+ results per page, causing timeout or resource exhaustion.
Credentials (HYBRIS_BASE_URL, HYBRIS_USERNAME, HYBRIS_PASSWORD) are loaded from environment variables at startup, which is correct. However, no audit logging of which tools were called or which user/agent made the calls.
all
Add validation and injection prevention hints to flexible_search and import_impex. Example for flexible_search: 'All user input in the query must be parameterized using {param} placeholders to prevent injection attacks.' For import_impex: 'Validate ImpEx syntax before calling, malformed imports may corrupt the catalog.'
Add error recovery guidance to tool descriptions. Example for execute_groovy: 'If you get a ParseException, check the Groovy syntax. If you get NullPointerException, the bean name may be incorrect, try System.out.println(applicationContext.getBeanDefinitionNames()) first.'
Document relationships between tools. For example, trigger_cronjob should note that you can call get_cronjobs first to find the cronJobCode, and trigger_catalog_sync should note that catalogId and versions must exist (call get_categories or get_system_info first to discover them).
Add rate limiting or concurrency warnings. Example: 'This tool locks the Hybris scripting console during execution. Concurrent calls will queue. Expect 5-30 second latencies under load.'
Implement structured error responses. Instead of returning raw exception messages, return {error: 'invalid_query', detail: 'FlexibleSearch syntax error at line 2: missing FROM clause', suggestion: 'Example: SELECT {pk}, {code} FROM {Product}'}. This guides LLM recovery.