Loading ToolBench
A package containing a Python class and associated methods to create a MCP server script from published SAS Micro Analytic Service (MAS) modules.
This server has 23 tools with significant quality gaps. Most tools lack proper descriptions (many under 20 chars or generic). Input schemas are present but sparse, many parameters lack type information or descriptions. Tool naming is inconsistent: some follow verb_noun (get_initial_count, list_modules) but others are vague (filter_modules, build_fastmcp_tool_def). No output schemas documented. Error handling is minimal, most tools return generic exceptions without recovery guidance. The server generates MCP server scripts but does not itself follow MCP best practices for tool definition.
Given a module specification, this tool builds a FastMCP server definition for HTTP transport.
Build starter commands for an HTTP FastMCP server.
Build footer (starting service) commands for an HTTP MCP server.
Build starter commands for a RAM FastMCP server.
Given a module specification, this tool builds a FastMCP server definition for RAM.
Build starter commands for an stdio FastMCP server.
Build footer (starting service) commands for an stdio server.
Vague tool names lack action verbs. 'filter_module_type', 'build_fastmcp_tool_def', 'build_http_fastmcp_header' do not clearly signal what happens when called. LLMs cannot infer intent from names alone.
Most tool descriptions are under 20 characters or generic ('Build starter commands for...'). Descriptions do not explain WHEN to use the tool or what it returns.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields (e.g., module_id, endpoint URL) without knowing response structure.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 45 | 2026-07-28+ | v2 |
Check if the access token is valid by making a test API call to the Viya environment.
Filter the selected modules based on the specified module types (e.g., "Model", "Decision").
Filter the selected modules based on custom criteria provided as a dictionary.
Retrieve and enhance module properties from the SAS Viya environment, classifying modules as Model or Decision type.
Retrieve all MAS modules from the SAS Viya environment.
Retrieve the count of MAS modules available in the SAS Viya environment.
Retrieve module contracts (input/output specifications) from module endpoints.
Extract module URLs and metadata from module specifications.
List the names and types of the selected modules.
Load environment variables from a .env file.
Generate a complete HTTP FastMCP server script from module contracts.
Generate a complete RAM FastMCP server script from module contracts.
Generate a complete stdio FastMCP server script from module contracts.
Given a module contract, this method generates a Python script that serves the module as a FastMCP tool using standard input/output.
Given a module contract, this method generates a Python script that serves the module as a FastMCP tool using standard input/output.
Given an endpoint, returns a tool name by extracting the relevant part from the URL path.
Error handling is minimal. Tools catch exceptions and return generic {'error': str(e)} without recovery guidance. LLMs cannot determine if errors are retryable or what to do next.
Credentials (viya_host, viya_access_token) exposed as tool parameters in load_env_variables and get_initial_count. Per pattern:secret-injection, credentials must use server-side injection via environment variables, not tool parameters.