Loading ToolBench

Reading the current corpus…

siteminder-policy-mcp MCP server · F · ToolBench

siteminder-policy-mcp

github.com/tl-brcm/siteminder-policy-mcp

MCP server for interacting with SiteMinder policy objects via FastMCP. Provides tools to list, search, and manage SiteMinder objects with support for OIDC, JWT, and static token authentication.

<=2025-11-25STREAMABLE_HTTP40 tools

49/100

Definition Quality44

Protocol Readiness62

SupportabilityNot measured-

Spec posture

Inferred effective spec <=2025-11-25·current baseline 2026-07-28

Static source inference · medium confidence · detected: Logging, Dynamic Client Registration

Deprecated protocol patterns detected

Summary

This server presents 40 tools across 8 SiteMinder object types (SmAgentConfig, SmAgent, SmPolicyDomain, SmRule, SmResponse, SmUser, SmGroup, SmRealm, SmResource, SmPolicy). Each type has 5 tools: list_*_summary, search_*, get_*_detail, create_*. While the tool names follow a clear verb_noun pattern and descriptions are present, critical deficiencies emerge: (1) Input schemas for all 40 tools lack type constraints and enums. The 'detail' parameter for all create_* tools is typed as 'string|object' with minimal guidance, LLMs cannot infer what fields are required or valid. (2) Output schemas are completely undocumented, no indication of what fields list_*, search_*, or get_detail_* return. (3) Parameter descriptions are minimal and generic. The 'filter_expression' param for all search tools shows the same boilerplate examples (Name contains 'login', Desc != null, IsEnabled = true, Level > 500) repeated across all 8 object types without object-specific guidance. (4) The create_* tools provide no schema for the 'detail' object, no required fields, no valid attributes, forcing agents to guess or hallucinate field names. (5) No error handling guidance (e.g., what happens if filter syntax is invalid, or required fields are missing from create detail). (6) The server does implement proper naming conventions (list_, search_, get_, create_) and descriptions exist for all tools, which prevents a lower score, but the lack of actionable input/output schemas and parameter specificity is a major gap in production readiness.

Tools (40)

create_smagentwriteauth50/100

Create a new SmAgent object with the provided details.

create_smagentconfigwriteauth50/100

Create a new SmAgentConfig object with the provided details.

create_smgroupwriteauth50/100

Create a new SmGroup object with the provided details.

create_smpolicywriteauth50/100

Create a new SmPolicy object with the provided details.

create_smpolicydomainwriteauth50/100

Create a new SmPolicyDomain object with the provided details.

create_smrealmwriteauth50/100

Create a new SmRealm object with the provided details.

create_smresourcewriteauth50/100

Create a new SmResource object with the provided details.

create_smresponsewriteauth50/100

Issues (6)

CRITICALtool ↗

All 40 tools lack output schema documentation. list_*, search_*, and get_detail_* tools do not declare what fields are returned (e.g., does list_smagentconfig_summary return [name, id, status] or [name, id, status, description, enabled, type]?). Without output schemas, LLMs cannot extract chaining IDs for downstream calls or plan multi-step workflows.

list_smagentconfig_summarysearch_smagentconfigget_smagentconfig_detaillist_smagent_summarysearch_smagentget_smagent_detaillist_smpolicydomain_summarysearch_smpolicydomainget_smpolicydomain_detaillist_smrule_summarysearch_smruleget_smrule_detaillist_smresponse_summarysearch_smresponseget_smresponse_detaillist_smuser_summarysearch_smuserget_smuser_detaillist_smgroup_summarysearch_smgroupget_smgroup_detaillist_smrealm_summarysearch_smrealmget_smrealm_detaillist_smresource_summarysearch_smresourceget_smresource_detail

Recommendations

For every search_* tool, formally define the filter_expression grammar. Example: 'Supported operators: contains, =, !=, >, <, >=, <=. Boolean logic: AND, OR. Attributes are case-sensitive. Example: Name contains "test" AND IsEnabled = true'. Add validation examples showing what happens on invalid syntax.
For every list_* and search_* tool, add optional 'limit' (default 20, max 100) and 'offset' (default 0) parameters. Return a total_count field and indicate whether more results are available. Document the default behavior in descriptions.
For every create_* tool, replace the vague 'detail' parameter with explicit, typed fields. Example: create_smagent should have 'name' (required, string), 'description' (optional, string), 'type' (required, enum: [ConfigAgent, SmAgent]), 'enabled' (optional, boolean, default true). Provide a JSON example in the description.
For every get_*_detail tool, add an explicit output schema in the description. Example: 'Returns an object with fields: id (string), name (string), description (string), type (string), enabled (boolean), created_at (ISO 8601), last_modified (ISO 8601), href (string for chaining).'
For every list_*_summary tool, clarify what 'summary' means. Example: 'Returns a list of all SmAgent objects with core fields only: id, name, type, enabled. For complete details, call get_smagent_detail(id). Results are sorted by name, limited to 50 items. Call with offset parameter to paginate.'
Add error handling documentation to search_* and create_* tools. Example: 'On invalid filter syntax, returns a 400 error with the problematic clause and valid operator list. On missing required fields in create detail, returns a 422 error listing which fields are required.' This enables LLMs to self-correct.

Spec posture evidence

Inferred effective spec: <=2025-11-25.

Relies on Logging (deprecated) - log to stderr or use OpenTelemetry

Score history

Overall score trend

↑ 49 points across a rubric change (v1 → v2)

49/100

Scored	Grade	Overall	Spec posture	Rubric
2026-09-22	F	49	<=2025-11-25	v2
2026-03-09	F	0	-	v1