Loading ToolBench
FastMCP server exposing SSW Rules search to AI agents with semantic search across SSW Rules knowledge base
The SSW Rules MCP server provides 5 well-intentioned read-only tools with reasonable descriptions and basic input schemas. However, there are consistent gaps in parameter documentation, schema completeness, and error handling guidance. Tool names are clear and action-oriented (search_rules, get_rule, list_categories), and descriptions explain the general purpose. The main quality issues are: (1) Parameters lack descriptions in several cases, 'limit' and 'days' parameters have descriptions, but the descriptions are minimal and don't explain constraints; (2) Output schemas are completely undocumented, the code returns dict objects with various fields but there's no schema specification for what fields to expect; (3) Error handling is minimal and doesn't guide recovery, error responses are plain strings without actionable next steps; (4) No pagination metadata is documented (e.g., total count, next_cursor) despite returning lists. The tools are functional and the naming is good, but the implementation is at the 'fair' level of the rubric, with documentation that would require agents to reverse-engineer expected output structures.
Get all rules in a specific category. Works for both top-level categories (returns all rules across subcategories) and subcategories (returns just that subcategory's rules).
Get recently updated SSW Rules.
Get the full content of a specific SSW Rule by its URI slug. Returns the rule's title, URL, metadata, and full cleaned markdown content. The content has JSX components stripped for clean reading.
List all SSW Rules categories in the hierarchy. Returns top-level categories (e.g. Software Engineering, Communication) with their subcategories and rule counts.
Semantic search across all SSW Rules (https://ssw.com.au/rules). Returns matching rules with title, URL, description, and relevance score. Requires Qdrant to be running and indexed (run 'ssw-rules index' first). Falls back to text search if Qdrant is unavailable.
Output schemas completely undocumented. Tools return dict objects (e.g., search_rules returns list[dict], get_rule returns dict) with no schema specification. LLMs cannot plan downstream operations or extract specific fields without trial and error. Pattern 'response-shaper' requires documenting return type structure.
Parameter descriptions are minimal and lack constraint information. 'limit' (int, default 10) has no documentation of valid range (min/max bounds). 'days' (int, default 30) lacks information about valid range or what happens with invalid values.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 45 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 44 | - | v1 |
No input schema visible in source for any tool. The @mcp.tool decorator appears to use FastMCP's automatic schema inference from Python type hints (query: str, limit: int = 10). While this generates valid JSON Schema, the code does not show explicit schema definitions with proper descriptions for each parameter. However, FastMCP does generate schemas from docstrings, so partial credit (35-50 range) is applied, but confidence is low without seeing the generated output.
Error handling lacks recovery guidance. search_rules and get_rule return plain error dicts like {'error': 'SSW.Rules.Content not found. Run ssw-rules index to clone and index.'} and {'error': 'Rule not found: {uri}'}. Per pattern 'recovery-guide', errors should tell LLMs what to do next. Current errors hint at the fix (run ssw-rules index) but don't offer tool-based alternatives or suggest related tools to call.
No pagination or limits enforcement documented. search_rules and get_recent_rules return lists but do not document total count, next_cursor, or explicit cap on results. No mention of what happens if results exceed limits, or whether responses are capped.
list_categories has no input schema documented. The tool accepts no parameters (empty dict) but the description and input handling are not explicit in the code snippet. This makes it harder for LLMs to know when to call it vs. discover_categories or similar.