Loading ToolBench
MCP server for NYC Charter, Administrative Code, and Rules of the City of New York
Five read-only tools with complete JSON schemas and descriptions. All tools have proper enum constraints and parameter descriptions. Naming follows verb_noun pattern (search, get_*, list_*). Descriptions are substantive (150-250 chars) and include legal disclaimers. However, output schemas are not documented, LLMs cannot predict response structure for chaining. Error handling is absent; no guidance on recovery or disambiguation. No tool annotations (readOnlyHint present in metadata but not in schema). Composition is sound: each tool has one responsibility, and tools chain naturally (search → get_section → get_title).
Retrieve a specific section by its citation (e.g. '§ 259', 'Section 259', '11-602.1', 'Chapter 11'). Input is normalized (with or without '§', any case). Pass 'corpus' to disambiguate when the same citation exists in multiple documents (e.g. charter 'Chapter 3' vs a rules chapter); if multiple sections still match, a disambiguation list is returned. For informational purposes only. Not legal advice. Verify against official sources at codelibrary.amlegal.com before relying on any result.
Retrieve chapter/title records matching an identifier (whole-token match: 'Chapter 1' does not match 'Chapter 10'). Note: the index is flat — deep hierarchy (every section nested within a title) is not indexed, so this returns matching chapter/title-level records, not full title contents. For informational purposes only. Not legal advice. Verify against official sources at codelibrary.amlegal.com before relying on any result.
Return the currency date for each document — how current the Charter, Administrative Code, and Rules are. Each corpus updates on its own schedule. Always call this tool before answering legal questions so responses are grounded in the correct version of the law. For informational purposes only. Not legal advice. Verify against official sources at codelibrary.amlegal.com before relying on any result.
List the top-level chapters or titles of a document. For informational purposes only. Not legal advice. Verify against official sources at codelibrary.amlegal.com before relying on any result.
Output schemas not documented. LLMs cannot predict response structure (fields, types, nesting) for downstream tool chaining or data extraction. search() returns what? Array of objects with which fields?
No error handling guidance. When get_section() returns 'disambiguation list' (mentioned in description), what is the structure? How should the LLM handle ambiguous citations? No recovery path documented.
search() limit parameter lacks bounds. Description says 'default 10, max 50' but no minItems/maxItems in schema. LLMs may pass invalid values; validation happens server-side only.
Tool annotations missing from inputSchema. Risk metadata (READ_ONLY) is in tool metadata but not in schema as readOnlyHint. Spec 2026-07-28 expects annotations in schema for LLM visibility.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 64 | 2026-07-28+ | v2 |
Search across the NYC Charter, Administrative Code, and Rules of the City of New York by keyword or phrase. Results are relevance-ranked: heading matches rank above citation matches, which rank above body-text matches, and whole-word matches rank above substring matches. For informational purposes only. Not legal advice. Verify against official sources at codelibrary.amlegal.com before relying on any result.
No pagination support. search() accepts limit but no offset/cursor or total_count in response. Large result sets will blow context window; no guidance on how to iterate.