Voice-powered NYC mortgage bias explorer using CFPB HMDA federal lending data. Exposes tools to query mortgage approval/denial patterns by race, location, and lender.
Server has 5 tools with basic verb-noun naming (get_*, identify_*). All tools have descriptions (10-200 chars, within baseline range). However, input schemas are present but lack critical detail: parameter types are declared but descriptions are minimal or generic. Output schemas are not documented anywhere in the source. Error handling is absent, no recovery guidance, no categorization, no actionable error messages. Tool composition is reasonable (each does one thing), but the lack of schema documentation and error patterns significantly limits LLM usability. STDIO transport caps protocol readiness at 50, further limiting production viability.
Get mortgage denial rates for ALL 5 NYC boroughs simultaneously. Also returns zip_map for choropleth visualization.
Get mortgage originated vs denied counts and denial rate for a NYC borough or neighborhood.
Get a citywide NYC mortgage lending summary across all boroughs.
Get mortgage denial rates broken down by every race group for a borough. Reveals racial disparity in lending — key for bias analysis.
Identify which NYC borough a location or neighborhood belongs to.
No output schemas documented. Tools return data but LLMs cannot plan downstream calls or extract fields without seeing response structure. Baseline: 100% of A+ tools document return types.
Parameter descriptions are minimal or generic. 'year' is described as 'HMDA data year (default 2022)' but lacks range context (2018-2022 is mentioned in one tool only). 'race' parameter lacks enum constraint, should list valid values explicitly instead of relying on description text.
No error handling or recovery guidance. If HMDA API fails, times out, or returns no data, tools provide no actionable error messages. LLMs cannot self-correct or retry intelligently.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | 2026-07-28+ | v2 |
Tool descriptions lack WHEN-to-use guidance. 'Get mortgage denial rates for ALL 5 NYC boroughs' does not explain when to call this vs get_borough_mortgage_stats. LLMs must infer intent from names alone.
Parameter 'race' accepts free-form strings with no enum constraint. Description lists examples ('Asian, Black, Hispanic...') but LLMs may hallucinate invalid values like 'Caucasian' or 'African'. Should declare as enum with exact valid values.