Loading ToolBench
MCP server exposing the DataOps Knowledge Hub (FastAPI app) as tools for AI agents. Communicates over stdio per the MCP spec's local-agent transport. Every tool is a thin HTTP client call into the already-running FastAPI app.
Three tools with complete schemas and descriptions, but significant gaps in parameter documentation and error handling. query_knowledge_hub has good description (180+ chars) and proper enum constraints on 'sources'. check_platform_health and trigger_ingestion lack parameter descriptions entirely (empty required arrays). No output schemas documented. Error handling is minimal, HTTP errors logged but no recovery guidance returned to LLM. Tool names follow verb_noun pattern (query_, check_, trigger_) which is good. Missing: per-parameter descriptions, output schema documentation, actionable error messages, and idempotency hints for the write operation (trigger_ingestion).
Check the health status of all services in the DataOps Knowledge Hub: PostgreSQL, Qdrant, Neo4j, MongoDB, and SeaweedFS. Returns the status of each service and overall platform health.
Query the DataOps Knowledge Hub — an enterprise RAG system that searches across 3 data stores: PostgreSQL (factual/numerical data about customers, orders, products), Qdrant (policies, SLAs, runbooks, incident logs), and Neo4j (pipeline lineage, table dependencies, team ownership). The system automatically routes your question to the appropriate engine(s) and returns a synthesized answer with sources and recommendations.
Trigger a re-ingestion of all data sources (SeaweedFS documents + MongoDB logs) into the Memory engine (Qdrant). Use this after new documents have been added or when you want to refresh the search index.
No output schemas documented. LLMs cannot plan downstream calls or extract structured data. query_knowledge_hub returns {answer, recommendation, sources_consulted, processing_time_ms}; check_platform_health returns {status, services, uptime_seconds, version}; trigger_ingestion returns {status, message}. These must be declared in tool definitions.
Error handling returns raw HTTP errors (httpx.HTTPError) with no recovery guidance. When API call fails, LLM receives 'HTTP error calling tool' with no actionable next step. Should return structured errors like 'API unavailable. Retry in 30s' or 'Invalid question format. Rephrase as: What is [metric] for [entity]?'
trigger_ingestion is a write operation but has no idempotency hint, dry-run option, or confirmation step. Agents may retry on transient failures, causing duplicate ingestions. Should add idempotentHint=false annotation and document: 'This operation is NOT idempotent. Retrying may trigger duplicate ingestion. Confirm before executing.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | 2026-07-28+ | v2 |
check_platform_health and trigger_ingestion have empty inputSchema (no properties, no required fields). While technically valid, they lack parameter descriptions. If these tools ever accept parameters (e.g., timeout, service_filter), the schema must document them. Currently ambiguous whether parameters are intentionally absent or overlooked.
query_knowledge_hub 'sources' parameter uses enum ['ledger', 'memory', 'brain'] but description says 'Optional: restrict search to specific engines.' No guidance on what each engine contains or when to use each. Should clarify: 'ledger=PostgreSQL (facts), memory=Qdrant (policies), brain=Neo4j (lineage). Omit to search all.'