Loading ToolBench
MCP server implementation of LightRAG API - exposes tools for all endpoints in the LightRAG OpenAPI spec (version 0204). Supports document ingestion, RAG queries, knowledge graph operations, and authentication.
23 tools with generally present but incomplete definitions. All tools have basic input schemas and descriptions, but many descriptions are too brief (10-50 chars), lack actionable guidance, and parameter descriptions are missing entirely. Critical issues: parameters lack type descriptions, no output schemas documented, no error handling guidance, missing parameter validation rules. Tool naming follows verb_noun pattern consistently (health, auth_status, documents_scan, etc.), which is a strength. However, descriptions read as API documentation rather than LLM-optimized prompts, they cite HTTP methods and endpoints instead of explaining WHEN to use each tool and what it enables. The query and query_stream tools have detailed descriptions with clear mode explanations, raising their bar significantly, but most other tools fall short. No security considerations documented (auth tools should explain credential handling). Destructive operations (documents_clear_all, documents_delete_by_ids, documents_delete_entity, documents_delete_relation) lack confirmation or dry-run patterns.
Login via OAuth2 password flow (POST /login). Stores Bearer token in-session. Example input: {"username":"admin", "password":"secret"}
Get authentication status and guest token if auth is not configured (GET /auth-status). Example input: {}
Clear ALL documents and files (DELETE /documents). Irreversible. Example input: {}
Clear LLM cache (POST /documents/clear_cache). Example input: {}
Delete specific documents (DELETE /documents/delete_document). Example input: {"doc_ids":["doc_123"], "delete_file": false}
Delete an entity from the KG (DELETE /documents/delete_entity). Example input: {"entity_name":"Apple Inc."}
Parameter descriptions missing entirely. No tool documents what each parameter controls (e.g., 'file_path' in documents_upload_file has no description explaining format/path type/validation rules). Baseline: 100% of A+ tools have param descriptions.
Output schemas not documented. No tool declares what fields are returned, their types, or structure. LLMs cannot plan downstream calls or extract data without knowing the response shape. Baseline: 100% of A+ tools have documented return types.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 53 | <=2025-11-25 | v2 |
Delete a relation from the KG (DELETE /documents/delete_relation). Example input: {"source_entity":"A", "target_entity":"B"}
Insert a single text (POST /documents/text). Example input: {"text":"hello", "file_source":"notes.txt"}
Insert multiple texts (POST /documents/texts). Example input: {"texts":["a","b"], "file_sources":["a.txt","b.txt"]}
List documents grouped by processing status (GET /documents). Example input: {}
Paginated documents query (POST /documents/paginated). Example input: {"page":1,"page_size":50,"sort_field":"updated_at","sort_direction":"desc"}
Get pipeline status/progress (GET /documents/pipeline_status). Example input: {}
Trigger scanning the input directory (POST /documents/scan). Example input: {}
Counts of documents by status (GET /documents/status_counts). Example input: {}
Track processing status by track_id (GET /documents/track_status/{track_id}). Example input: {"track_id":"upload_20250729_170612_abc123"}
Upload **one** file to input dir and index it (POST /documents/upload). Example input: {"file_path": "/path/to/file.pdf"}
Upload multiple files (calls /documents/upload once per file). Example input: {"file_paths":["/a.pdf", "/b.md"]}
Check if an entity exists in the knowledge graph (GET /graph/entity/exists). Example input: {"name":"Apple Inc."}
List all labels/types in the knowledge graph (GET /graph/label/list). Example input: {}
Get knowledge graph data for a specific label (GET /graphs). Example input: {"label":"Person", "max_depth":3, "max_nodes":1000}
Get current system status (GET /health). Example input: {}
RAG query (POST /query). Returns **structured output**: {request, response}. Query Modes: naive: Simple vector similarity search. Use for quick, general semantic search. local: Entity-focused retrieval. Use for questions about a specific person, place, or thing. global: Relationship-focused retrieval. Use for understanding connections between entities. hybrid: Combined entity and relationship retrieval. Use for comprehensive answers. mix: LLM-enhanced keyword retrieval. Use for complex queries. bypass: Direct to LLM. Use for chatting directly with the LLM. Other Important Parameters: only_need_context: If True, returns the retrieved context without generating a response. Useful for debugging. response_type: Defines the response format. Examples: 'Multiple Paragraphs', 'Single Paragraph', 'Bullet Points'. top_k: Number of top items to retrieve from the knowledge graph. chunk_top_k: Number of text chunks to retrieve from vector search. conversation_history: A list of past user/assistant messages to maintain context. user_prompt: A custom prompt to guide the LLM's response. enable_rerank: If True, reranks the retrieved text chunks for better relevance.
RAG query with streaming response (POST /query/stream). Returns streamed output concatenated into response string.
Tool descriptions are API-centric, not LLM-optimized. Most read as HTTP endpoint documentation ('GET /health', 'POST /documents/scan') rather than explaining WHAT the tool does, WHEN to use it, or WHY an agent should call it. Average description length 40-50 chars; should be 50-200 chars with actionable context. Baseline: average tool description 194 chars in production tools.
Destructive operations lack confirmation or dry-run patterns. Tools like documents_clear_all (irreversible), documents_delete_by_ids, documents_delete_entity, documents_delete_relation allow destructive state changes without a confirmation step or recovery guidance. Agents make mistakes, no safeguard against catastrophic errors.
No error handling guidance. Tools do not document what errors can occur, whether they are retryable, or how agents should recover. E.g., 'Invalid entity_name' is not documented for documents_delete_entity. Baseline: all A+ tools categorize errors and guide recovery.
No validation constraints documented. Parameters like page_size in documents_paginated declare min/max in schema (10 - 200) but not in description text. LLMs cannot read JSON Schema pattern fields reliably, constraints must be stated in descriptions.
Query parameters in documents_paginated (top_k, chunk_top_k, max_*_tokens) in query/query_stream lack validation rules in descriptions. LLMs will pass arbitrary values without understanding constraints.
No security/permission documentation. Auth tools (auth_login, auth_status) do not document credential handling, scope declarations, or what permissions each tool requires. Credential injection pattern not documented.