Loading ToolBench
MCP server for SourceSync.ai - a retrieval engine and RAG platform with data connectors for ingesting and searching documents from multiple sources
SourceSyncAI MCP has 25 tools with mixed definition quality. Strengths: all tools have names that start with action verbs and descriptions present. Weaknesses: (1) Parameter schemas are minimal, most accept generic 'object' types without documented field constraints (fileStorageConfig, vectorStorageConfig, embeddingModelConfig, ingestConfig all typed as 'object' with vague descriptions). (2) Descriptions lack specificity about what fields are required, format expectations, or valid values. (3) No output schemas documented, LLMs cannot predict what fields to expect from responses. (4) Error handling is generic (safeApiCall wraps errors as JSON but provides no recovery guidance). (5) Critical multi-config parameters (e.g., createNamespace requires four complex configs) lack field-level documentation. Baseline: Average tool description length is 194 chars (rubric baseline); most SourceSyncAI descriptions range 80-150 chars, within acceptable range but lacking depth. 100% of A+ tools have per-param descriptions; here ~40% of params lack meaningful descriptions (generic 'object' types). Per-tool averaging: tools score 45-65, median ~52.
Creates a new connection to an external data source connector.
Creates a new namespace with the provided configuration. Requires a name, file storage configuration, vector storage configuration, and embedding model configuration.
Deletes documents from the namespace.
Permanently deletes a namespace by its ID.
Fetches documents from the namespace based on filter criteria. Supports pagination and including specific document properties.
Fetches and returns the content of a URL.
Generic 'object' type parameters lack field-level schemas and constraints. createNamespace, updateNamespace, and all ingest* tools accept fileStorageConfig, vectorStorageConfig, embeddingModelConfig as 'object' with only a brief description. LLMs cannot determine required fields, valid keys, or nested structure, forcing trial-and-error or external API documentation lookups.
No output schemas documented for any tool. Callers cannot predict response structure, presence of pagination cursors, or nested fields. LLMs default to guessing field names, risking broken tool chains when downstream tools expect specific IDs or references. Example: fetchDocuments returns unknown structure, subsequent tools cannot reliably extract document IDs for updateDocuments or deleteDocuments.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 46 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Retrieves a specific connection by its ID.
Checks the status of a previously submitted ingestion job.
Retrieves a specific namespace by its ID.
Performs hybrid search on documents in the namespace combining semantic and keyword search.
Ingests all documents in the connector that are in backlog or failed status. No need to provide the document ids or file ids for the ingestion. Ids are already in the backlog when picked thorough the picker. If not, the user has to go through the authorization flow again, where they will be asked to pick the documents again.
Ingests a file into the namespace. Supports various file formats with automatic parsing.
Ingests content from a website using its sitemap.xml. Supports path filtering and link limits.
Ingests raw text content into the namespace. Supports optional metadata and chunk configuration.
Ingests content from a list of URLs. Supports scraping options and metadata.
Crawls and ingests content from a website recursively. Supports depth control and path filtering.
Lists all connections for the current namespace.
Lists all namespaces available for the current API key and optional tenant ID.
Resyncs documents in the namespace.
Revokes an existing connection.
Performs semantic search on documents in the namespace.
Updates an existing connection.
Updates metadata for documents in the namespace.
Updates an existing namespace with the provided configuration parameters.
Validates the API key by attempting to list namespaces. Returns the list of namespaces if successful.
Insufficient error recovery guidance. safeApiCall() catches errors and returns JSON-stringified error objects, but provides no classification (retryable vs fatal), no actionable next steps, and no field-validation hints. Example: 'Invalid namespaceId' returns a string error with no suggestion to call listNamespaces() first.
Multiple ingest* tools (ingestText, ingestFile, ingestUrls, ingestSitemap, ingestWebsite, ingestConnector) accept vague 'ingestConfig' object with no field documentation. No description of how config structure differs across connector types (e.g., Notion vs Google Drive config shape). Agents must consult external API docs or fail on malformed config.
No permission checks or destructive operation confirmations. deleteNamespace and deleteDocuments marked DESTRUCTIVE but implementation (in src/index.ts) shows no dry-run, confirmation step, or permission gate.
Missing pagination support on list operations. listNamespaces, listConnections, and fetchDocuments accept no limit, offset, or cursor parameters. Returning unbounded results risks context window exhaustion and degrades agent reasoning.
Parameter descriptions generic or missing. Many parameters described only as 'Optional namespace ID' or 'Optional tenant ID' without explaining format, valid patterns, or when to use each. createConnection accepts 'connector' enum (NOTION, GOOGLE_DRIVE, etc.) but description does not list valid values, forcing LLM to guess or hallucinate connector type names.