The server exposes 42 tools with reasonable naming conventions and basic parameter schemas defined via Zod. However, critical gaps significantly limit production readiness: (1) Most tool descriptions are extremely terse (10-30 chars), far below the 50-200 char best practice for LLM reasoning. (2) Parameter descriptions are minimal or absent, parameters like 'properties' in bucket-create lack actionable detail about structure. (3) Output schemas are entirely undocumented, responses are unstructured strings, not structured JSON the LLM can reliably parse and chain. (4) No error handling guidance; errors return raw messages without recovery hints. (5) Three tools (search, fetch, answer_question) appear to be AI-powered research tools that may not have directly verifiable schemas in the source code provided. (6) Destructive operations (bucket-delete, bucket-data-delete, function-delete, passport-identity-delete, passport-apikey-delete, passport-policy-delete) lack confirmation/dry-run patterns. Overall, this reads as a functional but rushed integration, adequate for exploration, inadequate for mission-critical agent tasks.
Tools (42)
answer_questionread onlyauthsource verified65/100
Get AI-powered answers from your documentation collection. Use this after searching to get specific information about Spica APIs, endpoints, parameters, examples, and best practices and fetch to retrieve latest documents about this topics. Always consult documentation before performing any Spica operations.
Terse tool descriptions (10-30 chars) prevent LLMs from reasoning about tool selection. Examples: 'Get all buckets from Spica.' (30 chars), 'List all functions' (18 chars), 'Get all identities from Spica.' (30 chars). Best practice is 50-200 chars with context on WHEN to use the tool and WHAT it returns.
Output schemas completely undocumented. All tools return unstructured text strings (e.g. '✅ Bucket data retrieved successfully:\n{...}'). LLMs cannot reliably parse or chain unstructured output. Tools must return structured JSON with documented field types.
bucket-listbucket-createbucket-update
Recommendations
Expand tool descriptions to 100-150 chars. Include WHAT the tool does, WHEN to call it vs related tools, and a hint about what structure it returns. Example: 'bucket-list: Retrieve all buckets from Spica. Use this first to discover available data stores before querying specific buckets. Returns a list of bucket objects with id, title, description, and properties fields.'
Convert all unstructured string responses to structured JSON. Define explicit output schemas (using Zod or TypeScript interfaces) and return typed objects instead of formatted text strings. Example: bucket-data-list should return {success: boolean, data: Array<{id: string, ...}>, total: number, limit: number, skip: number}.
Document nested object parameters with explicit field descriptions. For bucket-create's 'properties' parameter, provide: 'Bucket properties schema: a JSON Schema object defining fields. Example: {title: {type: string}, age: {type: number, minimum: 0}}' or link to schema documentation.
Add error handling guidance to all tool descriptions. Specify what can go wrong and recovery steps. Example: 'bucket-create: If a bucket with this title already exists, you will receive a 409 error. Call bucket-list to check existing names first.'
Implement confirmation/dry-run for destructive operations. Add optional parameters like 'confirm: boolean' or 'dryRun: boolean' to delete tools. Document in description: 'Set confirm=true to proceed; omit or set false for a dry-run that returns what would be deleted.'
Add pagination caps and defaults. State in tool descriptions: 'Results limited to 100 items by default. Use limit (max 1000) and skip for pagination. Always check the returned total_count to plan follow-up queries.'
Score history
Overall score trend
↑ 7 points across a rubric change (v1 → v2)
53/100
Scored
Grade
Overall
Spec posture
Rubric
2026-09-22
D
53
2026-07-28+
v2
2026-03-09
F
46
-
v1
auth
source verified
53/100
Delete a bucket from Spica
bucket-listread onlyauthsource verified62/100
Get all buckets from Spica.
bucket-updatewriteauthsource verified55/100
Update an existing bucket in Spica
fetchread onlyauthsource verified63/100
Retrieve complete document content by ID from your documentation collection. Use this to get the full text of a specific document after using the search tool to find relevant documents.
ALWAYS USE THIS FIRST! Search for documents in your vector store collection to understand available APIs, endpoints, and functionality. You should default to calling this even if you think you already know the answer, since the documentation is always being updated. Use this before any Spica API operations to understand the correct syntax, parameters, and best practices.
workflow-guideread onlysource verified43/100
MANDATORY FIRST STEP: Always use this to understand the required documentation-first workflow before performing ANY Spica operations. This tool explains why and how to use search/fetch tools first.
Parameter descriptions are missing or minimal. 'properties' in bucket-create is described only as 'Bucket properties schema', what format? What fields are required? A JSON Schema example? Parameter descriptions must be specific and actionable.
No error handling guidance. Tools return raw error messages (e.g. 'Failed to create bucket data:\n{error message}') without recovery hints. LLMs cannot infer whether to retry, ask the user, or abandon the operation.
Three tools (search, fetch, answer_question, workflow-guide) are AI-powered research utilities that appear to integrate OpenAI's vector store APIs. Their actual implementation/schemas are not fully visible in the provided source (src/tools/deep-research.ts and src/tools/helper.ts are not shown in detail). Tool definition visibility is incomplete, schema scores for these tools capped at 50.
Parameter schema detail inconsistent. Some tools (function-logs, passport-policy-create) have richer nested object schemas, but many others (bucket-create with 'acl: object') expose nested structures without documenting their internal fields. LLMs cannot construct valid nested payloads without examples or field documentation.
No pagination result limits stated. bucket-data-list, function-logs, and passport-identity-list accept 'limit' and 'skip' but do not document maximum recommended limits or default behavior. Without caps, LLMs can accidentally request huge result sets that exhaust context.
Implement per-item error reporting for batch or multi-result operations. If bucket-data-list retrieves 20 items and 3 fail ACL checks, return partial results with success/failure markers per item instead of a blanket error.
Add IAM/permission scopes to tool descriptions. Document required permissions, e.g., 'passport-identity-delete requires admin:identity scope'. This guides agent configuration and audit trails.
Include IDs in responses that downstream tools need. If create_bucket returns a bucket object, ensure it includes the bucketId that update_bucket and delete_bucket require. Verify response field names match parameter names to prevent mapping mismatches.
Validate inputs early and return structured error responses. Instead of 'Failed to create bucket: (raw API error)', return {success: false, error: 'Invalid title: must be 1-255 characters', errorCode: 'VALIDATION_ERROR', retryable: false}.