MCP server for Labellerr SDK - manage annotation projects, monitor progress, and query data
The server provides 18 well-organized tools with complete JSON Schema definitions and structured descriptions. However, several critical definition quality issues limit production readiness: (1) Most tool descriptions are functional but generic, lacking the context-aware guidance needed for LLM tool selection (e.g., 'List all projects for the client' vs 'List all projects to see current annotation status, available team members, and rotation configs'); (2) Parameter descriptions are present but minimal, many lack explanation of valid ranges, format constraints, or dependencies (e.g., annotation_rotation_count lacks numeric bounds, review_rotation_count has a mysterious constraint 'must be 1' in the description but not enforced in schema); (3) Output schemas are not documented, callers cannot see what fields are returned, forcing LLMs to guess structure and missing chaining IDs for multi-step workflows; (4) Error handling is generic (McpError wrapping) with no recovery guidance or actionable messages; (5) Some parameter names are opaque (e.g., 'folder_to_upload' vs 'folder_path' inconsistency; 'data_type' repeated across many tools without clear values in each context). The schemas themselves are well-formed JSON Schema with enums and required fields, which is a strength. Baselines: average param description ~30 chars (well below the 72-char production baseline); no documented return types (0% compliance vs 100% A+ baseline); tool descriptions average ~60 chars (below 194-char baseline).
Check the status of export jobs
Get download URL for a completed export
Create an export of project annotations
Upload pre-annotations to a project (synchronous)
Upload pre-annotations to a project (asynchronous)
Create a new dataset
Output schemas not documented. Tools return results but callers cannot see field names, types, or structure. This forces LLMs to guess what's in responses and breaks multi-step tool chaining (e.g., project_list returns projects but doesn't document project_id field name, so follow-up tools may fail to extract it).
Parameter descriptions are minimal. Most lack format constraints, valid ranges, or dependency notes. E.g., annotation_rotation_count and review_rotation_count have no min/max documented (annotation_rotation_count could be 0, 1, 1000?); review_rotation_count has constraint 'must be 1' only in description, not enforced in schema; folder_path vs folder_to_upload naming is inconsistent.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 51 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 41 | - | v1 |
Get detailed information about a dataset
List all datasets (linked and unlinked)
Upload individual files to a dataset
Upload all files from a folder to a dataset
Monitor the status of a background job
Monitor annotation progress for a project
Create a new annotation project with dataset and guidelines
Get detailed information about a specific project
List all projects for the client
Update rotation configuration for a project
Query dataset contents and metadata
Query project data with filters
Tool descriptions are generic and do not provide LLM-optimized context for tool selection. E.g., 'List all projects for the client' does not explain when to use this vs project_get, what structure is returned, or what user intent it satisfies. Average description length ~60 chars vs production baseline of 194 chars.
No error recovery guidance. Generic McpError handling does not tell the LLM what to do next. E.g., if project_get fails with 'project not found', there's no suggestion to call project_list first, or what might have gone wrong.
Two tools with nearly identical names and parameters (annotation_upload_preannotations vs annotation_upload_preannotations_async) create confusion. The naming does not clearly signal the difference, one is sync, one is async, forcing the LLM to reason about which to pick. Names should be verb_adjective, e.g., upload_preannotations_sync vs upload_preannotations_background.
Enum constraints present but descriptions do not fully explain value meanings. E.g., data_type enum ['image', 'video', 'audio', 'document', 'text'] is clear, but option_type enum ['input', 'radio', 'boolean', 'select', 'dropdown', 'stt', 'imc', 'BoundingBox', 'polygon', 'dot', 'audio'] lacks explanation of what 'imc', 'stt' mean or when to use each.