Monorepo for APEX: a comprehensive agentic platform engineering reference implementation for Azure. Includes agents, skills, infrastructure-as-code, and operational tools.
APEX provides 15 tools with reasonable naming (verb_noun structure: apex-recall <action>) and descriptions present for all tools. However, critical gaps undermine the score: (1) Input schemas are INFERRED from parameter lists but NOT explicitly visible in source code, cannot verify actual JSON Schema registration. (2) Descriptions are terse (20 - 80 chars) and lack LLM-optimization guidance (when to use, prerequisites, what it returns). (3) Parameter descriptions exist but are minimal ('Output as JSON', 'Max results') without format specs, constraints, or ranges. (4) No documented output schemas, LLMs cannot predict response structure. (5) No error handling guidance or recovery paths. (6) Several parameters lack type validation hints (e.g., 'step' accepts '1, 2, 3, 3_5, 4, 5, 6, 7' but this is buried in description text, not an enum). (7) No mention of idempotency, confirmation steps for destructive operations, or audit trails. The tools are well-intentioned but fall short of production-grade agent-ready standards.
Tools (15)
apex-recall checkpointwrite50/100
Record a sub-step checkpoint with optional artifact tracking and telemetry data
apex-recall complete-stepwrite48/100
Mark a step as complete with optional challenger-findings validation
apex-recall decidewrite50/100
Record a decision or decision_log entry for a project
apex-recall decisionsread only47/100
Query decision logs across projects
apex-recall filesread only47/100
List recently modified artifact files from the indexed agent-output directory
Input schemas not explicitly visible in source code, parameter definitions are inferred from listings rather than showing actual JSON Schema registration. Cannot verify type definitions, constraints, or format specifications.
Descriptions are minimal (20 - 80 chars), lacking guidance on WHEN to use each tool, WHAT to expect from output, and dependencies on other tools. LLMs cannot reliably select tools without richer context.
apex-recall reindexapex-recall filesapex-recall sessionsapex-recall showapex-recall health
Recommendations
ADD explicit JSON Schema definitions for all tool inputs: use 'type': 'string'|'integer'|'boolean'|'array', 'enum' for constrained values, 'minLength'/'maxLength' for strings, 'minimum'/'maximum' for integers. Reference the actual schema files in tool definitions so MCP clients can validate inputs.
EXPAND tool descriptions to 100 - 200 characters following the pattern: '[WHAT], [WHEN to use], [Prerequisites or dependencies].' Example: 'List session states updated within N days. Call this first to discover available projects before running step transitions. Requires at least one active project.'
DOCUMENT output schemas in tool definitions or a supplementary doc. Specify response structure: e.g., { 'sessions': [{ 'project': string, 'step': string, 'status': enum, 'updated_at': ISO8601 }], 'total': integer, 'next_cursor': string? }. This lets LLMs predict chaining calls.
ADD error recovery guidance to each tool. Example for 'apex-recall search': 'If no results, try removing project filter or expanding days range. If term is ambiguous, use apex-recall files to inspect artifact structure.'
ADD confirmation gates or dry-run modes for destructive writes: 'init --force' should require 'confirm_overwrite: true' parameter or split into 'preview_init' + 'apply_init'. Document retry-safety and idempotency guarantees.
Score history
Overall score trend
↑ 2 points across a rubric change (v1 → v2)
43/100
Scored
Grade
Overall
Spec posture
Rubric
2026-09-22
F
43
2026-07-28+
v2
2026-03-09
F
41
1.0.0+
v1
write
43/100
Force rebuild of the SQLite index for agent-output artifacts
apex-recall review-auditwrite50/100
Manage review_audit entries for a step
apex-recall searchread only50/100
Full-text search across indexed artifact content
apex-recall sessionsread only47/100
List session states across projects
apex-recall showread only45/100
Full context dump for one project
apex-recall start-stepwrite45/100
Mark a step as in_progress
apex-recall transitionwrite50/100
Atomic step transition: complete current step, record decisions, start next step
No error handling guidance, tools return failures but do not tell the LLM what to do next. Absent recovery paths (e.g., 'User not found. Call search_* first.').
Destructive operations (init --force, reindex, complete-step, transition) lack confirmation steps or dry-run modes. No mention of idempotency. Agents could accidentally overwrite session state or audit records.
No audit trail or logging hints in tool descriptions. For compliance-sensitive operations (audit records, decision logs, session state mutations), LLMs should know calls are logged and immutable.
Parameter descriptions lack format specs and ranges. E.g., 'days' (integer) has no min/max; 'project' (string) has no length or pattern; 'model' can be 'repeatable' but cardinality limits are not stated.
INCLUDE audit and immutability hints in descriptions for compliance-sensitive tools. Example: 'complete-step: Marks step as complete and records immutable decision log entry. This action is irreversible and fully audited.'
ADD min/max constraints to numeric and string parameters. Example: 'days: { type: integer, minimum: 1, maximum: 365, description: "Only results within N days (1-365)..." }' and 'project: { type: string, minLength: 1, maxLength: 128, pattern: "^[a-zA-Z0-9_-]+$" }'
DOCUMENT parameter relationships and mutual exclusivity. Example for 'decide': 'Mode A (decisions key-value) requires key and value; Mode B (decision_log) requires decision and rationale. Do not mix modes in one call.'
ADD examples of successful tool chaining in descriptions. E.g., 'search' returns project and session IDs; point out that these feed directly into 'show --project <project>' for full context. Reduces wasteful discovery calls.