Code intelligence and AI pull-request review across repositories with graph-based symbol analysis, dependency auditing, and agent execution capabilities
Celmis provides 7 tools with mostly clear naming (verb-first: run, list_*, find_*, get_*) and reasonable descriptions. However, critical gaps exist: parameter descriptions are sparse or missing entirely, input schemas are not visible in the provided source, and output schemas are undocumented. The 'run' tool has a lengthy description (>1000 chars) that reads like internal documentation rather than LLM-optimized guidance. Error handling and recovery paths are absent. No tool annotations (readOnlyHint/destructiveHint) despite clear risk levels. Composition is reasonable, tools chain naturally (find_symbol → get_symbol → find_callers/callees), but missing pagination parameters on list_* tools and no documented limits on result sets.
Find symbols that are called by the target (outgoing CALLS edges).
Find symbols that call the target (incoming CALLS edges). BFS expansion over CALLS edges. depth=1 — direct callers; >1 — transitive.
Find symbols by name in a specific repo. exact=True — exact match (MCP tools); exact=False — case-insensitive substring (UI search). Returns: list of dict, each with repo_slug + symbol fields.
Pull the full information about a symbol by id.
Groups with summary info. With `workspace_id`, only that tenant's. Walked as PATHS. Listing names installation-wide and then opening each by bare name meant a tenant-scoped group was listed and could not be opened, so it vanished behind a warning — and the two spellings disagreed about which groups exist at all. The cross-repo edge count comes from the group's own graph file rather than one built from the bare name, which two tenants sharing a group name would otherwise share.
Input schemas not visible in source code. Cannot verify parameter types, constraints, or defaults.
'run' tool description is 1200+ characters, exceeds recommended 1024 char limit and reads as internal documentation (sandbox behavior, git details, test runner notes) rather than LLM-optimized guidance. Should be 50-200 chars explaining WHAT it does and WHEN to use it.
list_groups and list_repos lack pagination parameters (limit, offset, page_size, cursor). No documented result limits. Large result sets will blow context window.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | 2026-07-28+ | v2 |
List of repos: either in a specific group, or every one indexed on disk.
Run a shell command against a COPY of the current working tree, in an isolated sandbox container. Use it to run tests, linters, builds and type checks — this is the only way to execute anything. YOU ARE ALREADY IN THE REPOSITORY ROOT. Do not cd anywhere first — run `pytest`, not `cd /some/path && pytest`. The absolute paths your Read and Edit tools use do NOT exist here: the sandbox unpacks the tree at its own location, so a command starting with `cd /workspace/...` fails with 'No such file or directory' and costs you a turn. Use paths relative to the repository root. The copy includes edits you have already made. It has bash, python, node/npm, ripgrep, jq and outbound internet, so installing dependencies works. Anything the command writes is DISCARDED — make changes with Edit/Write, then run this to check them. Returns the exit code with stdout and stderr. GIT WORKS, WITH NO HISTORY. The tenant's own .git is not shipped — it holds the remote URL, and a credentialed one would carry a token — so a fresh repository is created there instead: one commit, on the real branch name, no remote. `git status`, `git diff`, `git rev-parse` and pre-commit all work, and `git diff` shows what YOUR COMMAND changed, which is what a formatter or codegen check is usually asking — the `tool --write . && git diff --exit-code` idiom works. Note `git diff` does NOT show your own earlier edits: they are already in that baseline commit. There are no tags and no earlier commits, and the two consequences differ. `git describe` fails outright ('No names found'). setuptools_scm does NOT fail — it returns a placeholder like 0.1.dev1+g<sha>, so a build succeeds with a wrong version and only a test that asserts the version number notices. If you see a version assertion fail here, that is this, not your change. Pushing happens outside the sandbox and is handled for you. A TEST RUNNER MAY NOT BE INSTALLED. `python -m pytest` can answer 'No module named pytest' even when the project's tests are written for it — the image ships an interpreter, not the project's dev dependencies. Install and run in ONE call (`pip install pytest -q && python -m pytest -q`) rather than discovering it over two. Installing dependencies is fine and gets a longer clock, but it costs more of the shared sandbox, so combine steps in one call (`pip install -r requirements.txt && pytest`) instead of making the install its own round trip.
Output schemas are not documented anywhere in the provided source. LLMs cannot plan downstream calls or extract required fields (e.g., what fields does find_symbol return? Is repo_slug always present?).
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels. 'run' is marked WRITE risk but has no destructiveHint. Others are READ_ONLY but lack readOnlyHint. Agents cannot infer safety without annotations.