Loading ToolBench
Sift provides 7 specialized tools with clear, domain-specific purposes (run, follow, outline, digest, digest_many, peek, tool). All tools have descriptions (10 - 200+ chars) and input schemas with typed parameters. However, descriptions are often lengthy and assume domain knowledge; parameter descriptions are minimal; output schemas are undocumented; and error handling guidance is absent. Tools follow a coherent composition pattern (distillation + capture management) but lack LLM-friendly clarity on when to use each variant.
The third question, and the one to reach for with a file you did not produce: a log, a saved CI transcript, a crash dump, a long export. It asks what happened in the file rather than what the file declares. Never read one of those with a plain file-reading tool -- the whole thing lands in the conversation and stays there.
The same as digest for several files at once, and it is worth reaching for whenever there is more than one: the waiting happens in parallel and the answer is one tool result instead of four.
Return only what the command has printed since the last time you asked, with the line numbers it has in the whole run. Nothing is shown to you twice. When you are finished with one, call `follow` with `stop` set: it ends the command, its children with it, and hands you the last of the output.
What does this file declare, without its bodies. It holds no list of languages and never looks at the suffix, so it can be asked about anything.
Bring any of the lines back unchanged from a capture. The capture itself, byte for byte, with no model involved.
Output schemas undocumented. No tool declares what fields it returns or their types. LLMs cannot plan downstream calls or extract structured data reliably.
Parameter descriptions are minimal or missing context. E.g., 'budget' lacks explanation of units (lines? bytes?), 'keep' lacks regex syntax guidance, 'grep' in peek lacks pattern format. LLMs cannot infer constraints.
Tool descriptions assume domain expertise ('distillation', 'capture', 'dense programs'). LLMs unfamiliar with sift's philosophy may misselect tools. E.g., 'digest' vs 'outline' distinction is unclear without reading the full docstring.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | 2026-07-28+ | v2 |
Run a shell command and return only the lines that mattered instead of all of its output. Every byte is kept on disk and never enters the conversation, so a 5,000-line test run costs a few dozen lines of context -- and goes on costing nothing on every later turn, because tool results are re-sent with the rest of the transcript. The lines shown are the command's own, byte for byte; each gap states how many lines it stands for, and `peek` with the returned handle brings any range back in full. Prefer this over a plain shell tool whenever the output may be long or noisy. When you already know what you
Run one of three dense programs -- `sg` (ast-grep), `diff` (difftastic), `loc` (scc) -- and distils what it printed. Each of them answers a question without opening a file, which is the cheapest way to answer anything: reach for `sg` rather than reading candidate files to find where a shape occurs.
No error handling guidance. Tools return errors as text (per server.py design), but descriptions do not explain error cases, recovery steps, or when to retry. LLMs cannot self-correct.
'tool' parameter 'name' accepts free-form string (sg|diff|loc) but lacks enum constraint. LLMs may hallucinate invalid tool names. 'args' is an untyped array with no format guidance.