Loading ToolBench
Universal scientific data I/O with plugin registry. Supports 30+ formats including CSV, JSON, YAML, NPY, HDF5, Zarr, PKL, images, and more.
Strong tool naming (all verb_noun pattern: io_load, io_save, io_glob). Descriptions are comprehensive and LLM-optimized (100-300 chars typical), with clear WHEN-to-use guidance. All 22 tools have input schemas with typed parameters. However, output schemas are not documented, LLMs cannot predict return structure. Error handling guidance is minimal. No tool annotations (readOnlyHint/destructiveHint) despite clear risk classifications. Parameter descriptions are excellent but lack format constraints (e.g., glob patterns, file extensions). Missing idempotency guarantees for cache operations.
Drop every entry from the load cache. Use when the user asks to "clear the cache", "force fresh reads", or after files change on disk.
Enable/disable the load cache or change its capacity. Use when the user asks to "turn off caching", "increase cache size", or wants to tune cache behavior.
Embed JSON metadata into an existing image file. Use to backfill provenance into a figure that wasn't saved through `io_save`.
Print the group/dataset tree of an HDF5 file. Use when the user asks to "explore", "inspect", "show structure of", or "preview" an `.h5/.hdf5` file. Returns success status; the tree is printed to stdout.
Print the group/array tree of a Zarr store. HDF5-equivalent for `.zarr` directories.
Report current load-cache state (entries, hits, misses, size). Use when the user asks "is caching on?", "how big is the cache?", or wants to debug repeated-load performance.
Output schemas not documented. LLMs cannot predict return structure (e.g., does io_load return raw data or wrapped object? Does io_glob return paths as list or dict?). Breaks downstream tool chaining and forces agents to guess.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk classifications in metadata (READ_ONLY, WRITE, REVERSIBLE). Agents cannot distinguish safe reads from destructive writes without parsing descriptions.
Error handling guidance missing. No recovery hints (e.g., 'File not found, check path with io_glob' or 'Format not supported, call io_list_formats to see available formats'). Agents cannot self-correct on failures.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 68 | 2026-07-28+ | v2 |
Look up the registered loader for a file extension. Use when the user asks "is .ext supported?" or wants to know which function will handle a given file type. Returns the function's qualified name (or null if no handler is registered).
Look up the registered saver for a file extension. Mirrors `io_get_loader`. Use to verify whether `io_save(obj, 'foo.ext')` will succeed for a given extension.
Glob filesystem paths with natural sort and optional placeholder parsing. Drop-in upgrade for `glob.glob`/`pathlib.Path.glob` that returns natsort-ordered hits and (with `parse=True`) extracts `{name}` placeholders from each match. Use whenever the user asks to "find files matching", "list files", or wants templated path discovery.
Check whether an HDF5 file contains a given dataset/group key. Cheaper than opening the whole file.
Check whether an image file contains scitex-io provenance metadata. Quick yes/no for `io_read_metadata`.
Check whether a Zarr store contains a given key. Zarr equivalent of `io_has_h5_key`.
Convert a JSON object/array into Markdown headings + bullet lists. Use when the user asks to "format this JSON as markdown", "render config as docs", or wants a human-readable view of nested data.
List every file extension scitex-io can save or load. Use to discover supported formats before saving/loading, or to check whether a custom format is registered. Covers 30+ formats: CSV, Parquet, NumPy, pickle, YAML, JSON, HDF5, MATLAB, images, matplotlib figures, PyTorch, MNE, EDF, MP4, and more.
Load ANY data file — auto-detects format from extension. Drop-in replacement for `pd.read_csv`, `np.load`, `pickle.load`, `json.load`, `yaml.safe_load`, `h5py.File`, `torch.load`, `mne.io.read_raw_*`, `scipy.io.loadmat`, `cv2.imread`, etc. Use whenever the user asks to "load", "read", "open", or "import" a file. Handles glob patterns (`*.csv` → list of loaded objects) and caches by path+mtime for fast repeat loads.
Load a directory of YAML config files as one merged dotted-access dict. Use whenever the user mentions "config", "configs", "settings file", "hyperparameters file", "PATH.yaml", "PARAMS.yaml", or wants to centralize constants/magic-numbers out of source code. Every filename stem and every YAML key is normalised to UPPER_CASE at load time, so `model.yaml` with `hidden_dim: 256` lands at `CONFIG.MODEL.HIDDEN_DIM` regardless of source casing — your callers always read the same shape. Case-conflicting siblings (e.g. `MODEL.yaml` + `model.yaml`) emit a `UserWarning` and keep the UPPER variant. Returns a DotDict so callers can do `CONFIG.PATH.DATA_DIR` instead of `CONFIG["PATH"]["DATA_DIR"]`. Auto-promotes `DEBUG_*` keys when `is_debug=True` or when CI is detected.
Parse `{placeholder}` values out of every glob match. Like `io_glob(parse=True)` but returns only the parsed-fields list. Use when the user wants the *values* embedded in matching paths, not the paths themselves.
Read provenance metadata embedded in an image (PNG/JPEG) by `io_save(fig, …)`. Use when the user asks "where did this figure come from?", "what script produced this PNG?", or wants to inspect figure provenance.
Show how to register a custom save/load handler for a new file extension. Use when the user wants to "add support for a new format", "register a custom saver/loader", "extend scitex-io with my own format", or "make scitex-io handle .xyz files". Returns copy-paste decorator examples for both `register_saver` and `register_loader`.
Save ANY data to a file — format auto-detected from extension. Drop-in replacement for `df.to_csv`, `np.save/savez`, `pickle.dump`, `json.dump`, `yaml.dump`, `h5py` writes, `torch.save`, `fig.savefig`, `cv2.imwrite`, `scipy.io.savemat`, etc. Use whenever the user asks to "save", "write", "export", "dump", or "persist" data. Auto-routes relative paths: from a script → `{script}_out/`, from a notebook → `{notebook}_out/`, from IPython / REPL → `$SCITEX_DIR/io/runtime/cache/` (default `~/.scitex/io/runtime/cache/`); absolute paths bypass routing. Intermediate directories are auto-created — no `os.makedirs()` needed. Data arrives as a JSON string.
Fetch the full Markdown content of one scitex-io skill page.
List the names of every skill page shipped by scitex-io.
Parameter constraints not formalized. 'path' accepts glob patterns but no regex/format constraint documented. 'format' accepts extension strings but no enum of valid formats. Agents may pass invalid values without validation feedback.
Cache operations (io_clear_load_cache, io_configure_cache) lack idempotency guarantees. Repeated calls with same params should be safe, but no documentation confirms this. Agents may hesitate to retry on ambiguous failures.