Loading ToolBench
A repository of skills and adapters for scientific research data standardization and DNAnexus integration
OneSkills exposes 11 tools across three domain adapters (validate_dxapp, TargetDiff data conversion, channel_flow_dns conversion, DeepCFD conversion). Critical issues: tool names lack action verbs and are often duplicated across adapters without disambiguation ('probe', 'plan', 'convert' appear 3 times each); input schemas are present but minimal with generic object types lacking field-level documentation; descriptions are present but generic (50-80 chars) and lack context on when to use each tool or what distinguishes them; no output schemas documented; parameter descriptions are sparse (1-2 line); no error recovery guidance. The tools appear to be thin wrappers around Python class methods from adapter modules, with minimal LLM-facing design. Tools 2, 5, 9 are all named 'probe' with identical signatures, and tools 3, 6, 10 are all named 'plan', LLMs cannot disambiguate them without domain knowledge. The 'convert' tools (4, 7, 11) write data but descriptions do not state this explicitly (critical for irreversible operations). No tool declares mutually exclusive parameters, no validation rules are stated in descriptions, and no dependency hints guide multi-step workflows.
Execute the conversion, writing LMDB outputs aligned with onescience-primitives/assets/bio/datasets/targetdiff under target_dir
Execute the conversion plan and write AI-Ready formatted output to target_dir for channel_flow_dns
Execute the conversion, writing paired pickle arrays dataX.pkl and dataY.pkl with deterministic train/test split and per-channel loss weights
Generate a conversion plan based on probe report and spec for channel_flow_dns
Produce a conversion plan dict from probe report and primitives spec for TargetDiff adapter
Produce a conversion plan dict from probe report and spec for DeepCFD adapter
Probe the source directory for channel_flow_dns data and return a probe report with file inventory
Three tools named 'probe', three named 'plan', three named 'convert' with no disambiguation. LLMs cannot select the correct adapter-specific tool.
Tool names lack action verbs. 'probe', 'plan', 'validate', 'convert' are ambiguous, 'validate' could mean validate-input or validate-output; 'convert' is a generic verb that doesn't convey 'convert-and-write'. Use 'check_dxapp', 'inspect_source_dir', 'generate_conversion_plan', 'execute_conversion'.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 39 | 2026-07-28+ | v2 |
Probe the source directory for DeepCFD pickle files and identify layout pattern (paired, per_case, or monolithic)
Inspect raw data layout for TargetDiff-style pocket-ligand collection and return a probe report dict
Validate the converted dataset for required files and JSON structure validity
Offline validation and safety linting for a DNAnexus dxapp.json file
Input parameters use generic 'object' type with no field-level schema documentation. 'manifest', 'probe_report', 'spec', 'plan' are typed as objects but their internal structure is not defined. LLMs cannot validate input structure or understand required/optional fields.
Descriptions are 40-80 characters, generic, and lack context. 'Inspect raw data layout for TargetDiff-style pocket-ligand collection and return a probe report dict' doesn't explain when to call it vs other tools, what fields are in the returned dict, or what errors to expect. No dependency hints linking 'probe' → 'plan' → 'convert' workflow.
Output schemas are not documented. Tools return dicts (probe_report, plan, arrays of pickles) but the response structure is not visible in tool definitions. LLMs cannot plan downstream calls or extract fields.
Three tools write data ('convert' for TargetDiff, channel_flow_dns, DeepCFD) but descriptions do not explicitly state 'This tool modifies the filesystem by writing LMDB/pickle outputs to target_dir.' Critical for agents to know which calls have irreversible consequences.
No error handling guidance. Descriptions do not explain what errors can occur (missing files, invalid manifest, conversion failures) or how to recover. No mention of validation or pre-flight checks before calling 'convert'.
Parameter descriptions are minimal (1-2 lines) and lack format/constraint guidance. 'Path to the raw source directory' doesn't specify if it must exist, whether relative/absolute paths are allowed, or what happens if the path doesn't exist. No validation hints for LLMs.
No workflow documentation. The tools are designed as a 'probe → plan → convert' pipeline for each adapter, but this dependency is not stated in descriptions. An LLM might call 'convert' directly without probing or planning, leading to failures.
'validate' (tool 8) and 'validate_dxapp' (tool 1) are both validation tools. No clear documentation of what distinguishes them. Tool 1 validates dxapp.json manifests; Tool 8 validates converted datasets. Names are not clear enough to prevent confusion.