Loading ToolBench
Multi-server MCP framework for industrial asset operations benchmarking, providing IoT sensor data, failure mode reasoning, time-series forecasting/anomaly detection, vibration analysis, and work order management.
AssetOpsBench defines 22 tools across 6 servers with consistent naming (verb_noun pattern) and complete input schemas. All tools have descriptions (10-200 chars, within baseline 34-392 range). However, output schemas are entirely undocumented, no tool declares what fields it returns, forcing LLMs to infer structure. Parameter descriptions are present but often generic (e.g., 'Asset identifier' without format hints). Error handling is absent: no recovery guidance, no categorization of retryable vs fatal errors. Security is weak: no permission gates, no audit trail declarations. Composition is reasonable, tools are single-purpose, but chaining is fragile due to missing output documentation.
Classify vibration severity per ISO 10816.
Return registry details for one asset.
List asset identifiers registered at one site.
Compute bearing characteristic frequencies (BPFO, BPFI, BSF, FTF) from geometry.
Characterize the shape of a time-series dataset as structured evidence.
Compute the envelope spectrum for bearing fault detection.
Compute FFT amplitude spectrum of a stored vibration signal.
Output schemas completely undocumented. No tool declares return type, field names, or structure. LLMs cannot plan downstream calls or extract required data for chaining.
No error handling guidance. Tools provide no recovery hints, error categorization, or actionable messages. LLMs cannot self-correct on failures.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 68 | <=2025-11-25 | v2 |
Provides the current date time as a JSON object.
Returns the current time in English text.
Assess data quality for a time-series dataset and produce a cleaned file pointer.
GENERATE a new or extended failure-mode list for an asset class.
Return cataloged asset classes and categories.
Return cataloged failure modes by asset category.
READ the known failure modes for an asset class.
Return cataloged sensor types.
Fetch vibration sensor time-series from CouchDB and load into the analysis store.
Reads a JSON file, parses its content, and returns the parsed data.
List the standardized TSFM tasks available in the benchmark.
List available sensor fields for an asset.
List measurement fields observed in an asset's telemetry stream.
Profile a time-series dataset behind a file pointer.
List sorted site identifiers available in the asset registry.
Parameter descriptions lack format constraints. E.g., 'ISO 8601 start timestamp' in get_vibration_data is good, but most catalog/asset lookups omit format hints, allowing LLMs to pass invalid values.
No pagination support on list tools (sites, list_tasks, list_vibration_sensors). Large result sets will blow context window. Missing limit/offset parameters and total count in responses.
No security declarations. Tools lack permission gates, audit trail logging, or scope declarations (e.g., 'read:vibration', 'write:asset'). No indication of which tools are destructive or require authorization.