Loading ToolBench
Multi-cloud monitoring and compliance audit platform supporting AWS, Huawei Cloud, and Alibaba Cloud with intelligent agent-based querying, resource scanning, and risk assessment
Cloud Monitor exposes 17 tools for multi-cloud monitoring (Aliyun, Huawei, AWS). While all tools have names and basic descriptions, critical quality issues significantly reduce overall score: (1) Descriptions are often in Chinese and/or vague about WHEN to use each tool. Many lack the 'WHEN' context LLMs need for selection. (2) Input schemas exist but are minimal, most parameters lack type constraints, enums, or range validation. For example, 'stat' parameter in aliyun_get_metric_data has no enum of valid statistics types. (3) No documented output schemas, responses are unstructured and could be very large (e.g., aws_ec2 warns of 1-2 minute CloudWatch scans). (4) Error handling is absent, tools do not guide recovery. (5) Tool names use underscores but are provider-prefixed (aliyun_, huawei_, aws_), creating semantic coupling rather than clear action verbs. Most tools are READ_ONLY (good security posture) but lack the rich composition patterns needed for agent planning. Per-tool scores average 38/100.
查询阿里云指定实例的监控指标数据。需提供 namespace、metric_name、instance_id
列出阿里云 CDN 加速域名（概览+已停用域名详情），与 AWS CloudFront 输出格式一致。status_filter: 'offline'=仅停用, 'online'=仅启用, ''=全部
列出阿里云 ECS 云主机实例（概览+已停止实例详情），与 AWS EC2 输出格式一致
列出阿里云可用的监控指标(ECS)。不传参数返回常用指标，可通过 namespace 和 metric_name 过滤
列出阿里云 OSS 对象存储桶（含区域信息和区域分布统计），与 AWS S3 输出格式一致
AWS EC2 统一查询：一次返回实例概览 + 已停止实例详情 + 全部低利用率实例（紧凑表格）。默认检测条件: CPU<10% 或 内存<10%（最近360小时）。account: 账户名(空=所有账户)。region: 指定区域(空=所有区域)。首次调用会扫描 CloudWatch（耗时约1-2分钟），结果自动缓存，后续调用瞬间返回。
Missing output schemas. No tool documents what fields it returns or their types. LLMs cannot plan downstream calls or know what data to extract.
No parameter enums or type constraints. 'stat' in aliyun_get_metric_data accepts free-form strings with no list of valid values (e.g., Average, Sum, Max, Min). LLMs will hallucinate invalid stat types.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 46 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 43 | - | v1 |
Get AWS VPN connection status and bandwidth metrics for a specific VPN
List AWS CloudFront distributions with domain, origin, and cache behavior configuration
List AWS Elastic Load Balancers with health status and target details
列出 AWS S3 存储桶（含每个桶的区域信息和区域分布统计）。account: 账户名(空=所有账户)。region: 过滤特定区域的桶(空=全部)
列出 AWS VPN 连接并自动查询每个VPN最近1小时的带宽数据（1分钟粒度表格）。自动扫描所有配置区域。account: 账户名(空=所有账户)。region: 指定区域(空=所有区域)
华为云 CCE 巡检：工作负载副本数检查 + 节点 Pod 数量检查，一次返回全部结果
华为云 DDS (MongoDB) 巡检：网络类型检查
华为云 DMS 巡检：RabbitMQ 集群部署检查
华为云 ECS 巡检：安全组规则检查 + 反亲和性检查 + 闲置实例检查，一次返回全部结果
列出华为云已配置的所有巡检区域及对应的 project_id
华为云 RDS 巡检：高可用部署检查 + 网络类型检查 + 参数双1检查，一次返回全部结果
Vague or missing 'WHEN' context in descriptions. Many descriptions state WHAT (e.g., '列出华为云ECS实例') but not WHEN to call this vs. a similar tool. LLMs cannot disambiguate between aliyun_list_ecs and aws_ec2 without clearer guidance.
Descriptions in Chinese without English translation. This limits adoption beyond Chinese-speaking agents and makes code review difficult for non-Chinese reviewers.
No error handling or recovery guidance. Tools do not document what to do if a call fails (e.g., invalid credentials, region not found, quota exceeded). LLMs have no path forward on errors.
Unclear parameter types and ranges. 'hours' in aws_ec2 accepts a number but no min/max. 'cpu_threshold' and 'mem_threshold' have no documented range (0 - 100%?). LLMs will pass invalid values.
Provider-prefixed tool names (aliyun_*, huawei_*, aws_*) couple semantic action to vendor. Consider refactoring to action-first names (list_compute_instances with an optional 'provider' param) for better composability.
aws_ec2 description warns of 1-2 minute scans and caching, but no timeout or pagination parameters exposed. LLMs cannot control result size or know safe retry windows.
No tool annotations. No tools declare whether they are read-only, destructive, or idempotent. While all appear to be read-only (good), explicit declarations help agents reason about retry safety.