Loading ToolBench
Android keyboard application with AI-powered tools for LLM integration. Provides tools for calendar management, reminders, navigation, phone calls, SMS, contact lookup, and other device interactions.
Strong foundation with 17 well-named tools covering device, communication, and utility functions. All tools have descriptions (avg 150 chars) and explicit JSON schemas with typed parameters. Naming follows verb_noun convention (get_*, set_*, fetch_*, send_*). However, several tools lack output schema documentation, some parameter descriptions are generic, and error handling guidance is minimal. Calendar and reminder tools show sophisticated multi-step patterns with confirmation workflows. No security issues detected (no secrets in params). Composition is clean, each tool has one responsibility.
Return the device's current battery percentage, charging state, and temperature. Use when the user asks about battery.
Evaluate an arithmetic expression. Supports + - * / % ^ and parentheses. Use this for any numeric calculation — do NOT guess the result yourself.
Read, add, update, or delete events in the user's device calendar. Use action='list' with 'days_ahead' to list upcoming events (each line includes id=N), action='add' with 'title', 'start' (ISO 8601, or relative: 'tomorrow', 'morgen', 'overmorgen', 'next tuesday', 'volgende week dinsdag', optionally with time like 'tomorrow 15:00'), and optional 'end', 'location', 'description', 'all_day', action='update' with 'event_id' plus any fields to change, or action='delete' with 'event_id' to remove an event. IMPORTANT: update and delete require TWO calls. First call WITHOUT confirm to preview the change, then show the preview to the user and ask for confirmation, then call again WITH confirm=true. NEVER guess an event_id — always run action='list' first to get real ids. Times are interpreted in the device's local timezone unless they include an explicit offset.
Search the user's contacts by name and return matching names, phone numbers and email addresses. Use this when the user refers to a person by name and you need their phone number or email. Returns up to 5 matches. After getting the number, use phone_call or send_sms to act on it.
Output schemas not documented. Tools return results but LLMs cannot see field structure (e.g., what does web_search return? title, url, snippet fields?). Forces LLMs to infer output format.
No error handling guidance. Tools lack recovery hints (e.g., 'If URL fetch fails, try a shorter timeout' or 'If contact not found, try partial name search'). Agents cannot self-correct.
Parameter descriptions lack format/constraint details. 'location' in weather tool has no format spec (city name? coordinates? both?). 'expression' in calculator lacks examples of supported operators.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 65 | 2026-07-28+ | v2 |
Return info about the current Android device: manufacturer, model, Android version, locale, timezone, and free storage. Use when the user asks what phone/device they have, their Android version, or free space.
GET a web page over http/https and return its text content (HTML is stripped to plain text). Max 100 KB, 10 second timeout. Use this when the user asks you to read or summarize a specific URL.
Return the current date, time, day-of-week and timezone. Use this whenever the user asks about 'today', 'tomorrow', 'now', current year, or anything time-sensitive — your training data is out of date.
Open navigation to a destination. The device's default maps/navigation app will launch with directions. Use `destination` for an address or place name (e.g. 'Amsterdam Centraal', 'Kalverstraat 1, Amsterdam'). Set `mode` to 'd' (driving, default), 'w' (walking), 'b' (bicycling), or 'l' (two-wheeler).
Launch an installed app by package name (e.g. 'com.spotify.music', 'com.google.android.apps.maps'). The app must be installed on the device.
Open the phone dialer with a number ready to call. The user must press the call button themselves. Use `phone_number` for the number to dial (e.g. '+31612345678', '0612345678').
Return the current text contents of the system clipboard. Use when the user asks what is in their clipboard or wants you to act on the most recently copied text.
Open the SMS/messaging app with a message ready to send. The user must press send themselves. Provide `phone_number` for the recipient and `message` for the text.
Schedule a reminder. The user will receive a system notification at the chosen time; tapping it reopens this exact conversation with the reminder appended. Use `seconds_from_now` for relative times (e.g. 'in 10 minutes') or `datetime` (ISO 8601, e.g. '2026-04-09T09:30:00') for absolute times. At least one of the two must be provided. Keep `message` short (a few words).
Start a countdown timer in the device's Clock app. The user will see the Clock app open with the timer. Max 24 hours.
Convert a numeric value between units. Supports length (mm cm m km in ft yd mi), mass (mg g kg lb oz), temperature (c f k), volume (ml l gal fl_oz cup), time (s min h day), speed (kmh mph ms knot), data (b kb mb gb tb). Both 'from' and 'to' must be in the same category.
Get current weather for a location (city name or 'city, country'). Returns temperature, conditions, wind and humidity. Uses wttr.in.
Search the web via Brave Search and return the top results (title, URL, snippet). Use this when the user asks about recent events, current facts, or anything where you need fresh information. Follow up with fetch_url if you need the full content of a result.
No pagination or result limits documented for web_search. 'max_results' param exists (1-10) but no guidance on what happens if query matches 1000+ results or how to iterate.
calendar tool's confirmation workflow is complex. Description mentions 'TWO calls' but no explicit error response structure for preview vs. execute. LLMs may misunderstand the flow.