Loading ToolBench
Postern — the Personal Context Gateway: connect the sectors of your life once, then read them and act on them from any agent through one neutral surface.
Postern defines 7 tools with reasonable naming (verb_noun pattern: describe_context, get_schema, query, get_record, fetch_live, list_actions, invoke_action). Descriptions are present and substantive (100-300+ chars), explaining WHAT each tool does and WHEN to use it. However, input schemas are not visible in the provided source code, only tool names and descriptions are shown. The Dockerfile and sample code do not expose the actual JSON Schema definitions for parameters. Without seeing explicit schema definitions, parameter types, constraints, and output schemas, schema quality cannot be verified. Descriptions are strong (e.g., describe_context explains sectors, connections, freshness states, health status), but the absence of visible schemas is a critical gap. Risk annotations (READ_ONLY, WRITE) are present, which is good for security signaling.
What you can reach. `sectors` is the list of sectors you are GRANTED — a granted sector may have NO connection at all, so always check `connections` before concluding a source exists. Each connection carries `connection_id` (use it to name the source to the user), a best-effort `label` (the institution or account name, null when unknown), its sync mode (push|poll|on_demand) and a server-derived freshness tuple {state, last_heard_at, next_expected_at, stream_state, error}. state is ok|syncing|stale|unreachable|error|paused|disconnected|never_synced — trust it over raw timestamps: 'unreachable' means a push stream is down/retrying (the cache is aging even though the connection looks active); 'stale' means past the mode's expectation. Non-active connections (error/syncing/paused) ARE listed so a broken source is visible instead of vanishing; only revoked ones are omitted. Use mode + freshness to decide cache-vs-fetch_live: a push connection with state 'ok' is already live (skip fetch_live); poll/on_demand connections go stale between syncs. Plaid connections also carry health {status: ok|action_required|transient|dead, consentExpiresAt}: status='action_required' means the bank link is broken and the cache is stale even though last_synced_at advanced — prompt the operator to reconnect; non-Plaid connections carry health: null.
Reach out to a source at read time — the only path that touches a source synchronously. Use it when the cache is stale or you need the absolute latest state. Slower than query (seconds, not milliseconds) and subject to source availability.
Fetch a single record by its Postern id (the `id` field from query results). Works even on rows the source deleted (they are excluded from query but still readable here).
Input schemas not visible in source code. No JSON Schema definitions provided for any of the 7 tools. Cannot verify parameter types, constraints, enums, or required fields. Per HARD SCORING RULES, schema score must be 0 when schemas are not visible.
Output schemas not documented. Descriptions mention what tools return (e.g., 'query returns capped responses with has_more flag', 'get_schema returns queryable objects and fields'), but formal output schema definitions are not visible. LLMs cannot reliably extract fields without documented return types.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 49 | 2026-07-28+ | v2 |
The queryable objects and fields per sector — the contract query() validates against. Each object carries `notes` and each field may carry a `fieldDocs` entry: authored semantics you MUST read before filtering or aggregating, because they state the sign conventions, enums and unit traps the types cannot. Rules that hold everywhere, so they are stated once here instead of per field: `id` is the Postern row uuid (use it with get_record / fetch_live) while `source_object_id` is the provider's own id; timestamps are ISO-8601 UTC; sum/avg/min/max return precision-safe decimal STRINGS and count returns a number; rows the source deleted are excluded from query but still readable via get_record; `raw` is the untouched provider payload and may use different names, units and SIGNS than the normalized columns.
Act on a source — send a command, update state, or trigger a workflow. Actions are sector-specific; list_actions shows what is available.
The actions available on a sector — what you can invoke. Each action carries its parameters and their types.
Read one sector. No cross-sector joins — fan out and correlate yourself. Responses are capped; `has_more: true` means you are looking at a page, not an answer.
query tool description is vague. 'Read one sector. No cross-sector joins, fan out and correlate yourself. Responses are capped; has_more: true means you are looking at a page, not an answer.' Does not explain what parameters query accepts, what filters are available, or how pagination works. LLM cannot determine when to call query vs get_record.
No error handling guidance in tool descriptions. Tools do not document what errors can occur, how to recover, or what the LLM should do if a call fails. E.g., fetch_live says 'Slower than query (seconds, not milliseconds) and subject to source availability' but does not explain what happens when a source is unavailable or how to retry.
invoke_action (WRITE tool) lacks confirmation/dry-run pattern. No mention of whether the action is reversible, what side effects occur, or whether a confirmation step is available. Agents could invoke destructive actions without safeguards.