Loading ToolBench
A thin Model Context Protocol server (stdio transport) that exposes the GOSCE portfolio of remote agents running on getvda.ai as MCP tools. Discovery is static so enumeration works with no network access. Tool execution proxies over HTTPS to selected agents' live /mcp invoke endpoints.
This server exposes 28 tools as a static proxy gateway to remote getvda.ai agents. Critical issues: (1) Tool definitions are NOT visible in the source code, only tool names, titles, and descriptions are hardcoded in a JSON array. No explicit schema registration, parameter definitions, or input validation is present. (2) Descriptions are present but generic and lack actionable detail for LLM selection. (3) No input schemas, parameter types, or output documentation visible. (4) No error handling guidance, recovery paths, or validation logic in the server code. (5) Tool names are inconsistent and often opaque (e.g., 'ai-getvda-bcrypt-langchain-core-mcp-d0e0d0'), many lack clear action verbs. (6) Duplicate tools with identical descriptions (tools 20-27 are all 'JWT-gated LLM gateway' variants) suggest poor composition and naming discipline. Per-tool scores average ~28 due to missing schemas and inferred-only definitions.
Google-OAuth-gated LLM gateway: verify a Google ID token, then run a Gemini (Vertex AI) completion for the verified caller.
Proxy Gemini (Vertex AI) completions wrapped in OpenTelemetry trace spans; returns the answer plus the trace/span id.
JWT-gated LLM gateway: authenticate (bcrypt/JWT), then run a LangChain-on-Vertex Gemini completion. Unauthenticated calls are rejected.
JWT-gated LLM gateway: authenticate (bcrypt/JWT), then run a LangChain-on-Vertex Gemini completion. Unauthenticated calls are rejected.
JWT-gated LLM gateway: authenticate (bcrypt/JWT), then run a LangChain-on-Vertex Gemini completion. Unauthenticated calls are rejected.
No input schemas visible in source code. Tools are registered only as names and descriptions in a static JSON array. No parameter types, constraints, or validation logic present. Per HARD SCORING RULE: schema score must be 0 for all tools.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 34 | 2026-07-28+ | v2 |
Hash passwords with bcrypt and issue/verify JWT session tokens.
Hash passwords with bcrypt and issue/verify JWT session tokens.
Google-OAuth-gated LLM gateway: verify a Google ID token, then run a Gemini (Vertex AI) completion for the verified caller.
JWT-gated LLM gateway: authenticate (bcrypt/JWT), then run a LangChain-on-Vertex Gemini completion. Unauthenticated calls are rejected.
JWT-gated LLM gateway: authenticate (bcrypt/JWT), then run a LangChain-on-Vertex Gemini completion. Unauthenticated calls are rejected.
JWT-gated LLM gateway: authenticate (bcrypt/JWT), then run a LangChain-on-Vertex Gemini completion. Unauthenticated calls are rejected.
JWT-gated LLM gateway: authenticate (bcrypt/JWT), then run a LangChain-on-Vertex Gemini completion. Unauthenticated calls are rejected.
JWT-gated LLM gateway: authenticate (bcrypt/JWT), then run a LangChain-on-Vertex Gemini completion. Unauthenticated calls are rejected.
LangChain chains WITH LangSmith tracing: full run visibility, evaluation metrics, and debugging alongside orchestration.
Multi-step LangChain prompt chains over Gemini. Manages context across sequential steps.
Generate a production-ready docker-compose.yml for a multi-service infrastructure stack — messaging (Kafka/ZooKeeper, RabbitMQ, NATS), object storage (MinIO), search (Elasticsearch/OpenSearch), databases, caches, web servers and auth (Keycloak/Vault). Every service is health-checked, on a private bridge network, with named volumes and a matching .env.example. Kafka auto-wires its required ZooKeeper.
OAuth/OIDC-gated multi-model router: supports Anthropic + OpenAI + Gemini behind enterprise identity providers.
Schema-enforced JSON extraction: give a prompt + Pydantic model, get validated typed output.
Credential-gated single-model LLM: bcrypt + JWT auth, then Gemini completion. Rejects without valid token.
Credential-gated MCP tool runner: bcrypt + JWT auth, then invoke any registered MCP tool. Multi-capability, not just LLM.
Stateless JWT auth: hash a password or verify a token. No LLM, no external calls. Pure crypto.
Run a prompt through a LangChain (system + human) chain over Gemini on Vertex AI; optional LangSmith tracing.
Run a prompt through a LangChain (system + human) chain over Gemini on Vertex AI; optional LangSmith tracing.
Run a prompt through a LangChain (system + human) chain over Gemini on Vertex AI; optional LangSmith tracing.
Generate a production-ready docker-compose.yml for a Postgres + Redis (database + cache) stack — health checks, named volumes, PgBouncer pooling, Redis maxmemory/persistence policy, bridge networking and a matching .env.example.
Capability discovery broker: describe what you need, get routed to the right specialist agent with verified example output.
Turn a prompt + a field schema into validated, typed JSON (Instructor over Gemini 2.5 Flash on Vertex AI).
Observable LLM proxy: transparent pass-through that adds OpenTelemetry spans to every call. Zero logic, pure instrumentation.
Tool names lack clear action verbs and are often opaque (e.g., 'ai-getvda-bcrypt-langchain-core-mcp-d0e0d0', 'llm-orchestration-agent-d2feb0'). LLMs cannot infer intent from these names. Baseline: 90% of A+ tools start with action verbs (get_, create_, update_, delete_, search_, list_, send_).
Eight tools (tools 20-27) have identical descriptions ('JWT-gated LLM gateway: authenticate (bcrypt/JWT), then run a LangChain-on-Vertex Gemini completion. Unauthenticated calls are rejected.'). This violates composition principle: each tool should do exactly one thing with a distinct name and description. LLMs cannot disambiguate between them.
No error handling, recovery guidance, or validation logic visible in glama_server.py. The proxy simply forwards calls to remote agents without documenting what errors to expect, how to retry, or what the LLM should do on failure. Per pattern:recovery-guide, errors must guide the agent's next step.
Tool definitions are inferred from the JSON array only, no explicit Tool registration with schemas is visible in the code. Per HARD SCORING RULE: 'If you are inferring tool existence (not seeing explicit registration): cap that tool's overall at 50.' All 28 tools are capped at 50 maximum, but actual scores are lower due to missing schemas.