Loading ToolBench
MCP server that evaluates Kotlin scripts inside a running JetBrains IDE process, providing access to IntelliJ Platform APIs and loaded plugins
Single tool 'kotlin_eval' has a comprehensive, well-structured description (1,200+ chars) that clearly explains purpose, threading model, timeout behavior, and return contract. Input schema is fully defined with proper JSON Schema types and descriptions for all 5 parameters. However, the description is significantly longer than the 10-1024 char baseline (p90=392), risking token waste. Parameter descriptions are present but could be more concise. No output schema is documented, the tool returns unstructured text via TextContent, making it difficult for LLMs to plan downstream operations. Error handling is present but generic (timeout, invalid args, no project). Tool name 'kotlin_eval' lacks a clear action verb prefix (should be 'eval_kotlin' or 'execute_kotlin_script'). Overall solid for a specialized IDE integration tool, but falls short of A-grade polish.
Evaluate an unrestricted script inside the running JetBrains IDE process. The script language is Kotlin, but the exposed capabilities come from the current IntelliJ Platform IDE and the plugins loaded in that IDE, such as WebStorm, PyCharm, IntelliJ IDEA, GoLand, or other JetBrains IDEs. Do not assume Java, Kotlin, Gradle, Python, JavaScript, or GitHub APIs are present; probe plugin availability or class availability before using product- or language-specific APIs. This is a stateful project-scoped REPL. The script has one implicit receiver: project: com.intellij.openapi.project.Project. Top-level declarations from previous calls are kept, so you can define variables, helper functions, imports, and cached services once, then reuse them in later calls. Pass resetState=true when you want to clear that project's REPL history. Return contract: - The tool result is the script's final expression value. - stdout and stderr are not captured or returned. Do not use println as the answer channel. - Return concise strings or structured values from the final expression, for example: listOf("a", "b").joinToString("\n") Timeout behavior: - Scripts time out after the configured default unless timeoutMs is provided for a specific call. - timeoutMs must be greater than 0. - Blocking waits that respond to thread interruption, such as Thread.sleep or CountDownLatch.await, are interrupted on timeout. Pure CPU loops may not stop safely; avoid unbounded loops. - For long IDE operations, prefer starting asynchronous work and polling later instead of blocking the current MCP call. Threading and write safety: - Prefer IntelliJ coroutine APIs in scripts. Useful imports: import com.intellij.openapi.application.readAction import com.intellij.openapi.application.smartReadAction import com.intellij.openapi.application.writeAction import com.intellij.openapi.application.writeIntentReadAction import com.intellij.openapi.application.EDT import com.intellij.openapi.command.writeCommandAction - Signatures and call shapes: suspend fun <T> readAction(action: () -> T): T suspend fun <T> smartReadAction(project: Project, action: () -> T): T suspend fun <T> writeAction(action: () -> T): T suspend fun <T> writeIntentReadAction(action: () -> T): T suspend fun <T> writeCommandAction(project: Project, commandName: String, action: () -> T): T val Dispatchers.EDT: CoroutineContext - For PSI/VFS/model reads, use readAction { ... }. - For project-wide PSI analysis, index access, resolve, or anything that requires indexes, prefer smartReadAction(project) { ... }; it waits for smart mode and runs as a cancellable read action. - For PSI/document/project mutations, use writeAction { ... } or writeCommandAction(project, "Name") { ... }. - For model reads that must stay on the UI thread, use writeIntentReadAction { ... }. - For UI-only work, switch to withContext(Dispatchers.EDT) { ... }; prefer Dispatchers.EDT over Dispatchers.Main in IntelliJ Platform code. - Commit documents before reading PSI if editor changes matter: PsiDocumentManager.getInstance(project).commitAllDocuments() - Do not run long blocking waits on the EDT. Start the operation on the proper IDE API and wait from a background thread only when necessary.
Tool name 'kotlin_eval' does not start with a clear action verb. Should be 'eval_kotlin_script' or 'execute_kotlin_code' to match verb_noun convention and improve LLM intent inference.
No output schema documented. Tool returns unstructured TextContent; LLMs cannot infer result structure, making it impossible to plan chained operations or validate success. Document what the final expression value looks like (string, JSON, error format).
Description is 1,200+ characters, far exceeding the 10-1024 baseline (p90=392). Verbose descriptions waste tokens and bury key details. Condense to ~200 chars for the main description; move threading/timeout details to parameter descriptions or a separate guide.
Error handling is generic. Timeout and invalid-args errors do not guide recovery. Should include: 'If script times out, increase timeoutMs or break into smaller steps.' For 'no project found', suggest 'Ensure a JetBrains IDE is open with an active project.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 57 | 2026-07-28+ | v2 |
Parameter 'timeoutMs' lacks validation guidance in description. Should state: 'Must be > 0; defaults to server config (typically 30000ms). Blocking waits are interrupted; CPU loops may not stop safely.'