Loading ToolBench
An MCP server for analyzing MSBuild binary log files (binlog), extracting build performance data, project information, target and task execution times, analyzer metrics, diagnostics, and source files.
Strong foundational quality with clear naming conventions and mostly complete schemas. All 21 tools follow verb_noun pattern (get_, list_, search_). Descriptions are present for all tools and parameters, averaging 150-250 characters, well within the 10-1024 ideal range. Input schemas are fully defined with type information and parameter descriptions. However, output schemas are not explicitly documented in the source code (only visible through return type inference from method signatures), and error handling is minimal, tools return null or throw InvalidOperationException without actionable recovery guidance. No tool annotations (readOnlyHint, idempotentHint) visible in the McpServerTool attributes, despite all tools being read-only and idempotent. The search_binlog tool has exceptionally detailed documentation of query syntax, which is excellent, but other tools lack error recovery hints. Overall, the server demonstrates solid baseline quality but falls short of A-grade due to missing output schema documentation, weak error guidance, and absent tool annotations.
Extract diagnostic information (errors, warnings) from a binlog file with optional filtering.
Get the global properties for a specific evaluation in the loaded binary log file. You can use the `list_evaluations` command to find the evaluation IDs. Global properties are what make evaluations distinct from one another within the same project.
Get specific items by type name for a project evaluation in the loaded binary log file. You can use the `list_evaluations` command to find the evaluation IDs. Returns items organized by item type (e.g., 'Compile', 'PackageReference', 'Reference').
Get specific properties by name for a project evaluation in the loaded binary log file. You can use the `list_evaluations` command to find the evaluation IDs. This returns all properties (both global and non-global) matching the requested names.
Get the N most expensive Roslyn analyzers and source generators across the entire build, aggregated by analyzer name.
Output schemas not explicitly documented in source code. Return types are inferred from method signatures (e.g., Dictionary<string, AggregatedAnalyzerData>, string?, IEnumerable<string>), but there is no schema documentation showing field types, required/optional distinction, or structure details that LLMs need for planning downstream calls.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | B | 73 | 2025-06-18+ | v2 |
| 2026-03-09 | F | 47 | - | v1 |
Get the N most expensive projects in the loaded binary log file, aggregated at the project level with options to exclude specific targets and show exclusive vs inclusive time.
Get the N most expensive targets in the loaded binary log file
Get the N most expensive MSBuild tasks in the loaded binary log file, aggregated by task name.
Get a specific source file from the loaded binary log file.
Get the total build time for a specific project, calculating exclusive time across all its targets with optional filtering to exclude specific targets.
Get a list of targets for a specific project in the loaded binary log file. This includes the target's name, ID, and duration.
Get all target execution times for a specific project in one call, including both inclusive and exclusive durations.
Get some details about a specific target called in a project within the loaded binary log file. This includes the target's duration, its ID, why it was built, etc. This is more efficient than `get_target_info_by_name` if you already know the target ID, as it avoids searching by name.
Get some details about a specific target called in a project within the loaded binary log file. This includes the target's duration, its ID, why it was built, etc.
Extract Roslyn analyzer and source generator execution data from a specific Csc task invocation.
List all evaluations for a specific project in the loaded binary log file. You can use the `list_projects` command to find the project file paths.
List all source files from the loaded binary log file, optionally filtering by a path pattern.
List all projects in the loaded binary log file
Load a binary log file from a given absolute path
Perform freetext search within a binlog file using the same search capabilities as the MSBuild Structured Log Viewer. Query Language Syntax: - Basic text search: Simply type words to find nodes containing that text - Exact match: Use quotes "exact phrase" for exact string matching - Multiple terms: Space-separated terms are AND'd together (all must match) Node Type Filtering: - $<type>: Filter by node type (e.g., $project, $target, $task, $csc, $rar) - Shortcuts: $csc expands to "$task csc", $rar expands to "$task ResolveAssemblyReference" Property and Field Matching: - name=<value>: Match nodes where the name field equals the value - value=<value>: Match nodes where the value field equals the value Hierarchical Search: - under(<query>): Find nodes under/within nodes matching the nested query - notunder(<query>): Exclude nodes under/within nodes matching the nested query - project(<query>): Find nodes within projects matching the nested query - not(<query>): Exclude nodes matching the nested query Time-based Filtering: - start<"YYYY-MM-DD HH:mm:ss": Nodes that started before the specified time - start>"YYYY-MM-DD HH:mm:ss": Nodes that started after the specified time - end<"YYYY-MM-DD HH:mm:ss": Nodes that ended before the specified time - end>"YYYY-MM-DD HH:mm:ss": Nodes that ended after the specified time Special Properties: - skipped=true/false: For targets, filter by whether they were skipped - height=<number> or height=max: Filter by tree height/depth Node Index Search: - $<number>: Find node by its unique index (e.g., $123) Result Enhancement: - $time or $duration: Include timing information in results - $start or $starttime: Include start time in results - $end or $endtime: Include end time in results Examples: - "error CS1234": Find exact error message - $task Copy: Find all Copy tasks - under($project MyProject): Find all nodes under MyProject - $target Build start>"2023-01-01 09:00:00": Build targets that started after 9 AM - name=Configuration value=Debug: Find nodes named Configuration with Debug value
Find all executions of a specific target across all projects (e.g., 'CoreCompile') and return their timing information.
Error handling provides no recovery guidance. Tools throw InvalidOperationException ('Binlog X has not been loaded. Please load it using the `load_binlog` command first') or return null, but most errors offer no actionable next step. Pattern requires error responses to tell the LLM what to do: 'User not found. Try search_users()' or categorize as retryable/user-fixable/fatal.
Tool annotations missing. McpServerTool attributes declare Idempotent=true and ReadOnly=true in source code, but tool definitions in the spec do not include readOnlyHint or idempotentHint annotations. LLMs cannot discover these safety properties from the tool definition alone.
No pagination support on tools returning lists. get_diagnostics accepts maxResults (capped), but list_files_from_binlog, list_projects, list_evaluations, and search_binlog return potentially unbounded lists without offset/limit pagination. Large result sets blow context windows and waste tokens.
Parameter constraints not enforced at schema level. top_number and maxResults accept unbounded integers with no explicit min/max in descriptions. maxItemsPerType defaults to 100 but no upper bound stated. Validation descriptions mention constraints (e.g., search_binlog: 'default: 300') but not all tools enforce them consistently.