Multi-agent financial research system using LangGraph and LangChain with RAG capabilities for SEC filings and live market data
18 tools with mixed quality. Most have descriptions (good), but schemas are incomplete or missing type details. Many parameters lack descriptions or constraints. Tool naming is generally clear (verb_noun pattern), but several tools conflate multiple concerns. No error handling guidance visible. Output schemas undocumented. Financial tools (hybrid_search, live_finance_researcher) have good descriptions but lack parameter validation. File tools (write_file, cleanup_files) expose state injection patterns but lack confirmation/dry-run for destructive ops. Average tool score: 52/100.
Delete ALL files for this user/thread from the real filesystem. This is a destructive operation. Use ONLY when the human user explicitly asks to wipe, reset, or clean the workspace for this conversation.
Get financial statements (income statement, balance sheet, cashflow, etc.).
Get historical stock prices for a given ticker symbol with optional period and interval parameters.
Get holder information including major holders, institutional holders, mutual fund holders, insider transactions, insider purchases, and insider roster holders.
Get option chain data for a specific expiration date and option type.
Get available option expiration dates for a stock ticker.
Get analyst recommendations including upgrades and downgrades.
Output schemas undocumented. No tool returns a documented schema showing field names, types, or structure. LLMs cannot plan downstream calls or extract data reliably.
Destructive tool (cleanup_files) lacks confirmation/dry-run pattern. Agents can irreversibly delete all user files without a safety gate or preview step.
State injection (InjectedState) in file tools exposes user_id/thread_id as implicit parameters. No validation that injected state matches caller identity, potential privilege escalation if state is spoofed.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 56 | 2026-07-28+ | v2 |
Get dividends and stock splits information.
Get comprehensive stock information including price, metrics, financials, etc.
Get current weather for a location using WeatherAPI.com. Use for queries about weather, temperature, or conditions in any city.
Get latest news for a stock ticker.
Search historical financial documents (SEC filings: 10-K, 10-Q, 8-K) using hybrid search. This is the PRIMARY tool for financial research. ALWAYS call this tool FIRST for ANY financial question unless user explicitly asks for "current", "live", "real-time", or "latest" market data.
Research live stock data using Yahoo Finance MCP. Use this tool to get current stock prices and real-time market data, latest financial news, stock recommendations and analyst ratings, option chains and expiration dates, recent stock actions (splits, dividends).
List available files for this user/thread on the real filesystem.
Read a file from the real filesystem.
Tool for strategic reflection on research progress and decision-making. Use this tool after each search to analyze results and plan next steps systematically. This creates a deliberate pause in the research workflow for quality decision-making.
Perform a live web search using Ollama Cloud Web Search API for real-time information and news.
Write content to a file on the real filesystem.
Yahoo Finance tools (get_stock_info, get_yahoo_finance_news, etc.) have minimal descriptions (60 chars avg). Parameter descriptions missing for 'ticker' and 'holder_type'/'financial_type' enums, LLMs cannot infer valid values.
No error handling guidance visible in any tool. No recovery hints, retryability classification, or actionable error messages. Agents cannot self-correct on failures.