Scoring was not performed
Tool descriptions are critically short (10-15 chars). Baseline for A+ tools is 50-200 chars. 'Get the last concept from the bank_data table.' and 'Add a new concept and expense to the bank_data table.' lack context for LLM selection and do not explain when to use them vs. alternatives.
Parameter descriptions are completely absent. Schema shows 'concept' (string) and 'expense' (number) but provides no guidance on valid values, ranges, format, or usage. Parameter 'expense' has no min/max constraint visible in schema or description.
No output schema documentation. Tools return simple strings ('The last concept is: {result[0]}', 'Added concept... with expense...') but the schema structure is not formally documented for the LLM. No indication of success/failure format.
No error handling or recovery guidance. Tools do not validate inputs (expense < 0 not rejected), do not return actionable error messages, and do not guide LLM recovery paths on failure.
Naming issue: 'add_concept' is vague. Does it add a financial category, a budget item, a note? 'add_expense_category' or 'add_bank_transaction' would be clearer and match the underlying bank_data table schema better.
Repository contains intentional security vulnerabilities (unparameterized SQL queries for bank_data, no input sanitization, exposed tool metadata). This is a PoC for security education, not a production server.
Tool list provided shows 6 entries, but only 2 unique tools exist (last_concept and add_concept, each repeated 3x). This indicates duplication across the 3 PoC implementations or data reporting error.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 20 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 36 | - | v1 |