# CODE_QA evaluation summary

**8/18** cases passed.


| Case ID                        | Query                              | Expected scenario | Actual scenario  | Target                        | Evidence | Answer mode  | Pass |
| ------------------------------ | ---------------------------------- | ----------------- | ---------------- | ----------------------------- | -------- | ------------ | ---- |
| open_file_main_positive        | Открой файл main.py                | OPEN_FILE         | OPEN_FILE        | main.py                       | ✓        | normal       | ✓    |
| open_file_api_positive         | Покажи src/order_app/api/orders.py | OPEN_FILE         | OPEN_FILE        | src/order_app/api/orders.py   | ✓        | normal       | ✓    |
| open_file_borderline           | Open main                          | OPEN_FILE         | FIND_ENTRYPOINTS | src/order_app/api/orders.py   | ✓        | normal       | ✗    |
| open_file_negative             | Открой файл nonexistent/foo.py     | OPEN_FILE         | OPEN_FILE        | —                             | ✗        | insufficient | ✗    |
| explain_order_positive         | Объясни класс Order                | EXPLAIN           | EXPLAIN          | Order                         | ✓        | normal       | ✓    |
| explain_order_service_positive | Как работает OrderService?         | EXPLAIN           | EXPLAIN          | OrderService                  | ✓        | normal       | ✓    |
| explain_borderline             | Что делает create_order?           | EXPLAIN           | EXPLAIN          | src/order_app/api/orders.py   | ✓        | normal       | ✗    |
| explain_negative               | Объясни класс NonExistentClass     | EXPLAIN           | EXPLAIN          | src/order_app/domain/order.py | ✓        | normal       | ✗    |
| find_tests_positive            | Где тесты для OrderService?        | FIND_TESTS        | FIND_TESTS       | tests/test_order_service.py   | ✓        | normal       | ✓    |
| find_tests_order_positive      | Найди тесты для Order              | FIND_TESTS        | FIND_TESTS       | tests/test_order_service.py   | ✓        | normal       | ✓    |
| find_tests_borderline          | Есть ли тесты на репозиторий?      | FIND_TESTS        | FIND_TESTS       | tests/test_order_service.py   | ✓        | normal       | ✓    |
| find_tests_negative            | Где тесты для NonExistent?         | FIND_TESTS        | FIND_TESTS       | tests/test_order_service.py   | ✓        | normal       | ✗    |
| find_entrypoints_positive      | Какие точки входа в приложение?    | FIND_ENTRYPOINTS  | FIND_ENTRYPOINTS | src/order_app/api/**init**.py | ✗        | degraded     | ✗    |
| find_entrypoints_english       | Find application entrypoints       | FIND_ENTRYPOINTS  | FIND_ENTRYPOINTS | src/order_app/api/orders.py   | ✓        | normal       | ✓    |
| find_entrypoints_borderline    | Где main?                          | FIND_ENTRYPOINTS  | FIND_ENTRYPOINTS | src/order_app/domain/order.py | ✗        | degraded     | ✗    |
| general_qa_positive            | Что делает этот проект?            | GENERAL_QA        | EXPLAIN          | src/order_app/api/**init**.py | ✓        | normal       | ✗    |
| general_qa_how                 | How does order creation work?      | GENERAL_QA        | ARCHITECTURE     | Order                         | ✓        | normal       | ✗    |
| general_qa_borderline          | Расскажи про код                   | GENERAL_QA        | EXPLAIN          | src/order_app/api/**init**.py | ✓        | normal       | ✗    |


## Failures

- **open_file_borderline**: sub_intent: expected OPEN_FILE, got FIND_ENTRYPOINTS
- **open_file_negative**: answer_mode: expected degraded, got insufficient
- **explain_borderline**: intent: expected CODE_QA, got PROJECT_MISC
- **explain_negative**: answer_mode: expected degraded, got normal
- **find_tests_negative**: answer_mode: expected degraded, got normal
- **find_entrypoints_positive**: intent: expected CODE_QA, got PROJECT_MISC; answer_mode: expected normal, got degraded
- **find_entrypoints_borderline**: intent: expected CODE_QA, got PROJECT_MISC; answer_mode: expected normal, got degraded
- **general_qa_positive**: intent: expected CODE_QA, got PROJECT_MISC; sub_intent: expected GENERAL_QA, got EXPLAIN
- **general_qa_how**: sub_intent: expected GENERAL_QA, got ARCHITECTURE
- **general_qa_borderline**: sub_intent: expected GENERAL_QA, got EXPLAIN

