74 lines
2.6 KiB
Markdown
74 lines
2.6 KiB
Markdown
# pipeline_setup_v4
|
|
|
|
`pipeline_setup_v4` is a component-level test harness built on top of the `v3` ideas.
|
|
|
|
Differences from `v3`:
|
|
|
|
- each YAML case targets a single isolated component;
|
|
- results are written next to the suite in `cases/.../test_runs/...`;
|
|
- supported components are `process_v2_intent_router` and `process_v2_retrieval_policy_resolver`.
|
|
Also available: `process_v2_router_plus_retrieval_policy` for the linked route -> plan chain,
|
|
`process_v2_router_plus_retrieval_policy_rag` for the linked route -> plan -> rag chain,
|
|
and `process_v2_full_chain` for the full route -> plan -> rag -> evidence -> workflow LLM chain.
|
|
|
|
## Run
|
|
|
|
```bash
|
|
PYTHONPATH=. python -m tests.pipeline_setup_v4.run \
|
|
--cases-dir tests/pipeline_setup_v4/cases/suite_01/process_v2_intent_router/router_only_docs_v2_matrix.yaml \
|
|
--run-name smoke_v4
|
|
```
|
|
|
|
LLM-first router suite:
|
|
|
|
```bash
|
|
PYTHONPATH=. python -m tests.pipeline_setup_v4.run \
|
|
--cases-dir tests/pipeline_setup_v4/cases/suite_02/process_v2_intent_router/router_llm_first_v3.yaml \
|
|
--run-name llm_first_v3
|
|
```
|
|
|
|
Retrieval policy resolver suite:
|
|
|
|
```bash
|
|
PYTHONPATH=. python -m tests.pipeline_setup_v4.run \
|
|
--cases-dir tests/pipeline_setup_v4/cases/suite_03/process_v2_retrieval_policy_resolver/cases.yaml \
|
|
--run-name retrieval_policy_v1
|
|
```
|
|
|
|
Linked router + retrieval policy suite:
|
|
|
|
```bash
|
|
PYTHONPATH=. python3 -m tests.pipeline_setup_v4.run \
|
|
--cases-dir tests/pipeline_setup_v4/cases/suite_04/process_v2_router_plus_retrieval_policy \
|
|
--run-name router_plus_policy_v1
|
|
```
|
|
|
|
Inside `suite_04`, cases are split into:
|
|
|
|
- `strict_regression_cases.yaml` for contract-level invariants
|
|
- `soft_observational_cases.yaml` for LLM-sensitive boundary scenarios
|
|
|
|
Quality-gate mini-pack:
|
|
|
|
```bash
|
|
PYTHONPATH=. python3 -m tests.pipeline_setup_v4.run \
|
|
--cases-dir tests/pipeline_setup_v4/cases/suite_05/process_v2_router_plus_retrieval_policy_quality_gate/cases.yaml \
|
|
--run-name router_plus_policy_qg_v1
|
|
```
|
|
|
|
Linked router + retrieval policy + rag suite:
|
|
|
|
```bash
|
|
PYTHONPATH=src:. DATABASE_URL='postgresql+psycopg://agent:agent@127.0.0.1:5432/agent' python3 -m tests.pipeline_setup_v4.run \
|
|
--cases-dir tests/pipeline_setup_v4/cases/suite_06/process_v2_router_plus_retrieval_policy_rag/cases.yaml \
|
|
--run-name router_plus_policy_rag_v1
|
|
```
|
|
|
|
Full process v2 chain with workflow LLM:
|
|
|
|
```bash
|
|
PYTHONPATH=src:. DATABASE_URL='postgresql+psycopg://agent:agent@127.0.0.1:5432/agent' python3 -m tests.pipeline_setup_v4.run \
|
|
--cases-dir tests/pipeline_setup_v4/cases/suite_07/process_v2_full_chain/cases.yaml \
|
|
--run-name process_v2_full_chain_v1
|
|
```
|