# cmdc_test

> CMDC 集成方 / 第三方 plugin 作者 / cmdc 子库测试 helpers 一站式包。

cmdc 主库自身的 `test/support/` 目录在 hex 发布时**不会打包**，导致集成方写
contract test 时都各自重写 mock provider / plugin runner / 事件断言族。
本子库把这些常见测试基础设施统一发布到 hex.pm。

## 安装

```elixir
defp deps do
  [
    {:cmdc, "~> 0.6"},
    {:cmdc_test, "~> 0.3", only: :test}
  ]
end
```

## 核心模块

| 模块 | 用途 |
|---|---|
| `CMDCTest.MockProvider` | Builder 式 mock LLM provider，配合 `CMDC.Config.provider_fn` 注入 |
| `CMDCTest.Plugin` + `Plugin.Spy` | `run_hook/3` 单元测 Plugin；Spy plugin 集成路径 inject anonymous handler |
| `CMDCTest.EventCapture` | EventBus 订阅 + 清理 |
| `CMDCTest.Assertions` | `assert_event_emitted` / `assert_event_count` / `refute_event_emitted` |
| `CMDCTest.RAG.Fixtures` | RAG / GraphRAG tool JSON、plugin event、evidence 与状态 fixture |
| `CMDCTest.RAG.MockBackend` | fake Arcana search/answer backend |
| `CMDCTest.RAG.MockGraphBackend` | fake GraphStore backend |
| `CMDCTest.RAG.MockMaintenanceBackend` | fake reembed / GraphRAG maintenance backend |
| `CMDCTest.RAG.MockPipelineRunner` | fake pipeline runner，返回 grounding 与 pipeline summary |
| `CMDCTest.RAG.MockStatusBackend` | fake knowledge index status backend |
| `CMDCTest.RAG.Policy` | collection ACL / pipeline / graph profile 测试 user_data helper |
| `CMDCTest.RAG.Assertions` | citation、grounding、pipeline、GraphRAG evidence 断言 |
| `CMDCTest.Workflow.Fixtures` | WorkflowSpec / Run / NodeRun / RunEvent / Gateway event fixture |
| `CMDCTest.Workflow.FakeRunStore` | shape-compatible fake `CMDCOrchestrator.RunStore` |
| `CMDCTest.Workflow.Assertions` | workflow completed、signal path、human_task、幂等与 Gateway event 断言 |
| `CMDCTest.Reasoning.Fixtures` | reasoning 事件与 Runner payload fixture |
| `CMDCTest.Reasoning.MockProvider` | prompt-routed mock provider，用于并行/递归推理分支测试 |
| `CMDCTest.Reasoning.Assertions` | reasoning done、strategy、progress、branch、score 断言 |

## Quick Start

### 1. Mock LLM Provider

```elixir
defmodule MyAgentTest do
  use ExUnit.Case
  alias CMDCTest.MockProvider

  test "Agent 收到 prompt 后 LLM 回复" do
    provider =
      MockProvider.new()
      |> MockProvider.respond("Hello from mock!")

    {:ok, session} =
      CMDC.create_agent(
        model: "mock:test",
        config: %{provider_fn: MockProvider.to_provider_fn(provider)}
      )

    CMDC.prompt(session, "hi")
    {:ok, reply} = CMDC.collect_reply(session, timeout: 2_000)
    assert reply == "Hello from mock!"
  end
end
```

### 2. 单元测试 Plugin

```elixir
import CMDCTest.Plugin

test "SecurityGuard 拦截危险 shell 命令" do
  assert {:ok, {:block_tool, reason, _state}} =
           run_hook(
             CMDC.Plugin.Builtin.SecurityGuard,
             {:before_tool, "shell", %{"cmd" => "rm -rf /"}}
           )

  assert reason =~ "dangerous"
end
```

### 3. 集成路径 hook 替换（Spy plugin）

```elixir
alias CMDCTest.Plugin.Spy

test "Spy 拦截 before_tool 收集所有工具调用" do
  test_pid = self()

  {:ok, session} =
    CMDC.create_agent(
      model: "mock:test",
      plugins: [
        {Spy,
         handler: fn
           {:before_tool, name, args}, state, _ctx ->
             send(test_pid, {:tool_intercepted, name, args})
             {:continue, state}

           _, state, _ -> {:continue, state}
         end}
      ]
    )

  CMDC.prompt(session, "go")
  assert_receive {:tool_intercepted, "shell", _}, 2_000
end
```

### 4. 事件断言族

```elixir
import CMDCTest.Assertions
alias CMDCTest.EventCapture

test "Agent 执行后发出 agent_end 事件" do
  {:ok, session} = CMDC.create_agent(...)
  :ok = EventCapture.start_capture(session)

  CMDC.prompt(session, "go")

  # 等待事件
  assert_event_emitted(session, :agent_end, timeout: 2_000)

  # payload 子集匹配
  assert_event_emitted(session, :tool_blocked, payload: %{tool: "shell"})

  # 事件不应出现
  refute_event_emitted(session, :approval_required, timeout: 200)

  # 数事件次数
  events = assert_event_count(session, :stream_chunk, 5, timeout: 1_000)
end
```

### 5. RAG / GraphRAG 测试支撑

```elixir
import CMDCTest.RAG.Assertions
alias CMDCTest.RAG.{Fixtures, Policy}

test "RAG 输出必须有 citation 和 GraphRAG evidence" do
  json = Fixtures.graph_search_tool_json()

  assert_citations(json, min: 1, collection: "policies")
  assert_graph_evidence(json, min_entities: 1, min_relationships: 1)
end

test "Agent user_data 使用 fake RAG backend" do
  user_data = Policy.user_data(collections: ["policies"])

  assert user_data.cmdc_rag_arcana[:backend] == CMDCTest.RAG.MockBackend
  assert user_data.cmdc_rag_arcana[:graph_backend] == CMDCTest.RAG.MockGraphBackend
end
```

RAG helpers 是 shape-compatible contract，不依赖真实 Arcana DB、GraphStore、
embedding 或 LLM。集成方可以把这些模块填进 `cmdc_rag_arcana` config，用于
CI、Gateway event contract、AgentOps Trace Viewer 和 Eval Gate 测试。

### 6. Workflow Runtime / AgentOps 测试支撑

```elixir
import CMDCTest.Workflow.Assertions

alias CMDCTest.Workflow.{FakeRunStore, Fixtures}

setup do
  FakeRunStore.reset!()
  :ok
end

test "workflow 运行事件 contract" do
  spec = Fixtures.workflow_spec()
  snapshot = Fixtures.status_snapshot()

  assert spec["workflow_id"] == "wf.contract_review"
  assert_workflow_completed(snapshot)
  assert_signal_path(snapshot, [{"risk_check", "true"}, {"legal_review", "approved"}])
  assert_human_task_created(snapshot, node_id: "legal_review")
end

test "orchestrator 可使用 fake RunStore" do
  {:ok, run_id} =
    CMDCOrchestrator.start_run(Fixtures.workflow_spec(),
      run_store: FakeRunStore,
      idempotency_key: "trigger-001"
    )

  {:ok, ^run_id} =
    CMDCOrchestrator.start_run(Fixtures.workflow_spec(),
      run_store: FakeRunStore,
      idempotency_key: "trigger-001"
    )
end
```

Workflow helpers 不依赖 Phoenix / Ecto / Oban / 真实 LLM。`FakeRunStore` 不在
编译期依赖 `cmdc_orchestrator`,但函数形状对齐 `CMDCOrchestrator.RunStore`
behaviour,适合企业 AgentOps CI、Gateway event snapshot 和发布门禁测试。

### 7. Reasoning 策略测试支撑

```elixir
import CMDCTest.Reasoning.Assertions
alias CMDCTest.Reasoning.{Fixtures, MockProvider}

test "reasoning trace contract" do
  events = Fixtures.events(strategy: "trm", answer: "answer 42", revise?: true)

  assert_reasoning_done(events, strategy: "trm", answer: "42")
  assert_reasoning_progress(events, :revise)
  assert_reasoning_branch_count(events, 1)
  assert_reasoning_score_min(events, 0.8)
end

test "parallel branch mock provider" do
  provider =
    MockProvider.to_provider_fn([
      {"branch 1", "first answer"},
      {~r/branch 2/, %{content: "second answer", usage: %{total_tokens: 3}}},
      {:default, "fallback"}
    ])

  {:ok, session} =
    CMDC.create_agent(
      model: "mock:test",
      config: %{provider_fn: provider}
    )

  {:ok, result} =
    CMDC.Reasoning.Runner.run(
      session,
      {CMDC.Reasoning.Strategy.ToT, beam_width: 2},
      "solve"
    )

  assert_reasoning_done(result)
end
```

## License

Apache 2.0