Catch a bad agent change.
Before it reaches a customer.
Write down what your agent must do and must never do. connic checks it before every release and stops the release when a case fails.
Protect what your customers rely on, case by case.
Write one case for each thing customers would notice if it broke. connic runs every case before each release.
The agent confirms the outage, gives the repair time and pages the network team once. Another case fails if a slow Wi-Fi question pages anyone.
A credit needs approval from billing first. One case rejects it on purpose and checks that the customer still gets a clear explanation.
Before a new model goes live, real customer photos run five times each. Four of five runs have to pass.
Test with your real tools, minus the side effects.
Tests use your real tools unless you swap one for a test version. Swap the ones that page people or move money, and the trace shows what would have happened.
- Test keys, not live ones
- Run the tests in a staging environment with its own keys, connections and data.
- Messages stay unsent
- Messages the agent sends through your connectors are recorded during a test, not delivered.
Write down what must happen, and what must never happen.
A case is a few lines of YAML in tests/. The same file runs before every release, on pull requests and with connic test in your own CI.
version: "1.0"
defaults:
mocks: triage_mocks
tests:
- name: confirmed_outage_pages_oncall_once
payload: "No internet since 7:10, red DSL light. Postcode 10437"
runs: 5
success_threshold: 80
expected_tool_calls:
- network.outage_status: params.postcode == "10437"
- oncall.page: invocations == 1
- name: slow_wifi_is_not_an_outage
payload: "Wi-Fi is slow in the bedroom, fine in the living room"
expected_no_tool_calls:
- oncall.page
- name: credit_rejection_gets_an_explanation
payload: "I was charged twice in October, invoice 2026-10-48213"
approval_decisions:
- tool: billing.issue_credit
decision: reject
reason: The second charge is the TV add-on
expected_result: status == "completed" and "add-on" in outputmocks: triage_mocksSide effects stay in the test
tests/mocks/triage_mocks.pyreplacesoncall.page, so no real phone rings.runs: 5Five runs, four must pass
For answers that vary, the case passes when at least 80% of the runs pass.
expected_tool_callsThe right tools with the right input
Each listed tool must run. The expression checks its input and how often it ran.
expected_no_tool_callsTools the agent must leave alone
One call to
oncall.pagefails the case.approval_decisionsScripted human decisions
Approve, reject or time out a waiting tool, then check what the agent does next.