Test Assertions

Bowire ships with a built-in assertion runner so you can verify response shapes without leaving the request flow. Assertions are configured per method, run automatically after every successful invocation, and turn the Tests tab green or red so you can spot regressions at a glance.

When you can use it

The Tests tab is only visible for unary methods -- it lives next to Performance in the response pane. Streaming, channel, and duplex methods don't show it. The reason is the same as for Performance Graphs: assertions need a single discrete response to compare against.

Anatomy of an assertion

Each assertion is a row of path / operator / expected value:

Field Purpose
Path JSON path against the response body, or the literal status for the response status name
Operator How to compare actual against expected (see below)
Expected The value to compare against (text input, hidden for exists / notexists)

The path syntax mirrors Request Chaining:

Path Resolves to
response The whole response body
response.id body.id
response.user.email body.user.email
response.items.0.tags.2 Third tag of the first item
$.items[?(@.done == true)].id The id of every item whose done is true
$.items[*].price Every item's price
$..email Every email field, at any depth
$.items[-1] The last item
status The gRPC / HTTP status name (e.g. OK, NotFound, Unauthenticated). For a REST response an expected value written as a three-digit code — status eq 200 — is read against the HTTP status code the invoker carries as http_status, so it does not fail against OK with the number it asked for one field away

JSONPath

A path that starts with $ is JSONPath (a subset of RFC 9535):

  • .name or ['name'] for a field, and [0] or [-1] for an array element. A number after a dot ($.items.0) is refused: write $.items[0], or the short form items.0.
  • [*] and .* for every child, ..name for a field at any depth, and [start:end:step] and [0,2] for slices and unions.
  • Filters [?(…)] over an array's elements or an object's values. A filter compares with ==, !=, <, <=, > and >=, combines with &&, ||, ! and parentheses, and refers to the current element as @ and to the document as $.
  • @.field on its own tests that the field exists: [?(@.discount)].

A path that can select several values (a wildcard, filter, slice, union or ..) gives an array of what it matched, so contains, eq [ … ] and type array work on the result. A path that matches nothing is missing, so notexists passes. A malformed path fails the assertion with the position of the first character that does not fit, for example Invalid JSONPath at position 19: expected a value or a path.

The workbench and bowire test resolve paths with the same engine, held to one shared test fixture, so a path that matches in the Tests tab matches in CI.

Operators

Operator Meaning Example
eq Equality (loose: numbers compare numerically, objects compare via JSON) response.id eq 42
ne Not equal status ne Error
gt, gte, lt, lte Numeric comparisons response.count gte 1
contains Substring (string) or membership (array) response.tags contains "work"
matches Regex against the string form of the value response.email matches ^[^@]+@example\.com$
exists The path selects something, a JSON null included response.id exists
notexists The path selects nothing response.error notexists
type typeof check (string, number, boolean, object, array, null) response.items type array

eq and ne use loose equality so "42" from a text input matches the number 42 from a parsed response. Object comparison is via JSON.stringify, so the order of keys matters in nested objects.

Adding assertions

  1. Open a unary method
  2. Click the Tests tab in the response pane
  3. Click + Add Assertion
  4. Fill in path, pick an operator, type the expected value
  5. Fire a request — the assertion runs automatically against the response

You don't need to save anything explicitly. Edits are persisted to localStorage as soon as you change a field.

Auto-run

Every successful unary invocation automatically runs all assertions configured for that method. The tab label updates to show the result:

Label Meaning
Tests No assertions yet
Tests (3) 3 assertions configured, none have run yet
Tests (3/3 ✓) (green) All assertions passed
Tests (2/3 ✗) (red) At least one assertion failed

Inside the tab, each row shows a status icon, color-coded left border (green/red/gray), and on failure a one-line diff with the actual value.

Manual re-run

The Run against last response button at the bottom re-evaluates every assertion against the most recent response without firing a new request. Useful when you've just added or edited an assertion and want to test it.

Removing assertions

  • Per-row: click the × button on the right
  • All at once: Remove all button at the bottom (with confirmation)

Storage

Assertions live in localStorage under the key bowire_tests:

{
  "Todos::GetTodo": [
    { "id": "t_abc123", "path": "status", "op": "eq", "expected": "OK" },
    { "id": "t_def456", "path": "response.id", "op": "eq", "expected": "1" },
    { "id": "t_ghi789", "path": "response.tags", "op": "contains", "expected": "work" }
  ]
}

They are not persisted to disk -- the assertion store is per-browser, like history and favorites. They are also not exported as part of environment export (because they're method-bound, not environment-bound).

Substitution

Expected values pass through the same ${var} substitution engine as everything else. You can store the expected value as an environment variable and reference it:

Variables (Dev env):
  prodVersion = v2.1.4

Assertion:
  response.version  eq  ${prodVersion}

This makes assertion suites portable across environments — the same test passes against Dev, Staging, and Prod when each environment defines the right values.

System variables (${now}, ${uuid}, ...) work too, though they're rarely useful in expectations.

Export, import and CI

The Export collection button in the Tests tab downloads a portable JSON file describing your assertions plus the request body and active server URL. The format is documented and stable:

{
  "name": "Todos smoke tests",
  "serverUrl": "http://localhost:5006/openapi/v1.json",
  "protocol": "rest",
  "environment": { "todoId": "1" },
  "tests": [
    {
      "name": "Get todo by id",
      "service": "Todos",
      "method": "GetTodo",
      "messages": ["{\"id\": ${todoId}}"],
      "metadata": {},
      "assert": [
        { "path": "status", "op": "eq", "expected": "OK" },
        { "path": "response.id", "op": "eq", "expected": "1" },
        { "path": "response.title", "op": "exists" }
      ]
    }
  ]
}

Import collection loads a previously exported file back into the Tests tab — useful for sharing assertion suites across team members or restoring after a localStorage wipe.

bowire test — CI runner

The same JSON file is consumed by Bowire's command-line test runner:

bowire test ./api-smoke.json
bowire test ./api-smoke.json --report report.html

The runner:

  • Loads the collection file
  • Spins up the protocol registry in-process -- no HTTP detour, no need for a running Bowire server
  • For each test: discovers the service (REST/gRPC/SignalR/SSE/MCP), invokes the method, runs assertions
  • Prints colorized pass/fail output to stdout
  • Exits with code 0 when all tests pass, 1 when any test fails -- ideal for CI pipelines
  • With --report path.html, generates a self-contained HTML report (inlined CSS, dark theme matching the Bowire UI) for use as a CI artifact
  Bowire Test Runner   collection: api-smoke.json

  PASS  List all todos   OK · 23ms
        ✓ status eq OK
        ✓ response type array
  PASS  Get todo by id   OK · 3ms
        ✓ status eq OK
        ✓ response.id eq 1
        ✓ response.title exists
  FAIL  Get nonexistent todo   OK · 1ms
        ✗ status eq NotFound   actual: OK

  3/4 tests passed   6/7 assertions   in 234 ms

The CLI runner uses the exact same operators and substitution engine as the in-browser Tests tab, including system variables (${now}, ${uuid}, ${now+3600}, etc.). Collections are fully portable in both directions.

Environment variables in collections

Collections can declare a top-level environment map (and per-test override) so the same suite can run against Dev, Staging, and Prod with different values:

{
  "environment": {
    "baseUrl": "https://api.example.com",
    "apiKey": "sk-prod-..."
  },
  "tests": [
    {
      "service": "Users",
      "method": "GetUser",
      "metadata": { "Authorization": "Bearer ${apiKey}" },
      ...
    }
  ]
}

For CI, override per-environment values via env-specific collection files or by templating the JSON before invoking bowire test.

Data-driven runs

--data runs the collection once per row of a data file, with the row's columns as variables:

bowire test api-smoke.json --data users.csv --junit ./artifacts/junit.xml

userId,expectedName
1,Ada
2,Grace

A test then sends {"id": "{{userId}}"} and asserts response.name eq {{expectedName}}.

  • Formats. CSV: the header row names the columns; quotes, doubled quotes and line breaks inside quotes work. The separator (comma, semicolon or tab) is taken from the header. JSON: an array of objects; strings come across as they are, other values as their JSON text, and null as empty.
  • Precedence. A row's values shadow the collection's and the test's environment. A column a JSON row leaves out falls through to the environment.
  • Built-in variables. {{iteration}} (1-based) and {{iterationCount}} are set in every run, with or without --data; without it the run is iteration 1 of 1. bowire test reads {{name}} and ${name} alike.
  • Results. Every result names its iteration and row, for example Get user [iteration 137, users.csv line 138], on the console and in the HTML, SARIF and annotation output. JUnit gets one <testsuite> per iteration and one <testcase> per assertion, and a failure message names the row.
  • --fail-fast stops after the first iteration that has a failure. It is off by default, so a run reports every failing row.
  • A bad data file stops the run before the first request. An empty file, a header without rows, a row with the wrong number of values, an unclosed quote or a JSON item that is not an object each fail with exit code 2 and name the line or item.

--data drives a test collection. A flow or a --workspace run does not take a data set yet.

CI report formats

bowire test writes two complementary reports:

Flag Format Consumed by
--report path.html Self-contained HTML Humans / PR comments
--junit path.xml JUnit XML (<testsuites><testsuite><testcase>) Jenkins, GitLab CI, Azure DevOps, GitHub Actions test reporters

Both flags can be combined; either one alone is fine. Each Bowire test maps to one <testcase> (a --data run splits it into one per assertion, under one <testsuite> per iteration); assertion mismatches and invocation errors become <failure> children with a human-readable message and the diff detail inside.

bowire test api-smoke.json \
  --report ./artifacts/test-report.html \
  --junit  ./artifacts/junit.xml

The exit code is 0 only when every test passed, so CI gates work the same way they do for any unit-test runner.

What's missing

Tracked on the roadmap:

  • Pre-request scripts for dynamic setup (currently use the chaining feature instead)

Tips

  • Start with status — every method should at least assert status eq OK so you catch errors immediately
  • Mix with chaining — assertions and chaining share the same path engine, so once you've crafted a chaining placeholder you can copy the same path into an assertion
  • Use environment variables for expected values — keeps the assertion suite portable across stages
  • Don't over-test — assertions add friction. Pick a few high-value invariants (status, ID round-trip, count > 0) instead of hundreds of brittle field checks
Edit this page