Messy plant and business data has two problems: you do not know yet what is wrong with it, and
once you know, you want the flow to say so loudly instead of passing it on. OpexFlow has one tool
for each:

| Tool | Question it answers |
|------|---------------------|
| **Profile** (editor, `GET /api/flow/nodes/{id}/profile`, MCP `profile_node`) | What do the records at this node look like, field by field? |
| **`quality` node** | Are the values acceptable? Warn, park or drop the ones that are not, and count every broken rule. |

## Profile a node

Open a deployed flow, select any node and open **Profile** in the inspector. The editor arms a
bounded sampler on that node and refreshes the view every two seconds while it is open. Nothing is
sampled for nodes nobody profiles: with the sampler off a record costs one atomic read.

**What is sampled.** The records the node receives (**In**) or emits (**Out**, the default; a sink
defaults to In), after the privacy masker. Two buffer modes, up to 1000 records:

| Mode | Buffer | Use it for |
|------|--------|------------|
| **Recent** (default) | the newest `size` records | what is flowing now |
| **Sample** | a uniform random sample of everything since arming | a long, even picture of a busy stream |

At most 8 nodes are sampled at once, one record is skipped when it is bigger than 64 KB, each node
holds at most 4 MB, and a sampler nobody asked about for 10 minutes switches itself off and frees
its memory. Nothing is stored on disk: a restart starts with every sampler off.

**What you see, per field** (nested objects are flattened to dot paths such as
`metadata.raw.rpm`; arrays are one value):

| Figure | Meaning |
|--------|---------|
| Type mix | integer, float, boolean, string, **numeric_string** (text such as `12,5` or `1.234,5` that reads as a number), timestamp, array, object. |
| Missing % | absent + `null` + blank text, as a share of the records. |
| Distinct | different scalar values in the buffer. |
| Numbers | min, max, mean, standard deviation (sample, n − 1), median, a 10-bin histogram and a sparkline of the values in arrival order. Number-like text is included and counted separately, so a column of `"12,5"` strings still shows its range. |
| Top values | the five most frequent values with their share (not for float columns, the histogram covers those). |
| Examples | up to three different values. |
| Timestamps | earliest and latest. |

Rows are badged when something looks wrong: missing values, mixed types, number-like text, a
constant column, a column that is always missing.

Decimal-comma handling is the same in the profile and in the `quality` node: with both separators
the last one is the decimal mark (`1.234,5` and `1,234.5` are 1234.5), a single comma is a decimal
comma (`1,234` is 1.234), several of the same separator group thousands (`1.234.567`), and `NaN`
or `inf` are not numbers.

### API and MCP

```bash
# arm the sampler and get the profile (waits up to wait_ms for the first record)
curl -X POST -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"phase":"out","mode":"recent","size":500,"wait_ms":5000}' \
  http://127.0.0.1:3000/api/flow/nodes/clean/profile

# read it again (also keeps the sampler armed)
curl -H "Authorization: Bearer $TOKEN" http://127.0.0.1:3000/api/flow/nodes/clean/profile

# stop and free the buffer
curl -X DELETE -H "Authorization: Bearer $TOKEN" http://127.0.0.1:3000/api/flow/nodes/clean/profile
```

Reading (`GET`) needs the **viewer** role like the debug socket; starting and stopping the sampler (`POST`, `DELETE`, and the MCP tool) needs **operator**. The data is masked the same way as in the debug feed. The
response lists `fields[]` with the figures above plus `records` (in the buffer), `seen` (offered
since arming) and `armed`. The MCP tool `profile_node {node_id, phase?, mode?, size?, reset?,
wait_ms?, stop?}` does the same for an agent. An unknown node is 404, a ninth sampled node is 429.

## The `quality` node

`validate` checks a record against the **JSON Schema** of a catalogue event type: the shape and
types of a published contract ([business events](/docs/business-events#validate-node)). `quality`
is about the **values**: is the temperature plausible, is the timestamp fresh, did the counter go
backwards, is the batch id unique, did a machine stop reporting. Use `validate` at the edge where
data enters, `quality` wherever the numbers matter. They combine: `validate` first for shape, then
`quality` for value rules.

```json
{
  "id": "gate", "type": "quality",
  "rules": [
{ "check": "not_null", "field": "machine.id", "severity": "fail" },
{ "name": "rpm_range", "check": "range", "field": "metadata.raw.rpm", "min": 0, "max": 6000 },
{ "check": "fresh", "field": "timestamp", "max_age": "10m", "max_future": "5s", "severity": "drop" },
{ "check": "monotonic", "field": "metadata.counter", "by": "machine.id" },
{ "check": "compare", "field": "end", "op": ">=", "other": "start" }
  ]
}
```

Fields are dot paths into the record, a machine record (`machine.id`, `status.state`,
`metadata.raw.rpm`) or free JSON (`order.total`, `lines.0.sku`). The node only ever adds the
top-level `_quality` key, which `filter`, `branch` and the sinks ignore, so it works on both shapes
of record and never breaks the nodes after it.

### Severity

| `severity` | A record that breaks the rule |
|------------|-------------------------------|
| `warn` (default) | continues, with `_quality: {ok: false, issues: [{rule, check, field, severity, message}]}`; a second `quality` node appends to the same list |
| `fail` | fails the node: parked in [Dead letters](/docs/operations#dead-letters-correct-and-replay) with every failing rule's message, or sent down the node's `error` edge when one is wired |
| `drop` | discarded here, counted in the node's `drop` figure |

Every rule is always evaluated, so one record reports all it breaks. `fail` wins over `drop` when a
record breaks both. A record that breaks nothing is passed on untouched.

### Missing values

`null`, an absent field and blank text count as missing. Every rule except `not_null` **skips** a
missing value, so a range rule does not also complain about a field that a `not_null` rule already
covers. Add `"required": true` to make a missing value a violation of that rule. `row_count` has no
field and does not take `required`.

### Rules

| `check` | Options | Passes when |
|---------|---------|-------------|
| `not_null` | `field`, `allow_empty` | the value is present and not blank (`allow_empty: true` accepts blank text) |
| `type` | `field`, `is` | `is` is `string`, `number` (a JSON number), `integer` (no fraction, `5.0` counts), `boolean`, `object`, `array`, `timestamp` (RFC 3339, `YYYY-MM-DD[ T]HH:MM:SS`, `YYYY-MM-DD`, epoch seconds or milliseconds) or `numeric` (a number or number-like text) |
| `range` | `field`, `min`, `max` (inclusive), `gt`, `lt` (exclusive) | the number, or text like `12,5`, is within the bounds |
| `regex` | `field`, `pattern` | the text matches (unanchored: use `^…$`); numbers match as their text |
| `allowed` | `field`, `values` | the value equals one of `values` (numbers by value, text exactly) |
| `unique` | `field`, `window` (default `10m`), `by` | the value was not seen in the last `window` (per `by` value). Exactly one window apart is not a duplicate. |
| `fresh` | `field` (default `timestamp`), `max_age`, `max_future` | the timestamp is at most `max_age` old and at most `max_future` ahead of the clock (each optional, one required) |
| `monotonic` | `field`, `direction` (`increasing` default, `decreasing`), `strict`, `by` | the number or timestamp never goes the wrong way compared with the previous record (per `by` value); `strict` also refuses a repeat. After a reset the new level is the baseline: reported once. |
| `row_count` | `min`, `max`, `window` (default `1m`), `by` | records per tumbling window are within bounds. `max` flags the record that exceeds it. `min` is judged when the window closes, on the next record, so it can only `warn` (deploy error otherwise); silent windows between two records are reported with the next one. |
| `compare` | `field`, `op` (`==` `!=` `>` `>=` `<` `<=`), `other` (a field) or `value` (a literal) | numbers (or number text) compare as numbers, timestamps as time (zones honoured); text only supports `==` / `!=` |

Durations are `500ms`, `30s`, `10m`, `2h`, `1d` or a plain number of seconds. Every rule can have a
`name` (used in issues and stats); the default is `{field}.{check}` with `#2`, `#3` for repeats.
A misspelt or irrelevant option (`max` on a regex rule) is a deploy error that names the rule.

Windows (`unique`, `row_count`) use the **processing clock** (when the node sees the record), so
late or out-of-order records do not disturb them; `fresh` judges the record's own timestamp, so a
late record is judged on how old it is.

### Counting, stats and Health

Violations are counted per rule: the node's stats carry `rules: {name: count}` (shown next to each
rule in the editor, `GET /api/status`, MCP `get_node_stats`, the live socket). Health counts
records checked and records with at least one violation: `GET /api/health/report` shows
`counters.quality_checked_total`, `counters.quality_violated_total` and
`data_quality.rule_violation_pct`.

### State and restarts

Unique values, the last monotonic value and the window counts live in memory, capped at 10 000
entries per rule (the oldest unique value is evicted first, a `by` value beyond the cap is not
tracked). A restart or a deploy that rebuilds the node empties them. That can only miss a duplicate
or a regression right after the restart, never invent one, and the first `row_count` window is never
judged because it may be partial. A node used by several sources keeps one state per source.

### Example

`plant-quality-gate` in the [Examples gallery](/docs/examples): a machine feed with a `not_null`
that parks anonymous records, a range and an allowed-states rule that annotate, a monotonic
counter, and the sample record shows its `_quality` issues in a dry run.