Sensor data is irregular, noisy and arrives out of order. Six flow nodes turn it into something a
model, a dashboard or a CMMS can use, with no JavaScript:

| Node | What it does |
|------|--------------|
| [**window**](#window) | Aggregates a stream per key (machine, sensor) over tumbling, sliding or count windows: mean, RMS, kurtosis, percentiles, integral and 20 more. One record per window. |
| [**resample**](#resample) | Puts an irregular stream onto a fixed time grid (linear, previous or no fill, with a maximum gap). |
| [**rolling**](#rolling) | Adds moving mean / std / min / max over the last N records or the last N seconds to every record. |
| [**outlier**](#outliers) | z-score, IQR, MAD and Hampel tests over a rolling reference. Flag, drop, replace or route the odd values. |
| [**normalize**](#normalise) | z-score, min-max or centring with fixed training statistics or running estimates. |
| [**deadband**](#deadband) | Passes a record only when a value moved (report by exception) or a state changed. |

They work on both kinds of record, see [the data contract](/docs/data-contract):

- A **machine record** (it has a top-level `machine` object) stays a valid machine record. A field a
  node *creates* (rolling statistics, outlier flags, window aggregates) is written under
  `metadata`, never at the top level, so `filter`, `branch` and the sinks keep working. A window
  emits `{source, timestamp = window end, machine, status, axes: [], metadata: {window, <field>: {...}}}`.
- **Any other JSON** (webhook, ingest, an `http` response) is read and written at the paths you
  give; a window emits a plain object `{key, window, <field>: {...}}`.

Paths are dot paths: `metadata.vibration.x`, `axes.0.position`, `payload.sensors.temp`. Numbers may be
JSON numbers or text with a decimal comma (`"0,588399"`, `"1.234,5"`, `"1,234.5"`, the datalogger
convention); `NaN`, `inf`, empty text and `null` count as missing.

A typical vibration flow: source `ingest:vibration` -> **resample** (100 ms grid) -> **outlier** (Hampel, replace with
the median) -> **normalize** (the model's training mean/std) -> **window** (10 s: RMS, kurtosis, peak)
-> **filter** `metadata.x.rms > 3` -> **sink** `push`. The shipped example **Vibration: clean and
summarise a sensor stream** wires exactly this.

## Keys, time and durations

Every node that remembers something does it **per key**. `key` is a `${path}` template:
`${machine.id}`, `${machine.id}/${metadata.sensorId}`. Left out, a machine record's key is its
machine id and any other record shares one state. A record whose key cannot be built is dropped and
counted (`on_missing_key: fail` fails it instead).

Windowing and resampling use **event time**: the value at `time` (default `timestamp`, which a machine
record always has). It may be RFC 3339 text (`2026-03-01T10:00:00.5Z`), `YYYY-MM-DD HH:MM:SS`,
time of day `HH:MM:SS.f` (`time_unit: hms`) or epoch seconds, milliseconds, microseconds or
nanoseconds (`time_unit: auto` tells them apart by magnitude; set `s`, `ms`, `us`, `ns` or `iso` to
be explicit). A record without a readable time uses its arrival time (`on_bad_time: arrival`, the
default), or is dropped (`drop`) or failed (`fail`).

A duration is `"500ms"`, `"10s"`, `"2m"`, `"1h"`, `"1d"` (also `us`) or a plain number of seconds.

The per-node counters on the canvas gain a `drop` figure: records a data-preparation node dropped or
routed away on purpose (late, outlier, unusable, inside the deadband).

## Window

```json
{ "id": "w", "type": "window", "key": "${machine.id}",
  "mode": "tumbling", "size": "10s",
  "fields": [
"metadata.temp",
{ "path": "metadata.vibration.x", "as": "x",
  "aggregates": ["rms", "peak_to_peak", "crest_factor", "kurtosis", "p95"] }
  ],
  "aggregates": ["count", "mean", "min", "max"],
  "out_of_order": "2s", "allowed_lateness": "0s", "late": "drop",
  "idle_flush": "auto" }
```

| Option | Meaning |
|--------|---------|
| `mode` | `tumbling` (default): fixed windows `[start, start + size)`, epoch-aligned (`offset` shifts them). `sliding`: `size` long, one every `every`, overlapping. `count`: the last `size` records, one window every `every` records (`every` = `size` when left out). |
| `fields` | A path, or `{path, as, aggregates}`. The output name is `as`, else the last path segment (two fields must not end up with the same name). |
| `aggregates` | Aggregates of fields that do not list their own (default `count, mean, min, max`). |
| `emit_empty` | Also emit windows that received no record (`count` 0, other aggregates `null`), at most 1000 in a row. Default: empty windows are skipped. |
| `layout` | `nested` (default) `{x: {rms: 1}}` or `flat` `{x_rms: 1}` for SQL / CSV / ClickHouse sinks. |
| `max_records` | Most records buffered per key (default 10000). Beyond it the oldest are dropped and the window is flagged `truncated`. |
| `max_keys` | Most keys held (default 1000). The least recently used key is flushed (its open windows emitted `partial`) and removed. |

### Aggregates

Computed from the window's records, ordered by event time (ties in arrival order), two-pass, so
there is no streaming round-off.

| Name | Meaning |
|------|---------|
| `count` | numeric samples; `missing` counts the records without a usable number |
| `sum`, `min`, `max`, `mean` | as named |
| `var`, `std` | **sample** variance / deviation (divisor n-1; `null` below 2 samples) |
| `var_pop`, `std_pop` | population variance / deviation (divisor n). `std_pop` is the AC RMS (RMS with the mean removed) |
| `median`, `p50` ... `p99` (any `pNN`, 0 to 100) | quantile by linear interpolation between order statistics, position `(n-1)·p` (numpy's default) |
| `first`, `last` | by event time |
| `range`, `peak_to_peak` | `max - min` |
| `rms` | `sqrt(mean(x²))` of the raw values (DC offset included) |
| `peak` | `max(abs(x))` |
| `crest_factor` | `peak / rms` (`null` when rms is 0) |
| `kurtosis` | `m4 / m2²` with population moments: 3 for a normal distribution, the condition-monitoring convention. `kurtosis_excess` is `kurtosis - 3` |
| `skewness` | `m3 / m2^1.5` with population moments |
| `rate_of_change` (`rate`) | `(last - first) / (t_last - t_first)` per second |
| `integral` | trapezoid rule over the time-ordered samples, time in seconds |

For the sample `2, 4, 4, 4, 5, 5, 7, 9` one second apart: mean 5, `var_pop` 4, `std_pop` 2, `var`
32/7, median 4.5, p90 7.6, rms √29, peak 9, skewness 0.65625, kurtosis 2.78125, rate 1, integral 34.5.
Aggregates that are undefined (kurtosis of a constant signal, a rate from one sample) are `null`,
never `NaN`.

### The output record

```json
{ "source": "vibration-edge", "timestamp": "2026-03-01T10:00:10.000Z",
  "machine": { "id": "pump-1" }, "status": { "state": "running" }, "axes": [],
  "metadata": {
"key": "pump-1",
"window": { "start": "2026-03-01T10:00:00.000Z", "end": "2026-03-01T10:00:10.000Z",
            "start_ms": 1772359200000, "end_ms": 1772359210000, "count": 10,
            "mode": "tumbling", "size_ms": 10000 },
"temp": { "count": 10, "mean": 61.2, "min": 60.1, "max": 62.4 },
"x": { "rms": 0.41, "peak_to_peak": 1.9, "crest_factor": 3.1, "kurtosis": 2.9, "p95": 0.7 } } }
```

`window` also carries `every_ms` (sliding), `revision` (a re-emitted window, see lateness),
`partial: true` (emitted before the window was complete: idle flush or key eviction),
`truncated: true` (the buffer cap dropped records of it) and `suspect: true` (see
[State and restarts](#state-and-restarts)).

### Watermark, out-of-order and late records

Per key the **watermark** is the newest event time seen minus `out_of_order` and only moves forward.
A window is emitted, once, when the watermark reaches its end. So a record up to `out_of_order`
behind the newest one still lands in its window; the price is that every window is emitted
`out_of_order` later.

A record is **late** when every window it belongs to is already closed.

- `allowed_lateness: "5s"`: a late record whose newest window ended less than that before the
  watermark is still added, and each closed window containing it is **emitted again** with
  `revision` 1, 2, ... (downstream upserts on `window.start`). The buffer keeps the records needed for that.
- Anything later is dropped and counted (`late: "drop"`, the default), or sent out of a second port
  named `late` with `_late: {window_end, watermark}` added (`late: "port"`; wire the `late` port, the
  deploy refuses an unwired port).

Count windows have no event-time lateness; they take records in arrival order.

### Idle flush

Windows only close when a later record moves the watermark, so the last window of a sensor that
stopped would wait forever. `idle_flush` emits the open windows of a key that sent nothing for that
long (processing time), flagged `partial: true`:

- `"auto"` (default): twice `size + out_of_order`, at least 5 s; 30 s for count windows.
- A duration, or `"off"`.

The flushed windows are closed: a straggler for them is then late. The flush runs in the connector's
series driver once a second and only on the active node of an HA pair.

### Sliding and count windows

A sliding window `size: "60s"`, `every: "10s"` emits every 10 s the last 60 s; each record belongs to
`size / every` windows (at most 1000) and the buffer is shared, so memory does not grow with the
overlap. Windows are on the epoch grid, so the first ones overlap the start of the stream: a first
record at 10:00:12 belongs to the six windows starting 09:59:20 to 10:00:10, and those hold few records
(`window.count` says how many).

A count window `size: 100`, `every: 25` emits after every 25 records the last 100 (the first one
after 100). `window.start` / `end` are the event times of its first and last record.

## Resample

```json
{ "id": "rs", "type": "resample", "key": "${machine.id}", "interval": "100ms",
  "fields": ["metadata.vibration.x", "metadata.vibration.y", "metadata.vibration.z"],
  "fill": "linear", "max_gap": "2s", "align": "first" }
```

An irregular stream (a datalogger that writes in bursts with 1-second timestamps) onto a uniform grid,
with the semantics of the vibration edge's CSV replay (`csv_source::resample`): the grid is
`anchor + k · interval`, a grid time that equals a sample passes it unchanged, otherwise `linear`
interpolates between the two samples around it. A stream has no future, so the grid stops at the
newest sample and goes on when the next one arrives; one record is emitted per slot completed.

| Option | Meaning |
|--------|---------|
| `align` | `epoch` (default): grid times are multiples of `interval` (plus `offset`), so two sensors resampled separately share instants. `first`: the grid starts at the key's first sample, as the edge does. |
| `fill` | `linear` (default); `previous` holds the last sample; `none` synthesises nothing: only slots hit exactly by a sample exist, every other slot is a gap. |
| `max_gap` | Where the two samples around a slot are further apart than this, the slot is a **gap** whatever the fill (default: unlimited). A gap is skipped, or with `on_gap: "null"` emitted with null values and `_gap: true` (on a machine record only `metadata.*` fields can be null; other slots are skipped). |
| `reorder` | Hold samples back this long and sort them first. Default 0: input must be in time order. A sample at or before the last one used (late or duplicate) is dropped and counted. |
| `max_slots` | Most grid slots one sample may produce (default 100000); a longer stretch counts as a gap, so a one-hour hole on a 1 ms grid cannot flood the flow. |
| `on_missing` | Every listed field must be numeric in a record for it to be a sample. `skip` (default) drops the record, `fail` fails it. Fill gaps first with a cleansing step if you want them interpolated. |

The output record is a copy of the source record at or before the slot, with the listed fields
replaced and the slot time written at `time` in the format it came in (RFC 3339 stays RFC 3339,
epoch milliseconds stay epoch milliseconds). `timestamp`-less records get the grid time at the
`time` path.

With `reorder`, `idle_flush` (default twice `reorder`, at least 5 s) releases the held samples of a key that went quiet.

## Rolling

```json
{ "id": "r", "type": "rolling", "key": "${machine.id}",
  "fields": ["metadata.temp"], "records": 10,
  "aggregates": ["mean", "std", "min", "max"], "into": "rolling" }
```

Adds the statistics of the last `records` records (or the trailing `duration` of event time,
`(t - duration, t]`), the current record included, at `into.<field>.<aggregate>` (default `into` is
`rolling`; `layout: "flat"` writes `rolling_temp_mean`). Any aggregate of the table above works. The
record itself is unchanged and always passes. `min_periods` (default 1) nulls the statistics until the
window holds that many numeric samples; `count` and `missing` are always filled.

With `duration`, records may arrive out of order: each is computed against its own window, as long as it
is less than `duration` behind the newest record (the buffer keeps `2 · duration`).

## Outliers

```json
{ "id": "o", "type": "outlier", "key": "${machine.id}",
  "fields": ["metadata.temp"], "method": "hampel", "window": 30, "threshold": 3,
  "action": "replace", "replace_with": "median" }
```

Each value is judged against the **last `window` values** of its key and field (default 30), **not
counting itself**: a spike inside its own reference would inflate the very spread it is judged by (a
z-score in a window of n values can never exceed (n-1)/√n). Raw values enter the reference whatever the
verdict, so a level shift stops being an outlier once the window has caught up. With fewer than
`min_samples` (default 10) earlier values a record passes.

| `method` | Outlier when | Bounds |
|----------|--------------|--------|
| `zscore` | `abs(x - mean) > threshold · std` (sample std; `threshold` default 3) | `mean ± threshold·std` |
| `iqr` | `x < Q1 - k·IQR` or `x > Q3 + k·IQR` (`k` default 1.5; quartiles as in `p25` / `p75`) | the fences |
| `mad` | modified z `0.6745 · abs(x - median) / MAD > threshold` (default 3.5) | `median ± threshold·MAD/0.6745` |
| `hampel` | `abs(x - median) > threshold · 1.4826 · MAD` (default 3) | `median ± threshold·1.4826·MAD` |
| `range` | `x < min` or `x > max` (fixed limits, either may be left out, no reference) | the limits |

A flat reference (spread 0) makes every different value an outlier; `min_scale` is the smallest
spread, in the signal's units, that counts. The robust methods (`iqr`, `mad`, `hampel`) are not fooled
by the outliers already sitting in the reference; the z-score is.

| `action` | Effect |
|----------|--------|
| `flag` (default) | The record passes with `outlier: {flag, fields, scores}` written at `into` (default `outlier`) on every record, so `filter metadata.outlier.flag == true` works. |
| `drop` | A record with an outlier is discarded (counted). |
| `replace` | The outlier value is overwritten: `replace_with` `median` (of the reference, default), `limit` (the violated bound), `previous` (the last accepted value) or `null` (`metadata.*` only on a machine record). `outlier: {flag, fields, scores, replaced}` is added, on records that had an outlier only. |
| `route` | A record with an outlier leaves through a second port named `outlier` (with the info above), the others through `out`. Wire both. |

A record is an outlier when any listed field is.

## Normalise

Fixed parameters, for example the training statistics of a model. The vibration edge's
`models/inference_config.json` stores `norm_mean` and `norm_std` per channel:

```json
{ "id": "n", "type": "normalize", "method": "zscore",
  "fields": ["metadata.vibration.x", "metadata.vibration.y", "metadata.vibration.z"],
  "mean": [0.006168725434690714, 0.0084125567227602, 0.008863953873515129],
  "std":  [0.17158810794353485, 0.21506984531879425, 0.174563929438591] }
```

`mean`, `std`, `min`, `max` are arrays aligned with `fields` (paste them from the training config), or
are set per field (`{"path": "...", "mean": 0.1, "std": 0.2}`). Running estimates per key instead:
`"mode": "running"`.

| Option | Meaning |
|--------|---------|
| `method` | `zscore` `(x - mean) / std`; `minmax` `lo + (x - min) / (max - min) · (hi - lo)` into `range` (default `[0, 1]`, `clip: true` clamps); `center` `x - mean` (AC coupling). |
| `mode: running` | Welford estimates per key (population std, as numpy's default, which training pipelines store). `alpha` (0 to 1) makes them exponentially weighted so old data is forgotten; the current value is part of the estimate. Needs `min_samples` values (default 2); until then `on_warmup` drops the record (default) or passes it unscaled. |
| `as` (per field) | Write the result to another path and keep the original. |
| `on_missing` | A field that is missing or not numeric stays as it is (`pass`, default), drops the record or fails it. |

A zero spread gives 0 (z-score) or the low end of the range (min-max) instead of a division by zero. A
fixed z-score with `std <= 0` is refused at deploy.

## Deadband

```json
{ "id": "db", "type": "deadband", "key": "${machine.id}",
  "fields": [{ "path": "metadata.temp", "absolute": 0.5 },
         { "path": "metadata.load", "percent": 2 },
         "status.state"],
  "heartbeat": "60s" }
```

A numeric field moved when it differs from the last value that was let through by at least `absolute`,
or `percent` of that value's magnitude (either is enough, the same rule as the push path's
`preprocess.deadband`; an unchanged value never passes). A field with neither threshold is **change
detection**: any difference counts, for numbers, text and booleans. Node-level `absolute` / `percent`
are the defaults of the fields that set none.

`mode: any` (default) passes the record when any field moved, `all` when every one did. The first record
of a key passes (`emit_first`). With `heartbeat` a record passes anyway after that long without one, so a
quiet but alive sensor is still seen. A passing record is unchanged and becomes the new reference of
every field; the others are dropped (counted).

## State and restarts

Windows, rolling buffers, outlier references, running normalisation estimates, the resample grid and
the deadband reference live in memory **per key**, bounded: `max_keys` (default 1000, least recently
used evicted), `max_records` per key (window, rolling and resample reorder buffers). The state of a node
is kept across a redeploy as long as what it computes is unchanged (window size, fields, key, ...);
changing that starts the node empty and the open windows of the old state are discarded.

With a data dir the state is persisted in `series.db` (`persist: false` keeps it in memory only):

- changed keys are written once a second; a graceful stop writes everything and marks the file clean;
- a **graceful restart** loses nothing: open windows continue, and a key that stayed quiet through the
  downtime is idle-flushed right after start;
- after a **crash** (kill -9, power loss) up to the last second of records is missing from the restored
  state; windows that were open at the last write are emitted with `window.suspect: true` (count can be
  lower than it should) and the log says so;
- a node removed from the flow deletes its state at the first idle tick that finds it gone; a key unseen
  for 7 days is deleted;
- rows hold the buffered values (not the whole record, except `resample`, which keeps the previous
  record as its template). They are not encrypted by the master key.

The window output is at-least-once: a crash between emitting a window and writing the state can emit it
again after the restart. Downstream, key on `window.start` + `key`.

## Dry runs

Test runs (`/api/flow/test`, the editor's Test) never touch live state: a throwaway state is used and a
single record seldom closes a window, so a window test shows the record buffered. Use the sandbox with
several records, or deploy and watch the node counters.