OpexFlow keeps credentials out of flows and config, masks personal data wherever it is shown,
and encrypts what it stores. Three independent pieces, one shared key:

| Piece | What it does | Config |
| --- | --- | --- |
| [Secrets](#secrets) | `${secret:name}` references resolved from Vault/OpenBao or a local encrypted store | `secrets:` |
| [Masking](#masking-personal-data) | Personal fields replaced by `***` in traces, debug, dead letters, event log, job log, samples | `privacy.mask`, `x-sensitive` |
| [Encryption at rest](#encryption-at-rest) | AES-256-GCM on the payload columns of the SQLite stores | `OPEX_MASTER_KEY` |

## Master key

`OPEX_MASTER_KEY` is the one secret the connector itself needs. It is any string of at least 16
characters; generate a strong one with:

```bash
opex-flow secret keygen        # prints 64 hex characters
```

The connector reads it, in this order, from the environment variable `OPEX_MASTER_KEY`, from the file
named by `OPEX_MASTER_KEY_FILE` (Docker / Kubernetes secrets), or from the systemd credential
`OPEX_MASTER_KEY` (`$CREDENTIALS_DIRECTORY/OPEX_MASTER_KEY`):

```ini
# /etc/systemd/system/opex-flow.service.d/key.conf
[Service]
LoadCredential=OPEX_MASTER_KEY:/etc/opex-flow/master.key
# or, bound to the host's TPM2 / machine id:
# LoadCredentialEncrypted=OPEX_MASTER_KEY:/etc/opex-flow/master.key.cred
```

It is never written to disk by the connector. Sub-keys for data at rest, the secrets file and
dedupe keys are derived from it (HKDF-SHA256), so none of them can be used in place of another.
**Back the key up separately from the data directory.** Without it the encrypted stores and
`secrets.enc` cannot be read; the connector refuses to start on encrypted data without the key
rather than silently ignoring it. A different key than the one the data was written with is
refused the same way (the error names the key id, which is also logged at startup).

Key rotation is not automatic: to change the key, stop the connector, export what you need
(dead letters you still want to replay, secrets via `opex-flow secret list` and your records), and
start with the new key on a fresh data directory.

## Secrets

A flow or config never holds a credential. It holds a reference, resolved when it is used:

```text
${secret:erp_token}                    # local store first, then Vault (see secrets.order)
${secret:local:erp_token}              # the local encrypted store only
${secret:vault:erp/netsuite#token}     # Vault / OpenBao KV v2: path erp/netsuite, field token
```

A secret reference works wherever `${ENV}` does:

- **Config values**: any string, resolved when the config loads (`connector.hmac_secret`,
  `connector.api_token`, sink passwords, SMTP credentials, ...).
- **http node `auth`**: the secret fields (`token`, `password`, `client_secret`, `consumer_*`, `token_*`)
  take a single `${secret:name}` instead of `${ENV}`; a literal is still refused on Deploy.
- **http node templates**: `url`, `headers` and `body`.
- **Signed webhooks**: `webhook.signature.secret`, resolved at deploy.

In an http node, a secret must also be admitted by `transform.executor.http_env_allowlist`, which
takes `secret:` entries beside environment names: `secret:erp_token`, `secret:erp_*` or `secret:*`.
A secret nobody admitted is a deploy/run error, the same deny-by-default as environment variables.

### Local store

`<data_dir>/secrets.enc` holds the secrets, encrypted with a sub-key of `OPEX_MASTER_KEY`
(file mode `0600`). Manage it with the CLI, or in the UI under **Account menu, Secrets** (admin):

```bash
printf %s "$TOKEN" | opex-flow secret set erp_token     # value from stdin, never from the command line
opex-flow secret list                                   # names and change times, never values
opex-flow secret rm erp_token
```

`opex-flow secret ...` works on `OPEX_DATA_DIR`, else on the directory of `--config` (default
`./config.yaml`); pass `--data-dir` to be explicit. A running connector notices a changed file within
a second. Values are write-only: `GET /api/secrets` lists names, `PUT /api/secrets/{name}` with
`{"value": "..."}` stores one, `DELETE` removes one (admin role, audited, request bodies never logged).
Writing needs the master key (`409` without it).

### Vault / OpenBao

```yaml
secrets:
  cache_ttl_secs: 300            # how long a Vault value is cached (0 = never)
  order: [local, vault]          # lookup order of a bare ${secret:name}
  vault:
addr: https://vault.example.com:8200
mount: secret                # KV v2 mount (default "secret")
prefix: opex-flow/           # prepended to every path
namespace: ops               # Enterprise namespace, optional
# token auth ...
token: ${VAULT_TOKEN}        # or token_file: /run/secrets/vault-token (Vault Agent sink)
# ... or AppRole
# role_id: 7f3a...
# secret_id: ${VAULT_SECRET_ID}
# approle_mount: approle
ca_cert: /etc/opex-flow/vault-ca.pem   # private PKI, optional
```

The `secrets:` section itself accepts `${ENV}` only (it configures the providers). A reference
`${secret:vault:erp/netsuite#token}` reads `secret/data/opex-flow/erp/netsuite` and returns its field
`token`; without `#field` the field `value` is used, or the only field of the secret. An AppRole token
is renewed at 80 % of its lease and once more on a `403`.

### What a secret never does

A resolved value is never logged and never returned by an API. Every value the connector resolves is
remembered in a redaction set and scrubbed (`***`) from error texts, traces, dead letters, alerts, debug
events, dry-run results and the audit log. `GET /api/config` shows the file as written (references,
not values) with secret-looking keys redacted as before. `GET /api/flow`, exports and MCP drafts only
ever contain the reference.

## Masking personal data

Mask by field name in the config, and tag fields in the event catalogue:

```yaml
privacy:
  mask:
- "@pii"                # preset: *email*, *phone*, *mobile*, nik, ktp, *passport*, date_of_birth,
                        # full_name, *account_no*, address, password, ...
- "@pii-values"         # preset: e-mail addresses, card-like and 16-digit numbers inside any string
- customer.tax_id       # a key `tax_id` below a key `customer`, at any depth
- "*voucher*"           # * wildcards, case-insensitive
- "regex:ORD-\\d{8}"    # a value pattern, replaced inside any string
```

A name without a dot matches that key at any depth (arrays are transparent); a dotted name matches a
path suffix. A matching value (object, array or scalar) becomes the string `***`. There is no
substring match unless you write the `*`: `nik` does not touch `technician`.

Event types mark personal fields in their JSON Schema with `"x-sensitive": true`:

```json
{ "properties": { "email": { "type": "string", "format": "email", "x-sensitive": true } } }
```

The shipped types tag their personal fields (`guest.updated`: name, e-mail, phone, nationality, birth
date; `vendor.updated`: tax id, e-mail, bank account number and holder; `employee.updated`: name,
e-mail; `feedback.received`: comment). Inside an event envelope of that type, those fields are masked
wherever the event is shown. The [AsyncAPI export](/docs/business-events) carries the tags:
`x-sensitive` stays on the property, and every message has `x-sensitive-fields` (`data.email`, ...)
and a `personal-data` tag; `GET /api/events` returns `sensitive_fields` per type.

Masking is applied where data is **shown**:

| Where | Masked |
| --- | --- |
| Trace entries (`/api/pipeline/trace*`), per-node `last_error`, alert texts | error texts (value patterns, secrets) |
| Debug socket (`/api/ws/flow`), MCP `watch_debug` | node payloads and errors |
| Dead letters (list, detail, edit/discard responses, audit trail) | value, origin, edits, error |
| Event log (`/api/events/log`) and live feed (`/api/ws/events`) | event data |
| Printer job log (`/api/printers/jobs`, ingest replies) | label fields, detail |
| `last-record`, `flow/test` results, AI generator samples, MCP tools | the whole record |

The record a dead letter **stores** stays whole (encrypted at rest), so a replay still sends the real
data. When an operator edits a dead letter, a field that still reads `***` keeps its stored value, so
fixing one field never overwrites a personal one with the mask; to really change a masked field, send a
value other than `***`. Dry runs started from a masked `last-record` carry `***`: leave `record` out of
`POST /api/flow/test` to run on the real last record and read the (masked) result. A change to
`privacy.mask` applies on the next config load (`POST /api/config/reload`).

Not masked: the `subject` and `correlation_id` columns used to search the event log are stored and
filterable as written (put identifiers there, not personal data), and `GET /api/history` returns
machine telemetry (no personal data by design).

## Encryption at rest

With `OPEX_MASTER_KEY` set, these columns are stored as `enc1:<key id>:<base64>` (AES-256-GCM, a random
nonce per value, the table and column bound as associated data):

| Store | Encrypted |
| --- | --- |
| `pipeline_traces.db` | trace entries; dead-letter value, origin, edits; dead-letter audit detail |
| `events.db` | the event envelope |
| `push_queue.db` | every queued and dead-lettered payload |
| `printer_jobs.db` | label fields, counter, detail |
| `data.db` | the history record (`data_json`) |
| `dedupe.db` | keys become keyed digests (HMAC-SHA256), no identifier is stored |
| `secrets.enc` | the whole local secrets file |

Rows written before the key was set are encrypted in place at the next start (the log line
`Encrypted N existing row(s)` names each column), so enabling it needs no export or import. A
database that already holds encrypted rows is **not opened** without the matching key.

Without a master key the stores stay plaintext and the connector logs a warning at every start:

```text
WARN OPEX_MASTER_KEY is not set: dead letters, event history, the push queue, job logs and stored history
 are written UNENCRYPTED, and the local secrets file is unavailable. ...
```

What stays outside column encryption: indexed columns used for lookups (device id, timestamps,
event type, subject, correlation id, source, dead-letter error text), `reconcile.db` (ledger keys and
amounts), `lookups.db`, `flow.json`, the event catalogue files and `license.json`. None holds
credentials (those are references) and the personal-data fields of events are encrypted in
`events.db`, but a stolen disk would show them. **Where that matters, combine this with full-disk
encryption** of the data directory (LUKS/dm-crypt on Linux, BitLocker on Windows, an encrypted
volume or cloud-provider disk encryption for containers), which covers every file including SQLite
WAL files and temporary pages. The two layers are independent: disk encryption protects a lost
disk; column encryption also protects database copies, backups and anyone who can read the data
directory but does not hold the key.

Back up with `sqlite3 data.db ".backup out.db"` as before: backups stay encrypted.

Encryption in transit: put TLS in front of the API (reverse proxy), use `https://` endpoints, and
`postgres.sslmode: verify-full`; see [Security](/docs/security).