OpexFlow runs in two places at once: an **edge** node in the plant (it polls the machines and keeps working when the WAN is down) and a **hosted** node in the cloud (it receives records from every plant, runs the business flows, and lets people work in the UI from anywhere). Both are the same binary. This page is the supported recipe for the hosted node and for the link between the two. The files are in `deploy/hosted/` of the repository. ``` plant cloud (one VPS / VM) ┌─────────────────────┐ HTTPS, out ┌──────────────────────────────────────────────┐ │ edge OpexFlow │ signed POST │ Cloudflare: Access (SSO) in front │ │ adapters + flow ├────────────────►│ /api/hooks/edge Bypass (HMAC is the auth) │ │ push_queue.db │ │ cloudflared ──► opex-flow :3000 ──► /data │ └─────────────────────┘ └──────────────────────────────────────────────┘ no inbound port no published port, no inbound firewall rule ``` ## Why a VPS and not Workers or Containers OpexFlow is a native binary whose state is SQLite files (`data.db`, `push_queue.db`, `flow.json`, `versions.db`, ...). - **Cloudflare Workers cannot run it**: no native process, no local filesystem. - **Cloudflare Containers have an ephemeral disk**: every restart or redeploy would start from an empty `/data` and lose the history, the flow versions and the queue. So the hosted target is a VPS or VM with a **persistent volume** (any provider; x86_64 or arm64). Cloudflare is still the front door: a Tunnel and Access. ## Start the hosted node On the VPS (Docker Engine with the compose plugin; it only needs outbound HTTPS): ```bash cd opex-flow/deploy/hosted cp .env.example .env && chmod 600 .env # OPEX_API_TOKEN, OPEX_MASTER_KEY, TUNNEL_TOKEN, EDGE_HMAC_SECRET docker compose up -d --build docker compose ps # opex-flow (healthy), cloudflared (running) ``` The image is built from the **published static release**, so there is no compiler and no private token on the VPS. Pin the release and verify its digest for production: `OPEX_FLOW_VERSION=2.1.0 OPEX_FLOW_SHA256=` (the build prints the digest it downloaded and fails on a mismatch). To use an image built from source, see the header of the repository `Dockerfile` and set `OPEX_FLOW_IMAGE`. What the container is: non-root user `app` (uid 10001), state in the `/data` volume, read-only root filesystem, no capabilities, `HEALTHCHECK` on `GET /api/ready`, SIGTERM drains the queues for up to 30 s (`stop_grace_period: 45s`). Environment overrides: `OPEX_API_TOKEN`, `OPEX_MASTER_KEY` (encrypts data at rest and the local secret store; keep a copy elsewhere), `OPEX_DATA_DIR`, `OPEX_BIND_ADDR`, and every `${VAR}` of the config file. A node fed only by webhooks or the ingest API has no adapters. It is **ready** as soon as its pipeline engine is up, and it needs no `push_endpoint`: leave it unset and the `push` sink is skipped (`hosted.yaml` does exactly that). Anything that pushes upstream (`backfill`, `batch`) still needs one. ### The intake flow Deploy `flow.hosted.json` once (it is kept in the volume): ```bash jq -n --slurpfile f flow.hosted.json '{flow: $f[0], message: "edge intake"}' | curl -fsS -X PUT https://flow.example.com/api/flow \ -H "Authorization: Bearer $OPEX_API_TOKEN" -H 'Content-Type: application/json' --data @- ``` It is an ordinary flow, editable on the canvas: | Node | What it does | | --- | --- | | `webhook:edge` source | Verifies `X-Opex-Signature` = `sha256=` HMAC-SHA256 over `{timestamp}.{body}` with `${EDGE_HMAC_SECRET}`, timestamp within 300 s (the pusher's own signing scheme). A bad signature is a `401` and never routed. | | `dedupe` | Key = edge key id + machine + timestamp, 2 days: a record sent twice is stored once. | | `branch` | `status.state == "alarm"` takes the alarm port. | | `transform` (DSL) | Tags `metadata.hosted_route` = `alarm` / `normal`. | | `sink` | `storage` (history in `data.db`); alarms also go to the live feed (`hub`). | Swap the sinks for `http` nodes to deliver to an ERP or an iPaaS (see [Connectors](/docs/connectors)). ## Cloudflare Tunnel and Access 1. **Tunnel.** Zero Trust, Networks, Tunnels: create a tunnel, copy its token into `TUNNEL_TOKEN`, add the public hostname `flow.example.com` pointing at `HTTP` `opex-flow:3000` (the compose service name). 2. **Access for people.** A self-hosted application on the hostname with an *Allow* policy for your identity provider group: SSO and MFA happen at Cloudflare before a request reaches the VPS. 3. **Bypass for the edge push.** A second self-hosted application on `flow.example.com/api/hooks/edge` with action **Bypass** (Cloudflare applies the most specific matching application). The hook has no bearer token and is authenticated by its HMAC signature alone, so nothing else is exposed. If you prefer Access on that path too, use *Service Auth* with a service token and set `connector.push_headers` on the edge (`CF-Access-Client-Id`, `CF-Access-Client-Secret`). 4. **No inbound ports.** The compose file publishes nothing and `cloudflared` dials out, so the origin is reachable only through the tunnel and Access cannot be bypassed by hitting the VPS address. **Identity in the app.** Cloudflare authenticates the browser; OpexFlow still authorises it with a bearer token or its own OIDC login, with roles (`viewer` to `admin`), a session cookie and an audit log. Configure `auth.oidc` against your IdP to avoid a second prompt (see [Security](/docs/security#users-roles-and-single-sign-on)); `OPEX_API_TOKEN` stays the break-glass admin credential. ## The edge-to-hosted link The cleanest outbound-only link uses what OpexFlow already has: the edge's `push` sink and the hosted node's signed webhook source. No extra component, nothing listens at the plant. ```yaml # edge.yaml (plant) connector: name: "plant-sg-01" push_endpoint: "${HOSTED_URL}/api/hooks/edge" hmac_secret: "${EDGE_HMAC_SECRET}" # same value as on the hosted node hmac_key_id: "plant-sg-01" # push_headers: # only with Access Service Auth on that path # CF-Access-Client-Id: "${CF_ACCESS_CLIENT_ID}" # CF-Access-Client-Secret: "${CF_ACCESS_CLIENT_SECRET}" push_queue: { enabled: true, max_quota_mb: 2000, on_full: evict_oldest } ``` `flow.edge.json` sends every record of the plant adapters to local history and to the `push` sink. What each end guarantees: | Situation | Behaviour | | --- | --- | | Normal | The record is written to `push_queue.db` first, then sent as one signed POST; the hosted flow answers `200` after it ran (`respond: on_done`). | | Hosted side or WAN down | Records queue on the edge disk, in order, up to `max_quota_mb`; sending retries with exponential backoff, `Retry-After` honoured. The edge restarting keeps the queue. | | Link back | The queue drains by itself, oldest first. | | Response lost after the hosted side stored the record | The record is sent again (at-least-once); `dedupe` makes the repeat harmless. | | Wrong secret / `401`, `403` | Nothing is lost: the records stay queued and `GET /api/push/status` shows `last_error`. Fix the secret and the queue drains. | | Other `4xx` (the hosted flow refused the record) | The row moves to the push dead letters (`GET /api/push/dead`, replay with `POST /api/push/dead/replay`). | | Queue at quota | `evict_oldest` drops the oldest records and counts the gap (alert at 80 %); `block` refuses new records instead (for business flows). | | Captured request replayed later | Refused: the signature covers a timestamp, the hook rejects more than 300 s. Each retry is re-signed, so a record that waited a day is still accepted. | This was run end to end with two local instances (edge and hosted, own data directories). Records injected on the edge (`POST /api/ingest/edge/{machine}`) arrived and were routed (`alarm` records tagged `alarm`, others `normal`). With the hosted instance stopped, five more records were injected and queued on the edge (`queued=4`, then `5` after the edge itself was restarted, with `last_error: transport error`), and once the hosted instance was started again the queue drained to `0` and every record was in the hosted history, none dead-lettered. `deploy/hosted/check-link.sh` repeats the first half against your own pair, and the test suite (`api/hosted_link.rs`) pins the outage, the redelivery, a wrong secret and a replay. > **Info** > > The `push` sink carries machine telemetry (`CanonicalMachineData`). For **business records** from the plant, use an `http` > node with `auth: bearer` and a named API token (role `operator`) to `POST /api/ingest/{source}/{id}` on the hosted node (add > the Access service-token headers if Access protects it). That path has no store-and-forward of its own: the record is > only as durable as its source (NATS redelivers, a schedule does not). ## Operating it **Sizing.** 1 vCPU, 1 GB RAM and 20 GB disk is a starting point, not a measured limit. The published 2.1.0 binary idles at about 15 MiB RSS in a container. Disk grows with records: 2,008 records with one axis, a spindle and two metadata values took about 0.43 KB each in `data.db` and 0.17 KB in `dedupe.db`, so plan `records per day × 0.45 KB × storage.retention_days` and cap it with `storage.max_size_mb`. Load-test your own rate: the 2,000-record smoke run (a debug build driven by a `curl` loop, 24 s, no backlog) is not a capacity figure. **Backups.** Never copy a live `data.db`. `deploy/hosted/backup.sh` snapshots every database with `sqlite3 .backup` (online, consistent), copies the small state files, and keeps the newest 14 timestamped directories (`KEEP`). The systemd timer in `deploy/hosted/systemd/` does the same without Docker. Ship the directory off the host. `OPEX_MASTER_KEY` is not in the backup and must be stored separately, or restored data is unreadable. To restore, stop the stack, copy the backup into the volume, start (`deploy/hosted/README.md`). A restore of such a backup came up with the same flow version and all of its records. **Upgrades.** Take a backup, change `OPEX_FLOW_VERSION` (or `OPEX_FLOW_IMAGE`) and run `docker compose build opex-flow && docker compose up -d opex-flow`. The old container drains on SIGTERM and the new one opens the same volume. Run one version per volume; to roll back, re-pin the old version, and restore the backup if the newer one had already changed a database. **Without Docker.** `deploy/hosted/systemd/` has a drop-in for the shipped `deploy/opex-flow.service` (`opex-flow-hosted.conf`: master key as a systemd credential, loopback bind, no device access), a hardened `cloudflared.service`, and the backup service and timer. Point the tunnel's public hostname at `HTTP` `localhost:3000`. **Monitoring.** `GET /api/ready` for the container, `GET /api/push/status` on the edge (`queued`, `consecutive_failures`, `oldest_age_secs`), `GET /api/status` → `engine.webhooks[]` on the hosted node (`received`, `bad_signature`), plus `/metrics` and the SLA dashboard as on any instance ([Monitoring and reports](/docs/monitoring-and-reports)). ## OpexFlow next to an iPaaS An iPaaS (Boomi, Workato, MuleSoft, n8n Cloud, Make, ...) and OpexFlow solve different parts of the same problem. The iPaaS is strong where the work is between SaaS applications, with a catalogue of prebuilt connectors and tooling for people who are not developers. OpexFlow is strong at the plant boundary (Modbus, OPC-UA, EtherNet/IP, FOCAS, printers, local store-and-forward) and as a self-hosted business flow engine with delivery guarantees. OpexFlow does not ship a catalogue of hundreds of SaaS connectors: if the business side needs one, keep the iPaaS for it. Three ways they sit together: **1. OpexFlow pushes into the iPaaS (signed).** Point `push_endpoint` at the iPaaS's inbound webhook trigger. Every request is signed (`X-Signature` over the body, `X-Opex-Signature` over `{timestamp}.{body}`, `X-Opex-Key-Id`), so the iPaaS can verify origin and freshness (the contract and sample verifiers are on [Data contract](/docs/data-contract#verifying-the-signature)); put an API key the platform wants in `connector.push_headers`. Delivery is at-least-once through the durable queue, so make the iPaaS flow idempotent (the batch envelope carries `X-Batch-Id`) and answer `5xx`/`429` to be retried, another `4xx` to dead-letter. ```yaml connector: push_endpoint: "https://hooks.ipaas.example.com/catch/abc123" hmac_secret: "${IPAAS_SIGNING_SECRET}" push_headers: { X-Api-Key: "${IPAAS_API_KEY}" } push_queue: { enabled: true } batch: { enabled: true, max_records: 100, flush_interval_secs: 5 } ``` **2. The iPaaS calls OpexFlow.** A flow source `webhook:{name}` is a public, signature-checked endpoint (`hmac_sha256` with a timestamp, or `doku`; see [Signed webhooks](/docs/flows#signed-webhooks)); `POST /api/ingest/{source}/{id}` with a named API token (role `operator`) is the authenticated alternative. Both answer `200 done`, `422 failed`, `503 retry` (with `Retry-After`) or `202 pending`, so the iPaaS can retry correctly. **3. Hosted OpexFlow as the hub.** The plants push to the hosted node (above); its flows call the ERP and the iPaaS with `http` nodes (OAuth2, JWT bearer, basic, bearer, NetSuite TBA, DOKU signing; retries and a dead-letter queue with replay), and people approve exceptions in the app. The iPaaS keeps doing what it is good at, fed by a record stream that already has the plant's identity, quality flags and ordering. Hosted OpexFlow replaces an iPaaS where the integration is **plant to ERP/MES with delivery guarantees and local autonomy**. It does not replace one where the integration is **SaaS to SaaS across many vendors**.