Skip to content

Runtime and deployment envelope

VerifiedOwner Backend / platformLast verified backend@efa38e04e2db37a444e9f17100323c2ad4b83647 nautilus@2.0.0rc3

Process shape

Each venue integration runs as its own process (python -m onix_engine.spine.service --integration <venue>), built around one Nautilus LiveNode. Process isolation — one integration never sharing a Python process with another — is enforced at the OS level (systemd Restart=always, per-unit process boundaries), not by an ONIX-managed supervisor: an earlier ONIX-owned supervisor layer was retired when the RC3 pin removed the LiveNode.start()/poll() methods it was built around. Alongside the per-venue spine processes, a fleet composition includes a public gateway process (aggregating reads across venue processes), an operator-facing gateway, a Rust screener process, and — where deployed — a frontend server. Exact process counts and network placement for a given deployment are declared in that deployment's own activation configuration (see ../venues/index.md); this documentation does not publish specific ports, hosts, or filesystem mount paths for any environment.

Hosting the Nautilus runtime

RC3 removed LiveNode.start() and LiveNode.poll(); the wheel now offers only a blocking run() and a coroutine run_async(). ONIX drives run() — never run_async() — through one object, NautilusRuntimeHost, for a measured reason: the pinned wheel's run_async() driver carries an intermittent lock-order inversion between the CPython GIL and the native runtime (the native waker re-enters Python while the run future is being polled with the GIL held), which stalled multi-client connects in direct measurement, while run() was clean on the identical composition. This is a choice between two official Nautilus run surfaces, not a patch to either.

The host's contract, as implemented:

  1. Everything a caller may legally read for the whole run — node.cache, node.portfolio, node.trader_id, node.environment, node.instance_id — is captured before run() is entered, because all of those raise once the run owns the node.
  2. run() owns the calling (main) thread for the whole run; auxiliary work (the gateway, catalog measurement, the screener bridge) lives on ordinary threads that touch only the captured surfaces and a thread-safe stop handle, never the node object itself.
  3. A single supervisor thread converts every stop intent — a signal, a startup timeout, a run deadline — into the same handle.stop() call.
  4. The market-spine node registers no execution or risk client of any kind — every venue process in this fleet runs Nautilus's data/portfolio/risk engine scaffolding with default configuration and zero exec clients attached. See ../trading/execution-runtime.md for what that means for order placement.

Live market projection: actor, shared state, and the SSE fanout

The actor. Each spine process registers one Nautilus DataActor (spine.actor.MarketSpineActor) with its LiveNode via the only released registration path for a custom Python actor, add_actor_from_config(ImportableActorConfig(...)) — there is no node.add_actor(instance) for actors. This actor discovers every provider instrument, subscribes live-only quotes/trades and the one canonical 1-minute bar type per instrument, backfills each instrument's rolling catalog window through the bounded, priority-ordered hydration queue (see ../data/catalog.md), and — for a bounded prefix of the universe — the live one-second session tier (see ../data/aggregation.md). It computes nothing itself: "no detector logic, no computed fields — every wire value is read straight off the official Nautilus object via spine.wire," stated directly in its own module docstring.

Shared state. spine.state.SpineState is the one thread-safe object the Nautilus actor thread (recording observations) and the gateway threads (reading snapshots, subscribing to the fanout) both touch — and it is the only thing gateway threads may touch: everything crossing into it is already a plain Python wire dict, never a retained Nautilus/pyo3 object, so a gateway thread can never accidentally reach into node-owned memory. Because LiveNode.add_actor_from_config only accepts a plain config dict (no live object handoff, and no way to retrieve the actor instance back afterward — a documented, empirically-probed absence in the released wheel), SpineState lives in a module-level registry keyed by a single opaque string; the actor looks itself up by that key in on_start and fails loudly (KeyError) rather than silently constructing a private, disconnected state object if the caller forgot to register first.

Ordering and recovery on the SSE fanout. SpineState's broadcast fanout gives each subscriber its own bounded queue (2,048 events); a slow consumer loses its oldest events rather than stalling the actor thread or growing without bound, and a subscriber's optional keep-predicate is applied at enqueue time, before put_nowait — so a single-instrument SSE client's relevant events are never evicted by an unrelated firehose sharing the same bounded queue. On the serving side (onix_engine.gateway_sse), every SSE event is the same schema-valid DTO bytes the equivalent GET route already serves — SSE never invents a second, divergent shape — each event carries a run-scoped sequence cursor, a resuming client passes ?cursor=<sequence>, and the module answers an explicit event: resync message (never a silent empty answer) when the requested cursor can no longer be satisfied. Plain HTTP text/event-stream throughout; no custom WebSocket protocol anywhere in this fanout.

What this documentation has, and has not, observed. The actor/state/wire/SSE mechanics above are verified against pinned source. This documentation did not have a live host to open a real SSE connection against, so the end-to-end behavior of a real client consuming a real stream — the "observed-SSE prerequisite" a full closure of this topic would require — is not something this documentation pass can claim to have watched happen; it is stated here as unverified for that specific end-to-end observation, distinct from the verified source-level mechanics above.

Engine status

A producer-authored onix.engine_status document tracks one run's own lifecycle transitions — STARTING → RUNNING → {CHECKPOINTED, COMPLETED, FAILED}, with CHECKPOINTED able to return to RUNNING — plus counters (bars accepted/rejected, signals emitted, orders submitted, fills, errors). It is emitted by the process itself and served verbatim; the gateway never synthesizes a health conclusion from file existence.

Liveness versus readiness

These are deliberately different questions, served by different routes:

  • GET /health — transport liveness only: the process is up and answering HTTP. It says nothing about data freshness or venue connectivity.
  • GET /readiness — condition/freshness: the latest engine_status lifecycle state, plus the run's own per-authority gap/duplicate/out-of-order counters, read verbatim from the producer's own record rather than re-derived by the gateway. This route exists specifically so a caller cannot confuse "the process is alive" with "the data is trustworthy right now."

Both are currently served by ONIX's older, fixture/runtime-mode v1 gateway, kept only because some MCP tools still read through it; the governed production surface for everything else is the /v2 spine gateway (spine.gateway_v2 / gateway_multi), documented in ../gateway-schemas.md. GET /v2/status additionally reports node running-state, per-venue connectivity, instrument/bar counters and rates, and hydration summary counts.

The tracked deployment record — and its limits

The one deployment state ONIX tracks as CURRENT_PRODUCTION is a feature-state record, not a live observation:

  • Nautilus: 2.0.0rc3 at the pinned commit.
  • Retained-ingestion manifest: mv5-epoch1 — a different, earlier manifest than the mv8-epoch1 manifest committed in the development tree (see ../data/ingestion.md).
  • Venue composition: Bybit only. Five production units are named: the Bybit market spine, the public gateway, the operator gateway, the screener, and a frontend server bound to loopback.
  • Live 1-minute closes: INTERNAL TimeBarAggregator bars built from trade ticks; external venue klines are used for history/backfill/repair only, not as the live source.

This record is attested, not observed. The evidence this documentation effort had access to was the record itself (a receipt identifier) and a reference to an external production ledger file that is not tracked in this repository and was not read by this documentation effort. No live host state was independently observed while writing this documentation. Every claim above is therefore production-record status: true according to the tracked record, not independently verified against a running system. Separately, the development pin's deploy/activation.json declares a broader composition — Bybit, Hyperliquid (ingest-only), and Binance (ingest-only) — that is development intent and has not been shown, in this repository, to have been promoted to production. See ../venues/index.md for the full venue-by-venue breakdown and ../capability-status.md for the consolidated register.

Evidence and source pins for this page

Verified. Current behaviour, confirmed in source at the pinned commit.

Verified on against the following immutable sources:

  • backend@efa38e04:python/src/onix_engine/runtime/lifecycle.py
  • backend@efa38e04:python/src/onix_engine/spine/service.py
  • backend@efa38e04:python/src/onix_engine/spine/actor.py
  • backend@efa38e04:python/src/onix_engine/spine/state.py
  • backend@efa38e04:python/src/onix_engine/gateway.py
  • backend@efa38e04:python/src/onix_engine/gateway_sse.py
  • backend@efa38e04:feature-state/rc3-data-plane-frozen-current-production.yaml

Status tokens are defined on the documentation and status model page. Every pin on this site is listed under versions and source pins.