Skip to content

Backtesting

VerifiedOwner Backend / platformLast verified backend@efa38e04e2db37a444e9f17100323c2ad4b83647 nautilus@2.0.0rc3

A backtest in ONIX runs a sealed strategy revision (see strategy-authoring.md) through the official Nautilus BacktestNode, fed from the same durable ParquetDataCatalog the market spine writes to. Only a revision that has passed the admission gate — sealed, verified, promotable — can be submitted; there is no unsealed-submission path.

Submission and run modes

POST /v2/backtests takes {strategy_id, revision_id, config?, instrument_id?, bar_type?, start?, end?, chunk_size?, run_mode?} and returns {job_id, status} immediately; the job itself runs in a separate child process, one sealed revision per child. Two run modes are available: the official BacktestNode one-shot path, and a streaming path (the default), which bounds the batch pulled per iteration via chunk_size — larger chunk sizes trade lower overhead for higher peak worker memory. GET /v2/backtests lists job history durably, surviving a process restart; GET /v2/backtests/{job_id} returns full job detail; POST /v2/backtests/{job_id}/cancel cancels an in-flight job (refused for one already in a terminal state).

What data a run actually loads

For a revision with a typed, sealed data demand naming multiple instruments, potentially across different venues, the backtest builds one Nautilus venue configuration per distinct venue those instruments name, all inside a single BacktestNode — a three-symbol, mixed-venue revision really receives all three in one run, not three separate runs. Every channel a run is configured to load has already been proven, by the submitting job runner, to be something the catalog can actually serve; a required series that cannot be served means the job is refused before this process is even spawned, rather than silently running with a gap.

Configs are consolidated wherever Nautilus's own loading model allows it — one BacktestDataConfig per (Nautilus data type, window), carrying a sorted identifier list, rather than one per (instrument, channel). This is not just tidiness: at this pin, BacktestNode streams data lazily from the catalog only for a run built from exactly one data config, and otherwise materializes every row of every config into memory up front. Which of those two shapes a given run took is recorded, not left implicit, precisely because a bars-only multi-symbol run can be one streaming config while a run mixing data types cannot.

Results are official Nautilus output, verbatim

Every value written into a job's result — order/fill/position rows, unrealized PnL, account state — is read directly off official Nautilus objects (BacktestResult, Cache.orders() / .positions(), Position.events(), Position.unrealized_pnl(), Portfolio.account()). ONIX computes none of these statistics itself. BacktestJobDetail publishes the resolved run spec (configuration, sealed revision provenance, data selection, execution model, run mode) verbatim alongside these official outputs, plus an honest catalog-coverage statement for the window actually used.

GET /v2/backtests/{job_id}/report?kind=ORDERS|FILLS|POSITIONS|ACCOUNT serves paginated report pages, each carrying its own onix.execution_provenance envelope (environment, account, venue, OMS type, and the exact strategy/revision/job/run ids that produced it) — a report row is never presented without the identity of the run that produced it.

Determinism and provenance — two paths, one shared identity scheme, one proof

The gateway/job path documented above and a separate, offline single-user workbench are not the same thing, and only one of them is proven byte-identical across a fresh-process rerun. Both read the same canonical-JSON identity scheme (docs/contracts/identity.md): a result_hash is computed over the canonical result document with the hash field itself blanked (replaced by an all-zero 64-character digest at its normal sorted position), not omitted — a consumer verifies by byte substitution against a document it already has, rather than reconstructing one it never saw. That scheme is shared infrastructure. What each path actually does with it differs:

  • POST /v2/backtests, the path this page otherwise documents, spawns one child process (spine.backtest_run) per submitted job. run_id_hex on this path is an optional field the caller's own submitted config may carry (spec["config"]["run_id_hex"]); backtest_run.py itself does not derive a content address from the job's own inputs, and this documentation found no test that resubmits the same sealed revision as a second job and diffs the two jobs' result bytes. A BacktestJobDetail's resolved run spec (configuration, sealed revision provenance, data selection, execution model, run mode) is published verbatim alongside the official outputs — that is real, verified provenance — but it is not, by itself, a determinism proof.
  • The separate offline workbench CLI (python -m onix_engine.workbench, a local, single-user tool distinct from the gateway/MCP surface — see strategy-authoring.md) is where content-addressed run identity and fresh-process determinism are actually computed and proven. workbench.backtest.run_workbench_backtest derives run_id_hex = digest_hex(RUN_ID_DOMAIN, engine_identity, strategy_id, config_hash, fixture_hash) from the project's own committed manifest and fixture — a genuinely content-addressed identity, not an optional caller-supplied field. backtest --stdout-canonical prints exactly the canonical evidence bytes workbench.rerun's subprocess byte-equivalence proof depends on, and python/tests/test_workbench_rerun.py ("the fresh-process byte-equivalence proof") is the executable test that actually reruns a scaffolded project in a fresh process and asserts the two runs' canonical bytes are byte-identical — "reruns" is defined there to mean a fresh process, not a fresh in-process object.

A CLI's existence is not a determinism receipt. This documentation states the determinism claim precisely: byte-identical fresh-process rerun is verified for the workbench path, by the cited test. It is unverified for the gateway POST /v2/backtests path specifically — this documentation found no equivalent rerun-and-diff test for a job submitted through the gateway, and does not extend the workbench's proof to that separate path by association.

What a backtest result is not

  • Not a claim about live execution. A backtest runs entirely inside Nautilus's own simulated fill/venue model over historical catalog data; nothing about it exercises a real venue connection or a real order.
  • Not ONIX-computed statistics. Every number in a report is Nautilus's own arithmetic, transported, not recomputed — including the float-arithmetic PnL characteristics that are true of Nautilus's own Position accounting generally (see portfolio-risk.md).
  • Not immune to catalog gaps. A run's coverage is reported honestly for the exact window requested; a revision's data demand can only be satisfied from what the durable catalog actually holds (see ../data/catalog.md).
  • Not a three-mode TypeScript harness. Every backtest path this backend implements, on either the gateway or the workbench side, runs through the official Nautilus BacktestEngine/ BacktestNode. This documentation found no TypeScript-based backtest compile/execution/harness code anywhere in this repository at the pin (see strategy-authoring.md's "What is not part of this backend" section for the full, honestly-bounded search this claim rests on) — a legacy three-mode TypeScript harness, if one existed, is not present in and not reachable from this backend today. This documentation cannot independently confirm from the backend repository alone that such a harness previously existed elsewhere in the ONIX product and was later retired; that gap is recorded, with the exact bounded search performed, in this documentation effort's internal evidence log.
Evidence and source pins for this page

Verified. Current behaviour, confirmed in source at the pinned commit.

Verified on against the following immutable sources:

  • backend@efa38e04:python/src/onix_engine/spine/backtest_run.py
  • backend@efa38e04:python/src/onix_engine/workbench/backtest.py
  • backend@efa38e04:python/src/onix_engine/workbench/cli.py
  • backend@efa38e04:python/tests/test_workbench_rerun.py
  • backend@efa38e04:docs/contracts/market-spine-v2.md

Status tokens are defined on the documentation and status model page. Every pin on this site is listed under versions and source pins.