Skip to content

NOT MERGE: Leios prototype remake on 11.0.1#6574

Open
ch1bo wants to merge 77 commits into
masterfrom
leios-prototype-remake
Open

NOT MERGE: Leios prototype remake on 11.0.1#6574
ch1bo wants to merge 77 commits into
masterfrom
leios-prototype-remake

Conversation

@ch1bo
Copy link
Copy Markdown
Contributor

@ch1bo ch1bo commented May 19, 2026

Note

This PR intends to fast-forward leios-prototype into the head of leios-prototype-remake. The remake now targets 11.0.1 (the released node), not master, so a single nix build lands a Leios-capable node binary at the same major.minor.patch downstream pipelines already track. For side-by-side comparison see diffs to the branch points:

Commits

Five commits on top of 21c430e68 Reset leios-prototype to 11.0.1 (chunked recreation follows) — a snapshot-merge whose first parent is the original leios-prototype tip (4ed4a1688 Merge pull request #6556 from IntersectMBO/ch1bo/leios-voting) and second parent is the 11.0.1 release tag (97036a66b). The merge's tree equals the 11.0.1 tree, so git log --first-parent leios-prototype-remake walks back through the original prototype's history (preserved in the DAG), while the topical commits build on the release base.

  • Pin SRPs at the Leios prototype remake forks (consensus / ledger / network). cabal.project SRPs point at the three rebased forks shipped alongside this PR:

    • ouroboros-consensus at leios-prototype-remake (the chunked Leios stack on top of release-ouroboros-consensus-3.0.1.0 — the consensus shipped in 11.0.1)
    • cardano-ledger at leios-prototype-remake (Maybe LeiosCert on DijkstraBody)
    • ouroboros-network at leios-prototype-remake (BearerBytes + Reception + trace-dispatcher 2.12 bump on top of ouroboros-network-1.1.0.0)

    Also drops the dmq-node extra-packages entry (the latest CHaP dmq-node-0.4.2.0 requires ouroboros-network:framework-tracing, a sublib whose layout doesn't match the rebased network fork). Vendors ekg-forward with a one-line patch (ConnectToArgs/NetworkConnectTracers keep the pre-bump 4-param shape that 1.1.0.0-base network exposes).

  • leiosdemo202511: fixup build for ouroboros-network:runDriver* and TraceSendRecv changes (cherry-pick of Nicolas Frisby's upstream leios-prototype commit). Adds LogFormatting / MetaTrace instances for the post-Reception 2-arg TraceSendMsg/TraceRecvMsg constructors, plus the small mux_tm formatter so the new arrival-time field renders. Duplicate LogFormatting (Simple/Stateful.TraceSendRecv) instances that the upstream branch carried are dropped — they're now provided by ouroboros-network:framework-tracing and would collide here.

  • N2: wire Leios tracers into cardano-node Tracers / OrphanInstances. The consensus Leios chunks added four fields to the consensus tracer records that cardano-node assembles in Cardano.Tracing.Tracers (old style) and Cardano.Node.Tracing.Tracers (new style):

    • Consensus.Tracers'leiosKernelTracer, leiosPeerTracer
    • NodeToNode.Tracers'tLeiosNotifyTracer, tLeiosFetchTracer

    Populates those fields in both mkTracers code paths so the record-construction sites are total. All four are wired to nullTracer for now; the corresponding Transformable / MetaTrace instances and EKG metrics are TODO (this surfaces explicitly as a TracerConsistencyWarnings for the four Leios namespaces declared in the demo config.yaml).

    Adds matching selectors traceLeiosKernel / traceLeiosPeer / traceLeiosNotifyProtocol / traceLeiosFetchProtocol to TraceSelection / PartialTraceSelection + JSON parsing.

  • N3: LeiosDbConfig configuration entry + thread to consensus. New Cardano.Node.Configuration.Leios.LeiosDbConfig:

    • LeiosDbInMemory — backed by LeiosDbConnection's in-memory backend.
    • LeiosDbSQLite !FilePath — SQLite backend, file relative to node CWD (default: LeiosDbSQLite "leios.db").

    Wired through PartialNodeConfiguration / NodeConfiguration as ncLeiosDbConfig with JSON parsing ("LeiosDbConfig":{"Backend":...,"Filepath":...}). At node startup (Cardano.Node.Run.handleSimpleNode) the configured backend is materialised into a LeiosDbHandle and passed into RunNodeArgs via a new rnLeiosDb :: Maybe (LeiosDbHandle m) field, which consensus's runWith threads into the Leios kernel's shared connection. Nothing falls back to the in-memory default.

    Also adds ResolveLeiosBlock to the existential constraints of SomeConsensusProtocol, plus orphan no-op ResolveLeiosBlock instances for HardForkBlock '[ByronBlock] and per-era ShelleyBlockHFC (the type-level instances consensus's HFC layer demands).

  • tx-generator: adapt to Leios-extended Codecs + Reception-wrapped recv. The consensus Leios chunks extended NTN.Codecs with two extra type parameters (bLN / bLF, for LeiosNotify / LeiosFetch), and the network-side BearerBytes/Reception work changed runPeerWithLimits to return Maybe (Reception bytes) and made byteLimitsKeepAlive parameter-less.

    • Pulls in ouroboros-network:framework-tracing as a sublib dependency.
    • Extends the local myCodecs type to 9 ByteString parameters (was 7).
    • kaClient return type becomes IO ((), Maybe (Mux.Reception ByteString)).
    • Drops the size-function argument from byteLimitsKeepAlive.

Verification

End-to-end proto-devnet run against ouroboros-leios/demo/proto-devnet (3 nodes, TC=0, tx-centrifuge workload) with TestDijkstraHardForkAtEpoch: 0 added to the demo's config.yaml:

  • All three cardano-node 11.0.1 binaries (from this branch) start and remain up; no crashes.
  • cardano-cli query tip reports era: "Dijkstra".
  • Per-node SQLite leios.db fills with EBs at the chain tip; over ~5 min one node reaches 7 EBs containing 70k txs (~2.5 MB of EB payload). EB counts differ across nodes (normal LeiosNotify/LeiosFetch diffusion lag).
  • Upstream cardano-cli-10.15.0.1 and tx-centrifuge (both from the leios devshell) talk happily to our 11.0.1 node (50k+ StateQuery round-trips, no version mismatch errors).

Known gaps (intentionally left for follow-up)

  • Tracers wired as nullTracer with TODO. Full Transformable Text IO TraceLeiosKernel / MetaTrace (TraceSendRecv ...) instances and EKG metrics are not yet ported; the four Leios* tracer namespaces show up under TracerConsistencyWarnings at startup. The kernel runs; events just don't reach the tracer backends.
  • cabal build all fails at cardano-ledger-byron:testlib (pre-existing CHaP issue: discoverPropArg Template Haskell error on GHC 9.6) and that transitively blocks cardano-testnet:exe. Node, tracer, tx-generator, profile, topology, locli all build clean.

Companion PRs

jutaro and others added 30 commits July 1, 2025 12:21
The configuration option `StartAsNonProducingNode` was ignored, one
could only use the switch `--non-producing-node` to set it.  In this
patch we fix this, and now it can be set either with the switch or if
it's not given, it can be set in the configuration file.
The default configuration value of peer sharing option depends now on
`pncStartAsNonProducingNode` option and `ncProtocolFiles`.  If a node
runs as a relay, peer sharing is on by default, if it is running as
a block producer, peer sharing is off by default.  The default value can
be overridden.
…11 Demo

NOTE:
- Tests and benchmarks fail to build, and that's ok
More consistent traces between immdb-server and cardano-node
@ch1bo ch1bo requested review from a team as code owners May 19, 2026 11:08
@ch1bo ch1bo changed the title Leios prototype remake on 11.0.1 NOT MERGE: Leios prototype remake on 11.0.1 May 19, 2026
@ch1bo ch1bo changed the base branch from leios-prototype to master May 19, 2026 11:09
@ch1bo ch1bo linked an issue May 19, 2026 that may be closed by this pull request
4 tasks
@ch1bo ch1bo requested a review from bladyjoker May 19, 2026 11:45
@ch1bo ch1bo force-pushed the leios-prototype-remake branch 6 times, most recently from 93d64b6 to 6c43d2f Compare May 22, 2026 18:51
ch1bo and others added 5 commits May 22, 2026 21:20
…twork)

Updates cabal.project to point at the three rebased Leios forks that
ship with this cardano-node 11.0.1 build:

  - ouroboros-consensus  @  IntersectMBO/ouroboros-consensus
    branch leios-prototype-remake-3.0.1.0
    Single squashed Leios commit on top of release-3.0.1.0
    (mempool + diffusion + chain inclusion + voting + demo).
  - ouroboros-network    @  IntersectMBO/ouroboros-network
    branch leios-prototype-remake-1.1.0.0
    Three Leios commits on top of ouroboros-network-1.1.0.0
    (BearerBytes, Reception arrival-time, drop unused imports)
    + a trace-dispatcher version bump for compat with cardano-node 11.0.1.
  - cardano-ledger       @  IntersectMBO/cardano-ledger
    branch leios-prototype-remake
    Adds 'Maybe LeiosCert' on the Dijkstra block body
    (Dijkstra-only, mirrors 'Maybe PerasCert').

Also drops the 'dmq-node' extra-package (its latest CHaP release
requires ouroboros-network:framework-tracing, a sublib name that
the remade network fork doesn't carry).

Vendors a one-line 'ekg-forward' source patch (left untouched here)
to match the network's pre-bump 'ConnectToArgs' shape.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The consensus Leios chunks added four fields to the consensus tracer
records that cardano-node assembles in 'Cardano.Tracing.Tracers' (old
style) and 'Cardano.Node.Tracing.Tracers' (new style):

  - Consensus.Tracers'      gained 'leiosKernelTracer' and 'leiosPeerTracer'
  - NodeToNode.Tracers'     gained 'tLeiosNotifyTracer' and 'tLeiosFetchTracer'

This commit populates those fields in both 'mkTracers' code paths so
the record-construction sites are total. All four are wired to
'nullTracer' for now; the corresponding 'Transformable' / 'MetaTrace'
instances and EKG metrics are left as TODO. The new-style tracing
system will accordingly emit a 'TracerConsistencyWarnings' for the
four Leios namespaces declared in the config.yaml — that surfaces the
gap explicitly rather than silently dropping the configuration.

Adds the corresponding selectors to 'TraceSelection' / 'PartialTraceSelection'
and their JSON parsing:

  - traceLeiosKernel          / Consensus.LeiosKernel
  - traceLeiosPeer            / Consensus.LeiosPeer
  - traceLeiosNotifyProtocol  / LeiosNotify.Remote
  - traceLeiosFetchProtocol   / LeiosFetch.Remote

Also drops the duplicate 'LogFormatting (Simple/Stateful.TraceSendRecv)'
and 'MetaTrace (Simple/Stateful.TraceSendRecv)' instances that the
upstream leios-prototype branch carried in
'Cardano/Node/Tracing/Tracers/NodeToClient.hs' — they are now provided
by 'ouroboros-network:framework-tracing' and would collide if redefined
here.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Introduces 'Cardano.Node.Configuration.Leios.LeiosDbConfig' with two
constructors:

  - LeiosDbInMemory          (matches 'LeiosDbConnection's in-memory backend)
  - LeiosDbSQLite !FilePath  (SQLite backend, file relative to node CWD)

Wired through 'PartialNodeConfiguration' / 'NodeConfiguration' as
'ncLeiosDbConfig' with JSON parsing ('"LeiosDbConfig":{"Backend":...,
"Filepath":...}') and a default of 'LeiosDbSQLite "leios.db"'.

At node startup ('Cardano.Node.Run.handleSimpleNode'), the configured
backend is materialised into a 'LeiosDbHandle' (via 'newLeiosDBInMemory'
/ 'newLeiosDBSQLite') and passed into 'RunNodeArgs' through a new
'rnLeiosDb :: Maybe (LeiosDbHandle m)' field, which consensus's
'runWith' threads into the Leios kernel's shared connection.
'Nothing' falls back to the in-memory default.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The consensus Leios chunks extended 'NTN.Codecs' with two extra type
parameters ('bLN' / 'bLF', for LeiosNotify and LeiosFetch), and the
network-side BearerBytes/Reception work changed 'runPeerWithLimits' to
return 'Maybe (Reception bytes)' (instead of 'Maybe bytes') and made
'byteLimitsKeepAlive' parameter-less.

Update 'tx-generator' to compile against the rebased stack:

  - Pull in 'ouroboros-network:framework-tracing' as a sublib dependency.
  - Extend the local 'myCodecs' type to 9 'ByteString' parameters (was 7).
  - Update 'kaClient' return type to 'IO ((), Maybe (Mux.Reception ByteString))'.
  - Drop the size-function argument from 'byteLimitsKeepAlive'.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@ch1bo ch1bo force-pushed the leios-prototype-remake branch from 6c43d2f to c5acc0a Compare May 22, 2026 19:20
Both had been on the leios-prototype and and need to be re-exposed after
resetting to a recent cardano-node version.

Only the rest-in-mempool execution units metric was not easily
applicable. That part of the code will change anyways though.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update prototype to Dijsktra

9 participants