NOT MERGE: Leios prototype remake on 11.0.1#6574
Open
ch1bo wants to merge 77 commits into
Open
Conversation
The configuration option `StartAsNonProducingNode` was ignored, one could only use the switch `--non-producing-node` to set it. In this patch we fix this, and now it can be set either with the switch or if it's not given, it can be set in the configuration file.
The default configuration value of peer sharing option depends now on `pncStartAsNonProducingNode` option and `ncProtocolFiles`. If a node runs as a relay, peer sharing is on by default, if it is running as a block producer, peer sharing is off by default. The default value can be overridden.
The old option is kept, but deprecated.
…ceSendRecv changes
…11 Demo NOTE: - Tests and benchmarks fail to build, and that's ok
More consistent traces between immdb-server and cardano-node
…-837 [Leios prototype] Integrate ouroboros-leios#837
Leios prototype: Update tracers with new voting message
4 tasks
4 tasks
93d64b6 to
6c43d2f
Compare
…twork)
Updates cabal.project to point at the three rebased Leios forks that
ship with this cardano-node 11.0.1 build:
- ouroboros-consensus @ IntersectMBO/ouroboros-consensus
branch leios-prototype-remake-3.0.1.0
Single squashed Leios commit on top of release-3.0.1.0
(mempool + diffusion + chain inclusion + voting + demo).
- ouroboros-network @ IntersectMBO/ouroboros-network
branch leios-prototype-remake-1.1.0.0
Three Leios commits on top of ouroboros-network-1.1.0.0
(BearerBytes, Reception arrival-time, drop unused imports)
+ a trace-dispatcher version bump for compat with cardano-node 11.0.1.
- cardano-ledger @ IntersectMBO/cardano-ledger
branch leios-prototype-remake
Adds 'Maybe LeiosCert' on the Dijkstra block body
(Dijkstra-only, mirrors 'Maybe PerasCert').
Also drops the 'dmq-node' extra-package (its latest CHaP release
requires ouroboros-network:framework-tracing, a sublib name that
the remade network fork doesn't carry).
Vendors a one-line 'ekg-forward' source patch (left untouched here)
to match the network's pre-bump 'ConnectToArgs' shape.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ceSendRecv changes
The consensus Leios chunks added four fields to the consensus tracer records that cardano-node assembles in 'Cardano.Tracing.Tracers' (old style) and 'Cardano.Node.Tracing.Tracers' (new style): - Consensus.Tracers' gained 'leiosKernelTracer' and 'leiosPeerTracer' - NodeToNode.Tracers' gained 'tLeiosNotifyTracer' and 'tLeiosFetchTracer' This commit populates those fields in both 'mkTracers' code paths so the record-construction sites are total. All four are wired to 'nullTracer' for now; the corresponding 'Transformable' / 'MetaTrace' instances and EKG metrics are left as TODO. The new-style tracing system will accordingly emit a 'TracerConsistencyWarnings' for the four Leios namespaces declared in the config.yaml — that surfaces the gap explicitly rather than silently dropping the configuration. Adds the corresponding selectors to 'TraceSelection' / 'PartialTraceSelection' and their JSON parsing: - traceLeiosKernel / Consensus.LeiosKernel - traceLeiosPeer / Consensus.LeiosPeer - traceLeiosNotifyProtocol / LeiosNotify.Remote - traceLeiosFetchProtocol / LeiosFetch.Remote Also drops the duplicate 'LogFormatting (Simple/Stateful.TraceSendRecv)' and 'MetaTrace (Simple/Stateful.TraceSendRecv)' instances that the upstream leios-prototype branch carried in 'Cardano/Node/Tracing/Tracers/NodeToClient.hs' — they are now provided by 'ouroboros-network:framework-tracing' and would collide if redefined here. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Introduces 'Cardano.Node.Configuration.Leios.LeiosDbConfig' with two
constructors:
- LeiosDbInMemory (matches 'LeiosDbConnection's in-memory backend)
- LeiosDbSQLite !FilePath (SQLite backend, file relative to node CWD)
Wired through 'PartialNodeConfiguration' / 'NodeConfiguration' as
'ncLeiosDbConfig' with JSON parsing ('"LeiosDbConfig":{"Backend":...,
"Filepath":...}') and a default of 'LeiosDbSQLite "leios.db"'.
At node startup ('Cardano.Node.Run.handleSimpleNode'), the configured
backend is materialised into a 'LeiosDbHandle' (via 'newLeiosDBInMemory'
/ 'newLeiosDBSQLite') and passed into 'RunNodeArgs' through a new
'rnLeiosDb :: Maybe (LeiosDbHandle m)' field, which consensus's
'runWith' threads into the Leios kernel's shared connection.
'Nothing' falls back to the in-memory default.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The consensus Leios chunks extended 'NTN.Codecs' with two extra type
parameters ('bLN' / 'bLF', for LeiosNotify and LeiosFetch), and the
network-side BearerBytes/Reception work changed 'runPeerWithLimits' to
return 'Maybe (Reception bytes)' (instead of 'Maybe bytes') and made
'byteLimitsKeepAlive' parameter-less.
Update 'tx-generator' to compile against the rebased stack:
- Pull in 'ouroboros-network:framework-tracing' as a sublib dependency.
- Extend the local 'myCodecs' type to 9 'ByteString' parameters (was 7).
- Update 'kaClient' return type to 'IO ((), Maybe (Mux.Reception ByteString))'.
- Drop the size-function argument from 'byteLimitsKeepAlive'.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
6c43d2f to
c5acc0a
Compare
Both had been on the leios-prototype and and need to be re-exposed after resetting to a recent cardano-node version. Only the rest-in-mempool execution units metric was not easily applicable. That part of the code will change anyways though.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
This PR intends to fast-forward
leios-prototypeinto the head ofleios-prototype-remake. The remake now targets11.0.1(the released node), notmaster, so a singlenix buildlands a Leios-capable node binary at the same major.minor.patch downstream pipelines already track. For side-by-side comparison see diffs to the branch points:Commits
Five commits on top of
21c430e68 Reset leios-prototype to 11.0.1 (chunked recreation follows)— a snapshot-merge whose first parent is the originalleios-prototypetip (4ed4a1688 Merge pull request #6556 from IntersectMBO/ch1bo/leios-voting) and second parent is the11.0.1release tag (97036a66b). The merge's tree equals the 11.0.1 tree, sogit log --first-parent leios-prototype-remakewalks back through the original prototype's history (preserved in the DAG), while the topical commits build on the release base.Pin SRPs at the Leios prototype remake forks (consensus / ledger / network).
cabal.projectSRPs point at the three rebased forks shipped alongside this PR:ouroboros-consensusatleios-prototype-remake(the chunked Leios stack on top ofrelease-ouroboros-consensus-3.0.1.0— the consensus shipped in 11.0.1)cardano-ledgeratleios-prototype-remake(Maybe LeiosCertonDijkstraBody)ouroboros-networkatleios-prototype-remake(BearerBytes + Reception + trace-dispatcher 2.12 bump on top ofouroboros-network-1.1.0.0)Also drops the
dmq-nodeextra-packagesentry (the latest CHaPdmq-node-0.4.2.0requiresouroboros-network:framework-tracing, a sublib whose layout doesn't match the rebased network fork). Vendorsekg-forwardwith a one-line patch (ConnectToArgs/NetworkConnectTracerskeep the pre-bump 4-param shape that 1.1.0.0-base network exposes).leiosdemo202511: fixup build for ouroboros-network:runDriver* and TraceSendRecv changes(cherry-pick of Nicolas Frisby's upstreamleios-prototypecommit). AddsLogFormatting/MetaTraceinstances for the post-Reception 2-argTraceSendMsg/TraceRecvMsgconstructors, plus the smallmux_tmformatter so the new arrival-time field renders. DuplicateLogFormatting (Simple/Stateful.TraceSendRecv)instances that the upstream branch carried are dropped — they're now provided byouroboros-network:framework-tracingand would collide here.N2: wire Leios tracers into cardano-node Tracers / OrphanInstances. The consensus Leios chunks added four fields to the consensus tracer records that cardano-node assembles in
Cardano.Tracing.Tracers(old style) andCardano.Node.Tracing.Tracers(new style):Consensus.Tracers'→leiosKernelTracer,leiosPeerTracerNodeToNode.Tracers'→tLeiosNotifyTracer,tLeiosFetchTracerPopulates those fields in both
mkTracerscode paths so the record-construction sites are total. All four are wired tonullTracerfor now; the correspondingTransformable/MetaTraceinstances and EKG metrics are TODO (this surfaces explicitly as aTracerConsistencyWarningsfor the four Leios namespaces declared in the democonfig.yaml).Adds matching selectors
traceLeiosKernel/traceLeiosPeer/traceLeiosNotifyProtocol/traceLeiosFetchProtocoltoTraceSelection/PartialTraceSelection+ JSON parsing.N3: LeiosDbConfig configuration entry + thread to consensus. New
Cardano.Node.Configuration.Leios.LeiosDbConfig:LeiosDbInMemory— backed byLeiosDbConnection's in-memory backend.LeiosDbSQLite !FilePath— SQLite backend, file relative to node CWD (default:LeiosDbSQLite "leios.db").Wired through
PartialNodeConfiguration/NodeConfigurationasncLeiosDbConfigwith JSON parsing ("LeiosDbConfig":{"Backend":...,"Filepath":...}). At node startup (Cardano.Node.Run.handleSimpleNode) the configured backend is materialised into aLeiosDbHandleand passed intoRunNodeArgsvia a newrnLeiosDb :: Maybe (LeiosDbHandle m)field, which consensus'srunWiththreads into the Leios kernel's shared connection.Nothingfalls back to the in-memory default.Also adds
ResolveLeiosBlockto the existential constraints ofSomeConsensusProtocol, plus orphan no-opResolveLeiosBlockinstances forHardForkBlock '[ByronBlock]and per-eraShelleyBlockHFC(the type-level instances consensus's HFC layer demands).tx-generator: adapt to Leios-extended Codecs + Reception-wrapped recv. The consensus Leios chunks extended
NTN.Codecswith two extra type parameters (bLN/bLF, for LeiosNotify / LeiosFetch), and the network-side BearerBytes/Reception work changedrunPeerWithLimitsto returnMaybe (Reception bytes)and madebyteLimitsKeepAliveparameter-less.ouroboros-network:framework-tracingas a sublib dependency.myCodecstype to 9ByteStringparameters (was 7).kaClientreturn type becomesIO ((), Maybe (Mux.Reception ByteString)).byteLimitsKeepAlive.Verification
End-to-end proto-devnet run against
ouroboros-leios/demo/proto-devnet(3 nodes,TC=0,tx-centrifugeworkload) withTestDijkstraHardForkAtEpoch: 0added to the demo'sconfig.yaml:cardano-node11.0.1 binaries (from this branch) start and remain up; no crashes.cardano-cli query tipreportsera: "Dijkstra".leios.dbfills with EBs at the chain tip; over ~5 min one node reaches 7 EBs containing 70k txs (~2.5 MB of EB payload). EB counts differ across nodes (normalLeiosNotify/LeiosFetchdiffusion lag).cardano-cli-10.15.0.1andtx-centrifuge(both from the leios devshell) talk happily to our 11.0.1 node (50k+ StateQuery round-trips, no version mismatch errors).Known gaps (intentionally left for follow-up)
nullTracerwith TODO. FullTransformable Text IO TraceLeiosKernel/MetaTrace (TraceSendRecv ...)instances and EKG metrics are not yet ported; the fourLeios*tracer namespaces show up underTracerConsistencyWarningsat startup. The kernel runs; events just don't reach the tracer backends.cabal build allfails atcardano-ledger-byron:testlib(pre-existing CHaP issue:discoverPropArgTemplate Haskell error on GHC 9.6) and that transitively blockscardano-testnet:exe. Node, tracer, tx-generator, profile, topology, locli all build clean.Companion PRs