Profile Linux coverage CI#4889
Draft
andrewsanchez wants to merge 3 commits into
Draft
Conversation
Rust coverage previously invoked cargo-llvm-cov's default test runner directly, while normal Rust tests run through cargo nextest via ninja check:rust_test. That created a gap where coverage could use different execution semantics and timing than the test path developers and CI rely on. Install the same pinned cargo-nextest binary into out/bin when coverage needs it, put that directory on PATH for cargo-llvm-cov's nextest integration, and pass through the same nextest reporter flags used by the normal Rust test target. This keeps coverage behavior aligned across Unix and Windows launchers while preserving the existing cargo-llvm-cov report generation.
Split the Linux check job into separate build, lint, and per-stack test steps so GitHub Actions reports coverage timing by stack. PR builds continue to enforce coverage on Linux, while non-PR Linux runs and the macOS/Windows jobs keep using the normal test path. Clear generated coverage and e2e report directories before testing and before saving the generic build output cache. This prevents stale reports from restored caches from affecting profiling or later baseline comparisons. Add a CI-specific coverage.py config that opts into the sysmon core and trims terminal reports by skipping covered files while preserving missing-line output.
Contributor
Author
|
Follow-up consideration: if Playwright e2e tests are added to normal CI, we should consider skipping them by default and running them only when a dedicated PR label is present. That keeps browser-based tests available for relevant changes without making every regular CI run pay the startup/browser cost. |
The profiling step previously passed the parent out directory and nested paths to a single du invocation. GNU du suppresses child entries in that shape, so the Actions log only showed the total out size and hid the coverage, tool, and browser directories we need to inspect. Loop over each path instead so the follow-up profiling run reports every existing directory on its own line while still ignoring paths that are not present in a given job.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Splits the Linux CI job into separately timed build, lint, and per-stack coverage test steps so we can profile the cost of Rust, Python, and TypeScript coverage independently.
Also clears generated coverage/e2e reports before tests and before saving the generic build cache, preventing stale coverage output from restored caches from affecting profiling or future baseline comparisons.
Adds a CI-specific coverage.py config using the sysmon core and shorter terminal reports.
Profiling goal
Use the PR CI run to compare:
Validation