Install and compile gems in parallel

[is-alnilam]

Currently the gem support code runs a single gem install to install the gems in the hook. Dependencies are resolved by the gem command, which installs each dependency in sequence. Any gems with native code are compiled, with the compilation stage of each gem using only a single compile task.

With this change, gems are installed in parallel (both top-level and dependencies), and any gems with native code allow these gems to use multiple compiler tasks (so using multiple cores where available). (Note that running multiple compilers will be the default gem behaviour in Ruby 4, so this ought to be safe).

The benefits of this will vary according to computer and hook, but an approximately 3x speed increase has been observed with the rubocop hook (from 38s down to 12s).

Additional speedup for #43

[codecov]

Codecov Report

❌ Patch coverage is 85.35354% with 29 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.57%. Comparing base (85d6dc6) to head (cec116a).
⚠️ Report is 8 commits behind head on master.

Files with missing lines	Patch %	Lines
crates/prek/src/languages/ruby/gem.rs	85.35%	29 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1674      +/-   ##
==========================================
+ Coverage   90.35%   91.57%   +1.21%     
==========================================
  Files          96       96              
  Lines       18661    18842     +181     
==========================================
+ Hits        16861    17254     +393     
+ Misses       1800     1588     -212     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[prek-ci-bot]

📦 Cargo Bloat Comparison

Binary size change: +0.42% (23.8 MiB → 23.9 MiB)

Expand for cargo-bloat output

Head Branch Results

 File  .text    Size             Crate Name
 0.3%   0.7% 71.3KiB              prek prek::languages::<impl prek::config::Language>::run::{{closure}}::{{closure}}
 0.3%   0.7% 66.4KiB             prek? <prek::cli::Command as clap_builder::derive::Subcommand>::augment_subcommands
 0.3%   0.6% 65.6KiB              prek prek::languages::<impl prek::config::Language>::run::{{closure}}::{{closure}}
 0.2%   0.5% 51.2KiB annotate_snippets annotate_snippets::renderer::render::render
 0.2%   0.5% 50.5KiB              prek prek::languages::<impl prek::config::Language>::install::{{closure}}
 0.2%   0.4% 41.2KiB              prek prek::cli::run::run::run::{{closure}}
 0.2%   0.4% 38.8KiB              prek prek::run::{{closure}}
 0.1%   0.3% 32.0KiB             prek? <prek::cli::RunArgs as clap_builder::derive::Args>::augment_args
 0.1%   0.3% 28.5KiB      serde_saphyr saphyr_parser_bw::scanner::Scanner<T>::fetch_more_tokens
 0.1%   0.2% 24.7KiB             prek? <prek::config::_::<impl serde_core::de::Deserialize for prek::config::Config>::deserialize::__Visitor as serde_core::de::Visitor>::visit_map
 0.1%   0.2% 22.8KiB      serde_saphyr saphyr_parser_bw::scanner::Scanner<T>::fetch_more_tokens
 0.1%   0.2% 22.7KiB              prek <prek::languages::ruby::ruby::Ruby as prek::languages::LanguageImpl>::install::{{closure}}
 0.1%   0.2% 22.7KiB              prek prek::hooks::meta_hooks::MetaHooks::run::{{closure}}
 0.1%   0.2% 22.4KiB              prek prek::hooks::meta_hooks::MetaHooks::run::{{closure}}
 0.1%   0.2% 21.1KiB      clap_builder clap_builder::parser::parser::Parser::get_matches_with
 0.1%   0.2% 20.0KiB               std core::ptr::drop_in_place<prek::languages::<impl prek::config::Language>::install::{{closure}}>
 0.1%   0.2% 19.5KiB              prek prek::archive::unzip::{{closure}}
 0.1%   0.2% 19.4KiB   cargo_metadata? <cargo_metadata::_::<impl serde_core::de::Deserialize for cargo_metadata::Package>::deserialize::__Visitor as serde_core::de::Visitor>::visit_map
 0.1%   0.2% 18.7KiB     serde_saphyr? <serde_saphyr::de::YamlDeserializer as serde_core::de::Deserializer>::deserialize_map
 0.1%   0.2% 18.6KiB     serde_saphyr? <serde_saphyr::de::YamlDeserializer as serde_core::de::Deserializer>::deserialize_map
38.3%  91.8%  9.1MiB                   And 21088 smaller methods. Use -n N to show more.
41.8% 100.0% 10.0MiB                   .text section size, the file size is 23.9MiB

Base Branch Results

 File  .text    Size             Crate Name
 0.3%   0.7% 71.4KiB              prek prek::languages::<impl prek::config::Language>::run::{{closure}}::{{closure}}
 0.3%   0.7% 70.6KiB             prek? <prek::cli::Command as clap_builder::derive::Subcommand>::augment_subcommands
 0.3%   0.6% 65.6KiB              prek prek::languages::<impl prek::config::Language>::run::{{closure}}::{{closure}}
 0.2%   0.5% 51.2KiB annotate_snippets annotate_snippets::renderer::render::render
 0.2%   0.5% 50.5KiB              prek prek::languages::<impl prek::config::Language>::install::{{closure}}
 0.2%   0.4% 41.1KiB              prek prek::cli::run::run::run::{{closure}}
 0.2%   0.4% 38.8KiB              prek prek::run::{{closure}}
 0.1%   0.3% 32.0KiB             prek? <prek::cli::RunArgs as clap_builder::derive::Args>::augment_args
 0.1%   0.3% 28.5KiB      serde_saphyr saphyr_parser_bw::scanner::Scanner<T>::fetch_more_tokens
 0.1%   0.2% 24.7KiB             prek? <prek::config::_::<impl serde_core::de::Deserialize for prek::config::Config>::deserialize::__Visitor as serde_core::de::Visitor>::visit_map
 0.1%   0.2% 22.8KiB      serde_saphyr saphyr_parser_bw::scanner::Scanner<T>::fetch_more_tokens
 0.1%   0.2% 22.7KiB              prek prek::hooks::meta_hooks::MetaHooks::run::{{closure}}
 0.1%   0.2% 21.1KiB      clap_builder clap_builder::parser::parser::Parser::get_matches_with
 0.1%   0.2% 20.5KiB              prek prek::hooks::meta_hooks::MetaHooks::run::{{closure}}
 0.1%   0.2% 20.0KiB   cargo_metadata? <cargo_metadata::_::<impl serde_core::de::Deserialize for cargo_metadata::Package>::deserialize::__Visitor as serde_core::de::Visitor>::visit_map
 0.1%   0.2% 19.6KiB              prek prek::archive::unzip::{{closure}}
 0.1%   0.2% 19.3KiB               std core::ptr::drop_in_place<prek::languages::<impl prek::config::Language>::install::{{closure}}>
 0.1%   0.2% 19.2KiB              prek <prek::languages::ruby::ruby::Ruby as prek::languages::LanguageImpl>::install::{{closure}}
 0.1%   0.2% 18.8KiB        serde_json serde_json::de::from_str
 0.1%   0.2% 18.7KiB     serde_saphyr? <serde_saphyr::de::YamlDeserializer as serde_core::de::Deserializer>::deserialize_map
38.3%  91.8%  9.1MiB                   And 21022 smaller methods. Use -n N to show more.
41.7% 100.0%  9.9MiB                   .text section size, the file size is 23.8MiB

[Copilot]

Pull request overview

This pull request adds parallel gem installation support for Ruby hooks, significantly improving installation performance (approximately 3x speedup observed). The implementation uses a two-phase approach: first resolving dependencies with gem install --explain, then installing each gem in parallel with --ignore-dependencies, with a fallback to sequential installation if the parallel approach fails.

Changes:

• Implemented parallel gem installation using gem install --explain for dependency resolution followed by concurrent installation
• Added MAKEFLAGS=-j<CONCURRENCY> to parallelize native extension compilation
• Added comprehensive unit tests for the gem name/version/platform parsing logic and integration tests for multiple gemspecs and mixed prebuilt/compiled gems

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
crates/prek/src/languages/ruby/gem.rs	Core implementation of parallel gem installation with dependency resolution, individual gem installation, parsing logic, and fallback to sequential installation
crates/prek/tests/languages/ruby.rs	Added integration tests for multiple gemspecs with shared dependencies and mixed prebuilt/compiled gems

[j178]

Thanks!