Skip to content

go/cmd/dolt: Add GOMEMLIMIT awareness via automemlimit and server config#10813

Closed
stebbins wants to merge 1 commit into
dolthub:mainfrom
skullbloc:do-jz0-gomemlimit
Closed

go/cmd/dolt: Add GOMEMLIMIT awareness via automemlimit and server config#10813
stebbins wants to merge 1 commit into
dolthub:mainfrom
skullbloc:do-jz0-gomemlimit

Conversation

@stebbins
Copy link
Copy Markdown
Contributor

@stebbins stebbins commented Apr 4, 2026

Dolt has no GOMEMLIMIT or GOGC awareness. For containerized deployments, this means the Go runtime doesn't know about cgroup memory limits, leading to OOM kills when RSS grows past the container limit.

This adds three-tier memory limit configuration:

  1. DOLT_GOMEMLIMIT env var — explicit, human-readable values like "1GiB"
  2. automemlimit cgroup auto-detection — sets GOMEMLIMIT to 90% of the cgroup limit
  3. Silent fallthrough on bare metal when neither applies

Also adds max_go_memory to the YAML server config, applied at sql-server startup via debug.SetMemoryLimit(), and MaxGoMemory() int64 to the ServerConfig interface.

Files changed:

  • go/cmd/dolt/memlimit.go (new) — configureMemoryLimit() with the three-tier logic
  • go/cmd/dolt/dolt.go — calls configureMemoryLimit() early in runMain()
  • go/cmd/dolt/commands/sqlserver/server.go — applies max_go_memory at sql-server startup
  • go/libraries/doltcore/servercfg/MaxGoMemory() int64 on interface and implementations

@stebbins stebbins changed the title feat: add GOMEMLIMIT awareness via automemlimit and config go/cmd/dolt: Add GOMEMLIMIT awareness via automemlimit and server config Apr 4, 2026
@stebbins stebbins force-pushed the do-jz0-gomemlimit branch from 4f24477 to ba8ffd6 Compare April 4, 2026 16:38
Copy link
Copy Markdown
Contributor

@reltuk reltuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general I'm not opposed to automatically calling SetMemoryLimit with cgroup-or-other derived values. I'm not 100% sure this needs to settable through config.yaml. I'm also not sure about the DOLT_GOMEMLIMIT, versus just relying on the already-existing mechanism of GOMEMLIMIT.

What are the tradeoffs for these two cases in your opinion?

Is the idea behind the non-standard env variable that someone wanting CLI limits on Dolt but not on random other processes which happened to have been written in Go would export DOLT_GOMEMLIMIT instead?

In general I would prefer to not add functionality to Dolt itself that seems more appropriately delegated to the operating context in which Dolt is running (cgroups, supervisor service definitions, kubernetes manifests, etc.)

@stebbins
Copy link
Copy Markdown
Contributor Author

stebbins commented Apr 7, 2026

#10815 (comment)

closing in favor of #10820

@stebbins stebbins closed this Apr 7, 2026
@stebbins
Copy link
Copy Markdown
Contributor Author

stebbins commented Apr 7, 2026

#10821

to make it easier since there are already a few PRs, i created an issue to track the concept

nicktobey added a commit that referenced this pull request May 1, 2026
…rwrite-ignore-sql-checkout-new-branch

#10813 fix(checkout): enforce --no-overwrite-ignore in DOLT_CHECKOUT -b via direct SQL
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants