go/cmd/dolt: Add GOMEMLIMIT awareness via automemlimit and server config#10813
go/cmd/dolt: Add GOMEMLIMIT awareness via automemlimit and server config#10813stebbins wants to merge 1 commit into
Conversation
4f24477 to
ba8ffd6
Compare
reltuk
left a comment
There was a problem hiding this comment.
In general I'm not opposed to automatically calling SetMemoryLimit with cgroup-or-other derived values. I'm not 100% sure this needs to settable through config.yaml. I'm also not sure about the DOLT_GOMEMLIMIT, versus just relying on the already-existing mechanism of GOMEMLIMIT.
What are the tradeoffs for these two cases in your opinion?
Is the idea behind the non-standard env variable that someone wanting CLI limits on Dolt but not on random other processes which happened to have been written in Go would export DOLT_GOMEMLIMIT instead?
In general I would prefer to not add functionality to Dolt itself that seems more appropriately delegated to the operating context in which Dolt is running (cgroups, supervisor service definitions, kubernetes manifests, etc.)
|
closing in favor of #10820 |
|
to make it easier since there are already a few PRs, i created an issue to track the concept |
…rwrite-ignore-sql-checkout-new-branch #10813 fix(checkout): enforce --no-overwrite-ignore in DOLT_CHECKOUT -b via direct SQL
Dolt has no GOMEMLIMIT or GOGC awareness. For containerized deployments, this means the Go runtime doesn't know about cgroup memory limits, leading to OOM kills when RSS grows past the container limit.
This adds three-tier memory limit configuration:
DOLT_GOMEMLIMITenv var — explicit, human-readable values like "1GiB"automemlimitcgroup auto-detection — sets GOMEMLIMIT to 90% of the cgroup limitAlso adds
max_go_memoryto the YAML server config, applied at sql-server startup viadebug.SetMemoryLimit(), andMaxGoMemory() int64to theServerConfiginterface.Files changed:
go/cmd/dolt/memlimit.go(new) —configureMemoryLimit()with the three-tier logicgo/cmd/dolt/dolt.go— callsconfigureMemoryLimit()early inrunMain()go/cmd/dolt/commands/sqlserver/server.go— appliesmax_go_memoryat sql-server startupgo/libraries/doltcore/servercfg/—MaxGoMemory() int64on interface and implementations