Merged PRs

dolt

• 11076: More revert dirty-set rules to sync with git
  Git will reject revert when there is anything staged, regardless of conflicts or not in the changed. This change makes dolt follow the same pattern.
  Related: #11073
• 11074: go/store/nbs: journal.go: Fix a bug which could result in broken databases after a process crash.
  If Dolt crashed immediately after creating an empty journal file, and before it wrote the initial set-root record to it, then the next time Dolt ran it would treat the root of the database as 0000....
  This could also happen for certain observed filesystem states after an operating system crash.
• 11072: go/store/nbs: Delete the in-flight table file in (*fileTablePersister).writeAndProtect when we experience an error on the write.
  Leaving it behind needlessly wastes disk space. It will never be usable and is in an indeterminate state.
  This is currently relevant even when FatalBehaviorFatal, because Conjoin's write is currently allowed to fail before we take the hard dependency on it. There are still some I/O errors which leave the file behind, including a failure at ftp.Open().
• 11063: Tests for IndexedJsonDocument.Compare
  When an IndexedJsonDocument compares a sql.JsonDocument of a different type, it falls back to using Json.Compare. This PR adds tests to ensure that the two implementations match.
• 10775: Fix open Dependabot security alerts
  Upgrade golang.org/x/image v0.18.0 -> v0.38.0 to address CVE for out-of-memory via crafted TIFF file. Upgrade @mikro-orm/core and @mikro-orm/mysql from v5 to v6.6.10+ to address critical SQL injection and high-severity prototype pollution vulnerabilities.

go-mysql-server

• 3551: star expression used in function
• 3550: modify CompareJSON to use sorted object keys

Closed Issues

Merged PRs

dolt

• 11073: dolt revert to more closely match git

  Related: dolthub/doltlite#961

• 11067: proto,go/serial: Bump some bazel version, some module versions.

• 11059: Fix SchemaTable.Updater panic on direct UPDATE dolt_schemas
  A direct UPDATE dolt_schemas reached SchemaTable.Updater, which panicked with "should never be called" — an unprivileged user could crash the SQL server with it. dolt_schemas is not directly modifiable via SQL, so this returns a clean error unconditionally instead of panicking; adds a regression test.

• 11058: go: sqle: Fix a dolt_branch_control bypass involving the session table cache.
  On a new connection, a SELECT against information_schema.tables or information_schema.columns would populate the session's table cache is a Table value that embedded a revision-less Database value. An incoming write request against the cached table would run its permissions check against the revision-less database, instead of the revisioned database which correctly encoded the branch the write was running against. This allowed connections to bypass dolt_branch_control, elevating to write permissions on all branches.

• 11057: Branch_control regression tests for more user-table write shapes
  Adds branch_control regression tests for the user-table write shapes that TestBranchControlBlocks wasn't already pinning; all were already correctly gated, so this is pure regression coverage with no behavior change.

• 11050: Gate branch_control on writable dolt system tables
  Most writable dolt system tables had no branch_control check, so a read- or merge-permission user could write to them; this gates the user-space tables (dolt_docs/ignore/query_catalog/nonlocal/tests via the shared createWriteableSystemTable helper), dolt_constraint_violations_, dolt_workspace_, and ConflictRootObjectTable with Permissions_Write. dolt_branches/dolt_remotes (already read-only), dolt_branch_control/namespace (own admin gating), and dolt_conflicts_ (already gated) are left as-is; tests cover every writer factory and pin the read-only behavior.

• 11049: Audit and test branch control coverage for dolt procedures
  Survey of dprocedures/dolt_*.go turned up four procedures with no branch_control check that mutate branch or working-set state. Three of them belong under Permissions_Write:

  • dolt_rebase — mutates the active rebase plan and the working set
  • dolt_stash — push/pop/apply/drop/clear all write through to working state
  • dolt_update_column_tag — schema change committed to the working root
    Gate added at the procedure entry in each.
    dolt_undrop was the fourth — leaving it alone. It restores a dropped database, not a table on the current branch, so the branch-control entry for the current branch isn't a sensible gate. MySQL SUPER already covers it.
    Test side: every other dolt_* procedure now has at least one branch_control assertion.

• 11047: Gate DOLT_CHECKOUT('<table>') with branch_control Write permission
  dolt_checkout(<table>) was previously ungated, so a user with a read or merge permission on a branch could clear working set changes. This gates dolt_checkout(<table>) while leaving dolt_checkout(<branch>) ungated

go-mysql-server

• 3551: star expression used in function
• 3546: Cache context to use for String() methods
  This is a replacement for:
  • dolthub/go-mysql-server#3525
    This accomplishes the same effective goal of passing context in the areas that we previously were not (primarily the String() function, which is load-bearing). The above PR changed the interfaces such that nodes and expressions no longer respected the fmt.Stringer interface. This PR instead caches the context inside of all nodes that need a context in their String() function by providing that context at node creation time. This was a tactic that we were already using before my original context threading PR:
  • dolthub/go-mysql-server#3513
    ...and should therefore be no worse than we were before I embarked on this journey.
    It's worth noting that the information_schema tables do not have a proper context inside of their String() function, as those tables are created when the engine is created, and SQL contexts only exist inside of connections/sessions. For now this doesn't impact anything as the Schema(ctx) methods (which use the context from String()) return a precomputed schema, however there are comments in place that warn about the nil context if those methods are ever modified.
• 3532: replace all decimal.Decimal with *apd.Decimal

Closed Issues

• 11066: Connection Close hangs indefinitely when attempting to usthe implicit sql transaction commit of aCALL DOLT_COMMIT`
• 11046: Overflowing TEXT type does not error and silently truncates, and mysql session variables do not work to make it error.

Merged PRs

dolt

• 11028: drop remote db cache, Close() after each use
  SSH subprocesses created by sql-server instances were failing to clean up because the Close() method was never being called. This was the result of some caching done by GetRemoteDB in the sqle package. Most of this change is to remove sqle.DoltDatabaseProvider.GetRemoteDB() caching, allowing for push/pull operations to safely Close() remote databases after use. Thus, preventing the PID leak in the associated issue (#10897)
  Git remote database singletons are a special case. Previously the push operation only would cache these instances - which are expensive to create initially since they need to perform git operations to get in sync with the remote. There is no reason go Close() them though, so we keep a singleton for the duration of the sql-server process.
  Fixes: #10897
• 10990: replace all decimal.Decimal with *apd.Decimal
  Depends on dolthub/go-mysql-server#3532
• 10953: fix autogc load scheduler to be independent on number of CPUs
  Before, we had a threshold that was incorrectly calculating the threshold based off the number of CPUs.
  The original intention is to prevent AutoGC if a single CPU core exceeds 50% usage.
  We've determined that CPU Load itself is not a great metric to schedule AutoGC, but this is a fix for now.
  addresses: #10944

go-mysql-server

• 3539: Bug fix for dropping sort nodes
  Also fixed a couple under-specified goup concat tests.
  The latter were relying on a particular row storage order which is not guaranteed and broke when Dolt changed some encoding parameters.
• 3532: replace all decimal.Decimal with *apd.Decimal

Closed Issues

• 10944: Auto-GC never fires on multi-core systems: loadAvgGCScheduler threshold formula is inverted
• 9248: support returning clause in inserts like mariadb ?
• 10897: dolt sql-server leaks one ssh child process per CALL dolt_fetch against an ssh:// remote

Merged PRs

dolt

• 11040: fixed compat tests for a post-2.0 CI
• 11038: remove adapative encoding matrixes
  Now that adaptive encoding is on by default, we don't need to double our CI testing efforts anymore.
• 11035: Refactor prollyWriteSession and prollyTableWriter for individual table flushing
  This PR refactors flushing logic between prollyWriteSession and prollyTableWriter.
  Now, the prollyTableWriter is in charge of materializing the table and handling the auto increment logic, while the prollyWriteSession is in charge of keeping the workingSet up to date.
  The flush all table logic has been reworked in prollyWriteSession to use channels instead of a mutex.
  Additionally, this PR allows us to flush individual tables rather than every table in the session; this avoids concurrency overheads and no-op flushes.
• 11032: Let merge-permission users write through dolt_conflicts_
  Adds a branch-control carve-out so a user with only Permissions_Merge on a branch can resolve a merge that produced data conflicts by writing through dolt_conflicts_<t> and via DOLT_CONFLICTS_RESOLVE. Lets a PR reviewer on the SQL workbench finish a conflicting merge without being granted full write access on the target branch.
• 11030: Gate DELETE on dolt_conflicts_ with branch_control Write permission
• 11029: Allow index lookups on DOLT_DIFF table function.
  Currently, the DOLT_DIFF table function does not expose any indexes. But it can.
• 11020: .github: Factor out build-dolt and setup-dolt-config actions. Use them from workflows that want dolt binaries, like bats and sql-server-integration tests.
• 10853: chore: fix function name in comment to match actual function
  fix function name in comment to match actual function

Closed Issues

• 11041: FK action ON DELETE SET DEFAULT (and ON UPDATE SET DEFAULT) rejected at CREATE TABLE

Performance

Merged PRs

dolt

• 11006: Preserve tables absent from the index during checkout, reset, and rebase
  Tables present only in the working tree (never added to staging) were being deleted or incorrectly staged by the three commands.
  • Scope checkout . table list to HEAD and staged root
  • Update working and staged roots independently in the specific-table checkout case
  • Add MoveUntrackedTables helper for the working-tree preservation step in reset --hard and
    rebase
  • Apply rebase preservation via direct UpdateWorkingSet before SwitchWorkingSet
  • Fix dolt reset --soft <rev> to leave the index untouched
  • Rename ResetSoftToRef to MoveHeadToRef and resetSoftToRef to resetMixedToRef so names match
    behavior
• 11002: When dolt diff -r sql encounters a schema change, it should either print the full diff or return an error
  Previously, whenever a table had a schema change, dolt diff -r sql would skip printing the data diff for that table. It would print a message to stderr, but still return a 0 error code. This is misleading, especially if the diff command is called by an automated process.
  This PR improves the situations where we successfully render the data diff. In situations where we can't, we return a nonzero error code.

Closed Issues

• 3470: Subscribe to changes to a document/query (live query)
• 11014: ROW_COUNT function returns 0 on INSERT statement

Dolt 2.0 is the second major release of Dolt. It includes all the performance improvements and new features of Dolt 1.x releases in the 3 years since the 1.0 release, as well as several major milestones:

• Automatic garbage collection on by default
• Archival storage on by default
• Faster than MySQL on sysbench
• Beta support for Vector data
• Adaptive storage for TEXT, JSON, GEOMETRY, and BLOB types (similar to TOAST in Postgres)

Dolt 2.0 is backwards compatible with all 1.x releases of Dolt. Databases written by 2.x releases will not be readable by 1.x clients in all cases.

Merged PRs

dolt

• 11019: .github/actions/setup-go-toolchain: Make a common toolchain setup action which go test and bats test workflows can depend on.
  This is responsible for installing Golang toolchain and getting ICU4C in the environment so it can be used.
• 11017: flip flag for adaptive encoding

Closed Issues

Performance

Merged PRs

dolt

• 11011: Final 1.x PR before turning on adaptive encoding by default for 2.0
  This PR contains all the changes necessary to make CI pass with adaptive encoding on by default.

go-mysql-server

• 3539: Bug fix for dropping sort nodes
  Also fixed a couple under-specified goup concat tests.
  The latter were relying on a particular row storage order which is not guaranteed and broke when Dolt changed some encoding parameters.

Closed Issues

• 11014: ROW_COUNT function returns 0 on INSERT statement

This is a major version release to mark it as the last major version release before 2.0. There are no known backwards incompatible changes.

Merged PRs

dolt

• 11005: add a breaking change to schemas with adaptive columns
  Previously released versions had a fatal bug in adaptive encoded value handling that would lead to data loss during garbage collection or pulls. This change forces these clients to upgrade to a newer release before reading databases that use such encodings.
  Because no 1.x release of Dolt uses adaptive encoding without undocumented configuration flags, the fatal bug above does not impact any customer. This change prevents it from impacting any customers after the release of 2.0.

Closed Issues

Backwards Incompatible Changes

Different author and committer additions introduced are not interpretable by old Dolt clients. If you attempt to set an explicit committer that is different from the author, this will create a new field in storage. The dolt_log system table and procedure schemas have also been changed to be static. Compatibility for older clients is not guaranteed for all flags, e.g. --show-signature. Furthermore, the new Dolt client requires the Dolt server to be updated to this latest version to access all expected columns.

Merged PRs

dolt

• 10996: cleaned up binlog serialization logic
• 10995: Fixed deserialization for adaptive types during binlog replication
• 10993: Fix dolt rm --cached to allow unstaged working changes
  dolt rm --cached incorrectly rejected tables with unstaged working changes or tables already dropped from the working set.
  • --cached branch filters tables absent from working via HasTable before calling RemoveTables.
    Fix #10987
• 10983: Fix dolt_log table and procedure schema to be static and --decoration behavior
  dolt_log now exposes a fixed 12-column schema across the system table and procedure. Parents and signature columns remain opt-in via projection for the system table, and arguments in the dolt_log() procedure.
  • --parents and --show-signature no longer mutate the result of schema
  • Make parents and signature columns populate only when projected for dolt_log system table
  • --decorate=auto is deprecated in the sql-server context since TTY cannot be determined
  • --decorate=auto in CLI now correctly detects interactive terminal
  • Update bats coverage to be bidirectional for the server and client testing in compatibility
    Blocked by dolthub/vitess#467
• 10979: adaptive encoding tests for mysql client integration
• 10975: Use TARGET_ROW_SIZE table attribute to guide how aggressively we move adaptive encoded values out-of-band.
  TARGET_ROW_SIZE is a table attribute that can be set on tables. Dolt interprets this value as a size in bytes. When building a tuple, we attempt to keep the tuple under this size by moving adaptive-encoded values out-of-band until the size of the tuple falls below the threshold.
  Because Dolt does not allow tuples larger than 64K, we can always store the target value in 16-bits.
  The GMS interfaces use a uint64 for TARGET_ROW_SIZE, since other storage backends wouldn't necessarily have the same limit. When setting a value for TARGET_ROW_SIZE, Dolt verifies that it fits in 16 bits.
  Currently, this value can only be set while creating the table and can't be modified afterward.
• 10974: Db/git remote UI
  This PR updates git remotes to push a branch visible on the Git remote's UI that contains a file with some info about the git ref used as the dolt remote. Addresses #10525
• 10969: go-sql-server-driver tests for large adaptive and out-of-band values
  These tests exercise the replication, pull, and GC behavior for various kinds of schemas with large values that may be stored out of band. These tests failed before the inclusion of address serialization bug fixes from #10966
  Depends on dolthub/go-mysql-server#3533
• 10966: bug fix: write addresses field in tuples if they contain only adaptive encoded fields
• 10964: .github: ci-lambdabats-unix-adaptive.yaml: Add lambdabats DOLT_USE_ADPATIVE_ENCODING=true runs.
• 10962: /.github/workflows: fix maven link
• 10959: When merging, use a tuple builder to ensure that we generate the correct representation for the merged row.
  This matters for adaptive encoding columns, whose correct representation depends on other values in the row.
• 10958: Fix obscure race: make sure that RunF finishes before StopF in sqlServer.Start()
  This is a candidate for race conditions leading to nil pointer exceptions in some doltgres harness setups.
• 10952: adaptive encoding for JSON columns
  This will be turned on for the 2.0 release.
• 10941: build(deps): bump go.opentelemetry.io/otel from 1.40.0 to 1.41.0 in /go
  Bumps go.opentelemetry.io/otel from 1.40.0 to 1.41.0.
• 10933: build(deps): bump fast-xml-parser from 5.5.9 to 5.7.2 in /.github/actions/ses-email-action
  Bumps fast-xml-parser from 5.5.9 to 5.7.2.
• 10832: build(deps): bump go.opentelemetry.io/otel/sdk from 1.40.0 to 1.43.0 in /go
  Bumps go.opentelemetry.io/otel/sdk from 1.40.0 to 1.43.0.
• 10827: build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.78.0 to 1.97.3 in /go
  Bumps github.com/aws/aws-sdk-go-v2/service/s3 from 1.78.0 to 1.97.3.
• 10804: #10813 fix(checkout): enforce --no-overwrite-ignore in DOLT_CHECKOUT -b via direct SQL

  Summary

  When calling DOLT_CHECKOUT('-b', ...) directly via SQL (without --move), the --no-overwrite-ignore flag was silently ignored. CheckOverwrittenIgnoredTables was never called for the isMove=false path in checkoutNewBranch, so creating a branch from a non-HEAD start point would succeed even when ignored tables in the working set differed from the start point.
  The fix captures the current working roots before commitTransaction (which resets session state, making GetRoots return false afterward), then runs the ignored table check before switching the working set to the new branch.

  Changes

  • Call CheckOverwrittenIgnoredTables in the isMove=false path of checkoutNewBranch
  • Capture roots before commitTransaction to avoid session state reset
  • Add engine script tests and bats tests covering --no-overwrite-ignore and
    --overwrite-ignore with a non-HEAD start point
    Closes: #10813
• 10801: build(deps): bump lodash from 4.17.23 to 4.18.1 in /integration-tests/mysql-client-tests/node
  Bumps lodash from 4.17.23 to 4.18.1.* 10104: Add separate author and committer support
  Dolt now supports separate author (original intent) and committer (actual writer) identity for Dolt commits! Expect commit, cherry-pick, revert and merge operations (including SQL procedures) to pick up different author and committers automatically. The dolt_log, dolt_commits, dolt_diff, dolt_column_diff, dolt_branches, dolt_remote_branches system tables and respective table functions have been updated with author columns too.

  ⚠️ The committer columns in system tables or functions now show the committer identity instead of author. In old Dolt versions this was the result of no committer identity existing in storage, but has since been updated. We maintain backward-compatibility with this old format by duplicating the author identity into the committer identity. At serialization, however, the committer is only stored if the name or email is different from the author identity, allowing us to maintain the same commit hash history.
  ⚠️ We've appended the author columns under to end of each table to prevent issues with older Dolt clients.

New environment variables for committer identity are now available: DOLT_COMMITTER_NAME, DOLT_COMMITTER_EMAIL, DOLT_COMMITTER_DATE which take inspiration from their git counterparts.

⚠️ Environment variables (not SQL system variables) are only interpretable by the Dolt client. MySQL or alternative SQL clients will not pick these Dolt-specific environment variables up automatically in your terminal.

export DOLT_COMMITTER_NAME="Committer User"
export DOLT_COMMITTER_EMAIL="committer@example.com"
export DOLT_COMMITTER_DATE="2025-01-01T12:00:00Z"
dolt commit -m "Commit with custom committer date"

+----------------------------------+----------------+-----------------------+---------------------+--------------------------------------+--------------+-----------+------------------+---------------------+
| commit_hash                      | committer      | email                 | date                | message                              | commit_order | author    | author_email     | author_date         |
+----------------------------------+----------------+-----------------------+---------------------+--------------------------------------+--------------+-----------+------------------+---------------------+
| tdf81ch4ug4c9uc2p78rls1iihj95unn | Committer User | committer@example.com | 2025-01-01 12:00:00 | Commit with custom committer date    | 4            | Test User | test@example.com | 2025-11-19 10:15:47 |
| 0gonkh11sf1esld53b66jdpbu78sdq6i | Committer User | committer@example.com | 202...

Merged PRs

dolt

• 10947: go/store: nbs,types: Fix some error paths in GC.
  1. If gcCopier.copyTablesToDir failed after its writer.Finish() call succeeded, it would cancel the writer but leave gcc.writer != nil. Then markAndSweeper.Close would call gcc.cancel and get a spurious error from the failed writer.Cancel call.
  2. ValueStore.gc would fail to Close MarkAndSweeper if Finalize returned an error. The contract was that Close needed to be called on all paths.
  3. markAndSweepChunks dropped an error from newRotatingGCCopier.
  4. markAndSweeper Close early returned on an error from gcc.cancel, failing to close and cleanup incrementalGcc in that case.
• 10750: Add incremental mode for garbage collection
  This feature is enabled by providing the --incremental-file-size flag to dolt gc or call dolt_gc() with a positive value, and can be enabled for automatic GC with a setting in the server's config.yaml, eg:

  behavior:
  auto_gc_behavior:
  incremental_file_size: 1000000
  

  In this example, after GC has written 1000000 bytes of leaf chunks (chunks that do not reference other chunks), it will finish the chunk file and begin writing a new one.
  There are two main reasons to do this:
  • Chunks written this way don't need to be tracked in the GC "visited set", reducing memory usage.
  • If GC is interrupted (for instance, it may be terminated by the OS for using too much memory), the already-processed chunks will be preserved, and subsequent runs of GC will have less work to do and require less memory. This provides an escape hatch for the scenario where a user has not run GC in so long that GC requires more memory to run than is available.

go-mysql-server

• 3527: Truncate invalid UTF-8 on INSERT IGNORE and warn on LIKE with bad charset pattern
  • INSERT IGNORE with invalid UTF-8 in a utf8mb4 column now truncates at the first bad byte
  • LIKE with an invalid UTF-8 pattern emits warning 1300 and return no match
  • Exported TruncateInvalidUTF8 from sql/types
    Fix #10924
    Blocks #10926
• 3526: Update for functional expression index parser changes
  Parser changes support multiple functional expressions, or a mix of columns and functional expressions in an index. GMS still restricts functional indexes to a single functional expression, but that restriction will be removed in the next PR.
  Also includes a new test interface that Doltgres will use when integrating with functional indexes.
  Depends on: dolthub/vitess#466

vitess

• 466: Add parser support for multiple functional expressions in an index
• 465: go.mod: Bump to golang 1.26.2.
• 462: Added context to Injectable
  Needed for:
  • dolthub/go-mysql-server#3513

Closed Issues

• 10924: dolt table import should error when importing invalid charset
• 10945: RemotesAPI push: working-set-stale bug from #10807 still reproducible in v1.86.5

Performance