Fix duplicate key detection when first value is null#3006
Conversation
MapTypeAdapterFactory detected duplicate map keys by checking the
return value of Map.put(): `if (replaced != null)`. This missed
duplicates where the first value was null, because Map.put() returns
null (the previous value) which passed the != null check.
For example, {"a":null,"a":1} was silently accepted as {a=1} instead
of throwing JsonSyntaxException("duplicate key: a").
Fix: check Map.containsKey() before put() instead of relying on the
return value of put(). This correctly detects duplicates regardless
of whether the first value is null.
Fixes both the object form and the array-of-entries form
(enableComplexMapKeySerialization).
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
|
Thanks! There's a small compatibility risk, in that some client code may have been implicitly relying on the old behaviour. I'm going to run this change past all of Google's internal tests and if there are no problems I think we can assume that the change is OK. |
eamonnmcmanus
left a comment
There was a problem hiding this comment.
No issues found when running all of Google's internal tests against this change.
…ip ci] Bumps [com.google.code.gson:gson](https://github.com/google/gson) from 2.13.2 to 2.14.0. Release notes *Sourced from [com.google.code.gson:gson's releases](https://github.com/google/gson/releases).* > Gson 2.14.0 > ----------- > > What's Changed > -------------- > > * Add type adapters for `java.time` classes by [`@eamonnmcmanus`](https://github.com/eamonnmcmanus) in [google/gson#2948](https://redirect.github.com/google/gson/pull/2948) > > When the `java.time` API is available, Gson automatically can read and write instances of classes like `Instant` and `Duration`. The format it uses essentially freezes the JSON representation that `ReflectiveTypeAdapterFactory` established by default, based on the private fields of `java.time` classes. That's not a great representation, but it is understandable. Changing it to anything else would break compatibility with systems that are expecting the current format. > > With this change, Gson no longer tries to access private fields of these classes using reflection. So it is no longer necessary to run with `--add-opens` for these classes on recent JDKs. > * Remove `com.google.gson.graph` by [`@eamonnmcmanus`](https://github.com/eamonnmcmanus) in [google/gson#2990](https://redirect.github.com/google/gson/pull/2990). > > This package was not part of any released artifact and depended on Gson internals in potentially problematic ways. > * Validate that strings being parsed as integers consist of ASCII characters by [`@eamonnmcmanus`](https://github.com/eamonnmcmanus) in [google/gson#2995](https://redirect.github.com/google/gson/pull/2995) > > Previously, strings could contain non-ASCII Unicode digits and still be parsed as integers. That's inconsistent with how JSON numbers are treated. > * Fix duplicate key detection when first value is null by [`@andrewstellman`](https://github.com/andrewstellman) in [google/gson#3006](https://redirect.github.com/google/gson/pull/3006) > > This could potentially break code that was relying on the incorrect behaviour. For example, this JSON string was previously accepted but will no longer be: `{"foo": null, "foo": bar}`. > * Remove `Serializable` from internal `Type` implementation classes. by [`@eamonnmcmanus`](https://github.com/eamonnmcmanus) in [google/gson#3011](https://redirect.github.com/google/gson/pull/3011) > > The nested classes `ParameterizedTypeImpl`, `GenericArrayTypeImpl`, and `WildcardTypeImpl` in `GsonTypes` are implementations of the corresponding types (without `Impl`) in `java.lang.reflect`. For some reason, they were serializable, even though the `java.lang.reflect` implementations are not. Having unnecessarily serializable classes could *conceivably* have been a security problem if they were part of a larger exploit using serialization. (We do not consider this a likely scenario and do not suggest that you need to update Gson just to get this change.) > * Add `LegacyProtoTypeAdapterFactory`. by [`@eamonnmcmanus`](https://github.com/eamonnmcmanus) in [google/gson#3014](https://redirect.github.com/google/gson/pull/3014) > > This is not part of any released artifact, but may be of use when trying to fix code that is currently accessing the internals of protobuf classes via reflection. > * Make AppendableWriter do flush and close if delegation object supports by [`@MukjepScarlet`](https://github.com/MukjepScarlet) in [google/gson#2925](https://redirect.github.com/google/gson/pull/2925) > > Other less visible changes > -------------------------- > > * Add default capacity to EnumTypeAdapter maps by [`@MukjepScarlet`](https://github.com/MukjepScarlet) in [google/gson#2959](https://redirect.github.com/google/gson/pull/2959) > * refactor: move derived adapters from Gson to TypeAdapters by [`@MukjepScarlet`](https://github.com/MukjepScarlet) in [google/gson#2951](https://redirect.github.com/google/gson/pull/2951) > * Optimize `new Gson()` by [`@MukjepScarlet`](https://github.com/MukjepScarlet) in [google/gson#2864](https://redirect.github.com/google/gson/pull/2864) > > New Contributors > ---------------- > > * [`@ThirdGoddess`](https://github.com/ThirdGoddess) made their first contribution in [google/gson#2944](https://redirect.github.com/google/gson/pull/2944) > * [`@lmj798`](https://github.com/lmj798) made their first contribution in [google/gson#2988](https://redirect.github.com/google/gson/pull/2988) > * [`@Eng-YasminKotb`](https://github.com/Eng-YasminKotb) made their first contribution in [google/gson#3005](https://redirect.github.com/google/gson/pull/3005) > * [`@andrewstellman`](https://github.com/andrewstellman) made their first contribution in [google/gson#3006](https://redirect.github.com/google/gson/pull/3006) > > **Full Changelog**: <google/gson@gson-parent-2.13.2...gson-parent-2.14.0> Commits * [`3ff35d6`](google/gson@3ff35d6) [maven-release-plugin] prepare release gson-parent-2.14.0 * [`a3024fd`](google/gson@a3024fd) Bump the maven group with 13 updates ([#3002](https://redirect.github.com/google/gson/issues/3002)) * [`5689ffe`](google/gson@5689ffe) Bump the github-actions group across 1 directory with 3 updates ([#3018](https://redirect.github.com/google/gson/issues/3018)) * [`48db33c`](google/gson@48db33c) Add `LegacyProtoTypeAdapterFactory`. ([#3014](https://redirect.github.com/google/gson/issues/3014)) * [`53d703e`](google/gson@53d703e) Update outdated comment regarding serializable types ([#3012](https://redirect.github.com/google/gson/issues/3012)) * [`0189b72`](google/gson@0189b72) Remove `Serializable` from internal `Type` implementation classes. ([#3011](https://redirect.github.com/google/gson/issues/3011)) * [`f4d371d`](google/gson@f4d371d) Fix duplicate key detection when first value is null ([#3006](https://redirect.github.com/google/gson/issues/3006)) * [`27d9ba1`](google/gson@27d9ba1) Fix typo in README (JPMS dependencies section) ([#3005](https://redirect.github.com/google/gson/issues/3005)) * [`1fa9b7a`](google/gson@1fa9b7a) Validate that strings being parsed as integers consist of ASCII characters (#... * [`b7d5954`](google/gson@b7d5954) Add iterator fail-fast tests for LinkedTreeMap.clear() ([#2992](https://redirect.github.com/google/gson/issues/2992)) * Additional commits viewable in [compare view](google/gson@gson-parent-2.13.2...gson-parent-2.14.0) [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
## Release notes Sourced from com.google.code.gson:gson's releases. Gson 2.14.0 What's Changed Add type adapters for java.time classes by @eamonnmcmanus in google/gson#2948 When the java.time API is available, Gson automatically can read and write instances of classes like Instant and Duration. The format it uses essentially freezes the JSON representation that ReflectiveTypeAdapterFactory established by default, based on the private fields of java.time classes. That's not a great representation, but it is understandable. Changing it to anything else would break compatibility with systems that are expecting the current format. With this change, Gson no longer tries to access private fields of these classes using reflection. So it is no longer necessary to run with --add-opens for these classes on recent JDKs. Remove com.google.gson.graph by @eamonnmcmanus in google/gson#2990. This package was not part of any released artifact and depended on Gson internals in potentially problematic ways. Validate that strings being parsed as integers consist of ASCII characters by @eamonnmcmanus in google/gson#2995 Previously, strings could contain non-ASCII Unicode digits and still be parsed as integers. That's inconsistent with how JSON numbers are treated. Fix duplicate key detection when first value is null by @andrewstellman in google/gson#3006 This could potentially break code that was relying on the incorrect behaviour. For example, this JSON string was previously accepted but will no longer be: {"foo": null, "foo": bar}. Remove Serializable from internal Type implementation classes. by @eamonnmcmanus in google/gson#3011 The nested classes ParameterizedTypeImpl, GenericArrayTypeImpl, and WildcardTypeImpl in GsonTypes are implementations of the corresponding types (without Impl) in java.lang.reflect. For some reason, they were serializable, even though the java.lang.reflect implementations are not. Having unnecessarily serializable classes could conceivably have been a security problem if they were part of a larger exploit using serialization. (We do not consider this a likely scenario and do not suggest that you need to update Gson just to get this change.) Add LegacyProtoTypeAdapterFactory. by @eamonnmcmanus in google/gson#3014 This is not part of any released artifact, but may be of use when trying to fix code that is currently accessing the internals of protobuf classes via reflection. Make AppendableWriter do flush and close if delegation object supports by @MukjepScarlet in google/gson#2925 Other less visible changes Add default capacity to EnumTypeAdapter maps by @MukjepScarlet in google/gson#2959 refactor: move derived adapters from Gson to TypeAdapters by @MukjepScarlet in google/gson#2951 Optimize new Gson() by @MukjepScarlet in google/gson#2864 New Contributors @ThirdGoddess made their first contribution in google/gson#2944 @lmj798 made their first contribution in google/gson#2988 @Eng-YasminKotb made their first contribution in google/gson#3005 @andrewstellman made their first contribution in google/gson#3006 Full Changelog: google/gson@gson-parent-2.13.2...gson-parent-2.14.0 Gson 2.13.2 The main changes in this release are just newer dependencies. ... (truncated) ## Changelog Sourced from com.google.code.gson:gson's changelog. Change Log The change log for versions newer than 2.10 is available only on the GitHub Releases page. Version 2.10 Support for serializing and deserializing Java records, on Java ≥ 16. (google/gson#2201) Add JsonArray.asList and JsonObject.asMap view methods (google/gson#2225) Fix TypeAdapterRuntimeTypeWrapper not detecting reflective TreeTypeAdapter and FutureTypeAdapter (google/gson#1787) Improve JsonReader.skipValue() (google/gson#2062) Perform numeric conversion for primitive numeric type adapters (google/gson#2158) Add Gson.fromJson(..., TypeToken) overloads (google/gson#1700) Fix changes to GsonBuilder affecting existing Gson instances (google/gson#1815) Make JsonElement conversion methods more consistent and fix javadoc (google/gson#2178) Throw UnsupportedOperationException when JsonWriter.jsonValue is not supported (google/gson#1651) Disallow JsonObject Entry.setValue(null) (google/gson#2167) Fix TypeAdapter.toJson throwing AssertionError for custom IOException (google/gson#2172) Convert null to JsonNull for JsonArray.set (google/gson#2170) Fixed nullSafe usage. (google/gson#1555) Validate TypeToken.getParameterized arguments (google/gson#2166) Fix #1702: Gson.toJson creates CharSequence which does not implement toString (google/gson#1703) Prefer existing adapter for concurrent Gson.getAdapter calls (google/gson#2153) Improve ArrayTypeAdapter for Object[] (google/gson#1716) Improve AppendableWriter performance (google/gson#1706) ## Commits 3ff35d6 [maven-release-plugin] prepare release gson-parent-2.14.0 a3024fd Bump the maven group with 13 updates (#3002) 5689ffe Bump the github-actions group across 1 directory with 3 updates (#3018) 48db33c Add LegacyProtoTypeAdapterFactory. (#3014) 53d703e Update outdated comment regarding serializable types (#3012) 0189b72 Remove Serializable from internal Type implementation classes. (#3011) f4d371d Fix duplicate key detection when first value is null (#3006) 27d9ba1 Fix typo in README (JPMS dependencies section) (#3005) 1fa9b7a Validate that strings being parsed as integers consist of ASCII characters (# b7d5954 Add iterator fail-fast tests for LinkedTreeMap.clear() (#2992) Additional commits viewable in compare view  Issue-ID: CIMAN-33 Signed-off-by: dependabot[bot] <support@github.com> Change-Id: I3707533b94e77d221389af97baae133d90bdb985 GitHub-PR: #37 GitHub-Hash: dc2f307754a500e1 Signed-off-by: onap.gh2gerrit <releng+onap-gh2gerrit@linuxfoundation.org>
Summary
MapTypeAdapterFactorydetected duplicate map keys during deserialization by checking the return value ofMap.put():This fails when the first occurrence of a duplicate key maps to
null.Map.put()returns the previous value, which isnull, so the check silently passes. The duplicate is accepted without throwing.Example:
gson.fromJson("{\"a\":null,\"a\":1}", type)returns{a=1}instead of throwingJsonSyntaxException("duplicate key: a").Fix
Check
Map.containsKey(key)before callingput()instead of relying on the return value ofput():Applied to both the object-form path and the array-of-entries path (
enableComplexMapKeySerialization).Tests added
MapTest.testMapDeserializationWithDuplicateKeysNullFirstValue— null first value, non-null secondMapTest.testMapDeserializationWithDuplicateKeysBothNull— both values nullMapAsArrayTypeAdapterTest.testDuplicateKeyWithNullFirstValueArrayForm— array-of-entries formAll 4,589 existing tests continue to pass.
How this was found
This bug was found by an AI-assisted code review generated by quality-playbook, an open-source skill for AI-driven code review. The playbook derived null-handling requirements from Gson's community documentation (GitHub issues #676, #913, #948, #1558), then used those requirements to ground a three-pass code review. The review flagged the
replaced != nullcheck as insufficient for null values — a finding that was then manually verified with a failing test.