Skip to content

perf(i18n): split en.json shell + lazy rest chunk (#3535)#3563

Open
koala73 wants to merge 4 commits into
mainfrom
fix/i18n-en-shell-lazy-3535
Open

perf(i18n): split en.json shell + lazy rest chunk (#3535)#3563
koala73 wants to merge 4 commits into
mainfrom
fix/i18n-en-shell-lazy-3535

Conversation

@koala73
Copy link
Copy Markdown
Owner

@koala73 koala73 commented May 2, 2026

Closes #3535.

Summary

  • Splits the eagerly-imported src/locales/en.json (131KB raw / ~30-40KB gzipped) into a small shell subset that stays in the main JS chunk and a larger rest chunk that loads as a separate Vite chunk during initI18n().
  • Source of truth stays at en.json — the two derived files are regenerated by scripts/build-i18n-shell.mjs (wired into every prebuild/build:variant script). A parity test in tests/i18n-shell-split.test.mts blocks drift if a contributor edits en.json without re-running the script.
  • initI18n() boots i18next with the shell so chrome paints immediately, then awaits the rest before resolving so every t() call sees the full merged dictionary by the time any panel renders. No "raw t('key.path') flash" tradeoff to worry about.

Partition

Shell (eager, ~25KB raw / ~5KB gzipped):

  • Top-level: app, auth, common, connectivity, contextMenu, header, panels, widgets, premium, preferences, alerts, commands
  • Carve-out from components: panel (Panel.ts chrome), deckgl (region picker labels), map (header show/hide toggle)

Rest (lazy, ~84KB raw / ~30KB gzipped):

  • Remaining components.* (per-panel content), plus popups, modals, signals, countryBrief, intel, mcp

Measured chunk sizes

VITE_VARIANT=full npm run build:

chunk before after
i18n-*.js (i18next core + en payload) ~183 KB raw 52 KB raw / 16.6 KB gzipped
locale-en-rest-*.js (new, lazy) 83 KB raw / 30 KB gzipped

The lazy chunk fetches in parallel with everything else after initI18n() triggers it; the main JS chunk drops by ~130KB raw, which is what #3535 was after.

Test plan

  • npm run build:i18n-shell regenerates en.shell.json + en.rest.json from en.json deterministically.
  • npx tsx --test tests/i18n-shell-split.test.mts — 7/7 pass (round-trip parity, partition disjointness, byte cap, leaf-key reachability).
  • Full unit suite green: npx tsx --test tests/*.test.{mjs,mts} — 7836 tests pass.
  • npx tsc --noEmit clean.
  • VITE_VARIANT=full vite build succeeds and emits locale-en-rest-*.js as a distinct chunk that matches the existing SW /assets/locale-*.js/ CacheFirst rule.

Static-importing the full 131KB English bundle put all of it in the
main JS chunk on every page-load — including ~84KB of per-panel keys
that the user may never trigger by opening that panel.

Splits en.json into:
  - en.shell.json (~25KB raw / ~5KB gzipped): header, panel chrome,
    common toasts, command palette, premium gates, region picker.
    Statically imported, inlined into the i18n chunk.
  - en.rest.json (~84KB raw / ~30KB gzipped): per-panel content,
    modals, popups, signals, country brief. Dynamic-imported — Vite
    emits it as a separate locale-en-rest chunk (matching the
    existing /assets/locale-*.js/ SW caching rule).

initI18n() awaits both before resolving, so every t() call sees the
full merged dictionary by the time any panel renders. Source of truth
stays at en.json; build:i18n-shell regenerates the two derived files
on every prebuild and a parity test (tests/i18n-shell-split.test.mts)
blocks drift.

Measured chunk sizes after build (VITE_VARIANT=full):
  i18n chunk:        52KB  (was ~183KB with full en.json inlined)
  locale-en-rest:    83KB  (new, lazy)

Closes #3535
@vercel
Copy link
Copy Markdown

vercel Bot commented May 2, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
worldmonitor Ready Ready Preview, Comment May 3, 2026 4:03am

Request Review

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 2, 2026

Greptile Summary

Splits the 131 KB en.json into a ~25 KB eagerly-loaded shell (header, auth, panel chrome) and an ~84 KB lazy-loaded rest chunk, cutting the main JS bundle by ~130 KB raw. The partition script, Vite chunk rule, SW cache pattern, and parity test suite are all well-integrated.

  • P1 – loadEnFull() shallow merge drops shell's components subkeys: { ...enShell, ...rest } is a shallow spread; rest.components overwrites enShell.components, silently removing the panel, deckgl, and map carve-outs from the cached promise. The current init path is rescued by i18next's addResourceBundle(..., deep=true) call, but the promise itself is semantically incorrect and unsafe for future use.
  • P2 – build:desktop bypasses prebuild: it calls vite build directly, skipping build:i18n-shell; the script should chain npm run build:i18n-shell && explicitly like the other variant scripts do.

Confidence Score: 3/5

Hold for the shallow-merge fix in loadEnFull() — the current init path works by accident but the cached promise is semantically wrong.

One P1 bug (shallow merge in loadEnFull) that is mitigated at runtime by i18next's deep-merge behavior but leaves an incorrect cached promise that could silently break future callers or a re-init scenario. Score is below the P1 ceiling of 4 because the fix is a single line but the failure mode is subtle and not covered by the existing test suite.

src/services/i18n.ts lines 35-38 (loadEnFull merge logic); package.json build:desktop script.

Important Files Changed

Filename Overview
src/services/i18n.ts Boots i18next with the shell dict then lazy-loads the rest chunk; contains a P1 shallow-merge bug in loadEnFull() where rest.components overwrites enShell.components, dropping the panel/deckgl/map carve-outs from the cached promise.
scripts/build-i18n-shell.mjs New build script that correctly partitions en.json into shell and rest subsets; partition logic is clean and exports are well-structured for test reuse.
package.json Adds build:i18n-shell script and wires it into prebuild and all variant build scripts, but build:desktop still calls vite build directly and therefore skips the i18n-shell step.
vite.config.ts Adds a manualChunks rule to emit locale-en-rest-*.js as its own chunk (matching the existing SW CacheFirst regex) and excludes en.shell.json from the lazy locale glob — correctly architected.
tests/i18n-shell-split.test.mts Comprehensive parity, disjointness, byte-cap, and leaf-key tests; uses deepMerge to verify correctness but notably does not test loadEnFull() directly, leaving the shallow-merge bug undetected.

Sequence Diagram

sequenceDiagram
    participant App
    participant i18n as i18n.ts (initI18n)
    participant Shell as en.shell.json (eager, main chunk)
    participant Rest as en.rest.json (lazy chunk)
    participant i18next

    App->>i18n: initI18n()
    i18n->>Shell: static import (already in bundle)
    i18n->>i18next: init({ resources: { en: enShell } })
    Note over i18next: Chrome renders with shell keys
    i18n->>Rest: dynamic import('../locales/en.rest.json')
    Rest-->>i18n: rest dict
    i18n->>i18n: loadEnFull() → shallow { ...enShell, ...rest }<br/>⚠ drops components.panel/deckgl/map
    i18n->>i18next: addResourceBundle('en', fullEn, deep=true, overwrite=true)
    Note over i18next: deep=true rescues missing keys<br/>from prior shell registration
    i18n->>i18next: ensureLanguageLoaded(detectedLang)
    i18next-->>App: initI18n() resolves — full dict available
Loading

Comments Outside Diff (1)

  1. package.json, line 14 (link)

    P2 build:desktop bypasses prebuild and skips build:i18n-shell

    build:desktop calls vite build directly, not npm run build, so npm's prebuild lifecycle hook (which now includes build:i18n-shell) does not fire. A developer who edits en.json and then runs build:desktop will produce a desktop build with stale en.shell.json / en.rest.json — the parity test won't catch this at desktop-build time, only at npm test time.

Reviews (1): Last reviewed commit: "perf(i18n): split en.json into shell (ea..." | Re-trigger Greptile

Comment thread src/services/i18n.ts
Comment on lines +35 to +38
enFullPromise = import('../locales/en.rest.json').then((mod) => {
const rest = (mod.default ?? mod) as TranslationDictionary;
return { ...(enShell as TranslationDictionary), ...rest };
});
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Shallow spread drops components.panel/deckgl/map from loadEnFull()

{ ...enShell, ...rest } is a shallow merge. Because components is a top-level key in both en.shell.json (containing panel, deckgl, map) and en.rest.json (containing all other sub-panels), rest.components silently clobbers enShell.components, making loadEnFull() return a dict that is missing the three carved-out subkeys.

At initI18n() this is rescued by i18next's addResourceBundle(..., true, true) deep-merge path (the shell was already registered first), but the cached enFullPromise itself is incorrect. Any future caller that reads the resolved dict directly — or any re-init path that calls addResourceBundle with a fresh i18next instance — would lose components.panel.*, components.deckgl.*, and components.map.*.

Suggested change
enFullPromise = import('../locales/en.rest.json').then((mod) => {
const rest = (mod.default ?? mod) as TranslationDictionary;
return { ...(enShell as TranslationDictionary), ...rest };
});
enFullPromise = import('../locales/en.rest.json').then((mod) => {
const rest = (mod.default ?? mod) as TranslationDictionary;
// Shallow spread is insufficient: `components` exists in both shell and
// rest, so rest.components would clobber shell.components (panel/deckgl/map).
// Merge the two components sub-objects explicitly.
const shellDict = enShell as TranslationDictionary;
const merged: TranslationDictionary = { ...shellDict, ...rest };
if (shellDict.components && rest.components) {
merged.components = {
...(shellDict.components as Record<string, unknown>),
...(rest.components as Record<string, unknown>),
};
}
return merged;
});

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in f3d871d. Confirmed the bug is real — both files have a top-level components key (shell: panel/deckgl/map; rest: everything else), so shallow spread silently dropped the three shell carve-outs from the cached enFullPromise. Production was being rescued by i18next's deep-merge at init order (shell-then-rest), but the cached value itself was wrong — any fresh-i18next path or direct caller would lose the keys. Added explicit components-level merge + regression test in tests/i18n-shell-split.test.mts that asserts shallow spread WOULD drop the keys, so future "simplify" PRs fail loudly.

The previous chunk name locale-en-rest matched the SW's
globIgnores '**/locale-*.js' rule, which excluded it from the
precache. But initI18n() awaits the rest chunk before any panel
renders, so an offline boot (PWA installed, network drops before the
chunk is runtime-cached) would hang on the dynamic import.

Before the split, those strings shipped inside the precached i18n-*.js
chunk, so offline-after-install worked. Renaming the new chunk to
i18n-en-rest keeps it inside the default **/*.js precache glob and
out of the locale-only exclusion, restoring offline parity.

The user-selected per-language chunks stay as locale-<lang> —
those legitimately don't need to be precached and are runtime-cached
by the existing /assets/locale-.*\.js/ CacheFirst rule.

Confirmed via build: dist/sw.js precache manifest now includes
assets/i18n-en-rest-*.js (108 entries, up from 107) while the
per-language locale chunks remain runtime-only.
@koala73
Copy link
Copy Markdown
Owner Author

koala73 commented May 2, 2026

Fixed in 60f217e — thanks for catching this.

Root cause: chunk name locale-en-rest-*.js matched globIgnores: '**/locale-*.js', so VitePWA excluded it from the precache. But initI18n() awaits the rest chunk before any panel renders, so an offline-after-install boot would hang on the dynamic import. Pre-PR, those strings shipped inside the precached i18n-*.js chunk, so offline parity worked.

Fix: renamed the chunk to i18n-en-rest. That puts it inside the default **/*.js precache glob and outside the locale-only exclusion. The user-selected per-language chunks stay as locale-<lang> — those legitimately don't need precache and are runtime-cached by the existing /assets/locale-.*\.js/ CacheFirst rule.

Verified: dist/sw.js precache manifest now includes assets/i18n-en-rest-*.js (108 entries, up from 107). Per-language locale-* chunks remain runtime-only as before.

koala73 added a commit that referenced this pull request May 3, 2026
…401 crashes (#3566)

* fix(seeders): warm-pings send X-WorldMonitor-Key — stop relying on Origin-trust alone

Sweep companion to PR #3563 (c9cb13c — ais-relay) for the four remaining
seed scripts that POST to api.worldmonitor.app with `Origin:
https://worldmonitor.app` as their only auth credential. On 2026-05-02 all
three seed-infra warm-pings started returning HTTP 401 simultaneously;
process.exit(ok > 0 ? 0 : 1) made it look like a Railway container crash.

Root cause is identical to ais-relay's: Origin-trust via `api/_api-key.js`
BROWSER_ORIGIN_PATTERNS is fragile because CF/Vercel intermediaries can
strip the Origin header (S2S calls, WAF rules, cache hits) and CF can
cache the resulting 401 for the full s-maxage, poisoning a POP for ~30 min.
Local curl reproduction with identical headers returned `HTTP/2 401`
+ `cache-control: public, max-age=1800` — the smoking gun for per-POP
cache poisoning.

Defense-in-depth: send X-WorldMonitor-Key in addition to Origin. The
gateway already accepts it via validateApiKey's explicit-key branch
(api/_api-key.js:62-66). When the env var is unset the script falls
through to the existing Origin-only path — preserves backward
compatibility for local dev and for Railway services before the env var
is provisioned. Failure-log lines explicitly note when the key is missing
so future incidents are debuggable in 30 seconds.

Affected scripts:
  - scripts/seed-infra.mjs              (was crashing — exits 1 on all-401)
  - scripts/seed-military-maritime-news.mjs  (same warm-ping shape; would crash next CDN-poisoning event)
  - scripts/seed-service-statuses.mjs   (degrades to TTL-extension; caches drift silently)
  - scripts/seed-insights.mjs           (warmDigestCache was sending NO Origin at all → always 401, silent LKG fallback)

REQUIRED OPS STEP after merge (per Railway service):
  1. Pick a value from Vercel project env WORLDMONITOR_VALID_KEYS.
  2. Set WORLDMONITOR_RELAY_KEY=<that-value> on the Railway service.
  3. Redeploy.

The PR ships only the code change. Without the env var the scripts
continue current Origin-only behaviour (no regression). With it, auth
survives any future Origin-stripping intermediary.

Sweep recipe to catch this class of bug going forward:
  grep -ln "api.worldmonitor.app" scripts/*.{mjs,cjs} \
    | xargs grep -L "X-WorldMonitor-Key"

* fix(seed-insights): hoist RELAY_API_KEY to module scope to match siblings

Round-2 review on PR #3566: the three other scripts in this PR
(seed-infra.mjs, seed-military-maritime-news.mjs, seed-service-statuses.mjs)
all declare `const RELAY_API_KEY = process.env.WORLDMONITOR_RELAY_KEY || ''`
at module scope. seed-insights.mjs read the env var inside warmDigestCache()
on every call. Hoist for consistency so future copies of this pattern
inherit the right shape.
koala73 added 2 commits May 3, 2026 07:55
… footgun

Greptile P1 on #3563: en.shell.json and en.rest.json both have a top-level
`components` key (shell holds panel/deckgl/map; rest holds everything
else). The previous `{ ...shell, ...rest }` shallow spread silently
clobbered shell.components, dropping panel/deckgl/map from the cached
loadEnFull() result.

Production was rescued by i18next's deep-merge `addResourceBundle(...,
true, true)` because the shell was registered FIRST at init — but the
cached promise's value was wrong, and any future fresh-i18next instance
or direct caller would lose the carved-out keys.

- Explicit components-level merge in loadEnFull().
- Regression test in tests/i18n-shell-split.test.mts: pins the trap so
  a future "simplify back to shallow spread" PR will fail loudly.
Greptile residual-risk note on #3563: every web-build entrypoint already
runs `build:i18n-shell` (via `prebuild` for plain `npm run build`, and
inline in build:full/tech/finance/happy/commodity), but `build:desktop`
did not. Since `src-tauri/tauri.conf.json:beforeBuildCommand` invokes
`npm run build:desktop` for every Tauri target (desktop:build:full/
tech/finance + tauri dev), an `en.json` edit followed by a desktop
build would ship stale en.shell.json / en.rest.json against the new
en.json — a future bundle-drift trap.

One edit on `build:desktop` covers every desktop entrypoint because all
of them route through this script via Tauri's beforeBuildCommand chain.
@koala73
Copy link
Copy Markdown
Owner Author

koala73 commented May 3, 2026

Both Greptile findings addressed:

  • P1 — loadEnFull() shallow-merge bug → fixed in f3d871d. Explicit components-level merge + regression test in tests/i18n-shell-split.test.mts that asserts shallow spread WOULD drop components.panel/deckgl/map so future "simplify" PRs fail loudly.
  • P2 — build:desktop bypasses prebuild → fixed in e381b48. Added npm run build:i18n-shell && to the front of build:desktop. Single chokepoint covers every Tauri target because src-tauri/tauri.conf.json:beforeBuildCommand invokes npm run build:desktop for all desktop entrypoints (desktop:build:full/tech/finance + tauri dev).

fuleinist pushed a commit to fuleinist/worldmonitor that referenced this pull request May 9, 2026
…401 crashes (koala73#3566)

* fix(seeders): warm-pings send X-WorldMonitor-Key — stop relying on Origin-trust alone

Sweep companion to PR koala73#3563 (c9cb13c — ais-relay) for the four remaining
seed scripts that POST to api.worldmonitor.app with `Origin:
https://worldmonitor.app` as their only auth credential. On 2026-05-02 all
three seed-infra warm-pings started returning HTTP 401 simultaneously;
process.exit(ok > 0 ? 0 : 1) made it look like a Railway container crash.

Root cause is identical to ais-relay's: Origin-trust via `api/_api-key.js`
BROWSER_ORIGIN_PATTERNS is fragile because CF/Vercel intermediaries can
strip the Origin header (S2S calls, WAF rules, cache hits) and CF can
cache the resulting 401 for the full s-maxage, poisoning a POP for ~30 min.
Local curl reproduction with identical headers returned `HTTP/2 401`
+ `cache-control: public, max-age=1800` — the smoking gun for per-POP
cache poisoning.

Defense-in-depth: send X-WorldMonitor-Key in addition to Origin. The
gateway already accepts it via validateApiKey's explicit-key branch
(api/_api-key.js:62-66). When the env var is unset the script falls
through to the existing Origin-only path — preserves backward
compatibility for local dev and for Railway services before the env var
is provisioned. Failure-log lines explicitly note when the key is missing
so future incidents are debuggable in 30 seconds.

Affected scripts:
  - scripts/seed-infra.mjs              (was crashing — exits 1 on all-401)
  - scripts/seed-military-maritime-news.mjs  (same warm-ping shape; would crash next CDN-poisoning event)
  - scripts/seed-service-statuses.mjs   (degrades to TTL-extension; caches drift silently)
  - scripts/seed-insights.mjs           (warmDigestCache was sending NO Origin at all → always 401, silent LKG fallback)

REQUIRED OPS STEP after merge (per Railway service):
  1. Pick a value from Vercel project env WORLDMONITOR_VALID_KEYS.
  2. Set WORLDMONITOR_RELAY_KEY=<that-value> on the Railway service.
  3. Redeploy.

The PR ships only the code change. Without the env var the scripts
continue current Origin-only behaviour (no regression). With it, auth
survives any future Origin-stripping intermediary.

Sweep recipe to catch this class of bug going forward:
  grep -ln "api.worldmonitor.app" scripts/*.{mjs,cjs} \
    | xargs grep -L "X-WorldMonitor-Key"

* fix(seed-insights): hoist RELAY_API_KEY to module scope to match siblings

Round-2 review on PR koala73#3566: the three other scripts in this PR
(seed-infra.mjs, seed-military-maritime-news.mjs, seed-service-statuses.mjs)
all declare `const RELAY_API_KEY = process.env.WORLDMONITOR_RELAY_KEY || ''`
at module scope. seed-insights.mjs read the env var inside warmDigestCache()
on every call. Hoist for consistency so future copies of this pattern
inherit the right shape.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

131KB en.json statically imported by i18n — split into shell + lazy subsets

1 participant