refactor: refactor sharded wal to handle coordinator by BioPhoton · Pull Request #1237 · code-pushup/cli

BioPhoton · 2026-02-03T18:35:42Z

Precondition:

refactor: simplify trace types and helper #1236

Implement performance profiling and trace file generation #1197

This PR includes:

split sharded wal logic
handle coordinator in sharder

Followup:

refactor: use sharded WAL #1231

nx-cloud · 2026-02-03T18:36:44Z

View your CI Pipeline Execution ↗ for commit 55816b9

Command	Status	Duration	Result
`nx run ci:code-pushup -- merge-diffs --files=/h...`	✅ Succeeded	5s	View ↗
`nx run-many --targets=code-pushup --parallel=fa...`	✅ Succeeded	1m 31s	View ↗
`nx run-many --targets=code-pushup --parallel=fa...`	✅ Succeeded	13m 49s	View ↗
`nx run-many -t unit-test,int-test`	✅ Succeeded	13s	View ↗

☁️ Nx Cloud last updated this comment at 2026-02-23 15:39:50 UTC

…rdinator-in-sharded-wal

# Conflicts: # packages/utils/src/lib/wal.ts # packages/utils/src/lib/wal.unit.test.ts

…-sharded-wal' into feat/utils/handle-coordinator-in-sharded-wal

pkg-pr-new · 2026-02-17T11:42:53Z

Open in StackBlitz

@code-pushup/ci

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/ci@1237

@code-pushup/cli

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/cli@1237

@code-pushup/core

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/core@1237

@code-pushup/create-cli

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/create-cli@1237

@code-pushup/models

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/models@1237

@code-pushup/axe-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/axe-plugin@1237

@code-pushup/nx-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/nx-plugin@1237

@code-pushup/eslint-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/eslint-plugin@1237

@code-pushup/js-packages-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/js-packages-plugin@1237

@code-pushup/coverage-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/coverage-plugin@1237

@code-pushup/jsdocs-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/jsdocs-plugin@1237

@code-pushup/lighthouse-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/lighthouse-plugin@1237

@code-pushup/typescript-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/typescript-plugin@1237

@code-pushup/utils

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/utils@1237

commit: 55816b9

github-actions · 2026-02-23T08:47:07Z

Code PushUp

🤨 Code PushUp report has both improvements and regressions – compared current commit b46851f with previous commit dd6e35e.

🕵️ See full comparison in Code PushUp portal 🔍

🏷️ Categories

🏷️ Category	⭐ Previous score	⭐ Current score	🔄 Score change
Security	🔴 39	🔴 0
Performance	🔴 36	🔴 45
Best Practices	🟢 100	🟢 97
Updates	🟡 75	🟡 72
Documentation	🟡 53	🟡 53
Code coverage	🟢 93	🟢 93
Bug prevention	🟡 75	🟡 75
Axe Accessibility	🟡 88	🟡 88
Code style	🟢 100	🟢 100	–
Type Safety	🟡 67	🟡 67	–
Miscellaneous	🟡 67	🟡 67	–
Accessibility	🟢 92	🟢 92	–
SEO	🟢 92	🟢 92	–

👍 3 groups improved, 👎 3 groups regressed, 👍 9 audits improved, 👎 10 audits regressed, 17 audits changed without impacting score

🗃️ Groups

🔌 Plugin	🗃️ Group	⭐ Previous score	⭐ Current score
JS packages	npm audit	🔴 39	🔴 0
Lighthouse	Performance	🔴 36	🔴 45
Lighthouse	Best Practices	🟢 100	🟢 97
JS packages	npm outdated dependencies	🟡 75	🟡 72
JSDocs coverage	Documentation coverage	🟡 53	🟡 53
Code coverage	Code coverage metrics	🟢 93	🟢 93

28 other groups are unchanged.

🛡️ Audits

🔌 Plugin	🛡️ Audit	📏 Previous value	📏 Current value
Lighthouse	Initial server response time was short	🟩 Root document took 510 ms	🟥 Root document took 670 ms
Lighthouse	No issues in the `Issues` panel in Chrome Devtools	🟩 passed	🟥 failed
Lighthouse	Reduce unused JavaScript	🟥 Potential savings of 225 KiB	🟨 Potential savings of 256 KiB
Lighthouse	Remove duplicate modules in JavaScript bundles	🟥 Potential savings of 93 KiB	🟨 Potential savings of 80 KiB
JS packages	Vulnerabilities for npm prod dependencies.	🟥 12 vulnerabilities (3 high, 2 moderate, 7 low)	🟥 22 vulnerabilities (13 high, 3 moderate, 6 low)
Lighthouse	Largest Contentful Paint	🟥 11.6 s	🟥 4.7 s
Lighthouse	Speed Index	🟥 7.3 s	🟥 6.2 s
Lighthouse	First Contentful Paint	🟥 3.1 s	🟥 3.2 s
JS packages	Outdated npm prod dependencies.	🟨 20 outdated package versions (6 major, 10 minor, 4 patch)	🟨 21 outdated package versions (7 major, 11 minor, 3 patch)
JSDocs coverage	Methods coverage	🟨 10 undocumented methods	🟨 10 undocumented methods
JSDocs coverage	Properties coverage	🟥 43 undocumented properties	🟥 47 undocumented properties
JS packages	Outdated npm dev dependencies.	🟨 62 outdated package versions (29 major, 25 minor, 8 patch)	🟨 61 outdated package versions (30 major, 25 minor, 6 patch)
Lighthouse	Total Blocking Time	🟥 1,630 ms	🟥 1,670 ms
JSDocs coverage	Variables coverage	🟥 273 undocumented variables	🟥 274 undocumented variables
Code coverage	Branch coverage	🟨 89.2 %	🟨 89 %
Code coverage	Function coverage	🟩 94.4 %	🟩 94.4 %
JSDocs coverage	Types coverage	🟥 264 undocumented types	🟥 265 undocumented types
Code coverage	Line coverage	🟩 92.7 %	🟩 92.7 %
JSDocs coverage	Functions coverage	🟥 585 undocumented functions	🟥 584 undocumented functions
Lighthouse	Avoids enormous network payloads	🟩 Total size was 2,091 KiB	🟩 Total size was 2,137 KiB
Lighthouse	Uses efficient cache policy on static assets	🟨 31 resources found	🟨 31 resources found
Lighthouse	Server Backend Latencies	🟩 310 ms	🟩 930 ms
Lighthouse	Minimizes main-thread work	🟥 10.3 s	🟥 9.9 s
Lighthouse	Reduce unused CSS	🟥 Potential savings of 111 KiB	🟥 Potential savings of 110 KiB
Lighthouse	JavaScript execution time	🟥 3.0 s	🟥 3.1 s
Lighthouse	Metrics	🟩 100%	🟩 100%
Lighthouse	Time to Interactive	🟥 13.5 s	🟥 13.6 s
Lighthouse	Max Potential First Input Delay	🟥 990 ms	🟥 940 ms
JS packages	Vulnerabilities for npm dev dependencies.	🟥 48 vulnerabilities (3 critical, 9 high, 32 moderate, 4 low)	🟥 81 vulnerabilities (3 critical, 65 high, 10 moderate, 3 low)
Lighthouse	Network Round Trip Times	🟩 30 ms	🟩 60 ms
Lighthouse	Avoids an excessive DOM size	🟥 2,366 elements	🟥 2,360 elements
Axe	Elements must only use supported ARIA attributes	🟩 101 elements	🟩 96 elements
Axe	ARIA attributes must be used as specified for the element's role	🟩 101 elements	🟩 96 elements
Axe	Elements must only use permitted ARIA attributes	🟩 101 elements	🟩 96 elements
Axe	ARIA attributes must conform to valid names	🟩 101 elements	🟩 96 elements
Lighthouse	Cumulative Layout Shift	🟩 0	🟩 0

643 other audits are unchanged.

github-actions · 2026-02-23T09:01:37Z

Code PushUp

🤨 Code PushUp report has both improvements and regressions – compared current commit b46851f with previous commit dd6e35e.

💼 Project `utils`

🤨 Code PushUp report has both improvements and regressions.

🕵️ See full comparison in Code PushUp portal 🔍

🏷️ Category	⭐ Previous score	⭐ Current score	🔄 Score change
Documentation	🟡 60	🟡 61
Code coverage	🟢 95	🟢 95

4 other categories are unchanged.

👍 1 group improved, 👎 1 group regressed, 👍 5 audits improved, 👎 3 audits regressed

🗃️ Groups

🔌 Plugin	🗃️ Group	⭐ Previous score	⭐ Current score	🔄 Score change
JSDocs coverage	Documentation coverage	🟡 60	🟡 61
Code coverage	Code coverage metrics	🟢 95	🟢 95

13 other groups are unchanged.

🛡️ Audits

🔌 Plugin	🛡️ Audit	📏 Previous value	📏 Current value
JSDocs coverage	Methods coverage	🟨 10 undocumented methods	🟨 10 undocumented methods
JSDocs coverage	Properties coverage	🟥 40 undocumented properties	🟥 44 undocumented properties
Code coverage	Branch coverage	🟩 91.8 %	🟩 91.3 %
JSDocs coverage	Types coverage	🟨 55 undocumented types	🟨 56 undocumented types
JSDocs coverage	Variables coverage	🟥 49 undocumented variables	🟥 50 undocumented variables
Code coverage	Function coverage	🟩 95.8 %	🟩 95.9 %
JSDocs coverage	Functions coverage	🟥 243 undocumented functions	🟥 242 undocumented functions
Code coverage	Line coverage	🟩 97.8 %	🟩 97.8 %

436 other audits are unchanged.

💼 Project `plugin-lighthouse`

🤨 Code PushUp report has both improvements and regressions.

🕵️ See full comparison in Code PushUp portal 🔍

All of 6 categories are unchanged.

1 audit changed without impacting score

🗃️ Groups

All of 15 groups are unchanged.

🛡️ Audits

🔌 Plugin	🛡️ Audit	📏 Previous value	📏 Current value	🔄 Value change
Code coverage	Branch coverage	🟩 98.8 %	🟩 98.8 %

443 other audits are unchanged.

12 other projects are unchanged.

packages/utils/src/lib/wal.int.test.ts

packages/utils/src/lib/wal-sharded.unit.test.ts

packages/utils/src/lib/wal-sharded.int.test.ts

hanna-skryl · 2026-02-23T14:09:51Z

packages/utils/src/lib/wal-sharded.ts

+      return ++ShardedWal.instanceCount;
+    },
+  });
+  readonly groupId = getUniqueTimeId();


The groupId default value is always overwritten and never used. Consider declaring the field without a default.

hanna-skryl · 2026-02-23T14:28:55Z

packages/utils/src/lib/wal-sharded.ts

+    // create dir if not existing
+    ensureDirectoryExistsSync(groupDir);


I found it a bit hard to follow which layer is responsible for creating which directory. The finalize path calls ensureDirectoryExistsSync three times across two methods.

What do you think about moving the directory creation into finalize() so that shardFiles() stays a pure read method?

hanna-skryl · 2026-02-23T14:41:54Z

packages/utils/src/lib/wal.ts

-}
-
-/**
+ * NOTE: this helper is only used in this file. The rest of the repo avoids sync methods so it is not reusable.


The comment is misleading as ensureDirectoryExistsSync is actually exported and used in other files.

hanna-skryl · 2026-02-23T14:44:43Z

packages/utils/src/lib/wal-sharded.ts

+        : this.getCreatedShardFiles();
+
+    return {
+      lastRecover: this.#lastRecovery,


Is there a specific reason why the names differ so that lastRecover maps to lastRecovery?

packages/utils/src/lib/wal-sharded.int.test.ts

hanna-skryl · 2026-02-23T14:59:38Z

packages/utils/src/lib/wal-sharded.ts

+ * Handles distributed logging across multiple processes/files with atomic finalization.
+ */
+
+export class ShardedWal<T extends WalRecord = WalRecord> {


Nit: stale profiler references in JSDoc.

setCoordinatorProcess and isCoordinatorProcess have JSDoc and parameter names from before the generalization. profilerID param name, references to "origin PID", "enabled profiling", and CP_PROFILER_ORIGIN_PID. These could be updated to match the generic coordinator terminology used everywhere else in the class.

packages/utils/src/lib/wal-sharded.ts

Co-authored-by: Hanna Skryl <80118140+hanna-skryl@users.noreply.github.com>

refactor: refactor sharded wal to handle coordinator

ffcc95d

BioPhoton requested a review from hanna-skryl February 3, 2026 18:35

github-actions bot assigned BioPhoton Feb 3, 2026

github-actions bot added 🔬 testing writing tests 🧩 utils labels Feb 3, 2026

BioPhoton removed the request for review from hanna-skryl February 3, 2026 18:37

This was referenced Feb 3, 2026

refactor: simplify trace types and helper #1236

Merged

Implement performance profiling and trace file generation #1197

Open

BioPhoton added 2 commits February 8, 2026 22:24

Merge remote-tracking branch 'origin/main' into feat/utils/handle-coo…

cc0b481

…rdinator-in-sharded-wal

refactor: wip

aa0395f

github-actions bot added the 🛠️ tooling label Feb 8, 2026

BioPhoton added 2 commits February 17, 2026 18:09

Merge branch 'main' into feat/utils/handle-coordinator-in-sharded-wal

000be48

# Conflicts: # packages/utils/src/lib/wal.ts # packages/utils/src/lib/wal.unit.test.ts

Merge remote-tracking branch 'origin/feat/utils/handle-coordinator-in…

234a83b

…-sharded-wal' into feat/utils/handle-coordinator-in-sharded-wal

github-actions bot removed the 🛠️ tooling label Feb 17, 2026

BioPhoton added 3 commits February 17, 2026 18:34

refactor: wip

eddf9cc

refactor: wip

f8f2847

refactor: wip

14e6390

BioPhoton added 5 commits February 17, 2026 20:47

refactor: wip

75ccc48

refactor: update const name

2d94b12

refactor: fix int tests

6719873

refactor: fix unit tests

2e8910b

refactor: fix lint

d150c70

BioPhoton mentioned this pull request Feb 23, 2026

refactor: use sharded WAL #1231

Open

BioPhoton marked this pull request as ready for review February 23, 2026 08:53

BioPhoton requested a review from hanna-skryl February 23, 2026 08:53

hanna-skryl reviewed Feb 23, 2026

View reviewed changes

BioPhoton and others added 5 commits February 23, 2026 16:16

Update packages/utils/src/lib/wal.int.test.ts

b46e330

Co-authored-by: Hanna Skryl <80118140+hanna-skryl@users.noreply.github.com>

Update packages/utils/src/lib/wal-sharded.int.test.ts

ca1c0c8

Co-authored-by: Hanna Skryl <80118140+hanna-skryl@users.noreply.github.com>

Update packages/utils/src/lib/wal-sharded.int.test.ts

0e72e8c

Co-authored-by: Hanna Skryl <80118140+hanna-skryl@users.noreply.github.com>

Update packages/utils/src/lib/wal-sharded.unit.test.ts

d2e885d

Co-authored-by: Hanna Skryl <80118140+hanna-skryl@users.noreply.github.com>

Update packages/utils/src/lib/wal-sharded.ts

55816b9

Co-authored-by: Hanna Skryl <80118140+hanna-skryl@users.noreply.github.com>

		// create dir if not existing
		ensureDirectoryExistsSync(groupDir);

Comments

Conversation

BioPhoton commented Feb 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nx-cloud bot commented Feb 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pkg-pr-new bot commented Feb 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code PushUp

🏷️ Categories

🗃️ Groups

🛡️ Audits

Uh oh!

github-actions bot commented Feb 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code PushUp

💼 Project utils

🗃️ Groups

🛡️ Audits

💼 Project plugin-lighthouse

🗃️ Groups

🛡️ Audits

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hanna-skryl Feb 23, 2026

Choose a reason for hiding this comment

Uh oh!

hanna-skryl Feb 23, 2026

Choose a reason for hiding this comment

Uh oh!

hanna-skryl Feb 23, 2026

Choose a reason for hiding this comment

Uh oh!

hanna-skryl Feb 23, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

hanna-skryl Feb 23, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

BioPhoton commented Feb 3, 2026 •

edited

Loading

nx-cloud bot commented Feb 3, 2026 •

edited

Loading

pkg-pr-new bot commented Feb 17, 2026 •

edited

Loading

github-actions bot commented Feb 23, 2026 •

edited

Loading

github-actions bot commented Feb 23, 2026 •

edited

Loading

💼 Project `utils`

💼 Project `plugin-lighthouse`