chore: Use local clickhouse instance for playwright tests #1711

teeohhem · 2026-02-06T15:15:50Z

TLDR: This PR changes playwright full-stack tests to run against a local clickhouse instance (with seeded data) instead of relying on the clickhouse demo server, which can be unpredictable at times. This workflow allows us to fully control the data to make tests more predictable.

This PR:

Adds local CH instance to the e2e dockerfile
Adds a schema creation script
Adds a data seeding script
Updates playwright config
Updates various tests to change hardcoded fields, metrics, or areas relying on play demo data
Updates github workflow to use the dockerfile instead of separate services
Runs against a local clickhouse instead of the demo server

Fixes: HDX-3193

changeset-bot · 2026-02-06T15:15:55Z

⚠️ No Changeset found

Latest commit: f0a2f64

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

vercel · 2026-02-06T15:15:56Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
hyperdx-v2-oss-app	Ready	Preview, Comment	Feb 13, 2026 2:31pm

claude · 2026-02-06T15:17:41Z

Code Review

✅ No critical issues found.

Minor observations:

seed-clickhouse.ts:24-29: ClickHouse credentials passed via URL params → Consider using HTTP headers for better security (though acceptable for local E2E tests)
seed-clickhouse.ts:154,274: Only first 10 logs/sessions linked to traces → This is intentional for testing trace correlation
docker-compose.yml:24: CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT=1 enables dynamic user management → Appropriate for E2E isolated environment
init-db-e2e.sh:56-57: 30-day TTL on all test tables → Good practice to prevent data accumulation

Strengths:

Proper separation of E2E tables (e2e_ prefix) to avoid collision with dev data
Comprehensive test data generation (logs, traces, sessions, K8s metrics)
Good error handling and timeout configuration throughout
Seeding uses time window that includes future data to prevent flaky tests
Fixtures centralized in e2e-fixtures.json for consistency

The PR successfully replaces dependency on external demo server with local ClickHouse, improving test reliability and control.

- Add Docker Compose service for local ClickHouse instance - Add init script to create e2e_* tables in ClickHouse - Add seed-clickhouse.ts to populate test data - Add global-setup-local.ts for local mode setup - Update test scripts to support local ClickHouse - Update .env.e2e to point to local ClickHouse instead of demo server - Add stop-e2e.sh script for cleanup This enables E2E tests to run against a local Docker ClickHouse instance with seeded test data, instead of relying on the public demo server. Both full-stack mode (MongoDB + API + ClickHouse) and local mode (frontend + ClickHouse) now use the same local ClickHouse instance.

The global-setup-fullstack.ts was missing the seedClickHouse() call, which meant tests had no data to work with.

- Add saveSearchAndWaitForNavigation() to avoid race condition - Starts waiting for URL change BEFORE submitting form - Eliminates flaky tests where modal closes but URL hasn't changed yet - Replaces manual pattern of await modal hidden + await URL change - Simplify search-filters.spec.ts to use known test data - Use hardcoded 'info' severity from seeded data instead of dynamic discovery - Remove 18 lines of complex conditional logic - More reliable and easier to understand These changes leverage consistent test data from ClickHouse seeding.

github-actions · 2026-02-06T15:20:57Z

E2E Test Results

❌ 1 test failed • 65 passed • 4 skipped • 791s

Status	Count
✅ Passed	65
❌ Failed	1
⚠️ Flaky	0
⏭️ Skipped	4

Tests ran across 4 shards in parallel.

View full report →

teeohhem · 2026-02-11T00:37:06Z

packages/app/tests/e2e/core/navigation.spec.ts

      await test.step('Navigate between each page', async () => {
        for (const { testId, contentTestId } of navLinks) {
          const link = page.locator(`[data-testid="${testId}"]`);
+          await link.scrollIntoViewIfNeeded();


reduces flakiness

teeohhem · 2026-02-11T00:41:38Z

packages/app/scripts/run-e2e.js

+  env.DEFAULT_CONNECTIONS = JSON.stringify(fixture.connections ?? []);
+  env.DEFAULT_SOURCES = JSON.stringify(fixture.sources ?? []);
+}
+


simplify loading connections/sources instead of one big blob in an env file

teeohhem · 2026-02-11T00:42:10Z

packages/app/tests/e2e/components/SavedSearchModalComponent.ts

+    // Optionally wait for modal to fully close
+    await expect(this.container).toBeHidden();
+  }
+


abstraction for saved searches (lots of flakiness exposed when moving away from play demo server)

teeohhem · 2026-02-11T00:42:43Z

packages/app/tests/e2e/features/search/saved-search.spec.ts

-
-        await expect(searchPage.savedSearchModal.container).toBeHidden();
-        await page.waitForURL(/\/search\/[a-f0-9]+/, { timeout: 5000 });
      });


simplify these tests with more predictable input

teeohhem · 2026-02-11T00:52:45Z

.github/workflows/main.yml

+        run:
+          docker compose -p e2e -f packages/app/tests/e2e/docker-compose.yml
+          down -v
+


use docker file instead

teeohhem · 2026-02-11T00:52:56Z

docker/clickhouse/local/init-db-e2e.sh

+ORDER BY (ServiceName, MetricName, Attributes, toUnixTimestamp64Nano(TimeUnix))
+TTL toDate(TimeUnix) + toIntervalDay(30)
+SETTINGS index_granularity = 8192, ttl_only_drop_parts = 1
+EOFSQL


create schema for local db

teeohhem · 2026-02-11T00:53:14Z

packages/api/.env.e2e

-
+# DEFAULT_CONNECTIONS and DEFAULT_SOURCES are injected from packages/app/tests/e2e/fixtures/e2e-fixtures.json
+# by the e2e API runner (run-api-with-fixtures.js) and by base-test for local mode.



moved to fixtures

teeohhem · 2026-02-11T00:53:41Z

packages/app/tests/e2e/features/search/search-filters.spec.ts

    //verify there is a pin icon
    const pinIcon = searchPage.page.getByTestId(
-      `filter-pin-${availableFilterValue!}-pinned`,
+      `filter-pin-${TEST_FILTER_VALUE}-pinned`,


greatly simplify this logic now that we completely control the data

pulpdrew · 2026-02-11T15:58:41Z

packages/app/tests/e2e/utils/constants.ts

@@ -1,3 +1,5 @@
 // We may want to add logs/metrics source names here in the future.


nit

Suggested change

// We may want to add logs/metrics source names here in the future.

pulpdrew · 2026-02-11T17:29:02Z

scripts/stop-e2e.sh

@@ -0,0 +1,10 @@
+#!/bin/bash


Is this file used anywhere?

pulpdrew · 2026-02-11T17:34:11Z

docker/clickhouse/local/init-db-e2e.sh

@@ -0,0 +1,209 @@
+#!/bin/bash


I had to run chmod +x docker/clickhouse/local/init-db-e2e.sh to get this script to run in docker desktop - are you able to run the same and commit the extra permission to this PR?

pulpdrew · 2026-02-11T17:35:38Z

Makefile

-	if [ -n "$(tags)" ]; then set -- "$$@" --tags "$(tags)"; fi; \
-	if [ "$(ui)" = "true" ]; then set -- "$$@" --ui; fi; \
-	./scripts/test-e2e.sh "$$@"
+	@# For more control (--ui, --last-failed, --headed, etc), call the script directly:


packages/app/tests/e2e/README.md still suggests make e2e ui=true, could that be updated?

pulpdrew · 2026-02-11T17:48:06Z

packages/app/tests/e2e/seed-clickhouse.ts

+  const rows: string[] = [];
+
+  for (let i = 0; i < count; i++) {
+    const timestampNs = (now - i * 60000) * 1000000; // 1 minute intervals


One thing I noticed while running the tests locally for a few minutes is that tests (the search tests for example) quickly start failing because data is not continuously being inserted for newer/more current timestamps, so when a search filters for "last 5 minutes" there is no data, if seeding ran >5 minutes ago. Would it make sense to see data "in the future" as well, to prevent this sort of thing in most cases?

…dx into tom/e2e-local-ch-minimal

teeohhem changed the title ~~Tom/e2e local ch minimal~~ feat: Use local clickhouse instance for playwright tests Feb 6, 2026

teeohhem changed the title ~~feat: Use local clickhouse instance for playwright tests~~ chore: Use local clickhouse instance for playwright tests Feb 6, 2026

vercel bot deployed to Preview February 6, 2026 15:18 View deployment

teeohhem added 6 commits February 6, 2026 10:18

fix: seed ClickHouse in fullstack mode global setup

912a098

The global-setup-fullstack.ts was missing the seedClickHouse() call, which meant tests had no data to work with.

small test updates

785cfb6

simplify search filters test

31135ca

saved search update

ee12780

small improvements

8d16aca

teeohhem force-pushed the tom/e2e-local-ch-minimal branch from 3d7cb2b to 8d16aca Compare February 6, 2026 15:25

vercel bot deployed to Preview February 6, 2026 15:28 View deployment

small network tweak

751645b

vercel bot deployed to Preview February 6, 2026 15:43 View deployment

cleanup

124324f

vercel bot deployed to Preview February 10, 2026 18:23 View deployment

use docker file in ci/cid

7a5eff8

vercel bot deployed to Preview February 10, 2026 20:02 View deployment

fix metric name

30209d3

vercel bot deployed to Preview February 10, 2026 20:48 View deployment

tweaks for local mode + cleanup

0e4c5cc

teeohhem commented Feb 11, 2026

View reviewed changes

vercel bot deployed to Preview February 11, 2026 00:38 View deployment

teeohhem commented Feb 11, 2026

View reviewed changes

teeohhem marked this pull request as ready for review February 11, 2026 00:52

teeohhem requested a review from pulpdrew February 11, 2026 00:52

teeohhem commented Feb 11, 2026

View reviewed changes

pulpdrew reviewed Feb 11, 2026

View reviewed changes

review feedback

00f89ed

vercel bot deployed to Preview February 11, 2026 20:23 View deployment

Merge remote-tracking branch 'origin/main' into tom/e2e-local-ch-minimal

945aaad

vercel bot deployed to Preview February 12, 2026 14:23 View deployment

vercel bot deployed to Preview February 12, 2026 14:39 View deployment

vercel bot deployed to Preview February 12, 2026 15:58 View deployment

search filter test adjustments

962daf9

teeohhem force-pushed the tom/e2e-local-ch-minimal branch from a19cf47 to 962daf9 Compare February 12, 2026 16:20

vercel bot deployed to Preview February 12, 2026 16:23 View deployment

fix saved search tests + seed data

959d3f8

vercel bot deployed to Preview February 12, 2026 18:34 View deployment

more cleanup

d0be30b

vercel bot deployed to Preview February 12, 2026 18:58 View deployment

teeohhem marked this pull request as draft February 12, 2026 19:07

update

60f0741

vercel bot deployed to Preview February 12, 2026 19:14 View deployment

Merge branch 'main' into tom/e2e-local-ch-minimal

f32ea16

vercel bot deployed to Preview February 12, 2026 20:43 View deployment

teeohhem added 2 commits February 13, 2026 09:28

improvements to reduce flakiness

682e41d

Merge branch 'tom/e2e-local-ch-minimal' of github.com:hyperdxio/hyper…

f0a2f64

…dx into tom/e2e-local-ch-minimal

vercel bot deployed to Preview February 13, 2026 14:31 View deployment


		# DEFAULT_CONNECTIONS and DEFAULT_SOURCES are injected from packages/app/tests/e2e/fixtures/e2e-fixtures.json
		# by the e2e API runner (run-api-with-fixtures.js) and by base-test for local mode.

		@@ -1,3 +1,5 @@
		// We may want to add logs/metrics source names here in the future.

chore: Use local clickhouse instance for playwright tests #1711

Are you sure you want to change the base?

chore: Use local clickhouse instance for playwright tests #1711

Conversation

teeohhem commented Feb 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

changeset-bot bot commented Feb 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

vercel bot commented Feb 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

claude bot commented Feb 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review

Minor observations:

Strengths:

Uh oh!

github-actions bot commented Feb 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

E2E Test Results

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

teeohhem commented Feb 6, 2026 •

edited

Loading

changeset-bot bot commented Feb 6, 2026 •

edited

Loading

vercel bot commented Feb 6, 2026 •

edited

Loading

claude bot commented Feb 6, 2026 •

edited

Loading

github-actions bot commented Feb 6, 2026 •

edited

Loading