Implement basics for E2E tests driven by playwright #6038
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
zoldar
force-pushed
the
e2e-exploration-playwright
branch
5 times, most recently
from
bdf0a5a to
80ee512
Compare
This reverts commit 6105473.
zoldar
force-pushed
the
e2e-exploration-playwright
branch
from
0386ab7 to
a44d103
Compare
zoldar
force-pushed
the
e2e-exploration-playwright
branch
from
33f6ec9 to
bd620a1
Compare
This reverts commit 3372a82.
zoldar
changed the title
apata
reviewed
Feb 5, 2026
apata
reviewed
Feb 5, 2026
apata
approved these changes
Feb 5, 2026
github-merge-queue bot
pushed a commit
that referenced
this pull request
Feb 5, 2026
* [PoC] Implement basics for E2E tests driven by playwright (JS) * Update tracker test run instructions in README * Update package config on e2e tests * Set playwright reporter format to list * Add mix task for running e2e tests in UI mode * Add e2e test run job to CI config * Add missing PG and CH services to e2e CI config * Increase timeouts for e2e CI tasks * Temporarily remove tzdata updater * Revert "Temporarily remove tzdata updater" This reverts commit 6105473. * Add step downloading geo data * Setup E2E seeds before running CI * Run e2e tests via mix task to ensure setup env variables * Set `BASE_URL` env var explicitly in CI config instead * Reduce shards to 1 * Change how cwd is set for playwright server * Show server log output in e2e tests * Install asset dependencies during e2e setup * Build assets as well * Hide server output in e2e tests again * Parallelize e2e tests again * Reduce test sharding from 4 to 2 * Cache more fetched and compiled assets * Remove redundant tests * Clean up playwright config slightly * Try reducing the time spent fetching system deps * Output screenshots on failure * Upload screenshots from failed tests to GH artifacts * Make one test fail on purpose * Revert "Make one test fail on purpose" This reverts commit 3372a82. * Update gitignore inside e2e to ignore outputDir * Add notes about how to run e2e tests locally * Add preferred envs for E2E mix tasks * Don't dump screenshots and don't upload them as GH artifacts * Rely on cached `tracker/node_modules` and not install tracker conditionally * Remove no longer relevant comments from mix.exs
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
This PR implements basis for running end-to-end dashboard tests using playwright.
e2e_testMIX environment is introduced for running the tests./priv/repo/e2e_seeds.exs- it might be subject to change if we decide to change out approach to setting up system state for tests - it's being discussed internallyThe tests can be run locally in two ways:
mix test.e2e- standard mode outputting test report in the terminalmix test.e2e.ui- interactive mode running in a dedicated window, with ability to run tests and review the outputs selectivelyWhen run on CI, the tests are sharded (just two shards for now).
Failed test report for a single shard:
https://github.com/plausible/analytics/actions/runs/21710287043/job/62611828894
Full test report summary available under
merge-sharded-e2e-test-reportjob (after expanding "Merge into list report"):https://github.com/plausible/analytics/actions/runs/21710287043/job/62612031552
Screenshots from failed tests on CI are available as downloadable artifacts from CI worflow summary page, like this one:
https://github.com/plausible/analytics/actions/runs/21710287043
While we could work towards fancier report output, I'd opt for keeping things simple for now and not adding en explicit report the the PR thread just yet, to reduce noise.
There are only a couple example tests for now. Follow-up work will involve expanding it into a more comprehensive set of test scenarios.