element-web/playwright.config.ts
Michael Telatynski dbce48b23d
Run Playwright tests against Dendrite & Pinecone periodically (#28888)
* Switch to TestContainers for manging services in Playwright

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Flip fixture dependency order

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Remove mas dep

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Update matrix-authentication-service in Playwright tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* delint

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix SMTP port

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Comments

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Strip ansi from playwright logs to make them more readable

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Actually do the update

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Remove access to homeserver.config.baseUrl field in favour of homeserver.baseUrl

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Use sane default_server_config and specify server.invalid in the specific tests which demand it

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix mas run

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* break cycle

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* typo

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* prettier

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Wire up basics of dendriteHomeserver

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Run Playwright tests against Dendrite & Pinecone periodically

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix playwright-image-updates.yaml workflow

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Add `X-Run-All-Tests` label for running all tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Ignore failing tests in stale-screenshot-reporter.ts to avoid confusing errors

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Specify Synapse ui_auth.session_timeout only on tests which require it

As Dendrite lacks this configuration option

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* React to MatrixEvent sender/target sentinels being updated for rendering state events

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Remove test code

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* React to sentinel changes in EventListSummary

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Docs

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Avoid reusing user1234

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix stale-screenshot-reporter.ts

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Clean up public rooms between tests on reused homeserver

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Deflake spotlight when homeserver is reused

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Deflake more tests using existing username

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Clean mailhog between tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix more flakes

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix missing _request

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix playwright flaky tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Wipe mailhog between test runs

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Delint

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* delint

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Deflake more tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix flaky tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix flaky tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix mas config

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix another flaky test

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix playwright flakes due to floating promises

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix flaky playwright tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Update services.ts

---------

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2025-01-14 18:00:30 +00:00

96 lines
3.0 KiB
TypeScript

/*
Copyright 2024 New Vector Ltd.
Copyright 2023 The Matrix.org Foundation C.I.C.
SPDX-License-Identifier: AGPL-3.0-only OR GPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE files in the repository root for full details.
*/
import { defineConfig, devices } from "@playwright/test";
import { Options } from "./playwright/services";
const baseURL = process.env["BASE_URL"] ?? "http://localhost:8080";
const chromeProject = {
...devices["Desktop Chrome"],
channel: "chromium",
permissions: ["clipboard-write", "clipboard-read", "microphone"],
launchOptions: {
args: ["--use-fake-ui-for-media-stream", "--use-fake-device-for-media-stream", "--mute-audio"],
},
};
export default defineConfig<Options>({
projects: [
{
name: "Chrome",
use: {
...chromeProject,
},
},
{
name: "Firefox",
use: {
...devices["Desktop Firefox"],
launchOptions: {
firefoxUserPrefs: {
"permissions.default.microphone": 1,
},
},
// This is needed to work around an issue between Playwright routes, Firefox, and Service workers
// https://github.com/microsoft/playwright/issues/33561#issuecomment-2471642120
serviceWorkers: "block",
},
ignoreSnapshots: true,
},
{
name: "WebKit",
use: {
...devices["Desktop Safari"],
// Seemingly WebKit has the same issue as Firefox in Playwright routes not working
// https://playwright.dev/docs/network#missing-network-events-and-service-workers
serviceWorkers: "block",
},
ignoreSnapshots: true,
},
{
name: "Dendrite",
use: {
...chromeProject,
homeserverType: "dendrite",
},
ignoreSnapshots: true,
},
{
name: "Pinecone",
use: {
...chromeProject,
homeserverType: "pinecone",
},
ignoreSnapshots: true,
},
],
use: {
viewport: { width: 1280, height: 720 },
ignoreHTTPSErrors: true,
video: "retain-on-failure",
baseURL,
trace: "on-first-retry",
},
webServer: {
command: process.env.CI ? "npx serve -p 8080 -L ./webapp" : "yarn start",
url: `${baseURL}/config.json`,
reuseExistingServer: true,
timeout: (process.env.CI ? 30 : 120) * 1000,
},
testDir: "playwright/e2e",
outputDir: "playwright/test-results",
workers: 1,
retries: process.env.CI ? 2 : 0,
reporter: process.env.CI ? [["blob"], ["github"]] : [["html", { outputFolder: "playwright/html-report" }]],
snapshotDir: "playwright/snapshots",
snapshotPathTemplate: "{snapshotDir}/{testFilePath}/{arg}-{platform}{ext}",
forbidOnly: !!process.env.CI,
});