element-web/playwright/e2e/crypto/backups.spec.ts
Michael Telatynski dbce48b23d
Run Playwright tests against Dendrite & Pinecone periodically (#28888)
* Switch to TestContainers for manging services in Playwright

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Flip fixture dependency order

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Remove mas dep

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Update matrix-authentication-service in Playwright tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* delint

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix SMTP port

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Comments

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Strip ansi from playwright logs to make them more readable

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Actually do the update

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Remove access to homeserver.config.baseUrl field in favour of homeserver.baseUrl

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Use sane default_server_config and specify server.invalid in the specific tests which demand it

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix mas run

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* break cycle

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* typo

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* prettier

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Wire up basics of dendriteHomeserver

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Run Playwright tests against Dendrite & Pinecone periodically

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix playwright-image-updates.yaml workflow

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Add `X-Run-All-Tests` label for running all tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Ignore failing tests in stale-screenshot-reporter.ts to avoid confusing errors

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Specify Synapse ui_auth.session_timeout only on tests which require it

As Dendrite lacks this configuration option

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* React to MatrixEvent sender/target sentinels being updated for rendering state events

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Remove test code

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* React to sentinel changes in EventListSummary

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Docs

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Avoid reusing user1234

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix stale-screenshot-reporter.ts

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Clean up public rooms between tests on reused homeserver

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Deflake spotlight when homeserver is reused

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Deflake more tests using existing username

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Clean mailhog between tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix more flakes

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix missing _request

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix playwright flaky tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Wipe mailhog between test runs

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Delint

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* delint

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Deflake more tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix flaky tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix flaky tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix mas config

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix another flaky test

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix playwright flakes due to floating promises

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix flaky playwright tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Update services.ts

---------

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2025-01-14 18:00:30 +00:00

119 lines
6.0 KiB
TypeScript

/*
Copyright 2024 New Vector Ltd.
Copyright 2023 The Matrix.org Foundation C.I.C.
SPDX-License-Identifier: AGPL-3.0-only OR GPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE files in the repository root for full details.
*/
import { type Page } from "@playwright/test";
import { test, expect } from "../../element-web-test";
import { isDendrite } from "../../plugins/homeserver/dendrite";
async function expectBackupVersionToBe(page: Page, version: string) {
await expect(page.locator(".mx_SecureBackupPanel_statusList tr:nth-child(5) td")).toHaveText(
version + " (Algorithm: m.megolm_backup.v1.curve25519-aes-sha2)",
);
await expect(page.locator(".mx_SecureBackupPanel_statusList tr:nth-child(6) td")).toHaveText(version);
}
test.describe("Backups", () => {
test.skip(isDendrite, "Dendrite lacks support for MSC3967 so requires additional auth here");
test.use({
displayName: "Hanako",
});
test(
"Create, delete and recreate a keys backup",
{ tag: "@no-webkit" },
async ({ page, user, app }, workerInfo) => {
// Create a backup
const securityTab = await app.settings.openUserSettings("Security & Privacy");
await expect(securityTab.getByRole("heading", { name: "Secure Backup" })).toBeVisible();
await securityTab.getByRole("button", { name: "Set up", exact: true }).click();
const currentDialogLocator = page.locator(".mx_Dialog");
// It's the first time and secure storage is not set up, so it will create one
await expect(currentDialogLocator.getByRole("heading", { name: "Set up Secure Backup" })).toBeVisible();
await currentDialogLocator.getByRole("button", { name: "Continue", exact: true }).click();
await expect(currentDialogLocator.getByRole("heading", { name: "Save your Security Key" })).toBeVisible();
await currentDialogLocator.getByRole("button", { name: "Copy", exact: true }).click();
// copy the recovery key to use it later
const securityKey = await app.getClipboard();
await currentDialogLocator.getByRole("button", { name: "Continue", exact: true }).click();
await expect(currentDialogLocator.getByRole("heading", { name: "Secure Backup successful" })).toBeVisible();
await currentDialogLocator.getByRole("button", { name: "Done", exact: true }).click();
// Open the settings again
await app.settings.openUserSettings("Security & Privacy");
await expect(securityTab.getByRole("heading", { name: "Secure Backup" })).toBeVisible();
// expand the advanced section to see the active version in the reports
await page
.locator(".mx_Dialog .mx_SettingsSubsection_content details .mx_SecureBackupPanel_advanced")
.locator("..")
.click();
await expectBackupVersionToBe(page, "1");
await securityTab.getByRole("button", { name: "Delete Backup", exact: true }).click();
await expect(currentDialogLocator.getByRole("heading", { name: "Delete Backup" })).toBeVisible();
// Delete it
await currentDialogLocator.getByTestId("dialog-primary-button").click(); // Click "Delete Backup"
// Create another
await securityTab.getByRole("button", { name: "Set up", exact: true }).click();
await expect(currentDialogLocator.getByRole("heading", { name: "Security Key" })).toBeVisible();
await currentDialogLocator.getByLabel("Security Key").fill(securityKey);
await currentDialogLocator.getByRole("button", { name: "Continue", exact: true }).click();
// Should be successful
await expect(currentDialogLocator.getByRole("heading", { name: "Success!" })).toBeVisible();
await currentDialogLocator.getByRole("button", { name: "OK", exact: true }).click();
// Open the settings again
await app.settings.openUserSettings("Security & Privacy");
await expect(securityTab.getByRole("heading", { name: "Secure Backup" })).toBeVisible();
// expand the advanced section to see the active version in the reports
await page
.locator(".mx_Dialog .mx_SettingsSubsection_content details .mx_SecureBackupPanel_advanced")
.locator("..")
.click();
await expectBackupVersionToBe(page, "2");
// ==
// Ensure that if you don't have the secret storage passphrase the backup won't be created
// ==
// First delete version 2
await securityTab.getByRole("button", { name: "Delete Backup", exact: true }).click();
await expect(currentDialogLocator.getByRole("heading", { name: "Delete Backup" })).toBeVisible();
// Click "Delete Backup"
await currentDialogLocator.getByTestId("dialog-primary-button").click();
// Try to create another
await securityTab.getByRole("button", { name: "Set up", exact: true }).click();
await expect(currentDialogLocator.getByRole("heading", { name: "Security Key" })).toBeVisible();
// But cancel the security key dialog, to simulate not having the secret storage passphrase
await currentDialogLocator.getByTestId("dialog-cancel-button").click();
await expect(currentDialogLocator.getByRole("heading", { name: "Starting backup…" })).toBeVisible();
// check that it failed
await expect(currentDialogLocator.getByText("Unable to create key backup")).toBeVisible();
// cancel
await currentDialogLocator.getByTestId("dialog-cancel-button").click();
// go back to the settings to check that no backup was created (the setup button should still be there)
await app.settings.openUserSettings("Security & Privacy");
await expect(securityTab.getByRole("button", { name: "Set up", exact: true })).toBeVisible();
},
);
});