fix(security): harden CSRF with Content-Type gate and expand E2E coverage (#2818)
Publish to PyPI / build-and-publish (push) Waiting to run
Python Linting / Run Ruff (push) Waiting to run

Defense-in-depth over GET→POST alone: reject the three CORS-safelisted
simple-form Content-Types (x-www-form-urlencoded, multipart/form-data,
text/plain) on 16 no-body POST handlers (glob + legacy) to block
<form method=POST> CSRF that bypasses method-only gating. Move
comfyui_switch_version to a JSON body so the preflight requirement applies.
Split db_mode/policy/update/channel_url_list into GET(read) + POST(write).
Tighten do_fix (high → high+) and gate three previously-ungated config
setters at middle. Resynchronize openapi.yaml (27 paths, 30 operations,
ComfyUISwitchVersionParams as a shared $ref component). Add E2E harness
variants, Playwright config, CSRF/secgate suites, 39-endpoint coverage,
and a CHANGELOG.

Breaking: legacy per-op POST routes (install/uninstall/fix/disable/update/
reinstall/abort_current) are removed; callers already use queue/batch.
Legacy /manager/notice (v1) is removed; /v2/manager/notice is retained.

Reported-by: XlabAI Team of Tencent Xuanwu Lab
CVSS: 8.1 (AV:N/AC:L/PR:N/UI:R/S:U/C:N/I:H/A:H)
This commit is contained in:
Dr.Lt.Data
2026-04-22 05:04:30 +09:00
committed by GitHub
parent 49e205acd4
commit 4410ebc6a6
70 changed files with 13638 additions and 434 deletions
+44
View File
@@ -0,0 +1,44 @@
# Playwright E2E Tests — Legacy Manager UI
Browser-based E2E tests for the ComfyUI-Manager legacy UI.
## Prerequisites
1. **E2E environment** built via `python tests/e2e/scripts/setup_e2e_env.py`
2. **Playwright installed**: `npx playwright install chromium`
3. **ComfyUI running** with legacy UI enabled:
```bash
E2E_ROOT=/tmp/e2e_full_test
PORT=8199
$E2E_ROOT/venv/bin/python $E2E_ROOT/comfyui/main.py \
--listen 127.0.0.1 --port $PORT \
--enable-manager-legacy-ui \
--cpu
```
## Running Tests
```bash
# With server already running:
PORT=8199 npx playwright test
# Single file:
PORT=8199 npx playwright test tests/playwright/legacy-ui-manager-menu.spec.ts
# Headed (visible browser):
PORT=8199 npx playwright test --headed
# Debug mode:
PORT=8199 npx playwright test --debug
```
## Test Files
| File | Scenarios |
|------|-----------|
| `legacy-ui-manager-menu.spec.ts` | Menu dialog rendering, settings dropdowns, API round-trip |
| `legacy-ui-custom-nodes.spec.ts` | Node list grid, filter, search, footer buttons |
| `legacy-ui-model-manager.spec.ts` | Model list grid, filter, search |
| `legacy-ui-snapshot.spec.ts` | Snapshot list, save, remove |
| `legacy-ui-navigation.spec.ts` | Dialog open/close, nested navigation, no duplicates |
+202
View File
@@ -0,0 +1,202 @@
# Legacy UI Playwright E2E — Test Scenarios
Scenario list based on the actual API call flow of the Legacy UI (runtime-verified).
## URL Convention
ComfyUI's `api.fetchApi()` automatically prepends the `/api` prefix to every path.
- JS call: `fetchApi('/v2/manager/db_mode')` → actual request: `GET /api/v2/manager/db_mode`
- When intercepted by Playwright `route`, the URL is captured in the `/api/v2/...` form.
## Install Flow (Runtime-verified)
```
[Install button click]
→ GET /api/v2/customnode/versions/{id} ← list available versions
→ Version-selection dialog (<select multiple> + "Select"/"Cancel")
→ "Select" click
→ POST /api/v2/manager/queue/batch ← actual install request
body: {"install":[{...nodeData, selected_version:"latest"}], "batch_id":"uuid"}
→ WebSocket push: cm-queue-status
{status:"in_progress", done_count, total_count}
{status:"batch-done", nodepack_result:{"hash":"success"}}
{status:"all-done"}
```
---
## 1. API calls at Manager Menu initialization
**File**: `legacy-ui-manager-menu.spec.ts` (existing + enhanced)
5 API calls made concurrently when the Manager Menu opens (runtime-verified):
| # | Scenario | Assertion | Endpoint |
|---|---------|------|----------|
| 1-1 | DB mode loading | Dropdown value shown | `GET /api/v2/manager/db_mode` |
| 1-2 | Channel list loading | Dropdown options shown | `GET /api/v2/manager/channel_url_list` |
| 1-3 | Update policy loading | Dropdown value shown | `GET /api/v2/manager/policy/update` |
| 1-4 | Notice loading | Right panel text is non-empty | `GET /api/v2/manager/notice` |
| 1-5 | DB mode change round-trip | POST → GET verification | `POST /api/v2/manager/db_mode` |
| 1-6 | Policy change round-trip | POST → GET verification | `POST /api/v2/manager/policy/update` |
| 1-7 | Channel change round-trip | POST → GET verification | `POST /api/v2/manager/channel_url_list` |
## 2. Custom Nodes Manager — list retrieval
**File**: `legacy-ui-custom-nodes.spec.ts` (existing + enhanced)
2 API calls made when the Custom Nodes Manager opens (runtime-verified):
| # | Scenario | UI action | Assertion | Endpoint |
|---|---------|---------|------|----------|
| 2-1 | List loading (cache) | Open Custom Nodes Manager | Grid rows > 0 | `GET /api/v2/customnode/getlist?mode=cache&skip_update=true` |
| 2-2 | Mapping loading | (concurrent with list) | Request observed | `GET /api/v2/customnode/getmappings?mode=cache` |
| 2-3 | Installed filter | Filter → "Installed" | rows ≤ All | Client-side filter |
| 2-4 | Not Installed filter | Filter → "Not Installed" | rows > 0 | Client-side filter |
| 2-5 | Import Failed filter | Filter → "Import Failed" | Filter works | Client-side filter |
| 2-6 | Check Update | "Check Update" button | Filter flips to "Update", API re-called | `GET /api/v2/customnode/getlist?mode=cache` (no `skip_update`) |
| 2-7 | Check Missing | "Check Missing" button | Filter flips to "Missing" | `GET /api/v2/customnode/getmappings?mode=cache` |
| 2-8 | Alternatives filter | Filter → "Alternatives of A1111" | Data loads | `GET /api/customnode/alternatives?mode=cache` |
## 3. Full node install lifecycle
**File**: `legacy-ui-node-lifecycle.spec.ts` (new)
Install flow: Install button → versions API → version-selection dialog → Select → queue/batch → WebSocket status push
| # | Scenario | UI action | Assertion | Endpoint |
|---|---------|---------|------|----------|
| 3-1 | Install — version query | "Not Installed" → "Install" click | Version-list dialog appears | `GET /api/v2/customnode/versions/{id}` |
| 3-2 | Install — select version + send batch | Pick version in `<select>` → "Select" click | `queue/batch` called with `install` key in body | `POST /api/v2/manager/queue/batch` |
| 3-3 | Install — WebSocket status | Install runs | `cm-queue-status` messages: in_progress → batch-done → all-done | WebSocket |
| 3-4 | Uninstall | "Installed" → "Uninstall" click → confirm dialog "OK" | `uninstall` key in batch body | `POST /api/v2/manager/queue/batch` |
| 3-5 | Disable | "Installed" → "Disable" click | `disable` key in batch body | `POST /api/v2/manager/queue/batch` |
| 3-6 | Enable | "Disabled" → "Enable" click → pick version | `install` key + `skip_post_install:true` in batch body | `GET disabled_versions/{id}``POST queue/batch` |
| 3-7 | Update | "Installed" → "Try update" click | `update` key in batch body | `POST /api/v2/manager/queue/batch` |
| 3-8 | Fix | import-fail node → "Try fix" click | `fix` key in batch body (skip if none) | `POST /api/v2/manager/queue/batch` |
## 4. Version management
**File**: `legacy-ui-node-versions.spec.ts` (new)
| # | Scenario | UI action | Assertion | Endpoint |
|---|---------|---------|------|----------|
| 4-1 | Installed node version list | "Installed" → "Switch Ver" click | Version list shown in `<select multiple>` dialog | `GET /api/v2/customnode/versions/{id}` |
| 4-2 | Disabled node version list | "Disabled" → "Enable" click | `disabled_versions` call observed | `GET /api/v2/customnode/disabled_versions/{id}` |
## 5. Batch operations + stop
**File**: `legacy-ui-batch-operations.spec.ts` (new)
| # | Scenario | UI action | Assertion | Endpoint |
|---|---------|---------|------|----------|
| 5-1 | Update All | Manager Menu → "Update All" click | `update_all` key in batch body | `POST /api/v2/manager/queue/batch` |
| 5-2 | Update ComfyUI | Manager Menu → "Update ComfyUI" click | `update_comfyui` key in batch body | `POST /api/v2/manager/queue/batch` |
| 5-3 | Stop (Manager Menu) | "Restart" toggle → "Stop" click | `queue/reset` invoked | `POST /api/v2/manager/queue/reset` |
| 5-4 | Stop (Custom Nodes Manager) | "Stop" button click | `queue/reset` invoked | `POST /api/v2/manager/queue/reset` |
Note: `queue/abort_current` is not called directly from JS (server-only). Stop uses `queue/reset`.
## 6. Git URL / PIP install
**File**: `legacy-ui-install-methods.spec.ts` (new)
| # | Scenario | UI action | Assertion | Endpoint |
|---|---------|---------|------|----------|
| 6-1 | Git URL install | "Install via Git URL" → enter URL → confirm | 200 or 403 | `POST /api/v2/customnode/install/git_url` |
| 6-2 | Git URL cancel | "Install via Git URL" → cancel | No API call | — |
| 6-3 | PIP package install | "Install PIP packages" → enter package name | 200 or 403 | `POST /api/v2/customnode/install/pip` |
## 7. Import failure details
**File**: `legacy-ui-custom-nodes.spec.ts` (enhanced)
| # | Scenario | UI action | Assertion | Endpoint |
|---|---------|---------|------|----------|
| 7-1 | Import failure details | "Import Failed" filter → "IMPORT FAILED ↗" click | Error dialog appears | `POST /api/v2/customnode/import_fail_info` |
Note: skipped when no import-failed nodes are present.
## 8. Model management
**File**: `legacy-ui-model-manager.spec.ts` (existing + enhanced)
| # | Scenario | UI action | Assertion | Endpoint |
|---|---------|---------|------|----------|
| 8-1 | Model list loading | Open Model Manager | Grid rows > 0 | `GET /api/v2/externalmodel/getlist?mode=cache` |
| 8-2 | Model search | Enter query | Grid filtered | Client-side filter |
| 8-3 | Model install | Row's "Install" click | `install_model` key in batch body | `POST /api/v2/manager/queue/batch` |
## 9. Full snapshot lifecycle
**File**: `legacy-ui-snapshot.spec.ts` (existing + enhanced)
| # | Scenario | UI action | Assertion | Endpoint |
|---|---------|---------|------|----------|
| 9-1 | Snapshot list | Open Snapshot Manager | Table rows shown | `GET /api/v2/snapshot/getlist` |
| 9-2 | Snapshot save | "Save snapshot" click | "Current snapshot saved" message | `POST /api/v2/snapshot/save` |
| 9-3 | Snapshot restore | "Restore" click | 200 or 403, RESTART button shown | `POST /api/v2/snapshot/restore?target=X` |
| 9-4 | Snapshot remove | "Remove" click | Row removed from list | `POST /api/v2/snapshot/remove?target=X` |
## 10. Dialog navigation
**File**: `legacy-ui-navigation.spec.ts` (existing)
| # | Scenario | Assertion |
|---|---------|------|
| 10-1 | Manager → Custom Nodes → close → re-open Manager | Dialog transitions cleanly |
| 10-2 | Manager → Model Manager → close → re-open | Dialog transitions cleanly |
| 10-3 | API call while dialog is open | Server responds normally |
| 10-4 | Legacy UI enabled check | `is_legacy_manager_ui: true` |
---
## File composition summary
| File | New/Enhanced | Scenarios | Target Endpoints |
|------|----------|:-------:|---------------|
| `legacy-ui-manager-menu.spec.ts` | enhanced | 7 | db_mode, channel_url_list, policy/update, notice |
| `legacy-ui-custom-nodes.spec.ts` | enhanced | 9 | getlist, getmappings, alternatives, import_fail_info |
| `legacy-ui-node-lifecycle.spec.ts` | **new** | 8 | versions/{id}, disabled_versions/{id}, queue/batch (install/uninstall/update/fix/disable/enable) |
| `legacy-ui-node-versions.spec.ts` | **new** | 2 | versions/{id}, disabled_versions/{id} |
| `legacy-ui-batch-operations.spec.ts` | **new** | 4 | queue/batch (update_all, update_comfyui), queue/reset |
| `legacy-ui-install-methods.spec.ts` | **new** | 3 | install/git_url, install/pip |
| `legacy-ui-model-manager.spec.ts` | enhanced | 3 | externalmodel/getlist, queue/batch (install_model) |
| `legacy-ui-snapshot.spec.ts` | enhanced | 4 | snapshot/getlist, save, restore, remove |
| `legacy-ui-navigation.spec.ts` | existing | 4 | is_legacy_manager_ui, version |
**Total: 44 scenarios**
## Legacy-only endpoint coverage
| Endpoint | Scenarios |
|----------|---------|
| `GET /api/v2/customnode/getlist` | 2-1, 2-6 |
| `GET /api/v2/customnode/getmappings` | 2-2, 2-7 |
| `GET /api/customnode/alternatives` | 2-8 |
| `GET /api/v2/customnode/versions/{id}` | 3-1, 4-1 |
| `GET /api/v2/customnode/disabled_versions/{id}` | 3-6, 4-2 |
| `POST /api/v2/customnode/import_fail_info` | 7-1 |
| `POST /api/v2/customnode/install/git_url` | 6-1 |
| `POST /api/v2/customnode/install/pip` | 6-3 |
| `GET /api/v2/externalmodel/getlist` | 8-1 |
| `POST /api/v2/manager/queue/batch` | 3-2, 3-4~3-8, 5-1, 5-2, 8-3 |
| `POST /api/v2/manager/queue/reset` | 5-3, 5-4 |
| `GET /api/v2/manager/notice` | 1-4 |
| `GET /api/v2/snapshot/getlist` | 9-1 |
| `POST /api/v2/snapshot/save` | 9-2 |
| `POST /api/v2/snapshot/restore` | 9-3 |
| `POST /api/v2/snapshot/remove` | 9-4 |
## Exclusions
- `share_option` — per user instruction
- **External-service auth/integration** (9 endpoints) — external services are unreachable from E2E
- **Individual queue endpoints** (6) — unused by JS; delegated internally through `queue/batch`
- `queue/abort_current` — unused by JS (Stop uses `queue/reset`)
- `/manager/notice` (v1) — superseded by v2
## API call verification method
Capture the actual API call sequence via Playwright `page.route('**/*')` interception.
Verify job progress/completion via the `cm-queue-status` WebSocket event.
+128
View File
@@ -0,0 +1,128 @@
/**
* Shared helpers for ComfyUI Manager Playwright E2E tests.
*
* The legacy UI is dialog-based: a "Manager" menu button on the ComfyUI
* top-bar opens ManagerMenuDialog, from which sub-dialogs (CustomNodes,
* Model, Snapshot) are launched.
*/
import { type Page, expect } from '@playwright/test';
/** Wait for the ComfyUI page to be fully loaded (queue ready). */
export async function waitForComfyUI(page: Page) {
// ComfyUI shows the canvas once the app is ready. Wait for the
// system_stats endpoint to respond — same check the Python E2E uses.
await page.waitForFunction(
async () => {
try {
const r = await fetch('/system_stats');
return r.ok;
} catch {
return false;
}
},
{ timeout: 30_000, polling: 1_000 },
);
// Give the extensions a moment to register their menu items.
await page.waitForTimeout(3_000);
// Close any overlay that might be covering the toolbar.
// Press Escape to dismiss popups/modals/sidebars.
await page.keyboard.press('Escape');
await page.waitForTimeout(1_000);
await page.keyboard.press('Escape');
await page.waitForTimeout(500);
}
/** Open the Manager Menu dialog via the top-bar button. */
export async function openManagerMenu(page: Page) {
// The legacy UI registers a "Manager" button via ComfyButton (new style)
// or a plain <button> (old style). The new-style button uses the
// "puzzle" icon and has tooltip "ComfyUI Manager" / content "Manager".
//
// ComfyButton renders as a structure like:
// <button class="comfyui-button" title="ComfyUI Manager">
// <span class="icon">...</span>
// <span>Manager</span>
// </button>
//
// We try multiple selectors to handle both old and new ComfyUI layouts.
const selectors = [
'button[title="ComfyUI Manager"]', // new-style ComfyButton
'button.comfyui-button:has-text("Manager")', // new-style fallback
'button:has-text("Manager")', // old-style plain button
];
for (const sel of selectors) {
const btn = page.locator(sel).first();
if (await btn.isVisible({ timeout: 3_000 }).catch(() => false)) {
await btn.click();
await page.waitForSelector('#cm-manager-dialog, .comfy-modal', { timeout: 10_000 });
return;
}
}
// Last resort: find any button with "Manager" in tooltip or text via DOM
const found = await page.evaluate(() => {
const buttons = document.querySelectorAll('button');
for (const btn of buttons) {
const text = btn.textContent?.toLowerCase() || '';
const title = btn.getAttribute('title')?.toLowerCase() || '';
if (text.includes('manager') || title.includes('manager')) {
(btn as HTMLElement).click();
return true;
}
}
return false;
});
if (found) {
// Wait for the dialog by polling for the element in DOM
await page.waitForFunction(
() => !!document.getElementById('cm-manager-dialog'),
{ timeout: 10_000, polling: 500 },
);
return;
}
await page.screenshot({ path: 'test-results/debug-manager-btn-not-found.png' });
throw new Error('Could not find Manager button in ComfyUI toolbar');
}
/** Click a button inside the Manager Menu dialog by its visible text. */
export async function clickMenuButton(page: Page, text: string) {
const dialog = page.locator('#cm-manager-dialog').first();
await dialog.locator(`button:has-text("${text}")`).click();
}
/** Close the topmost dialog via its X (close) button or Escape. */
export async function closeDialog(page: Page) {
// Try clicking close buttons on visible dialogs. The manager-menu dialog
// (`#cm-manager-dialog`) is a ComfyDialog with `.p-dialog-close-button` (X),
// while sub-dialogs use `.cm-close-btn`. Try both.
for (const sel of [
'#cn-manager-dialog button.cm-close-btn',
'#cmm-manager-dialog button.cm-close-btn',
'#snapshot-manager-dialog button.cm-close-btn',
'#cm-manager-dialog button.cm-close-btn',
'#cm-manager-dialog .p-dialog-close-button',
'.cm-close-btn',
'.p-dialog-close-button',
]) {
const btn = page.locator(sel).last();
if (await btn.isVisible({ timeout: 500 }).catch(() => false)) {
await btn.click();
await page.waitForTimeout(300);
return;
}
}
// Fallback: press Escape (ComfyDialog may not honor this reliably)
await page.keyboard.press('Escape');
await page.waitForTimeout(300);
}
/** Assert the Manager Menu dialog is visible and contains expected sections. */
export async function assertManagerMenuVisible(page: Page) {
const dialog = page.locator('#cm-manager-dialog').first();
await expect(dialog).toBeVisible();
}
@@ -0,0 +1,152 @@
/**
* E2E tests: Legacy Custom Nodes Manager dialog.
*
* Tests the TurboGrid-based custom node list, filters, search,
* and basic row interactions.
*
* Requires ComfyUI running with --enable-manager-legacy-ui on PORT.
*/
import { test, expect } from '@playwright/test';
import { waitForComfyUI, openManagerMenu, clickMenuButton } from './helpers';
test.describe('Custom Nodes Manager', () => {
test.beforeEach(async ({ page }) => {
await page.goto('/');
await waitForComfyUI(page);
await openManagerMenu(page);
});
test('opens from Manager menu and renders grid', async ({ page }) => {
await clickMenuButton(page, 'Custom Nodes Manager');
// Wait for the custom nodes dialog to appear
await page.waitForSelector('#cn-manager-dialog', {
timeout: 10_000,
});
// The grid should be present
const grid = page.locator('.cn-manager-grid, .tg-body').first();
await expect(grid).toBeVisible({ timeout: 15_000 });
});
test('loads custom node list (non-empty)', async ({ page }) => {
await clickMenuButton(page, 'Custom Nodes Manager');
await page.waitForSelector('.cn-manager-grid, .tg-body', { timeout: 15_000 });
// Wait for data to load — grid rows should appear
await page.waitForFunction(
() => {
const rows = document.querySelectorAll('.tg-body .tg-row, .cn-manager-grid tr');
return rows.length > 0;
},
{ timeout: 30_000, polling: 1_000 },
);
const rows = page.locator('.tg-body .tg-row, .cn-manager-grid tr');
const count = await rows.count();
expect(count).toBeGreaterThan(0);
});
test('filter dropdown changes displayed nodes', async ({ page }) => {
await clickMenuButton(page, 'Custom Nodes Manager');
await page.waitForSelector('.cn-manager-grid, .tg-body', { timeout: 15_000 });
// Wait for initial data load
await page.waitForFunction(
() => document.querySelectorAll('.tg-body .tg-row, .cn-manager-grid tr').length > 0,
{ timeout: 30_000, polling: 1_000 },
);
// Find the filter select (class: cn-manager-filter) and switch to "Installed"
const filterSelect = page.locator('select.cn-manager-filter').first();
// Hard-fail if filter UI missing — that's a regression, not a skip condition
await expect(filterSelect).toBeVisible({ timeout: 5_000 });
const initialCount = await page.locator('.tg-body .tg-row').count();
await filterSelect.selectOption({ label: 'Installed' });
// Wait for row count to actually CHANGE (state-based, not wall-clock).
// If filter is broken and returns everything, this will fail within 10s.
await expect
.poll(async () => page.locator('.tg-body .tg-row').count(), { timeout: 10_000 })
.not.toBe(initialCount);
// Installed count should be <= total
const filteredCount = await page.locator('.tg-body .tg-row').count();
expect(filteredCount).toBeLessThanOrEqual(initialCount);
});
test('search input filters the grid', async ({ page }) => {
await clickMenuButton(page, 'Custom Nodes Manager');
await page.waitForSelector('.cn-manager-grid, .tg-body', { timeout: 15_000 });
await page.waitForFunction(
() => document.querySelectorAll('.tg-body .tg-row, .cn-manager-grid tr').length > 0,
{ timeout: 30_000, polling: 1_000 },
);
// Find search input
const searchInput = page.locator('.cn-manager-keywords, input[type="text"][placeholder*="earch"], input[type="search"]').first();
await expect(searchInput).toBeVisible({ timeout: 5_000 });
const initialCount = await page.locator('.tg-body .tg-row, .cn-manager-grid tr').count();
await searchInput.fill('ComfyUI-Manager');
// State-based wait: count must actually narrow (or become 0)
await expect
.poll(
async () => page.locator('.tg-body .tg-row, .cn-manager-grid tr').count(),
{ timeout: 10_000 },
)
.toBeLessThan(initialCount);
const filteredCount = await page.locator('.tg-body .tg-row, .cn-manager-grid tr').count();
expect(filteredCount).toBeLessThanOrEqual(initialCount);
});
test('footer buttons are present', async ({ page }) => {
// Wave3 WI-U Cluster H target 4: strengthen from OR-of-2 to AND-of-all-
// always-visible-admin-buttons. js/custom-nodes-manager.js:26-34 defines 6
// footer buttons, but `.cn-manager-restart` and `.cn-manager-stop` are
// `display: none` by default in custom-nodes-manager.css:47-62 (shown only
// via showRestart()/showStop() — conditional on restart-required /
// task-running state). In a clean Manager state, neither is visible.
//
// The 4 ALWAYS-visible footer admin buttons are:
// - "Install via Git URL" — primary install entrypoint
// - "Used In Workflow" — filter to workflow-referenced nodes
// - "Check Update" — refresh available-update list
// - "Check Missing" — scan for missing nodes
//
// We assert all 4 are visible (AND semantics). Hidden-by-default Restart/
// Stop are checked structurally — exist in DOM but may be hidden.
await clickMenuButton(page, 'Custom Nodes Manager');
await page.waitForSelector('#cn-manager-dialog', {
timeout: 15_000,
});
const dialog = page.locator('#cn-manager-dialog').last();
// AND semantics: every always-visible footer button MUST be visible.
const alwaysVisibleButtons = [
'Install via Git URL',
'Used In Workflow',
'Check Update',
'Check Missing',
];
for (const label of alwaysVisibleButtons) {
await expect(
dialog.locator(`button:has-text("${label}")`).first(),
`always-visible footer button "${label}" must be present and visible`,
).toBeVisible();
}
// Structural presence for conditional buttons — they exist in the DOM but
// are hidden until showRestart()/showStop() toggles `display: block`.
for (const cls of ['.cn-manager-restart', '.cn-manager-stop']) {
await expect(
dialog.locator(cls),
`conditional footer button ${cls} must be present in DOM (may be hidden)`,
).toHaveCount(1);
}
});
});
+294
View File
@@ -0,0 +1,294 @@
/**
* E2E tests: UI-driven install/uninstall effect verification.
*
* Contract: LEGACY UI tests must drive the action via UI elements (no direct API calls).
* Effect is observed through backend state (queue/status, installed list) and/or UI badges.
*
* Requires ComfyUI running with --enable-manager-legacy-ui on PORT.
* Test pack: ComfyUI_SigmoidOffsetScheduler (ltdrdata's test pack).
*/
import { test, expect } from '@playwright/test';
import { waitForComfyUI, openManagerMenu, clickMenuButton } from './helpers';
const PACK_CNR_ID = 'comfyui_sigmoidoffsetscheduler';
async function waitForAllDone(page: import('@playwright/test').Page, timeoutMs = 90_000): Promise<void> {
// Three-phase polling with DETERMINISTIC baseline:
// Phase 0 — snapshot baseline. To make the baseline deterministic across
// runs (and immune to leaking history from prior tests in the
// session), we FETCH the baseline immediately after the caller
// has triggered the UI action. The caller is expected to have
// called /v2/manager/queue/reset at the start of its test flow
// so that done_count starts at 0 for this test's session.
// Phase 1 — wait for task acceptance:
// total_count > 0 OR is_processing=true OR done_count > baseline
// Phase 2 — wait for drain (total_count === 0 && is_processing=false)
const deadline = Date.now() + timeoutMs;
// Phase 0: baseline. If fetch fails, treat as 0 but log so the test signal
// isn't silently degraded.
let baselineDone = 0;
const baselineResp = await page.request
.get('/v2/manager/queue/status')
.catch(() => null);
if (baselineResp && baselineResp.ok()) {
const baseline = await baselineResp.json();
baselineDone = baseline?.done_count ?? 0;
} else {
console.warn('[waitForAllDone] baseline fetch failed — treating as 0');
}
// Phase 1: task acceptance
const acceptDeadline = Math.min(Date.now() + 15_000, deadline);
let accepted = false;
while (Date.now() < acceptDeadline) {
const status = await page.request
.get('/v2/manager/queue/status')
.then((r) => r.json())
.catch(() => null);
if (
status &&
((status.total_count ?? 0) > 0 ||
status.is_processing === true ||
(status.done_count ?? 0) > baselineDone)
) {
accepted = true;
break;
}
await page.waitForTimeout(500);
}
if (!accepted) {
throw new Error('Queue never accepted the task (empty queue for 15s after UI action)');
}
// Phase 2: drain
while (Date.now() < deadline) {
const status = await page.request
.get('/v2/manager/queue/status')
.then((r) => r.json())
.catch(() => null);
if (status && status.is_processing === false && (status.total_count ?? 0) === 0) {
return;
}
await page.waitForTimeout(1_500);
}
throw new Error(`Queue did not drain within ${timeoutMs}ms`);
}
async function isPackInstalled(page: import('@playwright/test').Page): Promise<boolean> {
const resp = await page.request.get('/v2/customnode/installed');
if (!resp.ok()) return false;
const data = await resp.json();
for (const pkg of Object.values<unknown>(data)) {
if (
pkg &&
typeof pkg === 'object' &&
(pkg as { cnr_id?: string }).cnr_id?.toLowerCase() === PACK_CNR_ID
) {
return true;
}
}
return false;
}
test.describe('UI-driven install/uninstall', () => {
test.beforeEach(async ({ page }) => {
await page.goto('/');
await waitForComfyUI(page);
});
test('LB1 Install button triggers install effect', async ({ page }) => {
// Reset queue at start for deterministic done_count baseline in waitForAllDone
await page.request.post('/v2/manager/queue/reset');
// Precondition: pack must NOT be installed. If the seed pack is already
// installed (prior pytest runs, pre-seeded E2E environment), the
// "Not Installed" filter applied below would correctly exclude its row and
// `packRow.toBeVisible` would fail with "element(s) not found". Uninstall
// via API as test SETUP (not verification) — mirrors LB2's inverse pattern
// that API-installs if the pack is absent. queue/batch is used here (not
// queue/task) because queue/batch is the legacy manager_server endpoint
// for task enqueueing; queue/task is glob-only — under
// --enable-manager-legacy-ui (which this spec requires) POST /queue/task
// falls through to aiohttp's GET-only static catch-all and returns 405.
if (await isPackInstalled(page)) {
const queueResp = await page.request.post('/v2/manager/queue/batch', {
data: JSON.stringify({
batch_id: 'lb1-setup-uninstall',
uninstall: [{
id: 'ComfyUI_SigmoidOffsetScheduler',
ui_id: 'lb1-setup-uninstall',
version: '1.0.1',
selected_version: 'latest',
mode: 'local',
channel: 'default',
}],
}),
headers: { 'Content-Type': 'application/json' },
});
expect(queueResp.ok()).toBe(true);
await page.request.post('/v2/manager/queue/start');
await waitForAllDone(page, 60_000);
// Hard fail if setup itself couldn't uninstall the pack
expect(await isPackInstalled(page)).toBe(false);
}
// UI flow: open Manager → Custom Nodes Manager
await openManagerMenu(page);
await clickMenuButton(page, 'Custom Nodes Manager');
await page.waitForSelector('#cn-manager-dialog', { timeout: 15_000 });
// Wait for grid to populate before applying filter (avoids race on empty grid)
await expect(page.locator('.tg-body .tg-row').first()).toBeVisible({ timeout: 30_000 });
const initialRowCount = await page.locator('.tg-body .tg-row').count();
// Filter to Not Installed to make install buttons visible. Wait for
// filtered row count to actually change (DOM state, not wall-clock).
const filterSelect = page.locator('select.cn-manager-filter').first();
if (await filterSelect.isVisible().catch(() => false)) {
await filterSelect.selectOption({ value: 'not-installed' });
await expect
.poll(async () => page.locator('.tg-body .tg-row').count(), { timeout: 10_000 })
.not.toBe(initialRowCount);
}
// Search for the specific test pack. Wait for search to narrow results.
// Search matches title/author/description per custom-nodes-manager.js:605
// (NOT id). The pack's title is "ComfyUI Sigmoid Offset Scheduler" (with
// spaces), so "SigmoidOffsetScheduler" (no spaces) would miss — use
// "Sigmoid Offset Scheduler" to match the title substring.
const searchInput = page
.locator('.cn-manager-keywords, input[type="search"], input[type="text"][placeholder*="earch"]')
.first();
if (await searchInput.isVisible().catch(() => false)) {
await searchInput.fill('Sigmoid Offset Scheduler');
// Wait for search to settle — row count stabilizes
await expect
.poll(async () => page.locator('.tg-body .tg-row').count(), { timeout: 10_000 })
.toBeLessThanOrEqual(5);
}
// Scope button to the row containing the pack name (not arbitrary first row).
// Row DOM renders the title column, which reads "ComfyUI Sigmoid Offset
// Scheduler" — match the substring that appears there, not the id.
// TurboGrid splits each logical row into TWO DOM .tg-row elements (left
// frozen-column pane with the title + right scrollable-column pane with
// Version/Action/etc.). The Install button lives in the right pane, so
// filtering by title-text picks the left pane which has no Install button.
// Use `.tg-body` scope + `button[mode="install"]` directly, then assert
// only one such button exists (single search result narrows to 1 row).
const packRow = page.locator('.tg-body .tg-row', { hasText: 'Sigmoid Offset Scheduler' }).first();
await expect(packRow).toBeVisible({ timeout: 10_000 });
const installBtn = page.locator('.tg-body button[mode="install"]').first();
// Hard fail if the Install button isn't visible in the filtered result
await expect(installBtn).toBeVisible({ timeout: 5_000 });
await installBtn.click();
// Version selector dialog appears
const selectBtn = page.locator('.comfy-modal button:has-text("Select")').first();
await selectBtn.waitFor({ timeout: 10_000 });
await selectBtn.click();
// Effect verification: wait for queue to drain then check installed state
await waitForAllDone(page, 120_000);
const installed = await isPackInstalled(page);
expect(installed).toBe(true);
});
test('LB2 Uninstall button triggers uninstall effect', async ({ page }) => {
// Reset queue at start for deterministic done_count baseline in waitForAllDone
await page.request.post('/v2/manager/queue/reset');
// Precondition: pack must be installed. Install via API as test SETUP
// (not verification). This makes LB2 independent of LB1 — hard-failing
// on a UI bug rather than skipping on a missing precondition. queue/batch
// is the legacy manager_server endpoint (see LB1 comment above); install
// is async, so waitForAllDone is still required after queue/start.
const preinstalled = await isPackInstalled(page);
if (!preinstalled) {
await page.request.post('/v2/manager/queue/reset');
const queueResp = await page.request.post('/v2/manager/queue/batch', {
data: JSON.stringify({
batch_id: 'lb2-setup-install',
install: [{
id: 'ComfyUI_SigmoidOffsetScheduler',
ui_id: 'lb2-setup-install',
version: '1.0.1',
selected_version: 'latest',
mode: 'remote',
channel: 'default',
}],
}),
headers: { 'Content-Type': 'application/json' },
});
expect(queueResp.ok()).toBe(true);
const queueBody = await queueResp.json();
expect(queueBody.failed ?? []).toEqual([]);
await page.request.post('/v2/manager/queue/start');
// Poll the terminal state directly: isPackInstalled returning true is
// the unambiguous success signal. Using waitForAllDone here is racy —
// fast-path installs (pack already on disk / cached CNR artifacts) can
// complete before waitForAllDone's Phase 0 baseline fetch runs, leaving
// Phase 1 unable to distinguish "already done" from "never queued".
// Polling isPackInstalled avoids that ambiguity entirely.
await expect.poll(() => isPackInstalled(page), { timeout: 120_000 }).toBe(true);
}
await openManagerMenu(page);
await clickMenuButton(page, 'Custom Nodes Manager');
await page.waitForSelector('#cn-manager-dialog', { timeout: 15_000 });
await expect(page.locator('.tg-body .tg-row').first()).toBeVisible({ timeout: 30_000 });
const initialRowCount = await page.locator('.tg-body .tg-row').count();
// Filter to Installed to make Uninstall buttons visible
const filterSelect = page.locator('select.cn-manager-filter').first();
if (await filterSelect.isVisible().catch(() => false)) {
await filterSelect.selectOption({ label: 'Installed' });
await expect
.poll(async () => page.locator('.tg-body .tg-row').count(), { timeout: 10_000 })
.not.toBe(initialRowCount);
}
// Search matches title/author/description per custom-nodes-manager.js:605
// (NOT id). Pack title is "ComfyUI Sigmoid Offset Scheduler" (spaces) —
// use the space-separated form to match (WI-CC pattern).
const searchInput = page
.locator('.cn-manager-keywords, input[type="search"], input[type="text"][placeholder*="earch"]')
.first();
if (await searchInput.isVisible().catch(() => false)) {
await searchInput.fill('Sigmoid Offset Scheduler');
await expect
.poll(async () => page.locator('.tg-body .tg-row').count(), { timeout: 10_000 })
.toBeLessThanOrEqual(5);
}
// Scope packRow visibility to the specific pack title, but the Uninstall
// button lives in the right-pane .tg-row (TurboGrid dual-pane rendering),
// which is NOT a child of the title-bearing left-pane row. Scope the
// button lookup to the grid body + search-narrowed result set (WI-CC pattern).
const packRow = page.locator('.tg-body .tg-row', { hasText: 'Sigmoid Offset Scheduler' }).first();
await expect(packRow).toBeVisible({ timeout: 10_000 });
const uninstallBtn = page.locator('.tg-body button[mode="uninstall"]').first();
await expect(uninstallBtn).toBeVisible({ timeout: 5_000 });
await uninstallBtn.click();
// A confirmation dialog appears — custom-nodes-manager.js uses
// `customConfirm` (PrimeVue p-dialog), not `.comfy-modal`. The dialog
// is the last-opened one (on top of manager-menu + CustomNodes dialogs);
// its Confirm button accessible name has a leading space (icon + text),
// so match by visible text substring rather than exact name.
const confirmDialog = page.locator('dialog[open], [role="dialog"]').last();
const confirmBtn = confirmDialog.locator('button:has-text("Confirm"), button:has-text("Yes"), button:has-text("OK")').first();
if (await confirmBtn.isVisible({ timeout: 5_000 }).catch(() => false)) {
await confirmBtn.click();
}
// Poll isPackInstalled directly — the uninstall queue drains fast enough
// that waitForAllDone's Phase 0/1 baseline-vs-done race can miss
// acceptance. isPackInstalled==false is the unambiguous terminal signal.
await expect.poll(() => isPackInstalled(page), { timeout: 60_000 }).toBe(false);
});
});
@@ -0,0 +1,334 @@
/**
* E2E tests: Legacy Manager Menu Dialog.
*
* Verifies that the legacy UI manager menu opens correctly, renders
* all expected controls, and that settings dropdowns round-trip through
* the server API.
*
* Requires ComfyUI running with --enable-manager-legacy-ui on PORT.
*/
import { test, expect } from '@playwright/test';
import { waitForComfyUI, openManagerMenu, assertManagerMenuVisible, closeDialog } from './helpers';
test.describe('Manager Menu Dialog', () => {
test.beforeEach(async ({ page }) => {
await page.goto('/');
await waitForComfyUI(page);
});
test('opens via Manager button and shows 3-column layout', async ({ page }) => {
await openManagerMenu(page);
await assertManagerMenuVisible(page);
// The dialog should contain known buttons
const dialog = page.locator('#cm-manager-dialog').first();
await expect(dialog.locator('button:has-text("Custom Nodes Manager")')).toBeVisible();
await expect(dialog.locator('button:has-text("Model Manager")')).toBeVisible();
await expect(dialog.locator('button:has-text("Restart")')).toBeVisible();
});
test('shows DB mode and Update Policy dropdowns', async ({ page }) => {
// WI-OO Item 3 (bloat dev:ci-022 B8 title-mismatch): renamed from
// "shows settings dropdowns (DB, Channel, Policy)". The original title
// promised three dropdowns but the body only asserted DB + Policy; in
// this legacy-UI build the channel combo is populated via a separate
// code path and is not reliably surfaced as a <select> in `#cm-manager-dialog`
// at open time (the DB-mode combo's options overlap with channel names via
// the "Channel" entry, which is what the original filter regex accidentally
// caught). Renaming makes the test's actual contract match its name;
// channel-dropdown coverage belongs in a dedicated test once the combo's
// stable selector is established.
await openManagerMenu(page);
const dialog = page.locator('#cm-manager-dialog').first();
// DB mode combo — options include cache/local/channel/remote.
const dbCombo = dialog.locator('select').filter({ hasText: /Cache|Local|Channel/ }).first();
await expect(dbCombo).toBeVisible();
// Update policy combo — options include Stable/Nightly variants.
const policyCombo = dialog.locator('select').filter({ hasText: /Stable|Nightly/ }).first();
await expect(policyCombo).toBeVisible();
});
test('DB mode dropdown persists via UI (close-reopen verification)', async ({ page }) => {
// Wave3 WI-U Cluster H target 1: UI-only contract.
// No page.request / page.waitForResponse — pure UI interaction + dialog
// close-reopen as the persistence proof. networkidle is used only as a
// settle barrier (wait), never as assertion input. Close via the dialog's
// own `.p-dialog-close-button` (X button) because Escape doesn't close
// ComfyDialog reliably.
await openManagerMenu(page);
const dialog = page.locator('#cm-manager-dialog').first();
const dbCombo = dialog.locator('select').filter({ hasText: /Cache|Local|Channel/ }).first();
const original = await dbCombo.inputValue();
const newValue = original !== 'local' ? 'local' : 'cache';
try {
// Select via UI — the onchange handler fires the save. Wait for
// network quiescence so the save completes before we close.
await dbCombo.selectOption(newValue);
await page.waitForLoadState('networkidle');
// Close + reopen (UI-only persistence proof)
await dialog.locator('.p-dialog-close-button').first().click();
// ComfyDialog.close() sets display:none but keeps the element in DOM,
// so check visibility (toBeHidden), not presence (toHaveCount 0).
await expect(page.locator('#cm-manager-dialog').first()).toBeHidden({ timeout: 5_000 });
await openManagerMenu(page);
const reopenedDialog = page.locator('#cm-manager-dialog').first();
const reopenedCombo = reopenedDialog
.locator('select')
.filter({ hasText: /Cache|Local|Channel/ })
.first();
const persistedValue = await reopenedCombo.inputValue();
expect(persistedValue).toBe(newValue);
} finally {
// UI-only restore: reopen if needed + selectOption back to original.
// ComfyDialog keeps the element in DOM on close (display:none), so
// test visibility rather than presence.
const existing = page.locator('#cm-manager-dialog').first();
if ((await existing.count()) === 0 || !(await existing.isVisible().catch(() => false))) {
await openManagerMenu(page);
}
const cleanupDialog = page.locator('#cm-manager-dialog').first();
const cleanupCombo = cleanupDialog
.locator('select')
.filter({ hasText: /Cache|Local|Channel/ })
.first();
// selectOption is idempotent; if the value is already `original` this
// is a no-op. networkidle guarantees the save settles before
// subsequent tests run.
await cleanupCombo.selectOption(original);
await page.waitForLoadState('networkidle');
}
});
test('Update Policy dropdown persists via UI (close-reopen verification)', async ({ page }) => {
// Wave3 WI-U Cluster H target 2: same UI-only pattern as the DB mode test.
await openManagerMenu(page);
const dialog = page.locator('#cm-manager-dialog').first();
const policyCombo = dialog.locator('select').filter({ hasText: /Stable|Nightly/ }).first();
const original = await policyCombo.inputValue();
const newValue = original !== 'nightly-comfyui' ? 'nightly-comfyui' : 'stable-comfyui';
try {
await policyCombo.selectOption(newValue);
await page.waitForLoadState('networkidle');
await dialog.locator('.p-dialog-close-button').first().click();
// ComfyDialog.close() sets display:none but keeps the element in DOM,
// so check visibility (toBeHidden), not presence (toHaveCount 0).
await expect(page.locator('#cm-manager-dialog').first()).toBeHidden({ timeout: 5_000 });
await openManagerMenu(page);
const reopenedDialog = page.locator('#cm-manager-dialog').first();
const reopenedCombo = reopenedDialog
.locator('select')
.filter({ hasText: /Stable|Nightly/ })
.first();
const persistedValue = await reopenedCombo.inputValue();
expect(persistedValue).toBe(newValue);
} finally {
// UI-only restore
if ((await page.locator('#cm-manager-dialog').count()) === 0) {
await openManagerMenu(page);
}
const cleanupDialog = page.locator('#cm-manager-dialog').first();
const cleanupCombo = cleanupDialog
.locator('select')
.filter({ hasText: /Stable|Nightly/ })
.first();
await cleanupCombo.selectOption(original);
await page.waitForLoadState('networkidle');
}
});
test('closes and reopens without duplicating', async ({ page }) => {
await openManagerMenu(page);
await assertManagerMenuVisible(page);
await closeDialog(page);
// ComfyDialog.close() sets display:none but keeps the element in DOM —
// assert the (single) instance is now hidden instead of detached.
await expect(page.locator('#cm-manager-dialog').first()).toBeHidden({ timeout: 5_000 });
// Reopen
await openManagerMenu(page);
await assertManagerMenuVisible(page);
// Exactly one dialog instance expected. `=== 1` guards against real
// duplication bugs (ComfyDialog reuses the element, so a duplicate
// instance would be a real regression).
await expect(page.locator('#cm-manager-dialog')).toHaveCount(1);
});
// WI-VV coverage — close 4 LOW-risk Playwright P-gaps from
// reports/api-coverage-matrix.md. Each test exercises a UI trigger that
// the spec suite previously missed, without destructive action.
test('WI-VV wi-001: Switch ComfyUI button fetches comfyui_versions', async ({ page }) => {
// Clicking 'Switch ComfyUI' triggers GET /v2/comfyui_manager/comfyui_versions
// (comfyui-manager.js:612) and opens a secondary version-selector dialog.
// We assert the GET response populated with a non-empty version list
// and DO NOT select a version (selection would trigger the downstream
// POST /v2/comfyui_manager/comfyui_switch_version — out of scope for
// safe P-closure).
await openManagerMenu(page);
const dialog = page.locator('#cm-manager-dialog').first();
const switchBtn = dialog.locator('button:has-text("Switch ComfyUI")').first();
await expect(switchBtn).toBeVisible();
// Race the click with the response interception so we capture the GET
// that the click fires.
const [resp] = await Promise.all([
page.waitForResponse(
(r) =>
r.url().includes('/v2/comfyui_manager/comfyui_versions') &&
r.request().method() === 'GET',
{ timeout: 15_000 },
),
switchBtn.click(),
]);
expect(resp.status()).toBe(200);
const payload = await resp.json();
expect(payload).toHaveProperty('versions');
expect(Array.isArray(payload.versions)).toBe(true);
expect(payload.versions.length).toBeGreaterThan(0);
// Dismiss the secondary version-selector dialog without selecting by
// navigating away. Reloading the page collapses all ComfyDialogs and
// restores a clean slate for subsequent tests.
await page.goto('/');
await waitForComfyUI(page);
});
test('WI-VV wi-005: channel dropdown populates from channel_url_list GET', async ({ page }) => {
// Opening the manager menu triggers GET /v2/manager/channel_url_list
// (comfyui-manager.js:963) which async-populates the channel combo.
// Stable selector per reports/legacy-ui-channel-combo-dom-mapping.md:
// select[title^="Configure the channel"]
// Options are appended from `data.list` after the fetch resolves;
// `expect.poll` waits for population without racing the async fetch.
await openManagerMenu(page);
const dialog = page.locator('#cm-manager-dialog').first();
const channelCombo = dialog.locator(
'select[title^="Configure the channel"]',
);
await expect(channelCombo).toBeVisible();
await expect
.poll(
async () => (await channelCombo.locator('option').count()),
{ timeout: 10_000, message: 'channel combo should populate from GET /v2/manager/channel_url_list' },
)
.toBeGreaterThan(0);
// Current selection should be a non-empty string (the server-side
// `selected` field from the endpoint response).
const value = await channelCombo.inputValue();
expect(value).not.toBe('');
});
test('WI-VV wi-017: changing channel combo POSTs channel_url_list', async ({ page }) => {
// Changing the channel dropdown fires the onchange handler at
// comfyui-manager.js:975-977 which POSTs the new value to
// /v2/manager/channel_url_list. Teardown in finally restores the
// original selection to keep downstream tests clean.
await openManagerMenu(page);
const dialog = page.locator('#cm-manager-dialog').first();
const channelCombo = dialog.locator(
'select[title^="Configure the channel"]',
);
await expect(channelCombo).toBeVisible();
// Wait for options to populate before reading values.
await expect
.poll(async () => (await channelCombo.locator('option').count()), {
timeout: 10_000,
})
.toBeGreaterThan(0);
const original = await channelCombo.inputValue();
const optionValues = await channelCombo
.locator('option')
.evaluateAll((opts) => opts.map((o) => (o as HTMLOptionElement).value));
const alternative = optionValues.find((v) => v !== original && v !== '');
// If the server exposes only one channel, skip with reason — we
// cannot exercise the POST without a different selectable option.
if (!alternative) {
test.skip(
true,
`channel combo only offers one value (${original}); POST path unreachable in this env`,
);
}
try {
const [postResp] = await Promise.all([
page.waitForResponse(
(r) =>
r.url().includes('/v2/manager/channel_url_list') &&
r.request().method() === 'POST',
{ timeout: 10_000 },
),
channelCombo.selectOption(alternative!),
]);
expect(postResp.status()).toBe(200);
} finally {
// Restore — accept the POST but do not re-assert; a failure here
// should not mask the assertion failure above.
const restoreCombo = page
.locator('#cm-manager-dialog')
.first()
.locator('select[title^="Configure the channel"]');
if ((await restoreCombo.count()) > 0 && (await restoreCombo.inputValue()) !== original) {
await restoreCombo.selectOption(original).catch(() => undefined);
await page.waitForLoadState('networkidle').catch(() => undefined);
}
}
});
test('WI-VV wi-021: queue/reset POST succeeds at idle (API-level Playwright)', async ({ page, request }) => {
// UI-click path is NOT feasible at idle: comfyui-manager.js:795-802
// restart_stop_button reads "Restart" when no tasks are in progress and
// invokes rebootAPI() (server reboot) — clicking it at idle would
// kill the test server mid-run. The `.cn-manager-stop` /
// `.model-manager-stop` buttons that DO call `/v2/manager/queue/reset`
// (custom-nodes-manager.js:465, model-manager.js:173) are display:none
// at idle via CSS. Inducing in-progress state would require starting a
// real install — explicitly out-of-scope for this LOW-risk P-closure.
//
// Fallback: exercise the endpoint via page.request (Playwright's
// browser-context HTTP client). This verifies endpoint availability +
// idempotency at idle, which is the essential contract the UI-click
// would assert. The UI-wiring of the button is trivially visible from
// JS-source grep (3 callers, all with identical `fetchApi` POST).
await page.goto('/');
await waitForComfyUI(page);
// Pre-check: queue should be empty so reset is a true no-op.
const statusBefore = await request.get('/v2/manager/queue/status');
expect(statusBefore.status()).toBe(200);
const statusJson = await statusBefore.json();
const resetResp = await request.post('/v2/manager/queue/reset');
expect(resetResp.status()).toBe(200);
// Post-check: queue/status still callable (handler released locks
// cleanly) and the reset did not break queue introspection.
const statusAfter = await request.get('/v2/manager/queue/status');
expect(statusAfter.status()).toBe(200);
// Sanity: is_processing (or equivalent flag) should remain stable
// when reset was called on an empty queue — we don't strictly assert
// the flag here because the exact field name differs across Manager
// versions; the 200-on-status is the portable contract.
expect(await statusAfter.json()).toBeDefined();
void statusJson; // retained for debug, not asserted (pre/post shapes are impl-detail)
});
});
@@ -0,0 +1,135 @@
/**
* E2E tests: Legacy Model Manager dialog.
*
* Tests the model list grid, filters, and search.
*
* Requires ComfyUI running with --enable-manager-legacy-ui on PORT.
*/
import { test, expect } from '@playwright/test';
import { waitForComfyUI, openManagerMenu, clickMenuButton } from './helpers';
test.describe('Model Manager', () => {
test.beforeEach(async ({ page }) => {
await page.goto('/');
await waitForComfyUI(page);
await openManagerMenu(page);
});
test('opens from Manager menu and renders grid', async ({ page }) => {
await clickMenuButton(page, 'Model Manager');
await page.waitForSelector('#cmm-manager-dialog', {
timeout: 10_000,
});
const grid = page.locator('.cmm-manager-grid, .tg-body').first();
await expect(grid).toBeVisible({ timeout: 15_000 });
});
test('loads model list (non-empty)', async ({ page }) => {
// Wave3 WI-U Cluster H target 3 (LM1): previously rows>0 only. Now also
// verifies the install-state column is rendered for every logical model row.
//
// TurboGrid renders each logical row as TWO DOM `.tg-row` elements (left
// frozen-column pane + right scrollable-column pane). Only the right pane
// carries the "installed" column, which `model-manager.js:342-345` formats
// as EITHER `<div class="cmm-icon-passed">...</div>` (installed===True) OR
// `<button class="cmm-btn-install">Install</button>` (installed===False),
// with a `"Refresh Required"` fallback at :340.
//
// Invariant: the number of install-state indicators equals the number of
// logical rows, i.e. half the DOM-row count. This catches a regression
// where the installed column stops rendering for any model (partial or
// complete).
await clickMenuButton(page, 'Model Manager');
await page.waitForSelector('.cmm-manager-grid, .tg-body', { timeout: 15_000 });
await page.waitForFunction(
() => document.querySelectorAll('.tg-body .tg-row, .cmm-manager-grid tr').length > 0,
{ timeout: 30_000, polling: 1_000 },
);
const rows = page.locator('.tg-body .tg-row, .cmm-manager-grid tr');
const domRowCount = await rows.count();
expect(domRowCount).toBeGreaterThan(0);
// Count install indicators across the whole grid.
const installedCount = await page
.locator('.cmm-icon-passed, .cmm-btn-install')
.count();
const refreshCount = await page
.locator('.tg-body :text("Refresh Required"), .cmm-manager-grid :text("Refresh Required")')
.count();
const totalIndicators = installedCount + refreshCount;
// Each logical model row must expose an install-state indicator.
expect(totalIndicators, 'at least one row must have a valid install-state indicator').toBeGreaterThan(0);
// Expected indicator count: one per logical row. TurboGrid doubles DOM
// rows for the 2-pane layout, so logical_count = domRowCount / 2 when
// dual-pane. For single-pane (fallback) the ratio is 1:1. Accept either.
const logicalRowCount = domRowCount / 2;
const isDualPane = Number.isInteger(logicalRowCount) && totalIndicators === logicalRowCount;
const isSinglePane = totalIndicators === domRowCount;
expect(
isDualPane || isSinglePane,
`install-state indicator count mismatch: totalIndicators=${totalIndicators}, ` +
`domRowCount=${domRowCount}. Expected either ${logicalRowCount} (dual-pane) or ${domRowCount} (single-pane).`,
).toBe(true);
});
test('search input filters the model grid', async ({ page }) => {
await clickMenuButton(page, 'Model Manager');
await page.waitForSelector('.cmm-manager-grid, .tg-body', { timeout: 15_000 });
await page.waitForFunction(
() => document.querySelectorAll('.tg-body .tg-row, .cmm-manager-grid tr').length > 0,
{ timeout: 30_000, polling: 1_000 },
);
const searchInput = page.locator('.cmm-manager-keywords, input[type="text"][placeholder*="earch"], input[type="search"]').first();
await expect(searchInput).toBeVisible({ timeout: 5_000 });
const initialCount = await page.locator('.tg-body .tg-row, .cmm-manager-grid tr').count();
await searchInput.fill('stable diffusion');
// State-based wait: row count must change (or narrow). If the search
// is entirely broken and returns all rows, this will fail the poll.
await expect
.poll(
async () => page.locator('.tg-body .tg-row, .cmm-manager-grid tr').count(),
{ timeout: 10_000 },
)
.not.toBe(initialCount);
const filteredCount = await page.locator('.tg-body .tg-row, .cmm-manager-grid tr').count();
expect(filteredCount).toBeLessThanOrEqual(initialCount);
});
test('filter dropdown is present with expected options', async ({ page }) => {
// Wave3 WI-U Cluster H target 5: previously options.length>0 only.
// Now asserts the filter dropdown surfaces all 4 known states defined by
// ModelManager.initFilter() in js/model-manager.js:74-86 —
// `All`, `Installed`, `Not Installed`, `In Workflow`.
await clickMenuButton(page, 'Model Manager');
await page.waitForSelector('#cmm-manager-dialog', {
timeout: 15_000,
});
const dialog = page.locator('#cmm-manager-dialog').last();
const filterSelect = dialog.locator('select').filter({ hasText: /All|Installed/ }).first();
await expect(filterSelect).toBeVisible({ timeout: 5_000 });
const options = (await filterSelect.locator('option').allTextContents()).map((s) => s.trim());
// Exact set match (normalized): js/model-manager.js:74-86 defines this
// list. If labels change, update this assertion consciously.
const expected = ['All', 'Installed', 'Not Installed', 'In Workflow'];
const actual = new Set(options);
for (const label of expected) {
expect(
actual.has(label),
`filter dropdown missing expected option "${label}". Options=${JSON.stringify(options)}`,
).toBe(true);
}
});
});
@@ -0,0 +1,63 @@
/**
* E2E tests: Dialog navigation and lifecycle.
*
* Tests opening/closing dialogs, nested dialog navigation, and
* verifies no duplicate instances are created.
*
* Requires ComfyUI running with --enable-manager-legacy-ui on PORT.
*/
import { test, expect } from '@playwright/test';
import { waitForComfyUI, openManagerMenu, clickMenuButton, closeDialog, assertManagerMenuVisible } from './helpers';
test.describe('Dialog Navigation', () => {
test.beforeEach(async ({ page }) => {
await page.goto('/');
await waitForComfyUI(page);
});
test('Manager menu → Custom Nodes → close → Manager still visible', async ({ page }) => {
await openManagerMenu(page);
await assertManagerMenuVisible(page);
await clickMenuButton(page, 'Custom Nodes Manager');
await page.waitForSelector('#cn-manager-dialog', {
timeout: 15_000,
});
// Close the Custom Nodes dialog
await closeDialog(page);
await page.waitForTimeout(500);
// Manager menu should still be accessible (reopen if needed)
await openManagerMenu(page);
await assertManagerMenuVisible(page);
});
test('Manager menu → Model Manager → close → reopen', async ({ page }) => {
await openManagerMenu(page);
await clickMenuButton(page, 'Model Manager');
await page.waitForSelector('#cmm-manager-dialog', {
timeout: 15_000,
});
// Close the Model Manager dialog via its close button (p-dialog-close-button)
const mmMask = page.locator('.p-dialog-mask:has(#cmm-manager-dialog)');
const mmCloseBtn = mmMask.locator('button[aria-label="Close"], .p-dialog-close-button').first();
if (await mmCloseBtn.isVisible({ timeout: 2_000 }).catch(() => false)) {
await mmCloseBtn.click();
} else {
await page.keyboard.press('Escape');
}
await page.waitForTimeout(1_000);
// Reopen: need to open Manager menu first, then Model Manager
await openManagerMenu(page);
await clickMenuButton(page, 'Model Manager');
await page.waitForSelector('#cmm-manager-dialog', {
timeout: 15_000,
});
});
});
+124
View File
@@ -0,0 +1,124 @@
/**
* E2E tests: Legacy Snapshot Manager dialog.
*
* Tests UI-driven save and remove operations.
* Requires ComfyUI running with --enable-manager-legacy-ui on PORT.
*/
import { test, expect } from '@playwright/test';
import { waitForComfyUI, openManagerMenu, clickMenuButton } from './helpers';
const SNAPSHOT_ROW_SELECTOR = '#snapshot-manager-dialog tr, #snapshot-manager-dialog li';
async function getSnapshotNames(page: import('@playwright/test').Page): Promise<string[]> {
const resp = await page.request.get('/v2/snapshot/getlist');
if (!resp.ok()) return [];
const data = await resp.json();
return Array.isArray(data?.items) ? data.items : [];
}
test.describe('Snapshot Manager', () => {
// Track snapshots created during each test so afterEach can clean them up.
// Prevents test-run accumulation on disk across runs.
const createdDuringTest = new Set<string>();
test.beforeEach(async ({ page }) => {
createdDuringTest.clear();
await page.goto('/');
await waitForComfyUI(page);
await openManagerMenu(page);
});
test.afterEach(async ({ page }) => {
// Cleanup snapshots newly created during the test to avoid state leak.
for (const name of createdDuringTest) {
await page.request.post(`/v2/snapshot/remove?target=${encodeURIComponent(name)}`);
}
});
test('opens snapshot manager from Manager menu', async ({ page }) => {
await clickMenuButton(page, 'Snapshot Manager');
// Snapshot manager should appear
await page.waitForSelector('#snapshot-manager-dialog', {
timeout: 10_000,
});
});
test('SS1 Save button creates a new snapshot row', async ({ page }) => {
await clickMenuButton(page, 'Snapshot Manager');
await page.waitForSelector('#snapshot-manager-dialog', { timeout: 10_000 });
// Baseline snapshot names (not row count — more reliable)
const namesBefore = await getSnapshotNames(page);
// Click Save button (UI-driven). Hard fail if the button doesn't exist.
const saveBtn = page
.locator('#snapshot-manager-dialog button:has-text("Save"), #snapshot-manager-dialog button:has-text("Create")')
.first();
await expect(saveBtn).toBeVisible({ timeout: 5_000 });
await saveBtn.click();
// Wait for new snapshot to appear in backend list (UI row count may lag)
await expect
.poll(async () => (await getSnapshotNames(page)).length, { timeout: 15_000 })
.toBeGreaterThan(namesBefore.length);
const namesAfter = await getSnapshotNames(page);
const newNames = namesAfter.filter((n) => !namesBefore.includes(n));
expect(newNames.length).toBeGreaterThanOrEqual(1);
// Register for afterEach cleanup
newNames.forEach((n) => createdDuringTest.add(n));
// UI row count should also reflect the new snapshot
const rowsAfter = await page.locator(SNAPSHOT_ROW_SELECTOR).count();
expect(rowsAfter).toBeGreaterThan(0);
});
test('UI Remove button deletes a snapshot row', async ({ page }) => {
// SETUP: create a snapshot via API so we have a deterministic target
const saveResp = await page.request.post('/v2/snapshot/save');
expect(saveResp.ok()).toBe(true);
const namesAfterSave = await getSnapshotNames(page);
expect(namesAfterSave.length).toBeGreaterThan(0);
const targetName = namesAfterSave[0]; // desc-sorted — newest at [0]
// Open the Snapshot Manager via UI
await clickMenuButton(page, 'Snapshot Manager');
await page.waitForSelector('#snapshot-manager-dialog', { timeout: 10_000 });
// Locate the row containing our target snapshot
const targetRow = page
.locator(SNAPSHOT_ROW_SELECTOR, { hasText: targetName })
.first();
await expect(targetRow).toBeVisible({ timeout: 10_000 });
// Click the Remove/Delete button inside that row (UI-driven)
const removeBtn = targetRow.locator(
'button:has-text("Remove"), button:has-text("Delete"), button[title*="emove" i], button[title*="elete" i]',
);
if (!(await removeBtn.first().isVisible({ timeout: 2_000 }).catch(() => false))) {
// If the Remove UI is a right-click / hover / icon without text, register for
// cleanup via the afterEach and report a specific failure so the test surfaces
// the UI gap rather than pretending it verified deletion.
createdDuringTest.add(targetName);
throw new Error(
'Remove/Delete button not found in snapshot row — ' +
'UI regression or selector change; update selector to match current UI',
);
}
// Accept confirmation dialog if the UI raises one
page.once('dialog', async (d) => {
await d.accept();
});
await removeBtn.first().click();
// Effect verification: snapshot disappears from backend AND from UI
await expect
.poll(async () => (await getSnapshotNames(page)).includes(targetName), { timeout: 10_000 })
.toBe(false);
await expect(page.locator(SNAPSHOT_ROW_SELECTOR, { hasText: targetName })).toHaveCount(0);
});
});