mirror of
https://github.com/Comfy-Org/ComfyUI-Manager.git
synced 2026-03-07 18:17:36 +08:00
test(deps): add E2E scripts and update test documentation
Add automated E2E test scripts for unified dependency resolver: - setup_e2e_env.sh: idempotent environment setup (clone ComfyUI, create venv, install deps, symlink Manager, write config.ini) - start_comfyui.sh: foreground-blocking launcher using tail -f | grep -q readiness detection - stop_comfyui.sh: graceful SIGTERM → SIGKILL shutdown Update test documentation reflecting E2E testing findings: - TEST-environment-setup.md: add automated script usage, document caveats (PYTHONPATH, config.ini path, Manager v4 /v2/ prefix, Blocked by policy, bash ((var++)) trap, git+https:// rejection) - TEST-unified-dep-resolver.md: add TC-17 (restart dependency detection), TC-18 (real node pack integration), Validated Behaviors section, normalize API port to 8199 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
3d9c9ca8de
commit
ca8698533d
@ -2,13 +2,48 @@
|
||||
|
||||
Procedures for setting up a ComfyUI environment with ComfyUI-Manager installed for functional testing.
|
||||
|
||||
## Automated Setup (Recommended)
|
||||
|
||||
Three shell scripts in `tests/e2e/scripts/` automate the entire lifecycle:
|
||||
|
||||
```bash
|
||||
# 1. Setup: clone ComfyUI, create venv, install deps, symlink Manager
|
||||
E2E_ROOT=/tmp/e2e_test MANAGER_ROOT=/path/to/comfyui-manager-draft4 \
|
||||
bash tests/e2e/scripts/setup_e2e_env.sh
|
||||
|
||||
# 2. Start: launches ComfyUI in background, blocks until ready
|
||||
E2E_ROOT=/tmp/e2e_test bash tests/e2e/scripts/start_comfyui.sh
|
||||
|
||||
# 3. Stop: graceful SIGTERM → SIGKILL shutdown
|
||||
E2E_ROOT=/tmp/e2e_test bash tests/e2e/scripts/stop_comfyui.sh
|
||||
|
||||
# 4. Cleanup
|
||||
rm -rf /tmp/e2e_test
|
||||
```
|
||||
|
||||
### Script Details
|
||||
|
||||
| Script | Purpose | Input | Output |
|
||||
|--------|---------|-------|--------|
|
||||
| `setup_e2e_env.sh` | Full environment setup (8 steps) | `E2E_ROOT`, `MANAGER_ROOT`, `COMFYUI_BRANCH` (default: master), `PYTHON` (default: python3) | `E2E_ROOT=<path>` on last line |
|
||||
| `start_comfyui.sh` | Foreground-blocking launcher | `E2E_ROOT`, `PORT` (default: 8199), `TIMEOUT` (default: 120s) | `COMFYUI_PID=<pid> PORT=<port>` |
|
||||
| `stop_comfyui.sh` | Graceful shutdown | `E2E_ROOT`, `PORT` (default: 8199) | — |
|
||||
|
||||
**Idempotent**: `setup_e2e_env.sh` checks for a `.e2e_setup_complete` marker file and skips setup if the environment already exists.
|
||||
|
||||
**Blocking mechanism**: `start_comfyui.sh` uses `tail -n +1 -f | grep -q -m1 'To see the GUI'` to block until ComfyUI is ready. No polling loop needed.
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python 3.9+
|
||||
- Git
|
||||
- `uv` (pip alternative, install via `pip install uv`)
|
||||
- `uv` (install via `pip install uv` or [standalone](https://docs.astral.sh/uv/getting-started/installation/))
|
||||
|
||||
## Environment Setup
|
||||
## Manual Setup (Reference)
|
||||
|
||||
For understanding or debugging, the manual steps are documented below. The automated scripts execute these same steps.
|
||||
|
||||
### 1. ComfyUI Clone
|
||||
|
||||
@ -50,17 +85,44 @@ uv pip install -e "$MANAGER_ROOT"
|
||||
> **Note**: Editable mode (`-e`) reflects code changes without reinstalling.
|
||||
> For production-like testing, use `uv pip install "$MANAGER_ROOT"` (non-editable).
|
||||
|
||||
### 5. ComfyUI Launch
|
||||
### 5. Symlink Manager into custom_nodes
|
||||
|
||||
```bash
|
||||
ln -s "$MANAGER_ROOT" "$COMFY_ROOT/custom_nodes/ComfyUI-Manager"
|
||||
```
|
||||
|
||||
### 6. Write config.ini
|
||||
|
||||
```bash
|
||||
mkdir -p "$COMFY_ROOT/user/__manager"
|
||||
cat > "$COMFY_ROOT/user/__manager/config.ini" << 'EOF'
|
||||
[default]
|
||||
use_uv = true
|
||||
use_unified_resolver = true
|
||||
EOF
|
||||
```
|
||||
|
||||
> **IMPORTANT**: The config path is `$COMFY_ROOT/user/__manager/config.ini`, resolved by `folder_paths.get_system_user_directory("manager")`. It is NOT inside the symlinked Manager directory.
|
||||
|
||||
### 7. HOME Isolation
|
||||
|
||||
```bash
|
||||
export HOME=/tmp/e2e_home
|
||||
mkdir -p "$HOME/.config" "$HOME/.local/share"
|
||||
```
|
||||
|
||||
### 8. ComfyUI Launch
|
||||
|
||||
```bash
|
||||
cd "$COMFY_ROOT"
|
||||
python main.py --enable-manager --cpu
|
||||
PYTHONUNBUFFERED=1 python main.py --enable-manager --cpu --port 8199
|
||||
```
|
||||
|
||||
| Flag | Purpose |
|
||||
|------|---------|
|
||||
| `--enable-manager` | Enable ComfyUI-Manager (disabled by default) |
|
||||
| `--cpu` | Run without GPU (for functional testing) |
|
||||
| `--port 8199` | Use non-default port to avoid conflicts |
|
||||
| `--enable-manager-legacy-ui` | Enable legacy UI (optional) |
|
||||
| `--listen` | Allow remote connections (optional) |
|
||||
|
||||
@ -85,24 +147,48 @@ When Manager loads successfully, the following log lines appear:
|
||||
[START] ComfyUI-Manager # manager_server.py loaded
|
||||
```
|
||||
|
||||
The `Blocked by policy` message for Manager in custom_nodes is **expected** — `should_be_disabled()` in `comfyui_manager/__init__.py` prevents legacy double-loading when Manager is already pip-installed.
|
||||
|
||||
---
|
||||
|
||||
## Caveats & Known Issues
|
||||
|
||||
### PYTHONPATH for `comfy` imports
|
||||
|
||||
ComfyUI's `comfy` package is a **local package** inside the ComfyUI directory — it is NOT pip-installed. Any code that imports from `comfy` (including `comfyui_manager.__init__`) requires `PYTHONPATH` to include the ComfyUI directory:
|
||||
|
||||
```bash
|
||||
PYTHONPATH="$COMFY_ROOT" python -c "import comfy"
|
||||
PYTHONPATH="$COMFY_ROOT" python -c "import comfyui_manager"
|
||||
```
|
||||
|
||||
The automated scripts handle this via `PYTHONPATH` in verification checks and the ComfyUI process inherits it implicitly by running from the ComfyUI directory.
|
||||
|
||||
### config.ini path
|
||||
|
||||
The config file must be at `$COMFY_ROOT/user/__manager/config.ini`, **NOT** inside the Manager symlink directory. This is resolved by `folder_paths.get_system_user_directory("manager")` at `prestartup_script.py:65-73`.
|
||||
|
||||
### Manager v4 endpoint prefix
|
||||
|
||||
All Manager endpoints use the `/v2/` prefix (e.g., `/v2/manager/queue/status`, `/v2/snapshot/get_current`). Paths without the prefix will return 404.
|
||||
|
||||
### `Blocked by policy` is expected
|
||||
|
||||
When Manager detects that it's loaded as a custom_node but is already pip-installed, it prints `Blocked by policy` and skips legacy loading. This is intentional behavior in `comfyui_manager/__init__.py:39-51`.
|
||||
|
||||
### Bash `((var++))` trap
|
||||
|
||||
Under `set -e`, `((0++))` evaluates the pre-increment value (0), and `(( 0 ))` returns exit code 1, killing the script. Use `var=$((var + 1))` instead.
|
||||
|
||||
### `git+https://` URLs in requirements.txt
|
||||
|
||||
Some node packs (e.g., Impact Pack's SAM2 dependency) use `git+https://github.com/...` URLs. The unified resolver correctly rejects these with "rejected path separator" — they must be installed separately.
|
||||
|
||||
---
|
||||
|
||||
## Cleanup
|
||||
|
||||
```bash
|
||||
deactivate
|
||||
rm -rf "$COMFY_ROOT"
|
||||
```
|
||||
|
||||
## Quick Reference (Copy-Paste)
|
||||
|
||||
```bash
|
||||
# Full setup in one block
|
||||
COMFY_ROOT=$(mktemp -d)/ComfyUI
|
||||
MANAGER_ROOT=/path/to/comfyui-manager-draft4
|
||||
|
||||
git clone https://github.com/comfyanonymous/ComfyUI.git "$COMFY_ROOT"
|
||||
cd "$COMFY_ROOT"
|
||||
uv venv .venv && source .venv/bin/activate
|
||||
uv pip install -r requirements.txt --extra-index-url https://download.pytorch.org/whl/cpu
|
||||
uv pip install -e "$MANAGER_ROOT"
|
||||
python main.py --enable-manager --cpu
|
||||
```
|
||||
|
||||
@ -21,10 +21,12 @@ use_unified_resolver = true
|
||||
Node pack installation at runtime uses the task queue API:
|
||||
|
||||
```
|
||||
POST http://localhost:8188/v2/manager/queue/task
|
||||
POST http://localhost:8199/v2/manager/queue/task
|
||||
Content-Type: application/json
|
||||
```
|
||||
|
||||
> **Port**: E2E tests use port 8199 to avoid conflicts with running ComfyUI instances. Replace with your actual port if different.
|
||||
|
||||
**Payload** (`QueueTaskItem`):
|
||||
|
||||
| Field | Type | Description |
|
||||
@ -186,7 +188,7 @@ chmod -R u+w "$(python -c 'import site; print(site.getsitepackages()[0])')"
|
||||
**Steps**:
|
||||
1. While ComfyUI is running, install a node pack that has both `install.py` and `requirements.txt` via API:
|
||||
```bash
|
||||
curl -X POST http://localhost:8188/v2/manager/queue/task \
|
||||
curl -X POST http://localhost:8199/v2/manager/queue/task \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"ui_id": "test-installpy",
|
||||
@ -334,7 +336,7 @@ private-pkg --index-url https://user:token123@pypi.private.com/simple
|
||||
1. Start ComfyUI and confirm batch resolution succeeds
|
||||
2. While ComfyUI is running, install a new node pack via API:
|
||||
```bash
|
||||
curl -X POST http://localhost:8188/v2/manager/queue/task \
|
||||
curl -X POST http://localhost:8199/v2/manager/queue/task \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"ui_id": "test-defer-1",
|
||||
@ -377,7 +379,7 @@ Verify both code locations that guard per-node pip install behave correctly in u
|
||||
**Path 1 — Runtime API install (class method)**:
|
||||
```bash
|
||||
# While ComfyUI is running:
|
||||
curl -X POST http://localhost:8188/v2/manager/queue/task \
|
||||
curl -X POST http://localhost:8199/v2/manager/queue/task \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"ui_id": "test-path1",
|
||||
@ -426,7 +428,7 @@ echo "['$COMFY_ROOT/custom_nodes/test_pack_lazy', '#LAZY-INSTALL-SCRIPT', '$PYTH
|
||||
1. Set up conflicting deps (as in TC-4) and start ComfyUI (resolver fails, flag reset to `False`)
|
||||
2. While still running, install a new node pack via API:
|
||||
```bash
|
||||
curl -X POST http://localhost:8188/v2/manager/queue/task \
|
||||
curl -X POST http://localhost:8199/v2/manager/queue/task \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"ui_id": "test-postfallback",
|
||||
@ -478,6 +480,101 @@ chmod u+r "$COMFY_ROOT/custom_nodes"
|
||||
|
||||
---
|
||||
|
||||
## TC-17: Restart Dependency Detection [P0]
|
||||
|
||||
**Precondition**: `use_unified_resolver = true`, automated E2E scripts available
|
||||
|
||||
This test verifies that the resolver correctly detects and installs dependencies for node packs added between restarts, incrementally building the dependency set.
|
||||
|
||||
**Steps**:
|
||||
1. Boot ComfyUI with no custom node packs (Boot 1 — baseline)
|
||||
2. Verify baseline deps only (Manager's own deps)
|
||||
3. Stop ComfyUI
|
||||
4. Clone `ComfyUI-Impact-Pack` into `custom_nodes/`
|
||||
5. Restart ComfyUI (Boot 2)
|
||||
6. Verify Impact Pack deps are installed (`cv2`, `skimage`, `dill`, `scipy`, `matplotlib`)
|
||||
7. Stop ComfyUI
|
||||
8. Clone `ComfyUI-Inspire-Pack` into `custom_nodes/`
|
||||
9. Restart ComfyUI (Boot 3)
|
||||
10. Verify Inspire Pack deps are installed (`cachetools`, `webcolors`)
|
||||
|
||||
**Expected log (each boot)**:
|
||||
```
|
||||
[UnifiedDepResolver] Collected N deps from M sources (skipped S)
|
||||
[UnifiedDepResolver] running: ... uv pip compile ...
|
||||
[UnifiedDepResolver] running: ... uv pip install ...
|
||||
[UnifiedDepResolver] startup batch resolution succeeded
|
||||
```
|
||||
|
||||
**Verify**:
|
||||
- Boot 1: ~10 deps from ~10 sources; `cv2`, `dill`, `cachetools` are NOT installed
|
||||
- Boot 2: ~19 deps from ~18 sources; `cv2`, `skimage`, `dill`, `scipy`, `matplotlib` all importable
|
||||
- Boot 3: ~24 deps from ~21 sources; `cachetools`, `webcolors` also importable
|
||||
- Both packs show as loaded in logs
|
||||
|
||||
**Automation**: Use `tests/e2e/scripts/` (setup → start → stop) with node pack cloning between boots.
|
||||
|
||||
---
|
||||
|
||||
## TC-18: Real Node Pack Integration [P0]
|
||||
|
||||
**Precondition**: `use_unified_resolver = true`, network access to GitHub + PyPI
|
||||
|
||||
Full pipeline test with real-world node packs (`ComfyUI-Impact-Pack` + `ComfyUI-Inspire-Pack`) to verify the resolver handles production requirements.txt files correctly.
|
||||
|
||||
**Steps**:
|
||||
1. Set up E2E environment
|
||||
2. Clone both Impact Pack and Inspire Pack into `custom_nodes/`
|
||||
3. Direct-mode: instantiate `UnifiedDepResolver`, call `collect_requirements()` and `resolve_and_install()`
|
||||
4. Boot-mode: start ComfyUI and verify via logs
|
||||
|
||||
**Expected behavior (direct mode)**:
|
||||
```
|
||||
--- Discovered node packs (3) --- # Manager, Impact, Inspire
|
||||
ComfyUI-Impact-Pack
|
||||
ComfyUI-Inspire-Pack
|
||||
ComfyUI-Manager
|
||||
|
||||
--- Phase 1: Collect Requirements ---
|
||||
Total requirements: ~24
|
||||
Skipped: 1 # SAM2 git+https:// URL
|
||||
Extra index URLs: set()
|
||||
```
|
||||
|
||||
**Verify**:
|
||||
- `git+https://github.com/facebookresearch/sam2.git` is correctly rejected with "rejected path separator"
|
||||
- All other dependencies are collected and resolved
|
||||
- After install, `cv2`, `PIL`, `scipy`, `skimage`, `matplotlib` are all importable
|
||||
- No conflicting version errors during compile
|
||||
|
||||
**Automation**: Use `tests/e2e/scripts/` (setup → clone packs → start) with direct-mode resolver invocation.
|
||||
|
||||
---
|
||||
|
||||
## Validated Behaviors (from E2E Testing)
|
||||
|
||||
The following behaviors were confirmed during manual E2E testing:
|
||||
|
||||
### Resolver Pipeline
|
||||
- **3-phase pipeline**: Collect → `uv pip compile` → `uv pip install` works end-to-end
|
||||
- **Incremental detection**: Resolver discovers new node packs on each restart without reinstalling existing deps
|
||||
- **Dependency deduplication**: Overlapping deps from multiple packs are resolved to compatible versions
|
||||
|
||||
### Security & Filtering
|
||||
- **`git+https://` rejection**: URLs like `git+https://github.com/facebookresearch/sam2.git` are rejected with "rejected path separator" — SAM2 is the only dependency skipped from Impact Pack
|
||||
- **Blacklist filtering**: `PackageRequirement` objects have `.name`, `.spec`, `.source` attributes; `collected.skipped` returns `[(spec_string, reason_string)]` tuples
|
||||
|
||||
### Manager Integration
|
||||
- **Manager v4 endpoints**: All endpoints use `/v2/` prefix (e.g., `/v2/manager/queue/status`)
|
||||
- **`Blocked by policy`**: Expected when Manager is pip-installed and also symlinked in `custom_nodes/`; prevents legacy double-loading
|
||||
- **config.ini path**: Must be at `$COMFY_ROOT/user/__manager/config.ini`, not in the symlinked Manager dir
|
||||
|
||||
### Environment
|
||||
- **PYTHONPATH requirement**: `comfy` is a local package (not pip-installed); `comfyui_manager` imports from `comfy`, so both require `PYTHONPATH=$COMFY_ROOT`
|
||||
- **HOME isolation**: `HOME=$E2E_ROOT/home` prevents host config contamination during boot
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
| TC | P | Scenario | Key Verification |
|
||||
@ -498,6 +595,8 @@ chmod u+r "$COMFY_ROOT/custom_nodes"
|
||||
| 14 | P0 | Both unified resolver paths | runtime API (class method) + startup lazy install |
|
||||
| 15 | P1 | Post-fallback behavior | Per-node pip resumes in same process |
|
||||
| 16 | P1 | Generic exception fallback | Distinct from uv-absent and batch-failed |
|
||||
| 17 | P0 | Restart dependency detection | Incremental node pack discovery across restarts |
|
||||
| 18 | P0 | Real node pack integration | Impact + Inspire Pack full pipeline |
|
||||
|
||||
### Traceability
|
||||
|
||||
@ -513,3 +612,6 @@ chmod u+r "$COMFY_ROOT/custom_nodes"
|
||||
| Fallback behavior | TC-3, TC-4, TC-5, TC-15, TC-16 |
|
||||
| Disabled node pack exclusion | TC-11 |
|
||||
| Runtime defer behavior | TC-13, TC-14 |
|
||||
| FR-8: Restart discovery | TC-17 |
|
||||
| FR-9: Real-world compatibility | TC-17, TC-18 |
|
||||
| FR-2: Input sanitization (git URLs) | TC-8, TC-18 |
|
||||
|
||||
241
tests/e2e/scripts/setup_e2e_env.sh
Executable file
241
tests/e2e/scripts/setup_e2e_env.sh
Executable file
@ -0,0 +1,241 @@
|
||||
#!/usr/bin/env bash
|
||||
# setup_e2e_env.sh — Automated E2E environment setup for ComfyUI + Manager
|
||||
#
|
||||
# Creates an isolated ComfyUI installation with ComfyUI-Manager for E2E testing.
|
||||
# Idempotent: skips setup if marker file and key artifacts already exist.
|
||||
#
|
||||
# Input env vars:
|
||||
# E2E_ROOT — target directory (default: auto-generated via mktemp)
|
||||
# MANAGER_ROOT — manager repo root (default: auto-detected from script location)
|
||||
# COMFYUI_BRANCH — ComfyUI branch to clone (default: master)
|
||||
# PYTHON — Python executable (default: python3)
|
||||
#
|
||||
# Output (last line of stdout):
|
||||
# E2E_ROOT=/path/to/environment
|
||||
#
|
||||
# Exit: 0=success, 1=failure
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# --- Constants ---
|
||||
COMFYUI_REPO="https://github.com/comfyanonymous/ComfyUI.git"
|
||||
PYTORCH_CPU_INDEX="https://download.pytorch.org/whl/cpu"
|
||||
CONFIG_INI_CONTENT="[default]
|
||||
use_uv = true
|
||||
use_unified_resolver = true
|
||||
file_logging = false"
|
||||
|
||||
# --- Logging helpers ---
|
||||
log() { echo "[setup_e2e] $*"; }
|
||||
err() { echo "[setup_e2e] ERROR: $*" >&2; }
|
||||
die() { err "$@"; exit 1; }
|
||||
|
||||
# --- Detect manager root by walking up from script dir to find pyproject.toml ---
|
||||
detect_manager_root() {
|
||||
local dir
|
||||
dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
while [[ "$dir" != "/" ]]; do
|
||||
if [[ -f "$dir/pyproject.toml" ]]; then
|
||||
echo "$dir"
|
||||
return 0
|
||||
fi
|
||||
dir="$(dirname "$dir")"
|
||||
done
|
||||
return 1
|
||||
}
|
||||
|
||||
# --- Validate prerequisites ---
|
||||
validate_prerequisites() {
|
||||
local py="${PYTHON:-python3}"
|
||||
local missing=()
|
||||
|
||||
command -v git >/dev/null 2>&1 || missing+=("git")
|
||||
command -v uv >/dev/null 2>&1 || missing+=("uv")
|
||||
command -v "$py" >/dev/null 2>&1 || missing+=("$py")
|
||||
|
||||
if [[ ${#missing[@]} -gt 0 ]]; then
|
||||
die "Missing prerequisites: ${missing[*]}"
|
||||
fi
|
||||
|
||||
# Verify Python version >= 3.9
|
||||
local py_version
|
||||
py_version=$("$py" -c "import sys; print(f'{sys.version_info.major}.{sys.version_info.minor}')")
|
||||
local major minor
|
||||
major="${py_version%%.*}"
|
||||
minor="${py_version##*.}"
|
||||
if [[ "$major" -lt 3 ]] || { [[ "$major" -eq 3 ]] && [[ "$minor" -lt 9 ]]; }; then
|
||||
die "Python 3.9+ required, found $py_version"
|
||||
fi
|
||||
log "Prerequisites OK (python=$py_version, git=$(git --version | awk '{print $3}'), uv=$(uv --version 2>&1 | awk '{print $2}'))"
|
||||
}
|
||||
|
||||
# --- Check idempotency: skip if already set up ---
|
||||
check_already_setup() {
|
||||
local root="$1"
|
||||
if [[ -f "$root/.e2e_setup_complete" ]] \
|
||||
&& [[ -d "$root/comfyui" ]] \
|
||||
&& [[ -d "$root/venv" ]] \
|
||||
&& [[ -f "$root/comfyui/user/__manager/config.ini" ]] \
|
||||
&& [[ -L "$root/comfyui/custom_nodes/ComfyUI-Manager" ]]; then
|
||||
log "Environment already set up at $root (marker file exists). Skipping."
|
||||
echo "E2E_ROOT=$root"
|
||||
exit 0
|
||||
fi
|
||||
}
|
||||
|
||||
# --- Verify the setup ---
|
||||
verify_setup() {
|
||||
local root="$1"
|
||||
local manager_root="$2"
|
||||
local venv_py="$root/venv/bin/python"
|
||||
local errors=0
|
||||
|
||||
log "Running verification checks..."
|
||||
|
||||
# Check ComfyUI directory
|
||||
if [[ ! -f "$root/comfyui/main.py" ]]; then
|
||||
err "Verification FAIL: comfyui/main.py not found"
|
||||
((errors++))
|
||||
fi
|
||||
|
||||
# Check venv python
|
||||
if [[ ! -x "$venv_py" ]]; then
|
||||
err "Verification FAIL: venv python not executable"
|
||||
((errors++))
|
||||
fi
|
||||
|
||||
# Check symlink
|
||||
local link_target="$root/comfyui/custom_nodes/ComfyUI-Manager"
|
||||
if [[ ! -L "$link_target" ]]; then
|
||||
err "Verification FAIL: symlink $link_target does not exist"
|
||||
((errors++))
|
||||
elif [[ "$(readlink -f "$link_target")" != "$(readlink -f "$manager_root")" ]]; then
|
||||
err "Verification FAIL: symlink target mismatch"
|
||||
((errors++))
|
||||
fi
|
||||
|
||||
# Check config.ini
|
||||
if [[ ! -f "$root/comfyui/user/__manager/config.ini" ]]; then
|
||||
err "Verification FAIL: config.ini not found"
|
||||
((errors++))
|
||||
fi
|
||||
|
||||
# Check Python imports
|
||||
# comfy is a local package inside ComfyUI (not pip-installed), and
|
||||
# comfyui_manager.__init__ imports from comfy — both need PYTHONPATH
|
||||
if ! PYTHONPATH="$root/comfyui" "$venv_py" -c "import comfy" 2>/dev/null; then
|
||||
err "Verification FAIL: 'import comfy' failed"
|
||||
((errors++))
|
||||
fi
|
||||
|
||||
if ! PYTHONPATH="$root/comfyui" "$venv_py" -c "import comfyui_manager" 2>/dev/null; then
|
||||
err "Verification FAIL: 'import comfyui_manager' failed"
|
||||
((errors++))
|
||||
fi
|
||||
|
||||
if [[ "$errors" -gt 0 ]]; then
|
||||
die "Verification failed with $errors error(s)"
|
||||
fi
|
||||
log "Verification OK: all checks passed"
|
||||
}
|
||||
|
||||
# ===== Main =====
|
||||
|
||||
# Resolve MANAGER_ROOT
|
||||
if [[ -z "${MANAGER_ROOT:-}" ]]; then
|
||||
MANAGER_ROOT="$(detect_manager_root)" || die "Cannot detect MANAGER_ROOT. Set it explicitly."
|
||||
fi
|
||||
MANAGER_ROOT="$(cd "$MANAGER_ROOT" && pwd)"
|
||||
log "MANAGER_ROOT=$MANAGER_ROOT"
|
||||
|
||||
# Validate prerequisites
|
||||
validate_prerequisites
|
||||
|
||||
PYTHON="${PYTHON:-python3}"
|
||||
COMFYUI_BRANCH="${COMFYUI_BRANCH:-master}"
|
||||
|
||||
# Create or use E2E_ROOT
|
||||
CREATED_BY_US=false
|
||||
if [[ -z "${E2E_ROOT:-}" ]]; then
|
||||
E2E_ROOT="$(mktemp -d -t e2e_comfyui_XXXXXX)"
|
||||
CREATED_BY_US=true
|
||||
log "Created E2E_ROOT=$E2E_ROOT"
|
||||
else
|
||||
mkdir -p "$E2E_ROOT"
|
||||
log "Using E2E_ROOT=$E2E_ROOT"
|
||||
fi
|
||||
|
||||
# Idempotency check
|
||||
check_already_setup "$E2E_ROOT"
|
||||
|
||||
# Cleanup trap: remove E2E_ROOT on failure only if we created it
|
||||
cleanup_on_failure() {
|
||||
local exit_code=$?
|
||||
if [[ "$exit_code" -ne 0 ]] && [[ "$CREATED_BY_US" == "true" ]]; then
|
||||
err "Setup failed. Cleaning up $E2E_ROOT"
|
||||
rm -rf "$E2E_ROOT"
|
||||
fi
|
||||
}
|
||||
trap cleanup_on_failure EXIT
|
||||
|
||||
# Step 1: Clone ComfyUI
|
||||
log "Step 1/8: Cloning ComfyUI (branch=$COMFYUI_BRANCH)..."
|
||||
if [[ -d "$E2E_ROOT/comfyui/.git" ]]; then
|
||||
log " ComfyUI already cloned, skipping"
|
||||
else
|
||||
git clone --depth=1 --branch "$COMFYUI_BRANCH" "$COMFYUI_REPO" "$E2E_ROOT/comfyui"
|
||||
fi
|
||||
|
||||
# Step 2: Create virtual environment
|
||||
log "Step 2/8: Creating virtual environment..."
|
||||
if [[ -d "$E2E_ROOT/venv" ]]; then
|
||||
log " venv already exists, skipping"
|
||||
else
|
||||
uv venv "$E2E_ROOT/venv"
|
||||
fi
|
||||
VENV_PY="$E2E_ROOT/venv/bin/python"
|
||||
|
||||
# Step 3: Install ComfyUI dependencies
|
||||
log "Step 3/8: Installing ComfyUI dependencies (CPU-only)..."
|
||||
uv pip install \
|
||||
--python "$VENV_PY" \
|
||||
-r "$E2E_ROOT/comfyui/requirements.txt" \
|
||||
--extra-index-url "$PYTORCH_CPU_INDEX"
|
||||
|
||||
# Step 4: Install ComfyUI-Manager (non-editable, production-like)
|
||||
log "Step 4/8: Installing ComfyUI-Manager..."
|
||||
uv pip install --python "$VENV_PY" "$MANAGER_ROOT"
|
||||
|
||||
# Step 5: Create symlink for custom_nodes discovery
|
||||
log "Step 5/8: Creating custom_nodes symlink..."
|
||||
mkdir -p "$E2E_ROOT/comfyui/custom_nodes"
|
||||
local_link="$E2E_ROOT/comfyui/custom_nodes/ComfyUI-Manager"
|
||||
if [[ -L "$local_link" ]]; then
|
||||
log " Symlink already exists, updating"
|
||||
rm -f "$local_link"
|
||||
fi
|
||||
ln -s "$MANAGER_ROOT" "$local_link"
|
||||
|
||||
# Step 6: Write config.ini to correct path
|
||||
log "Step 6/8: Writing config.ini..."
|
||||
mkdir -p "$E2E_ROOT/comfyui/user/__manager"
|
||||
echo "$CONFIG_INI_CONTENT" > "$E2E_ROOT/comfyui/user/__manager/config.ini"
|
||||
|
||||
# Step 7: Create HOME isolation directories
|
||||
log "Step 7/8: Creating HOME isolation directories..."
|
||||
mkdir -p "$E2E_ROOT/home/.config"
|
||||
mkdir -p "$E2E_ROOT/home/.local/share"
|
||||
mkdir -p "$E2E_ROOT/logs"
|
||||
|
||||
# Step 8: Verify setup
|
||||
log "Step 8/8: Verifying setup..."
|
||||
verify_setup "$E2E_ROOT" "$MANAGER_ROOT"
|
||||
|
||||
# Write marker file
|
||||
date -Iseconds > "$E2E_ROOT/.e2e_setup_complete"
|
||||
|
||||
# Clear the EXIT trap since setup succeeded
|
||||
trap - EXIT
|
||||
|
||||
log "Setup complete."
|
||||
echo "E2E_ROOT=$E2E_ROOT"
|
||||
129
tests/e2e/scripts/start_comfyui.sh
Executable file
129
tests/e2e/scripts/start_comfyui.sh
Executable file
@ -0,0 +1,129 @@
|
||||
#!/usr/bin/env bash
|
||||
# start_comfyui.sh — Foreground-blocking ComfyUI launcher for E2E tests
|
||||
#
|
||||
# Starts ComfyUI in the background, then blocks the foreground until the server
|
||||
# is ready (or timeout). This makes it safe to call from subprocess.run() or
|
||||
# Claude's Bash tool — the call returns only when ComfyUI is accepting requests.
|
||||
#
|
||||
# Input env vars:
|
||||
# E2E_ROOT — (required) path to E2E environment from setup_e2e_env.sh
|
||||
# PORT — ComfyUI listen port (default: 8199)
|
||||
# TIMEOUT — max seconds to wait for readiness (default: 120)
|
||||
#
|
||||
# Output (last line on success):
|
||||
# COMFYUI_PID=<pid> PORT=<port>
|
||||
#
|
||||
# Exit: 0=ready, 1=timeout/failure
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# --- Defaults ---
|
||||
PORT="${PORT:-8199}"
|
||||
TIMEOUT="${TIMEOUT:-120}"
|
||||
|
||||
# --- Logging helpers ---
|
||||
log() { echo "[start_comfyui] $*"; }
|
||||
err() { echo "[start_comfyui] ERROR: $*" >&2; }
|
||||
die() { err "$@"; exit 1; }
|
||||
|
||||
# --- Validate environment ---
|
||||
[[ -n "${E2E_ROOT:-}" ]] || die "E2E_ROOT is not set"
|
||||
[[ -d "$E2E_ROOT/comfyui" ]] || die "ComfyUI not found at $E2E_ROOT/comfyui"
|
||||
[[ -x "$E2E_ROOT/venv/bin/python" ]] || die "venv python not found at $E2E_ROOT/venv/bin/python"
|
||||
[[ -f "$E2E_ROOT/.e2e_setup_complete" ]] || die "Setup marker not found. Run setup_e2e_env.sh first."
|
||||
|
||||
PY="$E2E_ROOT/venv/bin/python"
|
||||
COMFY_DIR="$E2E_ROOT/comfyui"
|
||||
LOG_DIR="$E2E_ROOT/logs"
|
||||
LOG_FILE="$LOG_DIR/comfyui.log"
|
||||
PID_FILE="$LOG_DIR/comfyui.pid"
|
||||
|
||||
mkdir -p "$LOG_DIR"
|
||||
|
||||
# --- Check/clear port ---
|
||||
if ss -tlnp 2>/dev/null | grep -q ":${PORT}\b"; then
|
||||
log "Port $PORT is in use. Attempting to stop existing process..."
|
||||
# Try to read existing PID file
|
||||
if [[ -f "$PID_FILE" ]]; then
|
||||
OLD_PID="$(cat "$PID_FILE")"
|
||||
if kill -0 "$OLD_PID" 2>/dev/null; then
|
||||
kill "$OLD_PID" 2>/dev/null || true
|
||||
sleep 2
|
||||
fi
|
||||
fi
|
||||
# Fallback: kill by port pattern
|
||||
if ss -tlnp 2>/dev/null | grep -q ":${PORT}\b"; then
|
||||
pkill -f "main\\.py.*--port $PORT" 2>/dev/null || true
|
||||
sleep 2
|
||||
fi
|
||||
# Final check
|
||||
if ss -tlnp 2>/dev/null | grep -q ":${PORT}\b"; then
|
||||
die "Port $PORT is still in use after cleanup attempt"
|
||||
fi
|
||||
log "Port $PORT cleared."
|
||||
fi
|
||||
|
||||
# --- Start ComfyUI ---
|
||||
log "Starting ComfyUI on port $PORT..."
|
||||
|
||||
# Create empty log file (ensures tail -f works from the start)
|
||||
: > "$LOG_FILE"
|
||||
|
||||
# Launch with unbuffered Python output so log lines appear immediately
|
||||
PYTHONUNBUFFERED=1 \
|
||||
HOME="$E2E_ROOT/home" \
|
||||
nohup "$PY" "$COMFY_DIR/main.py" \
|
||||
--cpu \
|
||||
--enable-manager \
|
||||
--port "$PORT" \
|
||||
> "$LOG_FILE" 2>&1 &
|
||||
COMFYUI_PID=$!
|
||||
|
||||
echo "$COMFYUI_PID" > "$PID_FILE"
|
||||
log "ComfyUI PID=$COMFYUI_PID, log=$LOG_FILE"
|
||||
|
||||
# Verify process didn't crash immediately
|
||||
sleep 1
|
||||
if ! kill -0 "$COMFYUI_PID" 2>/dev/null; then
|
||||
err "ComfyUI process died immediately. Last 30 lines of log:"
|
||||
tail -n 30 "$LOG_FILE" >&2
|
||||
rm -f "$PID_FILE"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# --- Block until ready ---
|
||||
# tail -n +1 -f: read from file start AND follow new content (no race condition)
|
||||
# grep -q -m1: exit on first match → tail gets SIGPIPE → pipeline ends
|
||||
# timeout: kill the pipeline after TIMEOUT seconds
|
||||
log "Waiting up to ${TIMEOUT}s for ComfyUI to become ready..."
|
||||
|
||||
if timeout "$TIMEOUT" bash -c \
|
||||
"tail -n +1 -f '$LOG_FILE' 2>/dev/null | grep -q -m1 'To see the GUI'"; then
|
||||
log "ComfyUI startup message detected."
|
||||
else
|
||||
err "Timeout (${TIMEOUT}s) waiting for ComfyUI. Last 30 lines of log:"
|
||||
tail -n 30 "$LOG_FILE" >&2
|
||||
kill "$COMFYUI_PID" 2>/dev/null || true
|
||||
rm -f "$PID_FILE"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Verify process is still alive after readiness detected
|
||||
if ! kill -0 "$COMFYUI_PID" 2>/dev/null; then
|
||||
err "ComfyUI process died after readiness signal. Last 30 lines:"
|
||||
tail -n 30 "$LOG_FILE" >&2
|
||||
rm -f "$PID_FILE"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Optional HTTP health check
|
||||
if command -v curl >/dev/null 2>&1; then
|
||||
if curl -sf "http://127.0.0.1:${PORT}/system_stats" >/dev/null 2>&1; then
|
||||
log "HTTP health check passed (/system_stats)"
|
||||
else
|
||||
log "HTTP health check skipped (endpoint not yet available, but startup message detected)"
|
||||
fi
|
||||
fi
|
||||
|
||||
log "ComfyUI is ready."
|
||||
echo "COMFYUI_PID=$COMFYUI_PID PORT=$PORT"
|
||||
75
tests/e2e/scripts/stop_comfyui.sh
Executable file
75
tests/e2e/scripts/stop_comfyui.sh
Executable file
@ -0,0 +1,75 @@
|
||||
#!/usr/bin/env bash
|
||||
# stop_comfyui.sh — Graceful ComfyUI shutdown for E2E tests
|
||||
#
|
||||
# Stops a ComfyUI process previously started by start_comfyui.sh.
|
||||
# Uses SIGTERM first, then SIGKILL after a grace period.
|
||||
#
|
||||
# Input env vars:
|
||||
# E2E_ROOT — (required) path to E2E environment
|
||||
# PORT — ComfyUI port for fallback pkill (default: 8199)
|
||||
#
|
||||
# Exit: 0=stopped, 1=failed
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
PORT="${PORT:-8199}"
|
||||
GRACE_PERIOD=10
|
||||
|
||||
# --- Logging helpers ---
|
||||
log() { echo "[stop_comfyui] $*"; }
|
||||
err() { echo "[stop_comfyui] ERROR: $*" >&2; }
|
||||
die() { err "$@"; exit 1; }
|
||||
|
||||
# --- Validate ---
|
||||
[[ -n "${E2E_ROOT:-}" ]] || die "E2E_ROOT is not set"
|
||||
|
||||
PID_FILE="$E2E_ROOT/logs/comfyui.pid"
|
||||
|
||||
# --- Read PID ---
|
||||
COMFYUI_PID=""
|
||||
if [[ -f "$PID_FILE" ]]; then
|
||||
COMFYUI_PID="$(cat "$PID_FILE")"
|
||||
log "Read PID=$COMFYUI_PID from $PID_FILE"
|
||||
fi
|
||||
|
||||
# --- Graceful shutdown via SIGTERM ---
|
||||
if [[ -n "$COMFYUI_PID" ]] && kill -0 "$COMFYUI_PID" 2>/dev/null; then
|
||||
log "Sending SIGTERM to PID $COMFYUI_PID..."
|
||||
kill "$COMFYUI_PID" 2>/dev/null || true
|
||||
|
||||
# Wait for graceful shutdown
|
||||
elapsed=0
|
||||
while kill -0 "$COMFYUI_PID" 2>/dev/null && [[ "$elapsed" -lt "$GRACE_PERIOD" ]]; do
|
||||
sleep 1
|
||||
elapsed=$((elapsed + 1))
|
||||
done
|
||||
|
||||
# Force kill if still alive
|
||||
if kill -0 "$COMFYUI_PID" 2>/dev/null; then
|
||||
log "Process still alive after ${GRACE_PERIOD}s. Sending SIGKILL..."
|
||||
kill -9 "$COMFYUI_PID" 2>/dev/null || true
|
||||
sleep 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- Fallback: kill by port pattern ---
|
||||
if ss -tlnp 2>/dev/null | grep -q ":${PORT}\b"; then
|
||||
log "Port $PORT still in use. Attempting pkill fallback..."
|
||||
pkill -f "main\\.py.*--port $PORT" 2>/dev/null || true
|
||||
sleep 2
|
||||
|
||||
if ss -tlnp 2>/dev/null | grep -q ":${PORT}\b"; then
|
||||
pkill -9 -f "main\\.py.*--port $PORT" 2>/dev/null || true
|
||||
sleep 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- Cleanup PID file ---
|
||||
rm -f "$PID_FILE"
|
||||
|
||||
# --- Verify port is free ---
|
||||
if ss -tlnp 2>/dev/null | grep -q ":${PORT}\b"; then
|
||||
die "Port $PORT is still in use after shutdown"
|
||||
fi
|
||||
|
||||
log "ComfyUI stopped."
|
||||
Loading…
Reference in New Issue
Block a user