test(deps): add E2E scripts and update test documentation
Some checks failed
CI / Validate OpenAPI Specification (push) Has been cancelled
CI / Code Quality Checks (push) Has been cancelled
Python Linting / Run Ruff (push) Has been cancelled

Add automated E2E test scripts for unified dependency resolver:
- setup_e2e_env.sh: idempotent environment setup (clone ComfyUI,
  create venv, install deps, symlink Manager, write config.ini)
- start_comfyui.sh: foreground-blocking launcher using
  tail -f | grep -q readiness detection
- stop_comfyui.sh: graceful SIGTERM → SIGKILL shutdown

Update test documentation reflecting E2E testing findings:
- TEST-environment-setup.md: add automated script usage, document
  caveats (PYTHONPATH, config.ini path, Manager v4 /v2/ prefix,
  Blocked by policy, bash ((var++)) trap, git+https:// rejection)
- TEST-unified-dep-resolver.md: add TC-17 (restart dependency
  detection), TC-18 (real node pack integration), Validated
  Behaviors section, normalize API port to 8199

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Dr.Lt.Data 2026-02-28 08:46:59 +09:00
parent 3d9c9ca8de
commit ca8698533d
5 changed files with 657 additions and 24 deletions

View File

@ -2,13 +2,48 @@
Procedures for setting up a ComfyUI environment with ComfyUI-Manager installed for functional testing.
## Automated Setup (Recommended)
Three shell scripts in `tests/e2e/scripts/` automate the entire lifecycle:
```bash
# 1. Setup: clone ComfyUI, create venv, install deps, symlink Manager
E2E_ROOT=/tmp/e2e_test MANAGER_ROOT=/path/to/comfyui-manager-draft4 \
bash tests/e2e/scripts/setup_e2e_env.sh
# 2. Start: launches ComfyUI in background, blocks until ready
E2E_ROOT=/tmp/e2e_test bash tests/e2e/scripts/start_comfyui.sh
# 3. Stop: graceful SIGTERM → SIGKILL shutdown
E2E_ROOT=/tmp/e2e_test bash tests/e2e/scripts/stop_comfyui.sh
# 4. Cleanup
rm -rf /tmp/e2e_test
```
### Script Details
| Script | Purpose | Input | Output |
|--------|---------|-------|--------|
| `setup_e2e_env.sh` | Full environment setup (8 steps) | `E2E_ROOT`, `MANAGER_ROOT`, `COMFYUI_BRANCH` (default: master), `PYTHON` (default: python3) | `E2E_ROOT=<path>` on last line |
| `start_comfyui.sh` | Foreground-blocking launcher | `E2E_ROOT`, `PORT` (default: 8199), `TIMEOUT` (default: 120s) | `COMFYUI_PID=<pid> PORT=<port>` |
| `stop_comfyui.sh` | Graceful shutdown | `E2E_ROOT`, `PORT` (default: 8199) | — |
**Idempotent**: `setup_e2e_env.sh` checks for a `.e2e_setup_complete` marker file and skips setup if the environment already exists.
**Blocking mechanism**: `start_comfyui.sh` uses `tail -n +1 -f | grep -q -m1 'To see the GUI'` to block until ComfyUI is ready. No polling loop needed.
---
## Prerequisites
- Python 3.9+
- Git
- `uv` (pip alternative, install via `pip install uv`)
- `uv` (install via `pip install uv` or [standalone](https://docs.astral.sh/uv/getting-started/installation/))
## Environment Setup
## Manual Setup (Reference)
For understanding or debugging, the manual steps are documented below. The automated scripts execute these same steps.
### 1. ComfyUI Clone
@ -50,17 +85,44 @@ uv pip install -e "$MANAGER_ROOT"
> **Note**: Editable mode (`-e`) reflects code changes without reinstalling.
> For production-like testing, use `uv pip install "$MANAGER_ROOT"` (non-editable).
### 5. ComfyUI Launch
### 5. Symlink Manager into custom_nodes
```bash
ln -s "$MANAGER_ROOT" "$COMFY_ROOT/custom_nodes/ComfyUI-Manager"
```
### 6. Write config.ini
```bash
mkdir -p "$COMFY_ROOT/user/__manager"
cat > "$COMFY_ROOT/user/__manager/config.ini" << 'EOF'
[default]
use_uv = true
use_unified_resolver = true
EOF
```
> **IMPORTANT**: The config path is `$COMFY_ROOT/user/__manager/config.ini`, resolved by `folder_paths.get_system_user_directory("manager")`. It is NOT inside the symlinked Manager directory.
### 7. HOME Isolation
```bash
export HOME=/tmp/e2e_home
mkdir -p "$HOME/.config" "$HOME/.local/share"
```
### 8. ComfyUI Launch
```bash
cd "$COMFY_ROOT"
python main.py --enable-manager --cpu
PYTHONUNBUFFERED=1 python main.py --enable-manager --cpu --port 8199
```
| Flag | Purpose |
|------|---------|
| `--enable-manager` | Enable ComfyUI-Manager (disabled by default) |
| `--cpu` | Run without GPU (for functional testing) |
| `--port 8199` | Use non-default port to avoid conflicts |
| `--enable-manager-legacy-ui` | Enable legacy UI (optional) |
| `--listen` | Allow remote connections (optional) |
@ -85,24 +147,48 @@ When Manager loads successfully, the following log lines appear:
[START] ComfyUI-Manager # manager_server.py loaded
```
The `Blocked by policy` message for Manager in custom_nodes is **expected**`should_be_disabled()` in `comfyui_manager/__init__.py` prevents legacy double-loading when Manager is already pip-installed.
---
## Caveats & Known Issues
### PYTHONPATH for `comfy` imports
ComfyUI's `comfy` package is a **local package** inside the ComfyUI directory — it is NOT pip-installed. Any code that imports from `comfy` (including `comfyui_manager.__init__`) requires `PYTHONPATH` to include the ComfyUI directory:
```bash
PYTHONPATH="$COMFY_ROOT" python -c "import comfy"
PYTHONPATH="$COMFY_ROOT" python -c "import comfyui_manager"
```
The automated scripts handle this via `PYTHONPATH` in verification checks and the ComfyUI process inherits it implicitly by running from the ComfyUI directory.
### config.ini path
The config file must be at `$COMFY_ROOT/user/__manager/config.ini`, **NOT** inside the Manager symlink directory. This is resolved by `folder_paths.get_system_user_directory("manager")` at `prestartup_script.py:65-73`.
### Manager v4 endpoint prefix
All Manager endpoints use the `/v2/` prefix (e.g., `/v2/manager/queue/status`, `/v2/snapshot/get_current`). Paths without the prefix will return 404.
### `Blocked by policy` is expected
When Manager detects that it's loaded as a custom_node but is already pip-installed, it prints `Blocked by policy` and skips legacy loading. This is intentional behavior in `comfyui_manager/__init__.py:39-51`.
### Bash `((var++))` trap
Under `set -e`, `((0++))` evaluates the pre-increment value (0), and `(( 0 ))` returns exit code 1, killing the script. Use `var=$((var + 1))` instead.
### `git+https://` URLs in requirements.txt
Some node packs (e.g., Impact Pack's SAM2 dependency) use `git+https://github.com/...` URLs. The unified resolver correctly rejects these with "rejected path separator" — they must be installed separately.
---
## Cleanup
```bash
deactivate
rm -rf "$COMFY_ROOT"
```
## Quick Reference (Copy-Paste)
```bash
# Full setup in one block
COMFY_ROOT=$(mktemp -d)/ComfyUI
MANAGER_ROOT=/path/to/comfyui-manager-draft4
git clone https://github.com/comfyanonymous/ComfyUI.git "$COMFY_ROOT"
cd "$COMFY_ROOT"
uv venv .venv && source .venv/bin/activate
uv pip install -r requirements.txt --extra-index-url https://download.pytorch.org/whl/cpu
uv pip install -e "$MANAGER_ROOT"
python main.py --enable-manager --cpu
```

View File

@ -21,10 +21,12 @@ use_unified_resolver = true
Node pack installation at runtime uses the task queue API:
```
POST http://localhost:8188/v2/manager/queue/task
POST http://localhost:8199/v2/manager/queue/task
Content-Type: application/json
```
> **Port**: E2E tests use port 8199 to avoid conflicts with running ComfyUI instances. Replace with your actual port if different.
**Payload** (`QueueTaskItem`):
| Field | Type | Description |
@ -186,7 +188,7 @@ chmod -R u+w "$(python -c 'import site; print(site.getsitepackages()[0])')"
**Steps**:
1. While ComfyUI is running, install a node pack that has both `install.py` and `requirements.txt` via API:
```bash
curl -X POST http://localhost:8188/v2/manager/queue/task \
curl -X POST http://localhost:8199/v2/manager/queue/task \
-H "Content-Type: application/json" \
-d '{
"ui_id": "test-installpy",
@ -334,7 +336,7 @@ private-pkg --index-url https://user:token123@pypi.private.com/simple
1. Start ComfyUI and confirm batch resolution succeeds
2. While ComfyUI is running, install a new node pack via API:
```bash
curl -X POST http://localhost:8188/v2/manager/queue/task \
curl -X POST http://localhost:8199/v2/manager/queue/task \
-H "Content-Type: application/json" \
-d '{
"ui_id": "test-defer-1",
@ -377,7 +379,7 @@ Verify both code locations that guard per-node pip install behave correctly in u
**Path 1 — Runtime API install (class method)**:
```bash
# While ComfyUI is running:
curl -X POST http://localhost:8188/v2/manager/queue/task \
curl -X POST http://localhost:8199/v2/manager/queue/task \
-H "Content-Type: application/json" \
-d '{
"ui_id": "test-path1",
@ -426,7 +428,7 @@ echo "['$COMFY_ROOT/custom_nodes/test_pack_lazy', '#LAZY-INSTALL-SCRIPT', '$PYTH
1. Set up conflicting deps (as in TC-4) and start ComfyUI (resolver fails, flag reset to `False`)
2. While still running, install a new node pack via API:
```bash
curl -X POST http://localhost:8188/v2/manager/queue/task \
curl -X POST http://localhost:8199/v2/manager/queue/task \
-H "Content-Type: application/json" \
-d '{
"ui_id": "test-postfallback",
@ -478,6 +480,101 @@ chmod u+r "$COMFY_ROOT/custom_nodes"
---
## TC-17: Restart Dependency Detection [P0]
**Precondition**: `use_unified_resolver = true`, automated E2E scripts available
This test verifies that the resolver correctly detects and installs dependencies for node packs added between restarts, incrementally building the dependency set.
**Steps**:
1. Boot ComfyUI with no custom node packs (Boot 1 — baseline)
2. Verify baseline deps only (Manager's own deps)
3. Stop ComfyUI
4. Clone `ComfyUI-Impact-Pack` into `custom_nodes/`
5. Restart ComfyUI (Boot 2)
6. Verify Impact Pack deps are installed (`cv2`, `skimage`, `dill`, `scipy`, `matplotlib`)
7. Stop ComfyUI
8. Clone `ComfyUI-Inspire-Pack` into `custom_nodes/`
9. Restart ComfyUI (Boot 3)
10. Verify Inspire Pack deps are installed (`cachetools`, `webcolors`)
**Expected log (each boot)**:
```
[UnifiedDepResolver] Collected N deps from M sources (skipped S)
[UnifiedDepResolver] running: ... uv pip compile ...
[UnifiedDepResolver] running: ... uv pip install ...
[UnifiedDepResolver] startup batch resolution succeeded
```
**Verify**:
- Boot 1: ~10 deps from ~10 sources; `cv2`, `dill`, `cachetools` are NOT installed
- Boot 2: ~19 deps from ~18 sources; `cv2`, `skimage`, `dill`, `scipy`, `matplotlib` all importable
- Boot 3: ~24 deps from ~21 sources; `cachetools`, `webcolors` also importable
- Both packs show as loaded in logs
**Automation**: Use `tests/e2e/scripts/` (setup → start → stop) with node pack cloning between boots.
---
## TC-18: Real Node Pack Integration [P0]
**Precondition**: `use_unified_resolver = true`, network access to GitHub + PyPI
Full pipeline test with real-world node packs (`ComfyUI-Impact-Pack` + `ComfyUI-Inspire-Pack`) to verify the resolver handles production requirements.txt files correctly.
**Steps**:
1. Set up E2E environment
2. Clone both Impact Pack and Inspire Pack into `custom_nodes/`
3. Direct-mode: instantiate `UnifiedDepResolver`, call `collect_requirements()` and `resolve_and_install()`
4. Boot-mode: start ComfyUI and verify via logs
**Expected behavior (direct mode)**:
```
--- Discovered node packs (3) --- # Manager, Impact, Inspire
ComfyUI-Impact-Pack
ComfyUI-Inspire-Pack
ComfyUI-Manager
--- Phase 1: Collect Requirements ---
Total requirements: ~24
Skipped: 1 # SAM2 git+https:// URL
Extra index URLs: set()
```
**Verify**:
- `git+https://github.com/facebookresearch/sam2.git` is correctly rejected with "rejected path separator"
- All other dependencies are collected and resolved
- After install, `cv2`, `PIL`, `scipy`, `skimage`, `matplotlib` are all importable
- No conflicting version errors during compile
**Automation**: Use `tests/e2e/scripts/` (setup → clone packs → start) with direct-mode resolver invocation.
---
## Validated Behaviors (from E2E Testing)
The following behaviors were confirmed during manual E2E testing:
### Resolver Pipeline
- **3-phase pipeline**: Collect → `uv pip compile``uv pip install` works end-to-end
- **Incremental detection**: Resolver discovers new node packs on each restart without reinstalling existing deps
- **Dependency deduplication**: Overlapping deps from multiple packs are resolved to compatible versions
### Security & Filtering
- **`git+https://` rejection**: URLs like `git+https://github.com/facebookresearch/sam2.git` are rejected with "rejected path separator" — SAM2 is the only dependency skipped from Impact Pack
- **Blacklist filtering**: `PackageRequirement` objects have `.name`, `.spec`, `.source` attributes; `collected.skipped` returns `[(spec_string, reason_string)]` tuples
### Manager Integration
- **Manager v4 endpoints**: All endpoints use `/v2/` prefix (e.g., `/v2/manager/queue/status`)
- **`Blocked by policy`**: Expected when Manager is pip-installed and also symlinked in `custom_nodes/`; prevents legacy double-loading
- **config.ini path**: Must be at `$COMFY_ROOT/user/__manager/config.ini`, not in the symlinked Manager dir
### Environment
- **PYTHONPATH requirement**: `comfy` is a local package (not pip-installed); `comfyui_manager` imports from `comfy`, so both require `PYTHONPATH=$COMFY_ROOT`
- **HOME isolation**: `HOME=$E2E_ROOT/home` prevents host config contamination during boot
---
## Summary
| TC | P | Scenario | Key Verification |
@ -498,6 +595,8 @@ chmod u+r "$COMFY_ROOT/custom_nodes"
| 14 | P0 | Both unified resolver paths | runtime API (class method) + startup lazy install |
| 15 | P1 | Post-fallback behavior | Per-node pip resumes in same process |
| 16 | P1 | Generic exception fallback | Distinct from uv-absent and batch-failed |
| 17 | P0 | Restart dependency detection | Incremental node pack discovery across restarts |
| 18 | P0 | Real node pack integration | Impact + Inspire Pack full pipeline |
### Traceability
@ -513,3 +612,6 @@ chmod u+r "$COMFY_ROOT/custom_nodes"
| Fallback behavior | TC-3, TC-4, TC-5, TC-15, TC-16 |
| Disabled node pack exclusion | TC-11 |
| Runtime defer behavior | TC-13, TC-14 |
| FR-8: Restart discovery | TC-17 |
| FR-9: Real-world compatibility | TC-17, TC-18 |
| FR-2: Input sanitization (git URLs) | TC-8, TC-18 |

View File

@ -0,0 +1,241 @@
#!/usr/bin/env bash
# setup_e2e_env.sh — Automated E2E environment setup for ComfyUI + Manager
#
# Creates an isolated ComfyUI installation with ComfyUI-Manager for E2E testing.
# Idempotent: skips setup if marker file and key artifacts already exist.
#
# Input env vars:
# E2E_ROOT — target directory (default: auto-generated via mktemp)
# MANAGER_ROOT — manager repo root (default: auto-detected from script location)
# COMFYUI_BRANCH — ComfyUI branch to clone (default: master)
# PYTHON — Python executable (default: python3)
#
# Output (last line of stdout):
# E2E_ROOT=/path/to/environment
#
# Exit: 0=success, 1=failure
set -euo pipefail
# --- Constants ---
COMFYUI_REPO="https://github.com/comfyanonymous/ComfyUI.git"
PYTORCH_CPU_INDEX="https://download.pytorch.org/whl/cpu"
CONFIG_INI_CONTENT="[default]
use_uv = true
use_unified_resolver = true
file_logging = false"
# --- Logging helpers ---
log() { echo "[setup_e2e] $*"; }
err() { echo "[setup_e2e] ERROR: $*" >&2; }
die() { err "$@"; exit 1; }
# --- Detect manager root by walking up from script dir to find pyproject.toml ---
detect_manager_root() {
local dir
dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
while [[ "$dir" != "/" ]]; do
if [[ -f "$dir/pyproject.toml" ]]; then
echo "$dir"
return 0
fi
dir="$(dirname "$dir")"
done
return 1
}
# --- Validate prerequisites ---
validate_prerequisites() {
local py="${PYTHON:-python3}"
local missing=()
command -v git >/dev/null 2>&1 || missing+=("git")
command -v uv >/dev/null 2>&1 || missing+=("uv")
command -v "$py" >/dev/null 2>&1 || missing+=("$py")
if [[ ${#missing[@]} -gt 0 ]]; then
die "Missing prerequisites: ${missing[*]}"
fi
# Verify Python version >= 3.9
local py_version
py_version=$("$py" -c "import sys; print(f'{sys.version_info.major}.{sys.version_info.minor}')")
local major minor
major="${py_version%%.*}"
minor="${py_version##*.}"
if [[ "$major" -lt 3 ]] || { [[ "$major" -eq 3 ]] && [[ "$minor" -lt 9 ]]; }; then
die "Python 3.9+ required, found $py_version"
fi
log "Prerequisites OK (python=$py_version, git=$(git --version | awk '{print $3}'), uv=$(uv --version 2>&1 | awk '{print $2}'))"
}
# --- Check idempotency: skip if already set up ---
check_already_setup() {
local root="$1"
if [[ -f "$root/.e2e_setup_complete" ]] \
&& [[ -d "$root/comfyui" ]] \
&& [[ -d "$root/venv" ]] \
&& [[ -f "$root/comfyui/user/__manager/config.ini" ]] \
&& [[ -L "$root/comfyui/custom_nodes/ComfyUI-Manager" ]]; then
log "Environment already set up at $root (marker file exists). Skipping."
echo "E2E_ROOT=$root"
exit 0
fi
}
# --- Verify the setup ---
verify_setup() {
local root="$1"
local manager_root="$2"
local venv_py="$root/venv/bin/python"
local errors=0
log "Running verification checks..."
# Check ComfyUI directory
if [[ ! -f "$root/comfyui/main.py" ]]; then
err "Verification FAIL: comfyui/main.py not found"
((errors++))
fi
# Check venv python
if [[ ! -x "$venv_py" ]]; then
err "Verification FAIL: venv python not executable"
((errors++))
fi
# Check symlink
local link_target="$root/comfyui/custom_nodes/ComfyUI-Manager"
if [[ ! -L "$link_target" ]]; then
err "Verification FAIL: symlink $link_target does not exist"
((errors++))
elif [[ "$(readlink -f "$link_target")" != "$(readlink -f "$manager_root")" ]]; then
err "Verification FAIL: symlink target mismatch"
((errors++))
fi
# Check config.ini
if [[ ! -f "$root/comfyui/user/__manager/config.ini" ]]; then
err "Verification FAIL: config.ini not found"
((errors++))
fi
# Check Python imports
# comfy is a local package inside ComfyUI (not pip-installed), and
# comfyui_manager.__init__ imports from comfy — both need PYTHONPATH
if ! PYTHONPATH="$root/comfyui" "$venv_py" -c "import comfy" 2>/dev/null; then
err "Verification FAIL: 'import comfy' failed"
((errors++))
fi
if ! PYTHONPATH="$root/comfyui" "$venv_py" -c "import comfyui_manager" 2>/dev/null; then
err "Verification FAIL: 'import comfyui_manager' failed"
((errors++))
fi
if [[ "$errors" -gt 0 ]]; then
die "Verification failed with $errors error(s)"
fi
log "Verification OK: all checks passed"
}
# ===== Main =====
# Resolve MANAGER_ROOT
if [[ -z "${MANAGER_ROOT:-}" ]]; then
MANAGER_ROOT="$(detect_manager_root)" || die "Cannot detect MANAGER_ROOT. Set it explicitly."
fi
MANAGER_ROOT="$(cd "$MANAGER_ROOT" && pwd)"
log "MANAGER_ROOT=$MANAGER_ROOT"
# Validate prerequisites
validate_prerequisites
PYTHON="${PYTHON:-python3}"
COMFYUI_BRANCH="${COMFYUI_BRANCH:-master}"
# Create or use E2E_ROOT
CREATED_BY_US=false
if [[ -z "${E2E_ROOT:-}" ]]; then
E2E_ROOT="$(mktemp -d -t e2e_comfyui_XXXXXX)"
CREATED_BY_US=true
log "Created E2E_ROOT=$E2E_ROOT"
else
mkdir -p "$E2E_ROOT"
log "Using E2E_ROOT=$E2E_ROOT"
fi
# Idempotency check
check_already_setup "$E2E_ROOT"
# Cleanup trap: remove E2E_ROOT on failure only if we created it
cleanup_on_failure() {
local exit_code=$?
if [[ "$exit_code" -ne 0 ]] && [[ "$CREATED_BY_US" == "true" ]]; then
err "Setup failed. Cleaning up $E2E_ROOT"
rm -rf "$E2E_ROOT"
fi
}
trap cleanup_on_failure EXIT
# Step 1: Clone ComfyUI
log "Step 1/8: Cloning ComfyUI (branch=$COMFYUI_BRANCH)..."
if [[ -d "$E2E_ROOT/comfyui/.git" ]]; then
log " ComfyUI already cloned, skipping"
else
git clone --depth=1 --branch "$COMFYUI_BRANCH" "$COMFYUI_REPO" "$E2E_ROOT/comfyui"
fi
# Step 2: Create virtual environment
log "Step 2/8: Creating virtual environment..."
if [[ -d "$E2E_ROOT/venv" ]]; then
log " venv already exists, skipping"
else
uv venv "$E2E_ROOT/venv"
fi
VENV_PY="$E2E_ROOT/venv/bin/python"
# Step 3: Install ComfyUI dependencies
log "Step 3/8: Installing ComfyUI dependencies (CPU-only)..."
uv pip install \
--python "$VENV_PY" \
-r "$E2E_ROOT/comfyui/requirements.txt" \
--extra-index-url "$PYTORCH_CPU_INDEX"
# Step 4: Install ComfyUI-Manager (non-editable, production-like)
log "Step 4/8: Installing ComfyUI-Manager..."
uv pip install --python "$VENV_PY" "$MANAGER_ROOT"
# Step 5: Create symlink for custom_nodes discovery
log "Step 5/8: Creating custom_nodes symlink..."
mkdir -p "$E2E_ROOT/comfyui/custom_nodes"
local_link="$E2E_ROOT/comfyui/custom_nodes/ComfyUI-Manager"
if [[ -L "$local_link" ]]; then
log " Symlink already exists, updating"
rm -f "$local_link"
fi
ln -s "$MANAGER_ROOT" "$local_link"
# Step 6: Write config.ini to correct path
log "Step 6/8: Writing config.ini..."
mkdir -p "$E2E_ROOT/comfyui/user/__manager"
echo "$CONFIG_INI_CONTENT" > "$E2E_ROOT/comfyui/user/__manager/config.ini"
# Step 7: Create HOME isolation directories
log "Step 7/8: Creating HOME isolation directories..."
mkdir -p "$E2E_ROOT/home/.config"
mkdir -p "$E2E_ROOT/home/.local/share"
mkdir -p "$E2E_ROOT/logs"
# Step 8: Verify setup
log "Step 8/8: Verifying setup..."
verify_setup "$E2E_ROOT" "$MANAGER_ROOT"
# Write marker file
date -Iseconds > "$E2E_ROOT/.e2e_setup_complete"
# Clear the EXIT trap since setup succeeded
trap - EXIT
log "Setup complete."
echo "E2E_ROOT=$E2E_ROOT"

View File

@ -0,0 +1,129 @@
#!/usr/bin/env bash
# start_comfyui.sh — Foreground-blocking ComfyUI launcher for E2E tests
#
# Starts ComfyUI in the background, then blocks the foreground until the server
# is ready (or timeout). This makes it safe to call from subprocess.run() or
# Claude's Bash tool — the call returns only when ComfyUI is accepting requests.
#
# Input env vars:
# E2E_ROOT — (required) path to E2E environment from setup_e2e_env.sh
# PORT — ComfyUI listen port (default: 8199)
# TIMEOUT — max seconds to wait for readiness (default: 120)
#
# Output (last line on success):
# COMFYUI_PID=<pid> PORT=<port>
#
# Exit: 0=ready, 1=timeout/failure
set -euo pipefail
# --- Defaults ---
PORT="${PORT:-8199}"
TIMEOUT="${TIMEOUT:-120}"
# --- Logging helpers ---
log() { echo "[start_comfyui] $*"; }
err() { echo "[start_comfyui] ERROR: $*" >&2; }
die() { err "$@"; exit 1; }
# --- Validate environment ---
[[ -n "${E2E_ROOT:-}" ]] || die "E2E_ROOT is not set"
[[ -d "$E2E_ROOT/comfyui" ]] || die "ComfyUI not found at $E2E_ROOT/comfyui"
[[ -x "$E2E_ROOT/venv/bin/python" ]] || die "venv python not found at $E2E_ROOT/venv/bin/python"
[[ -f "$E2E_ROOT/.e2e_setup_complete" ]] || die "Setup marker not found. Run setup_e2e_env.sh first."
PY="$E2E_ROOT/venv/bin/python"
COMFY_DIR="$E2E_ROOT/comfyui"
LOG_DIR="$E2E_ROOT/logs"
LOG_FILE="$LOG_DIR/comfyui.log"
PID_FILE="$LOG_DIR/comfyui.pid"
mkdir -p "$LOG_DIR"
# --- Check/clear port ---
if ss -tlnp 2>/dev/null | grep -q ":${PORT}\b"; then
log "Port $PORT is in use. Attempting to stop existing process..."
# Try to read existing PID file
if [[ -f "$PID_FILE" ]]; then
OLD_PID="$(cat "$PID_FILE")"
if kill -0 "$OLD_PID" 2>/dev/null; then
kill "$OLD_PID" 2>/dev/null || true
sleep 2
fi
fi
# Fallback: kill by port pattern
if ss -tlnp 2>/dev/null | grep -q ":${PORT}\b"; then
pkill -f "main\\.py.*--port $PORT" 2>/dev/null || true
sleep 2
fi
# Final check
if ss -tlnp 2>/dev/null | grep -q ":${PORT}\b"; then
die "Port $PORT is still in use after cleanup attempt"
fi
log "Port $PORT cleared."
fi
# --- Start ComfyUI ---
log "Starting ComfyUI on port $PORT..."
# Create empty log file (ensures tail -f works from the start)
: > "$LOG_FILE"
# Launch with unbuffered Python output so log lines appear immediately
PYTHONUNBUFFERED=1 \
HOME="$E2E_ROOT/home" \
nohup "$PY" "$COMFY_DIR/main.py" \
--cpu \
--enable-manager \
--port "$PORT" \
> "$LOG_FILE" 2>&1 &
COMFYUI_PID=$!
echo "$COMFYUI_PID" > "$PID_FILE"
log "ComfyUI PID=$COMFYUI_PID, log=$LOG_FILE"
# Verify process didn't crash immediately
sleep 1
if ! kill -0 "$COMFYUI_PID" 2>/dev/null; then
err "ComfyUI process died immediately. Last 30 lines of log:"
tail -n 30 "$LOG_FILE" >&2
rm -f "$PID_FILE"
exit 1
fi
# --- Block until ready ---
# tail -n +1 -f: read from file start AND follow new content (no race condition)
# grep -q -m1: exit on first match → tail gets SIGPIPE → pipeline ends
# timeout: kill the pipeline after TIMEOUT seconds
log "Waiting up to ${TIMEOUT}s for ComfyUI to become ready..."
if timeout "$TIMEOUT" bash -c \
"tail -n +1 -f '$LOG_FILE' 2>/dev/null | grep -q -m1 'To see the GUI'"; then
log "ComfyUI startup message detected."
else
err "Timeout (${TIMEOUT}s) waiting for ComfyUI. Last 30 lines of log:"
tail -n 30 "$LOG_FILE" >&2
kill "$COMFYUI_PID" 2>/dev/null || true
rm -f "$PID_FILE"
exit 1
fi
# Verify process is still alive after readiness detected
if ! kill -0 "$COMFYUI_PID" 2>/dev/null; then
err "ComfyUI process died after readiness signal. Last 30 lines:"
tail -n 30 "$LOG_FILE" >&2
rm -f "$PID_FILE"
exit 1
fi
# Optional HTTP health check
if command -v curl >/dev/null 2>&1; then
if curl -sf "http://127.0.0.1:${PORT}/system_stats" >/dev/null 2>&1; then
log "HTTP health check passed (/system_stats)"
else
log "HTTP health check skipped (endpoint not yet available, but startup message detected)"
fi
fi
log "ComfyUI is ready."
echo "COMFYUI_PID=$COMFYUI_PID PORT=$PORT"

View File

@ -0,0 +1,75 @@
#!/usr/bin/env bash
# stop_comfyui.sh — Graceful ComfyUI shutdown for E2E tests
#
# Stops a ComfyUI process previously started by start_comfyui.sh.
# Uses SIGTERM first, then SIGKILL after a grace period.
#
# Input env vars:
# E2E_ROOT — (required) path to E2E environment
# PORT — ComfyUI port for fallback pkill (default: 8199)
#
# Exit: 0=stopped, 1=failed
set -euo pipefail
PORT="${PORT:-8199}"
GRACE_PERIOD=10
# --- Logging helpers ---
log() { echo "[stop_comfyui] $*"; }
err() { echo "[stop_comfyui] ERROR: $*" >&2; }
die() { err "$@"; exit 1; }
# --- Validate ---
[[ -n "${E2E_ROOT:-}" ]] || die "E2E_ROOT is not set"
PID_FILE="$E2E_ROOT/logs/comfyui.pid"
# --- Read PID ---
COMFYUI_PID=""
if [[ -f "$PID_FILE" ]]; then
COMFYUI_PID="$(cat "$PID_FILE")"
log "Read PID=$COMFYUI_PID from $PID_FILE"
fi
# --- Graceful shutdown via SIGTERM ---
if [[ -n "$COMFYUI_PID" ]] && kill -0 "$COMFYUI_PID" 2>/dev/null; then
log "Sending SIGTERM to PID $COMFYUI_PID..."
kill "$COMFYUI_PID" 2>/dev/null || true
# Wait for graceful shutdown
elapsed=0
while kill -0 "$COMFYUI_PID" 2>/dev/null && [[ "$elapsed" -lt "$GRACE_PERIOD" ]]; do
sleep 1
elapsed=$((elapsed + 1))
done
# Force kill if still alive
if kill -0 "$COMFYUI_PID" 2>/dev/null; then
log "Process still alive after ${GRACE_PERIOD}s. Sending SIGKILL..."
kill -9 "$COMFYUI_PID" 2>/dev/null || true
sleep 1
fi
fi
# --- Fallback: kill by port pattern ---
if ss -tlnp 2>/dev/null | grep -q ":${PORT}\b"; then
log "Port $PORT still in use. Attempting pkill fallback..."
pkill -f "main\\.py.*--port $PORT" 2>/dev/null || true
sleep 2
if ss -tlnp 2>/dev/null | grep -q ":${PORT}\b"; then
pkill -9 -f "main\\.py.*--port $PORT" 2>/dev/null || true
sleep 1
fi
fi
# --- Cleanup PID file ---
rm -f "$PID_FILE"
# --- Verify port is free ---
if ss -tlnp 2>/dev/null | grep -q ":${PORT}\b"; then
die "Port $PORT is still in use after shutdown"
fi
log "ComfyUI stopped."