wangbo/ComfyUI-Manager
1
0
Fork 0
mirror of https://github.com/Comfy-Org/ComfyUI-Manager.git synced 2026-07-27 02:27:30 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

feat(security): add dedicated install flags decoupled from security_level

Browse Source 
Gate 'install via git URL' and 'install via pip' with dedicated opt-in
boolean flags (allow_git_url_install / allow_pip_install) in config.ini
[default], fully replacing the security_level term on those surfaces
(REPLACE, not AND — a strict level no longer denies when the flag is on;
a weak level no longer allows when the flag is off).

- glob/manager_server.py: pure predicate is_dedicated_install_allowed
  (flag AND loopback, request-time args.listen); REPLACE gates at
  /customnode/install/git_url and /customnode/install/pip; batch
  unknown-URL arm routes through the same full predicate at the risky
  position (loopback term is load-bearing — the middle entry gate has
  no network-position term; the entry gate itself stays in force);
  unknown-pip in batch stays unconditionally blocked; new
  SECURITY_MESSAGE_FLAG_* denial constants name the responsible flag;
  security_403_response gains flag_token (comfyui_outdated keeps precedence)
- glob/manager_core.py: register both keys (read via get_bool default-false,
  write list, exception fallback); "true"-only truthy; restart-only activation
- js/common.js: 403 dialog copy names the responsible flag at the two
  install call sites
- README.md: security-policy docs for both flags (per-surface scope incl.
  the batch entry-gate qualifier, REPLACE decoupling, loopback bound,
  opt-in config snippet, default-deny + migration note); stale tier lists
  corrected against the actual gates
- CHANGELOG.md: opt-in migration note + accepted residual risk (flags
  bypass the forced-strong outdated-ComfyUI hardening on loopback,
  opt-in only), decoupling claim qualified for the batch entry gate

Tests: unit suite (predicate truth table, REPLACE litmus both directions,
AST binding-proofs against live handlers, subprocess-isolated config
contract) plus a real-server E2E suite that mounts the Manager-under-test
via git worktree (exact-SHA pin, detached) against a real ComfyUI and
exercises both flag surfaces and both arms — deny arms (403 + flag-naming
body/log + no install artifact), git-URL allow arm (real clone), pip allow
arm as a two-phase reservation oracle — with zero-residual self-clean.
Module skips without E2E_COMFYUI_ROOT; unit suite unaffected.

The manager-v4 branch ships the identical policy (shared invariants +
config contract); this tree uses the degraded predicate 'flag AND
loopback' (no personal_cloud-equivalent mode here).
This commit is contained in:
Dr.Lt.Data
2026-06-15 02:44:26 +09:00
parent 3772432847
commit 6288fb0e2a
25 changed files with 9998 additions and 8815 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/conftest.py
+41
View File
@@ -0,0 +1,41 @@
"""Test-runner guard for the GOAL #32 tests/ modules.
WHY THIS FILE EXISTS (collection hazard, not test logic):
The repo root contains ``__init__.py`` — the ComfyUI plugin entrypoint —
which at import time appends ``glob/`` to sys.path and imports
``manager_server`` (which needs ``folder_paths`` / ``comfy.cli_args`` /
a constructed ``PromptServer``). pytest 8 collects any ancestor
directory that carries an ``__init__.py`` as a ``Package`` node and
IMPORTS that ``__init__.py`` during test setup (observed module name:
``__init__``). Outside a live ComfyUI process that import can never
succeed, so EVERY test under tests/ errors at setup — including the
pre-existing tests/test_csrf_content_type_helper.py — whenever pytest's
rootdir ends up at or above the repo root (e.g. running inside a git
worktree nested under the parent checkout).
The guard below pre-seeds ``sys.modules`` with an inert stub whose
``__file__`` matches the real path, so pytest's
``import_path(<repo-root>/__init__.py)`` resolves to the stub without
executing the plugin entrypoint. Conftest files load before the setup
phase, so the stub is always in place in time. This does NOT touch
production code and does NOT alter what the tests import themselves
(they use AST-extraction / subprocess isolation per the
tests/test_csrf_content_type_helper.py precedent — ``glob/`` is never
added to the runner's sys.path).
"""
import sys
import types
from pathlib import Path
_REPO_ROOT = Path(__file__).resolve().parent.parent
_ROOT_INIT = _REPO_ROOT / "__init__.py"
if _ROOT_INIT.exists() and "__init__" not in sys.modules:
_stub = types.ModuleType("__init__")
_stub.__file__ = str(_ROOT_INIT)
_stub.__doc__ = (
"Inert stand-in for the ComfyUI-Manager plugin entrypoint; "
"see tests/conftest.py for rationale."
)
sys.modules["__init__"] = _stub
tests/e2e/scripts/setup_e2e_env.sh
Executable
+169
View File
@@ -0,0 +1,169 @@
#!/usr/bin/env bash
# setup_e2e_env.sh — E2E environment builder for GOAL #60 (T1, spec §2).
#
# Builds the DISPOSABLE test ComfyUI root used by
# tests/e2e/test_e2e_install_flags.py. ENV BUILD ONLY: donor steps 4-5
# (editable pip install of the Manager + custom_nodes symlink) are
# deliberately DROPPED — the Manager is mounted via `git worktree add`
# by the `mount_worktree` session fixture in the test module, which is
# the SOLE owner of mount create/reuse/teardown (spec §2 T1
# single-ownership rule). This script never touches the Manager repo.
#
# Idempotent: re-run is a no-op when the marker + key artifacts exist
# (E2E-SC-01).
#
# Input env vars:
# E2E_COMFYUI_ROOT — target directory (default: mktemp -d)
# COMFYUI_BRANCH — ComfyUI clone ref (default: master; Q-2)
# PYTHON — python executable for version probe (default: python3)
#
# Output (last line of stdout):
# E2E_COMFYUI_ROOT=/path/to/environment
#
# Exit: 0=success, 1=failure
set -euo pipefail
COMFYUI_REPO="https://github.com/comfyanonymous/ComfyUI.git"
PYTORCH_CPU_INDEX="https://download.pytorch.org/whl/cpu"
# Minimal seed config. use_uv=false: the venv is seeded with pip
# (`uv venv --seed`), so the Manager's make_pip_cmd resolves to
# `<venv-python> -m pip` and the suite's pip-uninstall hygiene helpers
# work without uv on PATH at server runtime. The install flags are NOT
# seeded here — stage_flags.sh stages them per launch identity (SC-06).
CONFIG_INI_CONTENT="[default]
file_logging = false
use_uv = false"
log() { echo "[setup_e2e] $*"; }
err() { echo "[setup_e2e] ERROR: $*" >&2; }
die() { err "$@"; exit 1; }
validate_prerequisites() {
local py="${PYTHON:-python3}"
local missing=()
command -v git >/dev/null 2>&1 || missing+=("git")
command -v uv >/dev/null 2>&1 || missing+=("uv")
command -v "$py" >/dev/null 2>&1 || missing+=("$py")
if [[ ${#missing[@]} -gt 0 ]]; then
die "Missing prerequisites: ${missing[*]}"
fi
local py_version major minor
py_version=$("$py" -c "import sys; print(f'{sys.version_info.major}.{sys.version_info.minor}')")
major="${py_version%%.*}"
minor="${py_version##*.}"
if [[ "$major" -lt 3 ]] || { [[ "$major" -eq 3 ]] && [[ "$minor" -lt 9 ]]; }; then
die "Python 3.9+ required, found $py_version"
fi
log "Prerequisites OK (python=$py_version)"
}
check_already_setup() {
local root="$1"
if [[ -f "$root/.e2e_setup_complete" ]] \
&& [[ -d "$root/comfyui" ]] \
&& [[ -d "$root/venv" ]] \
&& [[ -f "$root/comfyui/user/__manager/config.ini" ]]; then
log "Environment already set up at $root (marker exists). Skipping. (E2E-SC-01 idempotence)"
echo "E2E_COMFYUI_ROOT=$root"
exit 0
fi
}
verify_setup() {
local root="$1"
local venv_py="$root/venv/bin/python"
local errors=0
log "Running verification checks..."
[[ -f "$root/comfyui/main.py" ]] || { err "Verification FAIL: comfyui/main.py not found"; ((errors++)); }
[[ -x "$venv_py" ]] || { err "Verification FAIL: venv python not executable"; ((errors++)); }
[[ -f "$root/comfyui/user/__manager/config.ini" ]] || { err "Verification FAIL: config.ini not found"; ((errors++)); }
# venv must carry pip (uv venv --seed) — the suite's hygiene helpers
# and the Manager's reservation-consuming boot both call `-m pip`.
if ! "$venv_py" -m pip --version >/dev/null 2>&1; then
err "Verification FAIL: venv pip not available"
((errors++))
fi
# comfy is a local package inside the ComfyUI checkout
if ! PYTHONPATH="$root/comfyui" "$venv_py" -c "import comfy" 2>/dev/null; then
err "Verification FAIL: 'import comfy' failed"
((errors++))
fi
# [D2] half-check at setup time: the Manager must NOT be pip-installed
# into this venv (the worktree mount is the only delivery mechanism).
if "$venv_py" -m pip show comfyui-manager >/dev/null 2>&1 \
|| "$venv_py" -m pip show ComfyUI-Manager >/dev/null 2>&1; then
err "Verification FAIL: a pip-installed Manager exists in the venv (wrong layout for [D2])"
((errors++))
fi
if [[ "$errors" -gt 0 ]]; then
die "Verification failed with $errors error(s)"
fi
log "Verification OK: all checks passed"
}
# ===== Main =====
validate_prerequisites
PYTHON="${PYTHON:-python3}"
COMFYUI_BRANCH="${COMFYUI_BRANCH:-master}"
CREATED_BY_US=false
if [[ -z "${E2E_COMFYUI_ROOT:-}" ]]; then
E2E_COMFYUI_ROOT="$(mktemp -d -t e2e_comfyui_XXXXXX)"
CREATED_BY_US=true
log "Created E2E_COMFYUI_ROOT=$E2E_COMFYUI_ROOT"
else
mkdir -p "$E2E_COMFYUI_ROOT"
log "Using E2E_COMFYUI_ROOT=$E2E_COMFYUI_ROOT"
fi
check_already_setup "$E2E_COMFYUI_ROOT"
cleanup_on_failure() {
local exit_code=$?
if [[ "$exit_code" -ne 0 ]] && [[ "$CREATED_BY_US" == "true" ]]; then
err "Setup failed. Cleaning up $E2E_COMFYUI_ROOT"
rm -rf "$E2E_COMFYUI_ROOT"
fi
}
trap cleanup_on_failure EXIT
log "Step 1/5: Cloning ComfyUI (branch=$COMFYUI_BRANCH)..."
if [[ -d "$E2E_COMFYUI_ROOT/comfyui/.git" ]]; then
log " ComfyUI already cloned, skipping"
else
git clone --depth=1 --branch "$COMFYUI_BRANCH" "$COMFYUI_REPO" "$E2E_COMFYUI_ROOT/comfyui"
fi
log "Step 2/5: Creating virtual environment (seeded with pip)..."
if [[ -d "$E2E_COMFYUI_ROOT/venv" ]]; then
log " venv already exists, skipping"
else
uv venv --seed "$E2E_COMFYUI_ROOT/venv"
fi
VENV_PY="$E2E_COMFYUI_ROOT/venv/bin/python"
log "Step 3/5: Installing ComfyUI dependencies (CPU-only torch index)..."
uv pip install \
--python "$VENV_PY" \
-r "$E2E_COMFYUI_ROOT/comfyui/requirements.txt" \
--extra-index-url "$PYTORCH_CPU_INDEX"
log "Step 4/5: Writing seed config.ini + HOME isolation dirs..."
mkdir -p "$E2E_COMFYUI_ROOT/comfyui/user/__manager"
echo "$CONFIG_INI_CONTENT" > "$E2E_COMFYUI_ROOT/comfyui/user/__manager/config.ini"
mkdir -p "$E2E_COMFYUI_ROOT/home/.config"
mkdir -p "$E2E_COMFYUI_ROOT/home/.local/share"
mkdir -p "$E2E_COMFYUI_ROOT/logs"
mkdir -p "$E2E_COMFYUI_ROOT/comfyui/custom_nodes"
log "Step 5/5: Verifying setup..."
verify_setup "$E2E_COMFYUI_ROOT"
# Marker written ONLY after verification passes (E2E-SC-01)
date -Iseconds > "$E2E_COMFYUI_ROOT/.e2e_setup_complete"
trap - EXIT
log "Setup complete."
echo "E2E_COMFYUI_ROOT=$E2E_COMFYUI_ROOT"
tests/e2e/scripts/stage_flags.sh
Executable
+78
View File
@@ -0,0 +1,78 @@
#!/usr/bin/env bash
# stage_flags.sh — Per-launch-identity config staging (T4, spec §1.4).
#
# Analog of the donor's start_comfyui_strict.sh config patching, split
# out as a pure staging script (launch is a separate step; the pytest
# fixture owns restore+delete of the backup at teardown — donor
# symmetry, spec §1.4).
#
# Modes (arg $1):
# deny — REMOVE both flag keys (L-D: flags ABSENT also live-proves
# "missing key reads false")
# allow — set allow_git_url_install = true AND allow_pip_install = true
# (L-A / L-P)
#
# Backup: config.ini.before-flags, created ONLY if not already present
# (crashed-run-safe — preserves the true baseline across crashed runs).
# Restore + DELETE of the backup happens in the pytest fixture teardown,
# NOT here (E2E-SC-06/07).
#
# Input env vars:
# E2E_COMFYUI_ROOT — (required)
#
# Exit: 0=staged, 1=failure
set -euo pipefail
MODE="${1:-}"
log() { echo "[stage_flags] $*"; }
err() { echo "[stage_flags] ERROR: $*" >&2; }
die() { err "$@"; exit 1; }
[[ -n "${E2E_COMFYUI_ROOT:-}" ]] || die "E2E_COMFYUI_ROOT is not set"
[[ "$MODE" == "deny" || "$MODE" == "allow" ]] || die "usage: stage_flags.sh deny|allow"
CONFIG="$E2E_COMFYUI_ROOT/comfyui/user/__manager/config.ini"
BACKUP="$CONFIG.before-flags"
[[ -f "$CONFIG" ]] || die "config not found at $CONFIG (run setup_e2e_env.sh first)"
# Backup ONLY if absent (crashed-run-safe; SC-06)
if [[ ! -f "$BACKUP" ]]; then
cp "$CONFIG" "$BACKUP"
log "Backed up original config to $BACKUP"
else
log "Backup already present at $BACKUP (preserving original baseline)"
fi
stage_key() {
local key="$1" value="$2"
if grep -qE "^${key}\s*=" "$CONFIG"; then
sed -i -E "s|^${key}\s*=.*|${key} = ${value}|" "$CONFIG"
else
sed -i -E "/^\[default\]/a ${key} = ${value}" "$CONFIG"
fi
}
remove_key() {
local key="$1"
sed -i -E "/^${key}\s*=/d" "$CONFIG"
}
case "$MODE" in
deny)
remove_key "allow_git_url_install"
remove_key "allow_pip_install"
log "Staged DENY config (both flags ABSENT — missing key reads false)"
;;
allow)
stage_key "allow_git_url_install" "true"
stage_key "allow_pip_install" "true"
log "Staged ALLOW config (both flags = true)"
;;
esac
# The staged value takes effect ONLY on the NEXT launch (restart-only
# cached_config — by construction, no hot-reload assumption; SC-06).
log "Staged config at $CONFIG (effective on next launch)"
tests/e2e/scripts/start_comfyui.sh
Executable
+131
View File
@@ -0,0 +1,131 @@
#!/usr/bin/env bash
# start_comfyui.sh — Foreground-blocking ComfyUI launcher (T2, spec §1.3).
#
# Starts ComfyUI in the background and blocks until the server answers
# GET /system_stats (or timeout). Deltas vs the donor script (binding,
# spec §1.3):
# - NO --enable-manager: the Manager is a custom-node-style plugin
# activated by ComfyUI's custom_nodes scan of the worktree mount.
# - NO COMFYUI_MANAGER_SKIP_MANAGER_REQUIREMENTS: that belt does not
# exist in this repo (Q-6 verified); watchdog row E2E-SC-42 covers
# the residual.
# - --listen is ALWAYS passed explicitly (LISTEN env): the predicate
# under test is request-time `flag AND is_loopback(args.listen)` —
# the listener value is LOAD-BEARING.
# - Per-launch log isolation (E2E-SC-04, MANDATORY): each launch
# identity writes a FRESH log file comfyui.<port>.<launch-id>.log,
# so a deny-copy substring from L-D is unfindable by L-A/R-A log
# assertions (stale-substring false-PASS class).
# - Readiness = poll GET /system_stats == 200; child exit code 0
# during the wait = Manager-triggered restart -> KEEP polling;
# non-zero exit -> fail fast with log tail.
#
# Input env vars:
# E2E_COMFYUI_ROOT — (required) root from setup_e2e_env.sh
# PORT — listen port (default: 8189)
# TIMEOUT — max seconds to readiness (default: 120)
# LISTEN — listener address (default: 127.0.0.1)
# LAUNCH_ID — launch identity tag for the log file (default: default)
#
# Output (last line on success):
# COMFYUI_PID=<pid> PORT=<port> LOG_FILE=<path>
#
# Exit: 0=ready, 1=timeout/failure
set -euo pipefail
PORT="${PORT:-8189}"
TIMEOUT="${TIMEOUT:-120}"
LISTEN="${LISTEN:-127.0.0.1}"
LAUNCH_ID="${LAUNCH_ID:-default}"
log() { echo "[start_comfyui] $*"; }
err() { echo "[start_comfyui] ERROR: $*" >&2; }
die() { err "$@"; exit 1; }
[[ -n "${E2E_COMFYUI_ROOT:-}" ]] || die "E2E_COMFYUI_ROOT is not set"
[[ -d "$E2E_COMFYUI_ROOT/comfyui" ]] || die "ComfyUI not found at $E2E_COMFYUI_ROOT/comfyui"
[[ -x "$E2E_COMFYUI_ROOT/venv/bin/python" ]] || die "venv python not found"
[[ -f "$E2E_COMFYUI_ROOT/.e2e_setup_complete" ]] || die "Setup marker not found. Run setup_e2e_env.sh first."
PY="$E2E_COMFYUI_ROOT/venv/bin/python"
COMFY_DIR="$E2E_COMFYUI_ROOT/comfyui"
LOG_DIR="$E2E_COMFYUI_ROOT/logs"
LOG_FILE="$LOG_DIR/comfyui.${PORT}.${LAUNCH_ID}.log"
# Port-namespaced PID file (donor WI-CC incident: shared PID file caused
# a cross-port kill).
PID_FILE="$LOG_DIR/comfyui.${PORT}.pid"
mkdir -p "$LOG_DIR"
# --- Pre-launch port clear ---
if ss -tlnp 2>/dev/null | grep -q ":${PORT}\b"; then
log "Port $PORT is in use. Attempting to stop existing process..."
if [[ -f "$PID_FILE" ]]; then
OLD_PID="$(cat "$PID_FILE")"
if kill -0 "$OLD_PID" 2>/dev/null; then
kill "$OLD_PID" 2>/dev/null || true
sleep 2
fi
fi
if ss -tlnp 2>/dev/null | grep -q ":${PORT}\b"; then
pkill -f "main\\.py.*--port $PORT" 2>/dev/null || true
sleep 2
fi
if ss -tlnp 2>/dev/null | grep -q ":${PORT}\b"; then
die "Port $PORT is still in use after cleanup attempt"
fi
log "Port $PORT cleared."
fi
# --- Launch (FRESH per-launch log file) ---
log "Starting ComfyUI on port $PORT (listen=$LISTEN, launch_id=$LAUNCH_ID)..."
: > "$LOG_FILE"
PYTHONUNBUFFERED=1 \
HOME="$E2E_COMFYUI_ROOT/home" \
nohup "$PY" -u "$COMFY_DIR/main.py" \
--cpu \
--port "$PORT" \
--listen "$LISTEN" \
> "$LOG_FILE" 2>&1 &
COMFYUI_PID=$!
echo "$COMFYUI_PID" > "$PID_FILE"
log "ComfyUI PID=$COMFYUI_PID, log=$LOG_FILE"
# --- Block until ready: poll /system_stats (restart-tolerant) ---
log "Waiting up to ${TIMEOUT}s for ComfyUI readiness (GET /system_stats)..."
DEADLINE=$(( $(date +%s) + TIMEOUT ))
READY=0
while [[ "$(date +%s)" -lt "$DEADLINE" ]]; do
if curl -sf --max-time 2 "http://127.0.0.1:${PORT}/system_stats" >/dev/null 2>&1; then
READY=1
break
fi
# Child exit handling: exit 0 = Manager-triggered restart -> keep
# polling (a restarted process will bind the port); non-zero -> fail fast.
if ! kill -0 "$COMFYUI_PID" 2>/dev/null; then
if wait "$COMFYUI_PID" 2>/dev/null; then
: # exit 0 — restart class, keep polling
else
RC=$?
err "ComfyUI exited with code $RC. Last 30 lines of log:"
tail -n 30 "$LOG_FILE" >&2
rm -f "$PID_FILE"
exit 1
fi
fi
sleep 1
done
if [[ "$READY" -ne 1 ]]; then
err "Timeout (${TIMEOUT}s) waiting for ComfyUI. Last 30 lines of log:"
tail -n 30 "$LOG_FILE" >&2
kill "$COMFYUI_PID" 2>/dev/null || true
rm -f "$PID_FILE"
exit 1
fi
log "ComfyUI is ready."
echo "COMFYUI_PID=$COMFYUI_PID PORT=$PORT LOG_FILE=$LOG_FILE"
tests/e2e/scripts/stop_comfyui.sh
Executable
+70
View File
@@ -0,0 +1,70 @@
#!/usr/bin/env bash
# stop_comfyui.sh — Graceful ComfyUI shutdown (T3, spec §1.3 stop contract).
#
# Donor mirror: SIGTERM -> 10s grace -> SIGKILL -> port-pattern pkill
# fallback -> port-free verify (incl. the legacy-PID-file warning from
# the donor WI-CC cross-port-kill incident).
#
# Input env vars:
# E2E_COMFYUI_ROOT — (required) path to the E2E environment
# PORT — ComfyUI port (default: 8189)
#
# Exit: 0=stopped, 1=failed
set -euo pipefail
PORT="${PORT:-8189}"
GRACE_PERIOD=10
log() { echo "[stop_comfyui] $*"; }
err() { echo "[stop_comfyui] ERROR: $*" >&2; }
die() { err "$@"; exit 1; }
[[ -n "${E2E_COMFYUI_ROOT:-}" ]] || die "E2E_COMFYUI_ROOT is not set"
PID_FILE="$E2E_COMFYUI_ROOT/logs/comfyui.${PORT}.pid"
LEGACY_PID_FILE="$E2E_COMFYUI_ROOT/logs/comfyui.pid"
if [[ -f "$LEGACY_PID_FILE" ]] && [[ ! -f "$PID_FILE" ]]; then
log "WARN: found legacy unported PID file $LEGACY_PID_FILE but no ${PID_FILE}. Cross-port risk — ignoring legacy file."
fi
COMFYUI_PID=""
if [[ -f "$PID_FILE" ]]; then
COMFYUI_PID="$(cat "$PID_FILE")"
log "Read PID=$COMFYUI_PID from $PID_FILE"
fi
if [[ -n "$COMFYUI_PID" ]] && kill -0 "$COMFYUI_PID" 2>/dev/null; then
log "Sending SIGTERM to PID $COMFYUI_PID..."
kill "$COMFYUI_PID" 2>/dev/null || true
elapsed=0
while kill -0 "$COMFYUI_PID" 2>/dev/null && [[ "$elapsed" -lt "$GRACE_PERIOD" ]]; do
sleep 1
elapsed=$((elapsed + 1))
done
if kill -0 "$COMFYUI_PID" 2>/dev/null; then
log "Process still alive after ${GRACE_PERIOD}s. Sending SIGKILL..."
kill -9 "$COMFYUI_PID" 2>/dev/null || true
sleep 1
fi
fi
# Fallback: kill by port pattern (covers Manager-restarted processes whose
# PID differs from the recorded one).
if ss -tlnp 2>/dev/null | grep -q ":${PORT}\b"; then
log "Port $PORT still in use. Attempting pkill fallback..."
pkill -f "main\\.py.*--port $PORT" 2>/dev/null || true
sleep 2
if ss -tlnp 2>/dev/null | grep -q ":${PORT}\b"; then
pkill -9 -f "main\\.py.*--port $PORT" 2>/dev/null || true
sleep 1
fi
fi
rm -f "$PID_FILE"
if ss -tlnp 2>/dev/null | grep -q ":${PORT}\b"; then
die "Port $PORT is still in use after shutdown"
fi
log "ComfyUI stopped."
tests/e2e/test_e2e_install_flags.py
+747
View File
@@ -0,0 +1,747 @@
"""GOAL #60 — Real-server E2E for the dedicated install flags (worktree-mounted Manager).
Boots a REAL ComfyUI server from a disposable test root
(`E2E_COMFYUI_ROOT`, built by tests/e2e/scripts/setup_e2e_env.sh) with
the Manager mounted via `git worktree add --detach` (NEVER pip-installed
— [D2]) and exercises both dedicated-flag surfaces over live HTTP.
Usage:
bash tests/e2e/scripts/setup_e2e_env.sh # once (E2E-SC-01)
E2E_COMFYUI_ROOT=/path/to/root pytest tests/e2e/test_e2e_install_flags.py -v
Per-row map (goal60-scenarios.md, 24 rows — spec §3 BINDING):
SC-01 setup_e2e_env.sh (pre-suite script; idempotent build + marker — not a pytest test)
SC-02 mount_worktree fixture create path (SHA pin, .git-file, no-pip, printed SHA)
SC-03 mount_worktree fixture reuse path + path-prefix scoping invariant
SC-04 _start_server via start_comfyui.sh (readiness poll, restart tolerance,
per-launch log comfyui.<port>.<launch-id>.log)
SC-05 test_00_smoke_manager_version in EVERY server-up class (+abort guard)
SC-06 _stage via stage_flags.sh (backup-if-absent; restart-only by construction)
SC-07 class fixture finalizers (stop + port-free + config restore + backup DELETE)
+ mount_worktree finalizer (unmount + prune + absence assert)
SC-10 TestDenyArms.test_01_sa_deny SC-11 TestDenyArms.test_02_sb_deny
SC-12 TestAllowArms.test_01_git_url_allow
SC-13 TestAllowArms.test_02_pip_allow_reserved (anti-false-PASS — VERBATIM)
SC-14 TestAllowArms.test_03_restart_consumes_reservation (R-A through the holder)
SC-20 _pre_guards in both class fixtures (before any request)
SC-21 TestAllowArms.test_04_clone_residual_cleanup (+ installed-index cross-check)
SC-22 TestAllowArms.test_05_pip_residual_uninstall
SC-23 _reservation_guard — UNCONDITIONAL fixture-teardown guard (failure path)
SC-24 TestZeroResidual.test_99_zero_residual_sweep (unmount half lives in the
mount_worktree finalizer — it cannot be asserted from inside the session)
SC-30 module-level pytestmark (env unset -> all SKIP; unit suite unaffected)
SC-31 module-level pytestmark (marker absent -> all SKIP)
SC-32 needs_network marker on fixture-dependent (allow-arm / public) rows
SC-33 collection safety by construction: stdlib + pytest + requests
(via pytest.importorskip) ONLY — no glob/ imports, no server imports,
HTTP only at test time
SC-40 TestPublicListener (opt-in E2E_PUBLIC_LISTEN=1; L-P @ 0.0.0.0)
SC-41 batch S-C/S-C' E2E — DEFERRED (Q-5; spec FREEZE item 3; recorded here)
SC-42 TestAllowArms.test_06_requirements_watchdog (L-A launch log)
Fixture-lifecycle ownership (spec §3 BINDING block):
- every class server fixture DECLARES mount_worktree (mount-before-launch);
- process handle lives in a MUTABLE ServerHolder owned by the fixture;
teardown stops the CURRENT holder content (whatever launch identity is live);
- SC-14 restarts THROUGH the holder (stop L-A -> launch R-A -> replace handle);
- stop-before-next-class is structural (pytest class-fixture scoping);
- `requests` is imported via pytest.importorskip (absence degrades to SKIP).
T6 note: no tests/e2e/conftest.py — all fixtures are single-module, so the
optional T6 file is not demanded (spec §2 T6 condition not met).
"""
from __future__ import annotations
import configparser
import os
import shutil
import socket
import subprocess
import sys
import time
import uuid
from pathlib import Path
import pytest
# ---------------------------------------------------------------------------
# Skip gates (E2E-SC-30/31) — BEFORE anything env-dependent
# ---------------------------------------------------------------------------
E2E_ROOT = os.environ.get("E2E_COMFYUI_ROOT", "")
_MARKER_OK = bool(E2E_ROOT) and os.path.isfile(os.path.join(E2E_ROOT, ".e2e_setup_complete"))
pytestmark = pytest.mark.skipif(
not _MARKER_OK,
reason="E2E_COMFYUI_ROOT not set or E2E environment not ready (.e2e_setup_complete missing)",
)
# requests: test-extra — absence degrades to SKIP, never a collection error
# (spec §3 binding block item 5; [D4]).
requests = pytest.importorskip("requests")
# ---------------------------------------------------------------------------
# Constants / paths
# ---------------------------------------------------------------------------
PORT = int(os.environ.get("PORT", "8189"))
TIMEOUT = int(os.environ.get("TIMEOUT", "120"))
BASE_URL = f"http://127.0.0.1:{PORT}"
THIS_DIR = Path(__file__).resolve().parent
SCRIPTS_DIR = THIS_DIR / "scripts"
MANAGER_REPO = THIS_DIR.parents[1] # repo root of the checkout running the suite
ROOT = Path(E2E_ROOT) if E2E_ROOT else Path(".")
COMFY_DIR = ROOT / "comfyui"
CN_DIR = COMFY_DIR / "custom_nodes"
MOUNT = CN_DIR / "comfyui-manager"
CFG = COMFY_DIR / "user" / "__manager" / "config.ini"
CFG_BACKUP = Path(str(CFG) + ".before-flags")
SCRIPTS_FILE = COMFY_DIR / "user" / "__manager" / "startup-scripts" / "install-scripts.txt"
LOGS_DIR = ROOT / "logs"
VENV_PY = ROOT / "venv" / "bin" / "python"
# Owned fixtures ONLY (goal60-scenarios.md Conventions; [D3])
NODEPACK_URL = "https://github.com/ltdrdata/nodepack-test1-do-not-install"
PACK_NAME = "nodepack-test1-do-not-install"
# pip stimulus uses the git+ scheme: pip/uv require it for VCS URLs — a
# plain GitHub repo URL serves HTML and cannot install (verified by probe;
# spec amendment requested via leader pushback 2026-06-08; the SC-13/14
# oracle itself is encoded VERBATIM).
PIP_URL = "git+https://github.com/ltdrdata/pip-test1-do-not-install"
PIP_PKG = "pip-test1-do-not-install"
PIP_IMPORT = "pip_test1_do_not_install"
PIP_MARKER = "pip-test1-do-not-install:ok"
# Amendment A2 (live-run finding, leader-approved): the S-A nodepack fixture
# is deliberately NOT zero-dep — it pins python-slugify==8.0.4 in its
# requirements as the invariant-4 ride-along proof vehicle. The SC-42
# watchdog therefore allowlists exactly that requirement, and the
# transitive-dep residual class is swept at allow-class teardown + SC-24.
# (The S-B pip fixture IS zero-dep as documented.)
NODEPACK_PINNED_REQ = "python-slugify==8.0.4"
TRANSITIVE_DEPS = ("python-slugify", "text-unidecode")
POLL_TIMEOUT = 60
POLL_INTERVAL = 1.0
# Distinctive substrings of the flag-naming denial constants @ d45c8e6b
DENY_COPY_GIT = "'allow_git_url_install = true' in config.ini"
DENY_COPY_PIP = "'allow_pip_install = true' in config.ini"
# Old security_level-attributing copy (must NOT appear on flag denials)
OLD_COPY_GENERAL = "is not allowed in this security_level"
OLD_COPY_NORMAL_MINUS = "set the security level to"
# ---------------------------------------------------------------------------
# Network probe (E2E-SC-32) — evaluated ONLY when the env gate is open, so
# collection without the env performs no network IO (SC-33).
# ---------------------------------------------------------------------------
def _network_available() -> bool:
try:
with socket.create_connection(("github.com", 443), timeout=5):
return True
except OSError:
return False
_NETWORK = _network_available() if _MARKER_OK else False
needs_network = pytest.mark.skipif(
not _NETWORK,
reason="github.com unreachable — network-dependent fixture row skipped (E2E-SC-32)",
)
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _run(cmd, check=False, timeout=180, env=None, cwd=None):
return subprocess.run(
cmd, capture_output=True, text=True, timeout=timeout, check=check,
env=env, cwd=cwd,
)
def _script(name: str) -> str:
return str(SCRIPTS_DIR / name)
def _script_env(**extra) -> dict:
env = {**os.environ, "E2E_COMFYUI_ROOT": str(ROOT), "PORT": str(PORT),
"TIMEOUT": str(TIMEOUT)}
env.update({k: str(v) for k, v in extra.items()})
return env
def _pack_dir(name: str = PACK_NAME) -> Path:
return CN_DIR / name
def _pack_exists(name: str = PACK_NAME) -> bool:
return _pack_dir(name).is_dir()
def _remove_pack(name: str = PACK_NAME) -> None:
"""Donor _remove_pack pattern: rmtree 3-retry + rename-to-.trash_ fallback."""
path = _pack_dir(name)
if path.is_symlink():
path.unlink()
return
if not path.is_dir():
return
for attempt in range(3):
try:
shutil.rmtree(path)
return
except OSError:
if attempt < 2:
time.sleep(1)
trash = CN_DIR / f".trash_{uuid.uuid4().hex[:8]}"
try:
os.rename(path, trash)
shutil.rmtree(trash, ignore_errors=True)
except OSError:
shutil.rmtree(path, ignore_errors=True)
def _wait_for(predicate, timeout=POLL_TIMEOUT, interval=POLL_INTERVAL) -> bool:
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
if predicate():
return True
time.sleep(interval)
return False
def _pip_import_rc() -> int:
return _run([str(VENV_PY), "-c", f"import {PIP_IMPORT}"]).returncode
def _pip_marker_rc() -> "subprocess.CompletedProcess":
return _run([
str(VENV_PY), "-c",
f"import {PIP_IMPORT} as m; assert m.MARKER == '{PIP_MARKER}'",
])
def _pip_uninstall() -> "subprocess.CompletedProcess":
return _run([str(VENV_PY), "-m", "pip", "uninstall", "-y", PIP_PKG])
def _scripts_clean() -> bool:
"""True when the reservation file is absent OR carries no pip-test1 line."""
if not SCRIPTS_FILE.exists():
return True
return "pip-test1" not in SCRIPTS_FILE.read_text(errors="ignore")
def _reservation_guard() -> None:
"""E2E-SC-23 — UNCONDITIONAL teardown guard for the unconsumed-reservation
leak class: a leaked line would pip-install on ANY next boot of this root."""
if SCRIPTS_FILE.exists() and "pip-test1" in SCRIPTS_FILE.read_text(errors="ignore"):
SCRIPTS_FILE.unlink()
assert _scripts_clean(), "reservation guard failed to clear pip-test1 line (SC-23)"
def _restore_config() -> None:
"""E2E-SC-07: restore from backup, then DELETE the backup and assert absence
(a surviving stale backup would silently restore an outdated config at a
FUTURE run's teardown via the create-only-if-absent rule)."""
if CFG_BACKUP.exists():
shutil.copyfile(CFG_BACKUP, CFG)
CFG_BACKUP.unlink()
assert not CFG_BACKUP.exists(), "config backup must be DELETED after restore (SC-07/24)"
def _port_free() -> bool:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
s.settimeout(1)
return s.connect_ex(("127.0.0.1", PORT)) != 0
finally:
s.close()
def _pre_guards() -> None:
"""E2E-SC-20 — before EACH arm's matrix rows; assert all three."""
_remove_pack(PACK_NAME)
assert not _pack_exists(PACK_NAME), (
f"pre-guard: failed to clean {PACK_NAME} (file locks?)"
)
_pip_uninstall() # ignore rc: not-installed is fine
assert _pip_import_rc() != 0, "pre-guard: pip fixture importable before test"
_reservation_guard()
assert _scripts_clean(), "pre-guard: stale pip-test1 reservation present"
# ---------------------------------------------------------------------------
# Server lifecycle (E2E-SC-04 + spec §3 binding holder contract)
# ---------------------------------------------------------------------------
class ServerHolder:
"""Mutable process-handle holder owned by the class server fixture.
The holder always points at the CURRENT launch identity; SC-14 replaces
its content when it restarts through it, so class teardown stops
whatever is live — no orphan."""
def __init__(self):
self.launch_id: str | None = None
self.log_path: Path | None = None
self.live = False
self.smoke_ok = False
def _stage(mode: str) -> None:
r = _run(["bash", _script("stage_flags.sh"), mode], env=_script_env(), check=False)
assert r.returncode == 0, f"stage_flags.sh {mode} failed:\n{r.stdout}\n{r.stderr}"
assert CFG_BACKUP.exists(), "backup must exist after staging (SC-06)"
def _start_server(holder: ServerHolder, launch_id: str, listen: str = "127.0.0.1") -> None:
r = _run(
["bash", _script("start_comfyui.sh")],
env=_script_env(LISTEN=listen, LAUNCH_ID=launch_id),
timeout=TIMEOUT + 90,
)
assert r.returncode == 0, (
f"start_comfyui.sh failed for launch {launch_id}:\n{r.stdout}\n{r.stderr}"
)
holder.launch_id = launch_id
holder.log_path = LOGS_DIR / f"comfyui.{PORT}.{launch_id}.log"
holder.live = True
assert holder.log_path.is_file(), "per-launch log file missing (SC-04)"
def _stop_server(holder: ServerHolder) -> None:
if not holder.live:
return
r = _run(["bash", _script("stop_comfyui.sh")], env=_script_env(), timeout=120)
assert r.returncode == 0, f"stop_comfyui.sh failed:\n{r.stdout}\n{r.stderr}"
holder.live = False
assert _port_free(), "port still bound after stop (SC-07)"
def _launch_log(holder: ServerHolder) -> str:
assert holder.log_path is not None and holder.log_path.is_file()
return holder.log_path.read_text(errors="ignore")
def _named_log(launch_id: str) -> str:
p = LOGS_DIR / f"comfyui.{PORT}.{launch_id}.log"
assert p.is_file(), f"launch log for {launch_id} missing"
return p.read_text(errors="ignore")
def _require_smoke(holder: ServerHolder) -> None:
"""SC-05 abort semantics: matrix rows refuse to run after a smoke failure
so a mount/activation problem cannot produce misleading 404 results."""
if not holder.smoke_ok:
pytest.fail(
"aborting matrix row: smoke (GET /manager/version) has not passed "
"for this launch — mount/activation problem or Q-2 bundled-manager "
"collision (E2E-SC-05)"
)
def _smoke(holder: ServerHolder) -> None:
r = requests.get(f"{BASE_URL}/manager/version", timeout=10)
assert r.status_code == 200, (
f"smoke FAILED: GET /manager/version -> {r.status_code}; the "
f"worktree-mounted plugin did not register its routes (E2E-SC-05). "
f"Log tail:\n{_launch_log(holder)[-2000:]}"
)
assert r.text.strip(), "smoke: /manager/version body empty"
holder.smoke_ok = True
# ---------------------------------------------------------------------------
# Fixtures
# ---------------------------------------------------------------------------
@pytest.fixture(scope="session")
def mount_worktree():
"""E2E-SC-02/03 — SOLE owner of mount create / reuse-verify / teardown."""
ref = os.environ.get("E2E_MANAGER_REF", "HEAD")
r = _run(["git", "-C", str(MANAGER_REPO), "rev-parse", f"{ref}^{{commit}}"], check=True)
sha = r.stdout.strip()
# Scoping invariant (SC-03): the mount path is {ROOT}-prefixed and is
# NEVER under the members' .claude/worktrees tree. Every mount/teardown
# command below references ONLY this path.
mount = MOUNT.resolve()
assert ".claude/worktrees" not in str(mount).replace(os.sep, "/"), (
"mount path must never live under member worktrees (SC-03 scoping)"
)
assert str(mount).startswith(str(ROOT.resolve())), (
"mount path must be {ROOT}-prefixed (SC-03 scoping)"
)
porcelain = _run(["git", "-C", str(MANAGER_REPO), "worktree", "list", "--porcelain"]).stdout
if f"worktree {mount}" in porcelain:
# Reuse path (SC-03)
head = _run(["git", "-C", str(mount), "rev-parse", "HEAD"]).stdout.strip()
if head != sha:
_run(["git", "-C", str(mount), "checkout", "--detach", sha], check=True)
else:
# Create path (SC-02)
_run(["git", "-C", str(MANAGER_REPO), "worktree", "add", "--detach",
str(mount), sha], check=True)
# Verify (every session)
head = _run(["git", "-C", str(mount), "rev-parse", "HEAD"]).stdout.strip()
assert head == sha, f"mount HEAD {head} != expected {sha} (SC-02)"
print(f"[mount_worktree] Manager mounted at {mount} @ SHA {sha}") # [D2] traceability
assert (mount / ".git").is_file(), (
".git in the mount must be a FILE (gitdir pointer) — worktree layout (SC-02)"
)
# [D2] other half: no pip-installed Manager in the venv; per MM §2.2 no
# assertion anywhere relies on the mounted Manager's OWN version/remote
# self-report (.git-file degradation accepted by design — spec R5).
for dist in ("comfyui-manager", "ComfyUI-Manager"):
rc = _run([str(VENV_PY), "-m", "pip", "show", dist]).returncode
assert rc != 0, f"pip-installed Manager '{dist}' found in venv — violates [D2]"
yield {"path": mount, "sha": sha}
if os.environ.get("E2E_KEEP_MOUNT"):
print(f"[mount_worktree] E2E_KEEP_MOUNT set — keeping {mount}")
return
# Exception-safe: prune + absence asserts run even when remove fails
# (review iter-2 — crash residue must still be surfaced honestly).
try:
_run(["git", "-C", str(MANAGER_REPO), "worktree", "remove", "--force", str(mount)],
check=True)
finally:
_run(["git", "-C", str(MANAGER_REPO), "worktree", "prune"])
porcelain = _run(["git", "-C", str(MANAGER_REPO), "worktree", "list", "--porcelain"]).stdout
assert f"worktree {mount}" not in porcelain, "mount still listed after remove (SC-07)"
assert not mount.exists(), "mount dir still present after remove (SC-07)"
@pytest.fixture(scope="class")
def deny_server(mount_worktree):
"""L-D: both flags ABSENT (live 'missing key reads false'), loopback."""
_pre_guards() # SC-20
_stage("deny") # SC-06
holder = ServerHolder()
_start_server(holder, "L-D") # SC-04
yield holder
# Exception-safe teardown chain (review iter-2 must-fix): a failing
# stop is exactly the crashed-run shape SC-23 exists for — the guard
# and the config restore MUST run regardless.
try:
_stop_server(holder) # SC-07 (current handle, whatever is live)
finally:
try:
_reservation_guard() # SC-23 — UNCONDITIONAL
finally:
_restore_config() # SC-07: restore + DELETE backup
@pytest.fixture(scope="class")
def allow_server(mount_worktree):
"""L-A: both flags true, loopback. SC-14 mutates the holder to R-A."""
_pre_guards() # SC-20 (re-guards before the allow arm)
_stage("allow") # SC-06
holder = ServerHolder()
_start_server(holder, "L-A")
yield holder
# Exception-safe teardown chain (review iter-2 must-fix): every
# residual guard runs even when the stop (or an earlier sweep step)
# raises — SC-23 is contractually UNCONDITIONAL (R3 leak class).
try:
_stop_server(holder) # stops the CURRENT identity (L-A or R-A)
finally:
try:
_remove_pack(PACK_NAME) # defensive re-sweep (primary assert is SC-21)
_pip_uninstall() # defensive (primary assert is SC-22)
# Amendment A2: sweep the S-A fixture's transitive-dep residual
# class (python-slugify + text-unidecode ride the git
# transaction; verified NOT in ComfyUI's own requirements).
_run([str(VENV_PY), "-m", "pip", "uninstall", "-y", *TRANSITIVE_DEPS])
finally:
try:
_reservation_guard() # SC-23 — UNCONDITIONAL (failure path cover)
finally:
_restore_config()
@pytest.fixture(scope="class")
def public_server(mount_worktree):
"""L-P (opt-in, Q-7): both flags true, 0.0.0.0 listener."""
_pre_guards()
_stage("allow")
holder = ServerHolder()
_start_server(holder, "L-P", listen="0.0.0.0")
yield holder
# Exception-safe teardown chain (review iter-2 must-fix).
try:
_stop_server(holder)
finally:
try:
_reservation_guard()
finally:
_restore_config()
# ---------------------------------------------------------------------------
# Classes — definition order IS execution order (deny first on the fresh env)
# ---------------------------------------------------------------------------
class TestDenyArms:
"""L-D launch: SC-05 smoke, SC-10, SC-11 (deny rows are offline-safe —
denial happens before any network access)."""
def test_00_smoke_manager_version(self, deny_server):
_smoke(deny_server) # SC-05
def test_01_sa_deny(self, deny_server):
"""E2E-SC-10: S-A deny — 403 + exact flag token + no artifact + honest log."""
_require_smoke(deny_server)
r = requests.post(f"{BASE_URL}/customnode/install/git_url",
data=NODEPACK_URL, timeout=30)
assert r.status_code == 403
assert r.json() == {"error": "allow_git_url_install"}, (
f"deny body must carry the flag token, got {r.text!r}"
)
assert not _pack_exists(PACK_NAME), "clone artifact created on DENY (SC-10)"
log = _launch_log(deny_server)
assert DENY_COPY_GIT in log, "flag-naming denial copy missing from L-D log"
assert OLD_COPY_GENERAL not in log and OLD_COPY_NORMAL_MINUS not in log, (
"denial attributed to security_level — honest-copy violation (SC-10)"
)
def test_02_sb_deny(self, deny_server):
"""E2E-SC-11: S-B deny — 403 + flag token + no reservation + not importable."""
_require_smoke(deny_server)
r = requests.post(f"{BASE_URL}/customnode/install/pip",
data=PIP_URL, timeout=30)
assert r.status_code == 403
assert r.json() == {"error": "allow_pip_install"}
assert _scripts_clean(), "reservation recorded on DENY (SC-11)"
assert _pip_import_rc() != 0, "pip fixture importable after DENY (SC-11)"
log = _launch_log(deny_server)
assert DENY_COPY_PIP in log, "flag-naming denial copy missing from L-D log"
class TestAllowArms:
"""L-A launch + R-A restart. ORDERED methods (donor sequential-class
precedent): SC-12 -> SC-13 -> SC-14 -> SC-21 -> SC-22 -> SC-42."""
def test_00_smoke_manager_version(self, allow_server):
_smoke(allow_server) # SC-05 (re-smoke on the new launch)
@needs_network
def test_01_git_url_allow(self, allow_server):
"""E2E-SC-12: S-A allow — 200 + real clone + clone-target proof."""
_require_smoke(allow_server)
r = requests.post(f"{BASE_URL}/customnode/install/git_url",
data=NODEPACK_URL, timeout=120)
assert r.status_code == 200, f"S-A allow expected 200, got {r.status_code}: {r.text!r}"
assert _wait_for(lambda: _pack_exists(PACK_NAME)), (
f"{PACK_NAME} not cloned within {POLL_TIMEOUT}s (SC-12)"
)
git_dir = _pack_dir() / ".git"
assert git_dir.is_dir(), "no .git DIRECTORY — not a real clone (SC-12)"
# Donor clone-target proof: .git/config [remote "origin"] url matches
# the requested URL modulo the .git suffix.
cp = configparser.ConfigParser()
cp.read(git_dir / "config")
section = 'remote "origin"'
assert section in cp, f'[{section}] missing from .git/config: {cp.sections()!r}'
remote_url = cp[section].get("url", "").rstrip("/")
expected = NODEPACK_URL.rstrip("/")
assert remote_url in (expected, expected + ".git"), (
f"clone targeted the WRONG repo: {remote_url!r} != {expected!r} (SC-12)"
)
@needs_network
def test_02_pip_allow_reserved(self, allow_server):
"""E2E-SC-13 (VERBATIM anti-false-PASS oracle): 200 = RESERVED, NOT
INSTALLED. Asserting import success here would be the exact false-PASS
the MM correction exists to prevent."""
_require_smoke(allow_server)
r = requests.post(f"{BASE_URL}/customnode/install/pip",
data=PIP_URL, timeout=30)
assert r.status_code == 200, f"S-B allow expected 200, got {r.status_code}: {r.text!r}"
assert SCRIPTS_FILE.is_file(), "no reservation file after S-B allow (SC-13)"
content = SCRIPTS_FILE.read_text(errors="ignore")
reserved_lines = [
ln for ln in content.splitlines()
if "'#FORCE'" in ln and PIP_PKG in ln
]
assert reserved_lines, (
f"no reservation line with '#FORCE' + {PIP_PKG!r} in {SCRIPTS_FILE}:\n{content}"
)
# MANDATORY: the package is NOT installed at this point.
assert _pip_import_rc() != 0, (
"pip fixture importable right after the 200 — reservation semantics "
"violated, or a previous run leaked state (SC-13 anti-false-PASS)"
)
@needs_network
def test_03_restart_consumes_reservation(self, allow_server):
"""E2E-SC-14: R-A restart THROUGH the holder; the consuming boot
executes + removes the reservation; MARKER import proves field-level."""
_require_smoke(allow_server)
assert SCRIPTS_FILE.is_file(), "precondition: reservation must exist (SC-13 first)"
# Restart THROUGH the holder (spec §3 binding item 3): stop the live
# L-A process, relaunch as R-A with the SAME staged config, replace
# the handle — class teardown then stops R-A.
_stop_server(allow_server)
_start_server(allow_server, "R-A")
_smoke(allow_server)
# Field-level positive proof (not just exit code):
marker = _pip_marker_rc()
assert marker.returncode == 0, (
f"MARKER import failed after the consuming restart (SC-14):\n"
f"{marker.stderr}\nR-A log tail:\n{_named_log('R-A')[-3000:]}"
)
assert not SCRIPTS_FILE.exists(), (
"install-scripts.txt NOT removed by the consuming boot (SC-14 self-clean)"
)
ra_log = _named_log("R-A")
assert "## ComfyUI-Manager: EXECUTE =>" in ra_log and PIP_PKG in ra_log, (
"R-A log lacks the startup-script execution block (SC-14)"
)
assert "Startup script completed." in ra_log, (
"R-A log lacks the startup-script completion line (SC-14)"
)
@needs_network
def test_04_clone_residual_cleanup(self, allow_server):
"""E2E-SC-21: clone-dir hygiene; FS-absence primary + installed-index
cross-check while the server is still up (defensive, donor pattern)."""
_remove_pack(PACK_NAME)
assert not _pack_exists(PACK_NAME), "clone dir still present (SC-21 primary)"
try:
r = requests.get(f"{BASE_URL}/customnode/installed", timeout=15)
if r.status_code == 200:
installed = r.json()
assert PACK_NAME not in installed, (
f"{PACK_NAME} still in /customnode/installed after removal (SC-21)"
)
for key, pkg in installed.items():
if isinstance(pkg, dict):
assert pkg.get("cnr_id") != PACK_NAME and pkg.get("aux_id") != PACK_NAME, (
f"installed entry {key!r} still references {PACK_NAME!r} (SC-21)"
)
except (ValueError, requests.RequestException):
# Spec SC-21: if the response schema proves awkward, FS-absence
# alone satisfies this row.
pass
@needs_network
def test_05_pip_residual_uninstall(self, allow_server):
"""E2E-SC-22: S-B residual class 1 (venv package)."""
r = _pip_uninstall()
assert r.returncode == 0, f"pip uninstall failed (SC-22):\n{r.stdout}\n{r.stderr}"
assert _pip_import_rc() != 0, "pip fixture importable after uninstall (SC-22)"
@needs_network
def test_06_requirements_watchdog(self, allow_server):
"""E2E-SC-42 (Q-6 watchdog, amendment A2): every management-script
EXECUTE in the L-A launch log must be attributable to the owned
fixture's OWN pinned requirements (python-slugify==8.0.4 — the
nodepack fixture's deliberate invariant-4 ride-along requirement).
Any other EXECUTE (e.g. a Manager-requirements install — the Q-6
risk this row guards) FAILS the watchdog.
The allowlisted line doubles as LIVE proof of the invariant-4
ride-along class: a dependency pip install executed inside the
git-URL transaction without consulting allow_pip_install."""
la_log = _named_log("L-A")
banner = "## ComfyUI-Manager: EXECUTE =>"
idx = 0
execs = []
while True:
idx = la_log.find(banner, idx)
if idx < 0:
break
execs.append(la_log[idx: idx + 600])
idx += len(banner)
# Non-vacuity (review iter-2 / A2 positive half): the ride-along
# MUST have happened — exactly ONE management-script execution,
# the fixture's single pinned requirement.
assert len(execs) == 1, (
f"expected exactly 1 management-script execution during L-A "
f"(the fixture's pinned requirement ride-along), found "
f"{len(execs)} (SC-42 / A2)"
)
window = execs[0]
assert NODEPACK_PINNED_REQ in window, (
"unexpected management-script execution during L-A — not "
"attributable to the fixture's pinned requirement "
f"({NODEPACK_PINNED_REQ}) (SC-42 watchdog):\n{window}"
)
# Line-level shape: it must be a pip-install command, not an
# arbitrary script that merely mentions the requirement string.
assert "'pip'" in window and "'install'" in window, (
f"EXECUTE block is not a pip-install command (SC-42):\n{window}"
)
@pytest.mark.skipif(
not os.environ.get("E2E_PUBLIC_LISTEN"),
reason="public-listener row is opt-in (E2E_PUBLIC_LISTEN=1) — Q-7 default-off",
)
class TestPublicListener:
"""E2E-SC-40 (opt-in): flags=true + 0.0.0.0 -> still 403 on both surfaces.
Live proof of invariant 2 (predicate = flag AND loopback at REQUEST time)."""
def test_00_smoke_manager_version(self, public_server):
_smoke(public_server)
def test_01_sa_public_deny(self, public_server):
_require_smoke(public_server)
r = requests.post(f"{BASE_URL}/customnode/install/git_url",
data=NODEPACK_URL, timeout=30)
assert r.status_code == 403
assert r.json() == {"error": "allow_git_url_install"}
assert not _pack_exists(PACK_NAME)
def test_02_sb_public_deny(self, public_server):
_require_smoke(public_server)
r = requests.post(f"{BASE_URL}/customnode/install/pip",
data=PIP_URL, timeout=30)
assert r.status_code == 403
assert r.json() == {"error": "allow_pip_install"}
assert _scripts_clean()
class TestZeroResidual:
"""E2E-SC-24: the complete [D3] residual inventory in one assertion block.
Runs AFTER the server classes (their class-scoped fixtures have finalized:
server stopped, config restored, backup deleted). The unmount half of the
inventory is asserted by the mount_worktree finalizer itself — it cannot
be asserted from inside the session while the mount is still live."""
def test_99_zero_residual_sweep(self, mount_worktree):
# custom_nodes clean (incl. .trash_ fallback leftovers)
assert not _pack_exists(PACK_NAME), "nodepack residue in custom_nodes (SC-24)"
leftovers = [p.name for p in CN_DIR.iterdir()
if p.name.startswith((".trash_", PACK_NAME))]
assert not leftovers, f"residual entries in custom_nodes: {leftovers} (SC-24)"
# venv clean
assert _pip_import_rc() != 0, "pip fixture still importable (SC-24)"
# Amendment A2: transitive-dep residual class swept
for dep in TRANSITIVE_DEPS:
rc = _run([str(VENV_PY), "-m", "pip", "show", dep]).returncode
assert rc != 0, f"transitive dep {dep!r} survived the sweep (SC-24 / A2)"
# reservation clean
assert _scripts_clean(), "pip-test1 reservation residue (SC-24)"
# config restored + backup DELETED
assert CFG.is_file(), "config.ini missing after restore (SC-24)"
cfg_text = CFG.read_text(errors="ignore")
assert "allow_git_url_install" not in cfg_text, "staged flag leaked into restored config (SC-24)"
assert "allow_pip_install" not in cfg_text, "staged flag leaked into restored config (SC-24)"
assert not CFG_BACKUP.exists(), "stale config backup survived (SC-24 / peer R2)"
# port free
assert _port_free(), f"port {PORT} still bound (SC-24)"
if __name__ == "__main__":
sys.exit(pytest.main([__file__, "-v"]))
tests/test_install_flag_predicate.py
+153
View File
@@ -0,0 +1,153 @@
"""Unit tests for the dedicated-install-flag predicate.
Covers `is_dedicated_install_allowed(flag_value, listen_address)` in
glob/manager_server.py:
- Truth table: allowed iff flag is true AND the listener is loopback.
- REPLACE-by-construction: the 2-arg signature has no security_level /
network_mode parameter and the body references no config machinery,
so security_level cannot influence the outcome in either direction.
- Cross-flag isolation: a single flag_value input cannot consult the
other flag.
- Request-time evaluation: the body must not read the import-time
`is_local_mode` snapshot (callers pass args.listen per request).
Harness: glob/manager_server.py is not importable under the test runner
(`from comfy.cli_args import args`, PromptServer), so we AST-parse the
file and exec only the wanted pure defs — `glob/` is never added to
sys.path (the dir name shadows the stdlib `glob`).
"""
import ast
import inspect
import unittest
from pathlib import Path
from typing import Any
REPO_ROOT = Path(__file__).resolve().parent.parent
MANAGER_SERVER_PATH = REPO_ROOT / "glob" / "manager_server.py"
_WANTED = {"is_loopback", "is_dedicated_install_allowed"}
def _load_predicates():
"""Parse manager_server.py; exec only the wanted pure function defs."""
source = MANAGER_SERVER_PATH.read_text()
tree = ast.parse(source)
nodes = []
node_by_name = {}
for node in tree.body:
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)) and node.name in _WANTED:
nodes.append(node)
node_by_name[node.name] = node
missing = _WANTED - node_by_name.keys()
assert not missing, f"expected pure defs missing from manager_server.py: {missing}"
module = ast.Module(body=nodes, type_ignores=[])
ns: dict = {"bool": bool}
exec(compile(module, "manager_server_predicates", "exec"), ns)
return ns, node_by_name
_NS, _NODES = _load_predicates()
IS_LOOPBACK: Any = _NS["is_loopback"]
PREDICATE: Any = _NS["is_dedicated_install_allowed"]
PREDICATE_NODE = _NODES["is_dedicated_install_allowed"]
class IsLoopbackBehaviorTest(unittest.TestCase):
"""Pins the loopback term the predicate composes."""
def test_ipv4_loopback(self):
self.assertTrue(IS_LOOPBACK("127.0.0.1"))
def test_public_address(self):
self.assertFalse(IS_LOOPBACK("0.0.0.0"))
def test_ipv6_loopback(self):
self.assertTrue(IS_LOOPBACK("::1"))
def test_invalid_address_reads_false(self):
# Non-IP strings deny-by-default (ValueError path).
self.assertFalse(IS_LOOPBACK("localhost"))
self.assertFalse(IS_LOOPBACK(""))
class DedicatedInstallPredicateTest(unittest.TestCase):
"""P-direct truth table + REPLACE-by-construction."""
def test_truth_table(self):
"""allowed iff flag AND loopback."""
cases = [
# (flag_value, listen_address, expected)
(True, "127.0.0.1", True),
(False, "127.0.0.1", False),
(True, "0.0.0.0", False),
(False, "0.0.0.0", False),
(True, "::1", True),
(True, "not-an-ip", False), # invalid listen -> deny
]
for flag_value, listen, expected in cases:
with self.subTest(flag=flag_value, listen=listen):
result = PREDICATE(flag_value, listen)
self.assertIsInstance(result, bool)
self.assertEqual(result, expected)
def test_falsy_flag_values_deny(self):
"""Secure-by-default: any falsy flag never allows."""
for falsy in (False, None, 0, ""):
with self.subTest(flag=falsy):
self.assertFalse(PREDICATE(falsy, "127.0.0.1"))
def test_signature_has_no_security_level(self):
"""Exactly (flag_value, listen_address) — no security_level term."""
params = list(inspect.signature(PREDICATE).parameters)
self.assertEqual(params, ["flag_value", "listen_address"])
for name in params:
self.assertNotIn("security", name)
self.assertNotIn("network_mode", name)
def test_body_free_of_config_machinery(self):
"""Body references no security_level plumbing, config reader, or the
import-time `is_local_mode` snapshot (request-time evaluation)."""
forbidden = {
"is_allowed_security_level",
"security_level",
"get_config",
"core",
"is_local_mode",
"network_mode",
"args",
}
seen = set()
for node in ast.walk(PREDICATE_NODE):
if isinstance(node, ast.Name):
seen.add(node.id)
elif isinstance(node, ast.Attribute):
seen.add(node.attr)
elif isinstance(node, ast.Constant) and isinstance(node.value, str):
seen.add(node.value)
self.assertEqual(
seen & forbidden, set(),
"predicate body must stay config-import-free",
)
def test_cross_flag_isolation_by_construction(self):
"""A single flag_value input cannot consult the other flag."""
seen_strings = {
node.value
for node in ast.walk(PREDICATE_NODE)
if isinstance(node, ast.Constant) and isinstance(node.value, str)
}
self.assertNotIn("allow_git_url_install", seen_strings)
self.assertNotIn("allow_pip_install", seen_strings)
self.assertTrue(PREDICATE(True, "127.0.0.1"))
self.assertFalse(PREDICATE(False, "127.0.0.1"))
def test_purity_deterministic(self):
"""Pure predicate — repeat calls identical."""
for _ in range(3):
self.assertTrue(PREDICATE(True, "127.0.0.1"))
self.assertFalse(PREDICATE(True, "0.0.0.0"))
if __name__ == "__main__":
unittest.main(verbosity=2)
tests/test_install_flags_config.py
+230
View File
@@ -0,0 +1,230 @@
"""Config-contract tests for the dedicated install flags.
Drives the real glob/manager_core config reader/writer through a
subprocess-isolated harness and pins: missing keys read False
(secure-by-default), only case-insensitive "true" is truthy, write
round-trips losslessly, edits need a restart (cached_config), the
exception-fallback path supplies False, no auto-migration seeds the
flags from a legacy security_level, and the get_bool missing->False
quirk the flags rely on stays frozen.
Harness: the child process injects a stub `folder_paths` (routing
import-time side effects into a tmpdir, and making has_system_user_api()
True so force_security_level_if_needed does not force 'strong'), prepends
`glob/` to ITS OWN sys.path (shadowing of stdlib `glob` confined to the
child), points manager_core.manager_config_path at a tmp config.ini,
resets cached_config, runs the scenario, and prints one JSON line for the
parent to assert.
"""
import json
import subprocess
import sys
import textwrap
import unittest
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
_CHILD_PREAMBLE = textwrap.dedent(
"""
import sys, types, tempfile, os, json
tmp = tempfile.mkdtemp(prefix="cm_flags_cfg_")
stub = types.ModuleType("folder_paths")
stub.get_user_directory = lambda: tmp
stub.get_system_user_directory = lambda *a, **k: os.path.join(tmp, "sysuser")
sys.modules["folder_paths"] = stub
sys.path.insert(0, {glob_path!r})
import manager_core
CONFIG_PATH = os.path.join(tmp, "config.ini")
manager_core.manager_config_path = CONFIG_PATH
manager_core.cached_config = None
def write_ini(text):
with open(CONFIG_PATH, "w") as f:
f.write(text)
def fresh_read():
manager_core.cached_config = None
return manager_core.get_config()
def flag_view(cfg):
return {{
"git": cfg.get("allow_git_url_install", "<ABSENT>"),
"pip": cfg.get("allow_pip_install", "<ABSENT>"),
}}
"""
)
def _run_child(body):
"""Run a scenario body in the isolated child; return its JSON payload."""
script = _CHILD_PREAMBLE.format(glob_path=str(REPO_ROOT / "glob")) + textwrap.dedent(body)
proc = subprocess.run(
[sys.executable, "-c", script],
capture_output=True,
text=True,
timeout=180,
cwd=str(REPO_ROOT),
)
if proc.returncode != 0:
raise AssertionError(
"config-harness child failed (rc=%d). stderr tail:\n%s"
% (proc.returncode, "\n".join(proc.stderr.strip().splitlines()[-8:]))
)
last_line = proc.stdout.strip().splitlines()[-1]
return json.loads(last_line)
class InstallFlagsConfigContractTest(unittest.TestCase):
def test_sc17_missing_keys_read_false(self):
"""Both keys absent from config.ini -> both flags read False
(secure-by-default)."""
payload = _run_child(
"""
write_ini("[default]\\nsecurity_level = normal\\n")
print(json.dumps(flag_view(fresh_read())))
"""
)
self.assertIs(payload["git"], False)
self.assertIs(payload["pip"], False)
def test_sc18_malformed_and_case_matrix(self):
"""Only case-insensitive "true" is truthy; malformed -> False."""
payload = _run_child(
"""
out = {}
for raw in ["1", "yes", "TRUE", "true ", "true"]:
write_ini("[default]\\nallow_git_url_install = %s\\nallow_pip_install = %s\\n" % (raw, raw))
cfg = fresh_read()
out[raw] = flag_view(cfg)
print(json.dumps(out))
"""
)
expected = {
"1": False, # malformed: numeric truthiness NOT honored
"yes": False, # malformed: yes/no NOT honored
"TRUE": True, # case-insensitive read (:1724)
"true ": True, # configparser strips surrounding whitespace
"true": True,
}
for raw, want in expected.items():
with self.subTest(value=raw):
self.assertIs(payload[raw]["git"], want)
self.assertIs(payload[raw]["pip"], want)
def test_sc19_write_round_trip(self):
"""write_config persists str(bool); round-trip is lossless."""
payload = _run_child(
"""
write_ini("[default]\\nsecurity_level = normal\\n")
cfg = fresh_read()
cfg["allow_git_url_install"] = True
cfg["allow_pip_install"] = False
manager_core.write_config()
raw = open(CONFIG_PATH).read()
reread = flag_view(fresh_read())
print(json.dumps({
"raw_has_git_true": "allow_git_url_install = True" in raw,
"raw_has_pip_false": "allow_pip_install = False" in raw,
"reread": reread,
}))
"""
)
self.assertTrue(
payload["raw_has_git_true"],
"write_config must persist allow_git_url_install = True in [default]",
)
self.assertTrue(
payload["raw_has_pip_false"],
"write_config must persist allow_pip_install = False in [default]",
)
self.assertIs(payload["reread"]["git"], True)
self.assertIs(payload["reread"]["pip"], False)
def test_sc20_restart_only_activation(self):
"""Editing config.ini without restart has NO effect (cache wins);
a reset (== restart) picks up the change."""
payload = _run_child(
"""
write_ini("[default]\\nallow_git_url_install = false\\n")
first = manager_core.get_config() # populates cached_config
before_edit = flag_view(first)
write_ini("[default]\\nallow_git_url_install = true\\n")
cached = flag_view(manager_core.get_config()) # NO reset: cache must win
after_restart = flag_view(fresh_read()) # reset == restart
print(json.dumps({
"before_edit": before_edit,
"cached_after_edit": cached,
"after_restart": after_restart,
}))
"""
)
self.assertIs(payload["before_edit"]["git"], False)
self.assertIs(
payload["cached_after_edit"]["git"],
False,
"cached_config must NOT hot-reload the edited flag",
)
self.assertIs(payload["after_restart"]["git"], True)
def test_sc21_exception_fallback_supplies_false(self):
"""Corrupted config.ini -> exception-fallback dict supplies flags False."""
payload = _run_child(
"""
# No [default] section header -> read_config raises inside try,
# lands in the exception-fallback dict.
write_ini("allow_git_url_install = true\\ngarbage without section\\n")
cfg = fresh_read()
print(json.dumps({
"flags": flag_view(cfg),
"fallback_marker_file_logging": cfg.get("file_logging"),
}))
"""
)
# file_logging True proves the FALLBACK dict was used (the parse
# path would yield False for a missing file_logging key).
self.assertIs(
payload["fallback_marker_file_logging"],
True,
"corrupted ini must route through the exception-fallback dict",
)
self.assertIs(payload["flags"]["git"], False)
self.assertIs(payload["flags"]["pip"], False)
def test_sc28_no_auto_migration_from_weak(self):
"""Legacy `security_level=weak` does NOT seed the flags (no auto-migration)."""
payload = _run_child(
"""
write_ini("[default]\\nsecurity_level = weak\\n")
cfg = fresh_read()
print(json.dumps({
"flags": flag_view(cfg),
"security_level": cfg.get("security_level"),
}))
"""
)
self.assertEqual(payload["security_level"], "weak")
self.assertIs(payload["flags"]["git"], False, "no auto-seed from weak")
self.assertIs(payload["flags"]["pip"], False, "no auto-seed from weak")
def test_sc42_get_bool_quirk_guard(self):
"""get_bool ignores its default param: missing `file_logging` reads
False despite a True default. The flags rely on this missing->False
quirk; this guard pins it."""
payload = _run_child(
"""
write_ini("[default]\\nsecurity_level = normal\\n")
cfg = fresh_read()
print(json.dumps({"file_logging": cfg.get("file_logging", "<ABSENT>")}))
"""
)
self.assertIs(
payload["file_logging"],
False,
"get_bool quirk changed: missing key no longer reads False — "
"new flags rely on missing->False",
)
if __name__ == "__main__":
unittest.main(verbosity=2)
tests/test_install_flags_gates.py
+461
View File
@@ -0,0 +1,461 @@
"""Handler-gate tests for the dedicated install flags.
Three layers, each covering what the others can't:
1. SCInstallGateMirrorTest — a slim mirror of the batch-install handler
(`install_custom_node`, S-C) wired to the REAL extracted gate
primitives. Covers the handler *composition* that the pure predicate
test cannot: risky-level routing, the load-bearing public canary
(the entry gate has no network term, so the deny must come from the
predicate's loopback term), block-arm unconditionality, the
security_level entry gate, and the CNR/middle false-pass guards.
2. DenialConstantsTest — content of the flag denial constants and the
`security_403_response` precedence, asserted directly (no server).
3. BindingProofTest — AST proof that the REAL handlers (S-A/S-B/S-C) are
wired to the predicate and that the old `is_allowed_security_level('high')`
gate is gone from S-A/S-B (closes the mirror-vs-real gap).
The direct S-A/S-B allow/deny behavior is covered by the binding proof
(wiring) plus the real-server E2E suite (behavior); the mirror here
focuses on S-C, whose multi-arm branching is the genuine logic risk.
Harness: glob/manager_server.py is not importable under the runner
(`from comfy.cli_args import args`, PromptServer), so we AST-extract the
gate primitives and exec them into a stub namespace — `glob/` is never
added to sys.path (the dir name shadows stdlib glob).
"""
import ast
import asyncio
import contextlib
import inspect
import json
import logging
import unittest
from pathlib import Path
from typing import Any
from aiohttp import web
from aiohttp.test_utils import TestClient, TestServer
REPO_ROOT = Path(__file__).resolve().parent.parent
MANAGER_SERVER_PATH = REPO_ROOT / "glob" / "manager_server.py"
_WANTED_FUNCS = {
"is_loopback",
"is_dedicated_install_allowed",
"is_allowed_security_level",
"security_403_response",
}
_WANTED_CONSTS = {
"SECURITY_MESSAGE_MIDDLE_OR_BELOW",
"SECURITY_MESSAGE_NORMAL_MINUS",
"SECURITY_MESSAGE_GENERAL",
"SECURITY_MESSAGE_FLAG_GIT_URL",
"SECURITY_MESSAGE_FLAG_PIP",
}
_HANDLER_NAMES = {
"install_custom_node_git_url", # S-A
"install_custom_node_pip", # S-B
"install_custom_node", # S-C
}
class _MigrationStub:
def __init__(self):
self.system_user_api = True
def has_system_user_api(self):
return self.system_user_api
class _CoreStub:
"""Stand-in for `core` consulted by is_allowed_security_level."""
def __init__(self):
self.security_level = "normal"
def get_config(self):
return {"security_level": self.security_level}
def _load_surfaces():
source = MANAGER_SERVER_PATH.read_text()
tree = ast.parse(source)
exec_nodes = []
handler_nodes = {}
for node in tree.body:
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
if node.name in _WANTED_FUNCS:
node.decorator_list = [] # exec needs no aiohttp routing context
exec_nodes.append(node)
if node.name in _HANDLER_NAMES:
handler_nodes[node.name] = node
elif isinstance(node, ast.Assign):
for target in node.targets:
if isinstance(target, ast.Name) and target.id in _WANTED_CONSTS:
exec_nodes.append(node)
module = ast.Module(body=exec_nodes, type_ignores=[])
ns: dict = {
"web": web,
"bool": bool,
"manager_migration": _MigrationStub(),
"core": _CoreStub(),
"is_local_mode": True,
}
exec(compile(module, "manager_server_gate_surfaces", "exec"), ns)
# Feature is implemented — these must resolve, else the extraction or
# the production code regressed.
for name in _WANTED_FUNCS | _WANTED_CONSTS:
assert ns.get(name) is not None, "missing gate primitive: %s" % name
for name in _HANDLER_NAMES:
assert name in handler_nodes, "missing handler: %s" % name
return ns, handler_nodes
NS, HANDLERS = _load_surfaces()
PREDICATE: Any = NS["is_dedicated_install_allowed"]
IS_LOOPBACK: Any = NS["is_loopback"]
IAS: Any = NS["is_allowed_security_level"]
SEC_403: Any = NS["security_403_response"]
CATALOG_URL = "https://github.com/catalog/listed-node"
UNKNOWN_URL = "https://github.com/x/not-in-catalog"
CATALOG_PIP = "torch"
UNKNOWN_PIP = "definitely-not-in-catalog-pkg"
def _body_unknown(files=None, pip=None):
"""version=='unknown' ingestion arm."""
return {
"version": "unknown",
"selected_version": "unknown",
"files": files or [UNKNOWN_URL],
"pip": pip or [],
"channel": "default",
"mode": "cache",
"ui_id": "test-row",
}
def _body_cnr_latest():
"""non-nightly CNR arm — risky='low' set statically."""
return {
"version": "1.0.0",
"selected_version": "latest",
"id": "catalog-cnr-pack",
"channel": "default",
"mode": "cache",
"ui_id": "test-row",
}
class _TrackingFlags(dict):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.reads = []
def __getitem__(self, key):
self.reads.append(key)
return super().__getitem__(key)
class GateEnv:
"""Injectable per-row environment for the mirror app."""
def __init__(self, git=False, pip=False, listen="127.0.0.1", security_level="normal"):
self.flags = _TrackingFlags(
{"allow_git_url_install": git, "allow_pip_install": pip}
)
self.listen = listen
self.security_level = security_level
self.task_queue = []
self.risky_calls = 0
self.catalog_urls = {CATALOG_URL}
self.catalog_pips = {CATALOG_PIP}
def get_risky_level(self, files, pip_packages):
"""Mirror of get_risky_level: URL check precedes pip check."""
self.risky_calls += 1
for x in files or []:
if x not in self.catalog_urls:
return "high"
for p in pip_packages or []:
if p not in self.catalog_pips:
return "block"
return "middle"
_SC_DENY_TEXT = "A security error has occurred. Please check the terminal logs"
def _apply_env(env):
"""Retained gates use the is_local_mode snapshot + config stub."""
NS["is_local_mode"] = IS_LOOPBACK(env.listen)
NS["core"].security_level = env.security_level
def _make_sc_install(env):
"""Slim mirror of install_custom_node (S-C) in its post-change gate
shape: security_level entry gate, then risky-level routing where the
'high' (unknown-URL) arm goes through the dedicated predicate and the
retained arms keep is_allowed_security_level."""
async def sc_install(request):
# ENTRY gate — UNCHANGED (security_level-governed)
if not IAS("middle"):
logging.error(NS["SECURITY_MESSAGE_MIDDLE_OR_BELOW"])
return web.Response(status=403, text=_SC_DENY_TEXT)
json_data = await request.json()
risky_level = None
git_url = None
selected_version = json_data.get("selected_version")
if json_data["version"] != "unknown" and selected_version != "unknown":
if selected_version != "nightly":
risky_level = "low" # static — get_risky_level NOT called
else:
git_url = [json_data.get("repository")]
else:
git_url = json_data.get("files")
if risky_level is None:
risky_level = env.get_risky_level(git_url, json_data.get("pip", []))
if risky_level == "high":
# unknown-URL arm -> dedicated predicate (flag AND loopback)
if not PREDICATE(env.flags["allow_git_url_install"], env.listen):
logging.error(NS["SECURITY_MESSAGE_FLAG_GIT_URL"])
return web.Response(status=404, text=_SC_DENY_TEXT)
elif not IAS(risky_level):
# 'block' is always False -> unconditional deny; 'middle'/'low'
# retained UNCHANGED.
logging.error(NS["SECURITY_MESSAGE_GENERAL"])
return web.Response(status=404, text=_SC_DENY_TEXT)
env.task_queue.append(("install", json_data.get("ui_id")))
return web.Response(status=200)
app = web.Application()
app.router.add_post("/manager/queue/install", sc_install)
return app
class _LogCapture(logging.Handler):
def __init__(self):
super().__init__()
self.messages = []
def emit(self, record):
self.messages.append(record.getMessage())
@contextlib.contextmanager
def _capture_logs():
handler = _LogCapture()
root = logging.getLogger()
old_level = root.level
root.addHandler(handler)
root.setLevel(logging.DEBUG)
try:
yield handler.messages
finally:
root.removeHandler(handler)
root.setLevel(old_level)
class SCInstallGateMirrorTest(unittest.TestCase):
"""Batch-install (S-C) gate composition via a slim handler mirror."""
def setUp(self):
self.loop = asyncio.new_event_loop()
asyncio.set_event_loop(self.loop)
def tearDown(self):
self.loop.close()
def _post(self, env, body):
_apply_env(env)
async def go():
server = TestServer(_make_sc_install(env))
client = TestClient(server)
await client.start_server()
try:
resp = await client.post("/manager/queue/install", json=body)
return resp.status, await resp.text()
finally:
await client.close()
return self.loop.run_until_complete(go())
def _installs(self, env):
return [item for item in env.task_queue if item[0] == "install"]
def _has(self, logs, const_name):
return any(NS[const_name] in m for m in logs)
def test_high_arm_allow(self):
env = GateEnv(git=True, listen="127.0.0.1")
with _capture_logs() as logs:
status, _ = self._post(env, _body_unknown())
self.assertEqual(status, 200)
self.assertEqual(len(self._installs(env)), 1)
self.assertFalse(self._has(logs, "SECURITY_MESSAGE_FLAG_GIT_URL"))
def test_high_arm_flag_deny(self):
env = GateEnv(git=False, listen="127.0.0.1")
with _capture_logs() as logs:
status, text = self._post(env, _body_unknown())
self.assertEqual(status, 404) # risky-position deny shape kept
self.assertIn("A security error has occurred", text)
self.assertEqual(self._installs(env), [])
self.assertTrue(self._has(logs, "SECURITY_MESSAGE_FLAG_GIT_URL"))
self.assertFalse(self._has(logs, "SECURITY_MESSAGE_NORMAL_MINUS"))
def test_load_bearing_public_canary(self):
"""Entry gate passes on a public listener; the deny MUST come from
the predicate's loopback term (the 'middle' set has no network
term). 404 (risky deny), not 403 (entry deny)."""
env = GateEnv(git=True, listen="0.0.0.0", security_level="normal")
with _capture_logs() as logs:
status, _ = self._post(env, _body_unknown())
self.assertEqual(status, 404)
self.assertEqual(self._installs(env), [])
self.assertFalse(
self._has(logs, "SECURITY_MESSAGE_MIDDLE_OR_BELOW"),
"entry gate must PASS here — deny must come from the predicate",
)
self.assertTrue(self._has(logs, "SECURITY_MESSAGE_FLAG_GIT_URL"))
def test_unknown_pip_block_unconditional(self):
"""Unknown-pip 'block' stays unconditional regardless of both flags
(catalog URL + non-catalog pip; URL check precedes pip check)."""
env = GateEnv(git=True, pip=True, listen="127.0.0.1")
body = _body_unknown(files=[CATALOG_URL], pip=[UNKNOWN_PIP])
with _capture_logs() as logs:
status, _ = self._post(env, body)
self.assertEqual(status, 404)
self.assertEqual(self._installs(env), [])
self.assertEqual(env.risky_calls, 1)
self.assertTrue(self._has(logs, "SECURITY_MESSAGE_GENERAL"))
self.assertEqual(env.flags.reads, [], "block arm must not consult the flags")
def test_entry_gate_strong_denies_despite_flags(self):
"""The security_level entry gate stays in force; flags do NOT bypass it."""
env = GateEnv(git=True, pip=True, listen="127.0.0.1", security_level="strong")
with _capture_logs() as logs:
status, _ = self._post(env, _body_unknown())
self.assertEqual(status, 403)
self.assertTrue(self._has(logs, "SECURITY_MESSAGE_MIDDLE_OR_BELOW"))
self.assertEqual(self._installs(env), [])
self.assertEqual(env.flags.reads, [], "entry deny must not consult the flags")
def test_cnr_latest_arm_never_consults_flags(self):
"""non-nightly CNR sets risky='low' statically — get_risky_level and
the flags are never consulted (false-pass guard)."""
env = GateEnv(git=False, pip=False, listen="127.0.0.1")
status, _ = self._post(env, _body_cnr_latest())
self.assertEqual(status, 200)
self.assertEqual(len(self._installs(env)), 1)
self.assertEqual(env.risky_calls, 0)
self.assertEqual(env.flags.reads, [], "flags must NOT be consulted on the CNR arm")
def test_middle_arm_retained(self):
"""all-catalog body -> risky='middle'; consults security_level
(UNCHANGED), not the flags."""
env = GateEnv(git=False, pip=False, listen="127.0.0.1")
body = _body_unknown(files=[CATALOG_URL], pip=[CATALOG_PIP])
status, _ = self._post(env, body)
self.assertEqual(status, 200)
self.assertEqual(len(self._installs(env)), 1)
self.assertEqual(env.risky_calls, 1)
self.assertEqual(env.flags.reads, [], "flags must NOT be consulted on the middle arm")
class DenialConstantsTest(unittest.TestCase):
"""Denial-copy honesty + security_403_response precedence (no server)."""
def _assert_honest_copy(self, const, flag_name):
self.assertIn(flag_name, const, "constant must name the responsible flag")
self.assertIn("config.ini", const, "constant must name config.ini")
for cause_phrasing in (
"is not allowed in this security_level",
"set the security level",
"a security_level of",
"security level configuration",
):
self.assertNotIn(cause_phrasing, const)
self.assertNotEqual(const, NS["SECURITY_MESSAGE_NORMAL_MINUS"])
self.assertNotEqual(const, NS["SECURITY_MESSAGE_GENERAL"])
def test_flag_constants_content(self):
self._assert_honest_copy(NS["SECURITY_MESSAGE_FLAG_GIT_URL"], "allow_git_url_install")
self._assert_honest_copy(NS["SECURITY_MESSAGE_FLAG_PIP"], "allow_pip_install")
def test_security_403_precedence(self):
"""outdated branch FIRST; flag_token names the flag; no-arg callers
stay byte-identical."""
self.assertIn("flag_token", inspect.signature(SEC_403).parameters)
NS["manager_migration"].system_user_api = False
try:
resp = SEC_403(flag_token="allow_git_url_install")
self.assertEqual(
json.loads(resp.text), {"error": "comfyui_outdated"},
"comfyui_outdated must take PRECEDENCE over flag_token",
)
finally:
NS["manager_migration"].system_user_api = True
resp = SEC_403(flag_token="allow_git_url_install")
self.assertEqual(json.loads(resp.text), {"error": "allow_git_url_install"})
resp = SEC_403()
self.assertEqual(
json.loads(resp.text), {"error": "security_level"},
"no-arg callers must stay byte-identical",
)
class BindingProofTest(unittest.TestCase):
"""AST proof that the REAL handlers are wired to the predicate (closes
the mirror-vs-real gap)."""
@staticmethod
def _ias_literal_calls(node):
out = []
for sub in ast.walk(node):
if (
isinstance(sub, ast.Call)
and isinstance(sub.func, ast.Name)
and sub.func.id == "is_allowed_security_level"
and sub.args
):
arg = sub.args[0]
out.append(arg.value if isinstance(arg, ast.Constant) else None)
return out
def test_handlers_bind_predicate(self):
"""S-A, S-B, S-C all gate via is_dedicated_install_allowed with the
right flag + args.listen (request-time); S-C keeps the 'middle'
entry gate and a variable-arg retained is_allowed_security_level path."""
for name, flag in (
("install_custom_node_git_url", "allow_git_url_install"),
("install_custom_node_pip", "allow_pip_install"),
("install_custom_node", "allow_git_url_install"),
):
with self.subTest(handler=name):
src = ast.unparse(HANDLERS[name])
self.assertIn("is_dedicated_install_allowed(", src)
self.assertIn(flag, src)
self.assertIn("args.listen", src)
sc_literals = self._ias_literal_calls(HANDLERS["install_custom_node"])
self.assertIn("middle", sc_literals, "entry gate must stay UNCHANGED")
self.assertIn(None, sc_literals, "a variable-arg retained path must remain")
def test_replace_no_high_literal_at_sa_sb(self):
"""REPLACE proof: no is_allowed_security_level('high') remains at
S-A / S-B (the flag fully replaced the old security_level gate)."""
for name in ("install_custom_node_git_url", "install_custom_node_pip"):
with self.subTest(handler=name):
self.assertNotIn(
"high", self._ias_literal_calls(HANDLERS[name]),
"%s still gates via is_allowed_security_level('high')" % name,
)
if __name__ == "__main__":
unittest.main(verbosity=2)
tests/test_install_flags_structural.py
+136
View File
@@ -0,0 +1,136 @@
"""Structural (grep/AST) guards for the dedicated install flags.
Cheap source-level guards that complement the behavioral tests:
- Frontend 403 copy: both install surfaces in js/common.js name their
responsible flag, and the generic fallback copy stays unchanged.
- No new HTTP install surface is added.
- cm-cli stays an ungated local operator tool.
- The migration module never references the flags (no auto-seed —
explicit opt-in only).
Harness: read/grep + AST over glob/*.py, cm-cli.py and js/*.js. No
imports of `glob/` modules (the dir name shadows stdlib glob).
"""
import re
import unittest
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
MANAGER_SERVER_PATH = REPO_ROOT / "glob" / "manager_server.py"
MANAGER_MIGRATION_PATH = REPO_ROOT / "glob" / "manager_migration.py"
CM_CLI_PATH = REPO_ROOT / "cm-cli.py"
JS_COMMON_PATH = REPO_ROOT / "js" / "common.js"
GENERIC_403_COPY = "This action is not allowed with this security level configuration."
FLAG_TOKENS = ("allow_git_url_install", "allow_pip_install")
def _js_function_block(source, func_name):
"""Slice an `export async function <name>` block (up to the next
export or EOF)."""
start = source.find("export async function %s" % func_name)
if start < 0:
raise AssertionError("function %s not found in js source" % func_name)
next_export = source.find("export ", start + 1)
return source[start: next_export if next_export > 0 else len(source)]
def _handle403_call_args(source):
"""All handle403Response(...) CALL argument strings (def/import lines
excluded)."""
calls = []
for match in re.finditer(r"handle403Response\s*\(([^()]*(?:\([^()]*\)[^()]*)*)\)", source):
line_start = source.rfind("\n", 0, match.start()) + 1
line = source[line_start: source.find("\n", match.start())]
if "function handle403Response" in line or line.lstrip().startswith("import"):
continue
calls.append(match.group(1).strip())
return calls
class JsCopyStructuralTest(unittest.TestCase):
"""Frontend honest-copy contract."""
@classmethod
def setUpClass(cls):
cls.common_src = JS_COMMON_PATH.read_text()
def test_surface_messages_name_their_flag(self):
"""Both install 403 branches pass a flag-naming defaultMessage."""
for func, flag in (
("install_via_git_url", "allow_git_url_install"),
("install_pip", "allow_pip_install"),
):
with self.subTest(func=func):
block = _js_function_block(self.common_src, func)
two_arg_calls = [a for a in _handle403_call_args(block) if "," in a]
self.assertTrue(
two_arg_calls,
"%s must call handle403Response with a defaultMessage" % func,
)
self.assertIn(flag, block)
self.assertIn("config.ini", block)
def test_generic_fallback_and_frozen_callers_unchanged(self):
"""The generic fallback copy stays (exactly its two occurrences in
handle403Response), and no other handle403Response caller across
js/ gains a defaultMessage."""
self.assertEqual(self.common_src.count(GENERIC_403_COPY), 2)
surface_blocks = "".join(
_js_function_block(self.common_src, name)
for name in ("install_pip", "install_via_git_url")
)
allowed_two_arg = {a for a in _handle403_call_args(surface_blocks) if "," in a}
for js_file in sorted((REPO_ROOT / "js").glob("*.js")):
source = js_file.read_text()
for args in _handle403_call_args(source):
if "," in args:
self.assertIn(
args, allowed_two_arg,
"frozen handle403Response caller in %s gained a "
"defaultMessage: handle403Response(%s)" % (js_file.name, args),
)
class StructuralSecurityGuardsTest(unittest.TestCase):
"""Source-level guards against scope bleed."""
def test_no_new_install_route_surface(self):
"""No new HTTP surface for git-URL/pip install."""
source = MANAGER_SERVER_PATH.read_text()
routes = set(re.findall(r"@routes\.post\(\"([^\"]+)\"\)", source))
expected_surfaces = {
"/customnode/install/git_url",
"/customnode/install/pip",
"/manager/queue/install",
"/manager/queue/reinstall",
}
self.assertTrue(expected_surfaces.issubset(routes))
install_like = {r for r in routes if "install" in r}
self.assertEqual(
install_like,
expected_surfaces
| {"/manager/queue/uninstall", "/manager/queue/install_model"},
"install-like route set drifted — no new install surface allowed",
)
def test_cm_cli_ungated(self):
"""cm-cli stays a local operator tool — no gate, no flag lookup."""
source = CM_CLI_PATH.read_text()
for token in FLAG_TOKENS + ("is_allowed_security_level", "is_dedicated_install_allowed"):
self.assertNotIn(token, source, "cm-cli.py must stay ungated")
def test_no_autoseed_in_migration(self):
"""The migration module never references the flags (explicit
opt-in only — no auto-seed from security_level)."""
source = MANAGER_MIGRATION_PATH.read_text()
for token in FLAG_TOKENS:
self.assertNotIn(
token, source,
"manager_migration.py must not seed/translate the new flags",
)
if __name__ == "__main__":
unittest.main(verbosity=2)