diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 00000000..60090316 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,57 @@ +# Changelog + +## Unreleased + +### Security policy: dedicated install flags (`allow_git_url_install` / `allow_pip_install`) + +Two new boolean keys in `config.ini` (`[default]` section), both defaulting to +`false`, now govern the arbitrary-install surfaces: + +| Flag | Governs | +|------|---------| +| `allow_git_url_install` | `POST /customnode/install/git_url` and the unknown-git-URL arm of `POST /manager/queue/install` (incl. reinstall delegation) — the entire install transaction, transitive dependency pip installs included. On the batch queue path the flag applies **in addition to** the queue's `security_level` entry gate (see below) | +| `allow_pip_install` | `POST /customnode/install/pip` only | + +These surfaces additionally require a **loopback listener** (`--listen` on a +loopback IP such as `127.0.0.1` or `::1` — not a general LAN/private address); +the flags never open a non-loopback deployment. On the two +**direct** endpoints (`POST /customnode/install/git_url` and +`POST /customnode/install/pip`), the flags fully **decouple** the surface +from `security_level`: it no longer has any effect in either direction — a +strict level cannot deny them when the flag is `true`, and a weak level +cannot allow them when the flag is `false`. On the **batch queue path** +(`POST /manager/queue/install`), the flag is **necessary but not +sufficient**: it gates the unknown-git-URL arm at the risky position, while +the queue's normal `security_level` entry gate (`middle`) remains in force — +at `security_level = strong`, batch unknown-URL installs stay denied even +with the flag set to `true`. `security_level` continues to govern every +other gated endpoint unchanged. Only the case-insensitive string `true` +enables a flag; a missing or malformed key reads as `false`. + +#### Migration note (no auto-seed) + +There is **no automatic migration** from `security_level`. Users who +previously relied on `security_level = weak` (or `normal-`) to use +install-via-git-URL / install-pip must now **opt in explicitly** by adding to +`config.ini`: + +```ini +[default] +allow_git_url_install = true +allow_pip_install = true +``` + +Changes take effect after a **restart** (no hot reload). + +#### Residual-risk note — outdated ComfyUI behavior change + +On outdated ComfyUI versions (no system-user API), the manager previously +forced `security_level = strong`, which unconditionally denied the +git-URL/pip install surfaces. After this change those surfaces are governed +by the new flags instead: an operator who explicitly sets a flag to `true` +on a **loopback** listener can now perform installs on outdated ComfyUI +where the forced-strong policy previously denied them. This is an accepted, +deliberate trade-off: it requires explicit operator opt-in, remains bounded +to loopback listeners, and the flag-deny path on outdated ComfyUI still +surfaces the `comfyui_outdated` notice. If you operate an outdated ComfyUI +deployment, leave both flags at their default `false` and update ComfyUI. diff --git a/README.md b/README.md index ea4919a7..7d2d01b9 100644 --- a/README.md +++ b/README.md @@ -384,19 +384,79 @@ When you run the `scan.sh` script: * all feature is available * `high` level risky features - * `Install via git url`, `pip install` - * Installation of custom nodes registered not in the `default channel`. - * Fix custom nodes + * Downloading models that are not in `.safetensors` format and not + registered in the `default channel` model list + * NOTE: `Install via git url`, `pip install`, and installation of custom nodes + not registered in the `default channel` are **no longer governed by + `security_level`** — they are governed by the dedicated install flags + described below. * `middle` level risky features * Uninstall/Update * Installation of custom nodes registered in the `default channel`. + * Fix custom nodes * Restore/Remove Snapshot * Restart * `low` level risky features * Update ComfyUI +### Dedicated install flags: `allow_git_url_install` / `allow_pip_install` + +The two arbitrary-install surfaces are governed by dedicated boolean keys in +`config.ini` (`[default]` section), fully **decoupled** from `security_level`: + + * `allow_git_url_install` + * governs `Install via Git URL` (`POST /customnode/install/git_url`) **and** + the unknown-git-URL arm of the batch install queue + (`POST /manager/queue/install`, including reinstall delegation) — i.e. + installing any custom node from a git URL that is not registered in the + `default channel` catalog + * on the **batch queue path**, the flag is **necessary but not + sufficient**: the queue's normal `security_level` entry gate (`middle`) + must ALSO pass — at `security_level = strong`, batch unknown-URL + installs stay denied even with the flag set to `true` (only the direct + `Install via Git URL` endpoint is fully independent of `security_level`) + * covers the **entire install transaction** it starts, including the + pack's transitive dependency pip installs + * `allow_pip_install` + * governs **only** the standalone `pip install` feature + (`POST /customnode/install/pip`) + +Key properties: + + * **Decoupled from `security_level` (replace, not and)** — on the two + **direct endpoints** (`Install via Git URL` and `pip install`), + `security_level` no longer has any effect in either direction: a strict + level cannot deny them when the flag is `true`, and a weak level cannot + allow them when the flag is `false`. (The batch queue path keeps its + `security_level` entry gate in ADDITION to the flag — see the scope bullet + above.) Every other gated feature remains governed by `security_level` as + described above. + * **Loopback only** — the flags take effect **only** when the server listens + on a loopback address (e.g. `--listen 127.0.0.1`). On a non-loopback + listener these surfaces stay denied regardless of the flags; the flags + never widen the exposure of a public deployment. + * **Default deny / explicit opt-in** — both flags default to `false`. Only + the case-insensitive string `true` enables a flag; a missing or malformed + key reads as `false`. + +To opt in, edit `config.ini`: + +```ini +[default] +allow_git_url_install = true +allow_pip_install = true +``` + +Changes take effect after a **restart** (no hot reload). + +> **Migration note**: there is no automatic migration from `security_level`. +> If you previously relied on `security_level = weak` (or `normal-`) to use +> install-via-git-URL / pip install, you must opt in explicitly with the flags +> above. See `CHANGELOG.md` for details, including a behavior note for +> outdated ComfyUI deployments. + # Disclaimer diff --git a/glob/manager_core.py b/glob/manager_core.py index 7c7e2fae..63ce1e4b 100644 --- a/glob/manager_core.py +++ b/glob/manager_core.py @@ -44,7 +44,7 @@ import manager_migration from node_package import InstalledNodePackage -version_code = [3, 40] +version_code = [3, 41] version_str = f"V{version_code[0]}.{version_code[1]}" + (f'.{version_code[2]}' if len(version_code) > 2 else '') @@ -1699,6 +1699,8 @@ def write_config(): 'always_lazy_install': get_config()['always_lazy_install'], 'network_mode': get_config()['network_mode'], 'db_mode': get_config()['db_mode'], + 'allow_git_url_install': get_config()['allow_git_url_install'], + 'allow_pip_install': get_config()['allow_pip_install'], } # Sanitize all string values to prevent CRLF injection attacks @@ -1745,6 +1747,8 @@ def read_config(): 'network_mode': default_conf.get('network_mode', 'public').lower(), 'security_level': default_conf.get('security_level', 'normal').lower(), 'db_mode': default_conf.get('db_mode', 'cache').lower(), + 'allow_git_url_install': get_bool('allow_git_url_install', False), + 'allow_pip_install': get_bool('allow_pip_install', False), } manager_migration.force_security_level_if_needed(result) return result @@ -1774,6 +1778,8 @@ def read_config(): 'network_mode': 'public', # public | private | offline 'security_level': 'normal', # strong | normal | normal- | weak 'db_mode': 'cache', # local | cache | remote + 'allow_git_url_install': False, + 'allow_pip_install': False, } manager_migration.force_security_level_if_needed(result) return result diff --git a/glob/manager_server.py b/glob/manager_server.py index 8474bb9c..069b0d8a 100644 --- a/glob/manager_server.py +++ b/glob/manager_server.py @@ -35,6 +35,8 @@ SECURITY_MESSAGE_MIDDLE_OR_BELOW = "ERROR: To use this action, a security_level SECURITY_MESSAGE_NORMAL_MINUS = "ERROR: To use this feature, you must either set '--listen' to a local IP and set the security level to 'normal-' or lower, or set the security level to 'middle' or 'weak'. Please contact the administrator.\nReference: https://github.com/ltdrdata/ComfyUI-Manager#security-policy" SECURITY_MESSAGE_GENERAL = "ERROR: This installation is not allowed in this security_level. Please contact the administrator.\nReference: https://github.com/ltdrdata/ComfyUI-Manager#security-policy" SECURITY_MESSAGE_NORMAL_MINUS_MODEL = "ERROR: Downloading models that are not in '.safetensors' format is only allowed for models registered in the 'default' channel at this security level. If you want to download this model, set the security level to 'normal-' or lower." +SECURITY_MESSAGE_FLAG_GIT_URL = "ERROR: This action requires 'allow_git_url_install = true' in config.ini ([default] section). This setting is independent of security_level. Reference: https://github.com/ltdrdata/ComfyUI-Manager#security-policy" +SECURITY_MESSAGE_FLAG_PIP = "ERROR: This action requires 'allow_pip_install = true' in config.ini ([default] section). This setting is independent of security_level. Reference: https://github.com/ltdrdata/ComfyUI-Manager#security-policy" routes = PromptServer.instance.routes @@ -82,6 +84,19 @@ def is_loopback(address): except ValueError: return False + +def is_dedicated_install_allowed(flag_value: bool, listen_address: str) -> bool: + """P-direct predicate (adopter-degraded form): flag AND loopback. + + Pure helper for the dedicated install flags + (allow_git_url_install / allow_pip_install) — callers pass the + flag value from their own config read and the listener address + from the CLI arguments (request-time evaluation; the import-time + snapshot above is NOT consulted). + """ + return bool(flag_value) and is_loopback(listen_address) + + is_local_mode = is_loopback(args.listen) @@ -305,10 +320,18 @@ import zipfile import urllib.request -def security_403_response(): - """Return appropriate 403 response based on ComfyUI version.""" +def security_403_response(flag_token=None): + """Return appropriate 403 response based on ComfyUI version. + + When `flag_token` is given (dedicated install flag denials), the + body names the responsible flag instead of "security_level". The + `comfyui_outdated` branch stays the FIRST check regardless, and + no-arg callers keep today's body byte-identical. + """ if not manager_migration.has_system_user_api(): return web.json_response({"error": "comfyui_outdated"}, status=403) + if flag_token is not None: + return web.json_response({"error": flag_token}, status=403) return web.json_response({"error": "security_level"}, status=403) @@ -1384,7 +1407,17 @@ async def install_custom_node(request): else: return web.Response(status=404, text=f"Following node pack doesn't provide `nightly` version: ${git_url}") - if not is_allowed_security_level(risky_level): + if risky_level == 'high': + # unknown-URL arm: governed by the dedicated flag predicate + # (flag AND loopback, evaluated at request time). The loopback + # term is load-bearing here — the 'middle' entry gate above has + # no network-position term. + if not is_dedicated_install_allowed(core.get_config()['allow_git_url_install'], args.listen): + logging.error(SECURITY_MESSAGE_FLAG_GIT_URL) + return web.Response(status=404, text="A security error has occurred. Please check the terminal logs") + elif not is_allowed_security_level(risky_level): + # 'block' arm stays an unconditional deny (is_allowed_security_level + # returns False for 'block'); 'middle'/'low' arms unchanged. logging.error(SECURITY_MESSAGE_GENERAL) return web.Response(status=404, text="A security error has occurred. Please check the terminal logs") @@ -1441,11 +1474,21 @@ async def fix_custom_node(request): @routes.post("/customnode/install/git_url") async def install_custom_node_git_url(request): - if not is_allowed_security_level('high'): - logging.error(SECURITY_MESSAGE_NORMAL_MINUS) - return security_403_response() + if not is_dedicated_install_allowed(core.get_config()['allow_git_url_install'], args.listen): + logging.error(SECURITY_MESSAGE_FLAG_GIT_URL) + return security_403_response(flag_token='allow_git_url_install') - url = await request.text() + # Read the body as JSON (not raw text): a cross-origin
+ # cannot forge an application/json body without a CORS preflight, which this + # server does not answer — same body-handler convention as every other + # request.json() route (see _reject_simple_form_content_type docstring). + # A malformed body or a missing 'url' is a client error (400), not a 500: + # don't let JSONDecodeError/KeyError bubble up as a server fault. + try: + json_data = await request.json() + url = json_data['url'] + except (KeyError, ValueError): + return web.Response(status=400, text="Invalid request body: expected JSON object with a 'url' field") res = await core.gitclone_install(url) if res.action == 'skip': @@ -1461,11 +1504,18 @@ async def install_custom_node_git_url(request): @routes.post("/customnode/install/pip") async def install_custom_node_pip(request): - if not is_allowed_security_level('high'): - logging.error(SECURITY_MESSAGE_NORMAL_MINUS) - return security_403_response() + if not is_dedicated_install_allowed(core.get_config()['allow_pip_install'], args.listen): + logging.error(SECURITY_MESSAGE_FLAG_PIP) + return security_403_response(flag_token='allow_pip_install') - packages = await request.text() + # JSON body (not raw text) for the same preflight-forcing reason as + # /customnode/install/git_url above; malformed body or missing 'packages' + # is a 400, not a 500. + try: + json_data = await request.json() + packages = json_data['packages'] + except (KeyError, ValueError): + return web.Response(status=400, text="Invalid request body: expected JSON object with a 'packages' field") core.pip_install(packages.split(' ')) return web.Response(status=200) diff --git a/js/common.js b/js/common.js index d51485da..a90855a2 100644 --- a/js/common.js +++ b/js/common.js @@ -223,16 +223,19 @@ function isValidURL(url) { } export async function install_pip(packages) { - if(packages.includes('&')) + if(packages.includes('&')) { app.ui.dialog.show(`Invalid PIP package enumeration: '${packages}'`); + return; + } const res = await api.fetchApi("/customnode/install/pip", { method: "POST", - body: packages, + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ packages: packages }), }); if(res.status == 403) { - await handle403Response(res); + await handle403Response(res, "To use this feature, set 'allow_pip_install = true' in config.ini ([default] section), then restart ComfyUI (the config is read once at startup). This setting is independent of security_level."); return; } @@ -263,11 +266,12 @@ export async function install_via_git_url(url, manager_dialog) { const res = await api.fetchApi("/customnode/install/git_url", { method: "POST", - body: url, + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ url: url }), }); if(res.status == 403) { - await handle403Response(res); + await handle403Response(res, "To use this feature, set 'allow_git_url_install = true' in config.ini ([default] section), then restart ComfyUI (the config is read once at startup). This setting is independent of security_level."); return; } diff --git a/pyproject.toml b/pyproject.toml index 8a154cef..8bc2ff14 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,8 +1,9 @@ [project] name = "comfyui-manager" description = "ComfyUI-Manager provides features to install and manage custom nodes for ComfyUI, as well as various functionalities to assist with ComfyUI." -version = "3.40" +version = "3.41" license = { file = "LICENSE.txt" } +requires-python = ">=3.9" dependencies = ["GitPython", "PyGithub", "matrix-nio", "transformers", "huggingface-hub>0.20", "typer", "rich", "typing-extensions", "toml", "uv", "chardet"] [project.urls] diff --git a/tests/conftest.py b/tests/conftest.py new file mode 100644 index 00000000..798d253e --- /dev/null +++ b/tests/conftest.py @@ -0,0 +1,41 @@ +"""Test-runner guard for the GOAL #32 tests/ modules. + +WHY THIS FILE EXISTS (collection hazard, not test logic): + +The repo root contains ``__init__.py`` — the ComfyUI plugin entrypoint — +which at import time appends ``glob/`` to sys.path and imports +``manager_server`` (which needs ``folder_paths`` / ``comfy.cli_args`` / +a constructed ``PromptServer``). pytest 8 collects any ancestor +directory that carries an ``__init__.py`` as a ``Package`` node and +IMPORTS that ``__init__.py`` during test setup (observed module name: +``__init__``). Outside a live ComfyUI process that import can never +succeed, so EVERY test under tests/ errors at setup — including the +pre-existing tests/test_csrf_content_type_helper.py — whenever pytest's +rootdir ends up at or above the repo root (e.g. running inside a git +worktree nested under the parent checkout). + +The guard below pre-seeds ``sys.modules`` with an inert stub whose +``__file__`` matches the real path, so pytest's +``import_path(/__init__.py)`` resolves to the stub without +executing the plugin entrypoint. Conftest files load before the setup +phase, so the stub is always in place in time. This does NOT touch +production code and does NOT alter what the tests import themselves +(they use AST-extraction / subprocess isolation per the +tests/test_csrf_content_type_helper.py precedent — ``glob/`` is never +added to the runner's sys.path). +""" +import sys +import types +from pathlib import Path + +_REPO_ROOT = Path(__file__).resolve().parent.parent +_ROOT_INIT = _REPO_ROOT / "__init__.py" + +if _ROOT_INIT.exists() and "__init__" not in sys.modules: + _stub = types.ModuleType("__init__") + _stub.__file__ = str(_ROOT_INIT) + _stub.__doc__ = ( + "Inert stand-in for the ComfyUI-Manager plugin entrypoint; " + "see tests/conftest.py for rationale." + ) + sys.modules["__init__"] = _stub diff --git a/tests/e2e/scripts/setup_e2e_env.sh b/tests/e2e/scripts/setup_e2e_env.sh new file mode 100755 index 00000000..778576bc --- /dev/null +++ b/tests/e2e/scripts/setup_e2e_env.sh @@ -0,0 +1,169 @@ +#!/usr/bin/env bash +# setup_e2e_env.sh — E2E environment builder for GOAL #60 (T1, spec §2). +# +# Builds the DISPOSABLE test ComfyUI root used by +# tests/e2e/test_e2e_install_flags.py. ENV BUILD ONLY: donor steps 4-5 +# (editable pip install of the Manager + custom_nodes symlink) are +# deliberately DROPPED — the Manager is mounted via `git worktree add` +# by the `mount_worktree` session fixture in the test module, which is +# the SOLE owner of mount create/reuse/teardown (spec §2 T1 +# single-ownership rule). This script never touches the Manager repo. +# +# Idempotent: re-run is a no-op when the marker + key artifacts exist +# (E2E-SC-01). +# +# Input env vars: +# E2E_COMFYUI_ROOT — target directory (default: mktemp -d) +# COMFYUI_BRANCH — ComfyUI clone ref (default: master; Q-2) +# PYTHON — python executable for version probe (default: python3) +# +# Output (last line of stdout): +# E2E_COMFYUI_ROOT=/path/to/environment +# +# Exit: 0=success, 1=failure + +set -euo pipefail + +COMFYUI_REPO="https://github.com/comfyanonymous/ComfyUI.git" +PYTORCH_CPU_INDEX="https://download.pytorch.org/whl/cpu" +# Minimal seed config. use_uv=false: the venv is seeded with pip +# (`uv venv --seed`), so the Manager's make_pip_cmd resolves to +# ` -m pip` and the suite's pip-uninstall hygiene helpers +# work without uv on PATH at server runtime. The install flags are NOT +# seeded here — stage_flags.sh stages them per launch identity (SC-06). +CONFIG_INI_CONTENT="[default] +file_logging = false +use_uv = false" + +log() { echo "[setup_e2e] $*"; } +err() { echo "[setup_e2e] ERROR: $*" >&2; } +die() { err "$@"; exit 1; } + +validate_prerequisites() { + local py="${PYTHON:-python3}" + local missing=() + command -v git >/dev/null 2>&1 || missing+=("git") + command -v uv >/dev/null 2>&1 || missing+=("uv") + command -v "$py" >/dev/null 2>&1 || missing+=("$py") + if [[ ${#missing[@]} -gt 0 ]]; then + die "Missing prerequisites: ${missing[*]}" + fi + local py_version major minor + py_version=$("$py" -c "import sys; print(f'{sys.version_info.major}.{sys.version_info.minor}')") + major="${py_version%%.*}" + minor="${py_version##*.}" + if [[ "$major" -lt 3 ]] || { [[ "$major" -eq 3 ]] && [[ "$minor" -lt 9 ]]; }; then + die "Python 3.9+ required, found $py_version" + fi + log "Prerequisites OK (python=$py_version)" +} + +check_already_setup() { + local root="$1" + if [[ -f "$root/.e2e_setup_complete" ]] \ + && [[ -d "$root/comfyui" ]] \ + && [[ -d "$root/venv" ]] \ + && [[ -f "$root/comfyui/user/__manager/config.ini" ]]; then + log "Environment already set up at $root (marker exists). Skipping. (E2E-SC-01 idempotence)" + echo "E2E_COMFYUI_ROOT=$root" + exit 0 + fi +} + +verify_setup() { + local root="$1" + local venv_py="$root/venv/bin/python" + local errors=0 + log "Running verification checks..." + [[ -f "$root/comfyui/main.py" ]] || { err "Verification FAIL: comfyui/main.py not found"; ((errors+=1)); } + [[ -x "$venv_py" ]] || { err "Verification FAIL: venv python not executable"; ((errors+=1)); } + [[ -f "$root/comfyui/user/__manager/config.ini" ]] || { err "Verification FAIL: config.ini not found"; ((errors+=1)); } + # venv must carry pip (uv venv --seed) — the suite's hygiene helpers + # and the Manager's reservation-consuming boot both call `-m pip`. + if ! "$venv_py" -m pip --version >/dev/null 2>&1; then + err "Verification FAIL: venv pip not available" + ((errors+=1)) + fi + # comfy is a local package inside the ComfyUI checkout + if ! PYTHONPATH="$root/comfyui" "$venv_py" -c "import comfy" 2>/dev/null; then + err "Verification FAIL: 'import comfy' failed" + ((errors+=1)) + fi + # [D2] half-check at setup time: the Manager must NOT be pip-installed + # into this venv (the worktree mount is the only delivery mechanism). + if "$venv_py" -m pip show comfyui-manager >/dev/null 2>&1 \ + || "$venv_py" -m pip show ComfyUI-Manager >/dev/null 2>&1; then + err "Verification FAIL: a pip-installed Manager exists in the venv (wrong layout for [D2])" + ((errors+=1)) + fi + if [[ "$errors" -gt 0 ]]; then + die "Verification failed with $errors error(s)" + fi + log "Verification OK: all checks passed" +} + +# ===== Main ===== +validate_prerequisites + +PYTHON="${PYTHON:-python3}" +COMFYUI_BRANCH="${COMFYUI_BRANCH:-master}" + +CREATED_BY_US=false +if [[ -z "${E2E_COMFYUI_ROOT:-}" ]]; then + E2E_COMFYUI_ROOT="$(mktemp -d -t e2e_comfyui_XXXXXX)" + CREATED_BY_US=true + log "Created E2E_COMFYUI_ROOT=$E2E_COMFYUI_ROOT" +else + mkdir -p "$E2E_COMFYUI_ROOT" + log "Using E2E_COMFYUI_ROOT=$E2E_COMFYUI_ROOT" +fi + +check_already_setup "$E2E_COMFYUI_ROOT" + +cleanup_on_failure() { + local exit_code=$? + if [[ "$exit_code" -ne 0 ]] && [[ "$CREATED_BY_US" == "true" ]]; then + err "Setup failed. Cleaning up $E2E_COMFYUI_ROOT" + rm -rf "$E2E_COMFYUI_ROOT" + fi +} +trap cleanup_on_failure EXIT + +log "Step 1/5: Cloning ComfyUI (branch=$COMFYUI_BRANCH)..." +if [[ -d "$E2E_COMFYUI_ROOT/comfyui/.git" ]]; then + log " ComfyUI already cloned, skipping" +else + git clone --depth=1 --branch "$COMFYUI_BRANCH" "$COMFYUI_REPO" "$E2E_COMFYUI_ROOT/comfyui" +fi + +log "Step 2/5: Creating virtual environment (seeded with pip)..." +if [[ -d "$E2E_COMFYUI_ROOT/venv" ]]; then + log " venv already exists, skipping" +else + uv venv --seed "$E2E_COMFYUI_ROOT/venv" +fi +VENV_PY="$E2E_COMFYUI_ROOT/venv/bin/python" + +log "Step 3/5: Installing ComfyUI dependencies (CPU-only torch index)..." +uv pip install \ + --python "$VENV_PY" \ + -r "$E2E_COMFYUI_ROOT/comfyui/requirements.txt" \ + --extra-index-url "$PYTORCH_CPU_INDEX" + +log "Step 4/5: Writing seed config.ini + HOME isolation dirs..." +mkdir -p "$E2E_COMFYUI_ROOT/comfyui/user/__manager" +echo "$CONFIG_INI_CONTENT" > "$E2E_COMFYUI_ROOT/comfyui/user/__manager/config.ini" +mkdir -p "$E2E_COMFYUI_ROOT/home/.config" +mkdir -p "$E2E_COMFYUI_ROOT/home/.local/share" +mkdir -p "$E2E_COMFYUI_ROOT/logs" +mkdir -p "$E2E_COMFYUI_ROOT/comfyui/custom_nodes" + +log "Step 5/5: Verifying setup..." +verify_setup "$E2E_COMFYUI_ROOT" + +# Marker written ONLY after verification passes (E2E-SC-01) +date -Iseconds > "$E2E_COMFYUI_ROOT/.e2e_setup_complete" + +trap - EXIT +log "Setup complete." +echo "E2E_COMFYUI_ROOT=$E2E_COMFYUI_ROOT" diff --git a/tests/e2e/scripts/stage_flags.sh b/tests/e2e/scripts/stage_flags.sh new file mode 100755 index 00000000..790908a2 --- /dev/null +++ b/tests/e2e/scripts/stage_flags.sh @@ -0,0 +1,82 @@ +#!/usr/bin/env bash +# stage_flags.sh — Per-launch-identity config staging (T4, spec §1.4). +# +# Analog of the donor's start_comfyui_strict.sh config patching, split +# out as a pure staging script (launch is a separate step; the pytest +# fixture owns restore+delete of the backup at teardown — donor +# symmetry, spec §1.4). +# +# Modes (arg $1): +# deny — REMOVE both flag keys (L-D: flags ABSENT also live-proves +# "missing key reads false") +# allow — set allow_git_url_install = true AND allow_pip_install = true +# (L-A / L-P) +# +# Backup: config.ini.before-flags, created ONLY if not already present +# (crashed-run-safe — preserves the true baseline across crashed runs). +# Restore + DELETE of the backup happens in the pytest fixture teardown, +# NOT here (E2E-SC-06/07). +# +# Input env vars: +# E2E_COMFYUI_ROOT — (required) +# +# Exit: 0=staged, 1=failure + +set -euo pipefail + +MODE="${1:-}" + +log() { echo "[stage_flags] $*"; } +err() { echo "[stage_flags] ERROR: $*" >&2; } +die() { err "$@"; exit 1; } + +[[ -n "${E2E_COMFYUI_ROOT:-}" ]] || die "E2E_COMFYUI_ROOT is not set" +[[ "$MODE" == "deny" || "$MODE" == "allow" ]] || die "usage: stage_flags.sh deny|allow" + +CONFIG="$E2E_COMFYUI_ROOT/comfyui/user/__manager/config.ini" +BACKUP="$CONFIG.before-flags" + +[[ -f "$CONFIG" ]] || die "config not found at $CONFIG (run setup_e2e_env.sh first)" + +# Backup ONLY if absent (crashed-run-safe; SC-06) +if [[ ! -f "$BACKUP" ]]; then + cp "$CONFIG" "$BACKUP" + log "Backed up original config to $BACKUP" +else + log "Backup already present at $BACKUP (preserving original baseline)" +fi + +stage_key() { + local key="$1" value="$2" + if grep -qE "^${key}\s*=" "$CONFIG"; then + sed -i -E "s|^${key}\s*=.*|${key} = ${value}|" "$CONFIG" + else + # The append-after target MUST exist, otherwise the sed below is a + # silent no-op and the flag never lands (false-PASS for the allow arm). + grep -qE "^\[default\]" "$CONFIG" \ + || die "no [default] section in $CONFIG — cannot stage '${key}' (would silently no-op)" + sed -i -E "/^\[default\]/a ${key} = ${value}" "$CONFIG" + fi +} + +remove_key() { + local key="$1" + sed -i -E "/^${key}\s*=/d" "$CONFIG" +} + +case "$MODE" in + deny) + remove_key "allow_git_url_install" + remove_key "allow_pip_install" + log "Staged DENY config (both flags ABSENT — missing key reads false)" + ;; + allow) + stage_key "allow_git_url_install" "true" + stage_key "allow_pip_install" "true" + log "Staged ALLOW config (both flags = true)" + ;; +esac + +# The staged value takes effect ONLY on the NEXT launch (restart-only +# cached_config — by construction, no hot-reload assumption; SC-06). +log "Staged config at $CONFIG (effective on next launch)" diff --git a/tests/e2e/scripts/start_comfyui.sh b/tests/e2e/scripts/start_comfyui.sh new file mode 100755 index 00000000..0860e2f0 --- /dev/null +++ b/tests/e2e/scripts/start_comfyui.sh @@ -0,0 +1,142 @@ +#!/usr/bin/env bash +# start_comfyui.sh — Foreground-blocking ComfyUI launcher (T2, spec §1.3). +# +# Starts ComfyUI in the background and blocks until the server answers +# GET /system_stats (or timeout). Deltas vs the donor script (binding, +# spec §1.3): +# - NO --enable-manager: the Manager is a custom-node-style plugin +# activated by ComfyUI's custom_nodes scan of the worktree mount. +# - NO COMFYUI_MANAGER_SKIP_MANAGER_REQUIREMENTS: that belt does not +# exist in this repo (Q-6 verified); watchdog row E2E-SC-42 covers +# the residual. +# - --listen is ALWAYS passed explicitly (LISTEN env): the predicate +# under test is request-time `flag AND is_loopback(args.listen)` — +# the listener value is LOAD-BEARING. +# - Per-launch log isolation (E2E-SC-04, MANDATORY): each launch +# identity writes a FRESH log file comfyui...log, +# so a deny-copy substring from L-D is unfindable by L-A/R-A log +# assertions (stale-substring false-PASS class). +# - Readiness = poll GET /system_stats == 200; child exit code 0 +# during the wait = Manager-triggered restart -> KEEP polling; +# non-zero exit -> fail fast with log tail. +# +# Input env vars: +# E2E_COMFYUI_ROOT — (required) root from setup_e2e_env.sh +# PORT — listen port (default: 8189) +# TIMEOUT — max seconds to readiness (default: 120) +# LISTEN — listener address (default: 127.0.0.1) +# LAUNCH_ID — launch identity tag for the log file (default: default) +# +# Output (last line on success): +# COMFYUI_PID= PORT= LOG_FILE= +# +# Exit: 0=ready, 1=timeout/failure + +set -euo pipefail + +PORT="${PORT:-8189}" +TIMEOUT="${TIMEOUT:-120}" +LISTEN="${LISTEN:-127.0.0.1}" +LAUNCH_ID="${LAUNCH_ID:-default}" + +log() { echo "[start_comfyui] $*"; } +err() { echo "[start_comfyui] ERROR: $*" >&2; } +die() { err "$@"; exit 1; } + +[[ -n "${E2E_COMFYUI_ROOT:-}" ]] || die "E2E_COMFYUI_ROOT is not set" +[[ -d "$E2E_COMFYUI_ROOT/comfyui" ]] || die "ComfyUI not found at $E2E_COMFYUI_ROOT/comfyui" +[[ -x "$E2E_COMFYUI_ROOT/venv/bin/python" ]] || die "venv python not found" +[[ -f "$E2E_COMFYUI_ROOT/.e2e_setup_complete" ]] || die "Setup marker not found. Run setup_e2e_env.sh first." + +PY="$E2E_COMFYUI_ROOT/venv/bin/python" +COMFY_DIR="$E2E_COMFYUI_ROOT/comfyui" +LOG_DIR="$E2E_COMFYUI_ROOT/logs" +LOG_FILE="$LOG_DIR/comfyui.${PORT}.${LAUNCH_ID}.log" +# Port-namespaced PID file (donor WI-CC incident: shared PID file caused +# a cross-port kill). +PID_FILE="$LOG_DIR/comfyui.${PORT}.pid" + +mkdir -p "$LOG_DIR" + +# --- Pre-launch port clear --- +if ss -tlnp 2>/dev/null | grep -q ":${PORT}\b"; then + log "Port $PORT is in use. Attempting to stop existing process..." + if [[ -f "$PID_FILE" ]]; then + OLD_PID="$(cat "$PID_FILE")" + if kill -0 "$OLD_PID" 2>/dev/null; then + kill "$OLD_PID" 2>/dev/null || true + sleep 2 + fi + fi + if ss -tlnp 2>/dev/null | grep -q ":${PORT}\b"; then + pkill -f "main\\.py.*--port $PORT" 2>/dev/null || true + sleep 2 + fi + if ss -tlnp 2>/dev/null | grep -q ":${PORT}\b"; then + die "Port $PORT is still in use after cleanup attempt" + fi + log "Port $PORT cleared." +fi + +# --- Launch (FRESH per-launch log file) --- +log "Starting ComfyUI on port $PORT (listen=$LISTEN, launch_id=$LAUNCH_ID)..." +: > "$LOG_FILE" + +PYTHONUNBUFFERED=1 \ +HOME="$E2E_COMFYUI_ROOT/home" \ + nohup "$PY" -u "$COMFY_DIR/main.py" \ + --cpu \ + --port "$PORT" \ + --listen "$LISTEN" \ + > "$LOG_FILE" 2>&1 & +COMFYUI_PID=$! + +echo "$COMFYUI_PID" > "$PID_FILE" +log "ComfyUI PID=$COMFYUI_PID, log=$LOG_FILE" + +# --- Block until ready: poll /system_stats (restart-tolerant) --- +log "Waiting up to ${TIMEOUT}s for ComfyUI readiness (GET /system_stats)..." +DEADLINE=$(( $(date +%s) + TIMEOUT )) +READY=0 +while [[ "$(date +%s)" -lt "$DEADLINE" ]]; do + if curl -sf --max-time 2 "http://127.0.0.1:${PORT}/system_stats" >/dev/null 2>&1; then + READY=1 + break + fi + # Child exit handling: exit 0 = Manager-triggered restart -> keep + # polling (a restarted process will bind the port); non-zero -> fail fast. + if ! kill -0 "$COMFYUI_PID" 2>/dev/null; then + if wait "$COMFYUI_PID" 2>/dev/null; then + : # exit 0 — restart class, keep polling + else + RC=$? + err "ComfyUI exited with code $RC. Last 30 lines of log:" + tail -n 30 "$LOG_FILE" >&2 + rm -f "$PID_FILE" + exit 1 + fi + fi + sleep 1 +done + +if [[ "$READY" -ne 1 ]]; then + err "Timeout (${TIMEOUT}s) waiting for ComfyUI. Last 30 lines of log:" + tail -n 30 "$LOG_FILE" >&2 + kill "$COMFYUI_PID" 2>/dev/null || true + rm -f "$PID_FILE" + exit 1 +fi + +# A Manager-triggered restart (child exit 0 above) leaves COMFYUI_PID pointing +# at the dead original child while a FRESH process now owns the port. Re-resolve +# the live listener PID so PID_FILE (consumed by stop_comfyui.sh) targets the +# process that must actually be killed at teardown. +LISTENER_PID="$(ss -tlnp 2>/dev/null | grep ":${PORT}\b" | grep -oE 'pid=[0-9]+' | head -n1 | cut -d= -f2)" +if [[ -n "$LISTENER_PID" && "$LISTENER_PID" != "$COMFYUI_PID" ]]; then + log "Listener PID $LISTENER_PID differs from launched PID $COMFYUI_PID (restart). Updating PID file." + COMFYUI_PID="$LISTENER_PID" + echo "$COMFYUI_PID" > "$PID_FILE" +fi + +log "ComfyUI is ready." +echo "COMFYUI_PID=$COMFYUI_PID PORT=$PORT LOG_FILE=$LOG_FILE" diff --git a/tests/e2e/scripts/stop_comfyui.sh b/tests/e2e/scripts/stop_comfyui.sh new file mode 100755 index 00000000..994ba5fa --- /dev/null +++ b/tests/e2e/scripts/stop_comfyui.sh @@ -0,0 +1,101 @@ +#!/usr/bin/env bash +# stop_comfyui.sh — Graceful ComfyUI shutdown (T3, spec §1.3 stop contract). +# +# Donor mirror: SIGTERM -> 10s grace -> SIGKILL -> port-pattern pkill +# fallback -> port-free verify (incl. the legacy-PID-file warning from +# the donor WI-CC cross-port-kill incident). +# +# Input env vars: +# E2E_COMFYUI_ROOT — (required) path to the E2E environment +# PORT — ComfyUI port (default: 8189) +# +# Exit: 0=stopped, 1=failed + +set -euo pipefail + +PORT="${PORT:-8189}" +GRACE_PERIOD=10 + +log() { echo "[stop_comfyui] $*"; } +err() { echo "[stop_comfyui] ERROR: $*" >&2; } +die() { err "$@"; exit 1; } + +[[ -n "${E2E_COMFYUI_ROOT:-}" ]] || die "E2E_COMFYUI_ROOT is not set" + +# Ownership guard: only signal a PID whose cmdline references THIS E2E root. +# On shared runners a bare "main.py --port N" pattern (or a reused PID) could +# otherwise match an unrelated process; the launcher invokes +# "$E2E_COMFYUI_ROOT/venv/bin/python $E2E_COMFYUI_ROOT/comfyui/main.py", so the +# root path is always present in our process's cmdline. +belongs_to_root() { + local pid="$1" + [[ -n "$pid" && -r "/proc/$pid/cmdline" ]] || return 1 + tr '\0' ' ' < "/proc/$pid/cmdline" 2>/dev/null | grep -qF "$E2E_COMFYUI_ROOT" +} + +# PIDs currently listening on PORT (deduped). +listener_pids() { + ss -tlnp 2>/dev/null | grep ":${PORT}\b" | grep -oE 'pid=[0-9]+' | cut -d= -f2 | sort -u +} + +PID_FILE="$E2E_COMFYUI_ROOT/logs/comfyui.${PORT}.pid" +LEGACY_PID_FILE="$E2E_COMFYUI_ROOT/logs/comfyui.pid" +if [[ -f "$LEGACY_PID_FILE" ]] && [[ ! -f "$PID_FILE" ]]; then + log "WARN: found legacy unported PID file $LEGACY_PID_FILE but no ${PID_FILE}. Cross-port risk — ignoring legacy file." +fi + +COMFYUI_PID="" +if [[ -f "$PID_FILE" ]]; then + COMFYUI_PID="$(cat "$PID_FILE")" + log "Read PID=$COMFYUI_PID from $PID_FILE" +fi + +if [[ -n "$COMFYUI_PID" ]] && kill -0 "$COMFYUI_PID" 2>/dev/null; then + if belongs_to_root "$COMFYUI_PID"; then + log "Sending SIGTERM to PID $COMFYUI_PID..." + kill "$COMFYUI_PID" 2>/dev/null || true + elapsed=0 + while kill -0 "$COMFYUI_PID" 2>/dev/null && [[ "$elapsed" -lt "$GRACE_PERIOD" ]]; do + sleep 1 + elapsed=$((elapsed + 1)) + done + if kill -0 "$COMFYUI_PID" 2>/dev/null; then + log "Process still alive after ${GRACE_PERIOD}s. Sending SIGKILL..." + kill -9 "$COMFYUI_PID" 2>/dev/null || true + sleep 1 + fi + else + log "WARN: PID $COMFYUI_PID does NOT belong to $E2E_COMFYUI_ROOT (reused/stale PID). Refusing to kill it." + fi +fi + +# Fallback: kill the port listener(s) (covers Manager-restarted processes whose +# PID differs from the recorded one) — but ONLY those verified to belong to this +# E2E root, never a broad pattern pkill that could hit unrelated CI processes. +if ss -tlnp 2>/dev/null | grep -q ":${PORT}\b"; then + log "Port $PORT still in use. Killing verified-own listener(s)..." + for pid in $(listener_pids); do + if belongs_to_root "$pid"; then + log "Sending SIGTERM to listener PID $pid..." + kill "$pid" 2>/dev/null || true + else + log "WARN: PID $pid on port $PORT is not part of $E2E_COMFYUI_ROOT — leaving it alone." + fi + done + sleep 2 + for pid in $(listener_pids); do + if belongs_to_root "$pid"; then + log "Listener PID $pid still alive. Sending SIGKILL..." + kill -9 "$pid" 2>/dev/null || true + fi + done + sleep 1 +fi + +rm -f "$PID_FILE" + +if ss -tlnp 2>/dev/null | grep -q ":${PORT}\b"; then + die "Port $PORT is still in use after shutdown" +fi + +log "ComfyUI stopped." diff --git a/tests/e2e/test_e2e_install_flags.py b/tests/e2e/test_e2e_install_flags.py new file mode 100644 index 00000000..106d6bcf --- /dev/null +++ b/tests/e2e/test_e2e_install_flags.py @@ -0,0 +1,752 @@ +"""GOAL #60 — Real-server E2E for the dedicated install flags (worktree-mounted Manager). + +Boots a REAL ComfyUI server from a disposable test root +(`E2E_COMFYUI_ROOT`, built by tests/e2e/scripts/setup_e2e_env.sh) with +the Manager mounted via `git worktree add --detach` (NEVER pip-installed +— [D2]) and exercises both dedicated-flag surfaces over live HTTP. + +Usage: + bash tests/e2e/scripts/setup_e2e_env.sh # once (E2E-SC-01) + E2E_COMFYUI_ROOT=/path/to/root pytest tests/e2e/test_e2e_install_flags.py -v + +Per-row map (goal60-scenarios.md, 24 rows — spec §3 BINDING): + SC-01 setup_e2e_env.sh (pre-suite script; idempotent build + marker — not a pytest test) + SC-02 mount_worktree fixture create path (SHA pin, .git-file, no-pip, printed SHA) + SC-03 mount_worktree fixture reuse path + path-prefix scoping invariant + SC-04 _start_server via start_comfyui.sh (readiness poll, restart tolerance, + per-launch log comfyui...log) + SC-05 test_00_smoke_manager_version in EVERY server-up class (+abort guard) + SC-06 _stage via stage_flags.sh (backup-if-absent; restart-only by construction) + SC-07 class fixture finalizers (stop + port-free + config restore + backup DELETE) + + mount_worktree finalizer (unmount + prune + absence assert) + SC-10 TestDenyArms.test_01_sa_deny SC-11 TestDenyArms.test_02_sb_deny + SC-12 TestAllowArms.test_01_git_url_allow + SC-13 TestAllowArms.test_02_pip_allow_reserved (anti-false-PASS — VERBATIM) + SC-14 TestAllowArms.test_03_restart_consumes_reservation (R-A through the holder) + SC-20 _pre_guards in both class fixtures (before any request) + SC-21 TestAllowArms.test_04_clone_residual_cleanup (+ installed-index cross-check) + SC-22 TestAllowArms.test_05_pip_residual_uninstall + SC-23 _reservation_guard — UNCONDITIONAL fixture-teardown guard (failure path) + SC-24 TestZeroResidual.test_99_zero_residual_sweep (unmount half lives in the + mount_worktree finalizer — it cannot be asserted from inside the session) + SC-30 module-level pytestmark (env unset -> all SKIP; unit suite unaffected) + SC-31 module-level pytestmark (marker absent -> all SKIP) + SC-32 needs_network marker on fixture-dependent (allow-arm / public) rows + SC-33 collection safety by construction: stdlib + pytest + requests + (via pytest.importorskip) ONLY — no glob/ imports, no server imports, + HTTP only at test time + SC-40 TestPublicListener (opt-in E2E_PUBLIC_LISTEN=1; L-P @ 0.0.0.0) + SC-41 batch S-C/S-C' E2E — DEFERRED (Q-5; spec FREEZE item 3; recorded here) + SC-42 TestAllowArms.test_06_requirements_watchdog (L-A launch log) + +Fixture-lifecycle ownership (spec §3 BINDING block): + - every class server fixture DECLARES mount_worktree (mount-before-launch); + - process handle lives in a MUTABLE ServerHolder owned by the fixture; + teardown stops the CURRENT holder content (whatever launch identity is live); + - SC-14 restarts THROUGH the holder (stop L-A -> launch R-A -> replace handle); + - stop-before-next-class is structural (pytest class-fixture scoping); + - `requests` is imported via pytest.importorskip (absence degrades to SKIP). + +T6 note: no tests/e2e/conftest.py — all fixtures are single-module, so the +optional T6 file is not demanded (spec §2 T6 condition not met). +""" +from __future__ import annotations + +import configparser +import os +import shutil +import socket +import subprocess +import sys +import time +import uuid +from pathlib import Path + +import pytest + +# --------------------------------------------------------------------------- +# Skip gates (E2E-SC-30/31) — BEFORE anything env-dependent +# --------------------------------------------------------------------------- + +E2E_ROOT = os.environ.get("E2E_COMFYUI_ROOT", "") +_MARKER_OK = bool(E2E_ROOT) and os.path.isfile(os.path.join(E2E_ROOT, ".e2e_setup_complete")) + +pytestmark = pytest.mark.skipif( + not _MARKER_OK, + reason="E2E_COMFYUI_ROOT not set or E2E environment not ready (.e2e_setup_complete missing)", +) + +# requests: test-extra — absence degrades to SKIP, never a collection error +# (spec §3 binding block item 5; [D4]). +requests = pytest.importorskip("requests") + +# --------------------------------------------------------------------------- +# Constants / paths +# --------------------------------------------------------------------------- + +PORT = int(os.environ.get("PORT", "8189")) +TIMEOUT = int(os.environ.get("TIMEOUT", "120")) +BASE_URL = f"http://127.0.0.1:{PORT}" + +THIS_DIR = Path(__file__).resolve().parent +SCRIPTS_DIR = THIS_DIR / "scripts" +MANAGER_REPO = THIS_DIR.parents[1] # repo root of the checkout running the suite + +ROOT = Path(E2E_ROOT) if E2E_ROOT else Path(".") +COMFY_DIR = ROOT / "comfyui" +CN_DIR = COMFY_DIR / "custom_nodes" +MOUNT = CN_DIR / "comfyui-manager" +CFG = COMFY_DIR / "user" / "__manager" / "config.ini" +CFG_BACKUP = Path(str(CFG) + ".before-flags") +SCRIPTS_FILE = COMFY_DIR / "user" / "__manager" / "startup-scripts" / "install-scripts.txt" +LOGS_DIR = ROOT / "logs" +VENV_PY = ROOT / "venv" / "bin" / "python" + +# Owned fixtures ONLY (goal60-scenarios.md Conventions; [D3]) +NODEPACK_URL = "https://github.com/ltdrdata/nodepack-test1-do-not-install" +PACK_NAME = "nodepack-test1-do-not-install" +# pip stimulus uses the git+ scheme: pip/uv require it for VCS URLs — a +# plain GitHub repo URL serves HTML and cannot install (verified by probe; +# spec amendment requested via leader pushback 2026-06-08; the SC-13/14 +# oracle itself is encoded VERBATIM). +PIP_URL = "git+https://github.com/ltdrdata/pip-test1-do-not-install" +PIP_PKG = "pip-test1-do-not-install" +PIP_IMPORT = "pip_test1_do_not_install" +PIP_MARKER = "pip-test1-do-not-install:ok" +# Amendment A2 (live-run finding, leader-approved): the S-A nodepack fixture +# is deliberately NOT zero-dep — it pins python-slugify==8.0.4 in its +# requirements as the invariant-4 ride-along proof vehicle. The SC-42 +# watchdog therefore allowlists exactly that requirement, and the +# transitive-dep residual class is swept at allow-class teardown + SC-24. +# (The S-B pip fixture IS zero-dep as documented.) +NODEPACK_PINNED_REQ = "python-slugify==8.0.4" +TRANSITIVE_DEPS = ("python-slugify", "text-unidecode") + +POLL_TIMEOUT = 60 +POLL_INTERVAL = 1.0 + +# Distinctive substrings of the flag-naming denial constants @ d45c8e6b +DENY_COPY_GIT = "'allow_git_url_install = true' in config.ini" +DENY_COPY_PIP = "'allow_pip_install = true' in config.ini" +# Old security_level-attributing copy (must NOT appear on flag denials) +OLD_COPY_GENERAL = "is not allowed in this security_level" +OLD_COPY_NORMAL_MINUS = "set the security level to" + + +# --------------------------------------------------------------------------- +# Network probe (E2E-SC-32) — evaluated ONLY when the env gate is open, so +# collection without the env performs no network IO (SC-33). +# --------------------------------------------------------------------------- + +def _network_available() -> bool: + try: + with socket.create_connection(("github.com", 443), timeout=5): + return True + except OSError: + return False + + +_NETWORK = _network_available() if _MARKER_OK else False +needs_network = pytest.mark.skipif( + not _NETWORK, + reason="github.com unreachable — network-dependent fixture row skipped (E2E-SC-32)", +) + + +# --------------------------------------------------------------------------- +# Helpers +# --------------------------------------------------------------------------- + +def _run(cmd, check=False, timeout=180, env=None, cwd=None): + return subprocess.run( + cmd, capture_output=True, text=True, timeout=timeout, check=check, + env=env, cwd=cwd, + ) + + +def _script(name: str) -> str: + return str(SCRIPTS_DIR / name) + + +def _script_env(**extra) -> dict: + env = {**os.environ, "E2E_COMFYUI_ROOT": str(ROOT), "PORT": str(PORT), + "TIMEOUT": str(TIMEOUT)} + env.update({k: str(v) for k, v in extra.items()}) + return env + + +def _pack_dir(name: str = PACK_NAME) -> Path: + return CN_DIR / name + + +def _pack_exists(name: str = PACK_NAME) -> bool: + return _pack_dir(name).is_dir() + + +def _remove_pack(name: str = PACK_NAME) -> None: + """Donor _remove_pack pattern: rmtree 3-retry + rename-to-.trash_ fallback.""" + path = _pack_dir(name) + if path.is_symlink(): + path.unlink() + return + if not path.is_dir(): + return + for attempt in range(3): + try: + shutil.rmtree(path) + return + except OSError: + if attempt < 2: + time.sleep(1) + trash = CN_DIR / f".trash_{uuid.uuid4().hex[:8]}" + try: + os.rename(path, trash) + shutil.rmtree(trash, ignore_errors=True) + except OSError: + shutil.rmtree(path, ignore_errors=True) + + +def _wait_for(predicate, timeout=POLL_TIMEOUT, interval=POLL_INTERVAL) -> bool: + deadline = time.monotonic() + timeout + while time.monotonic() < deadline: + if predicate(): + return True + time.sleep(interval) + return False + + +def _pip_import_rc() -> int: + return _run([str(VENV_PY), "-c", f"import {PIP_IMPORT}"]).returncode + + +def _pip_marker_rc() -> "subprocess.CompletedProcess": + return _run([ + str(VENV_PY), "-c", + f"import {PIP_IMPORT} as m; assert m.MARKER == '{PIP_MARKER}'", + ]) + + +def _pip_uninstall() -> "subprocess.CompletedProcess": + return _run([str(VENV_PY), "-m", "pip", "uninstall", "-y", PIP_PKG]) + + +def _scripts_clean() -> bool: + """True when the reservation file is absent OR carries no pip-test1 line.""" + if not SCRIPTS_FILE.exists(): + return True + return "pip-test1" not in SCRIPTS_FILE.read_text(errors="ignore") + + +def _reservation_guard() -> None: + """E2E-SC-23 — UNCONDITIONAL teardown guard for the unconsumed-reservation + leak class: a leaked line would pip-install on ANY next boot of this root.""" + if SCRIPTS_FILE.exists() and "pip-test1" in SCRIPTS_FILE.read_text(errors="ignore"): + SCRIPTS_FILE.unlink() + assert _scripts_clean(), "reservation guard failed to clear pip-test1 line (SC-23)" + + +def _restore_config() -> None: + """E2E-SC-07: restore from backup, then DELETE the backup and assert absence + (a surviving stale backup would silently restore an outdated config at a + FUTURE run's teardown via the create-only-if-absent rule).""" + if CFG_BACKUP.exists(): + shutil.copyfile(CFG_BACKUP, CFG) + CFG_BACKUP.unlink() + assert not CFG_BACKUP.exists(), "config backup must be DELETED after restore (SC-07/24)" + + +def _port_free() -> bool: + s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + try: + s.settimeout(1) + return s.connect_ex(("127.0.0.1", PORT)) != 0 + finally: + s.close() + + +def _pre_guards() -> None: + """E2E-SC-20 — before EACH arm's matrix rows; assert all three.""" + _remove_pack(PACK_NAME) + assert not _pack_exists(PACK_NAME), ( + f"pre-guard: failed to clean {PACK_NAME} (file locks?)" + ) + _pip_uninstall() # ignore rc: not-installed is fine + assert _pip_import_rc() != 0, "pre-guard: pip fixture importable before test" + _reservation_guard() + assert _scripts_clean(), "pre-guard: stale pip-test1 reservation present" + + +# --------------------------------------------------------------------------- +# Server lifecycle (E2E-SC-04 + spec §3 binding holder contract) +# --------------------------------------------------------------------------- + +class ServerHolder: + """Mutable process-handle holder owned by the class server fixture. + + The holder always points at the CURRENT launch identity; SC-14 replaces + its content when it restarts through it, so class teardown stops + whatever is live — no orphan.""" + + def __init__(self): + self.launch_id: str | None = None + self.log_path: Path | None = None + self.live = False + self.smoke_ok = False + + +def _stage(mode: str) -> None: + r = _run(["bash", _script("stage_flags.sh"), mode], env=_script_env(), check=False) + assert r.returncode == 0, f"stage_flags.sh {mode} failed:\n{r.stdout}\n{r.stderr}" + assert CFG_BACKUP.exists(), "backup must exist after staging (SC-06)" + + +def _start_server(holder: ServerHolder, launch_id: str, listen: str = "127.0.0.1") -> None: + r = _run( + ["bash", _script("start_comfyui.sh")], + env=_script_env(LISTEN=listen, LAUNCH_ID=launch_id), + timeout=TIMEOUT + 90, + ) + assert r.returncode == 0, ( + f"start_comfyui.sh failed for launch {launch_id}:\n{r.stdout}\n{r.stderr}" + ) + holder.launch_id = launch_id + holder.log_path = LOGS_DIR / f"comfyui.{PORT}.{launch_id}.log" + holder.live = True + assert holder.log_path.is_file(), "per-launch log file missing (SC-04)" + + +def _stop_server(holder: ServerHolder) -> None: + if not holder.live: + return + r = _run(["bash", _script("stop_comfyui.sh")], env=_script_env(), timeout=120) + assert r.returncode == 0, f"stop_comfyui.sh failed:\n{r.stdout}\n{r.stderr}" + holder.live = False + assert _port_free(), "port still bound after stop (SC-07)" + + +def _launch_log(holder: ServerHolder) -> str: + assert holder.log_path is not None and holder.log_path.is_file() + return holder.log_path.read_text(errors="ignore") + + +def _named_log(launch_id: str) -> str: + p = LOGS_DIR / f"comfyui.{PORT}.{launch_id}.log" + assert p.is_file(), f"launch log for {launch_id} missing" + return p.read_text(errors="ignore") + + +def _require_smoke(holder: ServerHolder) -> None: + """SC-05 abort semantics: matrix rows refuse to run after a smoke failure + so a mount/activation problem cannot produce misleading 404 results.""" + if not holder.smoke_ok: + pytest.fail( + "aborting matrix row: smoke (GET /manager/version) has not passed " + "for this launch — mount/activation problem or Q-2 bundled-manager " + "collision (E2E-SC-05)" + ) + + +def _smoke(holder: ServerHolder) -> None: + r = requests.get(f"{BASE_URL}/manager/version", timeout=10) + assert r.status_code == 200, ( + f"smoke FAILED: GET /manager/version -> {r.status_code}; the " + f"worktree-mounted plugin did not register its routes (E2E-SC-05). " + f"Log tail:\n{_launch_log(holder)[-2000:]}" + ) + assert r.text.strip(), "smoke: /manager/version body empty" + holder.smoke_ok = True + + +# --------------------------------------------------------------------------- +# Fixtures +# --------------------------------------------------------------------------- + +@pytest.fixture(scope="session") +def mount_worktree(): + """E2E-SC-02/03 — SOLE owner of mount create / reuse-verify / teardown.""" + ref = os.environ.get("E2E_MANAGER_REF", "HEAD") + r = _run(["git", "-C", str(MANAGER_REPO), "rev-parse", f"{ref}^{{commit}}"], check=True) + sha = r.stdout.strip() + + # Scoping invariant (SC-03): the mount path is {ROOT}-prefixed and is + # NEVER under the members' .claude/worktrees tree. Every mount/teardown + # command below references ONLY this path. + mount = MOUNT.resolve() + assert ".claude/worktrees" not in str(mount).replace(os.sep, "/"), ( + "mount path must never live under member worktrees (SC-03 scoping)" + ) + assert str(mount).startswith(str(ROOT.resolve())), ( + "mount path must be {ROOT}-prefixed (SC-03 scoping)" + ) + + porcelain = _run(["git", "-C", str(MANAGER_REPO), "worktree", "list", "--porcelain"]).stdout + if f"worktree {mount}" in porcelain: + # Reuse path (SC-03) + head = _run(["git", "-C", str(mount), "rev-parse", "HEAD"]).stdout.strip() + if head != sha: + _run(["git", "-C", str(mount), "checkout", "--detach", sha], check=True) + else: + # Create path (SC-02) + _run(["git", "-C", str(MANAGER_REPO), "worktree", "add", "--detach", + str(mount), sha], check=True) + + # From here a worktree exists at `mount`. Any failure between now and the + # yield (the verify asserts below) must STILL run teardown — otherwise a + # failed setup leaks an orphaned worktree into the next session (review + # follow-up). Hence the try/finally wraps verify + yield, not just yield. + try: + # Verify (every session) + head = _run(["git", "-C", str(mount), "rev-parse", "HEAD"]).stdout.strip() + assert head == sha, f"mount HEAD {head} != expected {sha} (SC-02)" + print(f"[mount_worktree] Manager mounted at {mount} @ SHA {sha}") # [D2] traceability + assert (mount / ".git").is_file(), ( + ".git in the mount must be a FILE (gitdir pointer) — worktree layout (SC-02)" + ) + # [D2] other half: no pip-installed Manager in the venv; per MM §2.2 no + # assertion anywhere relies on the mounted Manager's OWN version/remote + # self-report (.git-file degradation accepted by design — spec R5). + for dist in ("comfyui-manager", "ComfyUI-Manager"): + rc = _run([str(VENV_PY), "-m", "pip", "show", dist]).returncode + assert rc != 0, f"pip-installed Manager '{dist}' found in venv — violates [D2]" + + yield {"path": mount, "sha": sha} + finally: + if os.environ.get("E2E_KEEP_MOUNT"): + print(f"[mount_worktree] E2E_KEEP_MOUNT set — keeping {mount}") + else: + # Exception-safe: prune + absence asserts run even when remove fails + # (review iter-2 — crash residue must still be surfaced honestly). + try: + _run(["git", "-C", str(MANAGER_REPO), "worktree", "remove", "--force", str(mount)], + check=True) + finally: + _run(["git", "-C", str(MANAGER_REPO), "worktree", "prune"]) + porcelain = _run(["git", "-C", str(MANAGER_REPO), "worktree", "list", "--porcelain"]).stdout + assert f"worktree {mount}" not in porcelain, "mount still listed after remove (SC-07)" + assert not mount.exists(), "mount dir still present after remove (SC-07)" + + +@pytest.fixture(scope="class") +def deny_server(mount_worktree): + """L-D: both flags ABSENT (live 'missing key reads false'), loopback.""" + _pre_guards() # SC-20 + _stage("deny") # SC-06 + holder = ServerHolder() + _start_server(holder, "L-D") # SC-04 + yield holder + # Exception-safe teardown chain (review iter-2 must-fix): a failing + # stop is exactly the crashed-run shape SC-23 exists for — the guard + # and the config restore MUST run regardless. + try: + _stop_server(holder) # SC-07 (current handle, whatever is live) + finally: + try: + _reservation_guard() # SC-23 — UNCONDITIONAL + finally: + _restore_config() # SC-07: restore + DELETE backup + + +@pytest.fixture(scope="class") +def allow_server(mount_worktree): + """L-A: both flags true, loopback. SC-14 mutates the holder to R-A.""" + _pre_guards() # SC-20 (re-guards before the allow arm) + _stage("allow") # SC-06 + holder = ServerHolder() + _start_server(holder, "L-A") + yield holder + # Exception-safe teardown chain (review iter-2 must-fix): every + # residual guard runs even when the stop (or an earlier sweep step) + # raises — SC-23 is contractually UNCONDITIONAL (R3 leak class). + try: + _stop_server(holder) # stops the CURRENT identity (L-A or R-A) + finally: + try: + _remove_pack(PACK_NAME) # defensive re-sweep (primary assert is SC-21) + _pip_uninstall() # defensive (primary assert is SC-22) + # Amendment A2: sweep the S-A fixture's transitive-dep residual + # class (python-slugify + text-unidecode ride the git + # transaction; verified NOT in ComfyUI's own requirements). + _run([str(VENV_PY), "-m", "pip", "uninstall", "-y", *TRANSITIVE_DEPS]) + finally: + try: + _reservation_guard() # SC-23 — UNCONDITIONAL (failure path cover) + finally: + _restore_config() + + +@pytest.fixture(scope="class") +def public_server(mount_worktree): + """L-P (opt-in, Q-7): both flags true, 0.0.0.0 listener.""" + _pre_guards() + _stage("allow") + holder = ServerHolder() + _start_server(holder, "L-P", listen="0.0.0.0") + yield holder + # Exception-safe teardown chain (review iter-2 must-fix). + try: + _stop_server(holder) + finally: + try: + _reservation_guard() + finally: + _restore_config() + + +# --------------------------------------------------------------------------- +# Classes — definition order IS execution order (deny first on the fresh env) +# --------------------------------------------------------------------------- + +class TestDenyArms: + """L-D launch: SC-05 smoke, SC-10, SC-11 (deny rows are offline-safe — + denial happens before any network access).""" + + def test_00_smoke_manager_version(self, deny_server): + _smoke(deny_server) # SC-05 + + def test_01_sa_deny(self, deny_server): + """E2E-SC-10: S-A deny — 403 + exact flag token + no artifact + honest log.""" + _require_smoke(deny_server) + r = requests.post(f"{BASE_URL}/customnode/install/git_url", + json={"url": NODEPACK_URL}, timeout=30) + assert r.status_code == 403 + assert r.json() == {"error": "allow_git_url_install"}, ( + f"deny body must carry the flag token, got {r.text!r}" + ) + assert not _pack_exists(PACK_NAME), "clone artifact created on DENY (SC-10)" + log = _launch_log(deny_server) + assert DENY_COPY_GIT in log, "flag-naming denial copy missing from L-D log" + assert OLD_COPY_GENERAL not in log and OLD_COPY_NORMAL_MINUS not in log, ( + "denial attributed to security_level — honest-copy violation (SC-10)" + ) + + def test_02_sb_deny(self, deny_server): + """E2E-SC-11: S-B deny — 403 + flag token + no reservation + not importable.""" + _require_smoke(deny_server) + r = requests.post(f"{BASE_URL}/customnode/install/pip", + json={"packages": PIP_URL}, timeout=30) + assert r.status_code == 403 + assert r.json() == {"error": "allow_pip_install"} + assert _scripts_clean(), "reservation recorded on DENY (SC-11)" + assert _pip_import_rc() != 0, "pip fixture importable after DENY (SC-11)" + log = _launch_log(deny_server) + assert DENY_COPY_PIP in log, "flag-naming denial copy missing from L-D log" + + +class TestAllowArms: + """L-A launch + R-A restart. ORDERED methods (donor sequential-class + precedent): SC-12 -> SC-13 -> SC-14 -> SC-21 -> SC-22 -> SC-42.""" + + def test_00_smoke_manager_version(self, allow_server): + _smoke(allow_server) # SC-05 (re-smoke on the new launch) + + @needs_network + def test_01_git_url_allow(self, allow_server): + """E2E-SC-12: S-A allow — 200 + real clone + clone-target proof.""" + _require_smoke(allow_server) + r = requests.post(f"{BASE_URL}/customnode/install/git_url", + json={"url": NODEPACK_URL}, timeout=120) + assert r.status_code == 200, f"S-A allow expected 200, got {r.status_code}: {r.text!r}" + assert _wait_for(lambda: _pack_exists(PACK_NAME)), ( + f"{PACK_NAME} not cloned within {POLL_TIMEOUT}s (SC-12)" + ) + git_dir = _pack_dir() / ".git" + assert git_dir.is_dir(), "no .git DIRECTORY — not a real clone (SC-12)" + # Donor clone-target proof: .git/config [remote "origin"] url matches + # the requested URL modulo the .git suffix. + cp = configparser.ConfigParser() + cp.read(git_dir / "config") + section = 'remote "origin"' + assert section in cp, f'[{section}] missing from .git/config: {cp.sections()!r}' + remote_url = cp[section].get("url", "").rstrip("/") + expected = NODEPACK_URL.rstrip("/") + assert remote_url in (expected, expected + ".git"), ( + f"clone targeted the WRONG repo: {remote_url!r} != {expected!r} (SC-12)" + ) + + @needs_network + def test_02_pip_allow_reserved(self, allow_server): + """E2E-SC-13 (VERBATIM anti-false-PASS oracle): 200 = RESERVED, NOT + INSTALLED. Asserting import success here would be the exact false-PASS + the MM correction exists to prevent.""" + _require_smoke(allow_server) + r = requests.post(f"{BASE_URL}/customnode/install/pip", + json={"packages": PIP_URL}, timeout=30) + assert r.status_code == 200, f"S-B allow expected 200, got {r.status_code}: {r.text!r}" + assert SCRIPTS_FILE.is_file(), "no reservation file after S-B allow (SC-13)" + content = SCRIPTS_FILE.read_text(errors="ignore") + reserved_lines = [ + ln for ln in content.splitlines() + if "'#FORCE'" in ln and PIP_PKG in ln + ] + assert reserved_lines, ( + f"no reservation line with '#FORCE' + {PIP_PKG!r} in {SCRIPTS_FILE}:\n{content}" + ) + # MANDATORY: the package is NOT installed at this point. + assert _pip_import_rc() != 0, ( + "pip fixture importable right after the 200 — reservation semantics " + "violated, or a previous run leaked state (SC-13 anti-false-PASS)" + ) + + @needs_network + def test_03_restart_consumes_reservation(self, allow_server): + """E2E-SC-14: R-A restart THROUGH the holder; the consuming boot + executes + removes the reservation; MARKER import proves field-level.""" + _require_smoke(allow_server) + assert SCRIPTS_FILE.is_file(), "precondition: reservation must exist (SC-13 first)" + # Restart THROUGH the holder (spec §3 binding item 3): stop the live + # L-A process, relaunch as R-A with the SAME staged config, replace + # the handle — class teardown then stops R-A. + _stop_server(allow_server) + _start_server(allow_server, "R-A") + _smoke(allow_server) + # Field-level positive proof (not just exit code): + marker = _pip_marker_rc() + assert marker.returncode == 0, ( + f"MARKER import failed after the consuming restart (SC-14):\n" + f"{marker.stderr}\nR-A log tail:\n{_named_log('R-A')[-3000:]}" + ) + assert not SCRIPTS_FILE.exists(), ( + "install-scripts.txt NOT removed by the consuming boot (SC-14 self-clean)" + ) + ra_log = _named_log("R-A") + assert "## ComfyUI-Manager: EXECUTE =>" in ra_log and PIP_PKG in ra_log, ( + "R-A log lacks the startup-script execution block (SC-14)" + ) + assert "Startup script completed." in ra_log, ( + "R-A log lacks the startup-script completion line (SC-14)" + ) + + @needs_network + def test_04_clone_residual_cleanup(self, allow_server): + """E2E-SC-21: clone-dir hygiene; FS-absence primary + installed-index + cross-check while the server is still up (defensive, donor pattern).""" + _remove_pack(PACK_NAME) + assert not _pack_exists(PACK_NAME), "clone dir still present (SC-21 primary)" + try: + r = requests.get(f"{BASE_URL}/customnode/installed", timeout=15) + if r.status_code == 200: + installed = r.json() + assert PACK_NAME not in installed, ( + f"{PACK_NAME} still in /customnode/installed after removal (SC-21)" + ) + for key, pkg in installed.items(): + if isinstance(pkg, dict): + assert pkg.get("cnr_id") != PACK_NAME and pkg.get("aux_id") != PACK_NAME, ( + f"installed entry {key!r} still references {PACK_NAME!r} (SC-21)" + ) + except (ValueError, requests.RequestException): + # Spec SC-21: if the response schema proves awkward, FS-absence + # alone satisfies this row. + pass + + @needs_network + def test_05_pip_residual_uninstall(self, allow_server): + """E2E-SC-22: S-B residual class 1 (venv package).""" + r = _pip_uninstall() + assert r.returncode == 0, f"pip uninstall failed (SC-22):\n{r.stdout}\n{r.stderr}" + assert _pip_import_rc() != 0, "pip fixture importable after uninstall (SC-22)" + + @needs_network + def test_06_requirements_watchdog(self, allow_server): + """E2E-SC-42 (Q-6 watchdog, amendment A2): every management-script + EXECUTE in the L-A launch log must be attributable to the owned + fixture's OWN pinned requirements (python-slugify==8.0.4 — the + nodepack fixture's deliberate invariant-4 ride-along requirement). + Any other EXECUTE (e.g. a Manager-requirements install — the Q-6 + risk this row guards) FAILS the watchdog. + + The allowlisted line doubles as LIVE proof of the invariant-4 + ride-along class: a dependency pip install executed inside the + git-URL transaction without consulting allow_pip_install.""" + la_log = _named_log("L-A") + banner = "## ComfyUI-Manager: EXECUTE =>" + idx = 0 + execs = [] + while True: + idx = la_log.find(banner, idx) + if idx < 0: + break + execs.append(la_log[idx: idx + 600]) + idx += len(banner) + # Non-vacuity (review iter-2 / A2 positive half): the ride-along + # MUST have happened — exactly ONE management-script execution, + # the fixture's single pinned requirement. + assert len(execs) == 1, ( + f"expected exactly 1 management-script execution during L-A " + f"(the fixture's pinned requirement ride-along), found " + f"{len(execs)} (SC-42 / A2)" + ) + window = execs[0] + assert NODEPACK_PINNED_REQ in window, ( + "unexpected management-script execution during L-A — not " + "attributable to the fixture's pinned requirement " + f"({NODEPACK_PINNED_REQ}) (SC-42 watchdog):\n{window}" + ) + # Line-level shape: it must be a pip-install command, not an + # arbitrary script that merely mentions the requirement string. + assert "'pip'" in window and "'install'" in window, ( + f"EXECUTE block is not a pip-install command (SC-42):\n{window}" + ) + + +@pytest.mark.skipif( + not os.environ.get("E2E_PUBLIC_LISTEN"), + reason="public-listener row is opt-in (E2E_PUBLIC_LISTEN=1) — Q-7 default-off", +) +class TestPublicListener: + """E2E-SC-40 (opt-in): flags=true + 0.0.0.0 -> still 403 on both surfaces. + Live proof of invariant 2 (predicate = flag AND loopback at REQUEST time).""" + + def test_00_smoke_manager_version(self, public_server): + _smoke(public_server) + + def test_01_sa_public_deny(self, public_server): + _require_smoke(public_server) + r = requests.post(f"{BASE_URL}/customnode/install/git_url", + json={"url": NODEPACK_URL}, timeout=30) + assert r.status_code == 403 + assert r.json() == {"error": "allow_git_url_install"} + assert not _pack_exists(PACK_NAME) + + def test_02_sb_public_deny(self, public_server): + _require_smoke(public_server) + r = requests.post(f"{BASE_URL}/customnode/install/pip", + json={"packages": PIP_URL}, timeout=30) + assert r.status_code == 403 + assert r.json() == {"error": "allow_pip_install"} + assert _scripts_clean() + + +class TestZeroResidual: + """E2E-SC-24: the complete [D3] residual inventory in one assertion block. + Runs AFTER the server classes (their class-scoped fixtures have finalized: + server stopped, config restored, backup deleted). The unmount half of the + inventory is asserted by the mount_worktree finalizer itself — it cannot + be asserted from inside the session while the mount is still live.""" + + def test_99_zero_residual_sweep(self, mount_worktree): + # custom_nodes clean (incl. .trash_ fallback leftovers) + assert not _pack_exists(PACK_NAME), "nodepack residue in custom_nodes (SC-24)" + leftovers = [p.name for p in CN_DIR.iterdir() + if p.name.startswith((".trash_", PACK_NAME))] + assert not leftovers, f"residual entries in custom_nodes: {leftovers} (SC-24)" + # venv clean + assert _pip_import_rc() != 0, "pip fixture still importable (SC-24)" + # Amendment A2: transitive-dep residual class swept + for dep in TRANSITIVE_DEPS: + rc = _run([str(VENV_PY), "-m", "pip", "show", dep]).returncode + assert rc != 0, f"transitive dep {dep!r} survived the sweep (SC-24 / A2)" + # reservation clean + assert _scripts_clean(), "pip-test1 reservation residue (SC-24)" + # config restored + backup DELETED + assert CFG.is_file(), "config.ini missing after restore (SC-24)" + cfg_text = CFG.read_text(errors="ignore") + assert "allow_git_url_install" not in cfg_text, "staged flag leaked into restored config (SC-24)" + assert "allow_pip_install" not in cfg_text, "staged flag leaked into restored config (SC-24)" + assert not CFG_BACKUP.exists(), "stale config backup survived (SC-24 / peer R2)" + # port free + assert _port_free(), f"port {PORT} still bound (SC-24)" + + +if __name__ == "__main__": + sys.exit(pytest.main([__file__, "-v"])) diff --git a/tests/test_install_flag_predicate.py b/tests/test_install_flag_predicate.py new file mode 100644 index 00000000..510741df --- /dev/null +++ b/tests/test_install_flag_predicate.py @@ -0,0 +1,153 @@ +"""Unit tests for the dedicated-install-flag predicate. + +Covers `is_dedicated_install_allowed(flag_value, listen_address)` in +glob/manager_server.py: + + - Truth table: allowed iff flag is true AND the listener is loopback. + - REPLACE-by-construction: the 2-arg signature has no security_level / + network_mode parameter and the body references no config machinery, + so security_level cannot influence the outcome in either direction. + - Cross-flag isolation: a single flag_value input cannot consult the + other flag. + - Request-time evaluation: the body must not read the import-time + `is_local_mode` snapshot (callers pass args.listen per request). + +Harness: glob/manager_server.py is not importable under the test runner +(`from comfy.cli_args import args`, PromptServer), so we AST-parse the +file and exec only the wanted pure defs — `glob/` is never added to +sys.path (the dir name shadows the stdlib `glob`). +""" +import ast +import inspect +import unittest +from pathlib import Path +from typing import Any + +REPO_ROOT = Path(__file__).resolve().parent.parent +MANAGER_SERVER_PATH = REPO_ROOT / "glob" / "manager_server.py" + +_WANTED = {"is_loopback", "is_dedicated_install_allowed"} + + +def _load_predicates(): + """Parse manager_server.py; exec only the wanted pure function defs.""" + source = MANAGER_SERVER_PATH.read_text() + tree = ast.parse(source) + nodes = [] + node_by_name = {} + for node in tree.body: + if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)) and node.name in _WANTED: + nodes.append(node) + node_by_name[node.name] = node + missing = _WANTED - node_by_name.keys() + assert not missing, f"expected pure defs missing from manager_server.py: {missing}" + module = ast.Module(body=nodes, type_ignores=[]) + ns: dict = {"bool": bool} + exec(compile(module, "manager_server_predicates", "exec"), ns) + return ns, node_by_name + + +_NS, _NODES = _load_predicates() +IS_LOOPBACK: Any = _NS["is_loopback"] +PREDICATE: Any = _NS["is_dedicated_install_allowed"] +PREDICATE_NODE = _NODES["is_dedicated_install_allowed"] + + +class IsLoopbackBehaviorTest(unittest.TestCase): + """Pins the loopback term the predicate composes.""" + + def test_ipv4_loopback(self): + self.assertTrue(IS_LOOPBACK("127.0.0.1")) + + def test_public_address(self): + self.assertFalse(IS_LOOPBACK("0.0.0.0")) + + def test_ipv6_loopback(self): + self.assertTrue(IS_LOOPBACK("::1")) + + def test_invalid_address_reads_false(self): + # Non-IP strings deny-by-default (ValueError path). + self.assertFalse(IS_LOOPBACK("localhost")) + self.assertFalse(IS_LOOPBACK("")) + + +class DedicatedInstallPredicateTest(unittest.TestCase): + """P-direct truth table + REPLACE-by-construction.""" + + def test_truth_table(self): + """allowed iff flag AND loopback.""" + cases = [ + # (flag_value, listen_address, expected) + (True, "127.0.0.1", True), + (False, "127.0.0.1", False), + (True, "0.0.0.0", False), + (False, "0.0.0.0", False), + (True, "::1", True), + (True, "not-an-ip", False), # invalid listen -> deny + ] + for flag_value, listen, expected in cases: + with self.subTest(flag=flag_value, listen=listen): + result = PREDICATE(flag_value, listen) + self.assertIsInstance(result, bool) + self.assertEqual(result, expected) + + def test_falsy_flag_values_deny(self): + """Secure-by-default: any falsy flag never allows.""" + for falsy in (False, None, 0, ""): + with self.subTest(flag=falsy): + self.assertFalse(PREDICATE(falsy, "127.0.0.1")) + + def test_signature_has_no_security_level(self): + """Exactly (flag_value, listen_address) — no security_level term.""" + params = list(inspect.signature(PREDICATE).parameters) + self.assertEqual(params, ["flag_value", "listen_address"]) + for name in params: + self.assertNotIn("security", name) + self.assertNotIn("network_mode", name) + + def test_body_free_of_config_machinery(self): + """Body references no security_level plumbing, config reader, or the + import-time `is_local_mode` snapshot (request-time evaluation).""" + forbidden = { + "is_allowed_security_level", + "security_level", + "get_config", + "core", + "is_local_mode", + "network_mode", + "args", + } + seen = set() + for node in ast.walk(PREDICATE_NODE): + if isinstance(node, ast.Name): + seen.add(node.id) + elif isinstance(node, ast.Attribute): + seen.add(node.attr) + elif isinstance(node, ast.Constant) and isinstance(node.value, str): + seen.add(node.value) + self.assertEqual( + seen & forbidden, set(), + "predicate body must stay config-import-free", + ) + + def test_cross_flag_isolation_by_construction(self): + """A single flag_value input cannot consult the other flag.""" + seen_strings = { + node.value + for node in ast.walk(PREDICATE_NODE) + if isinstance(node, ast.Constant) and isinstance(node.value, str) + } + self.assertNotIn("allow_git_url_install", seen_strings) + self.assertNotIn("allow_pip_install", seen_strings) + self.assertTrue(PREDICATE(True, "127.0.0.1")) + self.assertFalse(PREDICATE(False, "127.0.0.1")) + + def test_purity_deterministic(self): + """Pure predicate — repeat calls identical.""" + for _ in range(3): + self.assertTrue(PREDICATE(True, "127.0.0.1")) + self.assertFalse(PREDICATE(True, "0.0.0.0")) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_install_flags_config.py b/tests/test_install_flags_config.py new file mode 100644 index 00000000..ac20bce3 --- /dev/null +++ b/tests/test_install_flags_config.py @@ -0,0 +1,242 @@ +"""Config-contract tests for the dedicated install flags. + +Drives the real glob/manager_core config reader/writer through a +subprocess-isolated harness and pins: missing keys read False +(secure-by-default), only case-insensitive "true" is truthy, write +round-trips losslessly, edits need a restart (cached_config), the +exception-fallback path supplies False, no auto-migration seeds the +flags from a legacy security_level, and the get_bool missing->False +quirk the flags rely on stays frozen. + +Harness: the child process injects a stub `folder_paths` (routing +import-time side effects into a tmpdir, and making has_system_user_api() +True so force_security_level_if_needed does not force 'strong'), prepends +`glob/` to ITS OWN sys.path (shadowing of stdlib `glob` confined to the +child), points manager_core.manager_config_path at a tmp config.ini, +resets cached_config, runs the scenario, and prints one JSON line for the +parent to assert. +""" +import json +import subprocess +import sys +import textwrap +import unittest +from pathlib import Path + +REPO_ROOT = Path(__file__).resolve().parent.parent + +_CHILD_PREAMBLE = textwrap.dedent( + """ + import sys, types, tempfile, os, json + tmp = tempfile.mkdtemp(prefix="cm_flags_cfg_") + stub = types.ModuleType("folder_paths") + stub.get_user_directory = lambda: tmp + stub.get_system_user_directory = lambda *a, **k: os.path.join(tmp, "sysuser") + sys.modules["folder_paths"] = stub + sys.path.insert(0, {glob_path!r}) + import manager_core + CONFIG_PATH = os.path.join(tmp, "config.ini") + manager_core.manager_config_path = CONFIG_PATH + manager_core.cached_config = None + + def write_ini(text): + with open(CONFIG_PATH, "w") as f: + f.write(text) + + def fresh_read(): + manager_core.cached_config = None + return manager_core.get_config() + + def flag_view(cfg): + return {{ + "git": cfg.get("allow_git_url_install", ""), + "pip": cfg.get("allow_pip_install", ""), + }} + """ +) + + +def _run_child(body): + """Run a scenario body in the isolated child; return its JSON payload.""" + script = _CHILD_PREAMBLE.format(glob_path=str(REPO_ROOT / "glob")) + textwrap.dedent(body) + proc = subprocess.run( + [sys.executable, "-c", script], + capture_output=True, + text=True, + timeout=180, + cwd=str(REPO_ROOT), + ) + if proc.returncode != 0: + raise AssertionError( + "config-harness child failed (rc=%d). stderr tail:\n%s" + % (proc.returncode, "\n".join(proc.stderr.strip().splitlines()[-8:])) + ) + lines = proc.stdout.strip().splitlines() + if not lines: + raise AssertionError( + "config-harness child exited 0 but produced no stdout. stderr tail:\n%s" + % "\n".join(proc.stderr.strip().splitlines()[-8:]) + ) + last_line = lines[-1] + try: + return json.loads(last_line) + except json.JSONDecodeError as e: + raise AssertionError( + "config-harness child emitted a non-JSON last line: %r\nfull stdout:\n%s" + % (last_line, proc.stdout) + ) from e + + +class InstallFlagsConfigContractTest(unittest.TestCase): + def test_sc17_missing_keys_read_false(self): + """Both keys absent from config.ini -> both flags read False + (secure-by-default).""" + payload = _run_child( + """ + write_ini("[default]\\nsecurity_level = normal\\n") + print(json.dumps(flag_view(fresh_read()))) + """ + ) + self.assertIs(payload["git"], False) + self.assertIs(payload["pip"], False) + + def test_sc18_malformed_and_case_matrix(self): + """Only case-insensitive "true" is truthy; malformed -> False.""" + payload = _run_child( + """ + out = {} + for raw in ["1", "yes", "TRUE", "true ", "true"]: + write_ini("[default]\\nallow_git_url_install = %s\\nallow_pip_install = %s\\n" % (raw, raw)) + cfg = fresh_read() + out[raw] = flag_view(cfg) + print(json.dumps(out)) + """ + ) + expected = { + "1": False, # malformed: numeric truthiness NOT honored + "yes": False, # malformed: yes/no NOT honored + "TRUE": True, # case-insensitive read (:1724) + "true ": True, # configparser strips surrounding whitespace + "true": True, + } + for raw, want in expected.items(): + with self.subTest(value=raw): + self.assertIs(payload[raw]["git"], want) + self.assertIs(payload[raw]["pip"], want) + + def test_sc19_write_round_trip(self): + """write_config persists str(bool); round-trip is lossless.""" + payload = _run_child( + """ + write_ini("[default]\\nsecurity_level = normal\\n") + cfg = fresh_read() + cfg["allow_git_url_install"] = True + cfg["allow_pip_install"] = False + manager_core.write_config() + raw = open(CONFIG_PATH).read() + reread = flag_view(fresh_read()) + print(json.dumps({ + "raw_has_git_true": "allow_git_url_install = True" in raw, + "raw_has_pip_false": "allow_pip_install = False" in raw, + "reread": reread, + })) + """ + ) + self.assertTrue( + payload["raw_has_git_true"], + "write_config must persist allow_git_url_install = True in [default]", + ) + self.assertTrue( + payload["raw_has_pip_false"], + "write_config must persist allow_pip_install = False in [default]", + ) + self.assertIs(payload["reread"]["git"], True) + self.assertIs(payload["reread"]["pip"], False) + + def test_sc20_restart_only_activation(self): + """Editing config.ini without restart has NO effect (cache wins); + a reset (== restart) picks up the change.""" + payload = _run_child( + """ + write_ini("[default]\\nallow_git_url_install = false\\n") + first = manager_core.get_config() # populates cached_config + before_edit = flag_view(first) + write_ini("[default]\\nallow_git_url_install = true\\n") + cached = flag_view(manager_core.get_config()) # NO reset: cache must win + after_restart = flag_view(fresh_read()) # reset == restart + print(json.dumps({ + "before_edit": before_edit, + "cached_after_edit": cached, + "after_restart": after_restart, + })) + """ + ) + self.assertIs(payload["before_edit"]["git"], False) + self.assertIs( + payload["cached_after_edit"]["git"], + False, + "cached_config must NOT hot-reload the edited flag", + ) + self.assertIs(payload["after_restart"]["git"], True) + + def test_sc21_exception_fallback_supplies_false(self): + """Corrupted config.ini -> exception-fallback dict supplies flags False.""" + payload = _run_child( + """ + # No [default] section header -> read_config raises inside try, + # lands in the exception-fallback dict. + write_ini("allow_git_url_install = true\\ngarbage without section\\n") + cfg = fresh_read() + print(json.dumps({ + "flags": flag_view(cfg), + "fallback_marker_file_logging": cfg.get("file_logging"), + })) + """ + ) + # file_logging True proves the FALLBACK dict was used (the parse + # path would yield False for a missing file_logging key). + self.assertIs( + payload["fallback_marker_file_logging"], + True, + "corrupted ini must route through the exception-fallback dict", + ) + self.assertIs(payload["flags"]["git"], False) + self.assertIs(payload["flags"]["pip"], False) + + def test_sc28_no_auto_migration_from_weak(self): + """Legacy `security_level=weak` does NOT seed the flags (no auto-migration).""" + payload = _run_child( + """ + write_ini("[default]\\nsecurity_level = weak\\n") + cfg = fresh_read() + print(json.dumps({ + "flags": flag_view(cfg), + "security_level": cfg.get("security_level"), + })) + """ + ) + self.assertEqual(payload["security_level"], "weak") + self.assertIs(payload["flags"]["git"], False, "no auto-seed from weak") + self.assertIs(payload["flags"]["pip"], False, "no auto-seed from weak") + + def test_sc42_get_bool_quirk_guard(self): + """get_bool ignores its default param: missing `file_logging` reads + False despite a True default. The flags rely on this missing->False + quirk; this guard pins it.""" + payload = _run_child( + """ + write_ini("[default]\\nsecurity_level = normal\\n") + cfg = fresh_read() + print(json.dumps({"file_logging": cfg.get("file_logging", "")})) + """ + ) + self.assertIs( + payload["file_logging"], + False, + "get_bool quirk changed: missing key no longer reads False — " + "new flags rely on missing->False", + ) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_install_flags_gates.py b/tests/test_install_flags_gates.py new file mode 100644 index 00000000..1dcac1ab --- /dev/null +++ b/tests/test_install_flags_gates.py @@ -0,0 +1,461 @@ +"""Handler-gate tests for the dedicated install flags. + +Three layers, each covering what the others can't: + + 1. SCInstallGateMirrorTest — a slim mirror of the batch-install handler + (`install_custom_node`, S-C) wired to the REAL extracted gate + primitives. Covers the handler *composition* that the pure predicate + test cannot: risky-level routing, the load-bearing public canary + (the entry gate has no network term, so the deny must come from the + predicate's loopback term), block-arm unconditionality, the + security_level entry gate, and the CNR/middle false-pass guards. + 2. DenialConstantsTest — content of the flag denial constants and the + `security_403_response` precedence, asserted directly (no server). + 3. BindingProofTest — AST proof that the REAL handlers (S-A/S-B/S-C) are + wired to the predicate and that the old `is_allowed_security_level('high')` + gate is gone from S-A/S-B (closes the mirror-vs-real gap). + +The direct S-A/S-B allow/deny behavior is covered by the binding proof +(wiring) plus the real-server E2E suite (behavior); the mirror here +focuses on S-C, whose multi-arm branching is the genuine logic risk. + +Harness: glob/manager_server.py is not importable under the runner +(`from comfy.cli_args import args`, PromptServer), so we AST-extract the +gate primitives and exec them into a stub namespace — `glob/` is never +added to sys.path (the dir name shadows stdlib glob). +""" +import ast +import asyncio +import contextlib +import inspect +import json +import logging +import unittest +from pathlib import Path +from typing import Any + +from aiohttp import web +from aiohttp.test_utils import TestClient, TestServer + +REPO_ROOT = Path(__file__).resolve().parent.parent +MANAGER_SERVER_PATH = REPO_ROOT / "glob" / "manager_server.py" + +_WANTED_FUNCS = { + "is_loopback", + "is_dedicated_install_allowed", + "is_allowed_security_level", + "security_403_response", +} +_WANTED_CONSTS = { + "SECURITY_MESSAGE_MIDDLE_OR_BELOW", + "SECURITY_MESSAGE_NORMAL_MINUS", + "SECURITY_MESSAGE_GENERAL", + "SECURITY_MESSAGE_FLAG_GIT_URL", + "SECURITY_MESSAGE_FLAG_PIP", +} +_HANDLER_NAMES = { + "install_custom_node_git_url", # S-A + "install_custom_node_pip", # S-B + "install_custom_node", # S-C +} + + +class _MigrationStub: + def __init__(self): + self.system_user_api = True + + def has_system_user_api(self): + return self.system_user_api + + +class _CoreStub: + """Stand-in for `core` consulted by is_allowed_security_level.""" + + def __init__(self): + self.security_level = "normal" + + def get_config(self): + return {"security_level": self.security_level} + + +def _load_surfaces(): + source = MANAGER_SERVER_PATH.read_text() + tree = ast.parse(source) + exec_nodes = [] + handler_nodes = {} + for node in tree.body: + if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)): + if node.name in _WANTED_FUNCS: + node.decorator_list = [] # exec needs no aiohttp routing context + exec_nodes.append(node) + if node.name in _HANDLER_NAMES: + handler_nodes[node.name] = node + elif isinstance(node, ast.Assign): + for target in node.targets: + if isinstance(target, ast.Name) and target.id in _WANTED_CONSTS: + exec_nodes.append(node) + module = ast.Module(body=exec_nodes, type_ignores=[]) + ns: dict = { + "web": web, + "bool": bool, + "manager_migration": _MigrationStub(), + "core": _CoreStub(), + "is_local_mode": True, + } + exec(compile(module, "manager_server_gate_surfaces", "exec"), ns) + # Feature is implemented — these must resolve, else the extraction or + # the production code regressed. + for name in _WANTED_FUNCS | _WANTED_CONSTS: + assert ns.get(name) is not None, "missing gate primitive: %s" % name + for name in _HANDLER_NAMES: + assert name in handler_nodes, "missing handler: %s" % name + return ns, handler_nodes + + +NS, HANDLERS = _load_surfaces() +PREDICATE: Any = NS["is_dedicated_install_allowed"] +IS_LOOPBACK: Any = NS["is_loopback"] +IAS: Any = NS["is_allowed_security_level"] +SEC_403: Any = NS["security_403_response"] + +CATALOG_URL = "https://github.com/catalog/listed-node" +UNKNOWN_URL = "https://github.com/x/not-in-catalog" +CATALOG_PIP = "torch" +UNKNOWN_PIP = "definitely-not-in-catalog-pkg" + + +def _body_unknown(files=None, pip=None): + """version=='unknown' ingestion arm.""" + return { + "version": "unknown", + "selected_version": "unknown", + "files": files or [UNKNOWN_URL], + "pip": pip or [], + "channel": "default", + "mode": "cache", + "ui_id": "test-row", + } + + +def _body_cnr_latest(): + """non-nightly CNR arm — risky='low' set statically.""" + return { + "version": "1.0.0", + "selected_version": "latest", + "id": "catalog-cnr-pack", + "channel": "default", + "mode": "cache", + "ui_id": "test-row", + } + + +class _TrackingFlags(dict): + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.reads = [] + + def __getitem__(self, key): + self.reads.append(key) + return super().__getitem__(key) + + +class GateEnv: + """Injectable per-row environment for the mirror app.""" + + def __init__(self, git=False, pip=False, listen="127.0.0.1", security_level="normal"): + self.flags = _TrackingFlags( + {"allow_git_url_install": git, "allow_pip_install": pip} + ) + self.listen = listen + self.security_level = security_level + self.task_queue = [] + self.risky_calls = 0 + self.catalog_urls = {CATALOG_URL} + self.catalog_pips = {CATALOG_PIP} + + def get_risky_level(self, files, pip_packages): + """Mirror of get_risky_level: URL check precedes pip check.""" + self.risky_calls += 1 + for x in files or []: + if x not in self.catalog_urls: + return "high" + for p in pip_packages or []: + if p not in self.catalog_pips: + return "block" + return "middle" + + +_SC_DENY_TEXT = "A security error has occurred. Please check the terminal logs" + + +def _apply_env(env): + """Retained gates use the is_local_mode snapshot + config stub.""" + NS["is_local_mode"] = IS_LOOPBACK(env.listen) + NS["core"].security_level = env.security_level + + +def _make_sc_install(env): + """Slim mirror of install_custom_node (S-C) in its post-change gate + shape: security_level entry gate, then risky-level routing where the + 'high' (unknown-URL) arm goes through the dedicated predicate and the + retained arms keep is_allowed_security_level.""" + + async def sc_install(request): + # ENTRY gate — UNCHANGED (security_level-governed) + if not IAS("middle"): + logging.error(NS["SECURITY_MESSAGE_MIDDLE_OR_BELOW"]) + return web.Response(status=403, text=_SC_DENY_TEXT) + json_data = await request.json() + risky_level = None + git_url = None + selected_version = json_data.get("selected_version") + if json_data["version"] != "unknown" and selected_version != "unknown": + if selected_version != "nightly": + risky_level = "low" # static — get_risky_level NOT called + else: + git_url = [json_data.get("repository")] + else: + git_url = json_data.get("files") + if risky_level is None: + risky_level = env.get_risky_level(git_url, json_data.get("pip", [])) + if risky_level == "high": + # unknown-URL arm -> dedicated predicate (flag AND loopback) + if not PREDICATE(env.flags["allow_git_url_install"], env.listen): + logging.error(NS["SECURITY_MESSAGE_FLAG_GIT_URL"]) + return web.Response(status=404, text=_SC_DENY_TEXT) + elif not IAS(risky_level): + # 'block' is always False -> unconditional deny; 'middle'/'low' + # retained UNCHANGED. + logging.error(NS["SECURITY_MESSAGE_GENERAL"]) + return web.Response(status=404, text=_SC_DENY_TEXT) + env.task_queue.append(("install", json_data.get("ui_id"))) + return web.Response(status=200) + + app = web.Application() + app.router.add_post("/manager/queue/install", sc_install) + return app + + +class _LogCapture(logging.Handler): + def __init__(self): + super().__init__() + self.messages = [] + + def emit(self, record): + self.messages.append(record.getMessage()) + + +@contextlib.contextmanager +def _capture_logs(): + handler = _LogCapture() + root = logging.getLogger() + old_level = root.level + root.addHandler(handler) + root.setLevel(logging.DEBUG) + try: + yield handler.messages + finally: + root.removeHandler(handler) + root.setLevel(old_level) + + +class SCInstallGateMirrorTest(unittest.TestCase): + """Batch-install (S-C) gate composition via a slim handler mirror.""" + + def setUp(self): + self.loop = asyncio.new_event_loop() + asyncio.set_event_loop(self.loop) + + def tearDown(self): + self.loop.close() + + def _post(self, env, body): + _apply_env(env) + + async def go(): + server = TestServer(_make_sc_install(env)) + client = TestClient(server) + await client.start_server() + try: + resp = await client.post("/manager/queue/install", json=body) + return resp.status, await resp.text() + finally: + await client.close() + + return self.loop.run_until_complete(go()) + + def _installs(self, env): + return [item for item in env.task_queue if item[0] == "install"] + + def _has(self, logs, const_name): + return any(NS[const_name] in m for m in logs) + + def test_high_arm_allow(self): + env = GateEnv(git=True, listen="127.0.0.1") + with _capture_logs() as logs: + status, _ = self._post(env, _body_unknown()) + self.assertEqual(status, 200) + self.assertEqual(len(self._installs(env)), 1) + self.assertFalse(self._has(logs, "SECURITY_MESSAGE_FLAG_GIT_URL")) + + def test_high_arm_flag_deny(self): + env = GateEnv(git=False, listen="127.0.0.1") + with _capture_logs() as logs: + status, text = self._post(env, _body_unknown()) + self.assertEqual(status, 404) # risky-position deny shape kept + self.assertIn("A security error has occurred", text) + self.assertEqual(self._installs(env), []) + self.assertTrue(self._has(logs, "SECURITY_MESSAGE_FLAG_GIT_URL")) + self.assertFalse(self._has(logs, "SECURITY_MESSAGE_NORMAL_MINUS")) + + def test_load_bearing_public_canary(self): + """Entry gate passes on a public listener; the deny MUST come from + the predicate's loopback term (the 'middle' set has no network + term). 404 (risky deny), not 403 (entry deny).""" + env = GateEnv(git=True, listen="0.0.0.0", security_level="normal") + with _capture_logs() as logs: + status, _ = self._post(env, _body_unknown()) + self.assertEqual(status, 404) + self.assertEqual(self._installs(env), []) + self.assertFalse( + self._has(logs, "SECURITY_MESSAGE_MIDDLE_OR_BELOW"), + "entry gate must PASS here — deny must come from the predicate", + ) + self.assertTrue(self._has(logs, "SECURITY_MESSAGE_FLAG_GIT_URL")) + + def test_unknown_pip_block_unconditional(self): + """Unknown-pip 'block' stays unconditional regardless of both flags + (catalog URL + non-catalog pip; URL check precedes pip check).""" + env = GateEnv(git=True, pip=True, listen="127.0.0.1") + body = _body_unknown(files=[CATALOG_URL], pip=[UNKNOWN_PIP]) + with _capture_logs() as logs: + status, _ = self._post(env, body) + self.assertEqual(status, 404) + self.assertEqual(self._installs(env), []) + self.assertEqual(env.risky_calls, 1) + self.assertTrue(self._has(logs, "SECURITY_MESSAGE_GENERAL")) + self.assertEqual(env.flags.reads, [], "block arm must not consult the flags") + + def test_entry_gate_strong_denies_despite_flags(self): + """The security_level entry gate stays in force; flags do NOT bypass it.""" + env = GateEnv(git=True, pip=True, listen="127.0.0.1", security_level="strong") + with _capture_logs() as logs: + status, _ = self._post(env, _body_unknown()) + self.assertEqual(status, 403) + self.assertTrue(self._has(logs, "SECURITY_MESSAGE_MIDDLE_OR_BELOW")) + self.assertEqual(self._installs(env), []) + self.assertEqual(env.flags.reads, [], "entry deny must not consult the flags") + + def test_cnr_latest_arm_never_consults_flags(self): + """non-nightly CNR sets risky='low' statically — get_risky_level and + the flags are never consulted (false-pass guard).""" + env = GateEnv(git=False, pip=False, listen="127.0.0.1") + status, _ = self._post(env, _body_cnr_latest()) + self.assertEqual(status, 200) + self.assertEqual(len(self._installs(env)), 1) + self.assertEqual(env.risky_calls, 0) + self.assertEqual(env.flags.reads, [], "flags must NOT be consulted on the CNR arm") + + def test_middle_arm_retained(self): + """all-catalog body -> risky='middle'; consults security_level + (UNCHANGED), not the flags.""" + env = GateEnv(git=False, pip=False, listen="127.0.0.1") + body = _body_unknown(files=[CATALOG_URL], pip=[CATALOG_PIP]) + status, _ = self._post(env, body) + self.assertEqual(status, 200) + self.assertEqual(len(self._installs(env)), 1) + self.assertEqual(env.risky_calls, 1) + self.assertEqual(env.flags.reads, [], "flags must NOT be consulted on the middle arm") + + +class DenialConstantsTest(unittest.TestCase): + """Denial-copy honesty + security_403_response precedence (no server).""" + + def _assert_honest_copy(self, const, flag_name): + self.assertIn(flag_name, const, "constant must name the responsible flag") + self.assertIn("config.ini", const, "constant must name config.ini") + for cause_phrasing in ( + "is not allowed in this security_level", + "set the security level", + "a security_level of", + "security level configuration", + ): + self.assertNotIn(cause_phrasing, const) + self.assertNotEqual(const, NS["SECURITY_MESSAGE_NORMAL_MINUS"]) + self.assertNotEqual(const, NS["SECURITY_MESSAGE_GENERAL"]) + + def test_flag_constants_content(self): + self._assert_honest_copy(NS["SECURITY_MESSAGE_FLAG_GIT_URL"], "allow_git_url_install") + self._assert_honest_copy(NS["SECURITY_MESSAGE_FLAG_PIP"], "allow_pip_install") + + def test_security_403_precedence(self): + """outdated branch FIRST; flag_token names the flag; no-arg callers + stay byte-identical.""" + self.assertIn("flag_token", inspect.signature(SEC_403).parameters) + NS["manager_migration"].system_user_api = False + try: + resp = SEC_403(flag_token="allow_git_url_install") + self.assertEqual( + json.loads(resp.text), {"error": "comfyui_outdated"}, + "comfyui_outdated must take PRECEDENCE over flag_token", + ) + finally: + NS["manager_migration"].system_user_api = True + resp = SEC_403(flag_token="allow_git_url_install") + self.assertEqual(json.loads(resp.text), {"error": "allow_git_url_install"}) + resp = SEC_403() + self.assertEqual( + json.loads(resp.text), {"error": "security_level"}, + "no-arg callers must stay byte-identical", + ) + + +class BindingProofTest(unittest.TestCase): + """AST proof that the REAL handlers are wired to the predicate (closes + the mirror-vs-real gap).""" + + @staticmethod + def _ias_literal_calls(node): + out = [] + for sub in ast.walk(node): + if ( + isinstance(sub, ast.Call) + and isinstance(sub.func, ast.Name) + and sub.func.id == "is_allowed_security_level" + and sub.args + ): + arg = sub.args[0] + out.append(arg.value if isinstance(arg, ast.Constant) else None) + return out + + def test_handlers_bind_predicate(self): + """S-A, S-B, S-C all gate via is_dedicated_install_allowed with the + right flag + args.listen (request-time); S-C keeps the 'middle' + entry gate and a variable-arg retained is_allowed_security_level path.""" + for name, flag in ( + ("install_custom_node_git_url", "allow_git_url_install"), + ("install_custom_node_pip", "allow_pip_install"), + ("install_custom_node", "allow_git_url_install"), + ): + with self.subTest(handler=name): + src = ast.unparse(HANDLERS[name]) + self.assertIn("is_dedicated_install_allowed(", src) + self.assertIn(flag, src) + self.assertIn("args.listen", src) + sc_literals = self._ias_literal_calls(HANDLERS["install_custom_node"]) + self.assertIn("middle", sc_literals, "entry gate must stay UNCHANGED") + self.assertIn(None, sc_literals, "a variable-arg retained path must remain") + + def test_replace_no_high_literal_at_sa_sb(self): + """REPLACE proof: no is_allowed_security_level('high') remains at + S-A / S-B (the flag fully replaced the old security_level gate).""" + for name in ("install_custom_node_git_url", "install_custom_node_pip"): + with self.subTest(handler=name): + self.assertNotIn( + "high", self._ias_literal_calls(HANDLERS[name]), + "%s still gates via is_allowed_security_level('high')" % name, + ) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_install_flags_structural.py b/tests/test_install_flags_structural.py new file mode 100644 index 00000000..20e7d89c --- /dev/null +++ b/tests/test_install_flags_structural.py @@ -0,0 +1,136 @@ +"""Structural (grep/AST) guards for the dedicated install flags. + +Cheap source-level guards that complement the behavioral tests: + + - Frontend 403 copy: both install surfaces in js/common.js name their + responsible flag, and the generic fallback copy stays unchanged. + - No new HTTP install surface is added. + - cm-cli stays an ungated local operator tool. + - The migration module never references the flags (no auto-seed — + explicit opt-in only). + +Harness: read/grep + AST over glob/*.py, cm-cli.py and js/*.js. No +imports of `glob/` modules (the dir name shadows stdlib glob). +""" +import re +import unittest +from pathlib import Path + +REPO_ROOT = Path(__file__).resolve().parent.parent +MANAGER_SERVER_PATH = REPO_ROOT / "glob" / "manager_server.py" +MANAGER_MIGRATION_PATH = REPO_ROOT / "glob" / "manager_migration.py" +CM_CLI_PATH = REPO_ROOT / "cm-cli.py" +JS_COMMON_PATH = REPO_ROOT / "js" / "common.js" + +GENERIC_403_COPY = "This action is not allowed with this security level configuration." +FLAG_TOKENS = ("allow_git_url_install", "allow_pip_install") + + +def _js_function_block(source, func_name): + """Slice an `export async function ` block (up to the next + export or EOF).""" + start = source.find("export async function %s" % func_name) + if start < 0: + raise AssertionError("function %s not found in js source" % func_name) + next_export = source.find("export ", start + 1) + return source[start: next_export if next_export > 0 else len(source)] + + +def _handle403_call_args(source): + """All handle403Response(...) CALL argument strings (def/import lines + excluded).""" + calls = [] + for match in re.finditer(r"handle403Response\s*\(([^()]*(?:\([^()]*\)[^()]*)*)\)", source): + line_start = source.rfind("\n", 0, match.start()) + 1 + line = source[line_start: source.find("\n", match.start())] + if "function handle403Response" in line or line.lstrip().startswith("import"): + continue + calls.append(match.group(1).strip()) + return calls + + +class JsCopyStructuralTest(unittest.TestCase): + """Frontend honest-copy contract.""" + + @classmethod + def setUpClass(cls): + cls.common_src = JS_COMMON_PATH.read_text() + + def test_surface_messages_name_their_flag(self): + """Both install 403 branches pass a flag-naming defaultMessage.""" + for func, flag in ( + ("install_via_git_url", "allow_git_url_install"), + ("install_pip", "allow_pip_install"), + ): + with self.subTest(func=func): + block = _js_function_block(self.common_src, func) + two_arg_calls = [a for a in _handle403_call_args(block) if "," in a] + self.assertTrue( + two_arg_calls, + "%s must call handle403Response with a defaultMessage" % func, + ) + self.assertIn(flag, block) + self.assertIn("config.ini", block) + + def test_generic_fallback_and_frozen_callers_unchanged(self): + """The generic fallback copy stays (exactly its two occurrences in + handle403Response), and no other handle403Response caller across + js/ gains a defaultMessage.""" + self.assertEqual(self.common_src.count(GENERIC_403_COPY), 2) + surface_blocks = "".join( + _js_function_block(self.common_src, name) + for name in ("install_pip", "install_via_git_url") + ) + allowed_two_arg = {a for a in _handle403_call_args(surface_blocks) if "," in a} + for js_file in sorted((REPO_ROOT / "js").glob("*.js")): + source = js_file.read_text() + for args in _handle403_call_args(source): + if "," in args: + self.assertIn( + args, allowed_two_arg, + "frozen handle403Response caller in %s gained a " + "defaultMessage: handle403Response(%s)" % (js_file.name, args), + ) + + +class StructuralSecurityGuardsTest(unittest.TestCase): + """Source-level guards against scope bleed.""" + + def test_no_new_install_route_surface(self): + """No new HTTP surface for git-URL/pip install.""" + source = MANAGER_SERVER_PATH.read_text() + routes = set(re.findall(r"@routes\.post\(\"([^\"]+)\"\)", source)) + expected_surfaces = { + "/customnode/install/git_url", + "/customnode/install/pip", + "/manager/queue/install", + "/manager/queue/reinstall", + } + self.assertTrue(expected_surfaces.issubset(routes)) + install_like = {r for r in routes if "install" in r} + self.assertEqual( + install_like, + expected_surfaces + | {"/manager/queue/uninstall", "/manager/queue/install_model"}, + "install-like route set drifted — no new install surface allowed", + ) + + def test_cm_cli_ungated(self): + """cm-cli stays a local operator tool — no gate, no flag lookup.""" + source = CM_CLI_PATH.read_text() + for token in FLAG_TOKENS + ("is_allowed_security_level", "is_dedicated_install_allowed"): + self.assertNotIn(token, source, "cm-cli.py must stay ungated") + + def test_no_autoseed_in_migration(self): + """The migration module never references the flags (explicit + opt-in only — no auto-seed from security_level).""" + source = MANAGER_MIGRATION_PATH.read_text() + for token in FLAG_TOKENS: + self.assertNotIn( + token, source, + "manager_migration.py must not seed/translate the new flags", + ) + + +if __name__ == "__main__": + unittest.main(verbosity=2)