mirror of
https://github.com/Comfy-Org/ComfyUI-Manager.git
synced 2026-06-20 14:59:22 +08:00
feat(security): dedicated install flags decouple git_url/pip install from security_level (#2962)
Some checks failed
Python Linting / Run Ruff (push) Has been cancelled
Some checks failed
Python Linting / Run Ruff (push) Has been cancelled
* feat(security): dedicated install flags decouple git_url/pip from security_level
Install via git URL and pip install are no longer gated by
security_level. Each surface gets a dedicated config.ini flag —
allow_git_url_install / allow_pip_install (both default false, secure
by default) — that fully REPLACES the security-level term for these two
features. The network-position invariant is retained: a non-local
listener stays denied regardless of the flags unless
network_mode = personal_cloud.
- New pure predicate is_dedicated_install_allowed() in
common/manager_security (no config access; callers resolve config)
- Legacy endpoints /v2/customnode/install/git_url and .../pip switch
from is_allowed_security_level('high+') to the flag gate; batch
installs of unknown git URLs likewise (middle+ entry gate unchanged,
unknown-pip 'block' stays unconditional; response shapes preserved)
- Config readers/writers (glob + legacy) parse and persist the flags;
denial logs and frontend 403 messages name the responsible flag and
note the non-local-listener requirement (network_mode=personal_cloud)
- No auto-seed from security_level — users previously on weak/normal-
must opt in explicitly (see CHANGELOG migration notes; README
documents the new contract)
- Update the pre-existing permissive E2E harness
(start_comfyui_permissive.sh + test_e2e_legacy_real_ops.py) to the
new contract: it now also sets allow_git_url_install /
allow_pip_install = true, since security_level = normal- alone no
longer opens the git_url/pip endpoints
Tests: predicate truth table proving security_level independence in
both directions, dual-reader config contract, security-level-matrix
freeze guards, legacy gate regression guards (121 unit), plus 22
real-server E2E tests incl. URL-form pip install with self-clean.
* test(e2e): fix fresh-env failures in customnode_info and git_clone harnesses
Two pre-existing harness defects that fail deterministically on a fresh
E2E environment (unrelated to the dedicated-install-flags change):
- test_e2e_customnode_info: TestInstalledPacks asserted the seed pack
ComfyUI_SigmoidOffsetScheduler is installed, but nothing seeded it —
the installing module (test_e2e_endpoint) runs alphabetically later
and uninstalls it at the end. Add a module-scoped autouse fixture
that installs the pack via cm-cli BEFORE the server starts (the
imported-mode test asserts against the startup-frozen snapshot, so
API-based seeding after boot cannot work) and removes it on teardown
only if the fixture installed it.
- test_e2e_git_clone: _ensure_cache ran cm-cli update-cache with a
120s timeout; the full DB download routinely exceeds that on slow
links, erroring the whole module at setup. Raise to 600s.
Verified from a fresh state (seed pack absent): both modules pass
(13 tests, incl. previously-failing TestInstalledPacks 2 and
TestNightlyInstallCycle 3).
This commit is contained in:
parent
8b98723b42
commit
fca7ef149d
25
CHANGELOG.md
25
CHANGELOG.md
@ -5,6 +5,31 @@ All notable changes to **ComfyUI-Manager** are documented in this file.
|
||||
The format is based on [Keep a Changelog 1.1.0](https://keepachangelog.com/en/1.1.0/),
|
||||
and this project adheres to [Semantic Versioning 2.0.0](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Security
|
||||
|
||||
- **Dedicated install flags decouple git-URL / pip installs from `security_level`**:
|
||||
`POST /v2/customnode/install/git_url` and `POST /v2/customnode/install/pip`
|
||||
(and the batch install path for git URLs not in the custom-node DB) are now
|
||||
gated by two new `config.ini` `[default]` flags — `allow_git_url_install`
|
||||
and `allow_pip_install` — instead of `security_level`. Both default to
|
||||
`false` (secure by default), and a non-loopback listener stays denied unless
|
||||
`network_mode = personal_cloud` (the existing network-position invariant is
|
||||
retained — the flags never widen exposure beyond what was possible before).
|
||||
`security_level` no longer has any effect on these two endpoints, in either
|
||||
direction. The unknown-pip-package block in batch installs remains
|
||||
unconditional. Activation requires a restart (no hot reload).
|
||||
|
||||
### Migration notes
|
||||
|
||||
- **Users running `security_level = weak` or `normal-`**: these environments
|
||||
could previously use the git-URL / pip install endpoints; after upgrading
|
||||
they are denied (HTTP 403) until you explicitly opt in by setting
|
||||
`allow_git_url_install = true` and/or `allow_pip_install = true` in the
|
||||
`[default]` section of `config.ini`. The flags are NOT auto-seeded from
|
||||
your `security_level` — explicit opt-in is intentional.
|
||||
|
||||
## [4.2.1] - 2026-04-22
|
||||
|
||||
Security-hardening release. Contains breaking-ish API changes for
|
||||
|
||||
20
README.md
20
README.md
@ -214,6 +214,8 @@ The following settings are applied based on the section marked as `is_default`.
|
||||
model_download_by_agent = <When downloading models, use an agent instead of torchvision_download_url.>
|
||||
downgrade_blacklist = <Set a list of packages to prevent downgrades. List them separated by commas.>
|
||||
security_level = <Set the security level => strong|normal|normal-|weak>
|
||||
allow_git_url_install = <Allow installing custom nodes from arbitrary git URLs. Independent of security_level. Default: False>
|
||||
allow_pip_install = <Allow installing arbitrary pip packages via the Manager. Independent of security_level. Default: False>
|
||||
always_lazy_install = <Whether to perform dependency installation on restart even in environments other than Windows.>
|
||||
network_mode = <Set the network mode => public|private|offline|personal_cloud>
|
||||
```
|
||||
@ -324,12 +326,14 @@ The security settings are applied based on whether the ComfyUI server's listener
|
||||
|
||||
| Risky Level | features |
|
||||
|-------------|---------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| high+ | * `Install via git url`, `pip install`<BR>* Installation of nodepack registered not in the `default channel`.<BR>* **Switch ComfyUI version**<BR>* **Fix nodepack** |
|
||||
| high+ | * **Switch ComfyUI version**<BR>* **Fix nodepack** |
|
||||
| high | _(no features at this tier — `Fix nodepack` promoted to `high+` to align the enforcement gate with the `SECURITY_MESSAGE_HIGH_P` log text)_ |
|
||||
| middle+ | * Uninstall/Update<BR>* Installation of nodepack registered in the `default channel`.<BR>* Restore/Remove Snapshot<BR>* Install model |
|
||||
| middle | * Restart |
|
||||
| low | * Update ComfyUI |
|
||||
|
||||
* **Note**: `Install via git url` and `pip install` are no longer gated by `security_level` — they moved to the dedicated flags `allow_git_url_install` / `allow_pip_install`. Installation of a nodepack registered not in the `default channel` likewise requires `allow_git_url_install` (in addition to the `middle+` level preconditions) instead of a `high+` security level. See the [Dedicated install flags](#dedicated-install-flags-allow_git_url_install--allow_pip_install) subsection below.
|
||||
|
||||
|
||||
### Security Level Table
|
||||
|
||||
@ -341,6 +345,20 @@ The security settings are applied based on whether the ComfyUI server's listener
|
||||
| weak | * All features are available | * All features are available | * `high+` and `middle+` level risky features are not allowed<BR>* `high`, `middle` and `low` level risky features are available
|
||||
|
||||
|
||||
### Dedicated install flags (`allow_git_url_install` / `allow_pip_install`)
|
||||
|
||||
The `Install via git url` and `pip install` features are governed by two dedicated `config.ini` flags instead of `security_level`:
|
||||
|
||||
* `allow_git_url_install`: Allows installing custom nodes from arbitrary git URLs.
|
||||
* `allow_pip_install`: Allows installing arbitrary pip packages via the Manager.
|
||||
|
||||
* Both flags default to `False` — secure by default. A missing or invalid value (anything other than `true`, case-insensitive) is read as `False`.
|
||||
* These flags fully **replace** `security_level` for these two features. `security_level` no longer affects them in either direction: a strict security level cannot deny them when the flag is `true`, and a weak security level cannot allow them when the flag is `false`.
|
||||
* The network-position rule still applies independently: even with a flag enabled, the feature is denied when the server listener is **non-local**, unless `network_mode = personal_cloud`.
|
||||
* Batch installs of git URLs not registered in the `default channel` are also gated by `allow_git_url_install`, and additionally require the normal batch-install preconditions (the `middle+` rules in the tables above). Unknown pip packages in batch installs remain blocked unconditionally — the flags do not open them.
|
||||
* Changes to these flags require a restart of ComfyUI to take effect.
|
||||
* Migration note: if you previously relied on `security_level = weak` or `normal-` to use these features, you must now opt in explicitly by setting the flags in the `[default]` section of `config.ini`. The flags are not auto-seeded from your `security_level`.
|
||||
|
||||
|
||||
# Disclaimer
|
||||
|
||||
|
||||
@ -83,6 +83,25 @@ def is_loopback(address):
|
||||
return False
|
||||
|
||||
|
||||
def is_dedicated_install_allowed(flag_value: bool, listen_address: str, network_mode: str) -> bool:
|
||||
"""P-direct predicate for the dedicated install flags (goal265-spec.md §1.2).
|
||||
|
||||
allowed iff flag AND (loopback listener OR network_mode == 'personal_cloud').
|
||||
|
||||
Gates the git-URL / standalone-pip install surfaces via the dedicated
|
||||
``allow_git_url_install`` / ``allow_pip_install`` config flags, fully
|
||||
decoupled from ``security_level`` (REPLACE, not AND — spec §1.1 inv. 1).
|
||||
The network-position term retains today's invariant that a public
|
||||
(non-loopback, non-personal_cloud) listener stays denied regardless of
|
||||
the flags (spec §1.1 inv. 2).
|
||||
|
||||
Pure function — NO config access; callers resolve ``flag_value`` and
|
||||
``network_mode`` through their own config reader and pass values in
|
||||
(preserves common/ layering: this module must stay config-import-free).
|
||||
"""
|
||||
return bool(flag_value) and (is_loopback(listen_address) or network_mode.lower() == 'personal_cloud')
|
||||
|
||||
|
||||
def do_nothing():
|
||||
pass
|
||||
|
||||
|
||||
@ -1706,6 +1706,8 @@ def write_config():
|
||||
'network_mode': get_config()['network_mode'],
|
||||
'db_mode': get_config()['db_mode'],
|
||||
'verbose': get_config()['verbose'],
|
||||
'allow_git_url_install': get_config()['allow_git_url_install'],
|
||||
'allow_pip_install': get_config()['allow_pip_install'],
|
||||
}
|
||||
|
||||
# Sanitize all string values to prevent CRLF injection attacks
|
||||
@ -1755,6 +1757,8 @@ def read_config():
|
||||
'security_level': default_conf.get('security_level', SecurityLevel.NORMAL.value).lower(),
|
||||
'db_mode': default_conf.get('db_mode', DBMode.CACHE.value).lower(),
|
||||
'verbose': get_bool('verbose', False),
|
||||
'allow_git_url_install': get_bool('allow_git_url_install', False),
|
||||
'allow_pip_install': get_bool('allow_pip_install', False),
|
||||
}
|
||||
|
||||
except Exception:
|
||||
@ -1783,6 +1787,8 @@ def read_config():
|
||||
'security_level': SecurityLevel.NORMAL.value,
|
||||
'db_mode': DBMode.CACHE.value,
|
||||
'verbose': False,
|
||||
'allow_git_url_install': False,
|
||||
'allow_pip_install': False,
|
||||
}
|
||||
|
||||
|
||||
|
||||
@ -216,7 +216,7 @@ export async function install_pip(packages) {
|
||||
});
|
||||
|
||||
if(res.status == 403) {
|
||||
show_message('This action is not allowed with this security level configuration.');
|
||||
show_message("To use this feature, set <code>allow_pip_install = true</code> in the [default] section of config.ini. This setting is independent of security_level.<BR>Note: if the ComfyUI listener is not local, <code>network_mode = personal_cloud</code> is also required.");
|
||||
return;
|
||||
}
|
||||
|
||||
@ -251,7 +251,7 @@ export async function install_via_git_url(url, manager_dialog) {
|
||||
});
|
||||
|
||||
if(res.status == 403) {
|
||||
show_message('This action is not allowed with this security level configuration.');
|
||||
show_message("To use this feature, set <code>allow_git_url_install = true</code> in the [default] section of config.ini. This setting is independent of security_level.<BR>Note: if the ComfyUI listener is not local, <code>network_mode = personal_cloud</code> is also required.");
|
||||
return;
|
||||
}
|
||||
|
||||
|
||||
@ -1690,6 +1690,8 @@ def write_config():
|
||||
'always_lazy_install': get_config()['always_lazy_install'],
|
||||
'network_mode': get_config()['network_mode'],
|
||||
'db_mode': get_config()['db_mode'],
|
||||
'allow_git_url_install': get_config()['allow_git_url_install'],
|
||||
'allow_pip_install': get_config()['allow_pip_install'],
|
||||
}
|
||||
|
||||
# Sanitize all string values to prevent CRLF injection attacks
|
||||
@ -1734,6 +1736,8 @@ def read_config():
|
||||
'network_mode': default_conf.get('network_mode', NetworkMode.PUBLIC.value).lower(),
|
||||
'security_level': default_conf.get('security_level', SecurityLevel.NORMAL.value).lower(),
|
||||
'db_mode': default_conf.get('db_mode', DBMode.CACHE.value).lower(),
|
||||
'allow_git_url_install': get_bool('allow_git_url_install', False),
|
||||
'allow_pip_install': get_bool('allow_pip_install', False),
|
||||
}
|
||||
|
||||
except Exception:
|
||||
@ -1757,6 +1761,8 @@ def read_config():
|
||||
'network_mode': NetworkMode.PUBLIC.value,
|
||||
'security_level': SecurityLevel.NORMAL.value,
|
||||
'db_mode': DBMode.CACHE.value,
|
||||
'allow_git_url_install': False,
|
||||
'allow_pip_install': False,
|
||||
}
|
||||
|
||||
|
||||
|
||||
@ -43,6 +43,8 @@ SECURITY_MESSAGE_HIGH_P = "ERROR: To use this action, '--listen' must be set to
|
||||
SECURITY_MESSAGE_NORMAL_MINUS = "ERROR: To use this feature, you must either set '--listen' to a local IP and set the security level to 'normal-' or lower, or set the security level to 'middle' or 'weak'. Please contact the administrator.\nReference: https://github.com/Comfy-Org/ComfyUI-Manager#security-policy"
|
||||
SECURITY_MESSAGE_GENERAL = "ERROR: This installation is not allowed in this security_level. Please contact the administrator.\nReference: https://github.com/Comfy-Org/ComfyUI-Manager#security-policy"
|
||||
SECURITY_MESSAGE_NORMAL_MINUS_MODEL = "ERROR: Downloading models that are not in '.safetensors' format is only allowed for models registered in the 'default' channel at this security level. If you want to download this model, set the security level to 'normal-' or lower."
|
||||
SECURITY_MESSAGE_FLAG_GIT_URL = "ERROR: This action requires 'allow_git_url_install = true' in config.ini ([default] section). This setting is independent of security_level. Please contact the administrator.\nReference: https://github.com/Comfy-Org/ComfyUI-Manager#security-policy"
|
||||
SECURITY_MESSAGE_FLAG_PIP = "ERROR: This action requires 'allow_pip_install = true' in config.ini ([default] section). This setting is independent of security_level. Please contact the administrator.\nReference: https://github.com/Comfy-Org/ComfyUI-Manager#security-policy"
|
||||
|
||||
routes = PromptServer.instance.routes
|
||||
|
||||
@ -122,6 +124,18 @@ def is_allowed_security_level(level):
|
||||
return True
|
||||
|
||||
|
||||
def _dedicated_install_allowed(flag_key: str) -> bool:
|
||||
"""goal265: P-direct gate for the dedicated install flags (spec §1.2).
|
||||
|
||||
allowed iff config[flag_key] AND (loopback listener OR
|
||||
network_mode == 'personal_cloud') — fully decoupled from security_level.
|
||||
Resolves config through the LEGACY reader and delegates the pure
|
||||
predicate to common/manager_security (which stays config-import-free).
|
||||
"""
|
||||
return manager_security.is_dedicated_install_allowed(
|
||||
core.get_config()[flag_key], args.listen, core.get_config()['network_mode'])
|
||||
|
||||
|
||||
async def get_risky_level(files, pip_packages):
|
||||
json_data1 = await core.get_data_by_mode('local', 'custom-node-list.json')
|
||||
json_data2 = await core.get_data_by_mode('cache', 'custom-node-list.json', channel_url='https://raw.githubusercontent.com/Comfy-Org/ComfyUI-Manager/main')
|
||||
@ -1473,7 +1487,15 @@ async def _install_custom_node(json_data):
|
||||
else:
|
||||
return web.Response(status=404, text=f"Following node pack doesn't provide `nightly` version: ${git_url}")
|
||||
|
||||
if not is_allowed_security_level(risky_level):
|
||||
# goal265 S-C (middle+ entry gate above UNCHANGED): unknown git URL ('high+')
|
||||
# -> dedicated-flag full predicate replaces the security_level check (spec §1.2);
|
||||
# unknown pip ('block') -> unconditional deny via is_allowed_security_level (Q1).
|
||||
# Flag-deny PRESERVES today's 404 response shape at this position (R1).
|
||||
if risky_level == 'high+':
|
||||
if not _dedicated_install_allowed('allow_git_url_install'):
|
||||
logging.error(SECURITY_MESSAGE_FLAG_GIT_URL)
|
||||
return web.Response(status=404, text="A security error has occurred. Please check the terminal logs")
|
||||
elif not is_allowed_security_level(risky_level):
|
||||
logging.error(SECURITY_MESSAGE_GENERAL)
|
||||
return web.Response(status=404, text="A security error has occurred. Please check the terminal logs")
|
||||
|
||||
@ -1527,8 +1549,9 @@ async def _fix_custom_node(json_data):
|
||||
|
||||
@routes.post("/v2/customnode/install/git_url")
|
||||
async def install_custom_node_git_url(request):
|
||||
if not is_allowed_security_level('high+'):
|
||||
logging.error(SECURITY_MESSAGE_NORMAL_MINUS)
|
||||
# goal265 S-A: dedicated-flag gate, decoupled from security_level (spec §1.2).
|
||||
if not _dedicated_install_allowed('allow_git_url_install'):
|
||||
logging.error(SECURITY_MESSAGE_FLAG_GIT_URL)
|
||||
return web.Response(status=403)
|
||||
|
||||
url = await request.text()
|
||||
@ -1547,8 +1570,9 @@ async def install_custom_node_git_url(request):
|
||||
|
||||
@routes.post("/v2/customnode/install/pip")
|
||||
async def install_custom_node_pip(request):
|
||||
if not is_allowed_security_level('high+'):
|
||||
logging.error(SECURITY_MESSAGE_NORMAL_MINUS)
|
||||
# goal265 S-B: dedicated-flag gate, decoupled from security_level (spec §1.2).
|
||||
if not _dedicated_install_allowed('allow_pip_install'):
|
||||
logging.error(SECURITY_MESSAGE_FLAG_PIP)
|
||||
return web.Response(status=403)
|
||||
|
||||
packages = await request.text()
|
||||
|
||||
177
tests/_install_flags_testutil.py
Normal file
177
tests/_install_flags_testutil.py
Normal file
@ -0,0 +1,177 @@
|
||||
"""Shared test-support for the goal265 dedicated-install-flag test modules.
|
||||
|
||||
NOT a test module (no ``test_`` prefix — pytest does not collect it).
|
||||
|
||||
Provides minimal runtime stubs so ``comfyui_manager`` package modules can be
|
||||
imported in a plain dev venv (outside a real ComfyUI runtime):
|
||||
|
||||
- ``comfy.cli_args.args`` — consumed by ``comfyui_manager/__init__.py`` at
|
||||
package import time (``from comfy.cli_args import args``). The stub pins
|
||||
``listen='127.0.0.1'`` (loopback) which matches the default precondition of
|
||||
every scenario row (goal265-scenarios.md §0 preamble).
|
||||
- ``folder_paths`` — consumed by ``comfyui_manager/common/context.py`` at
|
||||
import time to derive the manager user directory. The stub redirects it to
|
||||
a throwaway temp dir so importing the package NEVER creates directories
|
||||
inside the repository checkout.
|
||||
|
||||
SUITE-ORDER INDEPENDENCE (goal265 FU, task #293): every public ``import_*``
|
||||
entry point is self-sufficient — it repairs whatever ``sys.modules`` state
|
||||
earlier test modules left behind, instead of assuming a pristine interpreter:
|
||||
|
||||
- Other test modules legitimately install FAKE ``comfyui_manager`` lineage
|
||||
entries at module level (e.g. tests/test_unified_dep_resolver.py:63-73
|
||||
registers a plain ``types.ModuleType("comfyui_manager")`` — NOT a package,
|
||||
no ``__path__`` — to host a file-loaded module under a dotted name). Those
|
||||
modules are imported at pytest COLLECTION time, so under full-suite
|
||||
ordering the fake is already in ``sys.modules`` before any fixture here
|
||||
runs, and ``import comfyui_manager.glob.manager_core`` dies with
|
||||
``ModuleNotFoundError: 'comfyui_manager' is not a package``.
|
||||
- ``_purge_fake_comfyui_manager()`` therefore validates the cached lineage
|
||||
against the REAL package directory on disk and drops fake/broken entries
|
||||
before importing. Dropping ``sys.modules`` entries is safe for the other
|
||||
test modules: they hold direct object references to what they loaded;
|
||||
they do not re-resolve through ``sys.modules``.
|
||||
- ``ensure_comfy_stubs()`` likewise repairs half-formed ``comfy`` /
|
||||
``folder_paths`` entries (present but missing the attributes we need)
|
||||
rather than only handling the absent case.
|
||||
|
||||
Used by:
|
||||
- tests/test_install_flags_config.py (dual-reader config tests)
|
||||
- tests/test_install_flags_guards.py (out-of-scope guard tests)
|
||||
|
||||
Spec SoT: ~/.claude/pair-cowork/scratch/gm-team/goal265-spec.md §4
|
||||
(test surface contract — binding for Step-4 test authoring).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import types
|
||||
|
||||
#: repo root (parent of tests/)
|
||||
REPO_ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))
|
||||
|
||||
#: the real on-disk package directory the cached lineage must resolve to
|
||||
_REAL_PKG_DIR = os.path.join(REPO_ROOT, "comfyui_manager")
|
||||
|
||||
|
||||
def ensure_comfy_stubs() -> None:
|
||||
"""Install or REPAIR minimal ``comfy``/``folder_paths`` stubs (idempotent).
|
||||
|
||||
Handles three prior states per module:
|
||||
- absent -> install a fresh stub
|
||||
- present but inadequate -> patch the missing attributes in place
|
||||
(another test's stub, or a partially-initialized real module)
|
||||
- present and adequate -> leave untouched (e.g. real ComfyUI runtime)
|
||||
"""
|
||||
# --- comfy.cli_args.args ------------------------------------------------
|
||||
comfy = sys.modules.get("comfy")
|
||||
if comfy is None:
|
||||
comfy = types.ModuleType("comfy")
|
||||
sys.modules["comfy"] = comfy
|
||||
|
||||
cli_args = sys.modules.get("comfy.cli_args")
|
||||
if cli_args is None:
|
||||
cli_args = getattr(comfy, "cli_args", None)
|
||||
if not isinstance(cli_args, types.ModuleType):
|
||||
cli_args = types.ModuleType("comfy.cli_args")
|
||||
sys.modules["comfy.cli_args"] = cli_args
|
||||
|
||||
args = getattr(cli_args, "args", None)
|
||||
if args is None or not hasattr(args, "listen"):
|
||||
setattr(cli_args, "args", types.SimpleNamespace(
|
||||
listen="127.0.0.1",
|
||||
enable_manager=True,
|
||||
enable_manager_legacy_ui=False,
|
||||
))
|
||||
# re-link parent attr idempotently (a bare ``comfy`` stub from another
|
||||
# test may lack the submodule attribute even when both entries exist)
|
||||
setattr(comfy, "cli_args", cli_args)
|
||||
|
||||
# --- folder_paths -------------------------------------------------------
|
||||
fp = sys.modules.get("folder_paths")
|
||||
if fp is None:
|
||||
fp = types.ModuleType("folder_paths")
|
||||
sys.modules["folder_paths"] = fp
|
||||
if not hasattr(fp, "get_system_user_directory"):
|
||||
base = tempfile.mkdtemp(prefix="cm-flags-test-user-")
|
||||
setattr(fp, "get_system_user_directory",
|
||||
lambda name: os.path.join(base, name))
|
||||
|
||||
|
||||
def _module_is_real(mod: types.ModuleType) -> bool:
|
||||
"""True when a cached ``comfyui_manager*`` module resolves to the real
|
||||
on-disk package tree (by ``__path__`` for packages, ``__file__`` for
|
||||
leaf modules)."""
|
||||
real_root = os.path.abspath(_REAL_PKG_DIR)
|
||||
paths = getattr(mod, "__path__", None)
|
||||
if paths is not None:
|
||||
return any(os.path.abspath(p).startswith(real_root) for p in paths)
|
||||
file = getattr(mod, "__file__", None)
|
||||
if file is not None:
|
||||
return os.path.abspath(file).startswith(real_root)
|
||||
# neither __path__ nor __file__: a bare ModuleType fake
|
||||
return False
|
||||
|
||||
|
||||
def _purge_fake_comfyui_manager() -> None:
|
||||
"""Drop fake/broken ``comfyui_manager`` lineage entries from
|
||||
``sys.modules`` so a subsequent real-package import succeeds.
|
||||
|
||||
Uniform rule — an entry is dropped iff it does NOT resolve to the real
|
||||
on-disk package tree:
|
||||
|
||||
- A fake top-level entry (bare ``ModuleType`` without ``__path__`` —
|
||||
'not a package') is dropped so the real package can import.
|
||||
- Fake SUBentries injected under dotted names (e.g.
|
||||
``comfyui_manager.common.manager_util`` replaced by a stub) are
|
||||
dropped; the import machinery re-imports the real module on next
|
||||
access.
|
||||
- REAL-file modules registered under dotted names by other test
|
||||
modules (e.g. ``comfyui_manager.common.unified_dep_resolver`` loaded
|
||||
via spec_from_file_location by tests/test_unified_dep_resolver.py)
|
||||
are KEPT: those tests later resolve the SAME cached entry through
|
||||
``mock.patch("comfyui_manager.common.unified_dep_resolver.X")``;
|
||||
evicting it would make mock.patch import a fresh second instance and
|
||||
silently patch the wrong module object.
|
||||
"""
|
||||
lineage = [n for n in list(sys.modules)
|
||||
if n == "comfyui_manager" or n.startswith("comfyui_manager.")]
|
||||
for name in lineage:
|
||||
mod = sys.modules.get(name)
|
||||
if mod is None or not _module_is_real(mod):
|
||||
del sys.modules[name]
|
||||
|
||||
|
||||
def _import_real(dotted: str):
|
||||
"""ensure stubs -> purge fakes -> import; shared by all entry points."""
|
||||
ensure_comfy_stubs()
|
||||
_purge_fake_comfyui_manager()
|
||||
return importlib.import_module(dotted)
|
||||
|
||||
|
||||
def import_reader(reader: str):
|
||||
"""Import and return ``comfyui_manager.<reader>.manager_core``.
|
||||
|
||||
``reader`` is ``"glob"`` or ``"legacy"`` — the two independent config
|
||||
reader implementations sharing one config.ini (dual-reader rule,
|
||||
goal265-mm.md §1.1 / goal265-spec.md §3).
|
||||
"""
|
||||
assert reader in ("glob", "legacy"), reader
|
||||
return _import_real(f"comfyui_manager.{reader}.manager_core")
|
||||
|
||||
|
||||
def import_context():
|
||||
"""Import and return ``comfyui_manager.common.context`` (holds
|
||||
``manager_config_path``, consumed by both readers at call time)."""
|
||||
return _import_real("comfyui_manager.common.context")
|
||||
|
||||
|
||||
def import_glob_security_utils():
|
||||
"""Import and return ``comfyui_manager.glob.utils.security_utils``
|
||||
(the glob copy of the security_level gate matrix — guard rows
|
||||
SC-25C / SC-28)."""
|
||||
return _import_real("comfyui_manager.glob.utils.security_utils")
|
||||
267
tests/common/test_install_flag_predicate.py
Normal file
267
tests/common/test_install_flag_predicate.py
Normal file
@ -0,0 +1,267 @@
|
||||
"""[goal265 step4 — RED] Predicate truth table for ``is_dedicated_install_allowed``.
|
||||
|
||||
TARGET CONTRACT (NOT YET IMPLEMENTED — goal265-spec.md §1.2, LOCKED):
|
||||
|
||||
comfyui_manager/common/manager_security.py::is_dedicated_install_allowed(
|
||||
flag_value: bool, listen_address: str, network_mode: str) -> bool
|
||||
|
||||
P-direct: allowed iff bool(flag_value)
|
||||
AND (is_loopback(listen_address)
|
||||
OR network_mode.lower() == 'personal_cloud')
|
||||
|
||||
These tests are EXPECTED TO FAIL/ERROR against current code (the predicate
|
||||
does not exist yet) — RED confirmation is goal265 Step 5. Do NOT weaken them
|
||||
to pass against today's code.
|
||||
|
||||
SC rows covered (goal265-scenarios.md — predicate-level arm):
|
||||
SC-01, SC-02, SC-03, SC-05, SC-06, SC-07 ([D1] allow_git_url_install)
|
||||
SC-11, SC-12, SC-13, SC-14, SC-15 ([D2] allow_pip_install)
|
||||
SC-16, SC-17 (flag independence, [D1][D2])
|
||||
|
||||
security_level is ABSENT from the predicate signature — its irrelevance is
|
||||
proven BY CONSTRUCTION (spec §4 row 1): rows whose preconditions differ only
|
||||
in security_level (SC-01 vs SC-02; SC-06/SC-14 parametrizations) map onto the
|
||||
IDENTICAL predicate call, so no security_level value can change the outcome.
|
||||
|
||||
Fixtures: none — pure function (spec §4 binding patching constraint).
|
||||
The module under test is loaded directly by file path so the test does not
|
||||
import the ``comfyui_manager`` package (whose __init__ needs the ComfyUI
|
||||
runtime); ``manager_security.py`` itself is dependency-light by design
|
||||
(spec §1.2: MUST stay config-import-free).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib.util
|
||||
import inspect
|
||||
import pathlib
|
||||
|
||||
import pytest
|
||||
|
||||
_MANAGER_SECURITY_PATH = (
|
||||
pathlib.Path(__file__).resolve().parents[2]
|
||||
/ "comfyui_manager" / "common" / "manager_security.py"
|
||||
)
|
||||
|
||||
# Address / network-mode vocabulary (scenarios §0 preamble):
|
||||
LOOPBACK = "127.0.0.1"
|
||||
LOOPBACK_V6 = "::1"
|
||||
PUBLIC_ADDR = "203.0.113.5" # TEST-NET-3 — non-loopback ("listen=public")
|
||||
NM_PUBLIC = "public"
|
||||
NM_PERSONAL_CLOUD = "personal_cloud"
|
||||
|
||||
# security_level context values for irrelevance-by-construction params.
|
||||
# The predicate takes NO security_level argument; these ids document which
|
||||
# scenario-row context each identical call stands for.
|
||||
SECURITY_LEVELS = ("strong", "normal", "normal-", "weak")
|
||||
|
||||
|
||||
def _load_manager_security():
|
||||
spec = importlib.util.spec_from_file_location(
|
||||
"_manager_security_under_test", _MANAGER_SECURITY_PATH
|
||||
)
|
||||
assert spec is not None and spec.loader is not None, _MANAGER_SECURITY_PATH
|
||||
mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(mod)
|
||||
return mod
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def predicate():
|
||||
"""The predicate under test. FAILS (RED) while the function is absent."""
|
||||
mod = _load_manager_security()
|
||||
assert hasattr(mod, "is_dedicated_install_allowed"), (
|
||||
"RED: comfyui_manager/common/manager_security.py does not define "
|
||||
"is_dedicated_install_allowed yet (goal265-spec.md §1.2). "
|
||||
"This is the expected state before Step 6 (Develop)."
|
||||
)
|
||||
return mod.is_dedicated_install_allowed
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Structural: security_level is not even an input (irrelevance by construction)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_signature_has_no_security_level_parameter(predicate):
|
||||
"""[SC-01..SC-17 foundation] Spec §1.2 locks the signature to exactly
|
||||
(flag_value, listen_address, network_mode) — security_level CANNOT
|
||||
influence the decision because it is not an input."""
|
||||
params = list(inspect.signature(predicate).parameters)
|
||||
assert params == ["flag_value", "listen_address", "network_mode"], (
|
||||
f"Locked signature drifted: {params!r} "
|
||||
"(goal265-spec.md §1.2 — security_level must NOT appear)"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# [D1] allow_git_url_install — happy paths
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_sc01_flag_true_loopback_allows_under_strong_security_level(predicate):
|
||||
"""SC-01: git=true, sl=strong, listen=loopback, nm=public -> allowed.
|
||||
sl=strong is context only — the call has no security_level input."""
|
||||
assert predicate(True, LOOPBACK, NM_PUBLIC) is True
|
||||
|
||||
|
||||
def test_sc02_flag_true_loopback_allows_under_default_security_level(predicate):
|
||||
"""SC-02: git=true, sl=normal (default), listen=loopback, nm=public ->
|
||||
allowed. Identical call to SC-01: proves sl irrelevance by construction."""
|
||||
assert predicate(True, LOOPBACK, NM_PUBLIC) is True
|
||||
|
||||
|
||||
def test_sc03_flag_true_public_listener_personal_cloud_allows(predicate):
|
||||
"""SC-03: git=true, sl=normal, listen=public, nm=personal_cloud ->
|
||||
allowed (personal_cloud arm of the network-position invariant)."""
|
||||
assert predicate(True, PUBLIC_ADDR, NM_PERSONAL_CLOUD) is True
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# [D1] allow_git_url_install — failure paths
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_sc05_flag_false_denies_even_at_weak_security_level(predicate):
|
||||
"""SC-05: git=false, sl=weak, listen=loopback -> denied. Deny-direction
|
||||
proof of security_level irrelevance (today sl=weak would ALLOW)."""
|
||||
assert predicate(False, LOOPBACK, NM_PUBLIC) is False
|
||||
|
||||
|
||||
@pytest.mark.parametrize("security_level_context", SECURITY_LEVELS[:3],
|
||||
ids=[f"sl={s}" for s in SECURITY_LEVELS[:3]])
|
||||
def test_sc06_flag_false_denies_for_every_security_level(
|
||||
predicate, security_level_context):
|
||||
"""SC-06: git=false, sl in {strong, normal, normal-}, listen=loopback ->
|
||||
denied for EVERY sl value. The parametrize axis documents that all three
|
||||
scenario contexts collapse onto the same flag-only call — the flag is
|
||||
the sole decider."""
|
||||
assert predicate(False, LOOPBACK, NM_PUBLIC) is False
|
||||
|
||||
|
||||
def test_sc07_flag_true_cannot_open_public_listener(predicate):
|
||||
"""SC-07: git=true, sl=weak, listen=public, nm=public -> denied.
|
||||
Network-position invariant retained (spec §1.1 invariant 2): the flag
|
||||
must never widen exposure beyond what is possible today."""
|
||||
assert predicate(True, PUBLIC_ADDR, NM_PUBLIC) is False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# [D2] allow_pip_install — happy paths
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_sc11_pip_flag_true_loopback_allows_under_strong(predicate):
|
||||
"""SC-11: pip=true, sl=strong, listen=loopback, nm=public -> allowed."""
|
||||
assert predicate(True, LOOPBACK, NM_PUBLIC) is True
|
||||
|
||||
|
||||
def test_sc12_pip_flag_true_public_listener_personal_cloud_allows(predicate):
|
||||
"""SC-12: pip=true, sl=normal, listen=public, nm=personal_cloud ->
|
||||
allowed."""
|
||||
assert predicate(True, PUBLIC_ADDR, NM_PERSONAL_CLOUD) is True
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# [D2] allow_pip_install — failure paths
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_sc13_pip_flag_false_denies_even_at_weak(predicate):
|
||||
"""SC-13: pip=false, sl=weak, listen=loopback -> denied."""
|
||||
assert predicate(False, LOOPBACK, NM_PUBLIC) is False
|
||||
|
||||
|
||||
@pytest.mark.parametrize("security_level_context", SECURITY_LEVELS[:3],
|
||||
ids=[f"sl={s}" for s in SECURITY_LEVELS[:3]])
|
||||
def test_sc14_pip_flag_false_denies_for_every_security_level(
|
||||
predicate, security_level_context):
|
||||
"""SC-14: pip=false, sl in {strong, normal, normal-}, listen=loopback ->
|
||||
denied each. Same irrelevance-by-construction pattern as SC-06."""
|
||||
assert predicate(False, LOOPBACK, NM_PUBLIC) is False
|
||||
|
||||
|
||||
def test_sc15_pip_flag_true_cannot_open_public_listener(predicate):
|
||||
"""SC-15: pip=true, sl=weak, listen=public, nm=public -> denied
|
||||
(network invariant)."""
|
||||
assert predicate(True, PUBLIC_ADDR, NM_PUBLIC) is False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Flag independence (cross-link [D1]+[D2])
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_sc16_git_false_pip_true_independent_outcomes(predicate):
|
||||
"""SC-16: git=false, pip=true, sl=normal, listen=loopback -> git denied,
|
||||
pip allowed. At predicate level each surface passes ONLY its own flag —
|
||||
same network inputs, opposite outcomes."""
|
||||
git_allowed = predicate(False, LOOPBACK, NM_PUBLIC)
|
||||
pip_allowed = predicate(True, LOOPBACK, NM_PUBLIC)
|
||||
assert git_allowed is False
|
||||
assert pip_allowed is True
|
||||
|
||||
|
||||
def test_sc17_git_true_pip_false_independent_outcomes(predicate):
|
||||
"""SC-17: git=true, pip=false, sl=normal, listen=loopback -> git allowed,
|
||||
pip denied (mirror of SC-16)."""
|
||||
git_allowed = predicate(True, LOOPBACK, NM_PUBLIC)
|
||||
pip_allowed = predicate(False, LOOPBACK, NM_PUBLIC)
|
||||
assert git_allowed is True
|
||||
assert pip_allowed is False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Supplementary: exhaustive flag x loopback x personal_cloud truth table
|
||||
# (spec §4 row 1 — "flag x loopback x personal_cloud" full table)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("flag", "listen", "network_mode", "expected"),
|
||||
[
|
||||
(True, LOOPBACK, NM_PUBLIC, True), # SC-01/02/11
|
||||
(True, LOOPBACK, NM_PERSONAL_CLOUD, True), # both arms true
|
||||
(True, PUBLIC_ADDR, NM_PERSONAL_CLOUD, True), # SC-03/12
|
||||
(True, PUBLIC_ADDR, NM_PUBLIC, False), # SC-07/15
|
||||
(False, LOOPBACK, NM_PUBLIC, False), # SC-05/06/13/14
|
||||
(False, LOOPBACK, NM_PERSONAL_CLOUD, False),
|
||||
(False, PUBLIC_ADDR, NM_PERSONAL_CLOUD, False),
|
||||
(False, PUBLIC_ADDR, NM_PUBLIC, False),
|
||||
],
|
||||
ids=[
|
||||
"flag+loopback+nm_public",
|
||||
"flag+loopback+personal_cloud",
|
||||
"flag+public+personal_cloud",
|
||||
"flag+public+nm_public",
|
||||
"noflag+loopback+nm_public",
|
||||
"noflag+loopback+personal_cloud",
|
||||
"noflag+public+personal_cloud",
|
||||
"noflag+public+nm_public",
|
||||
],
|
||||
)
|
||||
def test_truth_table_flag_x_loopback_x_personal_cloud(
|
||||
predicate, flag, listen, network_mode, expected):
|
||||
"""Full 2x2x2 truth table for P-direct (spec §1.1): allowed iff flag AND
|
||||
(loopback OR personal_cloud). Consolidates SC-01/02/03/05/06/07 ([D1])
|
||||
and SC-11/12/13/14/15 ([D2]) plus the two combinations no single row
|
||||
pins (flag+loopback+personal_cloud, noflag+public+personal_cloud)."""
|
||||
assert predicate(flag, listen, network_mode) is expected
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Supplementary edge handling locked by the spec text
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_ipv6_loopback_counts_as_loopback(predicate):
|
||||
"""SC-01/SC-02 edge (loopback arm): `is_loopback` is ipaddress-based
|
||||
(manager_security.py) — ::1 must be treated as loopback too."""
|
||||
assert predicate(True, LOOPBACK_V6, NM_PUBLIC) is True
|
||||
|
||||
|
||||
@pytest.mark.parametrize("network_mode", ["Personal_Cloud", "PERSONAL_CLOUD"],
|
||||
ids=["mixed-case", "upper-case"])
|
||||
def test_network_mode_personal_cloud_is_case_insensitive(predicate, network_mode):
|
||||
"""SC-03/SC-12 edge (personal_cloud arm): spec §1.2 predicate body —
|
||||
network_mode.lower() == 'personal_cloud' (case-insensitive)."""
|
||||
assert predicate(True, PUBLIC_ADDR, network_mode) is True
|
||||
|
||||
|
||||
def test_unparseable_listen_address_is_not_loopback(predicate):
|
||||
"""SC-07/SC-15 edge (network invariant): is_loopback returns False for
|
||||
unparseable addresses (ValueError arm) — the predicate must fail CLOSED
|
||||
on a malformed listen address."""
|
||||
assert predicate(True, "not-an-ip-address", NM_PUBLIC) is False
|
||||
@ -10,15 +10,18 @@
|
||||
# script does NOT restore on its own, so fixture teardown MUST cleanup.
|
||||
#
|
||||
# Why permissive mode is needed:
|
||||
# Three endpoints check is_allowed_security_level('high+')
|
||||
# (security_utils.py:20-26): at is_local_mode=True (127.0.0.1 listen)
|
||||
# the gate requires security_level ∈ {weak, normal-}. Default
|
||||
# `security_level = normal` fails, so the POST returns 403.
|
||||
# - wi-014 POST /v2/comfyui_manager/comfyui_switch_version
|
||||
# - wi-037 POST /v2/customnode/install/git_url
|
||||
# - wi-038 POST /v2/customnode/install/pip
|
||||
# Setting security_level = normal- allows real E2E execution of these
|
||||
# endpoints with fixed, trusted inputs (never test-input-derived URLs).
|
||||
# - wi-014 POST /v2/comfyui_manager/comfyui_switch_version checks
|
||||
# is_allowed_security_level('high+'): at is_local_mode=True
|
||||
# (127.0.0.1 listen) the gate requires security_level ∈ {weak,
|
||||
# normal-}. Default `security_level = normal` fails (403).
|
||||
# - wi-037 POST /v2/customnode/install/git_url and
|
||||
# wi-038 POST /v2/customnode/install/pip are gated by the dedicated
|
||||
# config.ini flags `allow_git_url_install` / `allow_pip_install`
|
||||
# (goal265 / #2962) — independent of security_level, default false
|
||||
# (403).
|
||||
# Patching security_level = normal- plus both flags = true allows real
|
||||
# E2E execution of these endpoints with fixed, trusted inputs (never
|
||||
# test-input-derived URLs).
|
||||
#
|
||||
# SECURITY NOTE:
|
||||
# The endpoints are gated at high+ because they execute arbitrary remote
|
||||
@ -39,7 +42,8 @@
|
||||
# Exit: 0=ready, 1=timeout/failure
|
||||
#
|
||||
# Side effect: $E2E_ROOT/comfyui/user/__manager/config.ini gets
|
||||
# `security_level = normal-`. The original value is preserved at
|
||||
# `security_level = normal-`, `allow_git_url_install = true` and
|
||||
# `allow_pip_install = true`. The original config is preserved at
|
||||
# config.ini.before-permissive for the fixture to restore on teardown.
|
||||
|
||||
set -euo pipefail
|
||||
@ -68,4 +72,15 @@ else
|
||||
fi
|
||||
echo "[start_comfyui_permissive] Patched security_level = normal- in $CONFIG"
|
||||
|
||||
# Patch the dedicated install flags (goal265 / #2962): install/git_url and
|
||||
# install/pip are gated by these flags instead of security_level (idempotent).
|
||||
for flag in allow_git_url_install allow_pip_install; do
|
||||
if grep -qE "^${flag}\s*=" "$CONFIG"; then
|
||||
sed -i -E "s/^${flag}\s*=.*/${flag} = true/" "$CONFIG"
|
||||
else
|
||||
sed -i -E "/^\[default\]/a ${flag} = true" "$CONFIG"
|
||||
fi
|
||||
done
|
||||
echo "[start_comfyui_permissive] Patched allow_git_url_install/allow_pip_install = true in $CONFIG"
|
||||
|
||||
exec env ENABLE_LEGACY_UI=1 bash "$SCRIPT_DIR/start_comfyui.sh" "$@"
|
||||
|
||||
@ -17,13 +17,16 @@ Usage:
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
import requests
|
||||
|
||||
E2E_ROOT = os.environ.get("E2E_ROOT", "")
|
||||
COMFYUI_PATH = os.path.join(E2E_ROOT, "comfyui") if E2E_ROOT else ""
|
||||
CUSTOM_NODES = os.path.join(COMFYUI_PATH, "custom_nodes") if COMFYUI_PATH else ""
|
||||
SCRIPTS_DIR = os.path.join(
|
||||
os.path.dirname(os.path.abspath(__file__)), "scripts"
|
||||
)
|
||||
@ -31,6 +34,16 @@ SCRIPTS_DIR = os.path.join(
|
||||
PORT = 8199
|
||||
BASE_URL = f"http://127.0.0.1:{PORT}"
|
||||
|
||||
# Seed pack for the `installed` tests — same CNR test package used by
|
||||
# test_e2e_endpoint.py / test_e2e_task_operations.py. Installed by the
|
||||
# `seed_pack_on_disk` autouse fixture below (this module runs
|
||||
# alphabetically BEFORE test_e2e_endpoint.py, so it cannot rely on that
|
||||
# module having installed the pack — on a fresh E2E env nothing else
|
||||
# seeds it).
|
||||
PACK_ID = "ComfyUI_SigmoidOffsetScheduler"
|
||||
PACK_DIR_NAME = "ComfyUI_SigmoidOffsetScheduler"
|
||||
PACK_VERSION = "1.0.1"
|
||||
|
||||
pytestmark = pytest.mark.skipif(
|
||||
not E2E_ROOT
|
||||
or not os.path.isfile(os.path.join(E2E_ROOT, ".e2e_setup_complete")),
|
||||
@ -84,6 +97,51 @@ def comfyui():
|
||||
_stop_comfyui()
|
||||
|
||||
|
||||
def _pack_exists() -> bool:
|
||||
return os.path.isdir(os.path.join(CUSTOM_NODES, PACK_DIR_NAME))
|
||||
|
||||
|
||||
def _cm_cli_path() -> str:
|
||||
if sys.platform == "win32":
|
||||
return os.path.join(E2E_ROOT, "venv", "Scripts", "cm-cli.exe")
|
||||
return os.path.join(E2E_ROOT, "venv", "bin", "cm-cli")
|
||||
|
||||
|
||||
@pytest.fixture(scope="module", autouse=True)
|
||||
def seed_pack_on_disk():
|
||||
"""Ensure the seed pack is on disk BEFORE the server starts.
|
||||
|
||||
The `installed?mode=imported` test asserts against the startup
|
||||
snapshot, which is frozen when the server boots — so the pack must be
|
||||
installed before the module's `comfyui` fixture launches it. Autouse +
|
||||
module scope guarantees this fixture is instantiated ahead of the
|
||||
non-autouse `comfyui` fixture. Installs via cm-cli (no server needed,
|
||||
creates the same CNR `.tracking` layout) and removes the pack on
|
||||
teardown only if this fixture installed it, leaving the environment
|
||||
as found.
|
||||
"""
|
||||
installed_by_fixture = False
|
||||
if not _pack_exists():
|
||||
env = {**os.environ, "COMFYUI_PATH": COMFYUI_PATH}
|
||||
r = subprocess.run(
|
||||
[_cm_cli_path(), "install", f"{PACK_ID}@{PACK_VERSION}"],
|
||||
capture_output=True, text=True, timeout=300, env=env,
|
||||
)
|
||||
assert r.returncode == 0 and _pack_exists(), (
|
||||
f"seed fixture failed: cm-cli install {PACK_ID}@{PACK_VERSION} "
|
||||
f"(exit {r.returncode})\nSTDOUT: {r.stdout[-500:]}\n"
|
||||
f"STDERR: {r.stderr[-500:]}"
|
||||
)
|
||||
installed_by_fixture = True
|
||||
|
||||
yield PACK_ID
|
||||
|
||||
if installed_by_fixture:
|
||||
shutil.rmtree(
|
||||
os.path.join(CUSTOM_NODES, PACK_DIR_NAME), ignore_errors=True
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tests — getmappings
|
||||
# ---------------------------------------------------------------------------
|
||||
@ -164,10 +222,11 @@ class TestInstalledPacks:
|
||||
def test_installed_returns_dict(self, comfyui):
|
||||
"""GET /v2/customnode/installed returns dict containing seeded E2E pack with valid per-entry schema.
|
||||
|
||||
WI-M strengthening: previously only dict-type check. The E2E setup
|
||||
seeds `ComfyUI_SigmoidOffsetScheduler` (the test package used across
|
||||
task_operations/endpoint tests); its presence is a hard precondition
|
||||
for most other tests. We now assert it's in the installed dict AND
|
||||
WI-M strengthening: previously only dict-type check. The module's
|
||||
`seed_pack_on_disk` autouse fixture installs
|
||||
`ComfyUI_SigmoidOffsetScheduler` (the test package used across
|
||||
task_operations/endpoint tests) before the server starts.
|
||||
We now assert it's in the installed dict AND
|
||||
that its entry has the documented InstalledPack fields
|
||||
(cnr_id/ver/enabled). Defeats a regression where `installed` returns
|
||||
an empty dict despite packs existing on disk.
|
||||
|
||||
@ -64,11 +64,16 @@ def _cm_cli_path() -> str:
|
||||
|
||||
|
||||
def _ensure_cache():
|
||||
"""Run cm-cli update-cache (blocking) to populate Manager cache before tests."""
|
||||
"""Run cm-cli update-cache (blocking) to populate Manager cache before tests.
|
||||
|
||||
Timeout is generous (600s): update-cache downloads the full DB lists
|
||||
over the network and routinely exceeds 120s on slow links, which used
|
||||
to fail the whole module at setup.
|
||||
"""
|
||||
env = {**os.environ, "COMFYUI_PATH": COMFYUI_PATH}
|
||||
r = subprocess.run(
|
||||
[_cm_cli_path(), "update-cache"],
|
||||
capture_output=True, text=True, timeout=120, env=env,
|
||||
capture_output=True, text=True, timeout=600, env=env,
|
||||
)
|
||||
if r.returncode != 0:
|
||||
raise RuntimeError(f"update-cache failed:\n{r.stderr}")
|
||||
|
||||
@ -5,10 +5,13 @@ operation via HTTP:
|
||||
Default-security fixture (middle+ / no-gate endpoints):
|
||||
- wi-020 POST /v2/manager/queue/install_model (tiny TAEF1 model, <5MB)
|
||||
- wi-024 POST /v2/manager/queue/update_comfyui (safe via env var)
|
||||
Permissive-security fixture (high+ endpoints — normal- harness):
|
||||
- wi-014 POST /v2/comfyui_manager/comfyui_switch_version (no-op self-switch)
|
||||
- wi-037 POST /v2/customnode/install/git_url (nodepack-test1-do-not-install)
|
||||
- wi-038 POST /v2/customnode/install/pip (text-unidecode)
|
||||
Permissive-security fixture (normal- + dedicated install flags):
|
||||
- wi-014 POST /v2/comfyui_manager/comfyui_switch_version (no-op self-switch;
|
||||
high+ gate, opened by security_level = normal-)
|
||||
- wi-037 POST /v2/customnode/install/git_url (nodepack-test1-do-not-install;
|
||||
gated by allow_git_url_install — goal265 / #2962)
|
||||
- wi-038 POST /v2/customnode/install/pip (text-unidecode;
|
||||
gated by allow_pip_install — goal265 / #2962)
|
||||
Pre-seeded broken-pack fixture (no-gate endpoint, needs scan-time state):
|
||||
- wi-015 POST /v2/customnode/import_fail_info (pre-seeded broken pack)
|
||||
|
||||
@ -21,13 +24,17 @@ the test venv.
|
||||
|
||||
Permissive harness security rationale:
|
||||
wi-014/037/038 execute arbitrary remote code (version switch, git
|
||||
clone, pip install) and are gated at `high+` precisely to prevent
|
||||
such operations at default security. The permissive harness
|
||||
clone, pip install) and are gated precisely to prevent such
|
||||
operations at default config — wi-014 at `high+` (security_level),
|
||||
wi-037/038 by the dedicated `allow_git_url_install` /
|
||||
`allow_pip_install` flags (goal265 / #2962, independent of
|
||||
security_level, default false). The permissive harness
|
||||
(start_comfyui_permissive.sh) reflects the production use case
|
||||
these endpoints exist to serve — operators in a trusted environment
|
||||
lower security_level to normal-/weak to enable these features. The
|
||||
200 path IS a supported feature, and testing it requires exactly
|
||||
this configuration. Permissive harness uses HARDCODED trusted inputs:
|
||||
set security_level = normal- and opt in via the flags to enable
|
||||
these features. The 200 path IS a supported feature, and testing it
|
||||
requires exactly this configuration. Permissive harness uses
|
||||
HARDCODED trusted inputs:
|
||||
- wi-014: the CURRENT ComfyUI version (self-switch no-op)
|
||||
- wi-037: https://github.com/ltdrdata/nodepack-test1-do-not-install
|
||||
(project's test-fixture repo, also used by tests/cli/test_uv_compile.py)
|
||||
@ -162,7 +169,8 @@ def comfyui_legacy():
|
||||
|
||||
def _start_comfyui_permissive() -> int:
|
||||
"""Launch via start_comfyui_permissive.sh — patches config.ini to
|
||||
`security_level = normal-` (backup at config.ini.before-permissive)
|
||||
`security_level = normal-` plus `allow_git_url_install = true` /
|
||||
`allow_pip_install = true` (backup at config.ini.before-permissive)
|
||||
then delegates to start_comfyui.sh with ENABLE_LEGACY_UI=1.
|
||||
The permissive fixture MUST restore config on teardown.
|
||||
"""
|
||||
@ -198,9 +206,10 @@ def _restore_permissive_config():
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def comfyui_permissive():
|
||||
"""Module-scoped fixture: start server with security_level=normal-,
|
||||
tear down with config restore. Use for wi-014/037/038 which require
|
||||
`high+` (security_utils.py:20-26 allows weak/normal- at is_local_mode).
|
||||
"""Module-scoped fixture: start server with security_level=normal- and
|
||||
the dedicated install flags enabled, tear down with config restore.
|
||||
Use for wi-014 (high+ gate: weak/normal- at is_local_mode) and
|
||||
wi-037/038 (allow_git_url_install / allow_pip_install — goal265 / #2962).
|
||||
"""
|
||||
pid = _start_comfyui_permissive()
|
||||
try:
|
||||
@ -489,7 +498,7 @@ class TestInstallViaGitUrlRealClone:
|
||||
)
|
||||
assert resp.status_code == 200, (
|
||||
f"install/git_url with trusted URL {TRUSTED_GIT_URL!r} "
|
||||
f"should return 200 at security_level=normal-, got "
|
||||
f"should return 200 with allow_git_url_install=true, got "
|
||||
f"{resp.status_code}: {resp.text[:300]}"
|
||||
)
|
||||
assert os.path.isdir(target), (
|
||||
@ -555,7 +564,7 @@ class TestInstallPipRealExecute:
|
||||
)
|
||||
assert resp.status_code == 200, (
|
||||
f"install/pip with trusted pkg {TRUSTED_PIP_PKG!r} should "
|
||||
f"return 200 at security_level=normal-, got "
|
||||
f"return 200 with allow_pip_install=true, got "
|
||||
f"{resp.status_code}: {resp.text[:300]}"
|
||||
)
|
||||
|
||||
|
||||
408
tests/e2e/test_e2e_pip_url_form.py
Normal file
408
tests/e2e/test_e2e_pip_url_form.py
Normal file
@ -0,0 +1,408 @@
|
||||
"""[goal299 step3] E2E: URL-form pip install through the S-B endpoint.
|
||||
|
||||
CONTRACT UNDER TEST (SHIPPED — goal265; this module adds tests only):
|
||||
the ``allow_pip_install`` gate on POST /v2/customnode/install/pip is
|
||||
**argument-content-agnostic** (goal299-mm-addendum.md §1): a URL-form
|
||||
argument (``git+https://github.com/...``) exercises the SAME
|
||||
dedicated-flag gate as a bare package name — no code path inspects
|
||||
the argument before the gate decision (legacy/manager_server.py:1574).
|
||||
|
||||
FIXTURE NOTE (goal329 — WHY the owned fixture): the install arm
|
||||
originally exercised ``git+https://github.com/facebookresearch/sam2``;
|
||||
that external, unpinned, heavy dependency (torch-ecosystem build at
|
||||
restart time) made the test hostage to upstream maintainer drift
|
||||
(maintainer-fragility decision, goal329). It is replaced by the OWNED,
|
||||
purpose-built fixture ``git+https://github.com/ltdrdata/pip-test1-do-not-install``
|
||||
(PUBLIC; package ``pip-test1-do-not-install``, module
|
||||
``pip_test1_do_not_install``, ``MARKER = "pip-test1-do-not-install:ok"``,
|
||||
zero deps, pure Python, no import side effects). The gate contract is
|
||||
argument-content-agnostic, so the URL swap changes NOTHING about what
|
||||
is being proven — only the install payload shrinks to a harmless,
|
||||
owned package.
|
||||
|
||||
Two arms (goal299-spec.md §1.1, LOCKED):
|
||||
|
||||
DENY arm flag=false at security_level=weak -> 403, NO reservation
|
||||
entry appended, denial log names allow_pip_install
|
||||
(weak proves the FLAG, not the security_level, decides).
|
||||
INSTALL arm flag=true at security_level=strong, loopback -> 200 =
|
||||
gate-pass + RESERVATION (deferred-execution semantics,
|
||||
addendum §1: '#FORCE' reserves into install-scripts.txt;
|
||||
the real `uv pip install` runs at the NEXT server start)
|
||||
-> restart -> fixture module REALLY importable (MARKER
|
||||
verified) in the isolated E2E venv -> self-clean ->
|
||||
absent again.
|
||||
|
||||
Placement: NEW SIBLING of tests/e2e/test_e2e_secgate_legacy_flags.py
|
||||
(spec §1.2) — that module carries an engineered zero-install guarantee
|
||||
(its header :96-100), so the real-install arm must NOT live there.
|
||||
Helper reuse contract (spec §1.2, LOCKED): import-reuse is limited to
|
||||
``_stage_flags_config`` / ``_stop_legacy`` / ``_make_flags_server_fixture``
|
||||
/ ``_post_pip`` / ``_log_offset``; everything else needed here is a
|
||||
sibling-local definition (provenance-commented copies where applicable).
|
||||
|
||||
Self-cleaning + idempotent (spec §1.3 install-arm steps 1/6): pre-guard
|
||||
uninstall, teardown uninstall (runs even on failure), reservation-file
|
||||
hygiene on both ends. The venv is uv-managed and has NO pip module
|
||||
(addendum R4) — all (un)installs go through ``<venv>/bin/uv``.
|
||||
|
||||
Requires a pre-built E2E environment (setup_e2e_env.sh); skip-marked
|
||||
otherwise. The install arm additionally requires github.com reachability
|
||||
(addendum R5) and is skip-marked offline; the deny arm always runs.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ast
|
||||
import os
|
||||
import socket
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
|
||||
# Import-reuse: the LOCKED helper subset only (goal299-spec.md §1.2).
|
||||
from test_e2e_secgate_legacy_flags import (
|
||||
_log_offset,
|
||||
_make_flags_server_fixture,
|
||||
_post_pip,
|
||||
_stop_legacy,
|
||||
)
|
||||
|
||||
E2E_ROOT = os.environ.get("E2E_ROOT", "")
|
||||
COMFYUI_PATH = os.path.join(E2E_ROOT, "comfyui") if E2E_ROOT else ""
|
||||
SCRIPTS_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), "scripts")
|
||||
SERVER_LOG = os.path.join(E2E_ROOT, "logs", "comfyui.log") if E2E_ROOT else ""
|
||||
|
||||
# Same port as the flags module (its module constant is not in the locked
|
||||
# import set; the value is part of the shared legacy-fixture contract).
|
||||
PORT = 8199
|
||||
|
||||
# Owned fixture (goal329 — see FIXTURE NOTE in the module docstring).
|
||||
# Dist name uses hyphens, import name uses underscores; MARKER gives a
|
||||
# stronger installed-for-real check than a bare import.
|
||||
FIXTURE_URL = "git+https://github.com/ltdrdata/pip-test1-do-not-install"
|
||||
FIXTURE_DIST = "pip-test1-do-not-install"
|
||||
FIXTURE_MODULE = "pip_test1_do_not_install"
|
||||
FIXTURE_MARKER = "pip-test1-do-not-install:ok"
|
||||
VENV_PY = os.path.join(E2E_ROOT, "venv", "bin", "python") if E2E_ROOT else ""
|
||||
VENV_UV = os.path.join(E2E_ROOT, "venv", "bin", "uv") if E2E_ROOT else ""
|
||||
# Reservation file (path shape per tests/e2e/test_e2e_legacy_real_ops.py:538;
|
||||
# producer: reserve_script, legacy/manager_core.py:1836-1843; consumer:
|
||||
# comfyui_manager/prestartup_script.py:485 at next server start).
|
||||
SCRIPTS_PATH = (
|
||||
os.path.join(
|
||||
COMFYUI_PATH, "user", "__manager", "startup-scripts", "install-scripts.txt"
|
||||
)
|
||||
if E2E_ROOT
|
||||
else ""
|
||||
)
|
||||
|
||||
# Post-reservation restart budget (goal329): the owned fixture is pure
|
||||
# Python with zero deps — its `uv pip install` completes in seconds, so
|
||||
# the restart wall time is dominated by the plain server boot
|
||||
# (+prestartup resolver), observed well under 60s in this harness. 180s
|
||||
# equals the proven readiness budget the flags module's _start_legacy
|
||||
# uses for a plain legacy start (>=3x headroom over observed boot); the
|
||||
# python-side timeout is shell+60 per the helper invariant.
|
||||
RESTART_BUDGET_S = 180
|
||||
|
||||
pytestmark = pytest.mark.skipif(
|
||||
not E2E_ROOT
|
||||
or not os.path.isfile(os.path.join(E2E_ROOT, ".e2e_setup_complete")),
|
||||
reason="E2E_ROOT not set or E2E environment not ready",
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Sibling-local helpers (spec §1.2 — outside the locked import set)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _start_legacy_install(extra_env: dict, shell_timeout: int = RESTART_BUDGET_S) -> int:
|
||||
"""Parameterized variant of test_e2e_secgate_legacy_flags._start_legacy
|
||||
(:147-158) — provenance copy per goal299-spec.md §1.2. The flags
|
||||
module's helper hardcodes its env dict and python-side timeout, which
|
||||
can neither inject per-restart env nor take a per-call readiness
|
||||
budget. TIMEOUT is forwarded to start_comfyui_legacy.sh as the shell
|
||||
readiness budget; the python-side subprocess timeout MUST exceed it
|
||||
(+60s) so the start script, not python, owns the timeout."""
|
||||
env = {
|
||||
**os.environ,
|
||||
"E2E_ROOT": E2E_ROOT,
|
||||
"PORT": str(PORT),
|
||||
"TIMEOUT": str(shell_timeout),
|
||||
**extra_env,
|
||||
}
|
||||
r = subprocess.run(
|
||||
["bash", os.path.join(SCRIPTS_DIR, "start_comfyui_legacy.sh")],
|
||||
capture_output=True, text=True, timeout=shell_timeout + 60, env=env,
|
||||
)
|
||||
if r.returncode != 0:
|
||||
raise RuntimeError(
|
||||
f"Failed to start ComfyUI (legacy, install restart):\n{r.stderr}"
|
||||
)
|
||||
for part in r.stdout.strip().split():
|
||||
if part.startswith("COMFYUI_PID="):
|
||||
return int(part.split("=")[1])
|
||||
raise RuntimeError(f"Could not parse PID:\n{r.stdout}")
|
||||
|
||||
|
||||
def _log_slice(offset: int) -> str:
|
||||
"""Provenance copy of test_e2e_secgate_legacy_flags._log_slice
|
||||
(:241-246) — not in the locked import set (spec §1.2 fallback clause:
|
||||
fixture semantics are the lock, import path is not)."""
|
||||
if not os.path.isfile(SERVER_LOG):
|
||||
return ""
|
||||
with open(SERVER_LOG, "r", encoding="utf-8", errors="replace") as f:
|
||||
f.seek(offset)
|
||||
return f.read()
|
||||
|
||||
|
||||
def _scripts_state() -> str:
|
||||
"""Current install-scripts.txt content, or the sentinel 'ABSENT' —
|
||||
the file may legitimately not exist (spec §1.3 deny step 2: it is
|
||||
created lazily by reserve_script on first reservation)."""
|
||||
if not os.path.isfile(SCRIPTS_PATH):
|
||||
return "ABSENT"
|
||||
with open(SCRIPTS_PATH, "r", encoding="utf-8", errors="replace") as f:
|
||||
return f.read()
|
||||
|
||||
|
||||
def _venv_fixture_probe() -> subprocess.CompletedProcess:
|
||||
"""Import the fixture module in the isolated E2E venv and print its
|
||||
MARKER (subprocess — the observable for really-installed /
|
||||
really-absent). rc != 0 means absent; rc == 0 AND the MARKER value in
|
||||
stdout means installed for real (stronger than a bare import)."""
|
||||
return subprocess.run(
|
||||
[VENV_PY, "-c", f"import {FIXTURE_MODULE} as m; print(m.MARKER)"],
|
||||
capture_output=True, text=True, timeout=60,
|
||||
)
|
||||
|
||||
|
||||
def _uninstall_fixture() -> None:
|
||||
"""Uninstall the fixture distribution from the E2E venv via uv.
|
||||
|
||||
The venv has NO pip module (`python -m pip` fails — addendum R4);
|
||||
`<venv>/bin/uv pip uninstall --python <venv-python>` is the working
|
||||
path. A no-op when the dist is absent (uv warns, exits 0)."""
|
||||
subprocess.run(
|
||||
[VENV_UV, "pip", "uninstall", FIXTURE_DIST, "--python", VENV_PY],
|
||||
capture_output=True, text=True, timeout=180,
|
||||
)
|
||||
|
||||
|
||||
def _strip_fixture_reservation() -> None:
|
||||
"""Remove any fixture residual line from install-scripts.txt
|
||||
(addendum R8: a leftover reservation would silently mutate the venv
|
||||
on the NEXT restart, possibly a different test's). Matches both the
|
||||
hyphenated dist/URL form and the underscored module form."""
|
||||
if not os.path.isfile(SCRIPTS_PATH):
|
||||
return
|
||||
with open(SCRIPTS_PATH, "r", encoding="utf-8", errors="replace") as f:
|
||||
lines = f.readlines()
|
||||
kept = [
|
||||
ln for ln in lines
|
||||
if FIXTURE_DIST not in ln.lower() and FIXTURE_MODULE not in ln.lower()
|
||||
]
|
||||
if kept != lines:
|
||||
with open(SCRIPTS_PATH, "w", encoding="utf-8") as f:
|
||||
f.writelines(kept)
|
||||
|
||||
|
||||
def _github_reachable() -> bool:
|
||||
"""Runtime network probe for the install arm (addendum R5)."""
|
||||
try:
|
||||
socket.create_connection(("github.com", 443), timeout=10).close()
|
||||
return True
|
||||
except OSError:
|
||||
return False
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
# DENY arm — flag=false, security_level=weak (weak proves flag-not-level
|
||||
# decides; goal299-spec.md §1.3 TestPipUrlFormDeny)
|
||||
# ===========================================================================
|
||||
|
||||
comfyui_pip_url_deny = _make_flags_server_fixture(False, False, "weak")
|
||||
|
||||
|
||||
class TestPipUrlFormDeny:
|
||||
"""URL-form POST is denied by the allow_pip_install flag gate exactly
|
||||
like a bare package name (argument-content-agnostic, addendum §1):
|
||||
403, no install-side-effect (no reservation entry), denial log names
|
||||
the flag. Shared-POST shape (spec §1.3, LOCKED): exactly ONE POST via
|
||||
a class-scoped record fixture; the three tests assert exclusively
|
||||
against the record — independent of execution order, no re-reads of
|
||||
live state."""
|
||||
|
||||
@pytest.fixture(scope="class")
|
||||
def deny_post_record(self, comfyui_pip_url_deny):
|
||||
offset = _log_offset()
|
||||
snapshot_before = _scripts_state()
|
||||
resp = _post_pip(FIXTURE_URL)
|
||||
return {
|
||||
"log_offset_before": offset,
|
||||
"scripts_snapshot_before": snapshot_before,
|
||||
"status_code": resp.status_code,
|
||||
"log_slice_after": _log_slice(offset),
|
||||
"scripts_state_after": _scripts_state(),
|
||||
}
|
||||
|
||||
def test_url_form_denied_403(self, deny_post_record):
|
||||
"""403 from the dedicated-flag gate despite security_level=weak
|
||||
(deny-direction decoupling, mm §2 row 'deny arm, after POST')."""
|
||||
assert deny_post_record["status_code"] == 403, (
|
||||
f"URL-form pip POST: expected 403 (allow_pip_install=false "
|
||||
f"overrides sl=weak), got {deny_post_record['status_code']} — "
|
||||
f"the gate must be argument-content-agnostic."
|
||||
)
|
||||
|
||||
def test_url_form_deny_no_reservation(self, deny_post_record):
|
||||
"""No reservation entry appended on deny (mm §2: install-scripts.txt
|
||||
'no entry written'; HTTP 200 would have meant reservation, so the
|
||||
durable on-disk state is the decisive side-effect observable)."""
|
||||
before = deny_post_record["scripts_snapshot_before"]
|
||||
after = deny_post_record["scripts_state_after"]
|
||||
assert after == before, (
|
||||
f"deny arm: install-scripts.txt changed across the denied POST "
|
||||
f"— a reservation leaked past the gate.\nbefore:\n{before}\n"
|
||||
f"after:\n{after}"
|
||||
)
|
||||
if after != "ABSENT":
|
||||
assert FIXTURE_DIST not in after.lower(), (
|
||||
f"deny arm: a fixture line is present in install-scripts.txt "
|
||||
f"after a denied POST:\n{after}"
|
||||
)
|
||||
|
||||
def test_url_form_deny_log_names_flag(self, deny_post_record):
|
||||
"""Denial log names allow_pip_install (SECURITY_MESSAGE_FLAG_PIP,
|
||||
legacy/manager_server.py:47) and carries no security-level framing
|
||||
for this denial (goal265 spec §1.1 invariant 6 — same assertion
|
||||
shape as the flags module's SC-23)."""
|
||||
log = deny_post_record["log_slice_after"]
|
||||
assert "allow_pip_install" in log, (
|
||||
"deny arm: denial log does not name allow_pip_install "
|
||||
f"(SECURITY_MESSAGE_FLAG_PIP). Slice:\n{log[-1500:]}"
|
||||
)
|
||||
assert "security level to 'normal-'" not in log, (
|
||||
"deny arm: denial log still carries the misleading "
|
||||
"security-level copy (SECURITY_MESSAGE_NORMAL_MINUS)."
|
||||
)
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
# INSTALL arm — flag=true, security_level=strong (strong proves allow-side
|
||||
# decoupling; goal299-spec.md §1.3 TestPipUrlFormInstall)
|
||||
# ===========================================================================
|
||||
|
||||
comfyui_pip_url_install = _make_flags_server_fixture(False, True, "strong")
|
||||
|
||||
|
||||
@pytest.mark.network
|
||||
class TestPipUrlFormInstall:
|
||||
"""Full deferred-install round trip (mm §2 observable-outcome table):
|
||||
POST 200 = gate-pass + reservation (NOT installation) -> restart
|
||||
executes the reserved `uv pip install -U git+...` at prestartup
|
||||
-> fixture module imports with the expected MARKER in the isolated
|
||||
venv -> self-clean -> absent.
|
||||
|
||||
Asserting importability right after the 200 would be a guaranteed
|
||||
false-FAIL (mm §2 note) — the restart between POST and the import
|
||||
assertion is the production execution path (addendum §4.1)."""
|
||||
|
||||
@pytest.fixture(scope="class", autouse=True)
|
||||
def _network_guard(self):
|
||||
if not _github_reachable():
|
||||
pytest.skip("offline: github.com unreachable — install arm "
|
||||
"requires network (deny arm unaffected)")
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _self_clean(self):
|
||||
"""Teardown runs even on failure (spec §1.3 step 6): uninstall the
|
||||
dist, strip any residual reservation line (addendum R8), and prove
|
||||
the venv is back to baseline (import fails again)."""
|
||||
yield
|
||||
_uninstall_fixture()
|
||||
_strip_fixture_reservation()
|
||||
assert _venv_fixture_probe().returncode != 0, (
|
||||
f"self-clean: {FIXTURE_MODULE} is still importable in the E2E "
|
||||
f"venv after `uv pip uninstall {FIXTURE_DIST}` — venv NOT "
|
||||
f"restored to baseline."
|
||||
)
|
||||
|
||||
def test_url_form_install_end_to_end(self, comfyui_pip_url_install):
|
||||
# Step 1 — pre-guard (idempotency, spec §1.3 step 1): a previous
|
||||
# aborted run must not turn this into a false-PASS.
|
||||
_uninstall_fixture()
|
||||
_strip_fixture_reservation()
|
||||
assert _venv_fixture_probe().returncode != 0, (
|
||||
f"pre-guard: {FIXTURE_MODULE} already importable before the "
|
||||
f"test — uninstall guard failed; venv not at baseline."
|
||||
)
|
||||
|
||||
# Step 2 — POST the URL-form argument; 200 = gate-pass +
|
||||
# reservation under the deferred-execution semantics (mm §1).
|
||||
resp = _post_pip(FIXTURE_URL)
|
||||
assert resp.status_code == 200, (
|
||||
f"install arm: expected 200 (allow_pip_install=true overrides "
|
||||
f"sl=strong), got {resp.status_code} — allow-side decoupling "
|
||||
f"broken for the URL-form argument."
|
||||
)
|
||||
|
||||
# Step 3 — reservation entry: a Python-list-repr line containing
|
||||
# the fixture URL substring (producer reserve_script,
|
||||
# legacy/manager_core.py:1836-1843; precedent assertion shape
|
||||
# test_e2e_legacy_real_ops.py:516-538). NOT a shell-string match.
|
||||
state = _scripts_state()
|
||||
assert state != "ABSENT", (
|
||||
"install arm: install-scripts.txt missing after 200 — "
|
||||
"no reservation was written."
|
||||
)
|
||||
fixture_lines = [ln for ln in state.splitlines() if FIXTURE_URL in ln]
|
||||
assert fixture_lines, (
|
||||
f"install arm: no reservation line contains {FIXTURE_URL!r}.\n"
|
||||
f"file content:\n{state}"
|
||||
)
|
||||
reserved = ast.literal_eval(fixture_lines[0])
|
||||
assert isinstance(reserved, list) and "#FORCE" in reserved, (
|
||||
f"install arm: reservation line is not the expected "
|
||||
f"['..', '#FORCE', <pip cmd>...] list-repr shape: {reserved!r}"
|
||||
)
|
||||
|
||||
# Step 4 — restart: the production execution path for the
|
||||
# reservation (prestartup_script.py:485 consumes the file at
|
||||
# server start). Budget rationale at RESTART_BUDGET_S — the
|
||||
# owned zero-dep fixture installs in seconds; the budget covers
|
||||
# the plain server boot, no build env vars needed (goal329
|
||||
# retired the CUDA-build determinism knob with the heavy
|
||||
# upstream package).
|
||||
_stop_legacy()
|
||||
_start_legacy_install({}, shell_timeout=RESTART_BUDGET_S)
|
||||
|
||||
# Step 5 — post-restart: reservation CONSUMED (prestartup removes
|
||||
# the processed file) and the fixture REALLY importable in the
|
||||
# venv, proven by its MARKER value (stronger than bare import:
|
||||
# the marker ties the import to the owned fixture's content).
|
||||
post_state = _scripts_state()
|
||||
assert post_state == "ABSENT" or FIXTURE_URL not in post_state, (
|
||||
f"install arm: reservation NOT consumed by the restart — "
|
||||
f"install-scripts.txt still references the fixture:\n{post_state}"
|
||||
)
|
||||
probe = _venv_fixture_probe()
|
||||
assert probe.returncode == 0, (
|
||||
f"install arm: `import {FIXTURE_MODULE}` still fails in the "
|
||||
f"E2E venv after the reservation-executing restart — install "
|
||||
f"did not happen or did not target the venv.\n"
|
||||
f"stderr:\n{probe.stderr[-500:]}"
|
||||
)
|
||||
assert FIXTURE_MARKER in probe.stdout, (
|
||||
f"install arm: fixture imported but MARKER mismatch — "
|
||||
f"expected {FIXTURE_MARKER!r} in stdout, got: {probe.stdout!r}"
|
||||
)
|
||||
# Version breadcrumb for triage (owned repo, so drift risk is
|
||||
# retired — the breadcrumb now just documents what was installed).
|
||||
show = subprocess.run(
|
||||
[VENV_UV, "pip", "show", FIXTURE_DIST, "--python", VENV_PY],
|
||||
capture_output=True, text=True, timeout=60,
|
||||
)
|
||||
print(f"\n[goal329 install-evidence] uv pip show {FIXTURE_DIST}:\n"
|
||||
f"{show.stdout}")
|
||||
641
tests/e2e/test_e2e_secgate_legacy_flags.py
Normal file
641
tests/e2e/test_e2e_secgate_legacy_flags.py
Normal file
@ -0,0 +1,641 @@
|
||||
"""[goal265 step4 — RED] E2E endpoint binding for the dedicated install flags
|
||||
``allow_git_url_install`` / ``allow_pip_install`` on the LEGACY server.
|
||||
|
||||
TARGET CONTRACT (NOT YET IMPLEMENTED — goal265-spec.md §1, LOCKED):
|
||||
|
||||
S-A POST /v2/customnode/install/git_url gated by allow_git_url_install
|
||||
S-B POST /v2/customnode/install/pip gated by allow_pip_install
|
||||
S-C batch unknown-URL install: retained middle+ entry gate AND the
|
||||
allow_git_url_install full predicate replaces the high+ risky check;
|
||||
unknown-pip 'block' branch stays UNCONDITIONAL (Q1).
|
||||
The security_level term is fully REMOVED from S-A/S-B (spec §1.1 inv. 1);
|
||||
denial logs name the responsible flag, not the security level (inv. 6).
|
||||
|
||||
These tests are EXPECTED TO FAIL against current code (today S-A/S-B are
|
||||
``is_allowed_security_level('high+')``-gated) — RED confirmation is Step 5.
|
||||
Do NOT weaken them to pass against today's code.
|
||||
|
||||
This module delivers the LGU2/LPP2 legacy-fixture follow-up explicitly
|
||||
deferred by tests/e2e/test_e2e_secgate_default.py (its header, lines 31-34).
|
||||
|
||||
SC rows covered (goal265-scenarios.md), grouped by server config combination
|
||||
(one legacy-server launch per combination, config.ini staged per flags):
|
||||
|
||||
CFG-A (git=T, pip=T, sl=strong, nm=public): SC-01, SC-11, SC-08
|
||||
CFG-B (git=T, pip=F, sl=normal, nm=public): SC-02, SC-04, SC-10, SC-17,
|
||||
SC-18 (transitive-pip arm)
|
||||
CFG-C (git=F, pip=T, sl=normal, nm=public): SC-16
|
||||
CFG-D (git=F, pip=F, sl=weak, nm=public): SC-05, SC-09, SC-13,
|
||||
SC-23 (denial-log copy arm)
|
||||
CFG-E (git=T, pip=T, sl=weak, nm=public): SC-24 (Q1 unknown-pip block)
|
||||
CFG-F (git=T, pip=T, sl=normal, nm=public): SC-25A/B (guards, flags-on)
|
||||
CFG-G (git=F, pip=F, sl=normal, nm=public): SC-25A/B (guards, flags-off)
|
||||
CFG-H (git=T, pip=F, sl=normal, nm=personal_cloud): SC-30 (note below)
|
||||
|
||||
SC-30 limitation (deliberate, documented): the E2E harness listens on
|
||||
127.0.0.1, so the personal_cloud arm of the middle+ entry gate is satisfied
|
||||
via is_local_mode as well; the public-listener arm of personal_cloud is
|
||||
proven at predicate level (tests/common/test_install_flag_predicate.py
|
||||
SC-03/SC-12) — this row re-proves the endpoint binding with
|
||||
network_mode=personal_cloud staged.
|
||||
|
||||
SC-18 strategy: endpoint-level arm — git-URL install transaction completes
|
||||
(200) with allow_pip_install=false and the captured log slice contains NO
|
||||
allow_pip_install denial, proving the pip flag is never consulted inside a
|
||||
git transaction (MM §1.4). The spec's integration-level alternative
|
||||
(in-process gitclone_install with execute_install_script mocked) remains
|
||||
open to Step-7 if this arm proves under-discriminating.
|
||||
|
||||
Side-effect control: batch rows pre-seed the target node directory so the
|
||||
queue worker's gitclone_install short-circuits on "Already exists" — the
|
||||
GATE decision (the contract under test) happens synchronously in
|
||||
_install_custom_node BEFORE the worker runs, so the batch response's
|
||||
``failed`` list is a complete gate observable without real clones.
|
||||
|
||||
Requires a pre-built E2E environment (setup_e2e_env.sh); skip-marked
|
||||
otherwise (harness precedent: tests/e2e/test_e2e_secgate_default.py).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import configparser
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
import requests
|
||||
|
||||
E2E_ROOT = os.environ.get("E2E_ROOT", "")
|
||||
COMFYUI_PATH = os.path.join(E2E_ROOT, "comfyui") if E2E_ROOT else ""
|
||||
SCRIPTS_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), "scripts")
|
||||
|
||||
PORT = 8199
|
||||
BASE_URL = f"http://127.0.0.1:{PORT}"
|
||||
|
||||
CONFIG_PATH = os.path.join(COMFYUI_PATH, "user", "__manager", "config.ini")
|
||||
CONFIG_BACKUP = CONFIG_PATH + ".before-flags"
|
||||
SERVER_LOG = os.path.join(E2E_ROOT, "logs", "comfyui.log") if E2E_ROOT else ""
|
||||
|
||||
# Purpose-built test-fixture repo (same constant as
|
||||
# tests/e2e/test_e2e_legacy_real_ops.py TRUSTED_GIT_URL) — the DESIGNATED
|
||||
# do-not-install sample for every S-A surface test that performs a REAL
|
||||
# clone (direct /v2/customnode/install/git_url).
|
||||
UNKNOWN_GIT_URL = "https://github.com/ltdrdata/nodepack-test1-do-not-install"
|
||||
UNKNOWN_GIT_DIRNAME = "nodepack-test1-do-not-install"
|
||||
|
||||
# Batch S-C "unknown-URL" rows need a files URL genuinely NOT in the
|
||||
# custom-node DB (scenario preconditions SC-04/08/09/30: "file URL NOT in
|
||||
# node DB" -> get_risky_level returns 'high+' -> the dedicated-flag gate
|
||||
# engages). DISCOVERY (task #297 real-server run): the do-not-install
|
||||
# sample above IS registered in the Comfy-Org main-channel
|
||||
# custom-node-list (served from the manager cache), so get_risky_level
|
||||
# returns 'middle+' for it and the flag gate is bypassed — using it for
|
||||
# batch rows silently tests the wrong path (observed as an SC-09
|
||||
# false-FAIL: flag=false yet the item queued via middle+ at sl=weak).
|
||||
# The batch rows therefore use a SYNTHETIC nonexistent URL. ZERO-INSTALL
|
||||
# guarantee holds by construction: deny rows (SC-08/09) never reach the
|
||||
# worker, and allow rows (SC-04/30) pre-seed the matching placeholder dir
|
||||
# so the worker's gitclone_install short-circuits on "Already exists" —
|
||||
# the URL is never cloned (and could not be: the repo does not exist).
|
||||
BATCH_UNKNOWN_GIT_URL = "https://github.com/ltdrdata/goal265-nonexistent-unknown-node"
|
||||
BATCH_UNKNOWN_DIRNAME = "goal265-nonexistent-unknown-node"
|
||||
|
||||
# Long-standing custom-node-list.json entry — used by SC-24/SC-25B rows that
|
||||
# need a files URL the DB KNOWS (get_risky_level must not short-circuit at
|
||||
# high+ on the URL check).
|
||||
KNOWN_GIT_URL = "https://github.com/ltdrdata/ComfyUI-Impact-Pack"
|
||||
KNOWN_GIT_DIRNAME = "ComfyUI-Impact-Pack"
|
||||
|
||||
UNKNOWN_PIP_PKG = "cm-goal265-nonexistent-pip-pkg-xyz"
|
||||
|
||||
pytestmark = pytest.mark.skipif(
|
||||
not E2E_ROOT
|
||||
or not os.path.isfile(os.path.join(E2E_ROOT, ".e2e_setup_complete")),
|
||||
reason="E2E_ROOT not set or E2E environment not ready",
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Config staging + server lifecycle helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _stage_flags_config(git: bool, pip: bool, security_level: str,
|
||||
network_mode: str) -> None:
|
||||
"""Patch config.ini with the row preconditions (backup-once pattern,
|
||||
mirrors start_comfyui_strict.sh)."""
|
||||
if not os.path.isfile(CONFIG_BACKUP):
|
||||
shutil.copy(CONFIG_PATH, CONFIG_BACKUP)
|
||||
|
||||
cfg = configparser.ConfigParser(strict=False)
|
||||
cfg.read(CONFIG_PATH)
|
||||
if "default" not in cfg:
|
||||
cfg["default"] = {}
|
||||
cfg["default"]["allow_git_url_install"] = "true" if git else "false"
|
||||
cfg["default"]["allow_pip_install"] = "true" if pip else "false"
|
||||
cfg["default"]["security_level"] = security_level
|
||||
cfg["default"]["network_mode"] = network_mode
|
||||
with open(CONFIG_PATH, "w", encoding="utf-8") as f:
|
||||
cfg.write(f)
|
||||
|
||||
|
||||
def _restore_config() -> None:
|
||||
if os.path.isfile(CONFIG_BACKUP):
|
||||
shutil.move(CONFIG_BACKUP, CONFIG_PATH)
|
||||
|
||||
|
||||
def _start_legacy() -> int:
|
||||
env = {**os.environ, "E2E_ROOT": E2E_ROOT, "PORT": str(PORT)}
|
||||
r = subprocess.run(
|
||||
["bash", os.path.join(SCRIPTS_DIR, "start_comfyui_legacy.sh")],
|
||||
capture_output=True, text=True, timeout=180, env=env,
|
||||
)
|
||||
if r.returncode != 0:
|
||||
raise RuntimeError(f"Failed to start ComfyUI (legacy):\n{r.stderr}")
|
||||
for part in r.stdout.strip().split():
|
||||
if part.startswith("COMFYUI_PID="):
|
||||
return int(part.split("=")[1])
|
||||
raise RuntimeError(f"Could not parse PID:\n{r.stdout}")
|
||||
|
||||
|
||||
def _stop_legacy() -> None:
|
||||
env = {**os.environ, "E2E_ROOT": E2E_ROOT, "PORT": str(PORT)}
|
||||
subprocess.run(
|
||||
["bash", os.path.join(SCRIPTS_DIR, "stop_comfyui.sh")],
|
||||
capture_output=True, text=True, timeout=30, env=env,
|
||||
)
|
||||
|
||||
|
||||
def _make_flags_server_fixture(git: bool, pip: bool, security_level: str,
|
||||
network_mode: str = "public"):
|
||||
"""Class-scoped fixture factory: stage config -> start legacy server ->
|
||||
yield -> stop server -> restore config."""
|
||||
|
||||
@pytest.fixture(scope="class")
|
||||
def _server():
|
||||
_stage_flags_config(git, pip, security_level, network_mode)
|
||||
try:
|
||||
pid = _start_legacy()
|
||||
try:
|
||||
yield pid
|
||||
finally:
|
||||
_stop_legacy()
|
||||
finally:
|
||||
_restore_config()
|
||||
|
||||
return _server
|
||||
|
||||
|
||||
comfyui_flags_a = _make_flags_server_fixture(True, True, "strong")
|
||||
comfyui_flags_b = _make_flags_server_fixture(True, False, "normal")
|
||||
comfyui_flags_c = _make_flags_server_fixture(False, True, "normal")
|
||||
comfyui_flags_d = _make_flags_server_fixture(False, False, "weak")
|
||||
comfyui_flags_e = _make_flags_server_fixture(True, True, "weak")
|
||||
comfyui_flags_f = _make_flags_server_fixture(True, True, "normal")
|
||||
comfyui_flags_g = _make_flags_server_fixture(False, False, "normal")
|
||||
comfyui_flags_h = _make_flags_server_fixture(
|
||||
True, False, "normal", network_mode="personal_cloud")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Request / observation helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _post_git_url(url: str) -> requests.Response:
|
||||
return requests.post(
|
||||
f"{BASE_URL}/v2/customnode/install/git_url", data=url, timeout=120,
|
||||
)
|
||||
|
||||
|
||||
def _post_pip(packages: str) -> requests.Response:
|
||||
return requests.post(
|
||||
f"{BASE_URL}/v2/customnode/install/pip", data=packages, timeout=60,
|
||||
)
|
||||
|
||||
|
||||
def _batch_install_item(ui_id: str, files: list[str],
|
||||
pip: list[str] | None = None) -> dict:
|
||||
"""Unknown-version batch install item shape consumed by
|
||||
legacy _install_custom_node (version='unknown' -> files URL path)."""
|
||||
return {
|
||||
"id": ui_id,
|
||||
"ui_id": ui_id,
|
||||
"version": "unknown",
|
||||
"files": files,
|
||||
"pip": pip or [],
|
||||
"channel": "default",
|
||||
"mode": "cache",
|
||||
}
|
||||
|
||||
|
||||
def _post_batch(payload: dict) -> requests.Response:
|
||||
return requests.post(
|
||||
f"{BASE_URL}/v2/manager/queue/batch", json=payload, timeout=120,
|
||||
)
|
||||
|
||||
|
||||
def _log_offset() -> int:
|
||||
return os.path.getsize(SERVER_LOG) if os.path.isfile(SERVER_LOG) else 0
|
||||
|
||||
|
||||
def _log_slice(offset: int) -> str:
|
||||
if not os.path.isfile(SERVER_LOG):
|
||||
return ""
|
||||
with open(SERVER_LOG, "r", encoding="utf-8", errors="replace") as f:
|
||||
f.seek(offset)
|
||||
return f.read()
|
||||
|
||||
|
||||
def _custom_nodes_dir() -> str:
|
||||
return os.path.join(COMFYUI_PATH, "custom_nodes")
|
||||
|
||||
|
||||
def _preseed_node_dir(dirname: str) -> str:
|
||||
"""Create a placeholder node dir so the queue worker's gitclone_install
|
||||
short-circuits on 'Already exists' (no real clone; the gate decision
|
||||
under test already happened synchronously)."""
|
||||
target = os.path.join(_custom_nodes_dir(), dirname)
|
||||
os.makedirs(target, exist_ok=True)
|
||||
marker = os.path.join(target, ".goal265-preseed")
|
||||
with open(marker, "w", encoding="utf-8") as f:
|
||||
f.write("goal265 step4 gate-test placeholder\n")
|
||||
return target
|
||||
|
||||
|
||||
def _remove_node_dir(dirname: str) -> None:
|
||||
for candidate in (
|
||||
os.path.join(_custom_nodes_dir(), dirname),
|
||||
os.path.join(_custom_nodes_dir(), ".disabled", dirname),
|
||||
os.path.join(_custom_nodes_dir(), dirname + ".disabled"),
|
||||
):
|
||||
if os.path.isdir(candidate):
|
||||
shutil.rmtree(candidate, ignore_errors=True)
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
# CFG-A: git=T, pip=T, sl=strong, nm=public
|
||||
# ===========================================================================
|
||||
|
||||
class TestFlagsOnStrongLevel:
|
||||
"""Allow-direction decoupling: flags=true must allow S-A/S-B even at
|
||||
security_level=strong (today: 403). Batch entry gate stays
|
||||
security_level-coupled (middle+ fails at strong)."""
|
||||
|
||||
def test_sc01_git_url_allowed_at_strong_when_flag_true(
|
||||
self, comfyui_flags_a):
|
||||
"""SC-01: git=true, sl=strong, loopback -> POST git_url returns 200
|
||||
(today 403 — proves security_level irrelevance, allow direction).
|
||||
Pre-removes the fixture node so a real (tiny) clone proceeds; a
|
||||
200 via res.action=='skip' is equally a gate-pass observable."""
|
||||
_remove_node_dir(UNKNOWN_GIT_DIRNAME)
|
||||
try:
|
||||
resp = _post_git_url(UNKNOWN_GIT_URL)
|
||||
assert resp.status_code == 200, (
|
||||
f"SC-01: expected 200 (flag=true overrides sl=strong), got "
|
||||
f"{resp.status_code} — security_level still gates S-A. "
|
||||
f"Body: {resp.text[:200]}"
|
||||
)
|
||||
finally:
|
||||
_remove_node_dir(UNKNOWN_GIT_DIRNAME)
|
||||
|
||||
def test_sc11_pip_allowed_at_strong_when_flag_true(self, comfyui_flags_a):
|
||||
"""SC-11: pip=true, sl=strong, loopback -> POST pip returns 200
|
||||
(today 403). text-unidecode: pure-Python, tiny, idempotent — same
|
||||
trusted constant as test_e2e_legacy_real_ops.py."""
|
||||
resp = _post_pip("text-unidecode")
|
||||
assert resp.status_code == 200, (
|
||||
f"SC-11: expected 200 (flag=true overrides sl=strong), got "
|
||||
f"{resp.status_code} — security_level still gates S-B."
|
||||
)
|
||||
|
||||
def test_sc08_batch_unknown_url_denied_at_strong(self, comfyui_flags_a):
|
||||
"""SC-08: git=true, sl=strong -> batch unknown-URL install DENIED —
|
||||
the retained middle+ entry gate fails first (composite gate:
|
||||
middle+ AND flag; spec §1.2 S-C row, ':1427' gate UNCHANGED)."""
|
||||
item = _batch_install_item("sc08-unknown", [BATCH_UNKNOWN_GIT_URL])
|
||||
resp = _post_batch({"install": [item]})
|
||||
assert resp.status_code == 200, resp.text[:200]
|
||||
failed = resp.json().get("failed", [])
|
||||
assert "sc08-unknown" in failed, (
|
||||
"SC-08: batch unknown-URL install must stay denied at "
|
||||
"sl=strong (middle+ entry gate) even with "
|
||||
f"allow_git_url_install=true — failed={failed!r}"
|
||||
)
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
# CFG-B: git=T, pip=F, sl=normal, nm=public
|
||||
# ===========================================================================
|
||||
|
||||
class TestGitFlagOnDefaultLevel:
|
||||
"""Default security_level: git flag alone opens S-A; pip stays closed."""
|
||||
|
||||
def test_sc02_git_url_allowed_at_default_level(self, comfyui_flags_b):
|
||||
"""SC-02: git=true, sl=normal (default), loopback -> 200
|
||||
(today 403). Self-cleaning (pre+post remove): a leftover clone
|
||||
would turn the NEXT install of the same repo into an
|
||||
'Already exists' 400 (gitclone_install unknown-repo path has no
|
||||
skip arm — observed as an SC-17 false-FAIL in the first #297 run)."""
|
||||
_remove_node_dir(UNKNOWN_GIT_DIRNAME)
|
||||
try:
|
||||
resp = _post_git_url(UNKNOWN_GIT_URL)
|
||||
assert resp.status_code == 200, (
|
||||
f"SC-02: expected 200 at sl=normal with flag=true, got "
|
||||
f"{resp.status_code}. Body: {resp.text[:200]}"
|
||||
)
|
||||
finally:
|
||||
_remove_node_dir(UNKNOWN_GIT_DIRNAME)
|
||||
|
||||
def test_sc10_invalid_url_reaches_validation_400(self, comfyui_flags_b):
|
||||
"""SC-10: git=true, sl=normal -> POST an INVALID url -> 400 from
|
||||
installer validation, NOT 403 from the gate. Distinguishes gate-403
|
||||
from install-400 (today the gate 403s before URL validation)."""
|
||||
resp = _post_git_url("not-a-url")
|
||||
assert resp.status_code == 400, (
|
||||
f"SC-10: expected 400 (installer validation), got "
|
||||
f"{resp.status_code} — a 403 means the flag gate did not open."
|
||||
)
|
||||
|
||||
def test_sc17_git_open_pip_closed_independent(self, comfyui_flags_b):
|
||||
"""SC-17: git=true, pip=false, sl=normal -> git_url 200, pip 403
|
||||
(flags fully independent, [D1][D2]). Self-cleaning like SC-02 —
|
||||
a stale clone from an earlier S-A test would 400 the git arm."""
|
||||
_remove_node_dir(UNKNOWN_GIT_DIRNAME)
|
||||
try:
|
||||
git_resp = _post_git_url(UNKNOWN_GIT_URL)
|
||||
pip_resp = _post_pip("text-unidecode")
|
||||
assert git_resp.status_code == 200, (
|
||||
f"SC-17 git arm: expected 200, got {git_resp.status_code}"
|
||||
)
|
||||
assert pip_resp.status_code == 403, (
|
||||
f"SC-17 pip arm: expected 403, got {pip_resp.status_code}"
|
||||
)
|
||||
finally:
|
||||
_remove_node_dir(UNKNOWN_GIT_DIRNAME)
|
||||
|
||||
def test_sc18_git_transaction_never_consults_pip_flag(
|
||||
self, comfyui_flags_b):
|
||||
"""SC-18: git=true, pip=false -> a git-URL install transaction
|
||||
(clone + execute_install_script dependency step) COMPLETES, and the
|
||||
captured log slice contains no allow_pip_install denial — the pip
|
||||
flag governs ONLY the standalone S-B surface (MM §1.4 transaction
|
||||
scope; spec §1.1 invariant 4).
|
||||
|
||||
Fresh install forced (pre-remove) so execute_install_script actually
|
||||
runs inside the transaction window."""
|
||||
_remove_node_dir(UNKNOWN_GIT_DIRNAME)
|
||||
offset = _log_offset()
|
||||
try:
|
||||
resp = _post_git_url(UNKNOWN_GIT_URL)
|
||||
log = _log_slice(offset)
|
||||
assert resp.status_code == 200, (
|
||||
f"SC-18: git transaction failed ({resp.status_code}) with "
|
||||
"pip=false — transitive scope broken? Body: "
|
||||
f"{resp.text[:200]}"
|
||||
)
|
||||
assert "allow_pip_install" not in log, (
|
||||
"SC-18: the git-URL install transaction consulted/logged "
|
||||
"allow_pip_install — per-surface scope violated (MM §1.4):\n"
|
||||
+ log[-2000:]
|
||||
)
|
||||
finally:
|
||||
_remove_node_dir(UNKNOWN_GIT_DIRNAME)
|
||||
|
||||
def test_sc04_batch_unknown_url_queued_at_default_level(
|
||||
self, comfyui_flags_b):
|
||||
"""SC-04: git=true, sl=normal -> batch unknown-URL install is
|
||||
QUEUED (failed list empty): middle+ entry passes at normal local
|
||||
AND the flag replaces the high+ risky check (today 403: risky
|
||||
high+ fails at normal). Target dir pre-seeded — see module
|
||||
docstring 'Side-effect control'."""
|
||||
_preseed_node_dir(BATCH_UNKNOWN_DIRNAME)
|
||||
try:
|
||||
item = _batch_install_item("sc04-unknown", [BATCH_UNKNOWN_GIT_URL])
|
||||
resp = _post_batch({"install": [item]})
|
||||
assert resp.status_code == 200, resp.text[:200]
|
||||
failed = resp.json().get("failed", [])
|
||||
assert "sc04-unknown" not in failed, (
|
||||
"SC-04: batch unknown-URL install still denied at "
|
||||
"sl=normal with allow_git_url_install=true — the risky "
|
||||
f"high+ check was not replaced by the flag. failed={failed!r}"
|
||||
)
|
||||
finally:
|
||||
_remove_node_dir(BATCH_UNKNOWN_DIRNAME)
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
# CFG-C: git=F, pip=T, sl=normal, nm=public
|
||||
# ===========================================================================
|
||||
|
||||
class TestPipFlagOnDefaultLevel:
|
||||
def test_sc16_git_closed_pip_open_independent(self, comfyui_flags_c):
|
||||
"""SC-16: git=false, pip=true, sl=normal -> git_url 403, pip 200
|
||||
(flags fully independent, [D1][D2])."""
|
||||
git_resp = _post_git_url(UNKNOWN_GIT_URL)
|
||||
pip_resp = _post_pip("text-unidecode")
|
||||
assert git_resp.status_code == 403, (
|
||||
f"SC-16 git arm: expected 403, got {git_resp.status_code}"
|
||||
)
|
||||
assert pip_resp.status_code == 200, (
|
||||
f"SC-16 pip arm: expected 200, got {pip_resp.status_code}"
|
||||
)
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
# CFG-D: git=F, pip=F, sl=weak, nm=public
|
||||
# ===========================================================================
|
||||
|
||||
class TestFlagsOffWeakLevel:
|
||||
"""Deny-direction decoupling: flags=false must deny S-A/S-B even at
|
||||
security_level=weak (today: 200) — and the denial log must name the
|
||||
responsible flag (SC-23 log arm, spec §1.3)."""
|
||||
|
||||
def test_sc05_sc23_git_url_denied_at_weak_log_names_flag(
|
||||
self, comfyui_flags_d):
|
||||
"""SC-05: git=false, sl=weak, loopback -> 403 (today 200 — deny
|
||||
direction of security_level irrelevance).
|
||||
SC-23 (log arm): the denial log names allow_git_url_install and
|
||||
drops the security-level framing (SECURITY_MESSAGE_NORMAL_MINUS)."""
|
||||
offset = _log_offset()
|
||||
resp = _post_git_url(UNKNOWN_GIT_URL)
|
||||
log = _log_slice(offset)
|
||||
|
||||
assert resp.status_code == 403, (
|
||||
f"SC-05: expected 403 (flag=false overrides sl=weak), got "
|
||||
f"{resp.status_code} — security_level still opens S-A."
|
||||
)
|
||||
assert "allow_git_url_install" in log, (
|
||||
"SC-23: denial log does not name allow_git_url_install "
|
||||
f"(spec §1.3 SECURITY_MESSAGE_FLAG_GIT_URL). Slice:\n{log[-1500:]}"
|
||||
)
|
||||
assert "security level to 'normal-'" not in log, (
|
||||
"SC-23: denial log still carries the misleading "
|
||||
"security-level copy (SECURITY_MESSAGE_NORMAL_MINUS)."
|
||||
)
|
||||
|
||||
def test_sc13_sc23_pip_denied_at_weak_log_names_flag(
|
||||
self, comfyui_flags_d):
|
||||
"""SC-13: pip=false, sl=weak, loopback -> 403 (today 200).
|
||||
SC-23 (log arm): denial log names allow_pip_install
|
||||
(spec §1.3 SECURITY_MESSAGE_FLAG_PIP)."""
|
||||
offset = _log_offset()
|
||||
resp = _post_pip("text-unidecode")
|
||||
log = _log_slice(offset)
|
||||
|
||||
assert resp.status_code == 403, (
|
||||
f"SC-13: expected 403 (flag=false overrides sl=weak), got "
|
||||
f"{resp.status_code} — security_level still opens S-B."
|
||||
)
|
||||
assert "allow_pip_install" in log, (
|
||||
"SC-23: denial log does not name allow_pip_install "
|
||||
f"(spec §1.3 SECURITY_MESSAGE_FLAG_PIP). Slice:\n{log[-1500:]}"
|
||||
)
|
||||
|
||||
def test_sc09_batch_unknown_url_denied_when_flag_false(
|
||||
self, comfyui_flags_d):
|
||||
"""SC-09: git=false, sl=weak -> batch unknown-URL install denied —
|
||||
the flag gate fails despite weak (today: allowed). Composite gate's
|
||||
flag term is the decider here (middle+ passes at weak local)."""
|
||||
item = _batch_install_item("sc09-unknown", [BATCH_UNKNOWN_GIT_URL])
|
||||
resp = _post_batch({"install": [item]})
|
||||
assert resp.status_code == 200, resp.text[:200]
|
||||
failed = resp.json().get("failed", [])
|
||||
assert "sc09-unknown" in failed, (
|
||||
"SC-09: batch unknown-URL install must be denied with "
|
||||
f"allow_git_url_install=false even at sl=weak. failed={failed!r}"
|
||||
)
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
# CFG-E: git=T, pip=T, sl=weak, nm=public — Q1 guard
|
||||
# ===========================================================================
|
||||
|
||||
class TestUnknownPipStaysBlocked:
|
||||
def test_sc24_batch_unknown_pip_block_is_unconditional(
|
||||
self, comfyui_flags_e):
|
||||
"""SC-24 (Q1 guard): batch item whose files are ALL DB-known (no
|
||||
high+ short-circuit on the URL check) but whose pip list names an
|
||||
UNKNOWN package -> still denied. get_risky_level's 'block' branch
|
||||
stays UNCONDITIONAL — the flags do NOT open it (spec §5 freeze
|
||||
item 7), even with both flags true at sl=weak."""
|
||||
item = _batch_install_item(
|
||||
"sc24-known-url-unknown-pip",
|
||||
[KNOWN_GIT_URL],
|
||||
pip=[UNKNOWN_PIP_PKG],
|
||||
)
|
||||
resp = _post_batch({"install": [item]})
|
||||
assert resp.status_code == 200, resp.text[:200]
|
||||
failed = resp.json().get("failed", [])
|
||||
assert "sc24-known-url-unknown-pip" in failed, (
|
||||
"SC-24: unknown-pip 'block' branch opened — it must remain "
|
||||
f"unconditional regardless of the new flags. failed={failed!r}"
|
||||
)
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
# CFG-F / CFG-G: out-of-scope legacy guards (SC-25A/B) — flags have NO effect
|
||||
# ===========================================================================
|
||||
|
||||
def _batch_op_item(ui_id: str) -> dict:
|
||||
"""Item for middle-gated batch ops (fix/uninstall/update). The GATE
|
||||
check precedes node resolution, so a placeholder spec is sufficient:
|
||||
the queue worker's later failure on the nonexistent node is irrelevant
|
||||
to the synchronous gate observable (the response's failed list)."""
|
||||
return {
|
||||
"id": ui_id,
|
||||
"ui_id": ui_id,
|
||||
"version": "unknown",
|
||||
"files": [f"https://example.com/{ui_id}"],
|
||||
}
|
||||
|
||||
|
||||
class _LegacyGuardRows:
|
||||
"""Shared assertions for SC-25A/B — executed under BOTH flag combos
|
||||
((t,t) via CFG-F and (f,f) via CFG-G); outcomes must be IDENTICAL,
|
||||
proving the flags have zero effect on middle/middle+ surfaces."""
|
||||
|
||||
flags_label = ""
|
||||
|
||||
def _assert_sc25a(self):
|
||||
"""SC-25A: legacy middle-gated ops (fix/uninstall/update) stay
|
||||
allowed at sl=normal local for this flag combination."""
|
||||
payload = {
|
||||
"fix": [_batch_op_item(f"sc25a-fix-{self.flags_label}")],
|
||||
"uninstall": [_batch_op_item(f"sc25a-unin-{self.flags_label}")],
|
||||
"update": [_batch_op_item(f"sc25a-upd-{self.flags_label}")],
|
||||
}
|
||||
resp = _post_batch(payload)
|
||||
assert resp.status_code == 200, resp.text[:200]
|
||||
failed = resp.json().get("failed", [])
|
||||
gate_denied = [x for x in failed if x.startswith("sc25a-")]
|
||||
assert gate_denied == [], (
|
||||
f"SC-25A({self.flags_label}): middle-gated batch ops were "
|
||||
f"gate-denied at sl=normal — flags must have ZERO effect on "
|
||||
f"out-of-scope surfaces. failed={failed!r}"
|
||||
)
|
||||
|
||||
def _assert_sc25b(self):
|
||||
"""SC-25B: batch KNOWN-node install (middle+ path, DB-resolved
|
||||
files URL) stays allowed at sl=normal local for this flag
|
||||
combination. Known-node dir pre-seeded — no real install."""
|
||||
_preseed_node_dir(KNOWN_GIT_DIRNAME)
|
||||
try:
|
||||
item = _batch_install_item(
|
||||
f"sc25b-known-{self.flags_label}", [KNOWN_GIT_URL])
|
||||
resp = _post_batch({"install": [item]})
|
||||
assert resp.status_code == 200, resp.text[:200]
|
||||
failed = resp.json().get("failed", [])
|
||||
assert f"sc25b-known-{self.flags_label}" not in failed, (
|
||||
f"SC-25B({self.flags_label}): KNOWN-node batch install was "
|
||||
f"denied at sl=normal — middle+ path must be unaffected by "
|
||||
f"the flags. failed={failed!r}"
|
||||
)
|
||||
finally:
|
||||
_remove_node_dir(KNOWN_GIT_DIRNAME)
|
||||
|
||||
|
||||
class TestGuardsFlagsOn(_LegacyGuardRows):
|
||||
flags_label = "flags-on"
|
||||
|
||||
def test_sc25a_middle_ops_allowed_with_flags_on(self, comfyui_flags_f):
|
||||
self._assert_sc25a()
|
||||
|
||||
def test_sc25b_known_install_allowed_with_flags_on(self, comfyui_flags_f):
|
||||
self._assert_sc25b()
|
||||
|
||||
|
||||
class TestGuardsFlagsOff(_LegacyGuardRows):
|
||||
flags_label = "flags-off"
|
||||
|
||||
def test_sc25a_middle_ops_allowed_with_flags_off(self, comfyui_flags_g):
|
||||
self._assert_sc25a()
|
||||
|
||||
def test_sc25b_known_install_allowed_with_flags_off(self, comfyui_flags_g):
|
||||
self._assert_sc25b()
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
# CFG-H: git=T, pip=F, sl=normal, nm=personal_cloud
|
||||
# ===========================================================================
|
||||
|
||||
class TestPersonalCloudBatch:
|
||||
def test_sc30_batch_unknown_url_queued_under_personal_cloud(
|
||||
self, comfyui_flags_h):
|
||||
"""SC-30: git=true, sl=normal, nm=personal_cloud -> batch
|
||||
unknown-URL install queued: the S-C composite gate's middle+ arm
|
||||
passes (is_local_mode OR is_personal_cloud) AND the flag is true.
|
||||
See module docstring for the loopback-harness limitation; the
|
||||
public-listener personal_cloud arm is proven at predicate level."""
|
||||
_preseed_node_dir(BATCH_UNKNOWN_DIRNAME)
|
||||
try:
|
||||
item = _batch_install_item("sc30-unknown", [BATCH_UNKNOWN_GIT_URL])
|
||||
resp = _post_batch({"install": [item]})
|
||||
assert resp.status_code == 200, resp.text[:200]
|
||||
failed = resp.json().get("failed", [])
|
||||
assert "sc30-unknown" not in failed, (
|
||||
"SC-30: batch unknown-URL install denied under "
|
||||
f"network_mode=personal_cloud with flag=true. "
|
||||
f"failed={failed!r}"
|
||||
)
|
||||
finally:
|
||||
_remove_node_dir(BATCH_UNKNOWN_DIRNAME)
|
||||
268
tests/test_install_flags_config.py
Normal file
268
tests/test_install_flags_config.py
Normal file
@ -0,0 +1,268 @@
|
||||
"""[goal265 step4 — RED] Dual-reader config contract for the dedicated
|
||||
install flags ``allow_git_url_install`` / ``allow_pip_install``.
|
||||
|
||||
TARGET CONTRACT (NOT YET IMPLEMENTED — goal265-spec.md §3, LOCKED):
|
||||
|
||||
- Keys: ``allow_git_url_install``, ``allow_pip_install`` in
|
||||
config.ini ``[default]``.
|
||||
- BOTH readers carry the keys (read dict + write_config list +
|
||||
exception-fallback dict — 3 anchors each):
|
||||
* ``comfyui_manager/glob/manager_core.py``
|
||||
* ``comfyui_manager/legacy/manager_core.py``
|
||||
(dual-reader rule — the gated endpoints resolve through the LEGACY
|
||||
reader; a glob-only registration would silently never reach the gates.)
|
||||
- Only case-insensitive string "true" is truthy on read; anything else
|
||||
(including "1", "yes") reads False.
|
||||
- Missing key reads False — the shared ``get_bool(key, default)`` quirk:
|
||||
the ``default`` param is IGNORED, missing key -> always False.
|
||||
- Missing file / missing [default] section -> exception-fallback dict
|
||||
returns False for both keys.
|
||||
- Activation is restart-only (module-level ``cached_config``).
|
||||
|
||||
These tests are EXPECTED TO FAIL against current code (the keys are not
|
||||
registered in either reader yet) — RED confirmation is goal265 Step 5.
|
||||
|
||||
SC rows covered (goal265-scenarios.md):
|
||||
SC-19 missing keys -> False (both readers)
|
||||
SC-20 malformed values -> only "true"/"TRUE"/"True" truthy (both readers)
|
||||
SC-21 write_config round-trip persistence (both readers)
|
||||
SC-22 cached_config staleness — restart-only activation (reader-level arm)
|
||||
SC-29 no config.ini / no [default] section -> fallback False (both readers)
|
||||
|
||||
Fixtures (spec §4 binding): tmp config.ini + monkeypatch
|
||||
``context.manager_config_path`` + per-reader ``cached_config`` reset.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
from _install_flags_testutil import import_context, import_reader # noqa: E402
|
||||
|
||||
FLAG_KEYS = ("allow_git_url_install", "allow_pip_install")
|
||||
READERS = ("glob", "legacy")
|
||||
|
||||
|
||||
def _reset_cache(core) -> None:
|
||||
"""Clear the reader's module-level cached_config (simulates restart)."""
|
||||
setattr(core, "cached_config", None)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Fixtures
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@pytest.fixture(params=READERS)
|
||||
def reader_core(request):
|
||||
"""Yield (reader_name, manager_core module) for BOTH readers, with the
|
||||
module-level cached_config saved/cleared around each test so one test's
|
||||
cache never leaks into another (or into other test modules)."""
|
||||
core = import_reader(request.param)
|
||||
saved = getattr(core, "cached_config", None)
|
||||
_reset_cache(core)
|
||||
yield request.param, core
|
||||
setattr(core, "cached_config", saved)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def point_config(monkeypatch):
|
||||
"""Return a function that points context.manager_config_path at a path.
|
||||
|
||||
Both readers resolve ``context.manager_config_path`` at CALL time
|
||||
(``config.read(context.manager_config_path)``), so monkeypatching the
|
||||
context attribute redirects glob AND legacy alike."""
|
||||
ctx = import_context()
|
||||
|
||||
def _point(path):
|
||||
monkeypatch.setattr(ctx, "manager_config_path", str(path))
|
||||
|
||||
return _point
|
||||
|
||||
|
||||
def _write_ini(tmp_path, body: str):
|
||||
ini = tmp_path / "config.ini"
|
||||
ini.write_text(body, encoding="utf-8")
|
||||
return ini
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SC-19 — missing keys read False (get_bool quirk: missing key -> False)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_sc19_missing_keys_read_false(reader_core, point_config, tmp_path):
|
||||
"""SC-19: config.ini WITHOUT either new key -> read_config returns
|
||||
allow_git_url_install == False AND allow_pip_install == False.
|
||||
|
||||
This is the observable manifestation of the shared ``get_bool(key,
|
||||
default)`` quirk (spec §3): the ``default`` parameter is IGNORED —
|
||||
a missing key always reads False, never the passed default. The quirk
|
||||
is documented, NOT fixed, by this GOAL (spec §5 freeze item 9)."""
|
||||
reader, core = reader_core
|
||||
point_config(_write_ini(tmp_path, "[default]\nsecurity_level = normal\n"))
|
||||
|
||||
cfg = core.read_config()
|
||||
|
||||
for key in FLAG_KEYS:
|
||||
assert key in cfg, (
|
||||
f"RED: {reader} read_config() does not register {key!r} "
|
||||
"(goal265-spec.md §3 reader registration)"
|
||||
)
|
||||
assert cfg[key] is False, f"{reader}: missing {key} must read False"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SC-20 — malformed values: only case-insensitive "true" is truthy
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("raw_value", "expected"),
|
||||
[
|
||||
("true", True),
|
||||
("TRUE", True),
|
||||
("True", True),
|
||||
("1", False),
|
||||
("yes", False),
|
||||
("false", False),
|
||||
("garbage", False),
|
||||
],
|
||||
ids=["true", "TRUE", "True", "1", "yes", "false", "garbage"],
|
||||
)
|
||||
@pytest.mark.parametrize("flag_key", FLAG_KEYS)
|
||||
def test_sc20_only_case_insensitive_true_is_truthy(
|
||||
reader_core, point_config, tmp_path, flag_key, raw_value, expected):
|
||||
"""SC-20: key present with malformed values -> only case-insensitive
|
||||
"true" reads True; "1"/"yes"/"false"/"garbage" read False (both flags,
|
||||
both readers)."""
|
||||
reader, core = reader_core
|
||||
point_config(_write_ini(
|
||||
tmp_path,
|
||||
f"[default]\nsecurity_level = normal\n{flag_key} = {raw_value}\n",
|
||||
))
|
||||
|
||||
cfg = core.read_config()
|
||||
|
||||
assert flag_key in cfg, (
|
||||
f"RED: {reader} read_config() does not register {flag_key!r}"
|
||||
)
|
||||
assert cfg[flag_key] is expected, (
|
||||
f"{reader}: {flag_key} = {raw_value!r} must read {expected}"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SC-21 — write_config round-trip (write list registration, both writers)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_sc21_write_config_round_trips_both_flags(
|
||||
reader_core, point_config, tmp_path):
|
||||
"""SC-21: both flags true in cached config -> write_config -> reset the
|
||||
reader's cached_config -> read_config -> both keys persist as True.
|
||||
|
||||
Failure mode this guards: keys registered in the read dict but NOT in
|
||||
the write_config persistence list — the value would silently drop on
|
||||
the next config save (spec §2 F3/F4 dual-anchor rule, residual risk R2)."""
|
||||
reader, core = reader_core
|
||||
ini = _write_ini(
|
||||
tmp_path,
|
||||
"[default]\nsecurity_level = normal\n"
|
||||
"allow_git_url_install = true\nallow_pip_install = true\n",
|
||||
)
|
||||
point_config(ini)
|
||||
|
||||
loaded = core.get_config()
|
||||
for key in FLAG_KEYS:
|
||||
assert loaded.get(key) is True, (
|
||||
f"RED: {reader} get_config() did not load {key!r}=True "
|
||||
"from staged config.ini"
|
||||
)
|
||||
|
||||
core.write_config()
|
||||
|
||||
raw = ini.read_text(encoding="utf-8")
|
||||
for key in FLAG_KEYS:
|
||||
assert key in raw, (
|
||||
f"{reader}: write_config() dropped {key!r} — key missing from "
|
||||
"the write_config persistence list (spec §3 reader registration)"
|
||||
)
|
||||
|
||||
_reset_cache(core) # per-reader cache reset REQUIRED before re-read
|
||||
cfg = core.read_config()
|
||||
for key in FLAG_KEYS:
|
||||
assert cfg.get(key) is True, (
|
||||
f"{reader}: {key} did not survive the write/read round-trip"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SC-22 — cached_config staleness: restart-only activation (reader arm)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@pytest.mark.parametrize("flag_key", FLAG_KEYS)
|
||||
def test_sc22_flag_edit_without_restart_stays_stale(
|
||||
reader_core, point_config, tmp_path, flag_key):
|
||||
"""SC-22: process running with cached flag=false -> config.ini edited to
|
||||
flag=true WITHOUT restart -> the reader still serves False (module-level
|
||||
cached_config; identical semantics to security_level today, MM §1.5).
|
||||
Clearing the cache (= restart) then serves True."""
|
||||
reader, core = reader_core
|
||||
ini = _write_ini(
|
||||
tmp_path,
|
||||
f"[default]\nsecurity_level = normal\n{flag_key} = false\n",
|
||||
)
|
||||
point_config(ini)
|
||||
|
||||
first = core.get_config()
|
||||
assert first.get(flag_key) is False, (
|
||||
f"RED: {reader} get_config() does not register {flag_key!r}"
|
||||
)
|
||||
|
||||
# Edit config.ini on disk — NO cache reset (no restart).
|
||||
_write_ini(
|
||||
tmp_path,
|
||||
f"[default]\nsecurity_level = normal\n{flag_key} = true\n",
|
||||
)
|
||||
|
||||
stale = core.get_config()
|
||||
assert stale.get(flag_key) is False, (
|
||||
f"{reader}: {flag_key} hot-reloaded — activation MUST be "
|
||||
"restart-only (cached_config, spec §3)"
|
||||
)
|
||||
|
||||
_reset_cache(core) # simulate restart
|
||||
fresh = core.get_config()
|
||||
assert fresh.get(flag_key) is True, (
|
||||
f"{reader}: {flag_key}=true not visible after cache reset (restart)"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SC-29 — exception-fallback path carries the new keys as False
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@pytest.mark.parametrize("breakage", ["missing_file", "no_default_section"])
|
||||
def test_sc29_fallback_path_returns_false_for_both_flags(
|
||||
reader_core, point_config, tmp_path, breakage):
|
||||
"""SC-29: NO config.ini present / config.ini WITHOUT [default] section ->
|
||||
read_config's exception-fallback dict returns False for BOTH new keys
|
||||
(fallback dicts are the third registration anchor per reader,
|
||||
spec §2 F3/F4)."""
|
||||
reader, core = reader_core
|
||||
if breakage == "missing_file":
|
||||
point_config(tmp_path / "does-not-exist" / "config.ini")
|
||||
else:
|
||||
point_config(_write_ini(tmp_path, "[other]\nsomething = 1\n"))
|
||||
|
||||
cfg = core.read_config()
|
||||
|
||||
for key in FLAG_KEYS:
|
||||
assert key in cfg, (
|
||||
f"RED: {reader} exception-fallback dict does not carry {key!r} "
|
||||
"(goal265-spec.md §3 'Reader registration' — fallback anchor)"
|
||||
)
|
||||
assert cfg[key] is False, (
|
||||
f"{reader}: fallback {key} must be False (secure by default)"
|
||||
)
|
||||
256
tests/test_install_flags_guards.py
Normal file
256
tests/test_install_flags_guards.py
Normal file
@ -0,0 +1,256 @@
|
||||
"""[goal265 step4] Out-of-scope GUARD tests + structural frontend-copy
|
||||
assertion for the dedicated install flags GOAL.
|
||||
|
||||
Two populations in this module (intentionally different RED/GREEN states):
|
||||
|
||||
1. GUARD rows (SC-25C, SC-26, SC-27, SC-28) — regression-intent '=' rows:
|
||||
they assert the out-of-scope FREEZE (goal265-spec.md §5) and are expected
|
||||
to PASS both TODAY and AFTER Step 6. A failure at any point means the
|
||||
implementation leaked outside the locked scope.
|
||||
|
||||
2. SC-23 frontend arm — Δ row, EXPECTED TO FAIL TODAY (RED): the
|
||||
js/common.js 403 error copy must name the responsible flag after Step 6
|
||||
(spec §1.4, Q2 in scope). Do NOT weaken it to pass against today's code.
|
||||
|
||||
Placement per spec §4 rows 4-5 ('existing glob suite extension or unit' /
|
||||
'structural — copy-string assertion'): authored as a NEW unit/structural
|
||||
module so the pre-existing suite stays untouched (Step-4 acceptance bar (d));
|
||||
layout follows the Step-2 hand-off recommendation
|
||||
(goal265-scenarios.md §6: test_install_flags_guards.py).
|
||||
|
||||
Functional rows import the GLOB copy of the gate matrix
|
||||
(comfyui_manager/glob/utils/security_utils.py) with the lightweight runtime
|
||||
stubs from tests/_install_flags_testutil.py; the legacy copy of the same
|
||||
matrix is exercised end-to-end by tests/e2e/test_e2e_secgate_legacy_flags.py
|
||||
(SC-25A/B) under a real server.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
from _install_flags_testutil import ( # noqa: E402
|
||||
REPO_ROOT,
|
||||
import_glob_security_utils,
|
||||
)
|
||||
|
||||
GLOB_SERVER_PATH = os.path.join(
|
||||
REPO_ROOT, "comfyui_manager", "glob", "manager_server.py")
|
||||
CM_CLI_PATH = os.path.join(REPO_ROOT, "cm_cli", "__main__.py")
|
||||
COMMON_JS_PATH = os.path.join(REPO_ROOT, "comfyui_manager", "js", "common.js")
|
||||
|
||||
FLAG_KEYS = ("allow_git_url_install", "allow_pip_install")
|
||||
|
||||
|
||||
def _read(path: str) -> str:
|
||||
with open(path, "r", encoding="utf-8") as f:
|
||||
return f.read()
|
||||
|
||||
|
||||
def _extract_js_function(source: str, name: str) -> str:
|
||||
"""Slice an `export async function <name>` block out of common.js
|
||||
(up to the next exported/function declaration or EOF)."""
|
||||
m = re.search(
|
||||
rf"export\s+async\s+function\s+{re.escape(name)}\b.*?"
|
||||
r"(?=\nexport\s|\nfunction\s|\Z)",
|
||||
source,
|
||||
re.DOTALL,
|
||||
)
|
||||
assert m, f"could not locate function {name!r} in js/common.js"
|
||||
return m.group(0)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SC-27 — glob route-table absence (guard, '=')
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_sc27_glob_registers_no_git_url_or_pip_install_route():
|
||||
"""SC-27 (guard): glob/manager_server.py must register NO
|
||||
/v2/customnode/install/git_url or /v2/customnode/install/pip route —
|
||||
no new glob HTTP surface under this GOAL (Q4 settled; spec §5 item 5).
|
||||
Structural: route-decorator scan of the glob server source."""
|
||||
src = _read(GLOB_SERVER_PATH)
|
||||
for path in ("/v2/customnode/install/git_url",
|
||||
"/v2/customnode/install/pip"):
|
||||
pattern = rf"routes\.(post|get|put|delete)\(\s*[\"']{re.escape(path)}[\"']"
|
||||
assert not re.search(pattern, src), (
|
||||
f"SC-27: glob manager_server registers {path} — Q4 forbids a "
|
||||
"new glob surface for these capabilities under this GOAL"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SC-26 — cm-cli path stays ungated (guard, '=')
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_sc26_cm_cli_install_path_has_no_gate():
|
||||
"""SC-26 (guard): the cm-cli install path (install_node ->
|
||||
core.gitclone_install, cm_cli/__main__.py) is a local operator tool —
|
||||
it must consult NEITHER the security_level gate NOR the new flags
|
||||
(spec §5 item 4: cm-cli untouched). Static source assertion — no clone
|
||||
is executed."""
|
||||
src = _read(CM_CLI_PATH)
|
||||
assert "is_allowed_security_level" not in src, (
|
||||
"SC-26: cm_cli/__main__.py grew a security_level gate — cm-cli "
|
||||
"must stay ungated under this GOAL"
|
||||
)
|
||||
for key in FLAG_KEYS:
|
||||
assert key not in src, (
|
||||
f"SC-26: cm_cli/__main__.py consults {key!r} — the dedicated "
|
||||
"flags must not gate the local operator CLI"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SC-25C — glob comfyui_switch_version stays high+ security_level-coupled
|
||||
# (guard, '=') — structural arm
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_sc25c_glob_switch_version_keeps_high_plus_gate():
|
||||
"""SC-25C (guard, structural arm): the glob comfyui_switch_version
|
||||
handler keeps its is_allowed_security_level('high+') guard and does not
|
||||
consult the new flags (spec §5 item 1). The functional arm (high+ denies
|
||||
at sl=normal regardless of flags) is test_sc25c_sc28_* below."""
|
||||
src = _read(GLOB_SERVER_PATH)
|
||||
m = re.search(
|
||||
r"@routes\.post\(\s*[\"']/v2/comfyui_manager/comfyui_switch_version[\"']\s*\)"
|
||||
r".*?(?=\n@routes\.|\Z)",
|
||||
src,
|
||||
re.DOTALL,
|
||||
)
|
||||
assert m, "SC-25C: comfyui_switch_version route not found in glob server"
|
||||
handler = m.group(0)
|
||||
assert re.search(
|
||||
r"is_allowed_security_level\(\s*[\"']high\+[\"']", handler), (
|
||||
"SC-25C: comfyui_switch_version no longer carries the "
|
||||
"is_allowed_security_level('high+') guard — out-of-scope gate "
|
||||
"changed (spec §5 freeze item 1)"
|
||||
)
|
||||
for key in FLAG_KEYS:
|
||||
assert key not in handler, (
|
||||
f"SC-25C: comfyui_switch_version consults {key!r} — the new "
|
||||
"flags must not affect this surface"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SC-28 (+ SC-25C functional arm) — security_level matrix untouched
|
||||
# (guard, '=')
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# Frozen truth table of is_allowed_security_level at loopback listen,
|
||||
# network_mode=public (verified against legacy/manager_server.py:97-122 and
|
||||
# glob/utils/security_utils.py:22-48 as of base 01799f8c). The new flags
|
||||
# must leave every cell unchanged.
|
||||
_EXPECTED_MATRIX = {
|
||||
# (level, security_level): allowed
|
||||
("block", "weak"): False,
|
||||
("block", "normal"): False,
|
||||
("high+", "weak"): True,
|
||||
("high+", "normal-"): True,
|
||||
("high+", "normal"): False,
|
||||
("high+", "strong"): False,
|
||||
("high", "weak"): True,
|
||||
("high", "normal-"): True,
|
||||
("high", "normal"): False,
|
||||
("high", "strong"): False,
|
||||
("middle+", "weak"): True,
|
||||
("middle+", "normal-"): True,
|
||||
("middle+", "normal"): True,
|
||||
("middle+", "strong"): False,
|
||||
("middle", "weak"): True,
|
||||
("middle", "normal-"): True,
|
||||
("middle", "normal"): True,
|
||||
("middle", "strong"): False,
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.parametrize("flags_value", [True, False],
|
||||
ids=["flags-on", "flags-off"])
|
||||
def test_sc25c_sc28_security_level_matrix_unchanged_by_flags(
|
||||
monkeypatch, flags_value):
|
||||
"""SC-28 (guard): the is_allowed_security_level truth table for the
|
||||
non-target levels (middle/middle+/high/high+/block) is byte-identical
|
||||
whether both new flags are true or false — security_level semantics
|
||||
untouched (spec §5 item 2; MM §3).
|
||||
|
||||
SC-25C (functional arm): the ('high+', 'normal') -> False cell is the
|
||||
switch_version deny — it must hold for BOTH flag combinations.
|
||||
|
||||
Exercises the GLOB copy of the matrix (importable without a ComfyUI
|
||||
runtime); config access is monkeypatched at the consumer module so no
|
||||
real config.ini is read."""
|
||||
su = import_glob_security_utils()
|
||||
|
||||
fake_config = {
|
||||
"network_mode": "public",
|
||||
"allow_git_url_install": flags_value,
|
||||
"allow_pip_install": flags_value,
|
||||
}
|
||||
current_level = {"value": "normal"}
|
||||
|
||||
def fake_get_config():
|
||||
return {**fake_config, "security_level": current_level["value"]}
|
||||
|
||||
monkeypatch.setattr(su.core, "get_config", fake_get_config)
|
||||
monkeypatch.setattr(su.args, "listen", "127.0.0.1")
|
||||
|
||||
observed = {}
|
||||
for (level, sl) in _EXPECTED_MATRIX:
|
||||
current_level["value"] = sl
|
||||
observed[(level, sl)] = su.is_allowed_security_level(level)
|
||||
|
||||
mismatches = {
|
||||
cell: (got, _EXPECTED_MATRIX[cell])
|
||||
for cell, got in observed.items()
|
||||
if got is not _EXPECTED_MATRIX[cell]
|
||||
}
|
||||
assert not mismatches, (
|
||||
f"SC-28(flags={flags_value}): security_level matrix drifted — "
|
||||
f"{{cell: (got, expected)}} = {mismatches!r}. The dedicated-flag "
|
||||
"GOAL must not alter is_allowed_security_level semantics."
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SC-23 frontend arm — js/common.js error copy names the flag (Δ — RED today)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestSc23FrontendCopy:
|
||||
"""SC-23 (frontend arm, structural — spec §1.4 / Q2 in scope): on a
|
||||
non-200 install response, the user-visible copy must name the
|
||||
responsible flag, replacing the 'security level' framing.
|
||||
Copy-string assertion on js/common.js — not an executed-browser test
|
||||
(goal265-scenarios.md §6 note). EXPECTED TO FAIL TODAY (RED)."""
|
||||
|
||||
def test_sc23_install_pip_error_copy_names_pip_flag(self):
|
||||
"""install_pip error branch (~:213 call site) names
|
||||
allow_pip_install and drops the security-level framing."""
|
||||
block = _extract_js_function(_read(COMMON_JS_PATH), "install_pip")
|
||||
assert "allow_pip_install" in block, (
|
||||
"SC-23: js/common.js install_pip error copy does not name "
|
||||
"allow_pip_install (spec §1.4 frontend copy contract)"
|
||||
)
|
||||
assert "security level" not in block.lower(), (
|
||||
"SC-23: js/common.js install_pip still carries the misleading "
|
||||
"'security level' framing"
|
||||
)
|
||||
|
||||
def test_sc23_install_git_url_error_copy_names_git_flag(self):
|
||||
"""install_via_git_url error branch (~:248 call site) names
|
||||
allow_git_url_install and drops the security-level framing."""
|
||||
block = _extract_js_function(
|
||||
_read(COMMON_JS_PATH), "install_via_git_url")
|
||||
assert "allow_git_url_install" in block, (
|
||||
"SC-23: js/common.js install_via_git_url error copy does not "
|
||||
"name allow_git_url_install (spec §1.4 frontend copy contract)"
|
||||
)
|
||||
assert "security level" not in block.lower(), (
|
||||
"SC-23: js/common.js install_via_git_url still carries the "
|
||||
"misleading 'security level' framing"
|
||||
)
|
||||
340
tests/test_legacy_secgate_other_paths.py
Normal file
340
tests/test_legacy_secgate_other_paths.py
Normal file
@ -0,0 +1,340 @@
|
||||
"""[GOAL #347] Non-e2e unit regression guards for the OTHER legacy
|
||||
``security_level`` gate paths in ``comfyui_manager/legacy/manager_server.py``.
|
||||
|
||||
CONTEXT (GOAL #346 audit): the dedicated-install-flag work (git_url / pip
|
||||
endpoints) gained unit + e2e coverage, but the OTHER legacy gate sites — the
|
||||
ones that still gate on ``is_allowed_security_level(level)`` — had NO non-e2e
|
||||
coverage; only ``tests/e2e/`` exercised them, and only behind a real-server
|
||||
harness (E2E_ROOT). These guards close that gap WITHOUT a real server and
|
||||
WITHOUT modifying production code.
|
||||
|
||||
PATHS GUARDED (the OTHER security_level gates):
|
||||
P1 middle+ batch-install entry gate — ``_install_custom_node`` (L1441)
|
||||
P2 unknown-pip 'block' unconditional — ``_install_custom_node`` risky
|
||||
deny branch (``is_allowed_security_level
|
||||
('block')`` is hard-False; Q1)
|
||||
P3 nodepack/snapshot high+/middle — ``comfyui_switch_version`` (high+),
|
||||
gates ``restore_snapshot`` (middle+)
|
||||
P4 model non-.safetensors high+ gate — ``_install_model`` (L1713)
|
||||
|
||||
NON-VACUITY (the bar from the GOAL MM): every assertion below is written so
|
||||
that WEAKENING the underlying gate would FLIP the test result —
|
||||
* the decision-table tests drive the REAL ``is_allowed_security_level`` and
|
||||
pin the exact allow/deny truth for the level each path consumes; changing
|
||||
that function's logic for any level flips a row;
|
||||
* the response-shape tests invoke the REAL async handlers and assert the
|
||||
observable status (403/404/200) — each pairs a DENY case with an opposite
|
||||
(allow / flip) case proving the gate, not a constant, is what decides.
|
||||
A guard that merely imports or asserts a constant is explicitly avoided.
|
||||
|
||||
ISOLATION: reuses the ``tests/_install_flags_testutil.py`` stub approach
|
||||
(``ensure_comfy_stubs`` + ``_purge_fake_comfyui_manager``) and adds the
|
||||
minimal extra stubs ``manager_server`` needs at import (``server.PromptServer``
|
||||
route table, ``nodes``, ``folder_paths.__file__`` / ``get_folder_paths``), plus
|
||||
an inert ``manager_util.get_data`` so the module-level cache-warmup thread does
|
||||
no network I/O. No production code is touched.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import types
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
from _install_flags_testutil import ( # noqa: E402
|
||||
ensure_comfy_stubs,
|
||||
_purge_fake_comfyui_manager,
|
||||
)
|
||||
|
||||
_LEGACY_SERVER = None
|
||||
|
||||
|
||||
def _import_legacy_server():
|
||||
"""Import ``comfyui_manager.legacy.manager_server`` under minimal stubs
|
||||
(idempotent; memoized). Returns the real module object so the tests drive
|
||||
the REAL gate function and REAL handlers — only the runtime *environment*
|
||||
is stubbed, never the gate logic."""
|
||||
global _LEGACY_SERVER
|
||||
if _LEGACY_SERVER is not None:
|
||||
return _LEGACY_SERVER
|
||||
|
||||
ensure_comfy_stubs()
|
||||
_purge_fake_comfyui_manager()
|
||||
|
||||
# folder_paths: manager_server reads ``__file__`` and ``get_folder_paths``
|
||||
# at import (context.comfy_path derivation + check_invalid_nodes).
|
||||
fp = sys.modules["folder_paths"]
|
||||
if not hasattr(fp, "__file__"):
|
||||
fp.__file__ = os.path.join(tempfile.mkdtemp(prefix="cm-fp-"), "folder_paths.py")
|
||||
if not hasattr(fp, "get_folder_paths"):
|
||||
fp.get_folder_paths = lambda *a, **k: []
|
||||
|
||||
# nodes: imported but only used inside handlers we don't reach.
|
||||
sys.modules.setdefault("nodes", types.ModuleType("nodes"))
|
||||
|
||||
# server.PromptServer.instance.routes: the @routes.post/.get decorators run
|
||||
# at import. An identity-decorator route table lets every handler register
|
||||
# as a plain module-level function without a real aiohttp app.
|
||||
if "server" not in sys.modules:
|
||||
server = types.ModuleType("server")
|
||||
|
||||
class _Routes:
|
||||
def _identity(self, *a, **k):
|
||||
def deco(fn):
|
||||
return fn
|
||||
return deco
|
||||
|
||||
post = get = put = delete = _identity
|
||||
|
||||
class _Inst:
|
||||
routes = _Routes()
|
||||
|
||||
class PromptServer:
|
||||
instance = _Inst()
|
||||
|
||||
server.PromptServer = PromptServer
|
||||
sys.modules["server"] = server
|
||||
|
||||
# Make the module-level cache-warmup thread (manager_server L~1990
|
||||
# ``threading.Thread(... default_cache_update ...)``) inert — no network.
|
||||
import importlib
|
||||
mu = importlib.import_module("comfyui_manager.common.manager_util")
|
||||
|
||||
async def _inert_get_data(uri, *a, **k):
|
||||
return {"custom_nodes": [], "models": [], "node_map": {}}
|
||||
|
||||
mu.get_data = _inert_get_data
|
||||
|
||||
_LEGACY_SERVER = importlib.import_module("comfyui_manager.legacy.manager_server")
|
||||
return _LEGACY_SERVER
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def ms():
|
||||
return _import_legacy_server()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def gate(ms, monkeypatch):
|
||||
"""Return a function ``set_gate(security_level, network_mode='public',
|
||||
is_local_mode=True)`` that drives the gate inputs.
|
||||
|
||||
Builds a COMPLETE config dict off the reader's real fallback so any
|
||||
concurrent background reader (the warmup thread) never KeyErrors, then
|
||||
overrides only ``security_level`` / ``network_mode``. ``is_local_mode`` is
|
||||
the module global the gate consults. monkeypatch auto-restores."""
|
||||
base = dict(ms.core.read_config())
|
||||
|
||||
def set_gate(security_level, network_mode="public", is_local_mode=True):
|
||||
cfg = dict(base)
|
||||
cfg["security_level"] = security_level
|
||||
cfg["network_mode"] = network_mode
|
||||
monkeypatch.setattr(ms.core, "get_config", lambda: dict(cfg))
|
||||
monkeypatch.setattr(ms, "is_local_mode", is_local_mode)
|
||||
|
||||
return set_gate
|
||||
|
||||
|
||||
def _run(coro):
|
||||
return asyncio.run(coro)
|
||||
|
||||
|
||||
class FakeRequest:
|
||||
"""Minimal stand-in for an aiohttp request. ``content_type`` satisfies
|
||||
``reject_simple_form_post``; nothing else is read on the deny path (the
|
||||
gate returns before any body access)."""
|
||||
|
||||
def __init__(self, content_type="application/json"):
|
||||
self.content_type = content_type
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
# Decision-table guards — drive the REAL is_allowed_security_level for the
|
||||
# exact level each OTHER path consumes. Non-vacuous: weakening the function
|
||||
# for any level flips a row.
|
||||
# ===========================================================================
|
||||
|
||||
# (security_level, is_local_mode, network_mode, expected_allowed)
|
||||
_MIDDLE_PLUS = [ # P1 batch entry, _update_all, _install_model entry, restore_snapshot
|
||||
("weak", True, "public", True),
|
||||
("normal", True, "public", True),
|
||||
("normal-", True, "public", True),
|
||||
("strong", True, "public", False),
|
||||
("weak", False, "public", False), # public non-loopback -> denied
|
||||
("normal", False, "public", False),
|
||||
("weak", False, "personal_cloud", True), # personal_cloud arm of middle+
|
||||
("strong", False, "personal_cloud", False),
|
||||
]
|
||||
|
||||
_HIGH_PLUS = [ # P3 comfyui_switch_version, P4 model non-.safetensors sub-check
|
||||
("weak", True, "public", True),
|
||||
("normal-", True, "public", True),
|
||||
("normal", True, "public", False),
|
||||
("strong", True, "public", False),
|
||||
("weak", False, "public", False),
|
||||
("weak", False, "personal_cloud", True),
|
||||
("normal-", False, "personal_cloud", False), # personal_cloud high+ needs weak
|
||||
]
|
||||
|
||||
_MIDDLE = [ # P3 remove_snapshot and the other middle-gated legacy ops
|
||||
("weak", True, "public", True),
|
||||
("normal", True, "public", True),
|
||||
("normal-", True, "public", True),
|
||||
("strong", True, "public", False),
|
||||
("normal", False, "public", True), # 'middle' ignores locality
|
||||
("strong", False, "public", False),
|
||||
]
|
||||
|
||||
_BLOCK_LEVELS = ["weak", "normal", "normal-", "strong"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(("sl", "local", "nm", "expected"), _MIDDLE_PLUS)
|
||||
def test_p1_middle_plus_decision(ms, gate, sl, local, nm, expected):
|
||||
"""P1: the middle+ batch-install entry gate decision. is_allowed_security_level
|
||||
('middle+') must allow iff sl in {weak,normal,normal-} AND
|
||||
(loopback OR personal_cloud)."""
|
||||
gate(sl, network_mode=nm, is_local_mode=local)
|
||||
assert ms.is_allowed_security_level("middle+") is expected
|
||||
|
||||
|
||||
@pytest.mark.parametrize("sl", _BLOCK_LEVELS)
|
||||
@pytest.mark.parametrize("local", [True, False], ids=["loopback", "public"])
|
||||
@pytest.mark.parametrize("nm", ["public", "personal_cloud"])
|
||||
def test_p2_block_is_unconditional_deny(ms, gate, sl, local, nm):
|
||||
"""P2: unknown-pip 'block' is an UNCONDITIONAL deny — is_allowed_security_level
|
||||
('block') is False for EVERY security_level, locality, and network_mode
|
||||
(Q1: the dedicated flags do not open it). If 'block' ever became
|
||||
allowable, every row here flips."""
|
||||
gate(sl, network_mode=nm, is_local_mode=local)
|
||||
assert ms.is_allowed_security_level("block") is False
|
||||
|
||||
|
||||
@pytest.mark.parametrize(("sl", "local", "nm", "expected"), _HIGH_PLUS)
|
||||
def test_p3_high_plus_decision(ms, gate, sl, local, nm, expected):
|
||||
"""P3 (high+ arm): comfyui_switch_version gate decision. Allow iff
|
||||
(loopback AND sl in {weak,normal-}) OR (personal_cloud AND sl == weak)."""
|
||||
gate(sl, network_mode=nm, is_local_mode=local)
|
||||
assert ms.is_allowed_security_level("high+") is expected
|
||||
|
||||
|
||||
@pytest.mark.parametrize(("sl", "local", "nm", "expected"), _MIDDLE)
|
||||
def test_p3_middle_decision(ms, gate, sl, local, nm, expected):
|
||||
"""P3 (middle arm): snapshot remove / middle-gated legacy ops. Allow iff
|
||||
sl in {weak,normal,normal-} (locality-independent)."""
|
||||
gate(sl, network_mode=nm, is_local_mode=local)
|
||||
assert ms.is_allowed_security_level("middle") is expected
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
# Response-shape guards — invoke the REAL async handlers; assert the observable
|
||||
# status. Each pairs a DENY with an opposite case so the gate (not a constant)
|
||||
# is provably what decides.
|
||||
# ===========================================================================
|
||||
|
||||
def test_p1_install_custom_node_denies_at_strong_403(ms, gate):
|
||||
"""P1 deny: at sl=strong the middle+ entry gate (first statement of
|
||||
_install_custom_node) returns 403 before any further processing."""
|
||||
gate("strong", is_local_mode=True)
|
||||
resp = _run(ms._install_custom_node({}))
|
||||
assert resp.status == 403
|
||||
|
||||
|
||||
def test_p1_install_custom_node_passes_entry_gate_at_normal(ms, gate, monkeypatch):
|
||||
"""P1 allow-flip: at sl=normal the middle+ entry gate PASSES — with the
|
||||
downstream risky level forced to an allowed value the handler reaches 200,
|
||||
proving the 403 above came from the gate, not unconditionally."""
|
||||
gate("normal", is_local_mode=True)
|
||||
|
||||
async def _risky_middle_plus(files, pip):
|
||||
return "middle+"
|
||||
|
||||
monkeypatch.setattr(ms, "get_risky_level", _risky_middle_plus)
|
||||
json_data = {
|
||||
"version": "unknown",
|
||||
"files": ["https://github.com/example/unknown-node.git"],
|
||||
"pip": [],
|
||||
"id": "x",
|
||||
"ui_id": "u",
|
||||
"channel": "default",
|
||||
"mode": "cache",
|
||||
}
|
||||
resp = _run(ms._install_custom_node(dict(json_data)))
|
||||
assert resp.status == 200
|
||||
|
||||
|
||||
def test_p2_unknown_pip_block_denies_404_even_at_weak(ms, gate, monkeypatch):
|
||||
"""P2: with the risky level resolved to 'block' (unknown pip), the handler
|
||||
returns 404 even at sl=weak (the most permissive level) — proving 'block'
|
||||
is an unconditional deny, not a security_level-relative one."""
|
||||
gate("weak", is_local_mode=True)
|
||||
|
||||
async def _risky_block(files, pip):
|
||||
return "block"
|
||||
|
||||
monkeypatch.setattr(ms, "get_risky_level", _risky_block)
|
||||
json_data = {
|
||||
"version": "unknown",
|
||||
"files": ["https://github.com/example/unknown-node.git"],
|
||||
"pip": ["some-unknown-pkg"],
|
||||
"id": "x",
|
||||
"ui_id": "u",
|
||||
"channel": "default",
|
||||
"mode": "cache",
|
||||
}
|
||||
resp = _run(ms._install_custom_node(dict(json_data)))
|
||||
assert resp.status == 404
|
||||
|
||||
|
||||
def test_p3_comfyui_switch_version_denies_high_plus_403(ms, gate):
|
||||
"""P3 high+ handler: comfyui_switch_version denies with 403 at sl=normal
|
||||
(high+ requires normal- or lower on loopback)."""
|
||||
gate("normal", is_local_mode=True)
|
||||
resp = _run(ms.comfyui_switch_version(FakeRequest()))
|
||||
assert resp.status == 403
|
||||
|
||||
|
||||
def test_p3_restore_snapshot_denies_middle_plus_403(ms, gate):
|
||||
"""P3 middle+ handler: restore_snapshot denies with 403 at sl=strong
|
||||
(middle+ requires normal or lower)."""
|
||||
gate("strong", is_local_mode=True)
|
||||
resp = _run(ms.restore_snapshot(FakeRequest()))
|
||||
assert resp.status == 403
|
||||
|
||||
|
||||
def test_p4_install_model_non_safetensors_denies_high_plus_403(ms, gate, monkeypatch):
|
||||
"""P4 deny: a non-.safetensors model not on the whitelist requires high+;
|
||||
at sl=normal (middle+ passes, high+ fails) the handler returns 403."""
|
||||
gate("normal", is_local_mode=True)
|
||||
|
||||
async def _whitelist_ok(item):
|
||||
return True
|
||||
|
||||
async def _models_empty(*a, **k):
|
||||
return {"models": []}
|
||||
|
||||
monkeypatch.setattr(ms, "check_whitelist_for_model", _whitelist_ok)
|
||||
monkeypatch.setattr(ms.core, "get_data_by_mode", _models_empty)
|
||||
json_data = {"filename": "model.ckpt", "url": "http://example/model.ckpt", "ui_id": "u"}
|
||||
resp = _run(ms._install_model(dict(json_data)))
|
||||
assert resp.status == 403
|
||||
|
||||
|
||||
def test_p4_install_model_safetensors_skips_high_plus_gate_200(ms, gate, monkeypatch):
|
||||
"""P4 flip: a .safetensors model skips the non-.safetensors high+ sub-check
|
||||
entirely and reaches 200 at the SAME sl=normal — proving the 403 above is
|
||||
driven by the non-.safetensors + high+ compound, not the security level
|
||||
alone."""
|
||||
gate("normal", is_local_mode=True)
|
||||
|
||||
async def _whitelist_ok(item):
|
||||
return True
|
||||
|
||||
monkeypatch.setattr(ms, "check_whitelist_for_model", _whitelist_ok)
|
||||
json_data = {"filename": "model.safetensors", "url": "http://example/m.safetensors", "ui_id": "u"}
|
||||
resp = _run(ms._install_model(dict(json_data)))
|
||||
assert resp.status == 200
|
||||
Loading…
Reference in New Issue
Block a user