Commit Graph

22 Commits

Author SHA1 Message Date
Dr.Lt.Data
f042d73b72
feat(deps): add unified dependency resolver using uv pip compile (#2589)
Some checks are pending
Publish to PyPI / build-and-publish (push) Waiting to run
Python Linting / Run Ruff (push) Waiting to run
* feat(deps): add unified dependency resolver using uv pip compile

- Add UnifiedDepResolver module with 7 FRs: collect, compile, install pipeline
- Integrate startup batch resolution in prestartup_script.py (module scope)
- Skip per-node pip install in execute_install_script() when unified mode active
- Add use_unified_resolver config flag following use_uv pattern
- Input sanitization: reject -r, -e, --find-links, @ file://, path separators
- Handle --index-url/--extra-index-url separation with credential redaction
- Fallback to per-node pip on resolver failure or uv unavailability
- Add 98 unit tests across 20 test classes
- Add PRD and Design docs with cm_global integration marked as DEFERRED

* fix(deps): reset use_unified_resolver flag on startup fallback

When the unified resolver fails at startup (compile error, install
error, uv unavailable, or generic exception), the runtime flag was
not being reset to False. This caused subsequent runtime installs
to incorrectly defer pip dependencies instead of falling back to
per-node pip install.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test(deps): add manual test cases for unified dependency resolver

Add environment setup guide and 16 test cases covering:
- Normal batch resolution (TC-1), disabled state (TC-2)
- Fallback paths: uv unavailable (TC-3), compile fail (TC-4),
  install fail (TC-5), generic exception (TC-16)
- install.py preservation (TC-6), runtime defer (TC-13)
- Input sanitization: dangerous patterns (TC-7), path separators
  (TC-8), index-url separation (TC-9), credential redaction (TC-10)
- Disabled pack exclusion (TC-11), no-deps path (TC-12)
- Both unified resolver guard paths (TC-14), post-fallback (TC-15)

Includes API reference, traceability matrix, and out-of-scope items.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(deps): prevent read_config() from overriding resolver fallback state

read_config() in manager_core.py unconditionally re-read
use_unified_resolver from config.ini, undoing the False set by
prestartup_script.py on resolver fallback. This caused runtime
installs to still defer deps even after a startup batch failure.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(deps): support multiple index URLs per line and optimize downgrade check

- Rewrite _split_index_url() to handle multiple --index-url /
  --extra-index-url options on a single requirements.txt line using
  regex-based parsing instead of single split.
- Cache installed_packages snapshot in collect_requirements() to avoid
  repeated subprocess calls during downgrade blacklist checks.
- Add unit tests for multi-URL lines and bare --index-url edge case.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test(deps): add E2E scripts and update test documentation

Add automated E2E test scripts for unified dependency resolver:
- setup_e2e_env.sh: idempotent environment setup (clone ComfyUI,
  create venv, install deps, symlink Manager, write config.ini)
- start_comfyui.sh: foreground-blocking launcher using
  tail -f | grep -q readiness detection
- stop_comfyui.sh: graceful SIGTERM → SIGKILL shutdown

Update test documentation reflecting E2E testing findings:
- TEST-environment-setup.md: add automated script usage, document
  caveats (PYTHONPATH, config.ini path, Manager v4 /v2/ prefix,
  Blocked by policy, bash ((var++)) trap, git+https:// rejection)
- TEST-unified-dep-resolver.md: add TC-17 (restart dependency
  detection), TC-18 (real node pack integration), Validated
  Behaviors section, normalize API port to 8199

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(deps): harden input sanitization, expand test coverage, bump version

Security:
- Add _INLINE_DANGEROUS_OPTIONS regex to catch pip options after package
  names (--find-links, --constraint, --requirement, --editable, --trusted-host,
  --global-option, --install-option and short forms)
- Stage index URLs in pending_urls, commit only after full line validation
  to prevent URL injection from rejected lines

Tests:
- Add 50 new tests: inline sanitization, false-positive guards, parse
  helpers (_parse_conflicts, _parse_install_output), exception paths
  (91 → 141 total, all pass)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(cli): add uv-compile command and --uv-compile flag for batch dependency resolution

Add two CLI entry points for the unified dependency resolver:

- `cm_cli uv-compile`: standalone batch resolution of all installed
  node pack dependencies via uv pip compile
- `cm_cli install --uv-compile`: skip per-node pip, batch-resolve all
  deps after install completes (mutually exclusive with --no-deps)

Both use a shared `_run_unified_resolve()` helper that passes real
cm_global values (pip_blacklist, pip_overrides, pip_downgrade_blacklist)
and guarantees PIPFixer.fix_broken() runs via try/finally.

Update DESIGN, PRD, and TEST docs for consistency.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 06:51:53 +09:00
bymyself
2120a0aa79 [chore] Add dist/ to gitignore to exclude build artifacts 2025-06-13 19:40:27 -07:00
Christian Byrne
46a37907e6
add development guide (#1739) 2025-04-13 08:40:28 +09:00
Dr.Lt.Data
01ac9c895a Modify the structure to be installable via pip. 2025-03-19 22:15:53 +09:00
huchenlei
9d1ef85af8 Add NodePackage 2024-12-20 14:09:07 -08:00
Dr.Lt.Data
bff8dbee30 Merge branch 'main' into feat/cnr 2024-08-29 21:53:37 +09:00
Dr.Lt.Data
42f33a2dca update DB 2024-08-26 22:48:13 +09:00
Dr.Lt.Data
a6d20b0865 Merge branch 'main' into feat/cnr 2024-08-17 16:35:22 +09:00
Dr.Lt.Data
f896755a31 refactor: share features 2024-08-17 12:22:01 +09:00
Max Klein
f0299e07f9
added support for --no-deps option to node install and reinstall (#886)
* added support for `--no-deps` option to node install and reinstall

* post rebase fixup

* fixup help msg for --no-deps option
2024-07-27 01:52:07 +09:00
Dr.Lt.Data
c0c83ad977 update .gitignore 2024-04-12 22:30:09 +09:00
Liu Sida
abae9638ac
Add GitHub stats for custom nodes (#533)
* Add GitHub stats fetching feature
- Added PyGithub package to requirements.txt for GitHub API interaction
- Updated .gitignore to ignore github-stats-cache.json
- Produced github-stats.json for storing GitHub stats
- Modified scanner.py to include the GitHub stats fetching process

* Add sorting for 'GitHub Stars' and 'Last Update' columns

- Fetch 'GitHub Stars' and 'Last Update' data when getting the custom node list.
- Display 'GitHub Stars' and 'Last Update' information in the UI.
- Implement sorting functionality for these two columns, allowing users to sort both in descending and ascending order.

* fix: scanner - prevent stuck when exceed rate limit

---------

Co-authored-by: Dr.Lt.Data <dr.lt.data@gmail.com>
2024-04-02 20:56:09 +09:00
Dr.Lt.Data
8aada09e8f update DB
update .gitignore
2024-01-10 00:56:41 +09:00
Dr.Lt.Data
35dac5db65 .gitignore 2023-12-29 23:07:33 +09:00
dr.lt.data
61fd22415f improve: use channel.list instead of config.ini's channel_url.list
add: tutorial channel
2023-12-18 11:11:22 +09:00
Dr.Lt.Data
fa20899fa1 optimize: better badge
optimize: cache data mode
improve: robust data retrieve

update DB
2023-12-09 00:21:06 +09:00
Dr.Lt.Data
2f1ab1a327 update DB 2023-11-27 01:36:43 +09:00
Dr.Lt.Data
7fbe34f8db feat: snapshot 2023-11-04 17:17:55 +09:00
Dr.Lt.Data
91f8d07e3d modify .gitignore 2023-07-29 11:34:24 +09:00
Dr.Lt.Data
007093caa4 improve installer 2023-07-16 11:07:47 +09:00
Dr.Lt.Data
9894babccc add node scanner 2023-05-22 18:39:23 +09:00
Dr.Lt.Data
a0d143d1d0 prototype 2023-05-03 11:35:14 +09:00