Commit Graph

77 Commits

Author SHA1 Message Date
Dr.Lt.Data
f042d73b72
feat(deps): add unified dependency resolver using uv pip compile (#2589)
Some checks failed
Python Linting / Run Ruff (push) Has been cancelled
Publish to PyPI / build-and-publish (push) Has been cancelled
* feat(deps): add unified dependency resolver using uv pip compile

- Add UnifiedDepResolver module with 7 FRs: collect, compile, install pipeline
- Integrate startup batch resolution in prestartup_script.py (module scope)
- Skip per-node pip install in execute_install_script() when unified mode active
- Add use_unified_resolver config flag following use_uv pattern
- Input sanitization: reject -r, -e, --find-links, @ file://, path separators
- Handle --index-url/--extra-index-url separation with credential redaction
- Fallback to per-node pip on resolver failure or uv unavailability
- Add 98 unit tests across 20 test classes
- Add PRD and Design docs with cm_global integration marked as DEFERRED

* fix(deps): reset use_unified_resolver flag on startup fallback

When the unified resolver fails at startup (compile error, install
error, uv unavailable, or generic exception), the runtime flag was
not being reset to False. This caused subsequent runtime installs
to incorrectly defer pip dependencies instead of falling back to
per-node pip install.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test(deps): add manual test cases for unified dependency resolver

Add environment setup guide and 16 test cases covering:
- Normal batch resolution (TC-1), disabled state (TC-2)
- Fallback paths: uv unavailable (TC-3), compile fail (TC-4),
  install fail (TC-5), generic exception (TC-16)
- install.py preservation (TC-6), runtime defer (TC-13)
- Input sanitization: dangerous patterns (TC-7), path separators
  (TC-8), index-url separation (TC-9), credential redaction (TC-10)
- Disabled pack exclusion (TC-11), no-deps path (TC-12)
- Both unified resolver guard paths (TC-14), post-fallback (TC-15)

Includes API reference, traceability matrix, and out-of-scope items.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(deps): prevent read_config() from overriding resolver fallback state

read_config() in manager_core.py unconditionally re-read
use_unified_resolver from config.ini, undoing the False set by
prestartup_script.py on resolver fallback. This caused runtime
installs to still defer deps even after a startup batch failure.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(deps): support multiple index URLs per line and optimize downgrade check

- Rewrite _split_index_url() to handle multiple --index-url /
  --extra-index-url options on a single requirements.txt line using
  regex-based parsing instead of single split.
- Cache installed_packages snapshot in collect_requirements() to avoid
  repeated subprocess calls during downgrade blacklist checks.
- Add unit tests for multi-URL lines and bare --index-url edge case.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test(deps): add E2E scripts and update test documentation

Add automated E2E test scripts for unified dependency resolver:
- setup_e2e_env.sh: idempotent environment setup (clone ComfyUI,
  create venv, install deps, symlink Manager, write config.ini)
- start_comfyui.sh: foreground-blocking launcher using
  tail -f | grep -q readiness detection
- stop_comfyui.sh: graceful SIGTERM → SIGKILL shutdown

Update test documentation reflecting E2E testing findings:
- TEST-environment-setup.md: add automated script usage, document
  caveats (PYTHONPATH, config.ini path, Manager v4 /v2/ prefix,
  Blocked by policy, bash ((var++)) trap, git+https:// rejection)
- TEST-unified-dep-resolver.md: add TC-17 (restart dependency
  detection), TC-18 (real node pack integration), Validated
  Behaviors section, normalize API port to 8199

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(deps): harden input sanitization, expand test coverage, bump version

Security:
- Add _INLINE_DANGEROUS_OPTIONS regex to catch pip options after package
  names (--find-links, --constraint, --requirement, --editable, --trusted-host,
  --global-option, --install-option and short forms)
- Stage index URLs in pending_urls, commit only after full line validation
  to prevent URL injection from rejected lines

Tests:
- Add 50 new tests: inline sanitization, false-positive guards, parse
  helpers (_parse_conflicts, _parse_install_output), exception paths
  (91 → 141 total, all pass)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(cli): add uv-compile command and --uv-compile flag for batch dependency resolution

Add two CLI entry points for the unified dependency resolver:

- `cm_cli uv-compile`: standalone batch resolution of all installed
  node pack dependencies via uv pip compile
- `cm_cli install --uv-compile`: skip per-node pip, batch-resolve all
  deps after install completes (mutually exclusive with --no-deps)

Both use a shared `_run_unified_resolve()` helper that passes real
cm_global values (pip_blacklist, pip_overrides, pip_downgrade_blacklist)
and guarantees PIPFixer.fix_broken() runs via try/finally.

Update DESIGN, PRD, and TEST docs for consistency.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 06:51:53 +09:00
Dr.Lt.Data
0d88a3874d
refactor(cli): move cm_cli to top-level package and remove dead cli-only-mode (#2548)
Some checks failed
Python Linting / Run Ruff (push) Has been cancelled
Publish to PyPI / build-and-publish (push) Has been cancelled
- Move cm_cli from comfyui_manager/cm_cli/ to top-level cm_cli/ package
- Convert relative imports to absolute imports
- Remove non-functional cli-only-mode command (flag was never checked)
- Update docs: python cm-cli.py → cm-cli entrypoint
- Update prestartup snapshot restore to use -m cm_cli
- Version bump to 4.1b1

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 19:42:35 +09:00
Dr.Lt.Data
ef8703a3d7 security(api): add path traversal and CRLF injection protection
Some checks failed
Publish to PyPI / build-and-publish (push) Has been cancelled
Python Linting / Run Ruff (push) Has been cancelled
- Add is_safe_path_target() and get_safe_file_path() utilities
- Validate history id and snapshot target parameters in API endpoints
- Sanitize config string values to prevent CRLF injection
2026-01-08 18:35:03 +09:00
Dr.Lt.Data
f85a12f2a2 bump version to 4.0.4
Some checks failed
Publish to PyPI / build-and-publish (push) Has been cancelled
Python Linting / Run Ruff (push) Has been cancelled
2025-12-27 04:56:48 +09:00
Dr.Lt.Data
3f0fc85b95 refactor(core): add verbose config, improve module lookup, fix is_valid_url
- Add verbose config option to control CNR fetch logging
- Improve get_module_name with cnr_id/aux_id fallback via repo_cnr_map
- Fix is_valid_url misuse of try/finally that could cause runtime errors
- Move SSH_URL_PATTERN to module-level constant for performance
2025-12-27 03:57:19 +09:00
Dr.Lt.Data
b9def4cb6e refactor: remove preview_method and component legacy features
Some checks failed
Publish to PyPI / build-and-publish (push) Has been cancelled
Python Linting / Run Ruff (push) Has been cancelled
Preview Method Removal:
- Remove preview method UI from Manager settings panel
- Remove /v2/manager/preview_method API endpoint (legacy)
- Remove set_preview_method() and get_current_preview_method() functions
- Remove preview_method from config read/write operations
- Clean up latent_preview imports

Use ComfyUI Settings > Execution > Live preview method instead.

Component Feature Removal:
- Delete components-manager.js entirely
- Remove ComponentBuilderDialog, load_components, set_component_policy
- Remove component policy UI from Manager settings panel
- Remove /v2/manager/policy/component API endpoint
- Remove /v2/manager/component/save and /loads API endpoints
- Remove component_policy from config read/write operations
- Remove manager_components_path from context
2025-12-19 22:39:59 +09:00
Dr.Lt.Data
2779c66b39 feat(version): apply semver-based version sorting to glob and add master fallback
Some checks failed
Publish to PyPI / build-and-publish (push) Has been cancelled
Python Linting / Run Ruff (push) Has been cancelled
- Apply PR #2334 changes to glob/manager_core.py (was only in legacy)
- Add master branch fallback when remote/HEAD reference is unavailable
2025-12-15 03:39:13 +09:00
Dr.Lt.Data
952613c07b fix(api): improve import_fail_info_bulk lookup for cnr_id and aux_id
- Add aux_id format (author/repo) support in normalize_to_github_id()
- Fix get_module_name() to use URL normalization for unknown_active_nodes
- Use NormalizedKeyDict in reload() to maintain normalized key lookup
2025-12-15 02:54:30 +09:00
Dr.Lt.Data
8e8b6ca724 fix(git): handle divergent branches safely + datetime fallback
- Use --ff-only flag to detect non-fast-forward situations
- Create backup branch before resetting divergent local branch
- Reset to remote branch when fast-forward is not possible
- Add timestamp_utils.py for Mac datetime module compatibility
- Migrate all datetime usages to centralized utilities
- Bump version to 4.0.3b5

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-12 22:45:05 +09:00
Dr.Lt.Data
85ebcd9897 In response to the patch that separates manager_requirements.txt from requirements.txt, this update additionally refreshes manager_requirements.txt when it is present.
Some checks failed
Publish to PyPI / build-and-publish (push) Has been cancelled
Python Linting / Run Ruff (push) Has been cancelled
79fb96488a
2025-11-26 22:35:03 +09:00
Dr.Lt.Data
d3906e3cbc bump version 2025-10-21 07:25:56 +09:00
Dr.Lt.Data
079ac254ce
fixed: Bug fix in glob/manager_server.py that prevented cache updates when installed via pip. (#2237)
Until the cacheless implementation is fully applied, the cache must always be updated — otherwise, various parts of the system will malfunction.
2025-10-21 07:16:57 +09:00
Dr.Lt.Data
1ab2b1aeb3 modified: Reflection of changing --disable-manager to --enable-manager 2025-09-19 11:58:04 +09:00
Dr.Lt.Data
d7799964de fixed: Issue where an invalid channel exception occurred when using the default channel
- Mismatch issue between ltdrdata/ and Comfy-Org/
modified: /v2/customnode/installed – cnr_id was being returned in a normalized form
modified: /v2/customnode/installed – when both an enabled nodepack and a disabled nodepack existed, modified to report only the enabled nodepack
fixed: Removed unnecessary warning messages printed during nodepack installation
2025-09-06 03:35:43 +09:00
Jin Yi
e13bf68775
Fix JSON serialization error in bulk import fail info API (#2119)
* fix: import failed info bulk api bug fix

* fix: Remove unused ImportFailInfoBulkResponse import
2025-09-04 08:36:46 +09:00
Dr.Lt.Data
bdfb70a58a bump version 2025-08-24 15:58:23 +09:00
Dr.Lt.Data
00fb9c88e1 modified: remove matrix-nio dependency from the requirements.txt
modified: The matrix share feature is now only available when the `matrix-nio` dependency is installed.

If `matrix-nio` is not installed:
1. Apply a strikethrough to the matrix checkbox text in the share UI and display a tooltip.
2. A warning is logged at startup indicating that `matrix-nio` is missing, along with the installation command.

fixed: Corrected an issue where PR #2025 was merged into draft-v4 but applied only to `legacy/..` and not to `glob/..`
2025-08-22 20:46:32 +09:00
Jin Yi
300c6e7406 feat: Add Pydantic validation to import_fail_info_bulk endpoint
- Regenerated Pydantic models from updated OpenAPI specification
- Updated import_fail_info_bulk route handler to use ImportFailInfoBulkRequest/Response models
- Replaced manual JSON validation with Pydantic model validation
- Added proper error handling with ValidationError
- Updated data_models/__init__.py to export new models

Following the process outlined in data_models/README.md for type safety and consistency.
2025-07-31 14:15:21 +09:00
Jin Yi
60746c6253
[feat] Add bulk import failure info API endpoint (#2035)
* [feat] Add bulk import failure info API endpoint

- Add import_fail_info_bulk endpoint to both glob and legacy manager servers
- Supports bulk processing of cnr_ids and urls arrays in single request
- Maintains same error handling pattern as original import_fail_info API
- Reduces API calls from N to 1 for conflict detection optimization
- Validates input parameters and provides proper error responses

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* modified: remove manager button completely. Now, even when using the legacy UI, it must always be accessed through the menu.

* chore(api): Add temporary cache reload for import_fail_info_bulk

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Dr.Lt.Data <dr.lt.data@gmail.com>
2025-07-30 07:57:19 +09:00
Dr.Lt.Data
cf8029ecd4 Merge branch 'main' into draft-v4 2025-07-24 12:41:48 +09:00
Dr.Lt.Data
59264c1fd9 Merge branch 'main' into draft-v4 2025-07-20 19:23:24 +09:00
Dr.Lt.Data
a0b0c2b963 feat: initial implementation of middleware-based security policy 2025-07-12 11:31:07 +09:00
Dr.Lt.Data
f4ce0fd5f1 Merge branch 'main' into draft-v4 2025-07-08 12:21:47 +09:00
Dr.Lt.Data
43041cebed modified: Do not modify generated_models.py directly; use openapi.yaml instead. 2025-06-28 07:54:17 +09:00
Dr.Lt.Data
002e549a86 modified: security policy
- Strengthened the default security policy
- Subdivided the risky levels high and middle into high+, high, middle+, and middle
- Added support for personal_cloud network mode
- Updated README.md

fixed: invalid security message
fixed: legacy - crash when security policy violation occurred

modified: default 'use_uv' is now True
2025-06-27 01:38:38 +09:00
Dr.Lt.Data
223d6dad51 Merge branch 'main' into draft-v4 2025-06-25 00:46:12 +09:00
Christian Byrne
cb0fa5829d
Merge pull request #1915 from Comfy-Org/feat/implement-batch-tracking-clean
[feat] Implement comprehensive batch tracking and OpenAPI-driven data models
2025-06-21 19:46:23 -07:00
bymyself
a66f86d4af cleanup records older than 16 days 2025-06-21 16:57:54 -07:00
bymyself
35d98dcea8 add batch_id to history task items 2025-06-21 16:45:50 -07:00
bymyself
38fefde06d add embedded python to system state 2025-06-21 16:29:40 -07:00
bymyself
75ecb31f8c add frontend version to system state capture 2025-06-21 16:28:00 -07:00
bymyself
77133375ad [fix] Ensure batch history is written when queue becomes empty 2025-06-21 16:01:25 -07:00
bymyself
d1ca6288a3 apply formatting 2025-06-19 16:41:16 -07:00
bymyself
624ad4cfe6 remove debug comments 2025-06-19 16:39:14 -07:00
bymyself
d912fb0f8b [fix] Remove unused imports to fix Ruff linting errors 2025-06-17 15:27:21 -07:00
bymyself
ce3b2bab39 refactor 2025-06-17 14:58:34 -07:00
bymyself
15e3699535 [cleanup] Remove outdated temp_queue_batch comment 2025-06-17 14:44:58 -07:00
bymyself
a4bf6bddbf [refactor] Use Pydantic models for query parameter validation
- Added query parameter models to OpenAPI spec for GET endpoints
- Regenerated data models to include new query param models
- Replaced manual validation with Pydantic model validation
- Removed obsolete validate_required_params helper function
- Provides better error messages and type safety for API endpoints

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-17 14:42:25 -07:00
bymyself
f1b3c6b735 [refactor] Move model utility functions to model_utils module 2025-06-17 14:24:31 -07:00
bymyself
e923434d08 [fix] Update client filtering to handle tuple structure in pending_tasks 2025-06-17 13:52:00 -07:00
bymyself
ddc9cd0fd5 [fix] Use tuples in TaskQueue heap for proper comparison support 2025-06-17 13:42:47 -07:00
bymyself
d081db0c30 [cleanup] Remove dead code do_update_all function
- Removed do_update_all function that was never called and only returned an error
- Removed "update-all" from OperationType enum as it's no longer used
- Regenerated data models to reflect the enum change

The update_all functionality now properly creates individual update tasks through the API endpoint rather than being a single monolithic task.

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-17 13:27:51 -07:00
bymyself
14298b0859 [fix] Remove unused imports to fix linting errors 2025-06-17 13:08:52 -07:00
bymyself
03ecda3cfe [feat] Implement comprehensive system state capture for batch records 2025-06-17 13:08:35 -07:00
bymyself
1bdb026079 explain glob vs legacy in claude memory 2025-06-17 10:36:08 -07:00
Dr.Lt.Data
7e51286313 Merge branch 'main' into draft-v4 2025-06-17 00:33:31 +09:00
bymyself
c888ea6435 [fix] Reduce excessive logging output to debug level
- Convert batch tracking messages to debug level (batch start, history saved)
- Convert task processing details to debug level
- Convert cache update messages to debug level
- Replace print() with logging.debug() for task processing
- Keep user-relevant messages at info level (ComfyUI updates, installation success)
- Resolves verbose output appearing without --verbose flag
2025-06-13 20:39:18 -07:00
bymyself
b089db79c5 [fix] Restore proper thread-based TaskQueue worker management
- Fix async/sync mismatch in TaskQueue worker implementation
- Use threading.Thread with asyncio.run() as originally designed
- Remove incorrect async task approach that caused blocking issues
- TaskQueue now properly manages its own thread lifecycle
- Resolves WebSocket message delivery and task processing issues
2025-06-13 20:27:41 -07:00
bymyself
706b6d8317 [refactor] Remove legacy thread management for TaskQueue
- Add proper async worker management to TaskQueue class
- Remove redundant task_worker_thread and task_worker_lock global variables
- Replace manual threading with async task management
- Update is_processing() logic to use TaskQueue state instead of thread status
- Implement automatic worker cleanup when queue processing completes
- Simplify queue start endpoint to use TaskQueue.start_worker()
2025-06-13 19:40:27 -07:00
bymyself
a59e6e176e [refactor] Remove redundant ExecutionStatus NamedTuple
- Eliminate TaskQueue.ExecutionStatus NamedTuple in favor of generated TaskExecutionStatus Pydantic model
- Remove manual conversion logic between NamedTuple and Pydantic model
- Use single source of truth for task execution status
- Clean up unused imports (Literal, NamedTuple)
- Maintain consistent data model usage throughout TaskQueue
2025-06-13 19:37:57 -07:00