mirror of
https://github.com/Comfy-Org/ComfyUI-Manager.git
synced 2026-01-24 04:50:15 +08:00
Compare commits
4 Commits
f48b5e39b2
...
e47d79e979
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e47d79e979 | ||
|
|
b9def4cb6e | ||
|
|
a7eb93fff0 | ||
|
|
2866193baf |
@ -11,4 +11,5 @@ include extras.json
|
||||
include github-stats.json
|
||||
include model-list.json
|
||||
include alter-list.json
|
||||
include comfyui_manager/channels.list.template
|
||||
include comfyui_manager/channels.list.template
|
||||
include comfyui_manager/pip-policy.json
|
||||
@ -31,7 +31,6 @@ manager_startup_script_path:str = None
|
||||
manager_snapshot_path = None
|
||||
manager_pip_overrides_path = None
|
||||
manager_pip_blacklist_path = None
|
||||
manager_components_path = None
|
||||
manager_batch_history_path = None
|
||||
|
||||
def update_user_directory(manager_dir):
|
||||
@ -42,7 +41,6 @@ def update_user_directory(manager_dir):
|
||||
global manager_snapshot_path
|
||||
global manager_pip_overrides_path
|
||||
global manager_pip_blacklist_path
|
||||
global manager_components_path
|
||||
global manager_batch_history_path
|
||||
|
||||
manager_files_path = manager_dir
|
||||
@ -61,7 +59,6 @@ def update_user_directory(manager_dir):
|
||||
manager_channel_list_path = os.path.join(manager_files_path, 'channels.list')
|
||||
manager_pip_overrides_path = os.path.join(manager_files_path, "pip_overrides.json")
|
||||
manager_pip_blacklist_path = os.path.join(manager_files_path, "pip_blacklist.list")
|
||||
manager_components_path = os.path.join(manager_files_path, "components")
|
||||
manager_util.cache_dir = os.path.join(manager_files_path, "cache")
|
||||
manager_batch_history_path = os.path.join(manager_files_path, "batch_history")
|
||||
|
||||
|
||||
713
comfyui_manager/common/pip_util.design.en.md
Normal file
713
comfyui_manager/common/pip_util.design.en.md
Normal file
@ -0,0 +1,713 @@
|
||||
# Design Document for pip_util.py Implementation
|
||||
|
||||
This is designed to minimize breaking existing installed dependencies.
|
||||
|
||||
## List of Functions to Implement
|
||||
|
||||
## Global Policy Management
|
||||
|
||||
### Global Variables
|
||||
```python
|
||||
_pip_policy_cache = None # Policy cache (program-wide, loaded once)
|
||||
```
|
||||
|
||||
### Global Functions
|
||||
|
||||
* get_pip_policy(): Returns policy for resolving pip dependency conflicts (lazy loading)
|
||||
- **Call timing**: Called whenever needed (automatically loads only once on first call)
|
||||
- **Purpose**: Returns policy cache, automatically loads if cache is empty
|
||||
- **Execution flow**:
|
||||
1. Declare global _pip_policy_cache
|
||||
2. If _pip_policy_cache is already loaded, return immediately (prevent duplicate loading)
|
||||
3. Read base policy file:
|
||||
- Path: {manager_util.comfyui_manager_path}/pip-policy.json
|
||||
- Use empty dictionary if file doesn't exist
|
||||
- Log error and use empty dictionary if JSON parsing fails
|
||||
4. Read user policy file:
|
||||
- Path: {context.manager_files_path}/pip-policy.user.json
|
||||
- Create empty JSON file if doesn't exist ({"_comment": "User-specific pip policy overrides"})
|
||||
- Log warning and use empty dictionary if JSON parsing fails
|
||||
5. Apply merge rules (merge by package name):
|
||||
- Start with base policy as base
|
||||
- For each package in user policy:
|
||||
* Package only in user policy: add to base
|
||||
* Package only in base policy: keep in base
|
||||
* Package in both: completely replace with user policy (entire package replacement, not section-level)
|
||||
6. Store merged policy in _pip_policy_cache
|
||||
7. Log policy load success (include number of loaded package policies)
|
||||
8. Return _pip_policy_cache
|
||||
- **Return value**: Dict (merged policy dictionary)
|
||||
- **Exception handling**:
|
||||
- File read failure: Log warning and treat file as empty dictionary
|
||||
- JSON parsing failure: Log error and treat file as empty dictionary
|
||||
- **Notes**:
|
||||
- Lazy loading pattern automatically loads on first call
|
||||
- Not thread-safe, caution needed in multi-threaded environments
|
||||
|
||||
- Policy file structure should support the following scenarios:
|
||||
- Dictionary structure of {dependency name -> policy object}
|
||||
- Policy object has four policy sections:
|
||||
- **uninstall**: Package removal policy (pre-processing, condition optional)
|
||||
- **apply_first_match**: Evaluate top-to-bottom and execute only the first policy that satisfies condition (exclusive)
|
||||
- **apply_all_matches**: Execute all policies that satisfy conditions (cumulative)
|
||||
- **restore**: Package restoration policy (post-processing, condition optional)
|
||||
|
||||
- Condition types:
|
||||
- installed: Check version condition of already installed dependencies
|
||||
- spec is optional
|
||||
- package field: Specify package to check (optional, defaults to self)
|
||||
- Explicit: Reference another package (e.g., numba checks numpy version)
|
||||
- Omitted: Check own version (e.g., critical-package checks its own version)
|
||||
- platform: Platform conditions (os, has_gpu, comfyui_version, etc.)
|
||||
- If condition is absent, always considered satisfied
|
||||
|
||||
- uninstall policy (pre-removal policy):
|
||||
- Removal policy list (condition is optional, evaluate top-to-bottom and execute only first match)
|
||||
- When condition satisfied (or always if no condition): remove target package and abort installation
|
||||
- If this policy is applied, all subsequent steps are ignored
|
||||
- target field specifies package to remove
|
||||
- Example: Unconditionally remove if specific package is installed
|
||||
|
||||
- Actions available in apply_first_match (determine installation method, exclusive):
|
||||
- skip: Block installation of specific dependency
|
||||
- force_version: Force change to specific version during installation
|
||||
- extra_index_url field can specify custom package repository (optional)
|
||||
- replace: Replace with different dependency
|
||||
- extra_index_url field can specify custom package repository (optional)
|
||||
|
||||
- Actions available in apply_all_matches (installation options, cumulative):
|
||||
- pin_dependencies: Pin currently installed versions of other dependencies
|
||||
- pinned_packages field specifies package list
|
||||
- Example: `pip install requests urllib3==1.26.15 certifi==2023.7.22 charset-normalizer==3.2.0`
|
||||
- Real use case: Prevent urllib3 from upgrading to 2.x when installing requests
|
||||
- on_failure: "fail" or "retry_without_pin"
|
||||
- install_with: Specify additional dependencies to install together
|
||||
- warn: Record warning message in log
|
||||
|
||||
- restore policy (post-restoration policy):
|
||||
- Restoration policy list (condition is optional, evaluate top-to-bottom and execute only first match)
|
||||
- Executed after package installation completes (post-processing)
|
||||
- When condition satisfied (or always if no condition): force install target package to specific version
|
||||
- target field specifies package to restore (can be different package)
|
||||
- version field specifies version to install
|
||||
- extra_index_url field can specify custom package repository (optional)
|
||||
- Example: Reinstall/change version if specific package is deleted or wrong version
|
||||
|
||||
- Execution order:
|
||||
1. uninstall evaluation: If condition satisfied, remove package and **terminate** (ignore subsequent steps)
|
||||
2. apply_first_match evaluation:
|
||||
- Execute first policy that satisfies condition among skip/force_version/replace
|
||||
- If no matching policy, proceed with default installation of originally requested package
|
||||
3. apply_all_matches evaluation: Apply all pin_dependencies, install_with, warn that satisfy conditions
|
||||
4. Execute actual package installation (pip install or uv pip install)
|
||||
5. restore evaluation: If condition satisfied, restore target package (post-processing)
|
||||
|
||||
## Batch Unit Class (PipBatch)
|
||||
|
||||
### Class Structure
|
||||
```python
|
||||
class PipBatch:
|
||||
"""
|
||||
pip package installation batch unit manager
|
||||
Maintains pip freeze cache during batch operations for performance optimization
|
||||
|
||||
Usage pattern:
|
||||
# Batch operations (policy auto-loaded)
|
||||
with PipBatch() as batch:
|
||||
batch.ensure_not_installed()
|
||||
batch.install("numpy>=1.20")
|
||||
batch.install("pandas>=2.0")
|
||||
batch.install("scipy>=1.7")
|
||||
batch.ensure_installed()
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
self._installed_cache = None # Installed packages cache (batch-level)
|
||||
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, exc_type, exc_val, exc_tb):
|
||||
self._installed_cache = None
|
||||
```
|
||||
|
||||
### Private Methods
|
||||
|
||||
* PipBatch._refresh_installed_cache():
|
||||
- **Purpose**: Read currently installed package information and refresh cache
|
||||
- **Execution flow**:
|
||||
1. Generate command using manager_util.make_pip_cmd(["freeze"])
|
||||
2. Execute pip freeze via subprocess
|
||||
3. Parse output:
|
||||
- Each line is in "package_name==version" format
|
||||
- Parse "package_name==version" to create dictionary
|
||||
- Ignore editable packages (starting with -e)
|
||||
- Ignore comments (starting with #)
|
||||
4. Store parsed dictionary in self._installed_cache
|
||||
- **Return value**: None
|
||||
- **Exception handling**:
|
||||
- pip freeze failure: Set cache to empty dictionary and log warning
|
||||
- Parse failure: Ignore line and continue
|
||||
|
||||
* PipBatch._get_installed_packages():
|
||||
- **Purpose**: Return cached installed package information (refresh if cache is None)
|
||||
- **Execution flow**:
|
||||
1. If self._installed_cache is None, call _refresh_installed_cache()
|
||||
2. Return self._installed_cache
|
||||
- **Return value**: {package_name: version} dictionary
|
||||
|
||||
* PipBatch._invalidate_cache():
|
||||
- **Purpose**: Invalidate cache after package install/uninstall
|
||||
- **Execution flow**:
|
||||
1. Set self._installed_cache = None
|
||||
- **Return value**: None
|
||||
- **Call timing**: After install(), ensure_not_installed(), ensure_installed()
|
||||
|
||||
* PipBatch._parse_package_spec(package_info):
|
||||
- **Purpose**: Split package spec string into package name and version spec
|
||||
- **Parameters**:
|
||||
- package_info: "numpy", "numpy==1.26.0", "numpy>=1.20.0", "numpy~=1.20", etc.
|
||||
- **Execution flow**:
|
||||
1. Use regex to split package name and version spec
|
||||
2. Pattern: `^([a-zA-Z0-9_-]+)([><=!~]+.*)?$`
|
||||
- **Return value**: (package_name, version_spec) tuple
|
||||
- Examples: ("numpy", "==1.26.0"), ("pandas", ">=2.0.0"), ("scipy", None)
|
||||
- **Exception handling**:
|
||||
- Parse failure: Raise ValueError
|
||||
|
||||
* PipBatch._evaluate_condition(condition, package_name, installed_packages):
|
||||
- **Purpose**: Evaluate policy condition and return whether satisfied
|
||||
- **Parameters**:
|
||||
- condition: Policy condition object (dictionary)
|
||||
- package_name: Name of package currently being processed
|
||||
- installed_packages: {package_name: version} dictionary
|
||||
- **Execution flow**:
|
||||
1. If condition is None, return True (always satisfied)
|
||||
2. Branch based on condition["type"]:
|
||||
a. "installed" type:
|
||||
- target_package = condition.get("package", package_name)
|
||||
- Check current version with installed_packages.get(target_package)
|
||||
- If not installed (None), return False
|
||||
- If spec exists, compare version using packaging.specifiers.SpecifierSet
|
||||
- If no spec, only check installation status (True)
|
||||
b. "platform" type:
|
||||
- If condition["os"] exists, compare with platform.system()
|
||||
- If condition["has_gpu"] exists, check GPU presence (torch.cuda.is_available(), etc.)
|
||||
- If condition["comfyui_version"] exists, compare ComfyUI version
|
||||
- Return True if all conditions satisfied
|
||||
3. Return True if all conditions satisfied, False if any unsatisfied
|
||||
- **Return value**: bool
|
||||
- **Exception handling**:
|
||||
- Version comparison failure: Log warning and return False
|
||||
- Unknown condition type: Log warning and return False
|
||||
|
||||
|
||||
### Public Methods
|
||||
|
||||
* PipBatch.install(package_info, extra_index_url=None, override_policy=False):
|
||||
- **Purpose**: Perform policy-based pip package installation (individual package basis)
|
||||
- **Parameters**:
|
||||
- package_info: Package name and version spec (e.g., "numpy", "numpy==1.26.0", "numpy>=1.20.0")
|
||||
- extra_index_url: Additional package repository URL (optional)
|
||||
- override_policy: If True, skip policy application and install directly (default: False)
|
||||
- **Execution flow**:
|
||||
1. Call get_pip_policy() to get policy (lazy loading)
|
||||
2. Use self._parse_package_spec() to split package_info into package name and version spec
|
||||
3. Call self._get_installed_packages() to get cached installed package information
|
||||
4. If override_policy=True → Jump directly to step 10 (skip policy)
|
||||
5. Get policy for package name from policy dictionary
|
||||
6. If no policy → Jump to step 10 (default installation)
|
||||
7. **apply_first_match policy evaluation** (exclusive - only first match):
|
||||
- Iterate through policy list top-to-bottom
|
||||
- Evaluate each policy's condition with self._evaluate_condition()
|
||||
- When first condition-satisfying policy found:
|
||||
* type="skip": Log reason and return False (don't install)
|
||||
* type="force_version": Change package_info version to policy's version
|
||||
* type="replace": Completely replace package_info with policy's replacement package
|
||||
- If no matching policy, keep original package_info
|
||||
8. **apply_all_matches policy evaluation** (cumulative - all matches):
|
||||
- Iterate through policy list top-to-bottom
|
||||
- Evaluate each policy's condition with self._evaluate_condition()
|
||||
- For all condition-satisfying policies:
|
||||
* type="pin_dependencies":
|
||||
- For each package in pinned_packages, query current version with self._installed_cache.get(pkg)
|
||||
- Pin to installed version in "package==version" format
|
||||
- Add to installation package list
|
||||
* type="install_with":
|
||||
- Add additional_packages to installation package list
|
||||
* type="warn":
|
||||
- Output message as warning log
|
||||
- If allow_continue=false, wait for user confirmation (optional)
|
||||
9. Compose final installation package list:
|
||||
- Main package (modified/replaced package_info)
|
||||
- Packages pinned by pin_dependencies
|
||||
- Packages added by install_with
|
||||
10. Handle extra_index_url:
|
||||
- Parameter-passed extra_index_url takes priority
|
||||
- Otherwise use extra_index_url defined in policy
|
||||
11. Generate pip/uv command using manager_util.make_pip_cmd():
|
||||
- Basic format: ["pip", "install"] + package list
|
||||
- If extra_index_url exists: add ["--extra-index-url", url]
|
||||
12. Execute command via subprocess
|
||||
13. Handle installation failure:
|
||||
- If pin_dependencies's on_failure="retry_without_pin":
|
||||
* Retry with only main package excluding pinned packages
|
||||
- If on_failure="fail":
|
||||
* Raise exception and abort installation
|
||||
- Otherwise: Log warning and continue
|
||||
14. On successful installation:
|
||||
- Call self._invalidate_cache() (invalidate cache)
|
||||
- Log info if reason exists
|
||||
- Return True
|
||||
- **Return value**: Installation success status (bool)
|
||||
- **Exception handling**:
|
||||
- Policy parsing failure: Log warning and proceed with default installation
|
||||
- Installation failure: Log error and raise exception (depends on on_failure setting)
|
||||
- **Notes**:
|
||||
- restore policy not handled in this method (batch-processed in ensure_installed())
|
||||
- uninstall policy not handled in this method (batch-processed in ensure_not_installed())
|
||||
|
||||
* PipBatch.ensure_not_installed():
|
||||
- **Purpose**: Iterate through all policies and remove all packages satisfying uninstall conditions (batch processing)
|
||||
- **Parameters**: None
|
||||
- **Execution flow**:
|
||||
1. Call get_pip_policy() to get policy (lazy loading)
|
||||
2. Call self._get_installed_packages() to get cached installed package information
|
||||
3. Iterate through all package policies in policy dictionary:
|
||||
a. Check if each package has uninstall policy
|
||||
b. If uninstall policy exists:
|
||||
- Iterate through uninstall policy list top-to-bottom
|
||||
- Evaluate each policy's condition with self._evaluate_condition()
|
||||
- When first condition-satisfying policy found:
|
||||
* Check if target package exists in self._installed_cache
|
||||
* If installed:
|
||||
- Generate command with manager_util.make_pip_cmd(["uninstall", "-y", target])
|
||||
- Execute pip uninstall via subprocess
|
||||
- Log reason in info log
|
||||
- Add to removed package list
|
||||
- Remove package from self._installed_cache
|
||||
* Move to next package (only first match per package)
|
||||
4. Complete iteration through all package policies
|
||||
- **Return value**: List of removed package names (list of str)
|
||||
- **Exception handling**:
|
||||
- Individual package removal failure: Log warning only and continue to next package
|
||||
- **Call timing**:
|
||||
- Called at batch operation start to pre-remove conflicting packages
|
||||
- Called before multiple package installations to clean installation environment
|
||||
|
||||
* PipBatch.ensure_installed():
|
||||
- **Purpose**: Iterate through all policies and restore all packages satisfying restore conditions (batch processing)
|
||||
- **Parameters**: None
|
||||
- **Execution flow**:
|
||||
1. Call get_pip_policy() to get policy (lazy loading)
|
||||
2. Call self._get_installed_packages() to get cached installed package information
|
||||
3. Iterate through all package policies in policy dictionary:
|
||||
a. Check if each package has restore policy
|
||||
b. If restore policy exists:
|
||||
- Iterate through restore policy list top-to-bottom
|
||||
- Evaluate each policy's condition with self._evaluate_condition()
|
||||
- When first condition-satisfying policy found:
|
||||
* Get target package name (policy's "target" field)
|
||||
* Get version specified in version field
|
||||
* Check current version with self._installed_cache.get(target)
|
||||
* If current version is None or different from specified version:
|
||||
- Compose as package_spec = f"{target}=={version}" format
|
||||
- Generate command with manager_util.make_pip_cmd(["install", package_spec])
|
||||
- If extra_index_url exists, add ["--extra-index-url", url]
|
||||
- Execute pip install via subprocess
|
||||
- Log reason in info log
|
||||
- Add to restored package list
|
||||
- Update cache: self._installed_cache[target] = version
|
||||
* Move to next package (only first match per package)
|
||||
4. Complete iteration through all package policies
|
||||
- **Return value**: List of restored package names (list of str)
|
||||
- **Exception handling**:
|
||||
- Individual package installation failure: Log warning only and continue to next package
|
||||
- **Call timing**:
|
||||
- Called at batch operation end to restore essential package versions
|
||||
- Called for environment verification after multiple package installations
|
||||
|
||||
|
||||
## pip-policy.json Examples
|
||||
|
||||
### Base Policy File ({manager_util.comfyui_manager_path}/pip-policy.json)
|
||||
```json
|
||||
{
|
||||
"torch": {
|
||||
"apply_first_match": [
|
||||
{
|
||||
"type": "skip",
|
||||
"reason": "PyTorch installation should be managed manually due to CUDA compatibility"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
"opencv-python": {
|
||||
"apply_first_match": [
|
||||
{
|
||||
"type": "replace",
|
||||
"replacement": "opencv-contrib-python",
|
||||
"version": ">=4.8.0",
|
||||
"reason": "opencv-contrib-python includes all opencv-python features plus extras"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
"PIL": {
|
||||
"apply_first_match": [
|
||||
{
|
||||
"type": "replace",
|
||||
"replacement": "Pillow",
|
||||
"reason": "PIL is deprecated, use Pillow instead"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
"click": {
|
||||
"apply_first_match": [
|
||||
{
|
||||
"condition": {
|
||||
"type": "installed",
|
||||
"package": "colorama",
|
||||
"spec": "<0.5.0"
|
||||
},
|
||||
"type": "force_version",
|
||||
"version": "8.1.3",
|
||||
"reason": "click 8.1.3 compatible with colorama <0.5"
|
||||
}
|
||||
],
|
||||
"apply_all_matches": [
|
||||
{
|
||||
"type": "pin_dependencies",
|
||||
"pinned_packages": ["colorama"],
|
||||
"reason": "Prevent colorama upgrade that may break compatibility"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
"requests": {
|
||||
"apply_all_matches": [
|
||||
{
|
||||
"type": "pin_dependencies",
|
||||
"pinned_packages": ["urllib3", "certifi", "charset-normalizer"],
|
||||
"on_failure": "retry_without_pin",
|
||||
"reason": "Prevent urllib3 from upgrading to 2.x which has breaking changes"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
"six": {
|
||||
"restore": [
|
||||
{
|
||||
"target": "six",
|
||||
"version": "1.16.0",
|
||||
"reason": "six must be maintained at 1.16.0 for compatibility"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
"urllib3": {
|
||||
"restore": [
|
||||
{
|
||||
"condition": {
|
||||
"type": "installed",
|
||||
"spec": "!=1.26.15"
|
||||
},
|
||||
"target": "urllib3",
|
||||
"version": "1.26.15",
|
||||
"reason": "urllib3 must be 1.26.15 for compatibility with legacy code"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
"onnxruntime": {
|
||||
"apply_first_match": [
|
||||
{
|
||||
"condition": {
|
||||
"type": "platform",
|
||||
"os": "linux",
|
||||
"has_gpu": true
|
||||
},
|
||||
"type": "replace",
|
||||
"replacement": "onnxruntime-gpu",
|
||||
"reason": "Use GPU version on Linux with CUDA"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
"legacy-custom-node-package": {
|
||||
"apply_first_match": [
|
||||
{
|
||||
"condition": {
|
||||
"type": "platform",
|
||||
"comfyui_version": "<1.0.0"
|
||||
},
|
||||
"type": "force_version",
|
||||
"version": "0.9.0",
|
||||
"reason": "legacy-custom-node-package 0.9.0 is compatible with ComfyUI <1.0.0"
|
||||
},
|
||||
{
|
||||
"condition": {
|
||||
"type": "platform",
|
||||
"comfyui_version": ">=1.0.0"
|
||||
},
|
||||
"type": "force_version",
|
||||
"version": "1.5.0",
|
||||
"reason": "legacy-custom-node-package 1.5.0 is required for ComfyUI >=1.0.0"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
"tensorflow": {
|
||||
"apply_all_matches": [
|
||||
{
|
||||
"condition": {
|
||||
"type": "installed",
|
||||
"package": "torch"
|
||||
},
|
||||
"type": "warn",
|
||||
"message": "Installing TensorFlow alongside PyTorch may cause CUDA conflicts",
|
||||
"allow_continue": true
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
"some-package": {
|
||||
"uninstall": [
|
||||
{
|
||||
"condition": {
|
||||
"type": "installed",
|
||||
"package": "conflicting-package",
|
||||
"spec": ">=2.0.0"
|
||||
},
|
||||
"target": "conflicting-package",
|
||||
"reason": "conflicting-package >=2.0.0 conflicts with some-package"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
"banned-malicious-package": {
|
||||
"uninstall": [
|
||||
{
|
||||
"target": "banned-malicious-package",
|
||||
"reason": "Security vulnerability CVE-2024-XXXXX, always remove if attempting to install"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
"critical-package": {
|
||||
"restore": [
|
||||
{
|
||||
"condition": {
|
||||
"type": "installed",
|
||||
"package": "critical-package",
|
||||
"spec": "!=1.2.3"
|
||||
},
|
||||
"target": "critical-package",
|
||||
"version": "1.2.3",
|
||||
"extra_index_url": "https://custom-repo.example.com/simple",
|
||||
"reason": "critical-package must be version 1.2.3, restore if different or missing"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
"stable-package": {
|
||||
"apply_first_match": [
|
||||
{
|
||||
"condition": {
|
||||
"type": "installed",
|
||||
"package": "critical-dependency",
|
||||
"spec": ">=2.0.0"
|
||||
},
|
||||
"type": "force_version",
|
||||
"version": "1.5.0",
|
||||
"extra_index_url": "https://custom-repo.example.com/simple",
|
||||
"reason": "stable-package 1.5.0 is required when critical-dependency >=2.0.0 is installed"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
"new-experimental-package": {
|
||||
"apply_all_matches": [
|
||||
{
|
||||
"type": "pin_dependencies",
|
||||
"pinned_packages": ["numpy", "pandas", "scipy"],
|
||||
"on_failure": "retry_without_pin",
|
||||
"reason": "new-experimental-package may upgrade numpy/pandas/scipy, pin them to prevent breakage"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
"pytorch-addon": {
|
||||
"apply_all_matches": [
|
||||
{
|
||||
"condition": {
|
||||
"type": "installed",
|
||||
"package": "torch",
|
||||
"spec": ">=2.0.0"
|
||||
},
|
||||
"type": "pin_dependencies",
|
||||
"pinned_packages": ["torch", "torchvision", "torchaudio"],
|
||||
"on_failure": "fail",
|
||||
"reason": "pytorch-addon must not change PyTorch ecosystem versions"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Policy Structure Schema
|
||||
```json
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"type": "object",
|
||||
"patternProperties": {
|
||||
"^.*$": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"uninstall": {
|
||||
"type": "array",
|
||||
"description": "When condition satisfied (or always if no condition), remove package and terminate",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": ["target"],
|
||||
"properties": {
|
||||
"condition": {
|
||||
"type": "object",
|
||||
"description": "Optional: always remove if absent",
|
||||
"required": ["type"],
|
||||
"properties": {
|
||||
"type": {"enum": ["installed", "platform"]},
|
||||
"package": {"type": "string", "description": "Optional: defaults to self"},
|
||||
"spec": {"type": "string", "description": "Optional: version condition"},
|
||||
"os": {"type": "string"},
|
||||
"has_gpu": {"type": "boolean"},
|
||||
"comfyui_version": {"type": "string"}
|
||||
}
|
||||
},
|
||||
"target": {
|
||||
"type": "string",
|
||||
"description": "Package name to remove"
|
||||
},
|
||||
"reason": {"type": "string"}
|
||||
}
|
||||
}
|
||||
},
|
||||
"restore": {
|
||||
"type": "array",
|
||||
"description": "When condition satisfied (or always if no condition), restore package and terminate",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": ["target", "version"],
|
||||
"properties": {
|
||||
"condition": {
|
||||
"type": "object",
|
||||
"description": "Optional: always restore if absent",
|
||||
"required": ["type"],
|
||||
"properties": {
|
||||
"type": {"enum": ["installed", "platform"]},
|
||||
"package": {"type": "string", "description": "Optional: defaults to self"},
|
||||
"spec": {"type": "string", "description": "Optional: version condition"},
|
||||
"os": {"type": "string"},
|
||||
"has_gpu": {"type": "boolean"},
|
||||
"comfyui_version": {"type": "string"}
|
||||
}
|
||||
},
|
||||
"target": {
|
||||
"type": "string",
|
||||
"description": "Package name to restore"
|
||||
},
|
||||
"version": {
|
||||
"type": "string",
|
||||
"description": "Version to restore"
|
||||
},
|
||||
"extra_index_url": {"type": "string"},
|
||||
"reason": {"type": "string"}
|
||||
}
|
||||
}
|
||||
},
|
||||
"apply_first_match": {
|
||||
"type": "array",
|
||||
"description": "Execute only first condition-satisfying policy (exclusive)",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": ["type"],
|
||||
"properties": {
|
||||
"condition": {
|
||||
"type": "object",
|
||||
"description": "Optional: always apply if absent",
|
||||
"required": ["type"],
|
||||
"properties": {
|
||||
"type": {"enum": ["installed", "platform"]},
|
||||
"package": {"type": "string", "description": "Optional: defaults to self"},
|
||||
"spec": {"type": "string", "description": "Optional: version condition"},
|
||||
"os": {"type": "string"},
|
||||
"has_gpu": {"type": "boolean"},
|
||||
"comfyui_version": {"type": "string"}
|
||||
}
|
||||
},
|
||||
"type": {
|
||||
"enum": ["skip", "force_version", "replace"],
|
||||
"description": "Exclusive action: determines installation method"
|
||||
},
|
||||
"version": {"type": "string"},
|
||||
"replacement": {"type": "string"},
|
||||
"extra_index_url": {"type": "string"},
|
||||
"reason": {"type": "string"}
|
||||
}
|
||||
}
|
||||
},
|
||||
"apply_all_matches": {
|
||||
"type": "array",
|
||||
"description": "Execute all condition-satisfying policies (cumulative)",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": ["type"],
|
||||
"properties": {
|
||||
"condition": {
|
||||
"type": "object",
|
||||
"description": "Optional: always apply if absent",
|
||||
"required": ["type"],
|
||||
"properties": {
|
||||
"type": {"enum": ["installed", "platform"]},
|
||||
"package": {"type": "string", "description": "Optional: defaults to self"},
|
||||
"spec": {"type": "string", "description": "Optional: version condition"},
|
||||
"os": {"type": "string"},
|
||||
"has_gpu": {"type": "boolean"},
|
||||
"comfyui_version": {"type": "string"}
|
||||
}
|
||||
},
|
||||
"type": {
|
||||
"enum": ["pin_dependencies", "install_with", "warn"],
|
||||
"description": "Cumulative action: adds installation options"
|
||||
},
|
||||
"pinned_packages": {
|
||||
"type": "array",
|
||||
"items": {"type": "string"}
|
||||
},
|
||||
"on_failure": {"enum": ["fail", "retry_without_pin"]},
|
||||
"additional_packages": {"type": "array"},
|
||||
"message": {"type": "string"},
|
||||
"allow_continue": {"type": "boolean"},
|
||||
"reason": {"type": "string"}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
## Error Handling
|
||||
|
||||
* Default behavior when errors occur during policy execution:
|
||||
- Log error and continue
|
||||
- Only treat as installation failure when pin_dependencies's on_failure="fail"
|
||||
- For other cases, leave warning and attempt originally requested installation
|
||||
|
||||
|
||||
* pip_install: Performs pip package installation
|
||||
- Use manager_util.make_pip_cmd to generate commands for selective application of uv and pip
|
||||
- Provide functionality to skip policy application through override_policy flag
|
||||
614
comfyui_manager/common/pip_util.implementation-plan.en.md
Normal file
614
comfyui_manager/common/pip_util.implementation-plan.en.md
Normal file
@ -0,0 +1,614 @@
|
||||
# pip_util.py Implementation Plan Document
|
||||
|
||||
## 1. Project Overview
|
||||
|
||||
### Purpose
|
||||
Implement a policy-based pip package management system that minimizes breaking existing installed dependencies
|
||||
|
||||
### Core Features
|
||||
- JSON-based policy file loading and merging (lazy loading)
|
||||
- Per-package installation policy evaluation and application
|
||||
- Performance optimization through batch-level pip freeze caching
|
||||
- Automated conditional package removal/restoration
|
||||
|
||||
### Technology Stack
|
||||
- Python 3.x
|
||||
- packaging library (version comparison)
|
||||
- subprocess (pip command execution)
|
||||
- json (policy file parsing)
|
||||
|
||||
---
|
||||
|
||||
## 2. Architecture Design
|
||||
|
||||
### 2.1 Global Policy Management (Lazy Loading Pattern)
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────┐
|
||||
│ get_pip_policy() │
|
||||
│ - Auto-loads policy files on │
|
||||
│ first call via lazy loading │
|
||||
│ - Returns cache on subsequent calls│
|
||||
└─────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────┐
|
||||
│ _pip_policy_cache (global) │
|
||||
│ - Merged policy dictionary │
|
||||
│ - {package_name: policy_object} │
|
||||
└─────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### 2.2 Batch Operation Class (PipBatch)
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────┐
|
||||
│ PipBatch (Context Manager) │
|
||||
│ ┌───────────────────────────────┐ │
|
||||
│ │ _installed_cache │ │
|
||||
│ │ - Caches pip freeze results │ │
|
||||
│ │ - {package: version} │ │
|
||||
│ └───────────────────────────────┘ │
|
||||
│ │
|
||||
│ Public Methods: │
|
||||
│ ├─ install() │
|
||||
│ ├─ ensure_not_installed() │
|
||||
│ └─ ensure_installed() │
|
||||
│ │
|
||||
│ Private Methods: │
|
||||
│ ├─ _get_installed_packages() │
|
||||
│ ├─ _refresh_installed_cache() │
|
||||
│ ├─ _invalidate_cache() │
|
||||
│ ├─ _parse_package_spec() │
|
||||
│ └─ _evaluate_condition() │
|
||||
└─────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### 2.3 Policy Evaluation Flow
|
||||
|
||||
```
|
||||
install("numpy>=1.20") called
|
||||
│
|
||||
▼
|
||||
get_pip_policy() → Load policy (lazy)
|
||||
│
|
||||
▼
|
||||
Parse package name: "numpy"
|
||||
│
|
||||
▼
|
||||
Look up "numpy" policy in policy dictionary
|
||||
│
|
||||
├─ Evaluate apply_first_match (exclusive)
|
||||
│ ├─ skip → Return False (don't install)
|
||||
│ ├─ force_version → Change version
|
||||
│ └─ replace → Replace package
|
||||
│
|
||||
├─ Evaluate apply_all_matches (cumulative)
|
||||
│ ├─ pin_dependencies → Pin dependencies
|
||||
│ ├─ install_with → Additional packages
|
||||
│ └─ warn → Warning log
|
||||
│
|
||||
▼
|
||||
Execute pip install
|
||||
│
|
||||
▼
|
||||
Invalidate cache (_invalidate_cache)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Phase-by-Phase Implementation Plan
|
||||
|
||||
### Phase 1: Core Infrastructure Setup (2-3 hours)
|
||||
|
||||
#### Task 1.1: Project Structure and Dependency Setup (30 min)
|
||||
**Implementation**:
|
||||
- Create `pip_util.py` file
|
||||
- Add necessary import statements
|
||||
```python
|
||||
import json
|
||||
import logging
|
||||
import platform
|
||||
import re
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
from typing import Dict, List, Optional, Tuple
|
||||
|
||||
from packaging.specifiers import SpecifierSet
|
||||
from packaging.version import Version
|
||||
|
||||
from . import manager_util, context
|
||||
```
|
||||
- Set up logging
|
||||
```python
|
||||
logger = logging.getLogger(__name__)
|
||||
```
|
||||
|
||||
**Validation**:
|
||||
- Module loads without import errors
|
||||
- Logger works correctly
|
||||
|
||||
#### Task 1.2: Global Variable and get_pip_policy() Implementation (1 hour)
|
||||
**Implementation**:
|
||||
- Declare global variable
|
||||
```python
|
||||
_pip_policy_cache: Optional[Dict] = None
|
||||
```
|
||||
- Implement `get_pip_policy()` function
|
||||
- Check cache and early return
|
||||
- Read base policy file (`{manager_util.comfyui_manager_path}/pip-policy.json`)
|
||||
- Read user policy file (`{context.manager_files_path}/pip-policy.user.json`)
|
||||
- Create file if doesn't exist (for user policy)
|
||||
- Merge policies (complete package-level replacement)
|
||||
- Save to cache and return
|
||||
|
||||
**Exception Handling**:
|
||||
- `FileNotFoundError`: File not found → Use empty dictionary
|
||||
- `json.JSONDecodeError`: JSON parse failure → Warning log + empty dictionary
|
||||
- General exception: Warning log + empty dictionary
|
||||
|
||||
**Validation**:
|
||||
- Returns empty dictionary when policy files don't exist
|
||||
- Returns correct merged result when policy files exist
|
||||
- Confirms cache usage on second call (load log appears only once)
|
||||
|
||||
#### Task 1.3: PipBatch Class Basic Structure (30 min)
|
||||
**Implementation**:
|
||||
- Class definition and `__init__`
|
||||
```python
|
||||
class PipBatch:
|
||||
def __init__(self):
|
||||
self._installed_cache: Optional[Dict[str, str]] = None
|
||||
```
|
||||
- Context manager methods (`__enter__`, `__exit__`)
|
||||
```python
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, exc_type, exc_val, exc_tb):
|
||||
self._installed_cache = None
|
||||
return False
|
||||
```
|
||||
|
||||
**Validation**:
|
||||
- `with PipBatch() as batch:` syntax works correctly
|
||||
- Cache cleared on `__exit__` call
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: Caching and Utility Methods (2-3 hours)
|
||||
|
||||
#### Task 2.1: pip freeze Caching Methods (1 hour)
|
||||
**Implementation**:
|
||||
- Implement `_refresh_installed_cache()`
|
||||
- Call `manager_util.make_pip_cmd(["freeze"])`
|
||||
- Execute command via subprocess
|
||||
- Parse output (package==version format)
|
||||
- Exclude editable packages (-e) and comments (#)
|
||||
- Convert to dictionary and store in `self._installed_cache`
|
||||
|
||||
- Implement `_get_installed_packages()`
|
||||
- Call `_refresh_installed_cache()` if cache is None
|
||||
- Return cache
|
||||
|
||||
- Implement `_invalidate_cache()`
|
||||
- Set `self._installed_cache = None`
|
||||
|
||||
**Exception Handling**:
|
||||
- `subprocess.CalledProcessError`: pip freeze failure → Empty dictionary
|
||||
- Parse error: Ignore line + warning log
|
||||
|
||||
**Validation**:
|
||||
- pip freeze results correctly parsed into dictionary
|
||||
- New load occurs after cache invalidation and re-query
|
||||
|
||||
#### Task 2.2: Package Spec Parsing (30 min)
|
||||
**Implementation**:
|
||||
- Implement `_parse_package_spec(package_info)`
|
||||
- Regex pattern: `^([a-zA-Z0-9_-]+)([><=!~]+.*)?$`
|
||||
- Split package name and version spec
|
||||
- Return tuple: `(package_name, version_spec)`
|
||||
|
||||
**Exception Handling**:
|
||||
- Parse failure: Raise `ValueError`
|
||||
|
||||
**Validation**:
|
||||
- "numpy" → ("numpy", None)
|
||||
- "numpy==1.26.0" → ("numpy", "==1.26.0")
|
||||
- "pandas>=2.0.0" → ("pandas", ">=2.0.0")
|
||||
- Invalid format → ValueError
|
||||
|
||||
#### Task 2.3: Condition Evaluation Method (1.5 hours)
|
||||
**Implementation**:
|
||||
- Implement `_evaluate_condition(condition, package_name, installed_packages)`
|
||||
|
||||
**Handling by Condition Type**:
|
||||
1. **condition is None**: Always return True
|
||||
2. **"installed" type**:
|
||||
- `target_package = condition.get("package", package_name)`
|
||||
- Check version with `installed_packages.get(target_package)`
|
||||
- If spec exists, compare using `packaging.specifiers.SpecifierSet`
|
||||
- If no spec, only check installation status
|
||||
3. **"platform" type**:
|
||||
- `os` condition: Compare with `platform.system()`
|
||||
- `has_gpu` condition: Check `torch.cuda.is_available()` (False if torch unavailable)
|
||||
- `comfyui_version` condition: TODO (currently warning)
|
||||
|
||||
**Exception Handling**:
|
||||
- Version comparison failure: Warning log + return False
|
||||
- Unknown condition type: Warning log + return False
|
||||
|
||||
**Validation**:
|
||||
- Write test cases for each condition type
|
||||
- Verify edge case handling (torch not installed, invalid version format, etc.)
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: Core Installation Logic Implementation (4-5 hours)
|
||||
|
||||
#### Task 3.1: install() Method - Basic Flow (2 hours)
|
||||
**Implementation**:
|
||||
1. Parse package spec (`_parse_package_spec`)
|
||||
2. Query installed package cache (`_get_installed_packages`)
|
||||
3. If `override_policy=True`, install directly and return
|
||||
4. Call `get_pip_policy()` to load policy
|
||||
5. Default installation if no policy exists
|
||||
|
||||
**Validation**:
|
||||
- Verify policy ignored when override_policy=True
|
||||
- Verify default installation for packages without policy
|
||||
|
||||
#### Task 3.2: install() Method - apply_first_match Policy (1 hour)
|
||||
**Implementation**:
|
||||
- Iterate through policy list top-to-bottom
|
||||
- Evaluate each policy's condition (`_evaluate_condition`)
|
||||
- When condition satisfied:
|
||||
- **skip**: Log reason and return False
|
||||
- **force_version**: Force version change
|
||||
- **replace**: Replace package
|
||||
- Apply only first match (break)
|
||||
|
||||
**Validation**:
|
||||
- Verify installation blocked by skip policy
|
||||
- Verify version changed by force_version
|
||||
- Verify package replaced by replace
|
||||
|
||||
#### Task 3.3: install() Method - apply_all_matches Policy (1 hour)
|
||||
**Implementation**:
|
||||
- Iterate through policy list top-to-bottom
|
||||
- Evaluate each policy's condition
|
||||
- Apply all condition-satisfying policies:
|
||||
- **pin_dependencies**: Pin to installed version
|
||||
- **install_with**: Add to additional package list
|
||||
- **warn**: Output warning log
|
||||
|
||||
**Validation**:
|
||||
- Verify multiple policies applied simultaneously
|
||||
- Verify version pinning by pin_dependencies
|
||||
- Verify additional package installation by install_with
|
||||
|
||||
#### Task 3.4: install() Method - Installation Execution and Retry Logic (1 hour)
|
||||
**Implementation**:
|
||||
1. Compose final package list
|
||||
2. Generate command using `manager_util.make_pip_cmd()`
|
||||
3. Handle `extra_index_url`
|
||||
4. Execute installation via subprocess
|
||||
5. Handle failure based on on_failure setting:
|
||||
- `retry_without_pin`: Retry without pins
|
||||
- `fail`: Raise exception
|
||||
- Other: Warning log
|
||||
6. Invalidate cache on success
|
||||
|
||||
**Validation**:
|
||||
- Verify normal installation
|
||||
- Verify retry logic on pin failure
|
||||
- Verify error handling
|
||||
|
||||
---
|
||||
|
||||
### Phase 4: Batch Operation Methods Implementation (2-3 hours)
|
||||
|
||||
#### Task 4.1: ensure_not_installed() Implementation (1.5 hours)
|
||||
**Implementation**:
|
||||
1. Call `get_pip_policy()`
|
||||
2. Iterate through all package policies
|
||||
3. Check each package's uninstall policy
|
||||
4. When condition satisfied:
|
||||
- Check if target package is installed
|
||||
- If installed, execute `pip uninstall -y {target}`
|
||||
- Remove from cache
|
||||
- Add to removal list
|
||||
5. Execute only first match (per package)
|
||||
6. Return list of removed packages
|
||||
|
||||
**Exception Handling**:
|
||||
- Individual package removal failure: Warning log + continue
|
||||
|
||||
**Validation**:
|
||||
- Verify package removal by uninstall policy
|
||||
- Verify batch removal of multiple packages
|
||||
- Verify continued processing of other packages even on removal failure
|
||||
|
||||
#### Task 4.2: ensure_installed() Implementation (1.5 hours)
|
||||
**Implementation**:
|
||||
1. Call `get_pip_policy()`
|
||||
2. Iterate through all package policies
|
||||
3. Check each package's restore policy
|
||||
4. When condition satisfied:
|
||||
- Check target package's current version
|
||||
- If absent or different version:
|
||||
- Execute `pip install {target}=={version}`
|
||||
- Add extra_index_url if present
|
||||
- Update cache
|
||||
- Add to restoration list
|
||||
5. Execute only first match (per package)
|
||||
6. Return list of restored packages
|
||||
|
||||
**Exception Handling**:
|
||||
- Individual package installation failure: Warning log + continue
|
||||
|
||||
**Validation**:
|
||||
- Verify package restoration by restore policy
|
||||
- Verify reinstallation on version mismatch
|
||||
- Verify continued processing of other packages even on restoration failure
|
||||
|
||||
---
|
||||
|
||||
## 4. Testing Strategy
|
||||
|
||||
### 4.1 Unit Tests
|
||||
|
||||
#### Policy Loading Tests
|
||||
```python
|
||||
def test_get_pip_policy_empty():
|
||||
"""Returns empty dictionary when policy files don't exist"""
|
||||
|
||||
def test_get_pip_policy_merge():
|
||||
"""Correctly merges base and user policies"""
|
||||
|
||||
def test_get_pip_policy_cache():
|
||||
"""Uses cache on second call"""
|
||||
```
|
||||
|
||||
#### Package Parsing Tests
|
||||
```python
|
||||
def test_parse_package_spec_simple():
|
||||
"""'numpy' → ('numpy', None)"""
|
||||
|
||||
def test_parse_package_spec_version():
|
||||
"""'numpy==1.26.0' → ('numpy', '==1.26.0')"""
|
||||
|
||||
def test_parse_package_spec_range():
|
||||
"""'pandas>=2.0.0' → ('pandas', '>=2.0.0')"""
|
||||
|
||||
def test_parse_package_spec_invalid():
|
||||
"""Invalid format → ValueError"""
|
||||
```
|
||||
|
||||
#### Condition Evaluation Tests
|
||||
```python
|
||||
def test_evaluate_condition_none():
|
||||
"""None condition → True"""
|
||||
|
||||
def test_evaluate_condition_installed():
|
||||
"""Evaluates installed package condition"""
|
||||
|
||||
def test_evaluate_condition_platform():
|
||||
"""Evaluates platform condition"""
|
||||
```
|
||||
|
||||
### 4.2 Integration Tests
|
||||
|
||||
#### Installation Policy Tests
|
||||
```python
|
||||
def test_install_with_skip_policy():
|
||||
"""Blocks installation with skip policy"""
|
||||
|
||||
def test_install_with_force_version():
|
||||
"""Changes version with force_version policy"""
|
||||
|
||||
def test_install_with_replace():
|
||||
"""Replaces package with replace policy"""
|
||||
|
||||
def test_install_with_pin_dependencies():
|
||||
"""Pins versions with pin_dependencies"""
|
||||
```
|
||||
|
||||
#### Batch Operation Tests
|
||||
```python
|
||||
def test_ensure_not_installed():
|
||||
"""Removes packages with uninstall policy"""
|
||||
|
||||
def test_ensure_installed():
|
||||
"""Restores packages with restore policy"""
|
||||
|
||||
def test_batch_workflow():
|
||||
"""Tests complete batch workflow"""
|
||||
```
|
||||
|
||||
### 4.3 Edge Case Tests
|
||||
|
||||
```python
|
||||
def test_install_without_policy():
|
||||
"""Default installation for packages without policy"""
|
||||
|
||||
def test_install_override_policy():
|
||||
"""Ignores policy with override_policy=True"""
|
||||
|
||||
def test_pip_freeze_failure():
|
||||
"""Handles empty cache on pip freeze failure"""
|
||||
|
||||
def test_json_parse_error():
|
||||
"""Handles malformed JSON files"""
|
||||
|
||||
def test_subprocess_failure():
|
||||
"""Exception handling when pip command fails"""
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Error Handling Strategy
|
||||
|
||||
### 5.1 Policy Loading Errors
|
||||
- **File not found**: Warning log + empty dictionary
|
||||
- **JSON parse failure**: Error log + empty dictionary
|
||||
- **No read permission**: Warning log + empty dictionary
|
||||
|
||||
### 5.2 Package Installation Errors
|
||||
- **pip command failure**: Depends on on_failure setting
|
||||
- `retry_without_pin`: Retry
|
||||
- `fail`: Raise exception
|
||||
- Other: Warning log
|
||||
- **Invalid package spec**: Raise ValueError
|
||||
|
||||
### 5.3 Batch Operation Errors
|
||||
- **Individual package failure**: Warning log + continue to next package
|
||||
- **pip freeze failure**: Empty dictionary + warning log
|
||||
|
||||
---
|
||||
|
||||
## 6. Performance Optimization
|
||||
|
||||
### 6.1 Caching Strategy
|
||||
- **Policy cache**: Reused program-wide via global variable
|
||||
- **pip freeze cache**: Reused per batch, invalidated after install/remove
|
||||
- **lazy loading**: Load only when needed
|
||||
|
||||
### 6.2 Parallel Processing Considerations
|
||||
- Current implementation is not thread-safe
|
||||
- Consider adding threading.Lock if needed
|
||||
- Batch operations execute sequentially only
|
||||
|
||||
---
|
||||
|
||||
## 7. Documentation Requirements
|
||||
|
||||
### 7.1 Code Documentation
|
||||
- Docstrings required for all public methods
|
||||
- Specify parameters, return values, and exceptions
|
||||
- Include usage examples
|
||||
|
||||
### 7.2 User Guide
|
||||
- Explain `pip-policy.json` structure
|
||||
- Policy writing examples
|
||||
- Usage pattern examples
|
||||
|
||||
### 7.3 Developer Guide
|
||||
- Architecture explanation
|
||||
- Extension methods
|
||||
- Test execution methods
|
||||
|
||||
---
|
||||
|
||||
## 8. Deployment Checklist
|
||||
|
||||
### 8.1 Code Quality
|
||||
- [ ] All unit tests pass
|
||||
- [ ] All integration tests pass
|
||||
- [ ] Code coverage ≥80%
|
||||
- [ ] No linting errors (flake8, pylint)
|
||||
- [ ] Type hints complete (mypy passes)
|
||||
|
||||
### 8.2 Documentation
|
||||
- [ ] README.md written
|
||||
- [ ] API documentation generated
|
||||
- [ ] Example policy files written
|
||||
- [ ] Usage guide written
|
||||
|
||||
### 8.3 Performance Verification
|
||||
- [ ] Policy loading performance measured (<100ms)
|
||||
- [ ] pip freeze caching effectiveness verified (≥50% speed improvement)
|
||||
- [ ] Memory usage confirmed (<10MB)
|
||||
|
||||
### 8.4 Security Verification
|
||||
- [ ] Input validation complete
|
||||
- [ ] Path traversal prevention
|
||||
- [ ] Command injection prevention
|
||||
- [ ] JSON parsing safety confirmed
|
||||
|
||||
---
|
||||
|
||||
## 9. Future Improvements
|
||||
|
||||
### 9.1 Short-term (1-2 weeks)
|
||||
- Implement ComfyUI version check
|
||||
- Implement user confirmation prompt (allow_continue=false)
|
||||
- Thread-safe improvements (add Lock)
|
||||
|
||||
### 9.2 Mid-term (1-2 months)
|
||||
- Add policy validation tools
|
||||
- Policy migration tools
|
||||
- More detailed logging and debugging options
|
||||
|
||||
### 9.3 Long-term (3-6 months)
|
||||
- Web UI for policy management
|
||||
- Provide policy templates
|
||||
- Community policy sharing system
|
||||
|
||||
---
|
||||
|
||||
## 10. Risks and Mitigation Strategies
|
||||
|
||||
### Risk 1: Policy Conflicts
|
||||
**Description**: Policies for different packages may conflict
|
||||
**Mitigation**: Develop policy validation tools, conflict detection algorithm
|
||||
|
||||
### Risk 2: pip Version Compatibility
|
||||
**Description**: Must work across various pip versions
|
||||
**Mitigation**: Test on multiple pip versions, version-specific branching
|
||||
|
||||
### Risk 3: Performance Degradation
|
||||
**Description**: Installation speed may decrease due to policy evaluation
|
||||
**Mitigation**: Optimize caching, minimize condition evaluation
|
||||
|
||||
### Risk 4: Policy Misconfiguration
|
||||
**Description**: Users may write incorrect policies
|
||||
**Mitigation**: JSON schema validation, provide examples and guides
|
||||
|
||||
---
|
||||
|
||||
## 11. Timeline
|
||||
|
||||
### Week 1
|
||||
- Phase 1: Core Infrastructure Setup (Day 1-2)
|
||||
- Phase 2: Caching and Utility Methods (Day 3-4)
|
||||
- Write unit tests (Day 5)
|
||||
|
||||
### Week 2
|
||||
- Phase 3: Core Installation Logic Implementation (Day 1-3)
|
||||
- Phase 4: Batch Operation Methods Implementation (Day 4-5)
|
||||
|
||||
### Week 3
|
||||
- Integration and edge case testing (Day 1-2)
|
||||
- Documentation (Day 3)
|
||||
- Code review and refactoring (Day 4-5)
|
||||
|
||||
### Week 4
|
||||
- Performance optimization (Day 1-2)
|
||||
- Security verification (Day 3)
|
||||
- Final testing and deployment preparation (Day 4-5)
|
||||
|
||||
---
|
||||
|
||||
## 12. Success Criteria
|
||||
|
||||
### Feature Completeness
|
||||
- ✅ All policy types (uninstall, apply_first_match, apply_all_matches, restore) work correctly
|
||||
- ✅ Policy merge logic works correctly
|
||||
- ✅ Batch operations perform normally
|
||||
|
||||
### Quality Metrics
|
||||
- ✅ Test coverage ≥80%
|
||||
- ✅ All tests pass
|
||||
- ✅ 0 linting errors
|
||||
- ✅ 100% type hint completion
|
||||
|
||||
### Performance Metrics
|
||||
- ✅ Policy loading <100ms
|
||||
- ✅ ≥50% performance improvement with pip freeze caching
|
||||
- ✅ Memory usage <10MB
|
||||
|
||||
### Usability
|
||||
- ✅ Clear error messages
|
||||
- ✅ Sufficient documentation
|
||||
- ✅ Verified in real-world use cases
|
||||
629
comfyui_manager/common/pip_util.py
Normal file
629
comfyui_manager/common/pip_util.py
Normal file
@ -0,0 +1,629 @@
|
||||
"""
|
||||
pip_util - Policy-based pip package management system
|
||||
|
||||
This module provides a policy-based approach to pip package installation
|
||||
to minimize dependency conflicts and protect existing installed packages.
|
||||
|
||||
Usage:
|
||||
# Batch operations (policy auto-loaded)
|
||||
with PipBatch() as batch:
|
||||
batch.ensure_not_installed()
|
||||
batch.install("numpy>=1.20")
|
||||
batch.install("pandas>=2.0")
|
||||
batch.install("scipy>=1.7")
|
||||
batch.ensure_installed()
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
import platform
|
||||
import re
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
from typing import Dict, List, Optional, Tuple
|
||||
|
||||
from packaging.requirements import Requirement
|
||||
from packaging.specifiers import SpecifierSet
|
||||
from packaging.version import Version
|
||||
|
||||
from . import manager_util, context
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Global policy cache (lazy loaded on first access)
|
||||
_pip_policy_cache: Optional[Dict] = None
|
||||
|
||||
|
||||
def get_pip_policy() -> Dict:
|
||||
"""
|
||||
Get pip policy with lazy loading.
|
||||
|
||||
Returns the cached policy if available, otherwise loads it from files.
|
||||
This function automatically loads the policy on first access.
|
||||
|
||||
Thread safety: This function is NOT thread-safe.
|
||||
Ensure single-threaded access during initialization.
|
||||
|
||||
Returns:
|
||||
Dictionary of merged pip policies
|
||||
|
||||
Example:
|
||||
>>> policy = get_pip_policy()
|
||||
>>> numpy_policy = policy.get("numpy", {})
|
||||
"""
|
||||
global _pip_policy_cache
|
||||
|
||||
# Return cached policy if already loaded
|
||||
if _pip_policy_cache is not None:
|
||||
logger.debug("Returning cached pip policy")
|
||||
return _pip_policy_cache
|
||||
|
||||
logger.info("Loading pip policies...")
|
||||
|
||||
# Load base policy
|
||||
base_policy = {}
|
||||
base_policy_path = Path(manager_util.comfyui_manager_path) / "pip-policy.json"
|
||||
|
||||
try:
|
||||
if base_policy_path.exists():
|
||||
with open(base_policy_path, 'r', encoding='utf-8') as f:
|
||||
base_policy = json.load(f)
|
||||
logger.debug(f"Loaded base policy from {base_policy_path}")
|
||||
else:
|
||||
logger.warning(f"Base policy file not found: {base_policy_path}")
|
||||
except json.JSONDecodeError as e:
|
||||
logger.error(f"Failed to parse base policy JSON: {e}")
|
||||
base_policy = {}
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to read base policy file: {e}")
|
||||
base_policy = {}
|
||||
|
||||
# Load user policy
|
||||
user_policy = {}
|
||||
user_policy_path = Path(context.manager_files_path) / "pip-policy.user.json"
|
||||
|
||||
try:
|
||||
if user_policy_path.exists():
|
||||
with open(user_policy_path, 'r', encoding='utf-8') as f:
|
||||
user_policy = json.load(f)
|
||||
logger.debug(f"Loaded user policy from {user_policy_path}")
|
||||
else:
|
||||
# Create empty user policy file
|
||||
user_policy_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(user_policy_path, 'w', encoding='utf-8') as f:
|
||||
json.dump({"_comment": "User-specific pip policy overrides"}, f, indent=2)
|
||||
logger.info(f"Created empty user policy file: {user_policy_path}")
|
||||
except json.JSONDecodeError as e:
|
||||
logger.warning(f"Failed to parse user policy JSON: {e}")
|
||||
user_policy = {}
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to read user policy file: {e}")
|
||||
user_policy = {}
|
||||
|
||||
# Merge policies (package-level override: user completely replaces base per package)
|
||||
merged_policy = base_policy.copy()
|
||||
for package_name, package_policy in user_policy.items():
|
||||
if package_name.startswith("_"): # Skip metadata fields like _comment
|
||||
continue
|
||||
merged_policy[package_name] = package_policy # Complete package replacement
|
||||
|
||||
# Store in global cache
|
||||
_pip_policy_cache = merged_policy
|
||||
logger.info(f"Policy loaded successfully: {len(_pip_policy_cache)} package policies")
|
||||
|
||||
return _pip_policy_cache
|
||||
|
||||
|
||||
class PipBatch:
|
||||
"""
|
||||
Pip package installation batch manager.
|
||||
|
||||
Maintains pip freeze cache during a batch of operations for performance optimization.
|
||||
|
||||
Usage pattern:
|
||||
# Batch operations (policy auto-loaded)
|
||||
with PipBatch() as batch:
|
||||
batch.ensure_not_installed()
|
||||
batch.install("numpy>=1.20")
|
||||
batch.install("pandas>=2.0")
|
||||
batch.install("scipy>=1.7")
|
||||
batch.ensure_installed()
|
||||
|
||||
Attributes:
|
||||
_installed_cache: Cache of installed packages from pip freeze
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize PipBatch with empty cache."""
|
||||
self._installed_cache: Optional[Dict[str, str]] = None
|
||||
|
||||
def __enter__(self):
|
||||
"""Enter context manager."""
|
||||
return self
|
||||
|
||||
def __exit__(self, exc_type, exc_val, exc_tb):
|
||||
"""Exit context manager and clear cache."""
|
||||
self._installed_cache = None
|
||||
return False
|
||||
|
||||
def _refresh_installed_cache(self) -> None:
|
||||
"""
|
||||
Refresh the installed packages cache by executing pip freeze.
|
||||
|
||||
Parses pip freeze output into a dictionary of {package_name: version}.
|
||||
Ignores editable packages and comments.
|
||||
|
||||
Raises:
|
||||
No exceptions raised - failures result in empty cache with warning log
|
||||
"""
|
||||
try:
|
||||
cmd = manager_util.make_pip_cmd(["freeze"])
|
||||
result = subprocess.run(cmd, capture_output=True, text=True, check=True)
|
||||
|
||||
packages = {}
|
||||
for line in result.stdout.strip().split('\n'):
|
||||
line = line.strip()
|
||||
|
||||
# Skip empty lines
|
||||
if not line:
|
||||
continue
|
||||
|
||||
# Skip editable packages (-e /path/to/package or -e git+https://...)
|
||||
# Editable packages don't have version info and are typically development-only
|
||||
if line.startswith('-e '):
|
||||
continue
|
||||
|
||||
# Skip comments (defensive: pip freeze typically doesn't output comments,
|
||||
# but this handles manually edited requirements.txt or future pip changes)
|
||||
if line.startswith('#'):
|
||||
continue
|
||||
|
||||
# Parse package==version
|
||||
if '==' in line:
|
||||
try:
|
||||
package_name, version = line.split('==', 1)
|
||||
packages[package_name.strip()] = version.strip()
|
||||
except ValueError:
|
||||
logger.warning(f"Failed to parse pip freeze line: {line}")
|
||||
continue
|
||||
|
||||
self._installed_cache = packages
|
||||
logger.debug(f"Refreshed installed packages cache: {len(packages)} packages")
|
||||
|
||||
except subprocess.CalledProcessError as e:
|
||||
logger.warning(f"pip freeze failed: {e}")
|
||||
self._installed_cache = {}
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to refresh installed packages cache: {e}")
|
||||
self._installed_cache = {}
|
||||
|
||||
def _get_installed_packages(self) -> Dict[str, str]:
|
||||
"""
|
||||
Get cached installed packages, refresh if cache is None.
|
||||
|
||||
Returns:
|
||||
Dictionary of {package_name: version}
|
||||
"""
|
||||
if self._installed_cache is None:
|
||||
self._refresh_installed_cache()
|
||||
return self._installed_cache
|
||||
|
||||
def _invalidate_cache(self) -> None:
|
||||
"""
|
||||
Invalidate the installed packages cache.
|
||||
|
||||
Should be called after install/uninstall operations.
|
||||
"""
|
||||
self._installed_cache = None
|
||||
|
||||
def _parse_package_spec(self, package_info: str) -> Tuple[str, Optional[str]]:
|
||||
"""
|
||||
Parse package spec string into package name and version spec using PEP 508.
|
||||
|
||||
Uses the packaging library to properly parse package specifications according to
|
||||
PEP 508 standard, which handles complex cases like extras and multiple version
|
||||
constraints that simple regex cannot handle correctly.
|
||||
|
||||
Args:
|
||||
package_info: Package specification like "numpy", "numpy==1.26.0", "numpy>=1.20.0",
|
||||
or complex specs like "package[extra]>=1.0,<2.0"
|
||||
|
||||
Returns:
|
||||
Tuple of (package_name, version_spec)
|
||||
Examples: ("numpy", "==1.26.0"), ("pandas", ">=2.0.0"), ("scipy", None)
|
||||
Package names are normalized (e.g., "NumPy" -> "numpy")
|
||||
|
||||
Raises:
|
||||
ValueError: If package_info cannot be parsed according to PEP 508
|
||||
|
||||
Example:
|
||||
>>> batch._parse_package_spec("numpy>=1.20")
|
||||
("numpy", ">=1.20")
|
||||
>>> batch._parse_package_spec("requests[security]>=2.0,<3.0")
|
||||
("requests", ">=2.0,<3.0")
|
||||
"""
|
||||
try:
|
||||
req = Requirement(package_info)
|
||||
package_name = req.name # Normalized package name
|
||||
version_spec = str(req.specifier) if req.specifier else None
|
||||
return package_name, version_spec
|
||||
except Exception as e:
|
||||
raise ValueError(f"Invalid package spec: {package_info}") from e
|
||||
|
||||
def _evaluate_condition(self, condition: Optional[Dict], package_name: str,
|
||||
installed_packages: Dict[str, str]) -> bool:
|
||||
"""
|
||||
Evaluate policy condition and return whether it's satisfied.
|
||||
|
||||
Args:
|
||||
condition: Policy condition object (dict) or None
|
||||
package_name: Current package being processed
|
||||
installed_packages: Dictionary of {package_name: version}
|
||||
|
||||
Returns:
|
||||
True if condition is satisfied, False otherwise
|
||||
None condition always returns True
|
||||
|
||||
Example:
|
||||
>>> condition = {"type": "installed", "package": "numpy", "spec": ">=1.20"}
|
||||
>>> batch._evaluate_condition(condition, "numba", {"numpy": "1.26.0"})
|
||||
True
|
||||
"""
|
||||
# No condition means always satisfied
|
||||
if condition is None:
|
||||
return True
|
||||
|
||||
condition_type = condition.get("type")
|
||||
|
||||
if condition_type == "installed":
|
||||
# Check if a package is installed with optional version spec
|
||||
target_package = condition.get("package", package_name)
|
||||
installed_version = installed_packages.get(target_package)
|
||||
|
||||
# Package not installed
|
||||
if installed_version is None:
|
||||
return False
|
||||
|
||||
# Check version spec if provided
|
||||
spec = condition.get("spec")
|
||||
if spec:
|
||||
try:
|
||||
specifier = SpecifierSet(spec)
|
||||
return Version(installed_version) in specifier
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to compare version {installed_version} with spec {spec}: {e}")
|
||||
return False
|
||||
|
||||
# Package is installed (no spec check)
|
||||
return True
|
||||
|
||||
elif condition_type == "platform":
|
||||
# Check platform conditions (os, has_gpu, comfyui_version)
|
||||
conditions_met = True
|
||||
|
||||
# Check OS
|
||||
if "os" in condition:
|
||||
expected_os = condition["os"].lower()
|
||||
actual_os = platform.system().lower()
|
||||
if expected_os not in actual_os and actual_os not in expected_os:
|
||||
conditions_met = False
|
||||
|
||||
# Check GPU availability
|
||||
if "has_gpu" in condition:
|
||||
expected_gpu = condition["has_gpu"]
|
||||
try:
|
||||
import torch
|
||||
has_gpu = torch.cuda.is_available()
|
||||
except ImportError:
|
||||
has_gpu = False
|
||||
|
||||
if expected_gpu != has_gpu:
|
||||
conditions_met = False
|
||||
|
||||
# Check ComfyUI version
|
||||
if "comfyui_version" in condition:
|
||||
# TODO: Implement ComfyUI version check
|
||||
logger.warning("ComfyUI version condition not yet implemented")
|
||||
|
||||
return conditions_met
|
||||
|
||||
else:
|
||||
logger.warning(f"Unknown condition type: {condition_type}")
|
||||
return False
|
||||
|
||||
def install(self, package_info: str, extra_index_url: Optional[str] = None,
|
||||
override_policy: bool = False) -> bool:
|
||||
"""
|
||||
Install a pip package with policy-based modifications.
|
||||
|
||||
Args:
|
||||
package_info: Package specification (e.g., "numpy", "numpy==1.26.0", "numpy>=1.20.0")
|
||||
extra_index_url: Additional package repository URL (optional)
|
||||
override_policy: If True, skip policy application and install directly (default: False)
|
||||
|
||||
Returns:
|
||||
True if installation succeeded, False if skipped by policy
|
||||
|
||||
Raises:
|
||||
ValueError: If package_info cannot be parsed
|
||||
subprocess.CalledProcessError: If installation fails (depending on policy on_failure settings)
|
||||
|
||||
Example:
|
||||
>>> with PipBatch() as batch:
|
||||
... batch.install("numpy>=1.20")
|
||||
... batch.install("torch", override_policy=True)
|
||||
"""
|
||||
# Parse package spec
|
||||
try:
|
||||
package_name, version_spec = self._parse_package_spec(package_info)
|
||||
except ValueError as e:
|
||||
logger.error(f"Invalid package spec: {e}")
|
||||
raise
|
||||
|
||||
# Get installed packages cache
|
||||
installed_packages = self._get_installed_packages()
|
||||
|
||||
# Override policy - skip to direct installation
|
||||
if override_policy:
|
||||
logger.info(f"Installing {package_info} (policy override)")
|
||||
cmd = manager_util.make_pip_cmd(["install", package_info])
|
||||
if extra_index_url:
|
||||
cmd.extend(["--extra-index-url", extra_index_url])
|
||||
|
||||
try:
|
||||
subprocess.run(cmd, check=True)
|
||||
self._invalidate_cache()
|
||||
logger.info(f"Successfully installed {package_info}")
|
||||
return True
|
||||
except subprocess.CalledProcessError as e:
|
||||
logger.error(f"Failed to install {package_info}: {e}")
|
||||
raise
|
||||
|
||||
# Get policy (lazy loading)
|
||||
pip_policy = get_pip_policy()
|
||||
policy = pip_policy.get(package_name, {})
|
||||
|
||||
# If no policy, proceed with default installation
|
||||
if not policy:
|
||||
logger.debug(f"No policy found for {package_name}, proceeding with default installation")
|
||||
cmd = manager_util.make_pip_cmd(["install", package_info])
|
||||
if extra_index_url:
|
||||
cmd.extend(["--extra-index-url", extra_index_url])
|
||||
|
||||
try:
|
||||
subprocess.run(cmd, check=True)
|
||||
self._invalidate_cache()
|
||||
logger.info(f"Successfully installed {package_info}")
|
||||
return True
|
||||
except subprocess.CalledProcessError as e:
|
||||
logger.error(f"Failed to install {package_info}: {e}")
|
||||
raise
|
||||
|
||||
# Apply apply_first_match policies (exclusive - first match only)
|
||||
final_package_info = package_info
|
||||
final_extra_index_url = extra_index_url
|
||||
policy_reason = None
|
||||
|
||||
apply_first_match = policy.get("apply_first_match", [])
|
||||
for policy_item in apply_first_match:
|
||||
condition = policy_item.get("condition")
|
||||
if self._evaluate_condition(condition, package_name, installed_packages):
|
||||
policy_type = policy_item.get("type")
|
||||
|
||||
if policy_type == "skip":
|
||||
reason = policy_item.get("reason", "No reason provided")
|
||||
logger.info(f"Skipping installation of {package_name}: {reason}")
|
||||
return False
|
||||
|
||||
elif policy_type == "force_version":
|
||||
forced_version = policy_item.get("version")
|
||||
final_package_info = f"{package_name}=={forced_version}"
|
||||
policy_reason = policy_item.get("reason")
|
||||
if "extra_index_url" in policy_item:
|
||||
final_extra_index_url = policy_item["extra_index_url"]
|
||||
logger.info(f"Force version for {package_name}: {forced_version} ({policy_reason})")
|
||||
break # First match only
|
||||
|
||||
elif policy_type == "replace":
|
||||
replacement = policy_item.get("replacement")
|
||||
replacement_version = policy_item.get("version", "")
|
||||
if replacement_version:
|
||||
final_package_info = f"{replacement}{replacement_version}"
|
||||
else:
|
||||
final_package_info = replacement
|
||||
policy_reason = policy_item.get("reason")
|
||||
if "extra_index_url" in policy_item:
|
||||
final_extra_index_url = policy_item["extra_index_url"]
|
||||
logger.info(f"Replacing {package_name} with {final_package_info}: {policy_reason}")
|
||||
break # First match only
|
||||
|
||||
# Apply apply_all_matches policies (cumulative - all matches)
|
||||
additional_packages = []
|
||||
pinned_packages = []
|
||||
pin_on_failure = "fail"
|
||||
|
||||
apply_all_matches = policy.get("apply_all_matches", [])
|
||||
for policy_item in apply_all_matches:
|
||||
condition = policy_item.get("condition")
|
||||
if self._evaluate_condition(condition, package_name, installed_packages):
|
||||
policy_type = policy_item.get("type")
|
||||
|
||||
if policy_type == "pin_dependencies":
|
||||
pin_list = policy_item.get("pinned_packages", [])
|
||||
for pkg in pin_list:
|
||||
installed_version = installed_packages.get(pkg)
|
||||
if installed_version:
|
||||
pinned_packages.append(f"{pkg}=={installed_version}")
|
||||
else:
|
||||
logger.warning(f"Cannot pin {pkg}: not currently installed")
|
||||
pin_on_failure = policy_item.get("on_failure", "fail")
|
||||
reason = policy_item.get("reason", "")
|
||||
logger.info(f"Pinning dependencies: {pinned_packages} ({reason})")
|
||||
|
||||
elif policy_type == "install_with":
|
||||
additional = policy_item.get("additional_packages", [])
|
||||
additional_packages.extend(additional)
|
||||
reason = policy_item.get("reason", "")
|
||||
logger.info(f"Installing additional packages: {additional} ({reason})")
|
||||
|
||||
elif policy_type == "warn":
|
||||
message = policy_item.get("message", "")
|
||||
allow_continue = policy_item.get("allow_continue", True)
|
||||
logger.warning(f"Policy warning for {package_name}: {message}")
|
||||
if not allow_continue:
|
||||
# TODO: Implement user confirmation
|
||||
logger.info("User confirmation required (not implemented, continuing)")
|
||||
|
||||
# Build final package list
|
||||
packages_to_install = [final_package_info] + pinned_packages + additional_packages
|
||||
|
||||
# Execute installation
|
||||
cmd = manager_util.make_pip_cmd(["install"] + packages_to_install)
|
||||
if final_extra_index_url:
|
||||
cmd.extend(["--extra-index-url", final_extra_index_url])
|
||||
|
||||
try:
|
||||
subprocess.run(cmd, check=True)
|
||||
self._invalidate_cache()
|
||||
if policy_reason:
|
||||
logger.info(f"Successfully installed {final_package_info}: {policy_reason}")
|
||||
else:
|
||||
logger.info(f"Successfully installed {final_package_info}")
|
||||
return True
|
||||
|
||||
except subprocess.CalledProcessError as e:
|
||||
# Handle installation failure
|
||||
if pinned_packages and pin_on_failure == "retry_without_pin":
|
||||
logger.warning(f"Installation failed with pinned dependencies, retrying without pins")
|
||||
retry_cmd = manager_util.make_pip_cmd(["install", final_package_info])
|
||||
if final_extra_index_url:
|
||||
retry_cmd.extend(["--extra-index-url", final_extra_index_url])
|
||||
|
||||
try:
|
||||
subprocess.run(retry_cmd, check=True)
|
||||
self._invalidate_cache()
|
||||
logger.info(f"Successfully installed {final_package_info} (without pins)")
|
||||
return True
|
||||
except subprocess.CalledProcessError as retry_error:
|
||||
logger.error(f"Retry installation also failed: {retry_error}")
|
||||
raise
|
||||
|
||||
elif pin_on_failure == "fail":
|
||||
logger.error(f"Installation failed: {e}")
|
||||
raise
|
||||
|
||||
else:
|
||||
logger.warning(f"Installation failed, but continuing: {e}")
|
||||
return False
|
||||
|
||||
def ensure_not_installed(self) -> List[str]:
|
||||
"""
|
||||
Remove all packages matching uninstall policies (batch processing).
|
||||
|
||||
Iterates through all package policies and executes uninstall actions
|
||||
where conditions are satisfied.
|
||||
|
||||
Returns:
|
||||
List of removed package names
|
||||
|
||||
Example:
|
||||
>>> with PipBatch() as batch:
|
||||
... removed = batch.ensure_not_installed()
|
||||
... print(f"Removed: {removed}")
|
||||
"""
|
||||
# Get policy (lazy loading)
|
||||
pip_policy = get_pip_policy()
|
||||
|
||||
installed_packages = self._get_installed_packages()
|
||||
removed_packages = []
|
||||
|
||||
for package_name, policy in pip_policy.items():
|
||||
uninstall_policies = policy.get("uninstall", [])
|
||||
|
||||
for uninstall_policy in uninstall_policies:
|
||||
condition = uninstall_policy.get("condition")
|
||||
|
||||
if self._evaluate_condition(condition, package_name, installed_packages):
|
||||
target = uninstall_policy.get("target")
|
||||
reason = uninstall_policy.get("reason", "No reason provided")
|
||||
|
||||
# Check if target is installed
|
||||
if target in installed_packages:
|
||||
try:
|
||||
cmd = manager_util.make_pip_cmd(["uninstall", "-y", target])
|
||||
subprocess.run(cmd, check=True)
|
||||
|
||||
logger.info(f"Uninstalled {target}: {reason}")
|
||||
removed_packages.append(target)
|
||||
|
||||
# Remove from cache
|
||||
del installed_packages[target]
|
||||
|
||||
except subprocess.CalledProcessError as e:
|
||||
logger.warning(f"Failed to uninstall {target}: {e}")
|
||||
|
||||
# First match only per package
|
||||
break
|
||||
|
||||
return removed_packages
|
||||
|
||||
def ensure_installed(self) -> List[str]:
|
||||
"""
|
||||
Restore all packages matching restore policies (batch processing).
|
||||
|
||||
Iterates through all package policies and executes restore actions
|
||||
where conditions are satisfied.
|
||||
|
||||
Returns:
|
||||
List of restored package names
|
||||
|
||||
Example:
|
||||
>>> with PipBatch() as batch:
|
||||
... batch.install("numpy>=1.20")
|
||||
... restored = batch.ensure_installed()
|
||||
... print(f"Restored: {restored}")
|
||||
"""
|
||||
# Get policy (lazy loading)
|
||||
pip_policy = get_pip_policy()
|
||||
|
||||
installed_packages = self._get_installed_packages()
|
||||
restored_packages = []
|
||||
|
||||
for package_name, policy in pip_policy.items():
|
||||
restore_policies = policy.get("restore", [])
|
||||
|
||||
for restore_policy in restore_policies:
|
||||
condition = restore_policy.get("condition")
|
||||
|
||||
if self._evaluate_condition(condition, package_name, installed_packages):
|
||||
target = restore_policy.get("target")
|
||||
version = restore_policy.get("version")
|
||||
reason = restore_policy.get("reason", "No reason provided")
|
||||
extra_index_url = restore_policy.get("extra_index_url")
|
||||
|
||||
# Check if target needs restoration
|
||||
current_version = installed_packages.get(target)
|
||||
|
||||
if current_version is None or current_version != version:
|
||||
try:
|
||||
package_spec = f"{target}=={version}"
|
||||
cmd = manager_util.make_pip_cmd(["install", package_spec])
|
||||
|
||||
if extra_index_url:
|
||||
cmd.extend(["--extra-index-url", extra_index_url])
|
||||
|
||||
subprocess.run(cmd, check=True)
|
||||
|
||||
logger.info(f"Restored {package_spec}: {reason}")
|
||||
restored_packages.append(target)
|
||||
|
||||
# Update cache
|
||||
installed_packages[target] = version
|
||||
|
||||
except subprocess.CalledProcessError as e:
|
||||
logger.warning(f"Failed to restore {target}: {e}")
|
||||
|
||||
# First match only per package
|
||||
break
|
||||
|
||||
return restored_packages
|
||||
2916
comfyui_manager/common/pip_util.test-design.md
Normal file
2916
comfyui_manager/common/pip_util.test-design.md
Normal file
File diff suppressed because it is too large
Load Diff
@ -1574,9 +1574,6 @@ class ManagerFuncs:
|
||||
def __init__(self):
|
||||
pass
|
||||
|
||||
def get_current_preview_method(self):
|
||||
return "none"
|
||||
|
||||
def run_script(self, cmd, cwd='.'):
|
||||
if len(cmd) > 0 and cmd[0].startswith("#"):
|
||||
print(f"[ComfyUI-Manager] Unexpected behavior: `{cmd}`")
|
||||
@ -1594,14 +1591,12 @@ def write_config():
|
||||
config = configparser.ConfigParser(strict=False)
|
||||
|
||||
config['default'] = {
|
||||
'preview_method': manager_funcs.get_current_preview_method(),
|
||||
'git_exe': get_config()['git_exe'],
|
||||
'use_uv': get_config()['use_uv'],
|
||||
'channel_url': get_config()['channel_url'],
|
||||
'share_option': get_config()['share_option'],
|
||||
'bypass_ssl': get_config()['bypass_ssl'],
|
||||
"file_logging": get_config()['file_logging'],
|
||||
'component_policy': get_config()['component_policy'],
|
||||
'update_policy': get_config()['update_policy'],
|
||||
'windows_selector_event_loop_policy': get_config()['windows_selector_event_loop_policy'],
|
||||
'model_download_by_agent': get_config()['model_download_by_agent'],
|
||||
@ -1634,7 +1629,6 @@ def read_config():
|
||||
|
||||
return {
|
||||
'http_channel_enabled': get_bool('http_channel_enabled', False),
|
||||
'preview_method': default_conf.get('preview_method', manager_funcs.get_current_preview_method()).lower(),
|
||||
'git_exe': default_conf.get('git_exe', ''),
|
||||
'use_uv': get_bool('use_uv', True),
|
||||
'channel_url': default_conf.get('channel_url', DEFAULT_CHANNEL),
|
||||
@ -1642,7 +1636,6 @@ def read_config():
|
||||
'share_option': default_conf.get('share_option', 'all').lower(),
|
||||
'bypass_ssl': get_bool('bypass_ssl', False),
|
||||
'file_logging': get_bool('file_logging', True),
|
||||
'component_policy': default_conf.get('component_policy', 'workflow').lower(),
|
||||
'update_policy': default_conf.get('update_policy', 'stable-comfyui').lower(),
|
||||
'windows_selector_event_loop_policy': get_bool('windows_selector_event_loop_policy', False),
|
||||
'model_download_by_agent': get_bool('model_download_by_agent', False),
|
||||
@ -1661,7 +1654,6 @@ def read_config():
|
||||
|
||||
return {
|
||||
'http_channel_enabled': False,
|
||||
'preview_method': manager_funcs.get_current_preview_method(),
|
||||
'git_exe': '',
|
||||
'use_uv': manager_util.use_uv,
|
||||
'channel_url': DEFAULT_CHANNEL,
|
||||
@ -1669,7 +1661,6 @@ def read_config():
|
||||
'share_option': 'all',
|
||||
'bypass_ssl': manager_util.bypass_ssl,
|
||||
'file_logging': True,
|
||||
'component_policy': 'workflow',
|
||||
'update_policy': 'stable-comfyui',
|
||||
'windows_selector_event_loop_policy': False,
|
||||
'model_download_by_agent': False,
|
||||
|
||||
@ -27,7 +27,6 @@ from typing import Any, Optional
|
||||
from comfyui_manager.common.timestamp_utils import get_timestamp_for_filename, get_now
|
||||
|
||||
import folder_paths
|
||||
import latent_preview
|
||||
import nodes
|
||||
from aiohttp import web
|
||||
from comfy.cli_args import args
|
||||
@ -131,16 +130,6 @@ def error_response(
|
||||
|
||||
|
||||
class ManagerFuncsInComfyUI(core.ManagerFuncs):
|
||||
def get_current_preview_method(self):
|
||||
if args.preview_method == latent_preview.LatentPreviewMethod.Auto:
|
||||
return "auto"
|
||||
elif args.preview_method == latent_preview.LatentPreviewMethod.Latent2RGB:
|
||||
return "latent2rgb"
|
||||
elif args.preview_method == latent_preview.LatentPreviewMethod.TAESD:
|
||||
return "taesd"
|
||||
else:
|
||||
return "none"
|
||||
|
||||
def run_script(self, cmd, cwd="."):
|
||||
if len(cmd) > 0 and cmd[0].startswith("#"):
|
||||
logging.error(f"[ComfyUI-Manager] Unexpected behavior: `{cmd}`")
|
||||
@ -704,8 +693,6 @@ class TaskQueue:
|
||||
cli_args["listen"] = args.listen
|
||||
if hasattr(args, "port"):
|
||||
cli_args["port"] = args.port
|
||||
if hasattr(args, "preview_method"):
|
||||
cli_args["preview_method"] = str(args.preview_method)
|
||||
if hasattr(args, "enable_manager_legacy_ui"):
|
||||
cli_args["enable_manager_legacy_ui"] = args.enable_manager_legacy_ui
|
||||
if hasattr(args, "front_end_version"):
|
||||
@ -819,14 +806,6 @@ class TaskQueue:
|
||||
|
||||
task_queue = TaskQueue()
|
||||
|
||||
# Preview method initialization
|
||||
if args.preview_method == latent_preview.LatentPreviewMethod.NoPreviews:
|
||||
environment_utils.set_preview_method(core.get_config()["preview_method"])
|
||||
else:
|
||||
logging.warning(
|
||||
"[ComfyUI-Manager] Since --preview-method is set, ComfyUI-Manager's preview method feature will be ignored."
|
||||
)
|
||||
|
||||
|
||||
async def task_worker():
|
||||
logging.debug("[ComfyUI-Manager] Task worker started")
|
||||
|
||||
@ -5,8 +5,6 @@ import traceback
|
||||
|
||||
from comfyui_manager.common import context
|
||||
import folder_paths
|
||||
from comfy.cli_args import args
|
||||
import latent_preview
|
||||
|
||||
from comfyui_manager.glob import manager_core as core
|
||||
from comfyui_manager.common import cm_global
|
||||
@ -93,19 +91,6 @@ def print_comfyui_version():
|
||||
)
|
||||
|
||||
|
||||
def set_preview_method(method):
|
||||
if method == "auto":
|
||||
args.preview_method = latent_preview.LatentPreviewMethod.Auto
|
||||
elif method == "latent2rgb":
|
||||
args.preview_method = latent_preview.LatentPreviewMethod.Latent2RGB
|
||||
elif method == "taesd":
|
||||
args.preview_method = latent_preview.LatentPreviewMethod.TAESD
|
||||
else:
|
||||
args.preview_method = latent_preview.LatentPreviewMethod.NoPreviews
|
||||
|
||||
core.get_config()["preview_method"] = method
|
||||
|
||||
|
||||
def set_update_policy(mode):
|
||||
core.get_config()["update_policy"] = mode
|
||||
|
||||
@ -135,7 +120,6 @@ def initialize_environment():
|
||||
# manager_util.comfyui_manager_path, "extension-node-map.json"
|
||||
# )
|
||||
|
||||
set_preview_method(core.get_config()["preview_method"])
|
||||
print_comfyui_version()
|
||||
setup_environment()
|
||||
|
||||
|
||||
227
comfyui_manager/js/comfyui-gui-builder.js
Normal file
227
comfyui_manager/js/comfyui-gui-builder.js
Normal file
@ -0,0 +1,227 @@
|
||||
import { $el } from "../../scripts/ui.js";
|
||||
|
||||
function normalizeContent(content) {
|
||||
const tmp = document.createElement('div');
|
||||
if (typeof content === 'string') {
|
||||
tmp.innerHTML = content;
|
||||
return Array.from(tmp.childNodes);
|
||||
}
|
||||
if (content instanceof Node) {
|
||||
return content;
|
||||
}
|
||||
return content;
|
||||
}
|
||||
|
||||
export function createSettingsCombo(label, content) {
|
||||
const settingItem = $el("div.setting-item", {}, [
|
||||
$el("div.flex.flex-row.items-center.gap-2",[
|
||||
$el("div.form-label.flex.grow.items-center", [
|
||||
$el("span.text-muted", { textContent: label },)
|
||||
]),
|
||||
$el("div.form-input.flex.justify-end",
|
||||
[content]
|
||||
)
|
||||
]
|
||||
)
|
||||
]);
|
||||
return settingItem;
|
||||
}
|
||||
|
||||
export function buildGuiFrame(dialogId, title, iconClass, content, owner) {
|
||||
const dialog_mask = $el("div.p-dialog-mask.p-overlay-mask.p-overlay-mask-enter", {
|
||||
parent: document.body,
|
||||
style: {
|
||||
position: "fixed",
|
||||
height: "100%",
|
||||
width: "100%",
|
||||
left: "0px",
|
||||
top: "0px",
|
||||
display: "flex",
|
||||
justifyContent: "center",
|
||||
alignItems: "center",
|
||||
pointerEvents: "auto",
|
||||
zIndex: "1000"
|
||||
},
|
||||
onclick: (e) => {
|
||||
if (e.target === dialog_mask) {
|
||||
owner.close();
|
||||
}
|
||||
}
|
||||
// data-pc-section="mask"
|
||||
});
|
||||
|
||||
const header_actions = $el("div.p-dialog-header-actions", {
|
||||
// [TODO]
|
||||
// data-pc-section="headeractions"
|
||||
}
|
||||
);
|
||||
|
||||
const close_button = $el("button.p-button.p-component.p-button-icon-only.p-button-secondary.p-button-rounded.p-button-text.p-dialog-close-button", {
|
||||
parent: header_actions,
|
||||
type: "button",
|
||||
ariaLabel: "Close",
|
||||
onclick: () => owner.close(),
|
||||
// "data-pc-name": "pcclosebutton",
|
||||
// "data-p-disabled": "false",
|
||||
// "data-p-severity": "secondary",
|
||||
// "data-pc-group-section": "headericon",
|
||||
// "data-pc-extend": "button",
|
||||
// "data-pc-section": "root",
|
||||
// [FIXME] Not sure how to do most of the SVG using $el
|
||||
innerHTML: '<svg width="14" height="14" viewBox="0 0 14 14" fill="none" xmlns="http://www.w3.org/2000/svg" class="p-icon p-button-icon" aria-hidden="true"><path d="M8.01186 7.00933L12.27 2.75116C12.341 2.68501 12.398 2.60524 12.4375 2.51661C12.4769 2.42798 12.4982 2.3323 12.4999 2.23529C12.5016 2.13827 12.4838 2.0419 12.4474 1.95194C12.4111 1.86197 12.357 1.78024 12.2884 1.71163C12.2198 1.64302 12.138 1.58893 12.0481 1.55259C11.9581 1.51625 11.8617 1.4984 11.7647 1.50011C11.6677 1.50182 11.572 1.52306 11.4834 1.56255C11.3948 1.60204 11.315 1.65898 11.2488 1.72997L6.99067 5.98814L2.7325 1.72997C2.59553 1.60234 2.41437 1.53286 2.22718 1.53616C2.03999 1.53946 1.8614 1.61529 1.72901 1.74767C1.59663 1.88006 1.5208 2.05865 1.5175 2.24584C1.5142 2.43303 1.58368 2.61419 1.71131 2.75116L5.96948 7.00933L1.71131 11.2675C1.576 11.403 1.5 11.5866 1.5 11.7781C1.5 11.9696 1.576 12.1532 1.71131 12.2887C1.84679 12.424 2.03043 12.5 2.2219 12.5C2.41338 12.5 2.59702 12.424 2.7325 12.2887L6.99067 8.03052L11.2488 12.2887C11.3843 12.424 11.568 12.5 11.7594 12.5C11.9509 12.5 12.1346 12.424 12.27 12.2887C12.4053 12.1532 12.4813 11.9696 12.4813 11.7781C12.4813 11.5866 12.4053 11.403 12.27 11.2675L8.01186 7.00933Z" fill="currentColor"></path></svg><span class="p-button-label" data-pc-section="label"> </span><!---->'
|
||||
}
|
||||
);
|
||||
|
||||
const dialog_header = $el("div.p-dialog-header",
|
||||
[
|
||||
$el("div", [
|
||||
$el("div",
|
||||
{
|
||||
id: "frame-title-container",
|
||||
},
|
||||
[
|
||||
$el("h2.px-4", [
|
||||
$el(iconClass, {
|
||||
style: {
|
||||
"font-size": "1.25rem",
|
||||
"margin-right": ".5rem"
|
||||
}
|
||||
}),
|
||||
$el("span", { textContent: title })
|
||||
])
|
||||
]
|
||||
)
|
||||
]),
|
||||
header_actions
|
||||
]
|
||||
);
|
||||
|
||||
const contentFrame = $el("div.p-dialog-content", {}, normalizeContent(content));
|
||||
const manager_dialog = $el("div.p-dialog.p-component.global-dialog", {
|
||||
id: dialogId,
|
||||
parent: dialog_mask,
|
||||
style: {
|
||||
'display': 'flex',
|
||||
'flex-direction': 'column',
|
||||
'pointer-events': 'auto',
|
||||
'margin': '0px',
|
||||
},
|
||||
role: 'dialog',
|
||||
ariaModal: 'true',
|
||||
// [TODO]
|
||||
// ariaLabbelledby: 'cm-title',
|
||||
// maximized: 'false',
|
||||
// data-pc-name: 'dialog',
|
||||
// data-pc-section: 'root',
|
||||
// data-pd-focustrap: 'true'
|
||||
},
|
||||
[ dialog_header, contentFrame ]
|
||||
);
|
||||
|
||||
const hidden_accessible = $el("span.p-hidden-accessible.p-hidden-focusable", {
|
||||
parent: manager_dialog,
|
||||
tabindex: "0",
|
||||
role: "presentation",
|
||||
ariaHidden: "true",
|
||||
"data-p-hidden-accessible": "true",
|
||||
"data-p-hidden-focusable": "true",
|
||||
"data-pc-section": "firstfocusableelement"
|
||||
});
|
||||
|
||||
return dialog_mask;
|
||||
}
|
||||
|
||||
export function buildGuiFrameCustomHeader(dialogId, customHeader, content, owner) {
|
||||
const dialog_mask = $el("div.p-dialog-mask.p-overlay-mask.p-overlay-mask-enter", {
|
||||
parent: document.body,
|
||||
style: {
|
||||
position: "fixed",
|
||||
height: "100%",
|
||||
width: "100%",
|
||||
left: "0px",
|
||||
top: "0px",
|
||||
display: "flex",
|
||||
justifyContent: "center",
|
||||
alignItems: "center",
|
||||
pointerEvents: "auto",
|
||||
zIndex: "1000"
|
||||
},
|
||||
onclick: (e) => {
|
||||
if (e.target === dialog_mask) {
|
||||
owner.close();
|
||||
}
|
||||
}
|
||||
// data-pc-section="mask"
|
||||
});
|
||||
|
||||
const header_actions = $el("div.p-dialog-header-actions", {
|
||||
// [TODO]
|
||||
// data-pc-section="headeractions"
|
||||
}
|
||||
);
|
||||
|
||||
const close_button = $el("button.p-button.p-component.p-button-icon-only.p-button-secondary.p-button-rounded.p-button-text.p-dialog-close-button", {
|
||||
parent: header_actions,
|
||||
type: "button",
|
||||
ariaLabel: "Close",
|
||||
onclick: () => owner.close(),
|
||||
// "data-pc-name": "pcclosebutton",
|
||||
// "data-p-disabled": "false",
|
||||
// "data-p-severity": "secondary",
|
||||
// "data-pc-group-section": "headericon",
|
||||
// "data-pc-extend": "button",
|
||||
// "data-pc-section": "root",
|
||||
// [FIXME] Not sure how to do most of the SVG using $el
|
||||
innerHTML: '<svg width="14" height="14" viewBox="0 0 14 14" fill="none" xmlns="http://www.w3.org/2000/svg" class="p-icon p-button-icon" aria-hidden="true"><path d="M8.01186 7.00933L12.27 2.75116C12.341 2.68501 12.398 2.60524 12.4375 2.51661C12.4769 2.42798 12.4982 2.3323 12.4999 2.23529C12.5016 2.13827 12.4838 2.0419 12.4474 1.95194C12.4111 1.86197 12.357 1.78024 12.2884 1.71163C12.2198 1.64302 12.138 1.58893 12.0481 1.55259C11.9581 1.51625 11.8617 1.4984 11.7647 1.50011C11.6677 1.50182 11.572 1.52306 11.4834 1.56255C11.3948 1.60204 11.315 1.65898 11.2488 1.72997L6.99067 5.98814L2.7325 1.72997C2.59553 1.60234 2.41437 1.53286 2.22718 1.53616C2.03999 1.53946 1.8614 1.61529 1.72901 1.74767C1.59663 1.88006 1.5208 2.05865 1.5175 2.24584C1.5142 2.43303 1.58368 2.61419 1.71131 2.75116L5.96948 7.00933L1.71131 11.2675C1.576 11.403 1.5 11.5866 1.5 11.7781C1.5 11.9696 1.576 12.1532 1.71131 12.2887C1.84679 12.424 2.03043 12.5 2.2219 12.5C2.41338 12.5 2.59702 12.424 2.7325 12.2887L6.99067 8.03052L11.2488 12.2887C11.3843 12.424 11.568 12.5 11.7594 12.5C11.9509 12.5 12.1346 12.424 12.27 12.2887C12.4053 12.1532 12.4813 11.9696 12.4813 11.7781C12.4813 11.5866 12.4053 11.403 12.27 11.2675L8.01186 7.00933Z" fill="currentColor"></path></svg><span class="p-button-label" data-pc-section="label"> </span><!---->'
|
||||
}
|
||||
);
|
||||
|
||||
const _customHeader = normalizeContent(customHeader);
|
||||
const dialog_header = $el("div.p-dialog-header",
|
||||
[
|
||||
$el("div", [
|
||||
$el("div",
|
||||
{
|
||||
id: "frame-title-container",
|
||||
},
|
||||
Array.isArray(_customHeader) ? _customHeader : [_customHeader]
|
||||
)
|
||||
]),
|
||||
header_actions
|
||||
]
|
||||
);
|
||||
|
||||
const contentFrame = $el("div.p-dialog-content", {}, normalizeContent(content));
|
||||
const manager_dialog = $el("div.p-dialog.p-component.global-dialog", {
|
||||
id: dialogId,
|
||||
parent: dialog_mask,
|
||||
style: {
|
||||
'display': 'flex',
|
||||
'flex-direction': 'column',
|
||||
'pointer-events': 'auto',
|
||||
'margin': '0px',
|
||||
},
|
||||
role: 'dialog',
|
||||
ariaModal: 'true',
|
||||
// [TODO]
|
||||
// ariaLabbelledby: 'cm-title',
|
||||
// maximized: 'false',
|
||||
// data-pc-name: 'dialog',
|
||||
// data-pc-section: 'root',
|
||||
// data-pd-focustrap: 'true'
|
||||
},
|
||||
[ dialog_header, contentFrame ]
|
||||
);
|
||||
|
||||
const hidden_accessible = $el("span.p-hidden-accessible.p-hidden-focusable", {
|
||||
parent: manager_dialog,
|
||||
tabindex: "0",
|
||||
role: "presentation",
|
||||
ariaHidden: "true",
|
||||
"data-p-hidden-accessible": "true",
|
||||
"data-p-hidden-focusable": "true",
|
||||
"data-pc-section": "firstfocusableelement"
|
||||
});
|
||||
|
||||
return dialog_mask;
|
||||
}
|
||||
@ -16,10 +16,10 @@ import {
|
||||
rebootAPI, setManagerInstance, show_message, customAlert, customPrompt,
|
||||
infoToast, showTerminal, setNeedRestart, generateUUID
|
||||
} from "./common.js";
|
||||
import { ComponentBuilderDialog, load_components, set_component_policy } from "./components-manager.js";
|
||||
import { CustomNodesManager } from "./custom-nodes-manager.js";
|
||||
import { ModelManager } from "./model-manager.js";
|
||||
import { SnapshotManager } from "./snapshot.js";
|
||||
import { buildGuiFrame, createSettingsCombo } from "./comfyui-gui-builder.js";
|
||||
|
||||
let manager_version = await getVersion();
|
||||
|
||||
@ -44,12 +44,16 @@ docStyle.innerHTML = `
|
||||
|
||||
#cm-manager-dialog {
|
||||
width: 1000px;
|
||||
height: 455px;
|
||||
height: auto;
|
||||
box-sizing: content-box;
|
||||
z-index: 1000;
|
||||
overflow-y: auto;
|
||||
}
|
||||
|
||||
#cm-manager-dialog br {
|
||||
margin-bottom: 1em;
|
||||
}
|
||||
|
||||
.cb-widget {
|
||||
width: 400px;
|
||||
height: 25px;
|
||||
@ -80,6 +84,7 @@ docStyle.innerHTML = `
|
||||
}
|
||||
|
||||
.cm-menu-container {
|
||||
padding : calc(var(--spacing)*2);
|
||||
column-gap: 20px;
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
@ -140,8 +145,8 @@ docStyle.innerHTML = `
|
||||
}
|
||||
|
||||
.cm-notice-board {
|
||||
width: 290px;
|
||||
height: 230px;
|
||||
width: auto;
|
||||
height: 280px;
|
||||
overflow: auto;
|
||||
color: var(--input-text);
|
||||
border: 1px solid var(--descrip-text);
|
||||
@ -237,68 +242,50 @@ var batch_id = null;
|
||||
// copied style from https://github.com/pythongosssss/ComfyUI-Custom-Scripts
|
||||
const style = `
|
||||
#workflowgallery-button {
|
||||
width: 310px;
|
||||
height: 27px;
|
||||
height: 50px;
|
||||
padding: 0px !important;
|
||||
position: relative;
|
||||
overflow: hidden;
|
||||
font-size: 17px !important;
|
||||
}
|
||||
#cm-nodeinfo-button {
|
||||
width: 310px;
|
||||
height: 27px;
|
||||
padding: 0px !important;
|
||||
position: relative;
|
||||
overflow: hidden;
|
||||
font-size: 17px !important;
|
||||
|
||||
}
|
||||
#cm-manual-button {
|
||||
width: 310px;
|
||||
height: 27px;
|
||||
position: relative;
|
||||
overflow: hidden;
|
||||
|
||||
}
|
||||
|
||||
.cm-button {
|
||||
width: 310px;
|
||||
height: 30px;
|
||||
width: auto;
|
||||
position: relative;
|
||||
overflow: hidden;
|
||||
font-size: 17px !important;
|
||||
background-color: var(--comfy-menu-secondary-bg);
|
||||
border-color: var(--border-color);
|
||||
color: var(--input-text);
|
||||
}
|
||||
|
||||
.cm-button:hover {
|
||||
filter: brightness(125%);
|
||||
}
|
||||
|
||||
.cm-button-red {
|
||||
width: 310px;
|
||||
height: 30px;
|
||||
position: relative;
|
||||
overflow: hidden;
|
||||
font-size: 17px !important;
|
||||
background-color: #500000 !important;
|
||||
border-color: #88181b !important;
|
||||
color: white !important;
|
||||
}
|
||||
|
||||
.cm-button-red:hover {
|
||||
background-color: #88181b !important;
|
||||
}
|
||||
|
||||
.cm-button-orange {
|
||||
width: 310px;
|
||||
height: 30px;
|
||||
position: relative;
|
||||
overflow: hidden;
|
||||
font-size: 17px !important;
|
||||
font-weight: bold;
|
||||
background-color: orange !important;
|
||||
color: black !important;
|
||||
}
|
||||
|
||||
.cm-experimental-button {
|
||||
width: 290px;
|
||||
height: 30px;
|
||||
position: relative;
|
||||
overflow: hidden;
|
||||
font-size: 17px !important;
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
.cm-experimental {
|
||||
width: 310px;
|
||||
border: 1px solid #555;
|
||||
border-radius: 5px;
|
||||
padding: 10px;
|
||||
@ -325,8 +312,14 @@ const style = `
|
||||
|
||||
.cm-menu-combo {
|
||||
cursor: pointer;
|
||||
width: 310px;
|
||||
box-sizing: border-box;
|
||||
padding: 0.5em 0.5em;
|
||||
border: 1px solid var(--border-color);
|
||||
border-radius: 6px;
|
||||
background: var(--comfy-menu-secondary-bg);
|
||||
}
|
||||
|
||||
.cm-menu-combo:hover {
|
||||
filter: brightness(125%);
|
||||
}
|
||||
|
||||
.cm-small-button {
|
||||
@ -816,7 +809,7 @@ class ManagerMenuDialog extends ComfyDialog {
|
||||
const isElectron = 'electronAPI' in window;
|
||||
|
||||
update_comfyui_button =
|
||||
$el("button.cm-button", {
|
||||
$el("button.p-button.p-component.cm-button", {
|
||||
type: "button",
|
||||
textContent: "Update ComfyUI",
|
||||
style: {
|
||||
@ -827,7 +820,7 @@ class ManagerMenuDialog extends ComfyDialog {
|
||||
});
|
||||
|
||||
switch_comfyui_button =
|
||||
$el("button.cm-button", {
|
||||
$el("button.p-button.p-component.cm-button", {
|
||||
type: "button",
|
||||
textContent: "Switch ComfyUI",
|
||||
style: {
|
||||
@ -838,7 +831,7 @@ class ManagerMenuDialog extends ComfyDialog {
|
||||
});
|
||||
|
||||
restart_stop_button =
|
||||
$el("button.cm-button-red", {
|
||||
$el("button.p-button.p-component.cm-button-red", {
|
||||
type: "button",
|
||||
textContent: "Restart",
|
||||
onclick: () => restartOrStop()
|
||||
@ -846,7 +839,7 @@ class ManagerMenuDialog extends ComfyDialog {
|
||||
|
||||
if(isElectron) {
|
||||
update_all_button =
|
||||
$el("button.cm-button", {
|
||||
$el("button.p-button.p-component.cm-button", {
|
||||
type: "button",
|
||||
textContent: "Update All Custom Nodes",
|
||||
onclick:
|
||||
@ -855,7 +848,7 @@ class ManagerMenuDialog extends ComfyDialog {
|
||||
}
|
||||
else {
|
||||
update_all_button =
|
||||
$el("button.cm-button", {
|
||||
$el("button.p-button.p-component.cm-button", {
|
||||
type: "button",
|
||||
textContent: "Update All",
|
||||
onclick:
|
||||
@ -865,7 +858,7 @@ class ManagerMenuDialog extends ComfyDialog {
|
||||
|
||||
const res =
|
||||
[
|
||||
$el("button.cm-button", {
|
||||
$el("button.p-button.p-component.cm-button", {
|
||||
type: "button",
|
||||
textContent: "Custom Nodes Manager",
|
||||
onclick:
|
||||
@ -877,7 +870,7 @@ class ManagerMenuDialog extends ComfyDialog {
|
||||
}
|
||||
}),
|
||||
|
||||
$el("button.cm-button", {
|
||||
$el("button.p-button.p-component.cm-button", {
|
||||
type: "button",
|
||||
textContent: "Install Missing Custom Nodes",
|
||||
onclick:
|
||||
@ -889,7 +882,7 @@ class ManagerMenuDialog extends ComfyDialog {
|
||||
}
|
||||
}),
|
||||
|
||||
$el("button.cm-button", {
|
||||
$el("button.p-button.p-component.cm-button", {
|
||||
type: "button",
|
||||
textContent: "Custom Nodes In Workflow",
|
||||
onclick:
|
||||
@ -901,8 +894,8 @@ class ManagerMenuDialog extends ComfyDialog {
|
||||
}
|
||||
}),
|
||||
|
||||
$el("br", {}, []),
|
||||
$el("button.cm-button", {
|
||||
$el("div", {}, []),
|
||||
$el("button.p-button.p-component.cm-button", {
|
||||
type: "button",
|
||||
textContent: "Model Manager",
|
||||
onclick:
|
||||
@ -914,7 +907,7 @@ class ManagerMenuDialog extends ComfyDialog {
|
||||
}
|
||||
}),
|
||||
|
||||
$el("button.cm-button", {
|
||||
$el("button.p-button.p-component.cm-button", {
|
||||
type: "button",
|
||||
textContent: "Install via Git URL",
|
||||
onclick: async () => {
|
||||
@ -926,13 +919,13 @@ class ManagerMenuDialog extends ComfyDialog {
|
||||
}
|
||||
}),
|
||||
|
||||
$el("br", {}, []),
|
||||
$el("div", {}, []),
|
||||
update_all_button,
|
||||
update_comfyui_button,
|
||||
switch_comfyui_button,
|
||||
// fetch_updates_button,
|
||||
|
||||
$el("br", {}, []),
|
||||
$el("div", {}, []),
|
||||
restart_stop_button,
|
||||
];
|
||||
|
||||
@ -945,12 +938,13 @@ class ManagerMenuDialog extends ComfyDialog {
|
||||
let self = this;
|
||||
|
||||
// db mode
|
||||
|
||||
this.datasrc_combo = document.createElement("select");
|
||||
this.datasrc_combo.setAttribute("title", "Configure where to retrieve node/model information. If set to 'local,' the channel is ignored, and if set to 'channel (remote),' it fetches the latest information each time the list is opened.");
|
||||
this.datasrc_combo.className = "cm-menu-combo";
|
||||
this.datasrc_combo.appendChild($el('option', { value: 'cache', text: 'DB: Channel (1day cache)' }, []));
|
||||
this.datasrc_combo.appendChild($el('option', { value: 'local', text: 'DB: Local' }, []));
|
||||
this.datasrc_combo.appendChild($el('option', { value: 'remote', text: 'DB: Channel (remote)' }, []));
|
||||
this.datasrc_combo.className = "cm-menu-combo p-select p-component p-inputwrapper p-inputwrapper-filled ";
|
||||
this.datasrc_combo.appendChild($el('option', { value: 'cache', text: 'Channel (1day cache)' }, []));
|
||||
this.datasrc_combo.appendChild($el('option', { value: 'local', text: 'Local' }, []));
|
||||
this.datasrc_combo.appendChild($el('option', { value: 'remote', text: 'Channel (remote)' }, []));
|
||||
|
||||
api.fetchApi('/v2/manager/db_mode')
|
||||
.then(response => response.text())
|
||||
@ -960,27 +954,12 @@ class ManagerMenuDialog extends ComfyDialog {
|
||||
api.fetchApi(`/v2/manager/db_mode?value=${event.target.value}`);
|
||||
});
|
||||
|
||||
// preview method
|
||||
let preview_combo = document.createElement("select");
|
||||
preview_combo.setAttribute("title", "Configure how latent variables will be decoded during preview in the sampling process.");
|
||||
preview_combo.className = "cm-menu-combo";
|
||||
preview_combo.appendChild($el('option', { value: 'auto', text: 'Preview method: Auto' }, []));
|
||||
preview_combo.appendChild($el('option', { value: 'taesd', text: 'Preview method: TAESD (slow)' }, []));
|
||||
preview_combo.appendChild($el('option', { value: 'latent2rgb', text: 'Preview method: Latent2RGB (fast)' }, []));
|
||||
preview_combo.appendChild($el('option', { value: 'none', text: 'Preview method: None (very fast)' }, []));
|
||||
|
||||
api.fetchApi('/v2/manager/preview_method')
|
||||
.then(response => response.text())
|
||||
.then(data => { preview_combo.value = data; });
|
||||
|
||||
preview_combo.addEventListener('change', function (event) {
|
||||
api.fetchApi(`/v2/manager/preview_method?value=${event.target.value}`);
|
||||
});
|
||||
const dbRetrievalSetttingItem = createSettingsCombo("DB", this.datasrc_combo);
|
||||
|
||||
// channel
|
||||
let channel_combo = document.createElement("select");
|
||||
channel_combo.setAttribute("title", "Configure the channel for retrieving data from the Custom Node list (including missing nodes) or the Model list.");
|
||||
channel_combo.className = "cm-menu-combo";
|
||||
channel_combo.className = "cm-menu-combo p-select p-component p-inputwrapper p-inputwrapper-filled";
|
||||
api.fetchApi('/v2/manager/channel_url_list')
|
||||
.then(response => response.json())
|
||||
.then(async data => {
|
||||
@ -989,7 +968,7 @@ class ManagerMenuDialog extends ComfyDialog {
|
||||
for (let i in urls) {
|
||||
if (urls[i] != '') {
|
||||
let name_url = urls[i].split('::');
|
||||
channel_combo.appendChild($el('option', { value: name_url[0], text: `Channel: ${name_url[0]}` }, []));
|
||||
channel_combo.appendChild($el('option', { value: name_url[0], text: `${name_url[0]}` }, []));
|
||||
}
|
||||
}
|
||||
|
||||
@ -1004,11 +983,13 @@ class ManagerMenuDialog extends ComfyDialog {
|
||||
}
|
||||
});
|
||||
|
||||
const channelSetttingItem = createSettingsCombo("Channel", channel_combo);
|
||||
|
||||
|
||||
// share
|
||||
let share_combo = document.createElement("select");
|
||||
share_combo.setAttribute("title", "Hide the share button in the main menu or set the default action upon clicking it. Additionally, configure the default share site when sharing via the context menu's share button.");
|
||||
share_combo.className = "cm-menu-combo";
|
||||
share_combo.className = "cm-menu-combo p-select p-component p-inputwrapper p-inputwrapper-filled";
|
||||
const share_options = [
|
||||
['none', 'None'],
|
||||
['openart', 'OpenArt AI'],
|
||||
@ -1019,7 +1000,7 @@ class ManagerMenuDialog extends ComfyDialog {
|
||||
['all', 'All'],
|
||||
];
|
||||
for (const option of share_options) {
|
||||
share_combo.appendChild($el('option', { value: option[0], text: `Share: ${option[1]}` }, []));
|
||||
share_combo.appendChild($el('option', { value: option[0], text: `${option[1]}` }, []));
|
||||
}
|
||||
|
||||
api.fetchApi('/v2/manager/share_option')
|
||||
@ -1041,33 +1022,14 @@ class ManagerMenuDialog extends ComfyDialog {
|
||||
}
|
||||
});
|
||||
|
||||
let component_policy_combo = document.createElement("select");
|
||||
component_policy_combo.setAttribute("title", "When loading the workflow, configure which version of the component to use.");
|
||||
component_policy_combo.className = "cm-menu-combo";
|
||||
component_policy_combo.appendChild($el('option', { value: 'workflow', text: 'Component: Use workflow version' }, []));
|
||||
component_policy_combo.appendChild($el('option', { value: 'higher', text: 'Component: Use higher version' }, []));
|
||||
component_policy_combo.appendChild($el('option', { value: 'mine', text: 'Component: Use my version' }, []));
|
||||
api.fetchApi('/v2/manager/policy/component')
|
||||
.then(response => response.text())
|
||||
.then(data => {
|
||||
component_policy_combo.value = data;
|
||||
set_component_policy(data);
|
||||
});
|
||||
|
||||
component_policy_combo.addEventListener('change', function (event) {
|
||||
api.fetchApi(`/v2/manager/policy/component?value=${event.target.value}`);
|
||||
set_component_policy(event.target.value);
|
||||
});
|
||||
const shareSetttingItem = createSettingsCombo("Share", share_combo);
|
||||
|
||||
update_policy_combo = document.createElement("select");
|
||||
|
||||
if(isElectron)
|
||||
update_policy_combo.style.display = 'none';
|
||||
|
||||
update_policy_combo.setAttribute("title", "Sets the policy to be applied when performing an update.");
|
||||
update_policy_combo.className = "cm-menu-combo";
|
||||
update_policy_combo.appendChild($el('option', { value: 'stable-comfyui', text: 'Update: ComfyUI Stable Version' }, []));
|
||||
update_policy_combo.appendChild($el('option', { value: 'nightly-comfyui', text: 'Update: ComfyUI Nightly Version' }, []));
|
||||
update_policy_combo.className = "cm-menu-combo p-select p-component p-inputwrapper p-inputwrapper-filled";
|
||||
update_policy_combo.appendChild($el('option', { value: 'stable-comfyui', text: 'ComfyUI Stable Version' }, []));
|
||||
update_policy_combo.appendChild($el('option', { value: 'nightly-comfyui', text: 'ComfyUI Nightly Version' }, []));
|
||||
api.fetchApi('/v2/manager/policy/update')
|
||||
.then(response => response.text())
|
||||
.then(data => {
|
||||
@ -1078,20 +1040,20 @@ class ManagerMenuDialog extends ComfyDialog {
|
||||
api.fetchApi(`/v2/manager/policy/update?value=${event.target.value}`);
|
||||
});
|
||||
|
||||
return [
|
||||
$el("br", {}, []),
|
||||
this.datasrc_combo,
|
||||
channel_combo,
|
||||
preview_combo,
|
||||
share_combo,
|
||||
component_policy_combo,
|
||||
update_policy_combo,
|
||||
$el("br", {}, []),
|
||||
const updateSetttingItem = createSettingsCombo("Update", update_policy_combo);
|
||||
|
||||
if(isElectron)
|
||||
updateSetttingItem.style.display = 'none';
|
||||
|
||||
$el("br", {}, []),
|
||||
$el("filedset.cm-experimental", {}, [
|
||||
return [
|
||||
dbRetrievalSetttingItem,
|
||||
channelSetttingItem,
|
||||
shareSetttingItem,
|
||||
updateSetttingItem,
|
||||
//[TODO] replace mt-2 with wrapper div with flex column gap
|
||||
$el("filedset.cm-experimental.mt-auto", {}, [
|
||||
$el("legend.cm-experimental-legend", {}, ["EXPERIMENTAL"]),
|
||||
$el("button.cm-experimental-button", {
|
||||
$el("button.p-button.p-component.cm-button.cm-experimental-button", {
|
||||
type: "button",
|
||||
textContent: "Snapshot Manager",
|
||||
onclick:
|
||||
@ -1101,7 +1063,7 @@ class ManagerMenuDialog extends ComfyDialog {
|
||||
SnapshotManager.instance.show();
|
||||
}
|
||||
}),
|
||||
$el("button.cm-experimental-button", {
|
||||
$el("button.p-button.p-component.cm-button.cm-experimental-button.mt-2", {
|
||||
type: "button",
|
||||
textContent: "Install PIP packages",
|
||||
onclick:
|
||||
@ -1119,7 +1081,7 @@ class ManagerMenuDialog extends ComfyDialog {
|
||||
|
||||
createControlsRight() {
|
||||
const elts = [
|
||||
$el("button.cm-button", {
|
||||
$el("button.p-button.p-component.cm-button", {
|
||||
id: 'cm-manual-button',
|
||||
type: "button",
|
||||
textContent: "Community Manual",
|
||||
@ -1170,11 +1132,11 @@ class ManagerMenuDialog extends ComfyDialog {
|
||||
})
|
||||
]),
|
||||
|
||||
$el("button", {
|
||||
$el("button.p-button.p-component.cm-button", {
|
||||
id: 'workflowgallery-button',
|
||||
type: "button",
|
||||
style: {
|
||||
...(localStorage.getItem("wg_last_visited") ? {height: '50px'} : {})
|
||||
// ...(localStorage.getItem("wg_last_visited") ? {height: '50px'} : {})
|
||||
},
|
||||
onclick: (e) => {
|
||||
const last_visited_site = localStorage.getItem("wg_last_visited")
|
||||
@ -1197,7 +1159,7 @@ class ManagerMenuDialog extends ComfyDialog {
|
||||
}, [
|
||||
$el("p", {
|
||||
id: 'workflowgallery-button-last-visited-label',
|
||||
textContent: `(${localStorage.getItem("wg_last_visited") ? localStorage.getItem("wg_last_visited").split('/')[2] : ''})`,
|
||||
textContent: `(${localStorage.getItem("wg_last_visited") ? localStorage.getItem("wg_last_visited").split('/')[2] : 'none selected'})`,
|
||||
style: {
|
||||
'text-align': 'center',
|
||||
'color': 'var(--input-text)',
|
||||
@ -1213,13 +1175,12 @@ class ManagerMenuDialog extends ComfyDialog {
|
||||
})
|
||||
]),
|
||||
|
||||
$el("button.cm-button", {
|
||||
$el("button.p-button.p-component.cm-button", {
|
||||
id: 'cm-nodeinfo-button',
|
||||
type: "button",
|
||||
textContent: "Nodes Info",
|
||||
onclick: () => { window.open("https://ltdrdata.github.io/", "comfyui-node-info"); }
|
||||
}),
|
||||
$el("br", {}, []),
|
||||
];
|
||||
|
||||
var textarea = document.createElement("div");
|
||||
@ -1234,31 +1195,23 @@ class ManagerMenuDialog extends ComfyDialog {
|
||||
constructor() {
|
||||
super();
|
||||
|
||||
const close_button = $el("button", { id: "cm-close-button", type: "button", textContent: "Close", onclick: () => this.close() });
|
||||
const content = $el("div.cm-menu-container",
|
||||
[
|
||||
$el("div.cm-menu-column.gap-2", [...this.createControlsLeft()]),
|
||||
$el("div.cm-menu-column.gap-2", [...this.createControlsMid()]),
|
||||
$el("div.cm-menu-column.gap-2", [...this.createControlsRight()])
|
||||
]
|
||||
);
|
||||
|
||||
const content =
|
||||
$el("div.comfy-modal-content",
|
||||
[
|
||||
$el("tr.cm-title", {}, [
|
||||
$el("font", {size:6, color:"white"}, [`ComfyUI Manager ${manager_version}`])]
|
||||
),
|
||||
$el("br", {}, []),
|
||||
$el("div.cm-menu-container",
|
||||
[
|
||||
$el("div.cm-menu-column", [...this.createControlsLeft()]),
|
||||
$el("div.cm-menu-column", [...this.createControlsMid()]),
|
||||
$el("div.cm-menu-column", [...this.createControlsRight()])
|
||||
]),
|
||||
const frame = buildGuiFrame(
|
||||
'cm-manager-dialog', // dialog id
|
||||
`ComfyUI Manager ${manager_version}`, // dialog title
|
||||
"i.mdi.mdi-puzzle", // dialog icon class to show before title
|
||||
content, // dialog content element
|
||||
this
|
||||
); // send this so we can attach close functions
|
||||
|
||||
$el("br", {}, []),
|
||||
close_button,
|
||||
]
|
||||
);
|
||||
|
||||
content.style.width = '100%';
|
||||
content.style.height = '100%';
|
||||
|
||||
this.element = $el("div.comfy-modal", { id:'cm-manager-dialog', parent: document.body }, [ content ]);
|
||||
this.element = frame;
|
||||
}
|
||||
|
||||
get isVisible() {
|
||||
@ -1266,7 +1219,7 @@ class ManagerMenuDialog extends ComfyDialog {
|
||||
}
|
||||
|
||||
show() {
|
||||
this.element.style.display = "block";
|
||||
this.element.style.display = "flex";
|
||||
}
|
||||
|
||||
toggleVisibility() {
|
||||
@ -1435,14 +1388,6 @@ app.registerExtension({
|
||||
});
|
||||
},
|
||||
async setup() {
|
||||
let orig_clear = app.graph.clear;
|
||||
app.graph.clear = function () {
|
||||
orig_clear.call(app.graph);
|
||||
load_components();
|
||||
};
|
||||
|
||||
load_components();
|
||||
|
||||
const menu = document.querySelector(".comfy-menu");
|
||||
const separator = document.createElement("hr");
|
||||
|
||||
@ -1571,19 +1516,6 @@ app.registerExtension({
|
||||
node.prototype.getExtraMenuOptions = function (_, options) {
|
||||
origGetExtraMenuOptions?.apply?.(this, arguments);
|
||||
|
||||
if (node.category.startsWith('group nodes>')) {
|
||||
options.push({
|
||||
content: "Save As Component",
|
||||
callback: (obj) => {
|
||||
if (!ComponentBuilderDialog.instance) {
|
||||
ComponentBuilderDialog.instance = new ComponentBuilderDialog();
|
||||
}
|
||||
ComponentBuilderDialog.instance.target_node = node;
|
||||
ComponentBuilderDialog.instance.show();
|
||||
}
|
||||
}, null);
|
||||
}
|
||||
|
||||
if (isOutputNode(node)) {
|
||||
const { potential_outputs } = getPotentialOutputsAndOutputNodes([this]);
|
||||
const hasOutput = potential_outputs.length > 0;
|
||||
|
||||
@ -1,812 +0,0 @@
|
||||
import { app } from "../../scripts/app.js";
|
||||
import { api } from "../../scripts/api.js"
|
||||
import { sleep, show_message, customConfirm, customAlert } from "./common.js";
|
||||
import { GroupNodeConfig, GroupNodeHandler } from "../../extensions/core/groupNode.js";
|
||||
import { ComfyDialog, $el } from "../../scripts/ui.js";
|
||||
|
||||
const SEPARATOR = ">"
|
||||
|
||||
let pack_map = {};
|
||||
let rpack_map = {};
|
||||
|
||||
export function getPureName(node) {
|
||||
// group nodes/
|
||||
let category = null;
|
||||
if(node.category) {
|
||||
category = node.category.substring(12);
|
||||
}
|
||||
else {
|
||||
category = node.constructor.category?.substring(12);
|
||||
}
|
||||
if(category) {
|
||||
let purename = node.comfyClass.substring(category.length+1);
|
||||
return purename;
|
||||
}
|
||||
else if(node.comfyClass.startsWith('workflow/') || node.comfyClass.startsWith(`workflow${SEPARATOR}`)) {
|
||||
return node.comfyClass.substring(9);
|
||||
}
|
||||
else {
|
||||
return node.comfyClass;
|
||||
}
|
||||
}
|
||||
|
||||
function isValidVersionString(version) {
|
||||
const versionPattern = /^(\d+)\.(\d+)(\.(\d+))?$/;
|
||||
|
||||
const match = version.match(versionPattern);
|
||||
|
||||
return match !== null &&
|
||||
parseInt(match[1], 10) >= 0 &&
|
||||
parseInt(match[2], 10) >= 0 &&
|
||||
(!match[3] || parseInt(match[4], 10) >= 0);
|
||||
}
|
||||
|
||||
function register_pack_map(name, data) {
|
||||
if(data.packname) {
|
||||
pack_map[data.packname] = name;
|
||||
rpack_map[name] = data;
|
||||
}
|
||||
else {
|
||||
rpack_map[name] = data;
|
||||
}
|
||||
}
|
||||
|
||||
function storeGroupNode(name, data, register=true) {
|
||||
let extra = app.graph.extra;
|
||||
if (!extra) app.graph.extra = extra = {};
|
||||
let groupNodes = extra.groupNodes;
|
||||
if (!groupNodes) extra.groupNodes = groupNodes = {};
|
||||
groupNodes[name] = data;
|
||||
|
||||
if(register) {
|
||||
register_pack_map(name, data);
|
||||
}
|
||||
}
|
||||
|
||||
export async function load_components() {
|
||||
let data = await api.fetchApi('/v2/manager/component/loads', {method: "POST"});
|
||||
let components = await data.json();
|
||||
|
||||
let start_time = Date.now();
|
||||
let failed = [];
|
||||
let failed2 = [];
|
||||
|
||||
for(let name in components) {
|
||||
if(app.graph.extra?.groupNodes?.[name]) {
|
||||
if(data) {
|
||||
let data = components[name];
|
||||
|
||||
let category = data.packname;
|
||||
if(data.category) {
|
||||
category += SEPARATOR + data.category;
|
||||
}
|
||||
if(category == '') {
|
||||
category = 'components';
|
||||
}
|
||||
|
||||
const config = new GroupNodeConfig(name, data);
|
||||
await config.registerType(category);
|
||||
|
||||
register_pack_map(name, data);
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
let nodeData = components[name];
|
||||
|
||||
storeGroupNode(name, nodeData);
|
||||
|
||||
const config = new GroupNodeConfig(name, nodeData);
|
||||
|
||||
while(true) {
|
||||
try {
|
||||
let category = nodeData.packname;
|
||||
if(nodeData.category) {
|
||||
category += SEPARATOR + nodeData.category;
|
||||
}
|
||||
if(category == '') {
|
||||
category = 'components';
|
||||
}
|
||||
|
||||
await config.registerType(category);
|
||||
register_pack_map(name, nodeData);
|
||||
break;
|
||||
}
|
||||
catch {
|
||||
let elapsed_time = Date.now() - start_time;
|
||||
if (elapsed_time > 5000) {
|
||||
failed.push(name);
|
||||
break;
|
||||
} else {
|
||||
await sleep(100);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// fallback1
|
||||
for(let i in failed) {
|
||||
let name = failed[i];
|
||||
|
||||
if(app.graph.extra?.groupNodes?.[name]) {
|
||||
continue;
|
||||
}
|
||||
|
||||
let nodeData = components[name];
|
||||
|
||||
storeGroupNode(name, nodeData);
|
||||
|
||||
const config = new GroupNodeConfig(name, nodeData);
|
||||
while(true) {
|
||||
try {
|
||||
let category = nodeData.packname;
|
||||
if(nodeData.workflow.category) {
|
||||
category += SEPARATOR + nodeData.category;
|
||||
}
|
||||
if(category == '') {
|
||||
category = 'components';
|
||||
}
|
||||
|
||||
await config.registerType(category);
|
||||
register_pack_map(name, nodeData);
|
||||
break;
|
||||
}
|
||||
catch {
|
||||
let elapsed_time = Date.now() - start_time;
|
||||
if (elapsed_time > 10000) {
|
||||
failed2.push(name);
|
||||
break;
|
||||
} else {
|
||||
await sleep(100);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// fallback2
|
||||
for(let name in failed2) {
|
||||
let name = failed2[i];
|
||||
|
||||
let nodeData = components[name];
|
||||
|
||||
storeGroupNode(name, nodeData);
|
||||
|
||||
const config = new GroupNodeConfig(name, nodeData);
|
||||
while(true) {
|
||||
try {
|
||||
let category = nodeData.workflow.packname;
|
||||
if(nodeData.workflow.category) {
|
||||
category += SEPARATOR + nodeData.category;
|
||||
}
|
||||
if(category == '') {
|
||||
category = 'components';
|
||||
}
|
||||
|
||||
await config.registerType(category);
|
||||
register_pack_map(name, nodeData);
|
||||
break;
|
||||
}
|
||||
catch {
|
||||
let elapsed_time = Date.now() - start_time;
|
||||
if (elapsed_time > 30000) {
|
||||
failed.push(name);
|
||||
break;
|
||||
} else {
|
||||
await sleep(100);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function save_as_component(node, version, author, prefix, nodename, packname, category) {
|
||||
let component_name = `${prefix}::${nodename}`;
|
||||
|
||||
let subgraph = app.graph.extra?.groupNodes?.[component_name];
|
||||
if(!subgraph) {
|
||||
subgraph = app.graph.extra?.groupNodes?.[getPureName(node)];
|
||||
}
|
||||
|
||||
subgraph.version = version;
|
||||
subgraph.author = author;
|
||||
subgraph.datetime = Date.now();
|
||||
subgraph.packname = packname;
|
||||
subgraph.category = category;
|
||||
|
||||
let body =
|
||||
{
|
||||
name: component_name,
|
||||
workflow: subgraph
|
||||
};
|
||||
|
||||
pack_map[packname] = component_name;
|
||||
rpack_map[component_name] = subgraph;
|
||||
|
||||
const res = await api.fetchApi('/v2/manager/component/save', {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
|
||||
if(res.status == 200) {
|
||||
storeGroupNode(component_name, subgraph);
|
||||
const config = new GroupNodeConfig(component_name, subgraph);
|
||||
|
||||
let category = body.workflow.packname;
|
||||
if(body.workflow.category) {
|
||||
category += SEPARATOR + body.workflow.category;
|
||||
}
|
||||
if(category == '') {
|
||||
category = 'components';
|
||||
}
|
||||
|
||||
await config.registerType(category);
|
||||
|
||||
let path = await res.text();
|
||||
show_message(`Component '${component_name}' is saved into:\n${path}`);
|
||||
}
|
||||
else
|
||||
show_message(`Failed to save component.`);
|
||||
}
|
||||
|
||||
async function import_component(component_name, component, mode) {
|
||||
if(mode) {
|
||||
let body =
|
||||
{
|
||||
name: component_name,
|
||||
workflow: component
|
||||
};
|
||||
|
||||
const res = await api.fetchApi('/v2/manager/component/save', {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json", },
|
||||
body: JSON.stringify(body)
|
||||
});
|
||||
}
|
||||
|
||||
let category = component.packname;
|
||||
if(component.category) {
|
||||
category += SEPARATOR + component.category;
|
||||
}
|
||||
if(category == '') {
|
||||
category = 'components';
|
||||
}
|
||||
|
||||
storeGroupNode(component_name, component);
|
||||
const config = new GroupNodeConfig(component_name, component);
|
||||
await config.registerType(category);
|
||||
}
|
||||
|
||||
function restore_to_loaded_component(component_name) {
|
||||
if(rpack_map[component_name]) {
|
||||
let component = rpack_map[component_name];
|
||||
storeGroupNode(component_name, component, false);
|
||||
const config = new GroupNodeConfig(component_name, component);
|
||||
config.registerType(component.category);
|
||||
}
|
||||
}
|
||||
|
||||
// Using a timestamp prevents duplicate pastes and ensures the prevention of re-deletion of litegrapheditor_clipboard.
|
||||
let last_paste_timestamp = null;
|
||||
|
||||
function versionCompare(v1, v2) {
|
||||
let ver1;
|
||||
let ver2;
|
||||
if(v1 && v1 != '') {
|
||||
ver1 = v1.split('.');
|
||||
ver1[0] = parseInt(ver1[0]);
|
||||
ver1[1] = parseInt(ver1[1]);
|
||||
if(ver1.length == 2)
|
||||
ver1.push(0);
|
||||
else
|
||||
ver1[2] = parseInt(ver2[2]);
|
||||
}
|
||||
else {
|
||||
ver1 = [0,0,0];
|
||||
}
|
||||
|
||||
if(v2 && v2 != '') {
|
||||
ver2 = v2.split('.');
|
||||
ver2[0] = parseInt(ver2[0]);
|
||||
ver2[1] = parseInt(ver2[1]);
|
||||
if(ver2.length == 2)
|
||||
ver2.push(0);
|
||||
else
|
||||
ver2[2] = parseInt(ver2[2]);
|
||||
}
|
||||
else {
|
||||
ver2 = [0,0,0];
|
||||
}
|
||||
|
||||
if(ver1[0] > ver2[0])
|
||||
return -1;
|
||||
else if(ver1[0] < ver2[0])
|
||||
return 1;
|
||||
|
||||
if(ver1[1] > ver2[1])
|
||||
return -1;
|
||||
else if(ver1[1] < ver2[1])
|
||||
return 1;
|
||||
|
||||
if(ver1[2] > ver2[2])
|
||||
return -1;
|
||||
else if(ver1[2] < ver2[2])
|
||||
return 1;
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
function checkVersion(name, component) {
|
||||
let msg = '';
|
||||
if(rpack_map[name]) {
|
||||
let old_version = rpack_map[name].version;
|
||||
if(!old_version || old_version == '') {
|
||||
msg = ` '${name}' Upgrade (V0.0 -> V${component.version})`;
|
||||
}
|
||||
else {
|
||||
let c = versionCompare(old_version, component.version);
|
||||
if(c < 0) {
|
||||
msg = ` '${name}' Downgrade (V${old_version} -> V${component.version})`;
|
||||
}
|
||||
else if(c > 0) {
|
||||
msg = ` '${name}' Upgrade (V${old_version} -> V${component.version})`;
|
||||
}
|
||||
else {
|
||||
msg = ` '${name}' Same version (V${component.version})`;
|
||||
}
|
||||
}
|
||||
}
|
||||
else {
|
||||
msg = `'${name}' NEW (V${component.version})`;
|
||||
}
|
||||
|
||||
return msg;
|
||||
}
|
||||
|
||||
async function handle_import_components(components) {
|
||||
let msg = 'Components:\n';
|
||||
let cnt = 0;
|
||||
for(let name in components) {
|
||||
let component = components[name];
|
||||
let v = checkVersion(name, component);
|
||||
|
||||
if(cnt < 10) {
|
||||
msg += v + '\n';
|
||||
}
|
||||
else if (cnt == 10) {
|
||||
msg += '...\n';
|
||||
}
|
||||
else {
|
||||
// do nothing
|
||||
}
|
||||
|
||||
cnt++;
|
||||
}
|
||||
|
||||
let last_name = null;
|
||||
msg += '\nWill you load components?\n';
|
||||
const confirmed = await customConfirm(msg);
|
||||
if(confirmed) {
|
||||
const mode = await customConfirm('\nWill you save components?\n(cancel=load without save)');
|
||||
|
||||
for(let name in components) {
|
||||
let component = components[name];
|
||||
import_component(name, component, mode);
|
||||
last_name = name;
|
||||
}
|
||||
|
||||
if(mode) {
|
||||
show_message('Components are saved.');
|
||||
}
|
||||
else {
|
||||
show_message('Components are loaded.');
|
||||
}
|
||||
}
|
||||
|
||||
if(cnt == 1 && last_name) {
|
||||
const node = LiteGraph.createNode(`workflow${SEPARATOR}${last_name}`);
|
||||
node.pos = [app.canvas.graph_mouse[0], app.canvas.graph_mouse[1]];
|
||||
app.canvas.graph.add(node, false);
|
||||
}
|
||||
}
|
||||
|
||||
async function handlePaste(e) {
|
||||
let data = (e.clipboardData || window.clipboardData);
|
||||
const items = data.items;
|
||||
for(const item of items) {
|
||||
if(item.kind == 'string' && item.type == 'text/plain') {
|
||||
data = data.getData("text/plain");
|
||||
try {
|
||||
let json_data = JSON.parse(data);
|
||||
if(json_data.kind == 'ComfyUI Components' && last_paste_timestamp != json_data.timestamp) {
|
||||
last_paste_timestamp = json_data.timestamp;
|
||||
await handle_import_components(json_data.components);
|
||||
|
||||
// disable paste node
|
||||
localStorage.removeItem("litegrapheditor_clipboard", null);
|
||||
}
|
||||
else {
|
||||
console.log('This components are already pasted: ignored');
|
||||
}
|
||||
}
|
||||
catch {
|
||||
// nothing to do
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
document.addEventListener("paste", handlePaste);
|
||||
|
||||
|
||||
export class ComponentBuilderDialog extends ComfyDialog {
|
||||
constructor() {
|
||||
super();
|
||||
}
|
||||
|
||||
clear() {
|
||||
while (this.element.children.length) {
|
||||
this.element.removeChild(this.element.children[0]);
|
||||
}
|
||||
}
|
||||
|
||||
show() {
|
||||
this.invalidateControl();
|
||||
|
||||
this.element.style.display = "block";
|
||||
this.element.style.zIndex = 1099;
|
||||
this.element.style.width = "500px";
|
||||
this.element.style.height = "480px";
|
||||
}
|
||||
|
||||
invalidateControl() {
|
||||
this.clear();
|
||||
|
||||
let self = this;
|
||||
|
||||
const close_button = $el("button", { id: "cm-close-button", type: "button", textContent: "Close", onclick: () => self.close() });
|
||||
this.save_button = $el("button",
|
||||
{ id: "cm-save-button", type: "button", textContent: "Save", onclick: () =>
|
||||
{
|
||||
save_as_component(self.target_node, self.version_string.value.trim(), self.author.value.trim(), self.node_prefix.value.trim(),
|
||||
self.getNodeName(), self.getPackName(), self.category.value.trim());
|
||||
}
|
||||
});
|
||||
|
||||
let default_nodename = getPureName(this.target_node).trim();
|
||||
|
||||
let groupNode = app.graph.extra.groupNodes[default_nodename];
|
||||
let default_packname = groupNode.packname;
|
||||
if(!default_packname) {
|
||||
default_packname = '';
|
||||
}
|
||||
|
||||
let default_category = groupNode.category;
|
||||
if(!default_category) {
|
||||
default_category = '';
|
||||
}
|
||||
|
||||
this.default_ver = groupNode.version;
|
||||
if(!this.default_ver) {
|
||||
this.default_ver = '0.0';
|
||||
}
|
||||
|
||||
let default_author = groupNode.author;
|
||||
if(!default_author) {
|
||||
default_author = '';
|
||||
}
|
||||
|
||||
let delimiterIndex = default_nodename.indexOf('::');
|
||||
let default_prefix = "";
|
||||
if(delimiterIndex != -1) {
|
||||
default_prefix = default_nodename.substring(0, delimiterIndex);
|
||||
default_nodename = default_nodename.substring(delimiterIndex + 2);
|
||||
}
|
||||
|
||||
if(!default_prefix) {
|
||||
this.save_button.disabled = true;
|
||||
}
|
||||
|
||||
this.pack_list = this.createPackListCombo();
|
||||
|
||||
let version_string = this.createLabeledInput('input version (e.g. 1.0)', '*Version : ', this.default_ver);
|
||||
this.version_string = version_string[1];
|
||||
this.version_string.disabled = true;
|
||||
|
||||
let author = this.createLabeledInput('input author (e.g. Dr.Lt.Data)', 'Author : ', default_author);
|
||||
this.author = author[1];
|
||||
|
||||
let node_prefix = this.createLabeledInput('input node prefix (e.g. mypack)', '*Prefix : ', default_prefix);
|
||||
this.node_prefix = node_prefix[1];
|
||||
|
||||
let manual_nodename = this.createLabeledInput('input node name (e.g. MAKE_BASIC_PIPE)', 'Nodename : ', default_nodename);
|
||||
this.manual_nodename = manual_nodename[1];
|
||||
|
||||
let manual_packname = this.createLabeledInput('input pack name (e.g. mypack)', 'Packname : ', default_packname);
|
||||
this.manual_packname = manual_packname[1];
|
||||
|
||||
let category = this.createLabeledInput('input category (e.g. util/pipe)', 'Category : ', default_category);
|
||||
this.category = category[1];
|
||||
|
||||
this.node_label = this.createNodeLabel();
|
||||
|
||||
let author_mode = this.createAuthorModeCheck();
|
||||
this.author_mode = author_mode[0];
|
||||
|
||||
const content =
|
||||
$el("div.comfy-modal-content",
|
||||
[
|
||||
$el("tr.cm-title", {}, [
|
||||
$el("font", {size:6, color:"white"}, [`ComfyUI-Manager: Component Builder`])]
|
||||
),
|
||||
$el("br", {}, []),
|
||||
$el("div.cm-menu-container",
|
||||
[
|
||||
author_mode[0],
|
||||
author_mode[1],
|
||||
category[0],
|
||||
author[0],
|
||||
node_prefix[0],
|
||||
manual_nodename[0],
|
||||
manual_packname[0],
|
||||
version_string[0],
|
||||
this.pack_list,
|
||||
$el("br", {}, []),
|
||||
this.node_label
|
||||
]),
|
||||
|
||||
$el("br", {}, []),
|
||||
this.save_button,
|
||||
close_button,
|
||||
]
|
||||
);
|
||||
|
||||
content.style.width = '100%';
|
||||
content.style.height = '100%';
|
||||
|
||||
this.element = $el("div.comfy-modal", { id:'cm-manager-dialog', parent: document.body }, [ content ]);
|
||||
}
|
||||
|
||||
validateInput() {
|
||||
let msg = "";
|
||||
|
||||
if(!isValidVersionString(this.version_string.value)) {
|
||||
msg += 'Invalid version string: '+event.value+"\n";
|
||||
}
|
||||
|
||||
if(this.node_prefix.value.trim() == '') {
|
||||
msg += 'Node prefix cannot be empty\n';
|
||||
}
|
||||
|
||||
if(this.manual_nodename.value.trim() == '') {
|
||||
msg += 'Node name cannot be empty\n';
|
||||
}
|
||||
|
||||
if(msg != '') {
|
||||
// alert(msg);
|
||||
}
|
||||
|
||||
this.save_button.disabled = msg != "";
|
||||
}
|
||||
|
||||
getPackName() {
|
||||
if(this.pack_list.selectedIndex == 0) {
|
||||
return this.manual_packname.value.trim();
|
||||
}
|
||||
|
||||
return this.pack_list.value.trim();
|
||||
}
|
||||
|
||||
getNodeName() {
|
||||
if(this.manual_nodename.value.trim() != '') {
|
||||
return this.manual_nodename.value.trim();
|
||||
}
|
||||
|
||||
return getPureName(this.target_node);
|
||||
}
|
||||
|
||||
createAuthorModeCheck() {
|
||||
let check = $el("input",{type:'checkbox', id:"author-mode"},[])
|
||||
const check_label = $el("label",{for:"author-mode"},["Enable author mode"]);
|
||||
check_label.style.color = "var(--fg-color)";
|
||||
check_label.style.cursor = "pointer";
|
||||
check.checked = false;
|
||||
|
||||
let self = this;
|
||||
check.onchange = () => {
|
||||
self.version_string.disabled = !check.checked;
|
||||
|
||||
if(!check.checked) {
|
||||
self.version_string.value = self.default_ver;
|
||||
}
|
||||
else {
|
||||
customAlert('If you are not the author, it is not recommended to change the version, as it may cause component update issues.');
|
||||
}
|
||||
};
|
||||
|
||||
return [check, check_label];
|
||||
}
|
||||
|
||||
createNodeLabel() {
|
||||
let label = $el('p');
|
||||
label.className = 'cb-node-label';
|
||||
if(this.target_node.comfyClass.includes('::'))
|
||||
label.textContent = getPureName(this.target_node);
|
||||
else
|
||||
label.textContent = " _::" + getPureName(this.target_node);
|
||||
return label;
|
||||
}
|
||||
|
||||
createLabeledInput(placeholder, label, value) {
|
||||
let textbox = $el('input.cb-widget-input', {type:'text', placeholder:placeholder, value:value}, []);
|
||||
|
||||
let self = this;
|
||||
textbox.onchange = () => {
|
||||
this.validateInput.call(self);
|
||||
this.node_label.textContent = this.node_prefix.value + "::" + this.manual_nodename.value;
|
||||
}
|
||||
let row = $el('span.cb-widget', {}, [ $el('span.cb-widget-input-label', label), textbox]);
|
||||
|
||||
return [row, textbox];
|
||||
}
|
||||
|
||||
createPackListCombo() {
|
||||
let combo = document.createElement("select");
|
||||
combo.className = "cb-widget";
|
||||
let default_packname_option = { value: '##manual', text: 'Packname: Manual' };
|
||||
|
||||
combo.appendChild($el('option', default_packname_option, []));
|
||||
for(let name in pack_map) {
|
||||
combo.appendChild($el('option', { value: name, text: 'Packname: '+ name }, []));
|
||||
}
|
||||
|
||||
let self = this;
|
||||
combo.onchange = function () {
|
||||
if(combo.selectedIndex == 0) {
|
||||
self.manual_packname.disabled = false;
|
||||
}
|
||||
else {
|
||||
self.manual_packname.disabled = true;
|
||||
}
|
||||
};
|
||||
|
||||
return combo;
|
||||
}
|
||||
}
|
||||
|
||||
let orig_handleFile = app.handleFile;
|
||||
|
||||
async function handleFile(file) {
|
||||
if (file.name?.endsWith(".json") || file.name?.endsWith(".pack")) {
|
||||
const reader = new FileReader();
|
||||
reader.onload = async () => {
|
||||
let is_component = false;
|
||||
const jsonContent = JSON.parse(reader.result);
|
||||
for(let name in jsonContent) {
|
||||
let cand = jsonContent[name];
|
||||
is_component = cand.datetime && cand.version;
|
||||
break;
|
||||
}
|
||||
|
||||
if(is_component) {
|
||||
await handle_import_components(jsonContent);
|
||||
}
|
||||
else {
|
||||
orig_handleFile.call(app, file);
|
||||
}
|
||||
};
|
||||
reader.readAsText(file);
|
||||
|
||||
return;
|
||||
}
|
||||
|
||||
orig_handleFile.call(app, file);
|
||||
}
|
||||
|
||||
app.handleFile = handleFile;
|
||||
|
||||
let current_component_policy = 'workflow';
|
||||
try {
|
||||
api.fetchApi('/v2/manager/policy/component')
|
||||
.then(response => response.text())
|
||||
.then(data => { current_component_policy = data; });
|
||||
}
|
||||
catch {}
|
||||
|
||||
function getChangedVersion(groupNodes) {
|
||||
if(!Object.keys(pack_map).length || !groupNodes)
|
||||
return null;
|
||||
|
||||
let res = {};
|
||||
for(let component_name in groupNodes) {
|
||||
let data = groupNodes[component_name];
|
||||
|
||||
if(rpack_map[component_name]) {
|
||||
let v = versionCompare(data.version, rpack_map[component_name].version);
|
||||
res[component_name] = v;
|
||||
}
|
||||
}
|
||||
|
||||
return res;
|
||||
}
|
||||
|
||||
const loadGraphData = app.loadGraphData;
|
||||
app.loadGraphData = async function () {
|
||||
if(arguments.length == 0)
|
||||
return await loadGraphData.apply(this, arguments);
|
||||
|
||||
let graphData = arguments[0];
|
||||
let groupNodes = graphData.extra?.groupNodes;
|
||||
let res = getChangedVersion(groupNodes);
|
||||
|
||||
if(res) {
|
||||
let target_components = null;
|
||||
switch(current_component_policy) {
|
||||
case 'higher':
|
||||
target_components = Object.keys(res).filter(key => res[key] == 1);
|
||||
break;
|
||||
|
||||
case 'mine':
|
||||
target_components = Object.keys(res);
|
||||
break;
|
||||
|
||||
default:
|
||||
// do nothing
|
||||
}
|
||||
|
||||
if(target_components) {
|
||||
for(let i in target_components) {
|
||||
let component_name = target_components[i];
|
||||
let component = rpack_map[component_name];
|
||||
if(component && graphData.extra?.groupNodes) {
|
||||
graphData.extra.groupNodes[component_name] = component;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
else {
|
||||
console.log('Empty components: policy ignored');
|
||||
}
|
||||
|
||||
arguments[0] = graphData;
|
||||
return await loadGraphData.apply(this, arguments);
|
||||
};
|
||||
|
||||
export function set_component_policy(v) {
|
||||
current_component_policy = v;
|
||||
}
|
||||
|
||||
let graphToPrompt = app.graphToPrompt;
|
||||
app.graphToPrompt = async function () {
|
||||
let p = await graphToPrompt.call(app);
|
||||
try {
|
||||
let groupNodes = p.workflow.extra?.groupNodes;
|
||||
if(groupNodes) {
|
||||
p.workflow.extra = { ... p.workflow.extra};
|
||||
|
||||
// get used group nodes
|
||||
let used_group_nodes = new Set();
|
||||
for(let node of p.workflow.nodes) {
|
||||
if(node.type.startsWith(`workflow/`) || node.type.startsWith(`workflow${SEPARATOR}`)) {
|
||||
used_group_nodes.add(node.type.substring(9));
|
||||
}
|
||||
}
|
||||
|
||||
// remove unused group nodes
|
||||
let new_groupNodes = {};
|
||||
for (let key in p.workflow.extra.groupNodes) {
|
||||
if (used_group_nodes.has(key)) {
|
||||
new_groupNodes[key] = p.workflow.extra.groupNodes[key];
|
||||
}
|
||||
}
|
||||
p.workflow.extra.groupNodes = new_groupNodes;
|
||||
}
|
||||
}
|
||||
catch(e) {
|
||||
console.log(`Failed to filtering group nodes: ${e}`);
|
||||
}
|
||||
|
||||
return p;
|
||||
}
|
||||
@ -1,8 +1,9 @@
|
||||
.cn-manager {
|
||||
--grid-font: -apple-system, BlinkMacSystemFont, "Segoe UI", "Noto Sans", Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji";
|
||||
z-index: 1099;
|
||||
width: 80%;
|
||||
height: 80%;
|
||||
width: 80vw;
|
||||
height: 75vh;
|
||||
min-height: 30em;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 10px;
|
||||
@ -10,6 +11,7 @@
|
||||
font-family: arial, sans-serif;
|
||||
text-underline-offset: 3px;
|
||||
outline: none;
|
||||
margin: calc(var(--spacing)*2);
|
||||
}
|
||||
|
||||
.cn-manager .cn-flex-auto {
|
||||
@ -17,17 +19,21 @@
|
||||
}
|
||||
|
||||
.cn-manager button {
|
||||
width: auto;
|
||||
position: relative;
|
||||
overflow: hidden;
|
||||
font-size: 16px;
|
||||
color: var(--input-text);
|
||||
background-color: var(--comfy-input-bg);
|
||||
border-radius: 8px;
|
||||
border-color: var(--border-color);
|
||||
border-style: solid;
|
||||
margin: 0;
|
||||
padding: 4px 8px;
|
||||
min-width: 100px;
|
||||
}
|
||||
|
||||
.cn-manager button:hover {
|
||||
filter: brightness(125%);
|
||||
}
|
||||
|
||||
.cn-manager button:disabled,
|
||||
.cn-manager input:disabled,
|
||||
.cn-manager select:disabled {
|
||||
@ -40,8 +46,13 @@
|
||||
|
||||
.cn-manager .cn-manager-restart {
|
||||
display: none;
|
||||
background-color: #500000;
|
||||
color: white;
|
||||
background-color: #500000 !important;
|
||||
border-color: #88181b !important;
|
||||
color: white !important;
|
||||
}
|
||||
|
||||
.cn-manager .cn-manager-restart:hover {
|
||||
background-color: #88181b !important;
|
||||
}
|
||||
|
||||
.cn-manager .cn-manager-stop {
|
||||
@ -79,7 +90,6 @@
|
||||
flex-wrap: wrap;
|
||||
gap: 5px;
|
||||
align-items: center;
|
||||
padding: 0 5px;
|
||||
}
|
||||
|
||||
.cn-manager-header label {
|
||||
@ -91,16 +101,32 @@
|
||||
.cn-manager-filter {
|
||||
height: 28px;
|
||||
line-height: 28px;
|
||||
|
||||
cursor: pointer;
|
||||
padding: 0.5em 0.5em;
|
||||
border: 1px solid var(--border-color);
|
||||
border-radius: 6px;
|
||||
background: var(--comfy-input-bg);
|
||||
}
|
||||
|
||||
.cn-manager-filter:hover {
|
||||
filter: brightness(125%);
|
||||
}
|
||||
|
||||
.cn-manager-keywords {
|
||||
height: 28px;
|
||||
line-height: 28px;
|
||||
padding: 0 5px 0 26px;
|
||||
background: var(--comfy-input-bg);
|
||||
background-size: 16px;
|
||||
background-position: 5px center;
|
||||
background-repeat: no-repeat;
|
||||
background-image: url("data:image/svg+xml;charset=utf8,%3Csvg%20viewBox%3D%220%200%2024%2024%22%20width%3D%22100%25%22%20height%3D%22100%25%22%20pointer-events%3D%22none%22%20xmlns%3D%22http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%22%3E%3Cpath%20fill%3D%22none%22%20stroke%3D%22%23888%22%20stroke-linecap%3D%22round%22%20stroke-linejoin%3D%22round%22%20stroke-width%3D%222%22%20d%3D%22m21%2021-4.486-4.494M19%2010.5a8.5%208.5%200%201%201-17%200%208.5%208.5%200%200%201%2017%200%22%2F%3E%3C%2Fsvg%3E");
|
||||
|
||||
border: 1px solid var(--border-color);
|
||||
border-radius: 6px;
|
||||
|
||||
outline-color: transparent;
|
||||
}
|
||||
|
||||
.cn-manager-status {
|
||||
@ -588,6 +614,10 @@
|
||||
height: 100%;
|
||||
}
|
||||
|
||||
.cn-install-buttons button {
|
||||
padding: 4px 8px;
|
||||
}
|
||||
|
||||
.cn-selected-buttons {
|
||||
display: flex;
|
||||
gap: 5px;
|
||||
|
||||
@ -1,6 +1,7 @@
|
||||
import { app } from "../../scripts/app.js";
|
||||
import { ComfyDialog, $el } from "../../scripts/ui.js";
|
||||
import { api } from "../../scripts/api.js";
|
||||
import { buildGuiFrameCustomHeader, createSettingsCombo } from "./comfyui-gui-builder.js";
|
||||
|
||||
import {
|
||||
manager_instance, rebootAPI, install_via_git_url,
|
||||
@ -18,32 +19,19 @@ loadCss("./custom-nodes-manager.css");
|
||||
const gridId = "node";
|
||||
|
||||
const pageHtml = `
|
||||
<div class="cn-manager-header">
|
||||
<label>Filter
|
||||
<select class="cn-manager-filter"></select>
|
||||
</label>
|
||||
<input class="cn-manager-keywords" type="search" placeholder="Search" />
|
||||
<div class="cn-manager-status"></div>
|
||||
<div class="cn-flex-auto"></div>
|
||||
<div class="cn-manager-channel"></div>
|
||||
</div>
|
||||
<div class="cn-manager-grid"></div>
|
||||
<div class="cn-manager-selection"></div>
|
||||
<div class="cn-manager-message"></div>
|
||||
<div class="cn-manager-footer">
|
||||
<button class="cn-manager-back">
|
||||
<svg class="arrow-icon" width="14" height="14" viewBox="0 0 16 16" fill="none" xmlns="http://www.w3.org/2000/svg">
|
||||
<path d="M2 8H18M2 8L8 2M2 8L8 14" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"/>
|
||||
</svg>
|
||||
Back
|
||||
</button>
|
||||
<button class="cn-manager-restart">Restart</button>
|
||||
<button class="cn-manager-stop">Stop</button>
|
||||
<div class="cn-flex-auto"></div>
|
||||
<button class="cn-manager-used-in-workflow">Used In Workflow</button>
|
||||
<button class="cn-manager-check-update">Check Update</button>
|
||||
<button class="cn-manager-check-missing">Check Missing</button>
|
||||
<button class="cn-manager-install-url">Install via Git URL</button>
|
||||
<div class="cn-manager cn-manager-dark">
|
||||
<div class="cn-manager-grid"></div>
|
||||
<div class="cn-manager-selection"></div>
|
||||
<div class="cn-manager-message"></div>
|
||||
<div class="cn-manager-footer">
|
||||
<button class="cn-manager-restart p-button p-component">Restart</button>
|
||||
<button class="cn-manager-stop p-button p-component">Stop</button>
|
||||
<div class="cn-flex-auto"></div>
|
||||
<button class="cn-manager-used-in-workflow p-button p-component">Used In Workflow</button>
|
||||
<button class="cn-manager-check-update p-button p-component">Check Update</button>
|
||||
<button class="cn-manager-check-missing p-button p-component">Check Missing</button>
|
||||
<button class="cn-manager-install-url p-button p-component">Install via Git URL</button>
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
|
||||
@ -89,11 +77,26 @@ export class CustomNodesManager {
|
||||
}
|
||||
|
||||
init() {
|
||||
this.element = $el("div", {
|
||||
parent: document.body,
|
||||
className: "comfy-modal cn-manager"
|
||||
});
|
||||
this.element.innerHTML = pageHtml;
|
||||
const header = $el("div.cn-manager-header.px-2", {}, [
|
||||
// $el("label", {}, [
|
||||
// $el("span", { textContent: "Filter" }),
|
||||
// $el("select.cn-manager-filter")
|
||||
// ]),
|
||||
createSettingsCombo("Filter", $el("select.cn-manager-filter")),
|
||||
$el("input.cn-manager-keywords.p-inputtext.p-component", { type: "search", placeholder: "Search" }),
|
||||
$el("div.cn-manager-status"),
|
||||
$el("div.cn-flex-auto"),
|
||||
$el("div.cn-manager-channel")
|
||||
]);
|
||||
|
||||
const frame = buildGuiFrameCustomHeader(
|
||||
'cn-manager-dialog', // dialog id
|
||||
header, // custom header element
|
||||
pageHtml, // dialog content element
|
||||
this
|
||||
); // send this so we can attach close functions
|
||||
|
||||
this.element = frame;
|
||||
this.element.setAttribute("tabindex", 0);
|
||||
this.element.focus();
|
||||
|
||||
@ -372,7 +375,7 @@ export class CustomNodesManager {
|
||||
|
||||
return list.map(id => {
|
||||
const bt = buttons[id];
|
||||
return `<button class="cn-btn-${id}" group="${action}" mode="${bt.mode}">${bt.label}</button>`;
|
||||
return `<button class="cn-btn-${id} p-button p-component" group="${action}" mode="${bt.mode}">${bt.label}</button>`;
|
||||
}).join("");
|
||||
}
|
||||
|
||||
@ -655,7 +658,6 @@ export class CustomNodesManager {
|
||||
}
|
||||
|
||||
renderGrid() {
|
||||
|
||||
// update theme
|
||||
const globalStyle = window.getComputedStyle(document.body);
|
||||
this.colorVars = {
|
||||
|
||||
@ -1,13 +1,15 @@
|
||||
.cmm-manager {
|
||||
--grid-font: -apple-system, BlinkMacSystemFont, "Segoe UI", "Noto Sans", Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji";
|
||||
z-index: 1099;
|
||||
width: 80%;
|
||||
height: 80%;
|
||||
width: 80vw;
|
||||
height: 75vh;
|
||||
min-height: 30em;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 10px;
|
||||
color: var(--fg-color);
|
||||
font-family: arial, sans-serif;
|
||||
margin: calc(var(--spacing)*2);
|
||||
}
|
||||
|
||||
.cmm-manager .cmm-flex-auto {
|
||||
@ -18,14 +20,15 @@
|
||||
font-size: 16px;
|
||||
color: var(--input-text);
|
||||
background-color: var(--comfy-input-bg);
|
||||
border-radius: 8px;
|
||||
border-color: var(--border-color);
|
||||
border-style: solid;
|
||||
margin: 0;
|
||||
padding: 4px 8px;
|
||||
min-width: 100px;
|
||||
}
|
||||
|
||||
.cmm-manager button:hover {
|
||||
filter: brightness(125%);
|
||||
}
|
||||
|
||||
.cmm-manager button:disabled,
|
||||
.cmm-manager input:disabled,
|
||||
.cmm-manager select:disabled {
|
||||
@ -38,7 +41,7 @@
|
||||
|
||||
.cmm-manager .cmm-manager-refresh {
|
||||
display: none;
|
||||
background-color: #000080;
|
||||
background-color: #000080 !important;
|
||||
color: white;
|
||||
}
|
||||
|
||||
@ -53,7 +56,6 @@
|
||||
flex-wrap: wrap;
|
||||
gap: 5px;
|
||||
align-items: center;
|
||||
padding: 0 5px;
|
||||
}
|
||||
|
||||
.cmm-manager-header label {
|
||||
@ -67,16 +69,34 @@
|
||||
.cmm-manager-filter {
|
||||
height: 28px;
|
||||
line-height: 28px;
|
||||
|
||||
cursor: pointer;
|
||||
padding: 0.5em 0.5em;
|
||||
border: 1px solid var(--border-color);
|
||||
border-radius: 6px;
|
||||
background: var(--comfy-input-bg);
|
||||
}
|
||||
|
||||
.cmm-manager-type:hover,
|
||||
.cmm-manager-base:hover,
|
||||
.cmm-manager-filter:hover {
|
||||
filter: brightness(125%);
|
||||
}
|
||||
|
||||
.cmm-manager-keywords {
|
||||
height: 28px;
|
||||
line-height: 28px;
|
||||
padding: 0 5px 0 26px;
|
||||
background: var(--comfy-input-bg);
|
||||
background-size: 16px;
|
||||
background-position: 5px center;
|
||||
background-repeat: no-repeat;
|
||||
background-image: url("data:image/svg+xml;charset=utf8,%3Csvg%20viewBox%3D%220%200%2024%2024%22%20width%3D%22100%25%22%20height%3D%22100%25%22%20pointer-events%3D%22none%22%20xmlns%3D%22http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%22%3E%3Cpath%20fill%3D%22none%22%20stroke%3D%22%23888%22%20stroke-linecap%3D%22round%22%20stroke-linejoin%3D%22round%22%20stroke-width%3D%222%22%20d%3D%22m21%2021-4.486-4.494M19%2010.5a8.5%208.5%200%201%201-17%200%208.5%208.5%200%200%201%2017%200%22%2F%3E%3C%2Fsvg%3E");
|
||||
|
||||
border: 1px solid var(--border-color);
|
||||
border-radius: 6px;
|
||||
|
||||
outline-color: transparent;
|
||||
}
|
||||
|
||||
.cmm-manager-status {
|
||||
@ -148,6 +168,10 @@
|
||||
color: white;
|
||||
}
|
||||
|
||||
.cmm-btn-install {
|
||||
padding: 4px 8px;
|
||||
}
|
||||
|
||||
.cmm-btn-download {
|
||||
width: 18px;
|
||||
height: 18px;
|
||||
|
||||
@ -9,39 +9,22 @@ import { api } from "../../scripts/api.js";
|
||||
|
||||
// https://cenfun.github.io/turbogrid/api.html
|
||||
import TG from "./turbogrid.esm.js";
|
||||
import { buildGuiFrameCustomHeader, createSettingsCombo } from "./comfyui-gui-builder.js";
|
||||
|
||||
loadCss("./model-manager.css");
|
||||
|
||||
const gridId = "model";
|
||||
|
||||
const pageHtml = `
|
||||
<div class="cmm-manager-header">
|
||||
<label>Filter
|
||||
<select class="cmm-manager-filter"></select>
|
||||
</label>
|
||||
<label>Type
|
||||
<select class="cmm-manager-type"></select>
|
||||
</label>
|
||||
<label>Base
|
||||
<select class="cmm-manager-base"></select>
|
||||
</label>
|
||||
<input class="cmm-manager-keywords" type="search" placeholder="Search" />
|
||||
<div class="cmm-manager-status"></div>
|
||||
<div class="cmm-flex-auto"></div>
|
||||
</div>
|
||||
<div class="cmm-manager-grid"></div>
|
||||
<div class="cmm-manager-selection"></div>
|
||||
<div class="cmm-manager-message"></div>
|
||||
<div class="cmm-manager-footer">
|
||||
<button class="cmm-manager-back">
|
||||
<svg class="arrow-icon" width="14" height="14" viewBox="0 0 16 16" fill="none" xmlns="http://www.w3.org/2000/svg">
|
||||
<path d="M2 8H18M2 8L8 2M2 8L8 14" stroke="white" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"/>
|
||||
</svg>
|
||||
Back
|
||||
</button>
|
||||
<button class="cmm-manager-refresh">Refresh</button>
|
||||
<button class="cmm-manager-stop">Stop</button>
|
||||
<div class="cmm-flex-auto"></div>
|
||||
<div class="cmm-manager cmm-manager-dark">
|
||||
<div class="cmm-manager-grid"></div>
|
||||
<div class="cmm-manager-selection"></div>
|
||||
<div class="cmm-manager-message"></div>
|
||||
<div class="cmm-manager-footer">
|
||||
<button class="cmm-manager-refresh p-button p-component">Refresh</button>
|
||||
<button class="cmm-manager-stop p-button p-component">Stop</button>
|
||||
<div class="cmm-flex-auto"></div>
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
|
||||
@ -64,11 +47,23 @@ export class ModelManager {
|
||||
}
|
||||
|
||||
init() {
|
||||
this.element = $el("div", {
|
||||
parent: document.body,
|
||||
className: "comfy-modal cmm-manager"
|
||||
});
|
||||
this.element.innerHTML = pageHtml;
|
||||
const header = $el("div.cmm-manager-header", {}, [
|
||||
createSettingsCombo("Filter", $el("select.cmm-manager-filter")),
|
||||
createSettingsCombo("Type", $el("select.cmm-manager-type")),
|
||||
createSettingsCombo("Base", $el("select.cmm-manager-base")),
|
||||
$el("input.cmm-manager-keywords.p-inputtext.p-component", { type: "search", placeholder: "Search" }),
|
||||
$el("div.cmm-manager-status"),
|
||||
$el("div.cmm-flex-auto")
|
||||
]);
|
||||
|
||||
const frame = buildGuiFrameCustomHeader(
|
||||
'cmm-manager-dialog', // dialog id
|
||||
header, // custom header element
|
||||
pageHtml, // dialog content element
|
||||
this
|
||||
); // send this so we can attach close functions
|
||||
|
||||
this.element = frame;
|
||||
this.initFilter();
|
||||
this.bindEvents();
|
||||
this.initGrid();
|
||||
@ -347,7 +342,7 @@ export class ModelManager {
|
||||
if (installed === "True") {
|
||||
return `<div class="cmm-icon-passed">${icons.passed}</div>`;
|
||||
}
|
||||
return `<button class="cmm-btn-install" mode="install">Install</button>`;
|
||||
return `<button class="cmm-btn-install p-button p-component" mode="install">Install</button>`;
|
||||
}
|
||||
}, {
|
||||
id: 'url',
|
||||
@ -420,7 +415,7 @@ export class ModelManager {
|
||||
}
|
||||
|
||||
this.selectedModels = selectedList;
|
||||
this.showSelection(`<span>Selected <b>${selectedList.length}</b> models <button class="cmm-btn-install" mode="install">Install</button>`);
|
||||
this.showSelection(`<span>Selected <b>${selectedList.length}</b> models <button class="cmm-btn-install p-button p-component" mode="install">Install</button>`);
|
||||
}
|
||||
|
||||
focusInstall(item) {
|
||||
|
||||
65
comfyui_manager/js/snapshot.css
Normal file
65
comfyui_manager/js/snapshot.css
Normal file
@ -0,0 +1,65 @@
|
||||
.snapshot-manager {
|
||||
--grid-font: -apple-system, BlinkMacSystemFont, "Segoe UI", "Noto Sans", Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji";
|
||||
z-index: 1099;
|
||||
width: 80vw;
|
||||
height: 75vh;
|
||||
min-height: 30em;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 10px;
|
||||
color: var(--fg-color);
|
||||
font-family: arial, sans-serif;
|
||||
text-underline-offset: 3px;
|
||||
outline: none;
|
||||
margin: calc(var(--spacing)*2);
|
||||
}
|
||||
|
||||
.snapshot-manager button {
|
||||
width: auto;
|
||||
position: relative;
|
||||
overflow: hidden;
|
||||
font-size: 16px;
|
||||
color: var(--input-text);
|
||||
background-color: var(--comfy-input-bg);
|
||||
border-color: var(--border-color);
|
||||
margin: 0;
|
||||
min-width: 100px;
|
||||
padding: 4px 8px;
|
||||
}
|
||||
|
||||
.snapshot-manager .snapshot-restore-btn {
|
||||
background-color: #00158f !important;
|
||||
border-color: #2025b9 !important;
|
||||
color: white !important;
|
||||
}
|
||||
|
||||
.snapshot-manager .snapshot-remove-btn {
|
||||
background-color: #970000 !important;
|
||||
border-color: #be2127 !important;
|
||||
color: white !important;
|
||||
}
|
||||
|
||||
.snapshot-manager button:hover {
|
||||
filter: brightness(125%);
|
||||
}
|
||||
|
||||
.snapshot-manager .data-btns {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: calc(var(--spacing)*2);
|
||||
padding: calc(var(--spacing)*2);
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
height: 100%;
|
||||
}
|
||||
|
||||
.snapshot-footer {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
gap: 10px;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.snapshot-manager .cn-flex-auto {
|
||||
flex: auto;
|
||||
}
|
||||
@ -1,8 +1,10 @@
|
||||
import { app } from "../../scripts/app.js";
|
||||
import { api } from "../../scripts/api.js"
|
||||
import { ComfyDialog, $el } from "../../scripts/ui.js";
|
||||
import { manager_instance, rebootAPI, show_message } from "./common.js";
|
||||
import { manager_instance, rebootAPI, show_message, loadCss } from "./common.js";
|
||||
import { buildGuiFrame } from "./comfyui-gui-builder.js";
|
||||
|
||||
loadCss("./snapshot.css");
|
||||
|
||||
async function restore_snapshot(target) {
|
||||
if(SnapshotManager.instance) {
|
||||
@ -27,7 +29,7 @@ async function restore_snapshot(target) {
|
||||
}
|
||||
finally {
|
||||
await SnapshotManager.instance.invalidateControl();
|
||||
SnapshotManager.instance.updateMessage("<BR>To apply the snapshot, please <button id='cm-reboot-button2' class='cm-small-button'>RESTART</button> ComfyUI. And refresh browser.", 'cm-reboot-button2');
|
||||
SnapshotManager.instance.updateMessage("<BR>To apply the snapshot, please <button id='cm-reboot-button2' class='p-button p-component'>RESTART</button> ComfyUI. And refresh browser.", 'cm-reboot-button2');
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -88,6 +90,8 @@ export class SnapshotManager extends ComfyDialog {
|
||||
message_box = null;
|
||||
data = null;
|
||||
|
||||
content = $el("div.snapshot-manager");
|
||||
|
||||
clear() {
|
||||
this.restore_buttons = [];
|
||||
this.message_box = null;
|
||||
@ -96,9 +100,18 @@ export class SnapshotManager extends ComfyDialog {
|
||||
|
||||
constructor(app, manager_dialog) {
|
||||
super();
|
||||
this.manager_dialog = manager_dialog;
|
||||
// this.manager_dialog = manager_dialog;
|
||||
this.search_keyword = '';
|
||||
this.element = $el("div.comfy-modal", { parent: document.body }, []);
|
||||
|
||||
const frame = buildGuiFrame(
|
||||
'snapshot-manager-dialog', // dialog id
|
||||
'Snapshot Manager', // title
|
||||
'i.mdi.mdi-puzzle', // icon class
|
||||
this.content, // dialog content element
|
||||
this
|
||||
); // send this so we can attach close functions
|
||||
|
||||
this.element = frame;
|
||||
}
|
||||
|
||||
async remove_item() {
|
||||
@ -109,7 +122,7 @@ export class SnapshotManager extends ComfyDialog {
|
||||
|
||||
createControls() {
|
||||
return [
|
||||
$el("button.cm-small-button", {
|
||||
$el("button.p-button.p-component", {
|
||||
type: "button",
|
||||
textContent: "Close",
|
||||
onclick: () => { this.close(); }
|
||||
@ -132,8 +145,8 @@ export class SnapshotManager extends ComfyDialog {
|
||||
this.clear();
|
||||
this.data = (await getSnapshotList()).items;
|
||||
|
||||
while (this.element.children.length) {
|
||||
this.element.removeChild(this.element.children[0]);
|
||||
while (this.content.children.length) {
|
||||
this.content.removeChild(this.content.children[0]);
|
||||
}
|
||||
|
||||
await this.createGrid();
|
||||
@ -204,20 +217,21 @@ export class SnapshotManager extends ComfyDialog {
|
||||
data2.innerHTML = ` ${data}`;
|
||||
var data_button = document.createElement('td');
|
||||
data_button.style.textAlign = "center";
|
||||
data_button.className = "data-btns";
|
||||
|
||||
var restoreBtn = document.createElement('button');
|
||||
restoreBtn.className = "snapshot-restore-btn p-button p-component";
|
||||
restoreBtn.innerHTML = 'Restore';
|
||||
restoreBtn.style.width = "100px";
|
||||
restoreBtn.style.backgroundColor = 'blue';
|
||||
|
||||
restoreBtn.addEventListener('click', function() {
|
||||
restore_snapshot(data);
|
||||
});
|
||||
|
||||
var removeBtn = document.createElement('button');
|
||||
removeBtn.className = "snapshot-remove-btn p-button p-component";
|
||||
removeBtn.innerHTML = 'Remove';
|
||||
removeBtn.style.width = "100px";
|
||||
removeBtn.style.backgroundColor = 'red';
|
||||
|
||||
removeBtn.addEventListener('click', function() {
|
||||
remove_snapshot(data);
|
||||
@ -241,13 +255,14 @@ export class SnapshotManager extends ComfyDialog {
|
||||
let self = this;
|
||||
const panel = document.createElement('div');
|
||||
panel.style.width = "100%";
|
||||
panel.style.height = "100%";
|
||||
panel.appendChild(grid);
|
||||
|
||||
function handleResize() {
|
||||
const parentHeight = self.element.clientHeight;
|
||||
const gridHeight = parentHeight - 200;
|
||||
|
||||
grid.style.height = gridHeight + "px";
|
||||
// grid.style.height = gridHeight + "px";
|
||||
}
|
||||
window.addEventListener("resize", handleResize);
|
||||
|
||||
@ -256,25 +271,17 @@ export class SnapshotManager extends ComfyDialog {
|
||||
grid.style.width = "100%";
|
||||
grid.style.height = "100%";
|
||||
grid.style.overflowY = "scroll";
|
||||
this.element.style.height = "85%";
|
||||
this.element.style.width = "80%";
|
||||
this.element.appendChild(panel);
|
||||
|
||||
this.content.appendChild(panel);
|
||||
|
||||
handleResize();
|
||||
}
|
||||
|
||||
async createBottomControls() {
|
||||
var close_button = document.createElement("button");
|
||||
close_button.className = "cm-small-button";
|
||||
close_button.innerHTML = "Close";
|
||||
close_button.onclick = () => { this.close(); }
|
||||
close_button.style.display = "inline-block";
|
||||
|
||||
var save_button = document.createElement("button");
|
||||
save_button.className = "cm-small-button";
|
||||
save_button.className = "p-button p-component";
|
||||
save_button.innerHTML = "Save snapshot";
|
||||
save_button.onclick = () => { save_current_snapshot(); }
|
||||
save_button.style.display = "inline-block";
|
||||
save_button.style.horizontalAlign = "right";
|
||||
save_button.style.width = "170px";
|
||||
|
||||
@ -282,15 +289,19 @@ export class SnapshotManager extends ComfyDialog {
|
||||
this.message_box.style.height = '60px';
|
||||
this.message_box.style.verticalAlign = 'middle';
|
||||
|
||||
this.element.appendChild(this.message_box);
|
||||
this.element.appendChild(close_button);
|
||||
this.element.appendChild(save_button);
|
||||
const footer = $el("div.snapshot-footer");
|
||||
const spacer = $el("div.cn-flex-auto");
|
||||
footer.appendChild(spacer);
|
||||
footer.appendChild(save_button);
|
||||
|
||||
this.content.appendChild(this.message_box);
|
||||
this.content.appendChild(footer);
|
||||
}
|
||||
|
||||
async show() {
|
||||
try {
|
||||
this.invalidateControl();
|
||||
this.element.style.display = "block";
|
||||
this.element.style.display = "flex";
|
||||
this.element.style.zIndex = 1099;
|
||||
}
|
||||
catch(exception) {
|
||||
|
||||
@ -1577,9 +1577,6 @@ class ManagerFuncs:
|
||||
def __init__(self):
|
||||
pass
|
||||
|
||||
def get_current_preview_method(self):
|
||||
return "none"
|
||||
|
||||
def run_script(self, cmd, cwd='.'):
|
||||
if len(cmd) > 0 and cmd[0].startswith("#"):
|
||||
print(f"[ComfyUI-Manager] Unexpected behavior: `{cmd}`")
|
||||
@ -1597,14 +1594,12 @@ def write_config():
|
||||
config = configparser.ConfigParser(strict=False)
|
||||
|
||||
config['default'] = {
|
||||
'preview_method': manager_funcs.get_current_preview_method(),
|
||||
'git_exe': get_config()['git_exe'],
|
||||
'use_uv': get_config()['use_uv'],
|
||||
'channel_url': get_config()['channel_url'],
|
||||
'share_option': get_config()['share_option'],
|
||||
'bypass_ssl': get_config()['bypass_ssl'],
|
||||
"file_logging": get_config()['file_logging'],
|
||||
'component_policy': get_config()['component_policy'],
|
||||
'update_policy': get_config()['update_policy'],
|
||||
'windows_selector_event_loop_policy': get_config()['windows_selector_event_loop_policy'],
|
||||
'model_download_by_agent': get_config()['model_download_by_agent'],
|
||||
@ -1637,7 +1632,6 @@ def read_config():
|
||||
|
||||
return {
|
||||
'http_channel_enabled': get_bool('http_channel_enabled', False),
|
||||
'preview_method': default_conf.get('preview_method', manager_funcs.get_current_preview_method()).lower(),
|
||||
'git_exe': default_conf.get('git_exe', ''),
|
||||
'use_uv': get_bool('use_uv', True),
|
||||
'channel_url': default_conf.get('channel_url', DEFAULT_CHANNEL),
|
||||
@ -1645,7 +1639,6 @@ def read_config():
|
||||
'share_option': default_conf.get('share_option', 'all').lower(),
|
||||
'bypass_ssl': get_bool('bypass_ssl', False),
|
||||
'file_logging': get_bool('file_logging', True),
|
||||
'component_policy': default_conf.get('component_policy', 'workflow').lower(),
|
||||
'update_policy': default_conf.get('update_policy', 'stable-comfyui').lower(),
|
||||
'windows_selector_event_loop_policy': get_bool('windows_selector_event_loop_policy', False),
|
||||
'model_download_by_agent': get_bool('model_download_by_agent', False),
|
||||
@ -1662,7 +1655,6 @@ def read_config():
|
||||
|
||||
return {
|
||||
'http_channel_enabled': False,
|
||||
'preview_method': manager_funcs.get_current_preview_method(),
|
||||
'git_exe': '',
|
||||
'use_uv': True,
|
||||
'channel_url': DEFAULT_CHANNEL,
|
||||
@ -1670,7 +1662,6 @@ def read_config():
|
||||
'share_option': 'all',
|
||||
'bypass_ssl': manager_util.bypass_ssl,
|
||||
'file_logging': True,
|
||||
'component_policy': 'workflow',
|
||||
'update_policy': 'stable-comfyui',
|
||||
'windows_selector_event_loop_policy': False,
|
||||
'model_download_by_agent': False,
|
||||
|
||||
@ -61,7 +61,6 @@ def handle_stream(stream, prefix):
|
||||
|
||||
|
||||
from comfy.cli_args import args
|
||||
import latent_preview
|
||||
|
||||
def is_loopback(address):
|
||||
import ipaddress
|
||||
@ -146,16 +145,6 @@ async def get_risky_level(files, pip_packages):
|
||||
|
||||
|
||||
class ManagerFuncsInComfyUI(core.ManagerFuncs):
|
||||
def get_current_preview_method(self):
|
||||
if args.preview_method == latent_preview.LatentPreviewMethod.Auto:
|
||||
return "auto"
|
||||
elif args.preview_method == latent_preview.LatentPreviewMethod.Latent2RGB:
|
||||
return "latent2rgb"
|
||||
elif args.preview_method == latent_preview.LatentPreviewMethod.TAESD:
|
||||
return "taesd"
|
||||
else:
|
||||
return "none"
|
||||
|
||||
def run_script(self, cmd, cwd='.'):
|
||||
if len(cmd) > 0 and cmd[0].startswith("#"):
|
||||
logging.error(f"[ComfyUI-Manager] Unexpected behavior: `{cmd}`")
|
||||
@ -188,25 +177,6 @@ local_db_custom_node_list = os.path.join(manager_util.comfyui_manager_path, "cus
|
||||
local_db_extension_node_mappings = os.path.join(manager_util.comfyui_manager_path, "extension-node-map.json")
|
||||
|
||||
|
||||
def set_preview_method(method):
|
||||
if method == 'auto':
|
||||
args.preview_method = latent_preview.LatentPreviewMethod.Auto
|
||||
elif method == 'latent2rgb':
|
||||
args.preview_method = latent_preview.LatentPreviewMethod.Latent2RGB
|
||||
elif method == 'taesd':
|
||||
args.preview_method = latent_preview.LatentPreviewMethod.TAESD
|
||||
else:
|
||||
args.preview_method = latent_preview.LatentPreviewMethod.NoPreviews
|
||||
|
||||
core.get_config()['preview_method'] = method
|
||||
|
||||
|
||||
set_preview_method(core.get_config()['preview_method'])
|
||||
|
||||
|
||||
def set_component_policy(mode):
|
||||
core.get_config()['component_policy'] = mode
|
||||
|
||||
def set_update_policy(mode):
|
||||
core.get_config()['update_policy'] = mode
|
||||
|
||||
@ -1725,17 +1695,6 @@ async def _install_model(json_data):
|
||||
return web.Response(status=200)
|
||||
|
||||
|
||||
@routes.get("/v2/manager/preview_method")
|
||||
async def preview_method(request):
|
||||
if "value" in request.rel_url.query:
|
||||
set_preview_method(request.rel_url.query['value'])
|
||||
core.write_config()
|
||||
else:
|
||||
return web.Response(text=core.manager_funcs.get_current_preview_method(), status=200)
|
||||
|
||||
return web.Response(status=200)
|
||||
|
||||
|
||||
@routes.get("/v2/manager/db_mode")
|
||||
async def db_mode(request):
|
||||
if "value" in request.rel_url.query:
|
||||
@ -1747,18 +1706,6 @@ async def db_mode(request):
|
||||
return web.Response(status=200)
|
||||
|
||||
|
||||
|
||||
@routes.get("/v2/manager/policy/component")
|
||||
async def component_policy(request):
|
||||
if "value" in request.rel_url.query:
|
||||
set_component_policy(request.rel_url.query['value'])
|
||||
core.write_config()
|
||||
else:
|
||||
return web.Response(text=core.get_config()['component_policy'], status=200)
|
||||
|
||||
return web.Response(status=200)
|
||||
|
||||
|
||||
@routes.get("/v2/manager/policy/update")
|
||||
async def update_policy(request):
|
||||
if "value" in request.rel_url.query:
|
||||
@ -1897,61 +1844,6 @@ def restart(self):
|
||||
return os.execv(sys.executable, cmds)
|
||||
|
||||
|
||||
@routes.post("/v2/manager/component/save")
|
||||
async def save_component(request):
|
||||
try:
|
||||
data = await request.json()
|
||||
name = data['name']
|
||||
workflow = data['workflow']
|
||||
|
||||
if not os.path.exists(context.manager_components_path):
|
||||
os.mkdir(context.manager_components_path)
|
||||
|
||||
if 'packname' in workflow and workflow['packname'] != '':
|
||||
sanitized_name = manager_util.sanitize_filename(workflow['packname']) + '.pack'
|
||||
else:
|
||||
sanitized_name = manager_util.sanitize_filename(name) + '.json'
|
||||
|
||||
filepath = os.path.join(context.manager_components_path, sanitized_name)
|
||||
components = {}
|
||||
if os.path.exists(filepath):
|
||||
with open(filepath) as f:
|
||||
components = json.load(f)
|
||||
|
||||
components[name] = workflow
|
||||
|
||||
with open(filepath, 'w') as f:
|
||||
json.dump(components, f, indent=4, sort_keys=True)
|
||||
return web.Response(text=filepath, status=200)
|
||||
except Exception:
|
||||
return web.Response(status=400)
|
||||
|
||||
|
||||
@routes.post("/v2/manager/component/loads")
|
||||
async def load_components(request):
|
||||
if os.path.exists(context.manager_components_path):
|
||||
try:
|
||||
json_files = [f for f in os.listdir(context.manager_components_path) if f.endswith('.json')]
|
||||
pack_files = [f for f in os.listdir(context.manager_components_path) if f.endswith('.pack')]
|
||||
|
||||
components = {}
|
||||
for json_file in json_files + pack_files:
|
||||
file_path = os.path.join(context.manager_components_path, json_file)
|
||||
with open(file_path, 'r') as file:
|
||||
try:
|
||||
# When there is a conflict between the .pack and the .json, the pack takes precedence and overrides.
|
||||
components.update(json.load(file))
|
||||
except json.JSONDecodeError as e:
|
||||
logging.error(f"[ComfyUI-Manager] Error decoding component file in file {json_file}: {e}")
|
||||
|
||||
return web.json_response(components)
|
||||
except Exception as e:
|
||||
logging.error(f"[ComfyUI-Manager] failed to load components\n{e}")
|
||||
return web.Response(status=400)
|
||||
else:
|
||||
return web.json_response({})
|
||||
|
||||
|
||||
@routes.get("/v2/manager/version")
|
||||
async def get_version(request):
|
||||
return web.Response(text=core.version_str, status=200)
|
||||
|
||||
@ -5,7 +5,7 @@ build-backend = "setuptools.build_meta"
|
||||
[project]
|
||||
name = "comfyui-manager"
|
||||
license = { text = "GPL-3.0-only" }
|
||||
version = "4.0.3b6"
|
||||
version = "4.0.3b7"
|
||||
requires-python = ">= 3.9"
|
||||
description = "ComfyUI-Manager provides features to install and manage custom nodes for ComfyUI, as well as various functionalities to assist with ComfyUI."
|
||||
readme = "README.md"
|
||||
|
||||
34
tests/.gitignore
vendored
Normal file
34
tests/.gitignore
vendored
Normal file
@ -0,0 +1,34 @@
|
||||
# Test environment and artifacts
|
||||
|
||||
# Virtual environment
|
||||
test_venv/
|
||||
venv/
|
||||
env/
|
||||
|
||||
# pytest cache
|
||||
.pytest_cache/
|
||||
__pycache__/
|
||||
*.pyc
|
||||
*.pyo
|
||||
|
||||
# Coverage reports (module-specific naming)
|
||||
.coverage
|
||||
.coverage.*
|
||||
htmlcov*/
|
||||
coverage*.xml
|
||||
*.cover
|
||||
|
||||
# Test artifacts
|
||||
.tox/
|
||||
.hypothesis/
|
||||
|
||||
# IDE
|
||||
.vscode/
|
||||
.idea/
|
||||
*.swp
|
||||
*.swo
|
||||
*~
|
||||
|
||||
# OS
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
181
tests/README.md
Normal file
181
tests/README.md
Normal file
@ -0,0 +1,181 @@
|
||||
# ComfyUI Manager Test Suite
|
||||
|
||||
This directory contains all tests for the ComfyUI Manager project, organized by module structure.
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
tests/
|
||||
├── setup_test_env.sh # Setup isolated test environment
|
||||
├── requirements.txt # Test dependencies
|
||||
├── pytest.ini # Global pytest configuration
|
||||
├── .gitignore # Ignore test artifacts
|
||||
│
|
||||
└── common/ # Tests for comfyui_manager/common/
|
||||
└── pip_util/ # Tests for pip_util.py
|
||||
├── README.md # pip_util test documentation
|
||||
├── conftest.py # pip_util test fixtures
|
||||
├── pytest.ini # pip_util-specific pytest config
|
||||
└── test_*.py # Actual test files (to be created)
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Setup Test Environment (One Time)
|
||||
|
||||
```bash
|
||||
cd tests
|
||||
./setup_test_env.sh
|
||||
```
|
||||
|
||||
This creates an isolated virtual environment with all test dependencies.
|
||||
|
||||
### 2. Run Tests
|
||||
|
||||
```bash
|
||||
# Activate test environment
|
||||
source test_venv/bin/activate
|
||||
|
||||
# Run all tests from root
|
||||
cd tests
|
||||
pytest
|
||||
|
||||
# Run specific module tests
|
||||
cd tests/common/pip_util
|
||||
pytest
|
||||
|
||||
# Deactivate when done
|
||||
deactivate
|
||||
```
|
||||
|
||||
## Test Organization
|
||||
|
||||
Tests mirror the source code structure:
|
||||
|
||||
| Source Code | Test Location |
|
||||
|-------------|---------------|
|
||||
| `comfyui_manager/common/pip_util.py` | `tests/common/pip_util/test_*.py` |
|
||||
| `comfyui_manager/common/other.py` | `tests/common/other/test_*.py` |
|
||||
| `comfyui_manager/module/file.py` | `tests/module/file/test_*.py` |
|
||||
|
||||
## Writing Tests
|
||||
|
||||
1. Create test directory matching source structure
|
||||
2. Add `conftest.py` for module-specific fixtures
|
||||
3. Add `pytest.ini` for module-specific configuration (optional)
|
||||
4. Create `test_*.py` files with actual tests
|
||||
5. Document in module-specific README
|
||||
|
||||
## Test Categories
|
||||
|
||||
Use pytest markers to categorize tests:
|
||||
|
||||
```python
|
||||
@pytest.mark.unit
|
||||
def test_simple_function():
|
||||
pass
|
||||
|
||||
@pytest.mark.integration
|
||||
def test_complex_workflow():
|
||||
pass
|
||||
|
||||
@pytest.mark.e2e
|
||||
def test_full_system():
|
||||
pass
|
||||
```
|
||||
|
||||
Run by category:
|
||||
```bash
|
||||
pytest -m unit # Only unit tests
|
||||
pytest -m integration # Only integration tests
|
||||
pytest -m e2e # Only end-to-end tests
|
||||
```
|
||||
|
||||
## Coverage Reports
|
||||
|
||||
Coverage reports are generated per module:
|
||||
|
||||
```bash
|
||||
cd tests/common/pip_util
|
||||
pytest # Generates htmlcov_pip_util/ and coverage_pip_util.xml
|
||||
```
|
||||
|
||||
## Environment Isolation
|
||||
|
||||
**Why use venv?**
|
||||
- ✅ Prevents test dependencies from corrupting main environment
|
||||
- ✅ Allows safe package installation/uninstallation during tests
|
||||
- ✅ Consistent test results across machines
|
||||
- ✅ Easy to recreate clean environment
|
||||
|
||||
## Available Test Modules
|
||||
|
||||
- **[common/pip_util](common/pip_util/)** - Policy-based pip package management system tests
|
||||
- Unit tests for policy loading, parsing, condition evaluation
|
||||
- Integration tests for policy application (60% of tests)
|
||||
- End-to-end workflow tests
|
||||
|
||||
## Adding New Test Modules
|
||||
|
||||
1. Create directory structure: `tests/module_path/component_name/`
|
||||
2. Add `conftest.py` with fixtures
|
||||
3. Add `pytest.ini` if needed (optional)
|
||||
4. Add `README.md` documenting the tests
|
||||
5. Create `test_*.py` files
|
||||
|
||||
Example:
|
||||
```bash
|
||||
mkdir -p tests/data_models/config
|
||||
cd tests/data_models/config
|
||||
touch conftest.py README.md test_config_loader.py
|
||||
```
|
||||
|
||||
## CI/CD Integration
|
||||
|
||||
Tests are designed to run in CI/CD pipelines:
|
||||
|
||||
```yaml
|
||||
# Example GitHub Actions
|
||||
- name: Setup test environment
|
||||
run: |
|
||||
cd tests
|
||||
./setup_test_env.sh
|
||||
|
||||
- name: Run tests
|
||||
run: |
|
||||
source tests/test_venv/bin/activate
|
||||
pytest tests/
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Import errors
|
||||
```bash
|
||||
# Make sure venv is activated
|
||||
source test_venv/bin/activate
|
||||
|
||||
# Verify Python path
|
||||
python -c "import sys; print(sys.path)"
|
||||
```
|
||||
|
||||
### Tests not discovered
|
||||
```bash
|
||||
# Check pytest configuration
|
||||
pytest --collect-only
|
||||
|
||||
# Verify test file naming (must start with test_)
|
||||
ls test_*.py
|
||||
```
|
||||
|
||||
### Clean rebuild
|
||||
```bash
|
||||
# Remove and recreate test environment
|
||||
rm -rf test_venv/
|
||||
./setup_test_env.sh
|
||||
```
|
||||
|
||||
## Resources
|
||||
|
||||
- **pytest Documentation**: https://docs.pytest.org/
|
||||
- **Coverage.py**: https://coverage.readthedocs.io/
|
||||
- **Module-specific READMEs**: Check each test module directory
|
||||
423
tests/common/pip_util/CONTEXT_FILES_GUIDE.md
Normal file
423
tests/common/pip_util/CONTEXT_FILES_GUIDE.md
Normal file
@ -0,0 +1,423 @@
|
||||
# Context Files Guide for pip_util Tests
|
||||
|
||||
Quick reference for all context files created for extending pip_util tests.
|
||||
|
||||
---
|
||||
|
||||
## 📋 File Overview
|
||||
|
||||
| File | Purpose | When to Use |
|
||||
|------|---------|-------------|
|
||||
| **DEPENDENCY_TREE_CONTEXT.md** | Complete dependency trees with version analysis | Adding new test packages or updating scenarios |
|
||||
| **DEPENDENCY_ANALYSIS.md** | Analysis methodology and findings | Understanding why packages were chosen |
|
||||
| **TEST_SCENARIOS.md** | Detailed test specifications | Writing new tests or understanding existing ones |
|
||||
| **analyze_dependencies.py** | Interactive dependency analyzer | Exploring new packages before adding tests |
|
||||
| **requirements-test-base.txt** | Base test environment packages | Setting up or modifying test environment |
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Common Tasks
|
||||
|
||||
### Task 1: Adding a New Test Package
|
||||
|
||||
**Steps**:
|
||||
|
||||
1. **Analyze the package**:
|
||||
```bash
|
||||
python analyze_dependencies.py NEW_PACKAGE
|
||||
```
|
||||
|
||||
2. **Check size and dependencies**:
|
||||
```bash
|
||||
./test_venv/bin/pip download --no-deps NEW_PACKAGE
|
||||
ls -lh NEW_PACKAGE*.whl # Check size
|
||||
```
|
||||
|
||||
3. **Verify dependency tree**:
|
||||
- Open **DEPENDENCY_TREE_CONTEXT.md**
|
||||
- Follow "Adding New Test Scenarios" section
|
||||
- Document findings in the file
|
||||
|
||||
4. **Update requirements** (if pre-installation needed):
|
||||
- Add to `requirements-test-base.txt`
|
||||
- Run `./setup_test_env.sh` to recreate venv
|
||||
|
||||
5. **Write test**:
|
||||
- Follow patterns in `test_dependency_protection.py`
|
||||
- Use `reset_test_venv` fixture
|
||||
- Add scenario to **TEST_SCENARIOS.md**
|
||||
|
||||
6. **Verify**:
|
||||
```bash
|
||||
pytest test_YOUR_NEW_TEST.py -v --override-ini="addopts="
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Understanding Existing Tests
|
||||
|
||||
**Steps**:
|
||||
|
||||
1. **Read test scenario**:
|
||||
- Open **TEST_SCENARIOS.md**
|
||||
- Find your scenario (1-6)
|
||||
- Review initial state, action, expected result
|
||||
|
||||
2. **Check dependency details**:
|
||||
- Open **DEPENDENCY_TREE_CONTEXT.md**
|
||||
- Look up package in table of contents
|
||||
- Review dependency tree and version analysis
|
||||
|
||||
3. **Run analysis**:
|
||||
```bash
|
||||
python analyze_dependencies.py PACKAGE
|
||||
```
|
||||
|
||||
4. **Examine test code**:
|
||||
- Open relevant test file
|
||||
- Check policy fixture
|
||||
- Review assertions
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Updating for New Package Versions
|
||||
|
||||
**When**: PyPI releases major version updates (e.g., urllib3 3.0)
|
||||
|
||||
**Steps**:
|
||||
|
||||
1. **Check current environment**:
|
||||
```bash
|
||||
python analyze_dependencies.py --env
|
||||
```
|
||||
|
||||
2. **Analyze new versions**:
|
||||
```bash
|
||||
./test_venv/bin/pip index versions PACKAGE | head -20
|
||||
python analyze_dependencies.py PACKAGE
|
||||
```
|
||||
|
||||
3. **Update context files**:
|
||||
- Update version numbers in **DEPENDENCY_TREE_CONTEXT.md**
|
||||
- Update "Version Analysis" section
|
||||
- Document breaking changes
|
||||
|
||||
4. **Test with new versions**:
|
||||
- Update `requirements-test-base.txt` (if testing new base version)
|
||||
- OR update test to verify protection from new version
|
||||
- Run tests to verify behavior
|
||||
|
||||
5. **Update scenarios**:
|
||||
- Update **TEST_SCENARIOS.md** with new version numbers
|
||||
- Update expected results if behavior changed
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Debugging Dependency Issues
|
||||
|
||||
**Problem**: Test fails with unexpected dependency versions
|
||||
|
||||
**Steps**:
|
||||
|
||||
1. **Check what's installed**:
|
||||
```bash
|
||||
./test_venv/bin/pip freeze | grep -E "(urllib3|certifi|six|requests)"
|
||||
```
|
||||
|
||||
2. **Analyze what would install**:
|
||||
```bash
|
||||
python analyze_dependencies.py PACKAGE
|
||||
```
|
||||
|
||||
3. **Compare with expected**:
|
||||
- Open **DEPENDENCY_TREE_CONTEXT.md**
|
||||
- Check "Install Scenarios" for the package
|
||||
- Compare actual vs. expected
|
||||
|
||||
4. **Check for PyPI changes**:
|
||||
```bash
|
||||
./test_venv/bin/pip index versions PACKAGE
|
||||
```
|
||||
|
||||
5. **Verify test environment**:
|
||||
```bash
|
||||
rm -rf test_venv && ./setup_test_env.sh
|
||||
pytest test_FILE.py -v --override-ini="addopts="
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📚 Context File Details
|
||||
|
||||
### DEPENDENCY_TREE_CONTEXT.md
|
||||
|
||||
**Contents**:
|
||||
- Current test environment snapshot
|
||||
- Complete dependency trees for all test packages
|
||||
- Version analysis (current vs. latest)
|
||||
- Upgrade scenarios matrix
|
||||
- Guidelines for adding new scenarios
|
||||
- Quick reference tables
|
||||
|
||||
**Use when**:
|
||||
- Adding new test package
|
||||
- Understanding why a package was chosen
|
||||
- Checking version compatibility
|
||||
- Updating for new PyPI releases
|
||||
|
||||
**Key sections**:
|
||||
- Package Dependency Trees → See what each package depends on
|
||||
- Version Analysis → Understand version gaps and breaking changes
|
||||
- Adding New Test Scenarios → Step-by-step guide
|
||||
|
||||
---
|
||||
|
||||
### DEPENDENCY_ANALYSIS.md
|
||||
|
||||
**Contents**:
|
||||
- Detailed analysis of each test scenario
|
||||
- Real dependency verification using `pip --dry-run`
|
||||
- Version difference analysis
|
||||
- Rejected scenarios (and why)
|
||||
- Package size verification
|
||||
- Recommendations for implementation
|
||||
|
||||
**Use when**:
|
||||
- Understanding test design decisions
|
||||
- Evaluating new package candidates
|
||||
- Reviewing why certain packages were rejected
|
||||
- Learning the analysis methodology
|
||||
|
||||
**Key sections**:
|
||||
- Test Scenarios with Real Dependencies → Detailed scenarios
|
||||
- Rejected Scenarios → What NOT to use (e.g., click+colorama)
|
||||
- Validation Commands → How to verify analysis
|
||||
|
||||
---
|
||||
|
||||
### TEST_SCENARIOS.md
|
||||
|
||||
**Contents**:
|
||||
- Complete specifications for scenarios 1-6
|
||||
- Exact package versions and states
|
||||
- Policy configurations (JSON)
|
||||
- Expected pip commands
|
||||
- Expected final states
|
||||
- Key points for each scenario
|
||||
|
||||
**Use when**:
|
||||
- Writing new tests
|
||||
- Understanding test expectations
|
||||
- Debugging test failures
|
||||
- Documenting new scenarios
|
||||
|
||||
**Key sections**:
|
||||
- Each scenario section → Complete specification
|
||||
- Summary tables → Quick reference
|
||||
- Policy types summary → Available policy options
|
||||
|
||||
---
|
||||
|
||||
### analyze_dependencies.py
|
||||
|
||||
**Features**:
|
||||
- Interactive package analysis
|
||||
- Dry-run simulation
|
||||
- Version comparison
|
||||
- Pin impact analysis
|
||||
|
||||
**Use when**:
|
||||
- Exploring new packages
|
||||
- Verifying current environment
|
||||
- Checking upgrade impacts
|
||||
- Quick dependency checks
|
||||
|
||||
**Commands**:
|
||||
```bash
|
||||
# Analyze specific package
|
||||
python analyze_dependencies.py requests
|
||||
|
||||
# Analyze all test packages
|
||||
python analyze_dependencies.py --all
|
||||
|
||||
# Show current environment
|
||||
python analyze_dependencies.py --env
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### requirements-test-base.txt
|
||||
|
||||
**Contents**:
|
||||
- Base packages for test environment
|
||||
- Version specifications
|
||||
- Comments explaining each package's purpose
|
||||
|
||||
**Use when**:
|
||||
- Setting up test environment
|
||||
- Adding pre-installed packages
|
||||
- Modifying base versions
|
||||
- Recreating clean environment
|
||||
|
||||
**Format**:
|
||||
```txt
|
||||
# Scenario X: Purpose
|
||||
package==version # Comment explaining role
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔄 Workflow Examples
|
||||
|
||||
### Example 1: Adding flask Test
|
||||
|
||||
```bash
|
||||
# 1. Analyze flask
|
||||
python analyze_dependencies.py flask
|
||||
|
||||
# Output shows:
|
||||
# Would install: Flask, Jinja2, MarkupSafe, Werkzeug, blinker, click, itsdangerous
|
||||
|
||||
# 2. Check sizes
|
||||
./test_venv/bin/pip download --no-deps flask jinja2 werkzeug
|
||||
ls -lh *.whl
|
||||
|
||||
# 3. Document in DEPENDENCY_TREE_CONTEXT.md
|
||||
# Add section:
|
||||
### 3. flask → Dependencies
|
||||
**Package**: `flask==3.1.2`
|
||||
**Size**: ~100KB
|
||||
...
|
||||
|
||||
# 4. Write test
|
||||
# Create test_flask_dependencies.py
|
||||
|
||||
# 5. Test
|
||||
pytest test_flask_dependencies.py -v --override-ini="addopts="
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Example 2: Investigating Test Failure
|
||||
|
||||
```bash
|
||||
# Test failed: "urllib3 version mismatch"
|
||||
|
||||
# 1. Check installed
|
||||
./test_venv/bin/pip freeze | grep urllib3
|
||||
# Output: urllib3==2.5.0 (expected: 1.26.15)
|
||||
|
||||
# 2. Analyze what happened
|
||||
python analyze_dependencies.py requests
|
||||
|
||||
# 3. Check context
|
||||
# Open DEPENDENCY_TREE_CONTEXT.md
|
||||
# Section: "urllib3: Major Version Jump"
|
||||
# Confirms: 1.26.15 → 2.5.0 is expected without pin
|
||||
|
||||
# 4. Verify test has pin
|
||||
# Check test_dependency_protection.py for pin_policy fixture
|
||||
|
||||
# 5. Reset environment
|
||||
rm -rf test_venv && ./setup_test_env.sh
|
||||
|
||||
# 6. Re-run test
|
||||
pytest test_dependency_protection.py -v --override-ini="addopts="
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🎓 Best Practices
|
||||
|
||||
### When Adding New Tests
|
||||
|
||||
✅ **DO**:
|
||||
- Use `analyze_dependencies.py` first
|
||||
- Document in **DEPENDENCY_TREE_CONTEXT.md**
|
||||
- Add scenario to **TEST_SCENARIOS.md**
|
||||
- Verify with real pip operations
|
||||
- Keep packages lightweight (<500KB total)
|
||||
|
||||
❌ **DON'T**:
|
||||
- Add packages without verifying dependencies
|
||||
- Use packages with optional dependencies only
|
||||
- Add heavy packages (>1MB)
|
||||
- Skip documentation
|
||||
- Mock subprocess for integration tests
|
||||
|
||||
---
|
||||
|
||||
### When Updating Context
|
||||
|
||||
✅ **DO**:
|
||||
- Re-run `analyze_dependencies.py --all`
|
||||
- Update version numbers throughout
|
||||
- Document breaking changes
|
||||
- Test after updates
|
||||
- Note update date
|
||||
|
||||
❌ **DON'T**:
|
||||
- Update only one file
|
||||
- Skip verification
|
||||
- Forget to update TEST_SCENARIOS.md
|
||||
- Leave outdated version numbers
|
||||
|
||||
---
|
||||
|
||||
## 🆘 Quick Troubleshooting
|
||||
|
||||
| Problem | Check | Solution |
|
||||
|---------|-------|----------|
|
||||
| Test fails with version mismatch | `pip freeze` | Recreate venv with `./setup_test_env.sh` |
|
||||
| Package not found | `pip index versions PKG` | Check if package exists on PyPI |
|
||||
| Unexpected dependencies | `analyze_dependencies.py PKG` | Review dependency tree in context file |
|
||||
| Wrong test data | **TEST_SCENARIOS.md** | Verify against documented scenario |
|
||||
| Unclear why package chosen | **DEPENDENCY_ANALYSIS.md** | Read "Rejected Scenarios" section |
|
||||
|
||||
---
|
||||
|
||||
## 📞 Need Help?
|
||||
|
||||
1. **Check context files first**: Most answers are documented
|
||||
2. **Run analyze_dependencies.py**: Verify current state
|
||||
3. **Review test scenarios**: Understand expected behavior
|
||||
4. **Examine dependency trees**: Understand relationships
|
||||
5. **Check DEPENDENCY_ANALYSIS.md**: Learn the "why" behind decisions
|
||||
|
||||
---
|
||||
|
||||
## 📝 Maintenance Checklist
|
||||
|
||||
**Every 6 months or when major versions release**:
|
||||
|
||||
- [ ] Run `python analyze_dependencies.py --all`
|
||||
- [ ] Check for new major versions: `pip index versions urllib3 certifi six`
|
||||
- [ ] Update **DEPENDENCY_TREE_CONTEXT.md** version numbers
|
||||
- [ ] Update **TEST_SCENARIOS.md** expected versions
|
||||
- [ ] Test all scenarios: `pytest -v --override-ini="addopts="`
|
||||
- [ ] Document any breaking changes
|
||||
- [ ] Update this guide if workflow changed
|
||||
|
||||
---
|
||||
|
||||
## 🔗 File Relationships
|
||||
|
||||
```
|
||||
requirements-test-base.txt
|
||||
↓ (defines)
|
||||
Current Test Environment
|
||||
↓ (analyzed by)
|
||||
analyze_dependencies.py
|
||||
↓ (documents)
|
||||
DEPENDENCY_TREE_CONTEXT.md
|
||||
↓ (informs)
|
||||
TEST_SCENARIOS.md
|
||||
↓ (implemented in)
|
||||
test_*.py files
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2025-10-01
|
||||
**Python Version**: 3.12.3
|
||||
**pip Version**: 25.2
|
||||
261
tests/common/pip_util/DEPENDENCY_ANALYSIS.md
Normal file
261
tests/common/pip_util/DEPENDENCY_ANALYSIS.md
Normal file
@ -0,0 +1,261 @@
|
||||
# pip_util Test Package Dependency Analysis
|
||||
|
||||
Real dependency analysis using `pip install --dry-run` to verify meaningful test scenarios.
|
||||
|
||||
## Analysis Date
|
||||
|
||||
Generated: 2025-10-01
|
||||
Tool: `pip install --dry-run --ignore-installed`
|
||||
|
||||
## Test Scenarios with Real Dependencies
|
||||
|
||||
### Scenario 1: Dependency Version Protection (requests + urllib3)
|
||||
|
||||
**Purpose**: Verify pin_dependencies prevents unwanted upgrades
|
||||
|
||||
**Initial Environment**:
|
||||
```
|
||||
urllib3==1.26.15
|
||||
certifi==2023.7.22
|
||||
charset-normalizer==3.2.0
|
||||
```
|
||||
|
||||
**Without pin** (`pip install requests`):
|
||||
```bash
|
||||
Would install:
|
||||
certifi-2025.8.3 # UPGRADED from 2023.7.22 (+2 years)
|
||||
charset-normalizer-3.4.3 # UPGRADED from 3.2.0 (minor)
|
||||
idna-3.10 # NEW dependency
|
||||
requests-2.32.5 # NEW package
|
||||
urllib3-2.5.0 # UPGRADED from 1.26.15 (MAJOR 1.x→2.x!)
|
||||
```
|
||||
|
||||
**With pin** (`pip install requests urllib3==1.26.15 certifi==2023.7.22 charset-normalizer==3.2.0`):
|
||||
```bash
|
||||
Would install:
|
||||
idna-3.10 # NEW dependency (required by requests)
|
||||
requests-2.32.5 # NEW package
|
||||
|
||||
# Pinned packages stay at old versions:
|
||||
urllib3==1.26.15 ✅ PROTECTED (prevented 1.x→2.x jump)
|
||||
certifi==2023.7.22 ✅ PROTECTED
|
||||
charset-normalizer==3.2.0 ✅ PROTECTED
|
||||
```
|
||||
|
||||
**Key Finding**:
|
||||
- `urllib3` 1.26.15 → 2.5.0 is a **MAJOR version jump** (breaking changes!)
|
||||
- requests accepts both: `urllib3<3,>=1.21.1` (compatible with 1.x and 2.x)
|
||||
- Pin successfully prevents unwanted major upgrade
|
||||
|
||||
---
|
||||
|
||||
### Scenario 2: Package with Dependency (python-dateutil + six)
|
||||
|
||||
**Purpose**: Verify pin_dependencies with dependency chain
|
||||
|
||||
**Analysis**:
|
||||
```bash
|
||||
$ pip install --dry-run python-dateutil
|
||||
|
||||
Would install:
|
||||
python-dateutil-2.9.0.post0
|
||||
six-1.17.0 # DEPENDENCY
|
||||
```
|
||||
|
||||
**Initial Environment**:
|
||||
```
|
||||
six==1.16.0 # Older version
|
||||
```
|
||||
|
||||
**Without pin** (`pip install python-dateutil`):
|
||||
```bash
|
||||
Would install:
|
||||
python-dateutil-2.9.0.post0
|
||||
six-1.17.0 # UPGRADED from 1.16.0
|
||||
```
|
||||
|
||||
**With pin** (`pip install python-dateutil six==1.16.0`):
|
||||
```bash
|
||||
Would install:
|
||||
python-dateutil-2.9.0.post0
|
||||
|
||||
# Pinned package:
|
||||
six==1.16.0 ✅ PROTECTED
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Scenario 3: Package Deletion and Restore (six)
|
||||
|
||||
**Purpose**: Verify restore policy reinstalls deleted packages
|
||||
|
||||
**Initial Environment**:
|
||||
```
|
||||
six==1.16.0
|
||||
attrs==23.1.0
|
||||
packaging==23.1
|
||||
```
|
||||
|
||||
**Action Sequence**:
|
||||
1. Delete six: `pip uninstall -y six`
|
||||
2. Verify deletion: `pip freeze | grep six` (empty)
|
||||
3. Restore: `batch.ensure_installed()` → `pip install six==1.16.0`
|
||||
|
||||
**Expected Result**:
|
||||
```
|
||||
six==1.16.0 # ✅ RESTORED
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Scenario 4: Version Change and Restore (urllib3)
|
||||
|
||||
**Purpose**: Verify restore policy reverts version changes
|
||||
|
||||
**Initial Environment**:
|
||||
```
|
||||
urllib3==1.26.15
|
||||
```
|
||||
|
||||
**Action Sequence**:
|
||||
1. Upgrade: `pip install urllib3==2.5.0`
|
||||
2. Verify change: `pip freeze | grep urllib3` → `urllib3==2.5.0`
|
||||
3. Restore: `batch.ensure_installed()` → `pip install urllib3==1.26.15`
|
||||
|
||||
**Expected Result**:
|
||||
```
|
||||
urllib3==1.26.15 # ✅ RESTORED (downgraded from 2.5.0)
|
||||
```
|
||||
|
||||
**Key Finding**:
|
||||
- Downgrade from 2.x to 1.x requires explicit version specification
|
||||
- pip allows downgrades with `pip install urllib3==1.26.15`
|
||||
|
||||
---
|
||||
|
||||
## Rejected Scenarios
|
||||
|
||||
### click + colorama (NO REAL DEPENDENCY)
|
||||
|
||||
**Analysis**:
|
||||
```bash
|
||||
$ pip install --dry-run click
|
||||
Would install: click-8.3.0
|
||||
|
||||
$ pip install --dry-run click colorama==0.4.6
|
||||
Would install: click-8.3.0 # colorama not installed!
|
||||
```
|
||||
|
||||
**Finding**: click has **NO direct dependency** on colorama
|
||||
- colorama is **optional** and platform-specific (Windows only)
|
||||
- Not a good test case for dependency protection
|
||||
|
||||
**Recommendation**: Use python-dateutil + six instead
|
||||
|
||||
---
|
||||
|
||||
## Package Size Verification
|
||||
|
||||
```bash
|
||||
Package Size Version Purpose
|
||||
-------------------------------------------------------
|
||||
urllib3 ~140KB 1.26.15 Protected dependency
|
||||
certifi ~158KB 2023.7.22 SSL certificates
|
||||
charset-normalizer ~46KB 3.2.0 Charset detection
|
||||
idna ~69KB 3.10 NEW dep from requests
|
||||
requests ~100KB 2.32.5 Main package to install
|
||||
six ~11KB 1.16.0 Restore test
|
||||
python-dateutil ~280KB 2.9.0 Depends on six
|
||||
attrs ~61KB 23.1.0 Bystander
|
||||
packaging ~48KB 23.1 Bystander
|
||||
-------------------------------------------------------
|
||||
Total ~913KB (< 1MB) ✅ All lightweight
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Dependency Graph
|
||||
|
||||
```
|
||||
requests 2.32.5
|
||||
├── charset_normalizer<4,>=2 (have: 3.2.0)
|
||||
├── idna<4,>=2.5 (need: 3.10) ← NEW
|
||||
├── urllib3<3,>=1.21.1 (have: 1.26.15, latest: 2.5.0)
|
||||
└── certifi>=2017.4.17 (have: 2023.7.22, latest: 2025.8.3)
|
||||
|
||||
python-dateutil 2.9.0
|
||||
└── six>=1.5 (have: 1.16.0, latest: 1.17.0)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Version Compatibility Matrix
|
||||
|
||||
| Package | Old Version | Latest | Spec | Compatible? |
|
||||
|---------|------------|--------|------|-------------|
|
||||
| urllib3 | 1.26.15 | 2.5.0 | <3,>=1.21.1 | ✅ Both work |
|
||||
| certifi | 2023.7.22 | 2025.8.3 | >=2017.4.17 | ✅ Both work |
|
||||
| charset-normalizer | 3.2.0 | 3.4.3 | <4,>=2 | ✅ Both work |
|
||||
| six | 1.16.0 | 1.17.0 | >=1.5 | ✅ Both work |
|
||||
| idna | (none) | 3.10 | <4,>=2.5 | ⚠️ Must install |
|
||||
|
||||
---
|
||||
|
||||
## Test Data Justification
|
||||
|
||||
### Why urllib3 1.26.15?
|
||||
1. **Real world scenario**: Many projects pin urllib3<2 to avoid breaking changes
|
||||
2. **Meaningful test**: 1.26.15 → 2.5.0 is a major version jump (API changes)
|
||||
3. **Compatibility**: requests accepts both 1.x and 2.x (good for testing)
|
||||
|
||||
### Why certifi 2023.7.22?
|
||||
1. **Real world scenario**: Older environment with outdated SSL certificates
|
||||
2. **Meaningful test**: 2-year version gap (2023 → 2025)
|
||||
3. **Safety**: Still compatible with requests
|
||||
|
||||
### Why six 1.16.0?
|
||||
1. **Lightweight**: Only 11KB
|
||||
2. **Real dependency**: python-dateutil actually depends on it
|
||||
3. **Stable**: six is mature and rarely changes
|
||||
|
||||
---
|
||||
|
||||
## Recommendations for Test Implementation
|
||||
|
||||
### ✅ Keep These Scenarios:
|
||||
1. **requests + urllib3 pin** - Real major version protection
|
||||
2. **python-dateutil + six** - Real dependency chain
|
||||
3. **six deletion/restore** - Real package management
|
||||
4. **urllib3 version change** - Real downgrade scenario
|
||||
|
||||
### ❌ Remove These Scenarios:
|
||||
1. **click + colorama** - No real dependency (colorama is optional/Windows-only)
|
||||
|
||||
### 📝 Update Required Files:
|
||||
1. `requirements-test-base.txt` - Add idna (new dependency from requests)
|
||||
2. `TEST_SCENARIOS.md` - Update with real dependency analysis
|
||||
3. `test_dependency_protection.py` - Remove click-colorama test
|
||||
4. `pip_util.design.en.md` - Update examples with verified dependencies
|
||||
|
||||
---
|
||||
|
||||
## Validation Commands
|
||||
|
||||
Run these to verify analysis:
|
||||
|
||||
```bash
|
||||
# Check current environment
|
||||
./test_venv/bin/pip freeze
|
||||
|
||||
# Simulate requests installation without pin
|
||||
./test_venv/bin/pip install --dry-run requests
|
||||
|
||||
# Simulate requests installation with pin
|
||||
./test_venv/bin/pip install --dry-run requests urllib3==1.26.15 certifi==2023.7.22 charset-normalizer==3.2.0
|
||||
|
||||
# Check python-dateutil dependencies
|
||||
./test_venv/bin/pip install --dry-run python-dateutil
|
||||
|
||||
# Verify urllib3 version availability
|
||||
./test_venv/bin/pip index versions urllib3 | head -20
|
||||
```
|
||||
413
tests/common/pip_util/DEPENDENCY_TREE_CONTEXT.md
Normal file
413
tests/common/pip_util/DEPENDENCY_TREE_CONTEXT.md
Normal file
@ -0,0 +1,413 @@
|
||||
# Dependency Tree Context for pip_util Tests
|
||||
|
||||
**Generated**: 2025-10-01
|
||||
**Tool**: `pip install --dry-run --ignore-installed`
|
||||
**Python**: 3.12.3
|
||||
**pip**: 25.2
|
||||
|
||||
This document provides detailed dependency tree information for all test packages, verified against real PyPI data. Use this as a reference when extending tests.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Current Test Environment](#current-test-environment)
|
||||
2. [Package Dependency Trees](#package-dependency-trees)
|
||||
3. [Version Analysis](#version-analysis)
|
||||
4. [Upgrade Scenarios](#upgrade-scenarios)
|
||||
5. [Adding New Test Scenarios](#adding-new-test-scenarios)
|
||||
|
||||
---
|
||||
|
||||
## Current Test Environment
|
||||
|
||||
**Base packages installed in test_venv** (from `requirements-test-base.txt`):
|
||||
|
||||
```
|
||||
urllib3==1.26.15 # Protected from 2.x upgrade
|
||||
certifi==2023.7.22 # Protected from 2025.x upgrade
|
||||
charset-normalizer==3.2.0 # Protected from 3.4.x upgrade
|
||||
six==1.16.0 # For deletion/restore tests
|
||||
attrs==23.1.0 # Bystander package
|
||||
packaging==23.1 # Bystander package
|
||||
pytest==8.4.2 # Test framework
|
||||
```
|
||||
|
||||
**Total environment size**: ~913KB (all packages < 1MB)
|
||||
|
||||
---
|
||||
|
||||
## Package Dependency Trees
|
||||
|
||||
### 1. requests → Dependencies
|
||||
|
||||
**Package**: `requests==2.32.5`
|
||||
**Size**: ~100KB
|
||||
**Purpose**: Main test package for dependency protection
|
||||
|
||||
#### Dependency Tree
|
||||
|
||||
```
|
||||
requests==2.32.5
|
||||
├── charset-normalizer<4,>=2
|
||||
│ └── 3.2.0 (OLD) → 3.4.3 (LATEST)
|
||||
├── idna<4,>=2.5
|
||||
│ └── (NOT INSTALLED) → 3.10 (LATEST)
|
||||
├── urllib3<3,>=1.21.1
|
||||
│ └── 1.26.15 (OLD) → 2.5.0 (LATEST) ⚠️ MAJOR VERSION JUMP
|
||||
└── certifi>=2017.4.17
|
||||
└── 2023.7.22 (OLD) → 2025.8.3 (LATEST)
|
||||
```
|
||||
|
||||
#### Install Scenarios
|
||||
|
||||
**Scenario A: Without constraints (fresh install)**
|
||||
```bash
|
||||
$ pip install --dry-run --ignore-installed requests
|
||||
|
||||
Would install:
|
||||
certifi-2025.8.3 # Latest version
|
||||
charset-normalizer-3.4.3 # Latest version
|
||||
idna-3.10 # New dependency
|
||||
requests-2.32.5 # Target package
|
||||
urllib3-2.5.0 # Latest version (2.x!)
|
||||
```
|
||||
|
||||
**Scenario B: With pin constraints**
|
||||
```bash
|
||||
$ pip install --dry-run requests \
|
||||
urllib3==1.26.15 \
|
||||
certifi==2023.7.22 \
|
||||
charset-normalizer==3.2.0
|
||||
|
||||
Would install:
|
||||
certifi-2023.7.22 # Pinned to OLD version
|
||||
charset-normalizer-3.2.0 # Pinned to OLD version
|
||||
idna-3.10 # New dependency (not pinned)
|
||||
requests-2.32.5 # Target package
|
||||
urllib3-1.26.15 # Pinned to OLD version
|
||||
```
|
||||
|
||||
**Impact Analysis**:
|
||||
- ✅ Pin successfully prevents urllib3 1.x → 2.x major upgrade
|
||||
- ✅ Pin prevents certifi 2023 → 2025 upgrade (2 years)
|
||||
- ✅ Pin prevents charset-normalizer minor upgrade
|
||||
- ⚠️ idna is NEW and NOT pinned (acceptable - new dependency)
|
||||
|
||||
---
|
||||
|
||||
### 2. python-dateutil → Dependencies
|
||||
|
||||
**Package**: `python-dateutil==2.9.0.post0`
|
||||
**Size**: ~280KB
|
||||
**Purpose**: Real dependency chain test (depends on six)
|
||||
|
||||
#### Dependency Tree
|
||||
|
||||
```
|
||||
python-dateutil==2.9.0.post0
|
||||
└── six>=1.5
|
||||
└── 1.16.0 (OLD) → 1.17.0 (LATEST)
|
||||
```
|
||||
|
||||
#### Install Scenarios
|
||||
|
||||
**Scenario A: Without constraints**
|
||||
```bash
|
||||
$ pip install --dry-run --ignore-installed python-dateutil
|
||||
|
||||
Would install:
|
||||
python-dateutil-2.9.0.post0 # Target package
|
||||
six-1.17.0 # Latest version
|
||||
```
|
||||
|
||||
**Scenario B: With pin constraints**
|
||||
```bash
|
||||
$ pip install --dry-run python-dateutil six==1.16.0
|
||||
|
||||
Would install:
|
||||
python-dateutil-2.9.0.post0 # Target package
|
||||
six-1.16.0 # Pinned to OLD version
|
||||
```
|
||||
|
||||
**Impact Analysis**:
|
||||
- ✅ Pin successfully prevents six 1.16.0 → 1.17.0 upgrade
|
||||
- ✅ Real dependency relationship (verified via PyPI)
|
||||
|
||||
---
|
||||
|
||||
### 3. Other Test Packages (No Dependencies)
|
||||
|
||||
These packages have no dependencies or only have dependencies already in the test environment:
|
||||
|
||||
```
|
||||
attrs==23.1.0 # No dependencies
|
||||
packaging==23.1 # No dependencies (standalone)
|
||||
six==1.16.0 # No dependencies (pure Python)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Version Analysis
|
||||
|
||||
### urllib3: Major Version Jump (1.x → 2.x)
|
||||
|
||||
**Current**: 1.26.15 (2023)
|
||||
**Latest**: 2.5.0 (2025)
|
||||
**Breaking Changes**: YES - urllib3 2.0 removed deprecated APIs
|
||||
|
||||
**Available versions**:
|
||||
```
|
||||
2.x series: 2.5.0, 2.4.0, 2.3.0, 2.2.3, 2.2.2, 2.2.1, 2.2.0, 2.1.0, 2.0.7, ...
|
||||
1.26.x: 1.26.20, 1.26.19, 1.26.18, 1.26.17, 1.26.16, 1.26.15, ...
|
||||
1.25.x: 1.25.11, 1.25.10, 1.25.9, ...
|
||||
```
|
||||
|
||||
**Why test with 1.26.15?**
|
||||
- ✅ Real-world scenario: Many projects pin `urllib3<2` to avoid breaking changes
|
||||
- ✅ Meaningful test: 1.x → 2.x is a major API change
|
||||
- ✅ Compatibility: requests accepts both 1.x and 2.x (`urllib3<3,>=1.21.1`)
|
||||
|
||||
**Breaking changes in urllib3 2.0**:
|
||||
- Removed `urllib3.contrib.pyopenssl`
|
||||
- Removed `urllib3.contrib.securetransport`
|
||||
- Changed import paths for some modules
|
||||
- Updated connection pooling behavior
|
||||
|
||||
---
|
||||
|
||||
### certifi: Long-Term Version Gap (2023 → 2025)
|
||||
|
||||
**Current**: 2023.7.22 (July 2023)
|
||||
**Latest**: 2025.8.3 (August 2025)
|
||||
**Gap**: ~2 years of SSL certificate updates
|
||||
|
||||
**Available versions**:
|
||||
```
|
||||
2025: 2025.8.3, 2025.7.14, 2025.7.9, 2025.6.15, 2025.4.26, ...
|
||||
2024: 2024.12.25, 2024.11.28, 2024.10.29, 2024.9.19, ...
|
||||
2023: 2023.11.17, 2023.7.22, 2023.5.7, ...
|
||||
```
|
||||
|
||||
**Why test with 2023.7.22?**
|
||||
- ✅ Real-world scenario: Older environments with outdated SSL certificates
|
||||
- ✅ Meaningful test: 2-year gap shows protection of older versions
|
||||
- ✅ Safety: Still compatible with requests (`certifi>=2017.4.17`)
|
||||
|
||||
---
|
||||
|
||||
### charset-normalizer: Minor Version Updates
|
||||
|
||||
**Current**: 3.2.0 (2023)
|
||||
**Latest**: 3.4.3 (2025)
|
||||
**Breaking Changes**: NO - only minor/patch updates
|
||||
|
||||
**Available versions**:
|
||||
```
|
||||
3.4.x: 3.4.3, 3.4.2, 3.4.1, 3.4.0
|
||||
3.3.x: 3.3.2, 3.3.1, 3.3.0
|
||||
3.2.x: 3.2.0
|
||||
```
|
||||
|
||||
**Why test with 3.2.0?**
|
||||
- ✅ Demonstrates protection of minor version updates
|
||||
- ✅ Compatible with requests (`charset-normalizer<4,>=2`)
|
||||
|
||||
---
|
||||
|
||||
### six: Stable Version Update
|
||||
|
||||
**Current**: 1.16.0 (2021)
|
||||
**Latest**: 1.17.0 (2024)
|
||||
**Breaking Changes**: NO - six is very stable
|
||||
|
||||
**Available versions**:
|
||||
```
|
||||
1.17.0, 1.16.0, 1.15.0, 1.14.0, 1.13.0, 1.12.0, ...
|
||||
```
|
||||
|
||||
**Why test with 1.16.0?**
|
||||
- ✅ Real dependency of python-dateutil
|
||||
- ✅ Small size (11KB) - lightweight for tests
|
||||
- ✅ Demonstrates protection of stable packages
|
||||
|
||||
---
|
||||
|
||||
### idna: New Dependency
|
||||
|
||||
**Not pre-installed** - Added by requests
|
||||
|
||||
**Version**: 3.10
|
||||
**Size**: ~69KB
|
||||
**Dependency spec**: `idna<4,>=2.5` (from requests)
|
||||
|
||||
**Why NOT pre-installed?**
|
||||
- ✅ Tests that new dependencies are correctly added
|
||||
- ✅ Tests that pins only affect specified packages
|
||||
- ✅ Real-world scenario: new dependency introduced by package update
|
||||
|
||||
---
|
||||
|
||||
## Upgrade Scenarios
|
||||
|
||||
### Scenario Matrix
|
||||
|
||||
| Package | Initial | Without Pin | With Pin | Change Type |
|
||||
|---------|---------|-------------|----------|-------------|
|
||||
| **urllib3** | 1.26.15 | 2.5.0 ❌ | 1.26.15 ✅ | Major (breaking) |
|
||||
| **certifi** | 2023.7.22 | 2025.8.3 ❌ | 2023.7.22 ✅ | 2-year gap |
|
||||
| **charset-normalizer** | 3.2.0 | 3.4.3 ❌ | 3.2.0 ✅ | Minor update |
|
||||
| **six** | 1.16.0 | 1.17.0 ❌ | 1.16.0 ✅ | Stable update |
|
||||
| **idna** | (none) | 3.10 ✅ | 3.10 ✅ | New dependency |
|
||||
| **requests** | (none) | 2.32.5 ✅ | 2.32.5 ✅ | Target package |
|
||||
| **python-dateutil** | (none) | 2.9.0 ✅ | 2.9.0 ✅ | Target package |
|
||||
|
||||
---
|
||||
|
||||
## Adding New Test Scenarios
|
||||
|
||||
### Step 1: Identify Candidate Package
|
||||
|
||||
Use `pip install --dry-run` to analyze dependencies:
|
||||
|
||||
```bash
|
||||
# Analyze package dependencies
|
||||
./test_venv/bin/pip install --dry-run --ignore-installed PACKAGE
|
||||
|
||||
# Check what changes with current environment
|
||||
./test_venv/bin/pip install --dry-run PACKAGE
|
||||
|
||||
# List available versions
|
||||
./test_venv/bin/pip index versions PACKAGE
|
||||
```
|
||||
|
||||
### Step 2: Verify Real Dependencies
|
||||
|
||||
**Good candidates**:
|
||||
- ✅ Has 2+ dependencies
|
||||
- ✅ Dependencies have version upgrades available
|
||||
- ✅ Total size < 500KB (all packages combined)
|
||||
- ✅ Real-world use case (popular package)
|
||||
|
||||
**Examples**:
|
||||
```bash
|
||||
# flask → click, werkzeug, jinja2 (good: multiple dependencies)
|
||||
$ pip install --dry-run --ignore-installed flask
|
||||
Would install: Flask-3.1.2 Jinja2-3.1.6 MarkupSafe-3.0.3 Werkzeug-3.1.3 blinker-1.9.0 click-8.3.0 itsdangerous-2.2.0
|
||||
|
||||
# pytest-cov → pytest, coverage (good: popular testing tool)
|
||||
$ pip install --dry-run --ignore-installed pytest-cov
|
||||
Would install: coverage-7.10.7 pytest-8.4.2 pytest-cov-7.0.0
|
||||
```
|
||||
|
||||
**Bad candidates**:
|
||||
- ❌ click → colorama (no real dependency - colorama is optional/Windows-only)
|
||||
- ❌ pandas → numpy (too large - numpy is 50MB+)
|
||||
- ❌ torch → ... (too large - 800MB+)
|
||||
|
||||
### Step 3: Document Dependencies
|
||||
|
||||
Add to this file:
|
||||
|
||||
```markdown
|
||||
### Package: PACKAGE_NAME → Dependencies
|
||||
|
||||
**Package**: `PACKAGE==VERSION`
|
||||
**Size**: ~XXXKB
|
||||
**Purpose**: Brief description
|
||||
|
||||
#### Dependency Tree
|
||||
(Use tree format)
|
||||
|
||||
#### Install Scenarios
|
||||
(Show with/without pin)
|
||||
|
||||
#### Impact Analysis
|
||||
(What does pin protect?)
|
||||
```
|
||||
|
||||
### Step 4: Update Test Files
|
||||
|
||||
1. Add package to `requirements-test-base.txt` (if pre-installation needed)
|
||||
2. Create policy fixture in test file
|
||||
3. Write test function using `reset_test_venv` fixture
|
||||
4. Update `TEST_SCENARIOS.md` with detailed scenario
|
||||
|
||||
---
|
||||
|
||||
## Maintenance Notes
|
||||
|
||||
### Updating This Document
|
||||
|
||||
Re-run analysis when:
|
||||
- ✅ PyPI releases major version updates (e.g., urllib3 3.0)
|
||||
- ✅ Adding new test packages
|
||||
- ✅ Test environment base packages change
|
||||
- ✅ Every 6 months (to catch version drift)
|
||||
|
||||
### Verification Commands
|
||||
|
||||
```bash
|
||||
# Regenerate dependency tree
|
||||
./test_venv/bin/pip install --dry-run --ignore-installed requests
|
||||
./test_venv/bin/pip install --dry-run --ignore-installed python-dateutil
|
||||
|
||||
# Check current environment
|
||||
./test_venv/bin/pip freeze
|
||||
|
||||
# Verify test packages still available on PyPI
|
||||
./test_venv/bin/pip index versions urllib3
|
||||
./test_venv/bin/pip index versions certifi
|
||||
./test_venv/bin/pip index versions six
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference: Package Specs
|
||||
|
||||
From actual package metadata:
|
||||
|
||||
```python
|
||||
# requests dependencies (from requests==2.32.5)
|
||||
install_requires = [
|
||||
"charset_normalizer<4,>=2",
|
||||
"idna<4,>=2.5",
|
||||
"urllib3<3,>=1.21.1",
|
||||
"certifi>=2017.4.17"
|
||||
]
|
||||
|
||||
# python-dateutil dependencies (from python-dateutil==2.9.0)
|
||||
install_requires = [
|
||||
"six>=1.5"
|
||||
]
|
||||
|
||||
# six dependencies
|
||||
install_requires = [] # No dependencies
|
||||
|
||||
# attrs dependencies
|
||||
install_requires = [] # No dependencies
|
||||
|
||||
# packaging dependencies
|
||||
install_requires = [] # No dependencies
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Version Compatibility Table
|
||||
|
||||
| Package | Minimum | Maximum | Current Test | Latest | Notes |
|
||||
|---------|---------|---------|--------------|--------|-------|
|
||||
| urllib3 | 1.21.1 | <3.0 | 1.26.15 | 2.5.0 | Major version jump possible |
|
||||
| certifi | 2017.4.17 | (none) | 2023.7.22 | 2025.8.3 | Always backward compatible |
|
||||
| charset-normalizer | 2.0 | <4.0 | 3.2.0 | 3.4.3 | Within major version |
|
||||
| six | 1.5 | (none) | 1.16.0 | 1.17.0 | Very stable |
|
||||
| idna | 2.5 | <4.0 | (new) | 3.10 | Added by requests |
|
||||
|
||||
---
|
||||
|
||||
## See Also
|
||||
|
||||
- **DEPENDENCY_ANALYSIS.md** - Detailed analysis methodology
|
||||
- **TEST_SCENARIOS.md** - Complete test scenario specifications
|
||||
- **requirements-test-base.txt** - Base environment packages
|
||||
- **README.md** - Test suite overview and usage
|
||||
305
tests/common/pip_util/README.md
Normal file
305
tests/common/pip_util/README.md
Normal file
@ -0,0 +1,305 @@
|
||||
# pip_util Integration Tests
|
||||
|
||||
Real integration tests for `pip_util.py` using actual PyPI packages and pip operations.
|
||||
|
||||
## Overview
|
||||
|
||||
These tests use a **real isolated venv** to verify pip_util behavior with actual package installations, deletions, and version changes. No mocks - real pip operations only.
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Setup Test Environment
|
||||
|
||||
```bash
|
||||
cd tests/common/pip_util
|
||||
./setup_test_env.sh
|
||||
```
|
||||
|
||||
This creates `test_venv/` with base packages:
|
||||
- urllib3==1.26.15
|
||||
- certifi==2023.7.22
|
||||
- charset-normalizer==3.2.0
|
||||
- colorama==0.4.6
|
||||
- six==1.16.0
|
||||
- attrs==23.1.0
|
||||
- packaging==23.1
|
||||
- pytest (latest)
|
||||
|
||||
### 2. Run Tests
|
||||
|
||||
```bash
|
||||
# Run all integration tests
|
||||
pytest -v --override-ini="addopts="
|
||||
|
||||
# Run specific test
|
||||
pytest test_dependency_protection.py -v --override-ini="addopts="
|
||||
|
||||
# Run with markers
|
||||
pytest -m integration -v --override-ini="addopts="
|
||||
```
|
||||
|
||||
## Test Architecture
|
||||
|
||||
### Real venv Integration
|
||||
|
||||
- **No subprocess mocking** - uses real pip install/uninstall
|
||||
- **Isolated test venv** - prevents system contamination
|
||||
- **Automatic cleanup** - `reset_test_venv` fixture restores state after each test
|
||||
|
||||
### Test Fixtures
|
||||
|
||||
**venv Management**:
|
||||
- `test_venv_path` - Path to test venv (session scope)
|
||||
- `test_pip_cmd` - pip command for test venv
|
||||
- `reset_test_venv` - Restore venv to initial state after each test
|
||||
|
||||
**Helpers**:
|
||||
- `get_installed_packages()` - Get current venv packages
|
||||
- `install_packages(*packages)` - Install packages in test venv
|
||||
- `uninstall_packages(*packages)` - Uninstall packages in test venv
|
||||
|
||||
**Policy Configuration**:
|
||||
- `temp_policy_dir` - Temporary directory for base policies
|
||||
- `temp_user_policy_dir` - Temporary directory for user policies
|
||||
- `mock_manager_util` - Mock manager_util paths to use temp dirs
|
||||
- `mock_context` - Mock context paths to use temp dirs
|
||||
|
||||
## Test Scenarios
|
||||
|
||||
### Scenario 1: Dependency Version Protection
|
||||
**File**: `test_dependency_protection.py::test_dependency_version_protection_with_pin`
|
||||
|
||||
**Initial State**:
|
||||
```python
|
||||
urllib3==1.26.15
|
||||
certifi==2023.7.22
|
||||
charset-normalizer==3.2.0
|
||||
```
|
||||
|
||||
**Action**: Install `requests` with pin_dependencies policy
|
||||
|
||||
**Expected Result**:
|
||||
```python
|
||||
# Dependencies stay at old versions (protected by pin)
|
||||
urllib3==1.26.15 # NOT upgraded to 2.x
|
||||
certifi==2023.7.22 # NOT upgraded
|
||||
charset-normalizer==3.2.0 # NOT upgraded
|
||||
requests==2.31.0 # newly installed
|
||||
```
|
||||
|
||||
### Scenario 2: Click-Colorama Dependency Chain
|
||||
**File**: `test_dependency_protection.py::test_dependency_chain_with_click_colorama`
|
||||
|
||||
**Initial State**:
|
||||
```python
|
||||
colorama==0.4.6
|
||||
```
|
||||
|
||||
**Action**: Install `click` with force_version + pin_dependencies
|
||||
|
||||
**Expected Result**:
|
||||
```python
|
||||
colorama==0.4.6 # PINNED
|
||||
click==8.1.3 # FORCED to specific version
|
||||
```
|
||||
|
||||
### Scenario 3: Package Deletion and Restore
|
||||
**File**: `test_environment_recovery.py::test_package_deletion_and_restore`
|
||||
|
||||
**Initial State**:
|
||||
```python
|
||||
six==1.16.0
|
||||
attrs==23.1.0
|
||||
packaging==23.1
|
||||
```
|
||||
|
||||
**Action**: Delete `six` → call `batch.ensure_installed()`
|
||||
|
||||
**Expected Result**:
|
||||
```python
|
||||
six==1.16.0 # RESTORED to required version
|
||||
```
|
||||
|
||||
### Scenario 4: Version Change and Restore
|
||||
**File**: `test_environment_recovery.py::test_version_change_and_restore`
|
||||
|
||||
**Initial State**:
|
||||
```python
|
||||
urllib3==1.26.15
|
||||
```
|
||||
|
||||
**Action**: Upgrade `urllib3` to 2.1.0 → call `batch.ensure_installed()`
|
||||
|
||||
**Expected Result**:
|
||||
```python
|
||||
urllib3==1.26.15 # RESTORED to required version (downgraded)
|
||||
```
|
||||
|
||||
## Test Categories
|
||||
|
||||
### Priority 1 (Essential) ✅ ALL PASSING
|
||||
- ✅ Dependency version protection (enhanced with exact versions)
|
||||
- ✅ Package deletion and restore (enhanced with exact versions)
|
||||
- ✅ Version change and restore (enhanced with downgrade verification)
|
||||
- ✅ Pin only affects specified packages ✨ NEW
|
||||
- ✅ Major version jump prevention ✨ NEW
|
||||
|
||||
### Priority 2 (Important)
|
||||
- ✅ Complex dependency chains (python-dateutil + six)
|
||||
- ⏳ Full workflow integration (TODO: update to real venv)
|
||||
- ⏳ Pin failure retry (TODO: update to real venv)
|
||||
|
||||
### Priority 3 (Edge Cases)
|
||||
- ⏳ Platform conditions (TODO: update to real venv)
|
||||
- ⏳ Policy priority (TODO: update to real venv)
|
||||
- ⏳ Unit tests (no venv needed)
|
||||
- ⏳ Edge cases (no venv needed)
|
||||
|
||||
## Package Selection
|
||||
|
||||
All test packages are **real PyPI packages < 200KB**:
|
||||
|
||||
| Package | Size | Version | Purpose |
|
||||
|---------|------|---------|---------|
|
||||
| **urllib3** | ~100KB | 1.26.15 | Protected dependency (prevent 2.x upgrade) |
|
||||
| **certifi** | ~10KB | 2023.7.22 | SSL certificates (pinned) |
|
||||
| **charset-normalizer** | ~46KB | 3.2.0 | Charset detection (pinned) |
|
||||
| **requests** | ~100KB | 2.31.0 | Main package to install |
|
||||
| **colorama** | ~25KB | 0.4.6 | Terminal colors (pinned) |
|
||||
| **click** | ~90KB | 8.1.3 | CLI framework (forced version) |
|
||||
| **six** | ~11KB | 1.16.0 | Python 2/3 compatibility (restore) |
|
||||
| **attrs** | ~61KB | 23.1.0 | Bystander package |
|
||||
| **packaging** | ~48KB | 23.1 | Bystander package |
|
||||
|
||||
## Cleanup
|
||||
|
||||
### Manual Cleanup
|
||||
```bash
|
||||
# Remove test venv
|
||||
rm -rf test_venv/
|
||||
|
||||
# Recreate fresh venv
|
||||
./setup_test_env.sh
|
||||
```
|
||||
|
||||
### Automatic Cleanup
|
||||
The `reset_test_venv` fixture automatically:
|
||||
1. Records initial package state
|
||||
2. Runs test
|
||||
3. Removes all packages (except pip/setuptools/wheel)
|
||||
4. Reinstalls initial packages
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Error: "Test venv not found"
|
||||
**Solution**: Run `./setup_test_env.sh`
|
||||
|
||||
### Error: "Package not installed in initial state"
|
||||
**Solution**: Check `requirements-test-base.txt` and recreate venv
|
||||
|
||||
### Tests are slow
|
||||
**Reason**: Real pip operations take 2-3 seconds per test
|
||||
**This is expected** - we're doing actual pip install/uninstall
|
||||
|
||||
## Implementation Details
|
||||
|
||||
### How reset_test_venv Works
|
||||
|
||||
```python
|
||||
@pytest.fixture
|
||||
def reset_test_venv(test_pip_cmd):
|
||||
# 1. Record initial state
|
||||
initial = subprocess.run(test_pip_cmd + ["freeze"], ...)
|
||||
|
||||
yield # Run test here
|
||||
|
||||
# 2. Remove all packages
|
||||
current = subprocess.run(test_pip_cmd + ["freeze"], ...)
|
||||
subprocess.run(test_pip_cmd + ["uninstall", "-y", ...], ...)
|
||||
|
||||
# 3. Restore initial state
|
||||
subprocess.run(test_pip_cmd + ["install", "-r", initial], ...)
|
||||
```
|
||||
|
||||
### How make_pip_cmd is Patched
|
||||
|
||||
```python
|
||||
@pytest.fixture(autouse=True)
|
||||
def setup_pip_util(monkeypatch, test_pip_cmd):
|
||||
from comfyui_manager.common import pip_util
|
||||
|
||||
def make_test_pip_cmd(args: List[str]) -> List[str]:
|
||||
return test_pip_cmd + args # Use test venv pip
|
||||
|
||||
monkeypatch.setattr(
|
||||
pip_util.manager_util,
|
||||
"make_pip_cmd",
|
||||
make_test_pip_cmd
|
||||
)
|
||||
```
|
||||
|
||||
## Dependency Analysis Tool
|
||||
|
||||
Use `analyze_dependencies.py` to examine package dependencies before adding new tests:
|
||||
|
||||
```bash
|
||||
# Analyze specific package
|
||||
python analyze_dependencies.py requests
|
||||
|
||||
# Analyze all test packages
|
||||
python analyze_dependencies.py --all
|
||||
|
||||
# Show current environment
|
||||
python analyze_dependencies.py --env
|
||||
```
|
||||
|
||||
**Output includes**:
|
||||
- Latest available versions
|
||||
- Dependencies that would be installed
|
||||
- Version upgrades that would occur
|
||||
- Impact of pin constraints
|
||||
|
||||
**Example output**:
|
||||
```
|
||||
📦 Latest version: 2.32.5
|
||||
🔍 Scenario A: Install without constraints
|
||||
Would install 5 packages:
|
||||
• urllib3 1.26.15 → 2.5.0 ⚠️ UPGRADE
|
||||
|
||||
🔍 Scenario B: Install with pin constraints
|
||||
Would install 5 packages:
|
||||
• urllib3 1.26.15 (no change) 📌 PINNED
|
||||
|
||||
✅ Pin prevented 2 upgrade(s)
|
||||
```
|
||||
|
||||
## Test Statistics
|
||||
|
||||
**Current Status**: 6 tests, 100% passing
|
||||
|
||||
```
|
||||
test_dependency_version_protection_with_pin PASSED (2.28s)
|
||||
test_dependency_chain_with_six_pin PASSED (2.00s)
|
||||
test_pin_only_affects_specified_packages PASSED (2.25s) ✨ NEW
|
||||
test_major_version_jump_prevention PASSED (3.53s) ✨ NEW
|
||||
test_package_deletion_and_restore PASSED (2.25s)
|
||||
test_version_change_and_restore PASSED (2.24s)
|
||||
|
||||
Total: 14.10s
|
||||
```
|
||||
|
||||
**Test Improvements**:
|
||||
- ✅ All tests verify exact version numbers
|
||||
- ✅ All tests reference DEPENDENCY_TREE_CONTEXT.md
|
||||
- ✅ Added 2 new critical tests (pin selectivity, major version prevention)
|
||||
- ✅ Enhanced error messages with expected vs actual values
|
||||
|
||||
## Design Documents
|
||||
|
||||
- **TEST_IMPROVEMENTS.md** - Summary of test enhancements based on dependency context
|
||||
- **DEPENDENCY_TREE_CONTEXT.md** - Verified dependency trees for all test packages
|
||||
- **DEPENDENCY_ANALYSIS.md** - Dependency analysis methodology
|
||||
- **CONTEXT_FILES_GUIDE.md** - Guide for using context files
|
||||
- **TEST_SCENARIOS.md** - Detailed test scenario specifications
|
||||
- **pip_util.test-design.md** - Test design and architecture
|
||||
- **pip_util.design.en.md** - pip_util design documentation
|
||||
433
tests/common/pip_util/TEST_IMPROVEMENTS.md
Normal file
433
tests/common/pip_util/TEST_IMPROVEMENTS.md
Normal file
@ -0,0 +1,433 @@
|
||||
# Test Code Improvements Based on Dependency Context
|
||||
|
||||
**Date**: 2025-10-01
|
||||
**Basis**: DEPENDENCY_TREE_CONTEXT.md analysis
|
||||
|
||||
This document summarizes all test improvements made using verified dependency tree information.
|
||||
|
||||
---
|
||||
|
||||
## Summary of Changes
|
||||
|
||||
### Tests Enhanced
|
||||
|
||||
| Test File | Tests Modified | Tests Added | Total Tests |
|
||||
|-----------|----------------|-------------|-------------|
|
||||
| `test_dependency_protection.py` | 2 | 2 | 4 |
|
||||
| `test_environment_recovery.py` | 2 | 0 | 2 |
|
||||
| **Total** | **4** | **2** | **6** |
|
||||
|
||||
### Test Results
|
||||
|
||||
```bash
|
||||
$ pytest test_dependency_protection.py test_environment_recovery.py -v
|
||||
|
||||
test_dependency_protection.py::test_dependency_version_protection_with_pin PASSED
|
||||
test_dependency_protection.py::test_dependency_chain_with_six_pin PASSED
|
||||
test_dependency_protection.py::test_pin_only_affects_specified_packages PASSED ✨ NEW
|
||||
test_dependency_protection.py::test_major_version_jump_prevention PASSED ✨ NEW
|
||||
test_environment_recovery.py::test_package_deletion_and_restore PASSED
|
||||
test_environment_recovery.py::test_version_change_and_restore PASSED
|
||||
|
||||
6 passed in 14.10s
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Detailed Improvements
|
||||
|
||||
### 1. test_dependency_version_protection_with_pin
|
||||
|
||||
**File**: `test_dependency_protection.py:34-94`
|
||||
|
||||
**Enhancements**:
|
||||
- ✅ Added exact version assertions based on DEPENDENCY_TREE_CONTEXT.md
|
||||
- ✅ Verified initial versions: urllib3==1.26.15, certifi==2023.7.22, charset-normalizer==3.2.0
|
||||
- ✅ Added verification that idna is NOT pre-installed
|
||||
- ✅ Added assertion that idna==3.10 is installed as NEW dependency
|
||||
- ✅ Verified requests==2.32.5 is installed
|
||||
- ✅ Added detailed error messages explaining what versions are expected and why
|
||||
|
||||
**Key Assertions Added**:
|
||||
```python
|
||||
# Verify expected OLD versions
|
||||
assert initial_urllib3 == "1.26.15", f"Expected urllib3==1.26.15, got {initial_urllib3}"
|
||||
assert initial_certifi == "2023.7.22", f"Expected certifi==2023.7.22, got {initial_certifi}"
|
||||
assert initial_charset == "3.2.0", f"Expected charset-normalizer==3.2.0, got {initial_charset}"
|
||||
|
||||
# Verify idna is NOT installed initially
|
||||
assert "idna" not in initial, "idna should not be pre-installed"
|
||||
|
||||
# Verify new dependency was added (idna is NOT pinned, so it gets installed)
|
||||
assert "idna" in final_packages, "idna should be installed as new dependency"
|
||||
assert final_packages["idna"] == "3.10", f"Expected idna==3.10, got {final_packages['idna']}"
|
||||
```
|
||||
|
||||
**Based on Context**:
|
||||
- DEPENDENCY_TREE_CONTEXT.md Section 1: requests → Dependencies
|
||||
- Verified: Without pin, urllib3 would upgrade to 2.5.0 (MAJOR version jump)
|
||||
- Verified: idna is NEW dependency (not in requirements-test-base.txt)
|
||||
|
||||
---
|
||||
|
||||
### 2. test_dependency_chain_with_six_pin
|
||||
|
||||
**File**: `test_dependency_protection.py:117-162`
|
||||
|
||||
**Enhancements**:
|
||||
- ✅ Added exact version assertion for six==1.16.0
|
||||
- ✅ Added exact version assertion for python-dateutil==2.9.0.post0
|
||||
- ✅ Added detailed error messages
|
||||
- ✅ Added docstring reference to DEPENDENCY_TREE_CONTEXT.md
|
||||
|
||||
**Key Assertions Added**:
|
||||
```python
|
||||
# Verify expected OLD version
|
||||
assert initial_six == "1.16.0", f"Expected six==1.16.0, got {initial_six}"
|
||||
|
||||
# Verify final versions
|
||||
assert final_packages["python-dateutil"] == "2.9.0.post0", f"Expected python-dateutil==2.9.0.post0"
|
||||
assert final_packages["six"] == "1.16.0", "six should remain at 1.16.0 (prevented 1.17.0 upgrade)"
|
||||
```
|
||||
|
||||
**Based on Context**:
|
||||
- DEPENDENCY_TREE_CONTEXT.md Section 2: python-dateutil → Dependencies
|
||||
- Verified: six is a REAL dependency (not optional like colorama)
|
||||
- Verified: Without pin, six would upgrade from 1.16.0 to 1.17.0
|
||||
|
||||
---
|
||||
|
||||
### 3. test_pin_only_affects_specified_packages ✨ NEW
|
||||
|
||||
**File**: `test_dependency_protection.py:165-208`
|
||||
|
||||
**Purpose**: Verify that pin is selective, not global
|
||||
|
||||
**Test Logic**:
|
||||
1. Verify idna is NOT pre-installed
|
||||
2. Verify requests is NOT pre-installed
|
||||
3. Install requests with pin policy (only pins urllib3, certifi, charset-normalizer)
|
||||
4. Verify idna was installed at latest version (3.10) - NOT pinned
|
||||
5. Verify requests was installed at expected version (2.32.5)
|
||||
|
||||
**Key Assertions**:
|
||||
```python
|
||||
# Verify idna was installed (NOT pinned, so gets latest)
|
||||
assert "idna" in final_packages, "idna should be installed as new dependency"
|
||||
assert final_packages["idna"] == "3.10", "idna should be at latest version 3.10 (not pinned)"
|
||||
```
|
||||
|
||||
**Based on Context**:
|
||||
- DEPENDENCY_TREE_CONTEXT.md: "⚠️ idna is NEW and NOT pinned (acceptable - new dependency)"
|
||||
- Verified: Pin only affects specified packages in pinned_packages list
|
||||
|
||||
---
|
||||
|
||||
### 4. test_major_version_jump_prevention ✨ NEW
|
||||
|
||||
**File**: `test_dependency_protection.py:211-271`
|
||||
|
||||
**Purpose**: Verify that pin prevents MAJOR version jumps with breaking changes
|
||||
|
||||
**Test Logic**:
|
||||
1. Verify initial urllib3==1.26.15
|
||||
2. **Test WITHOUT pin**: Uninstall deps, install requests → urllib3 upgrades to 2.x
|
||||
3. Verify urllib3 was upgraded to 2.x (starts with "2.")
|
||||
4. Reset environment
|
||||
5. **Test WITH pin**: Install requests with pin → urllib3 stays at 1.x
|
||||
6. Verify urllib3 stayed at 1.26.15 (starts with "1.")
|
||||
|
||||
**Key Assertions**:
|
||||
```python
|
||||
# Without pin - verify urllib3 upgrades to 2.x
|
||||
assert without_pin["urllib3"].startswith("2."), \
|
||||
f"Without pin, urllib3 should upgrade to 2.x, got {without_pin['urllib3']}"
|
||||
|
||||
# With pin - verify urllib3 stays at 1.x
|
||||
assert final_packages["urllib3"] == "1.26.15", \
|
||||
"Pin should prevent urllib3 from upgrading to 2.x (breaking changes)"
|
||||
assert final_packages["urllib3"].startswith("1."), \
|
||||
f"urllib3 should remain at 1.x series, got {final_packages['urllib3']}"
|
||||
```
|
||||
|
||||
**Based on Context**:
|
||||
- DEPENDENCY_TREE_CONTEXT.md: "urllib3 1.26.15 → 2.5.0 is a MAJOR version jump"
|
||||
- DEPENDENCY_TREE_CONTEXT.md: "urllib3 2.0 removed deprecated APIs"
|
||||
- This is the MOST IMPORTANT test - prevents breaking changes
|
||||
|
||||
---
|
||||
|
||||
### 5. test_package_deletion_and_restore
|
||||
|
||||
**File**: `test_environment_recovery.py:33-78`
|
||||
|
||||
**Enhancements**:
|
||||
- ✅ Added exact version assertion for six==1.16.0
|
||||
- ✅ Added verification that six is restored to EXACT version (not latest)
|
||||
- ✅ Added detailed error messages
|
||||
- ✅ Added docstring reference to DEPENDENCY_TREE_CONTEXT.md
|
||||
|
||||
**Key Assertions Added**:
|
||||
```python
|
||||
# Verify six is initially installed at expected version
|
||||
assert initial["six"] == "1.16.0", f"Expected six==1.16.0, got {initial['six']}"
|
||||
|
||||
# Verify six was restored to EXACT required version (not latest)
|
||||
assert final_packages["six"] == "1.16.0", \
|
||||
"six should be restored to exact version 1.16.0 (not 1.17.0 latest)"
|
||||
```
|
||||
|
||||
**Based on Context**:
|
||||
- DEPENDENCY_TREE_CONTEXT.md: "six: 1.16.0 (OLD) → 1.17.0 (LATEST)"
|
||||
- Verified: Restore policy restores to EXACT version, not latest
|
||||
|
||||
---
|
||||
|
||||
### 6. test_version_change_and_restore
|
||||
|
||||
**File**: `test_environment_recovery.py:105-158`
|
||||
|
||||
**Enhancements**:
|
||||
- ✅ Added exact version assertions (1.26.15 initially, 2.1.0 after upgrade)
|
||||
- ✅ Added verification of major version change (1.x → 2.x)
|
||||
- ✅ Added verification of major version downgrade (2.x → 1.x)
|
||||
- ✅ Added detailed error messages explaining downgrade capability
|
||||
- ✅ Added docstring reference to DEPENDENCY_TREE_CONTEXT.md
|
||||
|
||||
**Key Assertions Added**:
|
||||
```python
|
||||
# Verify version was changed to 2.x
|
||||
assert installed_after["urllib3"] == "2.1.0", \
|
||||
f"urllib3 should be upgraded to 2.1.0, got {installed_after['urllib3']}"
|
||||
assert installed_after["urllib3"].startswith("2."), \
|
||||
"urllib3 should be at 2.x series"
|
||||
|
||||
# Verify version was DOWNGRADED from 2.x back to 1.x
|
||||
assert final["urllib3"] == "1.26.15", \
|
||||
"urllib3 should be downgraded to 1.26.15 (from 2.1.0)"
|
||||
assert final["urllib3"].startswith("1."), \
|
||||
f"urllib3 should be back at 1.x series, got {final['urllib3']}"
|
||||
```
|
||||
|
||||
**Based on Context**:
|
||||
- DEPENDENCY_TREE_CONTEXT.md: "urllib3 can upgrade from 1.26.15 (1.x) to 2.5.0 (2.x)"
|
||||
- Verified: Restore policy can DOWNGRADE (not just prevent upgrades)
|
||||
- Tests actual version downgrade capability (2.x → 1.x)
|
||||
|
||||
---
|
||||
|
||||
## Test Coverage Analysis
|
||||
|
||||
### Before Improvements
|
||||
|
||||
| Scenario | Coverage |
|
||||
|----------|----------|
|
||||
| Pin prevents upgrades | ✅ Basic |
|
||||
| New dependencies installed | ❌ Not tested |
|
||||
| Pin is selective | ❌ Not tested |
|
||||
| Major version jump prevention | ❌ Not tested |
|
||||
| Exact version restoration | ❌ Not tested |
|
||||
| Version downgrade capability | ❌ Not tested |
|
||||
|
||||
### After Improvements
|
||||
|
||||
| Scenario | Coverage | Test |
|
||||
|----------|----------|------|
|
||||
| Pin prevents upgrades | ✅ Enhanced | test_dependency_version_protection_with_pin |
|
||||
| New dependencies installed | ✅ Added | test_dependency_version_protection_with_pin |
|
||||
| Pin is selective | ✅ Added | test_pin_only_affects_specified_packages |
|
||||
| Major version jump prevention | ✅ Added | test_major_version_jump_prevention |
|
||||
| Exact version restoration | ✅ Enhanced | test_package_deletion_and_restore |
|
||||
| Version downgrade capability | ✅ Enhanced | test_version_change_and_restore |
|
||||
|
||||
---
|
||||
|
||||
## Key Testing Principles Applied
|
||||
|
||||
### 1. Exact Version Verification
|
||||
|
||||
**Before**:
|
||||
```python
|
||||
assert final_packages["urllib3"] == initial_urllib3 # Generic
|
||||
```
|
||||
|
||||
**After**:
|
||||
```python
|
||||
assert initial_urllib3 == "1.26.15", f"Expected urllib3==1.26.15, got {initial_urllib3}"
|
||||
assert final_packages["urllib3"] == "1.26.15", "urllib3 should remain at 1.26.15 (prevented 2.x upgrade)"
|
||||
```
|
||||
|
||||
**Benefit**: Fails with clear message if environment setup is wrong
|
||||
|
||||
---
|
||||
|
||||
### 2. Version Series Verification
|
||||
|
||||
**Added**:
|
||||
```python
|
||||
assert final_packages["urllib3"].startswith("1."), \
|
||||
f"urllib3 should remain at 1.x series, got {final_packages['urllib3']}"
|
||||
```
|
||||
|
||||
**Benefit**: Catches major version jumps even if exact version changes
|
||||
|
||||
---
|
||||
|
||||
### 3. Negative Testing (Verify NOT Installed)
|
||||
|
||||
**Added**:
|
||||
```python
|
||||
assert "idna" not in initial, "idna should not be pre-installed"
|
||||
```
|
||||
|
||||
**Benefit**: Ensures test environment is in expected state
|
||||
|
||||
---
|
||||
|
||||
### 4. Context-Based Documentation
|
||||
|
||||
**Every test now includes**:
|
||||
```python
|
||||
"""
|
||||
Based on DEPENDENCY_TREE_CONTEXT.md:
|
||||
<specific section reference>
|
||||
<expected behavior from context>
|
||||
"""
|
||||
```
|
||||
|
||||
**Benefit**: Links test expectations to verified dependency data
|
||||
|
||||
---
|
||||
|
||||
## Real-World Scenarios Tested
|
||||
|
||||
### Scenario 1: Preventing Breaking Changes
|
||||
|
||||
**Test**: `test_major_version_jump_prevention`
|
||||
|
||||
**Real-World Impact**:
|
||||
- urllib3 2.0 removed deprecated APIs
|
||||
- Many applications break when upgrading from 1.x to 2.x
|
||||
- Pin prevents this automatic breaking change
|
||||
|
||||
**Verified**: ✅ Pin successfully prevents 1.x → 2.x upgrade
|
||||
|
||||
---
|
||||
|
||||
### Scenario 2: Allowing New Dependencies
|
||||
|
||||
**Test**: `test_pin_only_affects_specified_packages`
|
||||
|
||||
**Real-World Impact**:
|
||||
- New dependencies are safe to add (idna)
|
||||
- Pin should not block ALL changes
|
||||
- Only specified packages are protected
|
||||
|
||||
**Verified**: ✅ idna installs at 3.10 even with pin policy active
|
||||
|
||||
---
|
||||
|
||||
### Scenario 3: Version Downgrade Recovery
|
||||
|
||||
**Test**: `test_version_change_and_restore`
|
||||
|
||||
**Real-World Impact**:
|
||||
- Sometimes packages get upgraded accidentally
|
||||
- Need to downgrade to known-good version
|
||||
- Downgrade is harder than upgrade prevention
|
||||
|
||||
**Verified**: ✅ Can downgrade urllib3 from 2.x to 1.x
|
||||
|
||||
---
|
||||
|
||||
## Test Execution Performance
|
||||
|
||||
```
|
||||
Test Performance Summary:
|
||||
|
||||
test_dependency_version_protection_with_pin 2.28s (enhanced)
|
||||
test_dependency_chain_with_six_pin 2.00s (enhanced)
|
||||
test_pin_only_affects_specified_packages 2.25s (NEW)
|
||||
test_major_version_jump_prevention 3.53s (NEW - does 2 install cycles)
|
||||
test_package_deletion_and_restore 2.25s (enhanced)
|
||||
test_version_change_and_restore 2.24s (enhanced)
|
||||
|
||||
Total: 14.10s for 6 tests
|
||||
Average: 2.35s per test
|
||||
```
|
||||
|
||||
**Note**: `test_major_version_jump_prevention` is slower because it tests both WITH and WITHOUT pin (2 install cycles).
|
||||
|
||||
---
|
||||
|
||||
## Files Modified
|
||||
|
||||
1. **test_dependency_protection.py**: +138 lines
|
||||
- Enhanced 2 existing tests
|
||||
- Added 2 new tests
|
||||
- Total: 272 lines (was 132 lines)
|
||||
|
||||
2. **test_environment_recovery.py**: +35 lines
|
||||
- Enhanced 2 existing tests
|
||||
- Total: 159 lines (was 141 lines)
|
||||
|
||||
---
|
||||
|
||||
## Verification Against Context
|
||||
|
||||
All test improvements verified against:
|
||||
|
||||
| Context Source | Usage |
|
||||
|----------------|-------|
|
||||
| **DEPENDENCY_TREE_CONTEXT.md** | All version numbers, dependency trees |
|
||||
| **DEPENDENCY_ANALYSIS.md** | Package selection rationale, rejected scenarios |
|
||||
| **TEST_SCENARIOS.md** | Scenario specifications, expected outcomes |
|
||||
| **requirements-test-base.txt** | Initial environment state |
|
||||
| **analyze_dependencies.py** | Real-time verification of expectations |
|
||||
|
||||
---
|
||||
|
||||
## Future Maintenance
|
||||
|
||||
### When to Update Tests
|
||||
|
||||
Update tests when:
|
||||
- ✅ PyPI releases new major versions (e.g., urllib3 3.0)
|
||||
- ✅ Base package versions change in requirements-test-base.txt
|
||||
- ✅ New test scenarios added to DEPENDENCY_TREE_CONTEXT.md
|
||||
- ✅ Policy behavior changes in pip_util.py
|
||||
|
||||
### How to Update Tests
|
||||
|
||||
1. Run `python analyze_dependencies.py --all`
|
||||
2. Update expected version numbers in tests
|
||||
3. Update DEPENDENCY_TREE_CONTEXT.md
|
||||
4. Update TEST_SCENARIOS.md
|
||||
5. Run tests to verify
|
||||
|
||||
### Verification Commands
|
||||
|
||||
```bash
|
||||
# Verify environment
|
||||
python analyze_dependencies.py --env
|
||||
|
||||
# Verify package dependencies
|
||||
python analyze_dependencies.py requests
|
||||
python analyze_dependencies.py python-dateutil
|
||||
|
||||
# Run all tests
|
||||
pytest test_dependency_protection.py test_environment_recovery.py -v --override-ini="addopts="
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
✅ **6 tests** now verify real PyPI package dependencies
|
||||
✅ **100% pass rate** with real pip operations
|
||||
✅ **All version numbers** verified against DEPENDENCY_TREE_CONTEXT.md
|
||||
✅ **Major version jump prevention** explicitly tested
|
||||
✅ **Selective pinning** verified (only specified packages)
|
||||
✅ **Version downgrade** capability tested
|
||||
|
||||
**Key Achievement**: Tests now verify actual PyPI behavior, not mocked expectations.
|
||||
573
tests/common/pip_util/TEST_SCENARIOS.md
Normal file
573
tests/common/pip_util/TEST_SCENARIOS.md
Normal file
@ -0,0 +1,573 @@
|
||||
# pip_util Test Scenarios - Test Data Specification
|
||||
|
||||
This document precisely defines all test scenarios, packages, versions, and expected behaviors used in the pip_util test suite.
|
||||
|
||||
## Table of Contents
|
||||
1. [Test Scenario 1: Dependency Version Protection](#scenario-1-dependency-version-protection)
|
||||
2. [Test Scenario 2: Complex Dependency Chain](#scenario-2-complex-dependency-chain)
|
||||
3. [Test Scenario 3: Package Deletion and Restore](#scenario-3-package-deletion-and-restore)
|
||||
4. [Test Scenario 4: Version Change and Restore](#scenario-4-version-change-and-restore)
|
||||
5. [Test Scenario 5: Full Workflow Integration](#scenario-5-full-workflow-integration)
|
||||
6. [Test Scenario 6: Pin Failure Retry](#scenario-6-pin-failure-retry)
|
||||
|
||||
---
|
||||
|
||||
## Scenario 1: Dependency Version Protection
|
||||
|
||||
**File**: `test_dependency_protection.py::test_dependency_version_protection_with_pin`
|
||||
|
||||
**Purpose**: Verify that `pin_dependencies` policy prevents dependency upgrades during package installation.
|
||||
|
||||
### Initial Environment State
|
||||
```python
|
||||
installed_packages = {
|
||||
"urllib3": "1.26.15", # OLD stable version
|
||||
"certifi": "2023.7.22", # OLD version
|
||||
"charset-normalizer": "3.2.0" # OLD version
|
||||
}
|
||||
```
|
||||
|
||||
### Policy Configuration
|
||||
```json
|
||||
{
|
||||
"requests": {
|
||||
"apply_all_matches": [
|
||||
{
|
||||
"type": "pin_dependencies",
|
||||
"pinned_packages": ["urllib3", "certifi", "charset-normalizer"],
|
||||
"on_failure": "retry_without_pin"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Action
|
||||
```python
|
||||
batch.install("requests")
|
||||
```
|
||||
|
||||
### Expected pip Command
|
||||
```bash
|
||||
pip install requests urllib3==1.26.15 certifi==2023.7.22 charset-normalizer==3.2.0
|
||||
```
|
||||
|
||||
### Expected Final State
|
||||
```python
|
||||
installed_packages = {
|
||||
"urllib3": "1.26.15", # PROTECTED - stayed at old version
|
||||
"certifi": "2023.7.22", # PROTECTED - stayed at old version
|
||||
"charset-normalizer": "3.2.0", # PROTECTED - stayed at old version
|
||||
"requests": "2.31.0" # NEWLY installed
|
||||
}
|
||||
```
|
||||
|
||||
### Without Pin (What Would Happen)
|
||||
```python
|
||||
# If pin_dependencies was NOT used:
|
||||
installed_packages = {
|
||||
"urllib3": "2.1.0", # UPGRADED to 2.x (breaking change)
|
||||
"certifi": "2024.2.2", # UPGRADED to latest
|
||||
"charset-normalizer": "3.3.2", # UPGRADED to latest
|
||||
"requests": "2.31.0"
|
||||
}
|
||||
```
|
||||
|
||||
**Key Point**: Pin prevents `urllib3` from upgrading to 2.x, which has breaking API changes.
|
||||
|
||||
---
|
||||
|
||||
## Scenario 2: Complex Dependency Chain
|
||||
|
||||
**File**: `test_dependency_protection.py::test_dependency_chain_with_click_colorama`
|
||||
|
||||
**Purpose**: Verify that `force_version` + `pin_dependencies` work together correctly.
|
||||
|
||||
### Initial Environment State
|
||||
```python
|
||||
installed_packages = {
|
||||
"colorama": "0.4.6" # Existing dependency
|
||||
}
|
||||
```
|
||||
|
||||
### Policy Configuration
|
||||
```json
|
||||
{
|
||||
"click": {
|
||||
"apply_first_match": [
|
||||
{
|
||||
"condition": {
|
||||
"type": "installed",
|
||||
"package": "colorama",
|
||||
"spec": "<0.5.0"
|
||||
},
|
||||
"type": "force_version",
|
||||
"version": "8.1.3",
|
||||
"reason": "click 8.1.3 compatible with colorama <0.5"
|
||||
}
|
||||
],
|
||||
"apply_all_matches": [
|
||||
{
|
||||
"type": "pin_dependencies",
|
||||
"pinned_packages": ["colorama"]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Condition Evaluation
|
||||
```python
|
||||
# Check: colorama installed AND version < 0.5.0?
|
||||
colorama_installed = True
|
||||
colorama_version = "0.4.6" # 0.4.6 < 0.5.0 → True
|
||||
# Result: Condition satisfied → apply force_version
|
||||
```
|
||||
|
||||
### Action
|
||||
```python
|
||||
batch.install("click")
|
||||
```
|
||||
|
||||
### Expected pip Command
|
||||
```bash
|
||||
pip install click==8.1.3 colorama==0.4.6
|
||||
```
|
||||
|
||||
### Expected Final State
|
||||
```python
|
||||
installed_packages = {
|
||||
"colorama": "0.4.6", # PINNED - version protected
|
||||
"click": "8.1.3" # FORCED to specific version
|
||||
}
|
||||
```
|
||||
|
||||
**Key Point**:
|
||||
- `force_version` forces click to install version 8.1.3
|
||||
- `pin_dependencies` ensures colorama stays at 0.4.6
|
||||
|
||||
---
|
||||
|
||||
## Scenario 3: Package Deletion and Restore
|
||||
|
||||
**File**: `test_environment_recovery.py::test_package_deletion_and_restore`
|
||||
|
||||
**Purpose**: Verify that deleted packages can be restored to required versions.
|
||||
|
||||
### Initial Environment State
|
||||
```python
|
||||
installed_packages = {
|
||||
"six": "1.16.0", # Critical package
|
||||
"attrs": "23.1.0",
|
||||
"packaging": "23.1"
|
||||
}
|
||||
```
|
||||
|
||||
### Policy Configuration
|
||||
```json
|
||||
{
|
||||
"six": {
|
||||
"restore": [
|
||||
{
|
||||
"target": "six",
|
||||
"version": "1.16.0",
|
||||
"reason": "six must be maintained at 1.16.0 for compatibility"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Action Sequence
|
||||
|
||||
**Step 1**: Install package that removes six
|
||||
```python
|
||||
batch.install("python-dateutil")
|
||||
```
|
||||
|
||||
**Step 1 Result**: six is DELETED
|
||||
```python
|
||||
installed_packages = {
|
||||
# "six": "1.16.0", # ❌ DELETED by python-dateutil
|
||||
"attrs": "23.1.0",
|
||||
"packaging": "23.1",
|
||||
"python-dateutil": "2.8.2" # ✅ NEW
|
||||
}
|
||||
```
|
||||
|
||||
**Step 2**: Restore deleted packages
|
||||
```python
|
||||
batch.ensure_installed()
|
||||
```
|
||||
|
||||
**Step 2 Result**: six is RESTORED
|
||||
```python
|
||||
installed_packages = {
|
||||
"six": "1.16.0", # ✅ RESTORED to required version
|
||||
"attrs": "23.1.0",
|
||||
"packaging": "23.1",
|
||||
"python-dateutil": "2.8.2"
|
||||
}
|
||||
```
|
||||
|
||||
### Expected pip Commands
|
||||
```bash
|
||||
# Step 1: Install
|
||||
pip install python-dateutil
|
||||
|
||||
# Step 2: Restore
|
||||
pip install six==1.16.0
|
||||
```
|
||||
|
||||
**Key Point**: `restore` policy automatically reinstalls deleted packages.
|
||||
|
||||
---
|
||||
|
||||
## Scenario 4: Version Change and Restore
|
||||
|
||||
**File**: `test_environment_recovery.py::test_version_change_and_restore`
|
||||
|
||||
**Purpose**: Verify that packages with changed versions can be restored to required versions.
|
||||
|
||||
### Initial Environment State
|
||||
```python
|
||||
installed_packages = {
|
||||
"urllib3": "1.26.15", # OLD version (required)
|
||||
"certifi": "2023.7.22"
|
||||
}
|
||||
```
|
||||
|
||||
### Policy Configuration
|
||||
```json
|
||||
{
|
||||
"urllib3": {
|
||||
"restore": [
|
||||
{
|
||||
"condition": {
|
||||
"type": "installed",
|
||||
"spec": "!=1.26.15"
|
||||
},
|
||||
"target": "urllib3",
|
||||
"version": "1.26.15",
|
||||
"reason": "urllib3 must be 1.26.15 for compatibility"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Action Sequence
|
||||
|
||||
**Step 1**: Install package that upgrades urllib3
|
||||
```python
|
||||
batch.install("requests")
|
||||
```
|
||||
|
||||
**Step 1 Result**: urllib3 is UPGRADED
|
||||
```python
|
||||
installed_packages = {
|
||||
"urllib3": "2.1.0", # ❌ UPGRADED from 1.26.15 to 2.1.0
|
||||
"certifi": "2023.7.22",
|
||||
"requests": "2.31.0" # ✅ NEW
|
||||
}
|
||||
```
|
||||
|
||||
**Step 2**: Check restore condition
|
||||
```python
|
||||
# Condition: urllib3 installed AND version != 1.26.15?
|
||||
urllib3_version = "2.1.0"
|
||||
condition_met = (urllib3_version != "1.26.15") # True
|
||||
# Result: Restore urllib3 to 1.26.15
|
||||
```
|
||||
|
||||
**Step 2**: Restore to required version
|
||||
```python
|
||||
batch.ensure_installed()
|
||||
```
|
||||
|
||||
**Step 2 Result**: urllib3 is DOWNGRADED
|
||||
```python
|
||||
installed_packages = {
|
||||
"urllib3": "1.26.15", # ✅ RESTORED to required version
|
||||
"certifi": "2023.7.22",
|
||||
"requests": "2.31.0"
|
||||
}
|
||||
```
|
||||
|
||||
### Expected pip Commands
|
||||
```bash
|
||||
# Step 1: Install (causes upgrade)
|
||||
pip install requests
|
||||
|
||||
# Step 2: Restore (downgrade)
|
||||
pip install urllib3==1.26.15
|
||||
```
|
||||
|
||||
**Key Point**: `restore` with condition can revert unwanted version changes.
|
||||
|
||||
---
|
||||
|
||||
## Scenario 5: Full Workflow Integration
|
||||
|
||||
**File**: `test_full_workflow_integration.py::test_uninstall_install_restore_workflow`
|
||||
|
||||
**Purpose**: Verify complete workflow: uninstall → install → restore.
|
||||
|
||||
### Initial Environment State
|
||||
```python
|
||||
installed_packages = {
|
||||
"old-package": "1.0.0", # To be removed
|
||||
"critical-package": "1.2.3", # To be restored
|
||||
"urllib3": "1.26.15",
|
||||
"certifi": "2023.7.22"
|
||||
}
|
||||
```
|
||||
|
||||
### Policy Configuration
|
||||
```json
|
||||
{
|
||||
"old-package": {
|
||||
"uninstall": [
|
||||
{
|
||||
"target": "old-package"
|
||||
}
|
||||
]
|
||||
},
|
||||
"requests": {
|
||||
"apply_all_matches": [
|
||||
{
|
||||
"type": "pin_dependencies",
|
||||
"pinned_packages": ["urllib3", "certifi"]
|
||||
}
|
||||
]
|
||||
},
|
||||
"critical-package": {
|
||||
"restore": [
|
||||
{
|
||||
"target": "critical-package",
|
||||
"version": "1.2.3"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Action Sequence
|
||||
|
||||
**Step 1**: Remove old packages
|
||||
```python
|
||||
removed = batch.ensure_not_installed()
|
||||
```
|
||||
|
||||
**Step 1 Result**:
|
||||
```python
|
||||
installed_packages = {
|
||||
# "old-package": "1.0.0", # ❌ REMOVED
|
||||
"critical-package": "1.2.3",
|
||||
"urllib3": "1.26.15",
|
||||
"certifi": "2023.7.22"
|
||||
}
|
||||
removed = ["old-package"]
|
||||
```
|
||||
|
||||
**Step 2**: Install new package with pins
|
||||
```python
|
||||
batch.install("requests")
|
||||
```
|
||||
|
||||
**Step 2 Result**:
|
||||
```python
|
||||
installed_packages = {
|
||||
"critical-package": "1.2.3",
|
||||
"urllib3": "1.26.15", # PINNED - no upgrade
|
||||
"certifi": "2023.7.22", # PINNED - no upgrade
|
||||
"requests": "2.31.0" # NEW
|
||||
}
|
||||
```
|
||||
|
||||
**Step 3**: Restore required packages
|
||||
```python
|
||||
restored = batch.ensure_installed()
|
||||
```
|
||||
|
||||
**Step 3 Result**:
|
||||
```python
|
||||
installed_packages = {
|
||||
"critical-package": "1.2.3", # Still present
|
||||
"urllib3": "1.26.15",
|
||||
"certifi": "2023.7.22",
|
||||
"requests": "2.31.0"
|
||||
}
|
||||
restored = [] # Nothing to restore (all present)
|
||||
```
|
||||
|
||||
### Expected pip Commands
|
||||
```bash
|
||||
# Step 1: Uninstall
|
||||
pip uninstall -y old-package
|
||||
|
||||
# Step 2: Install with pins
|
||||
pip install requests urllib3==1.26.15 certifi==2023.7.22
|
||||
|
||||
# Step 3: (No command - all packages present)
|
||||
```
|
||||
|
||||
**Key Point**: Complete workflow demonstrates policy coordination.
|
||||
|
||||
---
|
||||
|
||||
## Scenario 6: Pin Failure Retry
|
||||
|
||||
**File**: `test_pin_failure_retry.py::test_pin_failure_retry_without_pin_succeeds`
|
||||
|
||||
**Purpose**: Verify automatic retry without pins when installation with pins fails.
|
||||
|
||||
### Initial Environment State
|
||||
```python
|
||||
installed_packages = {
|
||||
"urllib3": "1.26.15",
|
||||
"certifi": "2023.7.22"
|
||||
}
|
||||
```
|
||||
|
||||
### Policy Configuration
|
||||
```json
|
||||
{
|
||||
"requests": {
|
||||
"apply_all_matches": [
|
||||
{
|
||||
"type": "pin_dependencies",
|
||||
"pinned_packages": ["urllib3", "certifi"],
|
||||
"on_failure": "retry_without_pin"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Action
|
||||
```python
|
||||
batch.install("requests")
|
||||
```
|
||||
|
||||
### Attempt 1: Install WITH pins (FAILS)
|
||||
```bash
|
||||
# Command:
|
||||
pip install requests urllib3==1.26.15 certifi==2023.7.22
|
||||
|
||||
# Result: FAILURE (dependency conflict)
|
||||
# Error: "Package conflict: requests requires urllib3>=2.0"
|
||||
```
|
||||
|
||||
### Attempt 2: Retry WITHOUT pins (SUCCEEDS)
|
||||
```bash
|
||||
# Command:
|
||||
pip install requests
|
||||
|
||||
# Result: SUCCESS
|
||||
```
|
||||
|
||||
**Final State**:
|
||||
```python
|
||||
installed_packages = {
|
||||
"urllib3": "2.1.0", # UPGRADED (pins removed)
|
||||
"certifi": "2024.2.2", # UPGRADED (pins removed)
|
||||
"requests": "2.31.0" # INSTALLED
|
||||
}
|
||||
```
|
||||
|
||||
### Expected Behavior
|
||||
1. **First attempt**: Install with pinned versions
|
||||
2. **On failure**: Log warning about conflict
|
||||
3. **Retry**: Install without pins
|
||||
4. **Success**: Package installed, dependencies upgraded
|
||||
|
||||
**Key Point**: `retry_without_pin` provides automatic fallback for compatibility issues.
|
||||
|
||||
---
|
||||
|
||||
## Scenario 6b: Pin Failure with Hard Fail
|
||||
|
||||
**File**: `test_pin_failure_retry.py::test_pin_failure_with_fail_raises_exception`
|
||||
|
||||
**Purpose**: Verify that `on_failure: fail` raises exception instead of retrying.
|
||||
|
||||
### Initial Environment State
|
||||
```python
|
||||
installed_packages = {
|
||||
"urllib3": "1.26.15",
|
||||
"certifi": "2023.7.22"
|
||||
}
|
||||
```
|
||||
|
||||
### Policy Configuration
|
||||
```json
|
||||
{
|
||||
"requests": {
|
||||
"apply_all_matches": [
|
||||
{
|
||||
"type": "pin_dependencies",
|
||||
"pinned_packages": ["urllib3", "certifi"],
|
||||
"on_failure": "fail"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Action
|
||||
```python
|
||||
batch.install("requests")
|
||||
```
|
||||
|
||||
### Attempt 1: Install WITH pins (FAILS)
|
||||
```bash
|
||||
# Command:
|
||||
pip install requests urllib3==1.26.15 certifi==2023.7.22
|
||||
|
||||
# Result: FAILURE (dependency conflict)
|
||||
# Error: "Package conflict: requests requires urllib3>=2.0"
|
||||
```
|
||||
|
||||
### Expected Behavior
|
||||
1. **First attempt**: Install with pinned versions
|
||||
2. **On failure**: Raise `subprocess.CalledProcessError`
|
||||
3. **No retry**: Exception propagates to caller
|
||||
4. **No changes**: Environment unchanged
|
||||
|
||||
**Key Point**: `on_failure: fail` ensures strict version requirements.
|
||||
|
||||
---
|
||||
|
||||
## Summary Table: All Test Packages
|
||||
|
||||
| Package | Initial Version | Action | Final Version | Role |
|
||||
|---------|----------------|--------|---------------|------|
|
||||
| **urllib3** | 1.26.15 | Pin | 1.26.15 | Protected dependency |
|
||||
| **certifi** | 2023.7.22 | Pin | 2023.7.22 | Protected dependency |
|
||||
| **charset-normalizer** | 3.2.0 | Pin | 3.2.0 | Protected dependency |
|
||||
| **requests** | (not installed) | Install | 2.31.0 | New package |
|
||||
| **colorama** | 0.4.6 | Pin | 0.4.6 | Protected dependency |
|
||||
| **click** | (not installed) | Force version | 8.1.3 | New package with forced version |
|
||||
| **six** | 1.16.0 | Delete→Restore | 1.16.0 | Deleted then restored |
|
||||
| **python-dateutil** | (not installed) | Install | 2.8.2 | Package that deletes six |
|
||||
| **attrs** | 23.1.0 | No change | 23.1.0 | Bystander package |
|
||||
| **packaging** | 23.1 | No change | 23.1 | Bystander package |
|
||||
|
||||
## Policy Types Summary
|
||||
|
||||
| Policy Type | Purpose | Example |
|
||||
|-------------|---------|---------|
|
||||
| **pin_dependencies** | Prevent dependency upgrades | Keep urllib3 at 1.26.15 |
|
||||
| **force_version** | Force specific package version | Install click==8.1.3 |
|
||||
| **restore** | Reinstall deleted/changed packages | Restore six to 1.16.0 |
|
||||
| **uninstall** | Remove obsolete packages | Remove old-package |
|
||||
| **on_failure** | Handle installation failures | retry_without_pin or fail |
|
||||
|
||||
## Test Data Design Principles
|
||||
|
||||
1. **Lightweight Packages**: All packages are <200KB for fast testing
|
||||
2. **Real Dependencies**: Use actual PyPI package relationships
|
||||
3. **Version Realism**: Use real version numbers from PyPI
|
||||
4. **Clear Scenarios**: Each test demonstrates one clear behavior
|
||||
5. **Reproducible**: Mock ensures consistent behavior across environments
|
||||
261
tests/common/pip_util/analyze_dependencies.py
Executable file
261
tests/common/pip_util/analyze_dependencies.py
Executable file
@ -0,0 +1,261 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Dependency Tree Analyzer for pip_util Tests
|
||||
|
||||
Usage:
|
||||
python analyze_dependencies.py [package]
|
||||
python analyze_dependencies.py --all
|
||||
python analyze_dependencies.py --update-context
|
||||
|
||||
Examples:
|
||||
python analyze_dependencies.py requests
|
||||
python analyze_dependencies.py python-dateutil
|
||||
python analyze_dependencies.py --all
|
||||
"""
|
||||
|
||||
import subprocess
|
||||
import sys
|
||||
from typing import Dict, List, Tuple, Optional
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
PIP = "./test_venv/bin/pip"
|
||||
|
||||
|
||||
def check_venv():
|
||||
"""Check if test venv exists"""
|
||||
if not Path(PIP).exists():
|
||||
print("❌ Test venv not found!")
|
||||
print(" Run: ./setup_test_env.sh")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def get_installed_packages() -> Dict[str, str]:
|
||||
"""Get currently installed packages"""
|
||||
result = subprocess.run(
|
||||
[PIP, "freeze"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True
|
||||
)
|
||||
|
||||
packages = {}
|
||||
for line in result.stdout.strip().split('\n'):
|
||||
if '==' in line:
|
||||
pkg, ver = line.split('==', 1)
|
||||
packages[pkg] = ver
|
||||
|
||||
return packages
|
||||
|
||||
|
||||
def analyze_package_dry_run(
|
||||
package: str,
|
||||
constraints: Optional[List[str]] = None
|
||||
) -> Tuple[List[Tuple[str, str]], Dict[str, str]]:
|
||||
"""
|
||||
Analyze what would be installed with --dry-run
|
||||
|
||||
Returns:
|
||||
- List of (package_name, version) tuples in install order
|
||||
- Dict of current_version → new_version for upgrades
|
||||
"""
|
||||
cmd = [PIP, "install", "--dry-run", "--ignore-installed", package]
|
||||
if constraints:
|
||||
cmd.extend(constraints)
|
||||
|
||||
result = subprocess.run(cmd, capture_output=True, text=True)
|
||||
|
||||
# Parse "Would install" line
|
||||
would_install = []
|
||||
for line in result.stdout.split('\n'):
|
||||
if 'Would install' in line:
|
||||
packages_str = line.split('Would install')[1].strip()
|
||||
for pkg_str in packages_str.split():
|
||||
parts = pkg_str.split('-', 1)
|
||||
if len(parts) == 2:
|
||||
would_install.append((parts[0], parts[1]))
|
||||
|
||||
# Check against current installed
|
||||
installed = get_installed_packages()
|
||||
changes = {}
|
||||
for pkg, new_ver in would_install:
|
||||
if pkg in installed:
|
||||
old_ver = installed[pkg]
|
||||
if old_ver != new_ver:
|
||||
changes[pkg] = (old_ver, new_ver)
|
||||
|
||||
return would_install, changes
|
||||
|
||||
|
||||
def get_available_versions(package: str, limit: int = 10) -> Tuple[str, List[str]]:
|
||||
"""
|
||||
Get available versions from PyPI
|
||||
|
||||
Returns:
|
||||
- Latest version
|
||||
- List of available versions (limited)
|
||||
"""
|
||||
result = subprocess.run(
|
||||
[PIP, "index", "versions", package],
|
||||
capture_output=True,
|
||||
text=True
|
||||
)
|
||||
|
||||
latest = None
|
||||
versions = []
|
||||
|
||||
for line in result.stdout.split('\n'):
|
||||
if 'LATEST:' in line:
|
||||
latest = line.split('LATEST:')[1].strip()
|
||||
elif 'Available versions:' in line:
|
||||
versions_str = line.split('Available versions:')[1].strip()
|
||||
versions = [v.strip() for v in versions_str.split(',')[:limit]]
|
||||
|
||||
return latest, versions
|
||||
|
||||
|
||||
def print_package_analysis(package: str, with_pin: bool = False):
|
||||
"""Print detailed analysis for a package"""
|
||||
print(f"\n{'='*80}")
|
||||
print(f"Package: {package}")
|
||||
print(f"{'='*80}")
|
||||
|
||||
installed = get_installed_packages()
|
||||
|
||||
# Get latest version
|
||||
latest, available = get_available_versions(package)
|
||||
if latest:
|
||||
print(f"\n📦 Latest version: {latest}")
|
||||
print(f"📋 Available versions: {', '.join(available[:5])}")
|
||||
|
||||
# Scenario 1: Without constraints
|
||||
print(f"\n🔍 Scenario A: Install without constraints")
|
||||
print(f" Command: pip install {package}")
|
||||
|
||||
would_install, changes = analyze_package_dry_run(package)
|
||||
|
||||
if would_install:
|
||||
print(f"\n Would install {len(would_install)} packages:")
|
||||
for pkg, ver in would_install:
|
||||
if pkg in changes:
|
||||
old_ver, new_ver = changes[pkg]
|
||||
print(f" • {pkg:25} {old_ver:15} → {new_ver:15} ⚠️ UPGRADE")
|
||||
elif pkg in installed:
|
||||
print(f" • {pkg:25} {ver:15} (already installed)")
|
||||
else:
|
||||
print(f" • {pkg:25} {ver:15} ✨ NEW")
|
||||
|
||||
# Scenario 2: With pin constraints (if dependencies exist)
|
||||
dependencies = [pkg for pkg, _ in would_install if pkg != package]
|
||||
if dependencies and with_pin:
|
||||
print(f"\n🔍 Scenario B: Install with pin constraints")
|
||||
|
||||
# Create pin constraints for all current dependencies
|
||||
constraints = []
|
||||
for dep in dependencies:
|
||||
if dep in installed:
|
||||
constraints.append(f"{dep}=={installed[dep]}")
|
||||
|
||||
if constraints:
|
||||
print(f" Command: pip install {package} {' '.join(constraints)}")
|
||||
|
||||
would_install_pinned, changes_pinned = analyze_package_dry_run(
|
||||
package, constraints
|
||||
)
|
||||
|
||||
print(f"\n Would install {len(would_install_pinned)} packages:")
|
||||
for pkg, ver in would_install_pinned:
|
||||
if pkg in constraints:
|
||||
print(f" • {pkg:25} {ver:15} 📌 PINNED")
|
||||
elif pkg in installed:
|
||||
print(f" • {pkg:25} {ver:15} (no change)")
|
||||
else:
|
||||
print(f" • {pkg:25} {ver:15} ✨ NEW")
|
||||
|
||||
# Show what was prevented
|
||||
prevented = set(changes.keys()) - set(changes_pinned.keys())
|
||||
if prevented:
|
||||
print(f"\n ✅ Pin prevented {len(prevented)} upgrade(s):")
|
||||
for pkg in prevented:
|
||||
old_ver, new_ver = changes[pkg]
|
||||
print(f" • {pkg:25} {old_ver:15} ❌→ {new_ver}")
|
||||
|
||||
|
||||
def analyze_all_test_packages():
|
||||
"""Analyze all packages used in tests"""
|
||||
print("="*80)
|
||||
print("ANALYZING ALL TEST PACKAGES")
|
||||
print("="*80)
|
||||
|
||||
test_packages = [
|
||||
("requests", True),
|
||||
("python-dateutil", True),
|
||||
]
|
||||
|
||||
for package, with_pin in test_packages:
|
||||
print_package_analysis(package, with_pin)
|
||||
|
||||
print(f"\n{'='*80}")
|
||||
print("ANALYSIS COMPLETE")
|
||||
print(f"{'='*80}")
|
||||
|
||||
|
||||
def print_current_environment():
|
||||
"""Print current test environment"""
|
||||
print("="*80)
|
||||
print("CURRENT TEST ENVIRONMENT")
|
||||
print("="*80)
|
||||
|
||||
installed = get_installed_packages()
|
||||
|
||||
print(f"\nTotal packages: {len(installed)}\n")
|
||||
|
||||
# Group by category
|
||||
test_packages = ["urllib3", "certifi", "charset-normalizer", "six", "attrs", "packaging"]
|
||||
framework = ["pytest", "iniconfig", "pluggy", "Pygments"]
|
||||
|
||||
print("Test packages:")
|
||||
for pkg in test_packages:
|
||||
if pkg in installed:
|
||||
print(f" {pkg:25} {installed[pkg]}")
|
||||
|
||||
print("\nTest framework:")
|
||||
for pkg in framework:
|
||||
if pkg in installed:
|
||||
print(f" {pkg:25} {installed[pkg]}")
|
||||
|
||||
other = set(installed.keys()) - set(test_packages) - set(framework)
|
||||
if other:
|
||||
print("\nOther packages:")
|
||||
for pkg in sorted(other):
|
||||
print(f" {pkg:25} {installed[pkg]}")
|
||||
|
||||
|
||||
def main():
|
||||
"""Main entry point"""
|
||||
check_venv()
|
||||
|
||||
if len(sys.argv) == 1:
|
||||
print("Usage: python analyze_dependencies.py [package|--all|--env]")
|
||||
print("\nExamples:")
|
||||
print(" python analyze_dependencies.py requests")
|
||||
print(" python analyze_dependencies.py --all")
|
||||
print(" python analyze_dependencies.py --env")
|
||||
sys.exit(0)
|
||||
|
||||
command = sys.argv[1]
|
||||
|
||||
if command == "--all":
|
||||
analyze_all_test_packages()
|
||||
elif command == "--env":
|
||||
print_current_environment()
|
||||
elif command.startswith("--"):
|
||||
print(f"Unknown option: {command}")
|
||||
sys.exit(1)
|
||||
else:
|
||||
# Analyze specific package
|
||||
print_package_analysis(command, with_pin=True)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
387
tests/common/pip_util/conftest.py
Normal file
387
tests/common/pip_util/conftest.py
Normal file
@ -0,0 +1,387 @@
|
||||
"""
|
||||
pytest configuration and shared fixtures for pip_util.py tests
|
||||
|
||||
This file provides common fixtures and configuration for all tests.
|
||||
Uses real isolated venv for actual pip operations.
|
||||
"""
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Dict, List
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Test venv Management
|
||||
# =============================================================================
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def test_venv_path():
|
||||
"""
|
||||
Get path to test venv (must be created by setup_test_env.sh)
|
||||
|
||||
Returns:
|
||||
Path: Path to test venv directory
|
||||
"""
|
||||
venv_path = Path(__file__).parent / "test_venv"
|
||||
if not venv_path.exists():
|
||||
pytest.fail(
|
||||
f"Test venv not found at {venv_path}.\n"
|
||||
"Please run: ./setup_test_env.sh"
|
||||
)
|
||||
return venv_path
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def test_pip_cmd(test_venv_path):
|
||||
"""
|
||||
Get pip command for test venv
|
||||
|
||||
Returns:
|
||||
List[str]: pip command prefix for subprocess
|
||||
"""
|
||||
pip_path = test_venv_path / "bin" / "pip"
|
||||
if not pip_path.exists():
|
||||
pytest.fail(f"pip not found at {pip_path}")
|
||||
return [str(pip_path)]
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def reset_test_venv(test_pip_cmd):
|
||||
"""
|
||||
Reset test venv to initial state before each test
|
||||
|
||||
This fixture:
|
||||
1. Records current installed packages
|
||||
2. Yields control to test
|
||||
3. Restores original packages after test
|
||||
"""
|
||||
# Get initial state
|
||||
result = subprocess.run(
|
||||
test_pip_cmd + ["freeze"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True
|
||||
)
|
||||
initial_packages = result.stdout.strip()
|
||||
|
||||
yield
|
||||
|
||||
# Restore initial state
|
||||
# Uninstall everything except pip, setuptools, wheel
|
||||
result = subprocess.run(
|
||||
test_pip_cmd + ["freeze"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True
|
||||
)
|
||||
current_packages = result.stdout.strip()
|
||||
|
||||
if current_packages:
|
||||
packages_to_remove = []
|
||||
for line in current_packages.split('\n'):
|
||||
if line and '==' in line:
|
||||
pkg = line.split('==')[0].lower()
|
||||
if pkg not in ['pip', 'setuptools', 'wheel']:
|
||||
packages_to_remove.append(pkg)
|
||||
|
||||
if packages_to_remove:
|
||||
subprocess.run(
|
||||
test_pip_cmd + ["uninstall", "-y"] + packages_to_remove,
|
||||
capture_output=True,
|
||||
check=False # Don't fail if package doesn't exist
|
||||
)
|
||||
|
||||
# Reinstall initial packages
|
||||
if initial_packages:
|
||||
# Create temporary requirements file
|
||||
import tempfile
|
||||
with tempfile.NamedTemporaryFile(mode='w', suffix='.txt', delete=False) as f:
|
||||
f.write(initial_packages)
|
||||
temp_req = f.name
|
||||
|
||||
try:
|
||||
subprocess.run(
|
||||
test_pip_cmd + ["install", "-r", temp_req],
|
||||
capture_output=True,
|
||||
check=True
|
||||
)
|
||||
finally:
|
||||
Path(temp_req).unlink()
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Directory and Path Fixtures
|
||||
# =============================================================================
|
||||
|
||||
@pytest.fixture
|
||||
def temp_policy_dir(tmp_path):
|
||||
"""
|
||||
Create temporary directory for policy files
|
||||
|
||||
Returns:
|
||||
Path: Temporary directory for storing test policy files
|
||||
"""
|
||||
policy_dir = tmp_path / "policies"
|
||||
policy_dir.mkdir()
|
||||
return policy_dir
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def temp_user_policy_dir(tmp_path):
|
||||
"""
|
||||
Create temporary directory for user policy files
|
||||
|
||||
Returns:
|
||||
Path: Temporary directory for storing user policy files
|
||||
"""
|
||||
user_dir = tmp_path / "user_policies"
|
||||
user_dir.mkdir()
|
||||
return user_dir
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Module Setup and Mocking
|
||||
# =============================================================================
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def setup_pip_util(monkeypatch, test_pip_cmd):
|
||||
"""
|
||||
Setup pip_util module for testing with real venv
|
||||
|
||||
This fixture:
|
||||
1. Mocks comfy module (not needed for tests)
|
||||
2. Adds comfyui_manager to path
|
||||
3. Patches make_pip_cmd to use test venv
|
||||
4. Resets policy cache
|
||||
"""
|
||||
# Mock comfy module before importing anything
|
||||
comfy_mock = MagicMock()
|
||||
cli_args_mock = MagicMock()
|
||||
cli_args_mock.args = MagicMock()
|
||||
comfy_mock.cli_args = cli_args_mock
|
||||
sys.modules['comfy'] = comfy_mock
|
||||
sys.modules['comfy.cli_args'] = cli_args_mock
|
||||
|
||||
# Add comfyui_manager parent to path so relative imports work
|
||||
comfyui_manager_path = str(Path(__file__).parent.parent.parent.parent)
|
||||
if comfyui_manager_path not in sys.path:
|
||||
sys.path.insert(0, comfyui_manager_path)
|
||||
|
||||
# Import pip_util
|
||||
from comfyui_manager.common import pip_util
|
||||
|
||||
# Patch make_pip_cmd to use test venv pip
|
||||
def make_test_pip_cmd(args: List[str]) -> List[str]:
|
||||
return test_pip_cmd + args
|
||||
|
||||
monkeypatch.setattr(
|
||||
pip_util.manager_util,
|
||||
"make_pip_cmd",
|
||||
make_test_pip_cmd
|
||||
)
|
||||
|
||||
# Reset policy cache
|
||||
pip_util._pip_policy_cache = None
|
||||
|
||||
yield
|
||||
|
||||
# Cleanup
|
||||
pip_util._pip_policy_cache = None
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_manager_util(monkeypatch, temp_policy_dir):
|
||||
"""
|
||||
Mock manager_util module paths
|
||||
|
||||
Args:
|
||||
monkeypatch: pytest monkeypatch fixture
|
||||
temp_policy_dir: Temporary policy directory
|
||||
"""
|
||||
from comfyui_manager.common import pip_util
|
||||
|
||||
monkeypatch.setattr(
|
||||
pip_util.manager_util,
|
||||
"comfyui_manager_path",
|
||||
str(temp_policy_dir)
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_context(monkeypatch, temp_user_policy_dir):
|
||||
"""
|
||||
Mock context module paths
|
||||
|
||||
Args:
|
||||
monkeypatch: pytest monkeypatch fixture
|
||||
temp_user_policy_dir: Temporary user policy directory
|
||||
"""
|
||||
from comfyui_manager.common import pip_util
|
||||
|
||||
monkeypatch.setattr(
|
||||
pip_util.context,
|
||||
"manager_files_path",
|
||||
str(temp_user_policy_dir)
|
||||
)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Platform Mocking Fixtures
|
||||
# =============================================================================
|
||||
|
||||
@pytest.fixture
|
||||
def mock_platform_linux(monkeypatch):
|
||||
"""Mock platform.system() to return 'Linux'"""
|
||||
monkeypatch.setattr("platform.system", lambda: "Linux")
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_platform_windows(monkeypatch):
|
||||
"""Mock platform.system() to return 'Windows'"""
|
||||
monkeypatch.setattr("platform.system", lambda: "Windows")
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_platform_darwin(monkeypatch):
|
||||
"""Mock platform.system() to return 'Darwin' (macOS)"""
|
||||
monkeypatch.setattr("platform.system", lambda: "Darwin")
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_torch_cuda_available(monkeypatch):
|
||||
"""Mock torch.cuda.is_available() to return True"""
|
||||
class MockCuda:
|
||||
@staticmethod
|
||||
def is_available():
|
||||
return True
|
||||
|
||||
class MockTorch:
|
||||
cuda = MockCuda()
|
||||
|
||||
import sys
|
||||
monkeypatch.setitem(sys.modules, "torch", MockTorch())
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_torch_cuda_unavailable(monkeypatch):
|
||||
"""Mock torch.cuda.is_available() to return False"""
|
||||
class MockCuda:
|
||||
@staticmethod
|
||||
def is_available():
|
||||
return False
|
||||
|
||||
class MockTorch:
|
||||
cuda = MockCuda()
|
||||
|
||||
import sys
|
||||
monkeypatch.setitem(sys.modules, "torch", MockTorch())
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_torch_not_installed(monkeypatch):
|
||||
"""Mock torch as not installed (ImportError)"""
|
||||
import sys
|
||||
if "torch" in sys.modules:
|
||||
monkeypatch.delitem(sys.modules, "torch")
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Helper Functions
|
||||
# =============================================================================
|
||||
|
||||
@pytest.fixture
|
||||
def get_installed_packages(test_pip_cmd):
|
||||
"""
|
||||
Helper to get currently installed packages in test venv
|
||||
|
||||
Returns:
|
||||
Callable that returns Dict[str, str] of installed packages
|
||||
"""
|
||||
def _get_installed() -> Dict[str, str]:
|
||||
result = subprocess.run(
|
||||
test_pip_cmd + ["freeze"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True
|
||||
)
|
||||
|
||||
packages = {}
|
||||
for line in result.stdout.strip().split('\n'):
|
||||
if line and '==' in line:
|
||||
pkg, ver = line.split('==', 1)
|
||||
packages[pkg] = ver
|
||||
|
||||
return packages
|
||||
|
||||
return _get_installed
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def install_packages(test_pip_cmd):
|
||||
"""
|
||||
Helper to install packages in test venv
|
||||
|
||||
Returns:
|
||||
Callable that installs packages
|
||||
"""
|
||||
def _install(*packages):
|
||||
subprocess.run(
|
||||
test_pip_cmd + ["install"] + list(packages),
|
||||
capture_output=True,
|
||||
check=True
|
||||
)
|
||||
|
||||
return _install
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def uninstall_packages(test_pip_cmd):
|
||||
"""
|
||||
Helper to uninstall packages in test venv
|
||||
|
||||
Returns:
|
||||
Callable that uninstalls packages
|
||||
"""
|
||||
def _uninstall(*packages):
|
||||
subprocess.run(
|
||||
test_pip_cmd + ["uninstall", "-y"] + list(packages),
|
||||
capture_output=True,
|
||||
check=False # Don't fail if package doesn't exist
|
||||
)
|
||||
|
||||
return _uninstall
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Test Data Factories
|
||||
# =============================================================================
|
||||
|
||||
@pytest.fixture
|
||||
def make_policy():
|
||||
"""
|
||||
Factory fixture for creating policy dictionaries
|
||||
|
||||
Returns:
|
||||
Callable that creates policy dict from parameters
|
||||
"""
|
||||
def _make_policy(
|
||||
package_name: str,
|
||||
policy_type: str,
|
||||
section: str = "apply_first_match",
|
||||
**kwargs
|
||||
) -> Dict:
|
||||
policy_item = {"type": policy_type}
|
||||
policy_item.update(kwargs)
|
||||
|
||||
return {
|
||||
package_name: {
|
||||
section: [policy_item]
|
||||
}
|
||||
}
|
||||
|
||||
return _make_policy
|
||||
52
tests/common/pip_util/pytest.ini
Normal file
52
tests/common/pip_util/pytest.ini
Normal file
@ -0,0 +1,52 @@
|
||||
[pytest]
|
||||
# pytest configuration for pip_util.py tests
|
||||
|
||||
# Test discovery
|
||||
testpaths = .
|
||||
|
||||
# Markers
|
||||
markers =
|
||||
unit: Unit tests for individual functions
|
||||
integration: Integration tests for workflows
|
||||
e2e: End-to-end tests for complete scenarios
|
||||
|
||||
# Output options - extend global config
|
||||
addopts =
|
||||
# Coverage options for pip_util
|
||||
--cov=../../../comfyui_manager/common/pip_util
|
||||
--cov-report=html:htmlcov_pip_util
|
||||
--cov-report=term-missing
|
||||
--cov-report=xml:coverage_pip_util.xml
|
||||
# Coverage fail threshold
|
||||
--cov-fail-under=80
|
||||
|
||||
# Coverage configuration
|
||||
[coverage:run]
|
||||
source = ../../../comfyui_manager/common
|
||||
omit =
|
||||
*/tests/*
|
||||
*/test_*.py
|
||||
*/__pycache__/*
|
||||
*/test_venv/*
|
||||
|
||||
[coverage:report]
|
||||
precision = 2
|
||||
show_missing = True
|
||||
skip_covered = False
|
||||
|
||||
exclude_lines =
|
||||
# Standard pragma
|
||||
pragma: no cover
|
||||
# Don't complain about missing debug code
|
||||
def __repr__
|
||||
# Don't complain if tests don't hit defensive assertion code
|
||||
raise AssertionError
|
||||
raise NotImplementedError
|
||||
# Don't complain if non-runnable code isn't run
|
||||
if __name__ == .__main__.:
|
||||
# Don't complain about abstract methods
|
||||
@abstractmethod
|
||||
|
||||
[coverage:html]
|
||||
directory = htmlcov
|
||||
|
||||
20
tests/common/pip_util/requirements-test-base.txt
Normal file
20
tests/common/pip_util/requirements-test-base.txt
Normal file
@ -0,0 +1,20 @@
|
||||
# Base packages for pip_util integration tests
|
||||
# These packages are installed initially to test various scenarios
|
||||
# All versions verified using: pip install --dry-run --ignore-installed
|
||||
|
||||
# Scenario 1: Dependency Version Protection (requests + urllib3)
|
||||
# Purpose: Pin prevents urllib3 1.26.15 → 2.5.0 major upgrade
|
||||
urllib3==1.26.15 # OLD stable version (prevent 2.x upgrade)
|
||||
certifi==2023.7.22 # OLD version (prevent 2025.x upgrade)
|
||||
charset-normalizer==3.2.0 # OLD version (prevent 3.4.x upgrade)
|
||||
# Note: idna is NOT pre-installed (will be added by requests)
|
||||
|
||||
# Scenario 2: Package Deletion and Restore (six)
|
||||
# Purpose: Restore policy reinstalls deleted packages
|
||||
six==1.16.0 # Will be deleted and restored to 1.16.0
|
||||
attrs==23.1.0 # Bystander package
|
||||
packaging==23.1 # Bystander package (NOT 23.1.0, not 25.0)
|
||||
|
||||
# Scenario 3: Version Change and Restore (urllib3)
|
||||
# Purpose: Restore policy reverts version changes
|
||||
# urllib3==1.26.15 (same as Scenario 1, will be upgraded to 2.5.0 then restored)
|
||||
47
tests/common/pip_util/setup_test_env.sh
Executable file
47
tests/common/pip_util/setup_test_env.sh
Executable file
@ -0,0 +1,47 @@
|
||||
#!/bin/bash
|
||||
# Setup script for pip_util integration tests
|
||||
# Creates a test venv and installs base packages
|
||||
|
||||
set -e # Exit on error
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
VENV_DIR="$SCRIPT_DIR/test_venv"
|
||||
|
||||
echo "Setting up test environment for pip_util integration tests..."
|
||||
|
||||
# Remove existing venv if present
|
||||
if [ -d "$VENV_DIR" ]; then
|
||||
echo "Removing existing test venv..."
|
||||
rm -rf "$VENV_DIR"
|
||||
fi
|
||||
|
||||
# Create new venv
|
||||
echo "Creating test venv at $VENV_DIR..."
|
||||
python3 -m venv "$VENV_DIR"
|
||||
|
||||
# Activate venv
|
||||
source "$VENV_DIR/bin/activate"
|
||||
|
||||
# Upgrade pip
|
||||
echo "Upgrading pip..."
|
||||
pip install --upgrade pip
|
||||
|
||||
# Install pytest
|
||||
echo "Installing pytest..."
|
||||
pip install pytest
|
||||
|
||||
# Install base test packages
|
||||
echo "Installing base test packages..."
|
||||
pip install -r "$SCRIPT_DIR/requirements-test-base.txt"
|
||||
|
||||
echo ""
|
||||
echo "Test environment setup complete!"
|
||||
echo "Installed packages:"
|
||||
pip freeze
|
||||
|
||||
echo ""
|
||||
echo "To activate the test venv, run:"
|
||||
echo " source $VENV_DIR/bin/activate"
|
||||
echo ""
|
||||
echo "To run tests:"
|
||||
echo " pytest -v"
|
||||
271
tests/common/pip_util/test_dependency_protection.py
Normal file
271
tests/common/pip_util/test_dependency_protection.py
Normal file
@ -0,0 +1,271 @@
|
||||
"""
|
||||
Test dependency version protection with pin (Priority 1)
|
||||
|
||||
Tests that existing dependency versions are protected by pin_dependencies policy
|
||||
"""
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def pin_policy(temp_policy_dir):
|
||||
"""Create policy with pin_dependencies for lightweight real packages"""
|
||||
policy_content = {
|
||||
"requests": {
|
||||
"apply_all_matches": [
|
||||
{
|
||||
"type": "pin_dependencies",
|
||||
"pinned_packages": ["urllib3", "certifi", "charset-normalizer"],
|
||||
"on_failure": "retry_without_pin"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
policy_file = temp_policy_dir / "pip-policy.json"
|
||||
policy_file.write_text(json.dumps(policy_content, indent=2))
|
||||
return policy_file
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
def test_dependency_version_protection_with_pin(
|
||||
pin_policy,
|
||||
mock_manager_util,
|
||||
mock_context,
|
||||
reset_test_venv,
|
||||
get_installed_packages
|
||||
):
|
||||
"""
|
||||
Test existing dependency versions are protected by pin
|
||||
|
||||
Priority: 1 (Essential)
|
||||
|
||||
Purpose:
|
||||
Verify that when installing a package that would normally upgrade
|
||||
dependencies, the pin_dependencies policy protects existing versions.
|
||||
|
||||
Based on DEPENDENCY_TREE_CONTEXT.md:
|
||||
Without pin: urllib3 1.26.15 → 2.5.0 (MAJOR upgrade)
|
||||
With pin: urllib3 stays at 1.26.15 (protected)
|
||||
"""
|
||||
from comfyui_manager.common.pip_util import PipBatch
|
||||
|
||||
# Verify initial packages are installed (from requirements-test-base.txt)
|
||||
initial = get_installed_packages()
|
||||
assert "urllib3" in initial
|
||||
assert "certifi" in initial
|
||||
assert "charset-normalizer" in initial
|
||||
|
||||
# Record initial versions (from DEPENDENCY_TREE_CONTEXT.md)
|
||||
initial_urllib3 = initial["urllib3"]
|
||||
initial_certifi = initial["certifi"]
|
||||
initial_charset = initial["charset-normalizer"]
|
||||
|
||||
# Verify expected OLD versions
|
||||
assert initial_urllib3 == "1.26.15", f"Expected urllib3==1.26.15, got {initial_urllib3}"
|
||||
assert initial_certifi == "2023.7.22", f"Expected certifi==2023.7.22, got {initial_certifi}"
|
||||
assert initial_charset == "3.2.0", f"Expected charset-normalizer==3.2.0, got {initial_charset}"
|
||||
|
||||
# Verify idna is NOT installed initially
|
||||
assert "idna" not in initial, "idna should not be pre-installed"
|
||||
|
||||
with PipBatch() as batch:
|
||||
result = batch.install("requests")
|
||||
final_packages = batch._get_installed_packages()
|
||||
|
||||
# Verify installation succeeded
|
||||
assert result is True
|
||||
assert "requests" in final_packages
|
||||
|
||||
# Verify versions were maintained (not upgraded to latest)
|
||||
# Without pin, these would upgrade to: urllib3==2.5.0, certifi==2025.8.3, charset-normalizer==3.4.3
|
||||
assert final_packages["urllib3"] == "1.26.15", "urllib3 should remain at 1.26.15 (prevented 2.x upgrade)"
|
||||
assert final_packages["certifi"] == "2023.7.22", "certifi should remain at 2023.7.22 (prevented 2025.x upgrade)"
|
||||
assert final_packages["charset-normalizer"] == "3.2.0", "charset-normalizer should remain at 3.2.0"
|
||||
|
||||
# Verify new dependency was added (idna is NOT pinned, so it gets installed)
|
||||
assert "idna" in final_packages, "idna should be installed as new dependency"
|
||||
assert final_packages["idna"] == "3.10", f"Expected idna==3.10, got {final_packages['idna']}"
|
||||
|
||||
# Verify requests was installed at expected version
|
||||
assert final_packages["requests"] == "2.32.5", f"Expected requests==2.32.5, got {final_packages['requests']}"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def python_dateutil_policy(temp_policy_dir):
|
||||
"""Create policy for python-dateutil with six pinning"""
|
||||
policy_content = {
|
||||
"python-dateutil": {
|
||||
"apply_all_matches": [
|
||||
{
|
||||
"type": "pin_dependencies",
|
||||
"pinned_packages": ["six"],
|
||||
"reason": "Protect six from upgrading"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
policy_file = temp_policy_dir / "pip-policy.json"
|
||||
policy_file.write_text(json.dumps(policy_content, indent=2))
|
||||
return policy_file
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
def test_dependency_chain_with_six_pin(
|
||||
python_dateutil_policy,
|
||||
mock_manager_util,
|
||||
mock_context,
|
||||
reset_test_venv,
|
||||
get_installed_packages
|
||||
):
|
||||
"""
|
||||
Test python-dateutil + six dependency chain with pin
|
||||
|
||||
Priority: 2 (Important)
|
||||
|
||||
Purpose:
|
||||
Verify that pin_dependencies protects actual dependencies
|
||||
(six is a real dependency of python-dateutil).
|
||||
|
||||
Based on DEPENDENCY_TREE_CONTEXT.md:
|
||||
python-dateutil depends on six>=1.5
|
||||
Without pin: six 1.16.0 → 1.17.0
|
||||
With pin: six stays at 1.16.0 (protected)
|
||||
"""
|
||||
from comfyui_manager.common.pip_util import PipBatch
|
||||
|
||||
# Verify six is installed
|
||||
initial = get_installed_packages()
|
||||
assert "six" in initial
|
||||
initial_six = initial["six"]
|
||||
|
||||
# Verify expected OLD version
|
||||
assert initial_six == "1.16.0", f"Expected six==1.16.0, got {initial_six}"
|
||||
|
||||
with PipBatch() as batch:
|
||||
result = batch.install("python-dateutil")
|
||||
final_packages = batch._get_installed_packages()
|
||||
|
||||
# Verify installation succeeded
|
||||
assert result is True
|
||||
|
||||
# Verify final versions
|
||||
assert "python-dateutil" in final_packages
|
||||
assert final_packages["python-dateutil"] == "2.9.0.post0", f"Expected python-dateutil==2.9.0.post0"
|
||||
|
||||
# Verify six was NOT upgraded (without pin, would upgrade to 1.17.0)
|
||||
assert "six" in final_packages
|
||||
assert final_packages["six"] == "1.16.0", "six should remain at 1.16.0 (prevented 1.17.0 upgrade)"
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
def test_pin_only_affects_specified_packages(
|
||||
pin_policy,
|
||||
mock_manager_util,
|
||||
mock_context,
|
||||
reset_test_venv,
|
||||
get_installed_packages
|
||||
):
|
||||
"""
|
||||
Test that pin only affects specified packages, not all dependencies
|
||||
|
||||
Priority: 1 (Essential)
|
||||
|
||||
Purpose:
|
||||
Verify that idna (new dependency) is installed even though
|
||||
other dependencies are pinned. This tests that pin is selective,
|
||||
not global.
|
||||
|
||||
Based on DEPENDENCY_TREE_CONTEXT.md:
|
||||
idna is a NEW dependency (not in initial environment)
|
||||
Pin only affects: urllib3, certifi, charset-normalizer
|
||||
idna should be installed at latest version (3.10)
|
||||
"""
|
||||
from comfyui_manager.common.pip_util import PipBatch
|
||||
|
||||
# Verify initial state
|
||||
initial = get_installed_packages()
|
||||
assert "idna" not in initial, "idna should not be pre-installed"
|
||||
assert "requests" not in initial, "requests should not be pre-installed"
|
||||
|
||||
with PipBatch() as batch:
|
||||
result = batch.install("requests")
|
||||
final_packages = batch._get_installed_packages()
|
||||
|
||||
# Verify installation succeeded
|
||||
assert result is True
|
||||
|
||||
# Verify idna was installed (NOT pinned, so gets latest)
|
||||
assert "idna" in final_packages, "idna should be installed as new dependency"
|
||||
assert final_packages["idna"] == "3.10", "idna should be at latest version 3.10 (not pinned)"
|
||||
|
||||
# Verify requests was installed
|
||||
assert "requests" in final_packages
|
||||
assert final_packages["requests"] == "2.32.5"
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
def test_major_version_jump_prevention(
|
||||
pin_policy,
|
||||
mock_manager_util,
|
||||
mock_context,
|
||||
reset_test_venv,
|
||||
get_installed_packages,
|
||||
install_packages,
|
||||
uninstall_packages
|
||||
):
|
||||
"""
|
||||
Test that pin prevents MAJOR version jumps (breaking changes)
|
||||
|
||||
Priority: 1 (Essential)
|
||||
|
||||
Purpose:
|
||||
Verify that pin prevents urllib3 1.x → 2.x major upgrade.
|
||||
This is the most important test because urllib3 2.0 has
|
||||
breaking API changes.
|
||||
|
||||
Based on DEPENDENCY_TREE_CONTEXT.md:
|
||||
urllib3 1.26.15 → 2.5.0 is a MAJOR version jump
|
||||
urllib3 2.0 removed deprecated APIs
|
||||
requests accepts both: urllib3<3,>=1.21.1
|
||||
"""
|
||||
from comfyui_manager.common.pip_util import PipBatch
|
||||
|
||||
# Verify initial urllib3 version
|
||||
initial = get_installed_packages()
|
||||
assert initial["urllib3"] == "1.26.15", "Expected urllib3==1.26.15"
|
||||
|
||||
# First, test WITHOUT pin to verify urllib3 would upgrade to 2.x
|
||||
# (This simulates what would happen without our protection)
|
||||
uninstall_packages("urllib3", "certifi", "charset-normalizer")
|
||||
install_packages("requests")
|
||||
|
||||
without_pin = get_installed_packages()
|
||||
|
||||
# Verify urllib3 was upgraded to 2.x without pin
|
||||
assert "urllib3" in without_pin
|
||||
assert without_pin["urllib3"].startswith("2."), \
|
||||
f"Without pin, urllib3 should upgrade to 2.x, got {without_pin['urllib3']}"
|
||||
|
||||
# Now reset and test WITH pin
|
||||
uninstall_packages("requests", "urllib3", "certifi", "charset-normalizer", "idna")
|
||||
install_packages("urllib3==1.26.15", "certifi==2023.7.22", "charset-normalizer==3.2.0")
|
||||
|
||||
with PipBatch() as batch:
|
||||
result = batch.install("requests")
|
||||
final_packages = batch._get_installed_packages()
|
||||
|
||||
# Verify installation succeeded
|
||||
assert result is True
|
||||
|
||||
# Verify urllib3 stayed at 1.x (prevented major version jump)
|
||||
assert final_packages["urllib3"] == "1.26.15", \
|
||||
"Pin should prevent urllib3 from upgrading to 2.x (breaking changes)"
|
||||
|
||||
# Verify it's specifically 1.x, not 2.x
|
||||
assert final_packages["urllib3"].startswith("1."), \
|
||||
f"urllib3 should remain at 1.x series, got {final_packages['urllib3']}"
|
||||
279
tests/common/pip_util/test_edge_cases.py
Normal file
279
tests/common/pip_util/test_edge_cases.py
Normal file
@ -0,0 +1,279 @@
|
||||
"""
|
||||
Edge cases and boundary conditions (Priority 3)
|
||||
|
||||
Tests empty policies, malformed JSON, and edge cases
|
||||
"""
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
def test_empty_base_policy_uses_default_installation(
|
||||
empty_policy_file,
|
||||
mock_manager_util,
|
||||
mock_context
|
||||
):
|
||||
"""
|
||||
Test default installation with empty policy
|
||||
|
||||
Priority: 3 (Recommended)
|
||||
|
||||
Purpose:
|
||||
Verify that when policy is empty, the system falls back
|
||||
to default installation behavior.
|
||||
"""
|
||||
import sys
|
||||
# Path setup handled by conftest.py
|
||||
|
||||
from comfyui_manager.common.pip_util import get_pip_policy
|
||||
|
||||
policy = get_pip_policy()
|
||||
|
||||
assert policy == {}
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def malformed_policy_file(temp_policy_dir):
|
||||
"""Create malformed JSON policy file"""
|
||||
policy_file = temp_policy_dir / "pip-policy.json"
|
||||
policy_file.write_text("{invalid json content")
|
||||
return policy_file
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
def test_json_parse_error_fallback_to_empty(
|
||||
malformed_policy_file,
|
||||
mock_manager_util,
|
||||
mock_context,
|
||||
capture_logs
|
||||
):
|
||||
"""
|
||||
Test empty dict on JSON parse error
|
||||
|
||||
Priority: 3 (Recommended)
|
||||
|
||||
Purpose:
|
||||
Verify that malformed JSON results in empty policy
|
||||
with appropriate error logging.
|
||||
"""
|
||||
import sys
|
||||
# Path setup handled by conftest.py
|
||||
|
||||
from comfyui_manager.common.pip_util import get_pip_policy
|
||||
|
||||
policy = get_pip_policy()
|
||||
|
||||
assert policy == {}
|
||||
# Should have error log about parsing failure
|
||||
assert any("parse" in record.message.lower() for record in capture_logs.records)
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
def test_unknown_condition_type_returns_false(
|
||||
mock_manager_util,
|
||||
mock_context,
|
||||
capture_logs
|
||||
):
|
||||
"""
|
||||
Test unknown condition type returns False
|
||||
|
||||
Priority: 3 (Recommended)
|
||||
|
||||
Purpose:
|
||||
Verify that unknown condition types are handled gracefully
|
||||
by returning False with a warning.
|
||||
"""
|
||||
import sys
|
||||
# Path setup handled by conftest.py
|
||||
|
||||
from comfyui_manager.common.pip_util import PipBatch
|
||||
|
||||
batch = PipBatch()
|
||||
condition = {"type": "unknown_type", "some_field": "value"}
|
||||
|
||||
result = batch._evaluate_condition(condition, "pkg", {})
|
||||
|
||||
assert result is False
|
||||
# Should have warning about unknown type
|
||||
assert any("unknown" in record.message.lower() for record in capture_logs.records)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def self_reference_policy(temp_policy_dir):
|
||||
"""Create policy with self-reference"""
|
||||
policy_content = {
|
||||
"critical-package": {
|
||||
"restore": [
|
||||
{
|
||||
"condition": {
|
||||
"type": "installed",
|
||||
"spec": "!=1.2.3"
|
||||
},
|
||||
"target": "critical-package",
|
||||
"version": "1.2.3"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
policy_file = temp_policy_dir / "pip-policy.json"
|
||||
policy_file.write_text(json.dumps(policy_content, indent=2))
|
||||
return policy_file
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_self_reference_subprocess(monkeypatch):
|
||||
"""Mock subprocess for self-reference test"""
|
||||
call_sequence = []
|
||||
|
||||
installed_packages = {
|
||||
"critical-package": "1.2.2"
|
||||
}
|
||||
|
||||
def mock_run(cmd, **kwargs):
|
||||
call_sequence.append(cmd)
|
||||
|
||||
# pip freeze
|
||||
if "freeze" in cmd:
|
||||
output = "\n".join([f"{pkg}=={ver}" for pkg, ver in installed_packages.items()])
|
||||
return subprocess.CompletedProcess(cmd, 0, output, "")
|
||||
|
||||
# pip install
|
||||
if "install" in cmd and "critical-package==1.2.3" in cmd:
|
||||
installed_packages["critical-package"] = "1.2.3"
|
||||
return subprocess.CompletedProcess(cmd, 0, "", "")
|
||||
|
||||
return subprocess.CompletedProcess(cmd, 0, "", "")
|
||||
|
||||
monkeypatch.setattr("subprocess.run", mock_run)
|
||||
return call_sequence, installed_packages
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
def test_restore_self_version_check(
|
||||
self_reference_policy,
|
||||
mock_manager_util,
|
||||
mock_context,
|
||||
mock_self_reference_subprocess
|
||||
):
|
||||
"""
|
||||
Test restore policy checking its own version
|
||||
|
||||
Priority: 3 (Recommended)
|
||||
|
||||
Purpose:
|
||||
Verify that when a condition omits the package field,
|
||||
it correctly defaults to checking the package itself.
|
||||
"""
|
||||
import sys
|
||||
# Path setup handled by conftest.py
|
||||
|
||||
from comfyui_manager.common.pip_util import PipBatch
|
||||
|
||||
call_sequence, installed_packages = mock_self_reference_subprocess
|
||||
|
||||
with PipBatch() as batch:
|
||||
restored = batch.ensure_installed()
|
||||
final = batch._get_installed_packages()
|
||||
|
||||
# Condition should evaluate with self-reference
|
||||
# "1.2.2" != "1.2.3" → True
|
||||
assert "critical-package" in restored
|
||||
assert final["critical-package"] == "1.2.3"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def partial_failure_policy(temp_policy_dir):
|
||||
"""Create policy for multiple uninstalls"""
|
||||
policy_content = {
|
||||
"pkg-a": {
|
||||
"uninstall": [{"target": "old-pkg-1"}]
|
||||
},
|
||||
"pkg-b": {
|
||||
"uninstall": [{"target": "old-pkg-2"}]
|
||||
},
|
||||
"pkg-c": {
|
||||
"uninstall": [{"target": "old-pkg-3"}]
|
||||
}
|
||||
}
|
||||
|
||||
policy_file = temp_policy_dir / "pip-policy.json"
|
||||
policy_file.write_text(json.dumps(policy_content, indent=2))
|
||||
return policy_file
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_partial_failure_subprocess(monkeypatch):
|
||||
"""Mock subprocess with one failure"""
|
||||
call_sequence = []
|
||||
|
||||
installed_packages = {
|
||||
"old-pkg-1": "1.0",
|
||||
"old-pkg-2": "1.0",
|
||||
"old-pkg-3": "1.0"
|
||||
}
|
||||
|
||||
def mock_run(cmd, **kwargs):
|
||||
call_sequence.append(cmd)
|
||||
|
||||
# pip freeze
|
||||
if "freeze" in cmd:
|
||||
output = "\n".join([f"{pkg}=={ver}" for pkg, ver in installed_packages.items()])
|
||||
return subprocess.CompletedProcess(cmd, 0, output, "")
|
||||
|
||||
# pip uninstall
|
||||
if "uninstall" in cmd:
|
||||
if "old-pkg-2" in cmd:
|
||||
# Fail on pkg-2
|
||||
raise subprocess.CalledProcessError(1, cmd, "", "Uninstall failed")
|
||||
else:
|
||||
# Success on others
|
||||
for pkg in ["old-pkg-1", "old-pkg-3"]:
|
||||
if pkg in cmd:
|
||||
installed_packages.pop(pkg, None)
|
||||
return subprocess.CompletedProcess(cmd, 0, "", "")
|
||||
|
||||
return subprocess.CompletedProcess(cmd, 0, "", "")
|
||||
|
||||
monkeypatch.setattr("subprocess.run", mock_run)
|
||||
return call_sequence, installed_packages
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
def test_ensure_not_installed_continues_on_individual_failure(
|
||||
partial_failure_policy,
|
||||
mock_manager_util,
|
||||
mock_context,
|
||||
mock_partial_failure_subprocess,
|
||||
capture_logs
|
||||
):
|
||||
"""
|
||||
Test partial failure handling
|
||||
|
||||
Priority: 2 (Important)
|
||||
|
||||
Purpose:
|
||||
Verify that when one package removal fails, the system
|
||||
continues processing other packages.
|
||||
"""
|
||||
import sys
|
||||
# Path setup handled by conftest.py
|
||||
|
||||
from comfyui_manager.common.pip_util import PipBatch
|
||||
|
||||
call_sequence, installed_packages = mock_partial_failure_subprocess
|
||||
|
||||
with PipBatch() as batch:
|
||||
removed = batch.ensure_not_installed()
|
||||
|
||||
# Verify partial success
|
||||
assert "old-pkg-1" in removed
|
||||
assert "old-pkg-3" in removed
|
||||
assert "old-pkg-2" not in removed # Failed
|
||||
|
||||
# Verify warning logged for failure
|
||||
assert any("warning" in record.levelname.lower() for record in capture_logs.records)
|
||||
158
tests/common/pip_util/test_environment_recovery.py
Normal file
158
tests/common/pip_util/test_environment_recovery.py
Normal file
@ -0,0 +1,158 @@
|
||||
"""
|
||||
Test environment corruption and recovery (Priority 1)
|
||||
|
||||
Tests that packages deleted or modified during installation are restored
|
||||
"""
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def restore_policy(temp_policy_dir):
|
||||
"""Create policy with restore section for lightweight packages"""
|
||||
policy_content = {
|
||||
"six": {
|
||||
"restore": [
|
||||
{
|
||||
"target": "six",
|
||||
"version": "1.16.0",
|
||||
"reason": "six must be maintained at 1.16.0 for compatibility"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
policy_file = temp_policy_dir / "pip-policy.json"
|
||||
policy_file.write_text(json.dumps(policy_content, indent=2))
|
||||
return policy_file
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
def test_package_deletion_and_restore(
|
||||
restore_policy,
|
||||
mock_manager_util,
|
||||
mock_context,
|
||||
reset_test_venv,
|
||||
get_installed_packages,
|
||||
install_packages,
|
||||
uninstall_packages
|
||||
):
|
||||
"""
|
||||
Test package deleted by installation is restored
|
||||
|
||||
Priority: 1 (Essential)
|
||||
|
||||
Purpose:
|
||||
Verify that when a package installation deletes another package,
|
||||
the restore policy can bring it back with the correct version.
|
||||
|
||||
Based on DEPENDENCY_TREE_CONTEXT.md:
|
||||
six==1.16.0 must be maintained for compatibility
|
||||
After deletion, should restore to exactly 1.16.0
|
||||
"""
|
||||
from comfyui_manager.common.pip_util import PipBatch
|
||||
|
||||
# Verify six is initially installed at expected version
|
||||
initial = get_installed_packages()
|
||||
assert "six" in initial
|
||||
assert initial["six"] == "1.16.0", f"Expected six==1.16.0, got {initial['six']}"
|
||||
|
||||
with PipBatch() as batch:
|
||||
# Manually remove six to simulate deletion by another package
|
||||
uninstall_packages("six")
|
||||
|
||||
# Check six was deleted
|
||||
installed_after_delete = batch._get_installed_packages()
|
||||
assert "six" not in installed_after_delete, "six should be deleted"
|
||||
|
||||
# Restore six
|
||||
restored = batch.ensure_installed()
|
||||
final_packages = batch._get_installed_packages()
|
||||
|
||||
# Verify six was restored to EXACT required version (not latest)
|
||||
assert "six" in restored, "six should be in restored list"
|
||||
assert final_packages["six"] == "1.16.0", \
|
||||
"six should be restored to exact version 1.16.0 (not 1.17.0 latest)"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def version_change_policy(temp_policy_dir):
|
||||
"""Create policy for version change test with real packages"""
|
||||
policy_content = {
|
||||
"urllib3": {
|
||||
"restore": [
|
||||
{
|
||||
"condition": {
|
||||
"type": "installed",
|
||||
"spec": "!=1.26.15"
|
||||
},
|
||||
"target": "urllib3",
|
||||
"version": "1.26.15",
|
||||
"reason": "urllib3 must be 1.26.15 for compatibility"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
policy_file = temp_policy_dir / "pip-policy.json"
|
||||
policy_file.write_text(json.dumps(policy_content, indent=2))
|
||||
return policy_file
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
def test_version_change_and_restore(
|
||||
version_change_policy,
|
||||
mock_manager_util,
|
||||
mock_context,
|
||||
reset_test_venv,
|
||||
get_installed_packages,
|
||||
install_packages
|
||||
):
|
||||
"""
|
||||
Test package version changed by installation is restored
|
||||
|
||||
Priority: 1 (Essential)
|
||||
|
||||
Purpose:
|
||||
Verify that when a package installation changes another package's
|
||||
version, the restore policy can revert it to the required version.
|
||||
|
||||
Based on DEPENDENCY_TREE_CONTEXT.md:
|
||||
urllib3 can upgrade from 1.26.15 (1.x) to 2.5.0 (2.x)
|
||||
Restore policy with condition "!=1.26.15" should downgrade back
|
||||
This tests downgrade capability (not just upgrade prevention)
|
||||
"""
|
||||
from comfyui_manager.common.pip_util import PipBatch
|
||||
|
||||
# Verify urllib3 1.26.15 is installed
|
||||
initial = get_installed_packages()
|
||||
assert "urllib3" in initial
|
||||
assert initial["urllib3"] == "1.26.15", f"Expected urllib3==1.26.15, got {initial['urllib3']}"
|
||||
|
||||
with PipBatch() as batch:
|
||||
# Manually upgrade urllib3 to 2.x to simulate version change
|
||||
# This is a MAJOR version upgrade (1.x → 2.x)
|
||||
install_packages("urllib3==2.1.0")
|
||||
|
||||
installed_after = batch._get_installed_packages()
|
||||
# Verify version was changed to 2.x
|
||||
assert installed_after["urllib3"] == "2.1.0", \
|
||||
f"urllib3 should be upgraded to 2.1.0, got {installed_after['urllib3']}"
|
||||
assert installed_after["urllib3"].startswith("2."), \
|
||||
"urllib3 should be at 2.x series"
|
||||
|
||||
# Restore urllib3 to 1.26.15 (this is a DOWNGRADE from 2.x to 1.x)
|
||||
restored = batch.ensure_installed()
|
||||
final = batch._get_installed_packages()
|
||||
|
||||
# Verify condition was satisfied (2.1.0 != 1.26.15) and restore was triggered
|
||||
assert "urllib3" in restored, "urllib3 should be in restored list"
|
||||
|
||||
# Verify version was DOWNGRADED from 2.x back to 1.x
|
||||
assert final["urllib3"] == "1.26.15", \
|
||||
"urllib3 should be downgraded to 1.26.15 (from 2.1.0)"
|
||||
assert final["urllib3"].startswith("1."), \
|
||||
f"urllib3 should be back at 1.x series, got {final['urllib3']}"
|
||||
204
tests/common/pip_util/test_full_workflow_integration.py
Normal file
204
tests/common/pip_util/test_full_workflow_integration.py
Normal file
@ -0,0 +1,204 @@
|
||||
"""
|
||||
Test full workflow integration (Priority 1)
|
||||
|
||||
Tests the complete uninstall → install → restore workflow
|
||||
"""
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def workflow_policy(temp_policy_dir):
|
||||
"""Create policy for full workflow test"""
|
||||
policy_content = {
|
||||
"target-package": {
|
||||
"uninstall": [
|
||||
{
|
||||
"condition": {
|
||||
"type": "installed",
|
||||
"package": "conflicting-pkg"
|
||||
},
|
||||
"target": "conflicting-pkg",
|
||||
"reason": "Conflicts with target-package"
|
||||
}
|
||||
],
|
||||
"apply_all_matches": [
|
||||
{
|
||||
"type": "pin_dependencies",
|
||||
"pinned_packages": ["numpy", "pandas"]
|
||||
}
|
||||
]
|
||||
},
|
||||
"critical-package": {
|
||||
"restore": [
|
||||
{
|
||||
"target": "critical-package",
|
||||
"version": "1.2.3",
|
||||
"reason": "Critical package must be 1.2.3"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
policy_file = temp_policy_dir / "pip-policy.json"
|
||||
policy_file.write_text(json.dumps(policy_content, indent=2))
|
||||
return policy_file
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_workflow_subprocess(monkeypatch):
|
||||
"""Mock subprocess for workflow test"""
|
||||
call_sequence = []
|
||||
|
||||
# Initial environment: conflicting-pkg, numpy, pandas, critical-package
|
||||
installed_packages = {
|
||||
"conflicting-pkg": "1.0.0",
|
||||
"numpy": "1.26.0",
|
||||
"pandas": "2.0.0",
|
||||
"critical-package": "1.2.3"
|
||||
}
|
||||
|
||||
def mock_run(cmd, **kwargs):
|
||||
call_sequence.append(cmd)
|
||||
|
||||
# pip freeze
|
||||
if "freeze" in cmd:
|
||||
output = "\n".join([f"{pkg}=={ver}" for pkg, ver in installed_packages.items()])
|
||||
return subprocess.CompletedProcess(cmd, 0, output, "")
|
||||
|
||||
# pip uninstall
|
||||
if "uninstall" in cmd:
|
||||
# Remove conflicting-pkg
|
||||
if "conflicting-pkg" in cmd:
|
||||
installed_packages.pop("conflicting-pkg", None)
|
||||
return subprocess.CompletedProcess(cmd, 0, "", "")
|
||||
|
||||
# pip install target-package (deletes critical-package)
|
||||
if "install" in cmd and "target-package" in cmd:
|
||||
# Simulate target-package installation deleting critical-package
|
||||
installed_packages.pop("critical-package", None)
|
||||
installed_packages["target-package"] = "1.0.0"
|
||||
return subprocess.CompletedProcess(cmd, 0, "", "")
|
||||
|
||||
# pip install critical-package (restore)
|
||||
if "install" in cmd and "critical-package==1.2.3" in cmd:
|
||||
installed_packages["critical-package"] = "1.2.3"
|
||||
return subprocess.CompletedProcess(cmd, 0, "", "")
|
||||
|
||||
return subprocess.CompletedProcess(cmd, 0, "", "")
|
||||
|
||||
monkeypatch.setattr("subprocess.run", mock_run)
|
||||
return call_sequence, installed_packages
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
def test_uninstall_install_restore_workflow(
|
||||
workflow_policy,
|
||||
mock_manager_util,
|
||||
mock_context,
|
||||
mock_workflow_subprocess
|
||||
):
|
||||
"""
|
||||
Test complete uninstall → install → restore workflow
|
||||
|
||||
Priority: 1 (Essential)
|
||||
|
||||
Purpose:
|
||||
Verify the complete workflow executes in correct order:
|
||||
1. ensure_not_installed() removes conflicting packages
|
||||
2. install() applies policies (pin_dependencies)
|
||||
3. ensure_installed() restores deleted packages
|
||||
"""
|
||||
import sys
|
||||
# Path setup handled by conftest.py
|
||||
|
||||
from comfyui_manager.common.pip_util import PipBatch
|
||||
|
||||
call_sequence, installed_packages = mock_workflow_subprocess
|
||||
|
||||
with PipBatch() as batch:
|
||||
# Step 1: uninstall - remove conflicting packages
|
||||
removed = batch.ensure_not_installed()
|
||||
|
||||
# Step 2: install target-package with pinned dependencies
|
||||
result = batch.install("target-package")
|
||||
|
||||
# Step 3: restore critical-package that was deleted
|
||||
restored = batch.ensure_installed()
|
||||
|
||||
# Verify Step 1: conflicting-pkg was removed
|
||||
assert "conflicting-pkg" in removed
|
||||
|
||||
# Verify Step 2: target-package was installed with pinned dependencies
|
||||
assert result is True
|
||||
# Check that pip install was called with pinned packages
|
||||
install_calls = [cmd for cmd in call_sequence if "install" in cmd and "target-package" in cmd]
|
||||
assert len(install_calls) > 0
|
||||
install_cmd = install_calls[0]
|
||||
assert "target-package" in install_cmd
|
||||
assert "numpy==1.26.0" in install_cmd
|
||||
assert "pandas==2.0.0" in install_cmd
|
||||
|
||||
# Verify Step 3: critical-package was restored
|
||||
assert "critical-package" in restored
|
||||
|
||||
# Verify final state
|
||||
assert "conflicting-pkg" not in installed_packages
|
||||
assert "critical-package" in installed_packages
|
||||
assert installed_packages["critical-package"] == "1.2.3"
|
||||
assert "target-package" in installed_packages
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
def test_cache_invalidation_across_workflow(
|
||||
workflow_policy,
|
||||
mock_manager_util,
|
||||
mock_context,
|
||||
mock_workflow_subprocess
|
||||
):
|
||||
"""
|
||||
Test cache is correctly refreshed at each workflow step
|
||||
|
||||
Priority: 1 (Essential)
|
||||
|
||||
Purpose:
|
||||
Verify that the cache is invalidated and refreshed after each
|
||||
operation (uninstall, install, restore) to reflect current state.
|
||||
"""
|
||||
import sys
|
||||
# Path setup handled by conftest.py
|
||||
|
||||
from comfyui_manager.common.pip_util import PipBatch
|
||||
|
||||
call_sequence, installed_packages = mock_workflow_subprocess
|
||||
|
||||
with PipBatch() as batch:
|
||||
# Initial cache state
|
||||
cache1 = batch._get_installed_packages()
|
||||
assert "conflicting-pkg" in cache1
|
||||
assert "critical-package" in cache1
|
||||
|
||||
# After uninstall
|
||||
removed = batch.ensure_not_installed()
|
||||
cache2 = batch._get_installed_packages()
|
||||
assert "conflicting-pkg" not in cache2 # Removed
|
||||
|
||||
# After install (critical-package gets deleted by target-package)
|
||||
batch.install("target-package")
|
||||
cache3 = batch._get_installed_packages()
|
||||
assert "target-package" in cache3 # Added
|
||||
assert "critical-package" not in cache3 # Deleted by target-package
|
||||
|
||||
# After restore
|
||||
restored = batch.ensure_installed()
|
||||
cache4 = batch._get_installed_packages()
|
||||
assert "critical-package" in cache4 # Restored
|
||||
|
||||
# Verify cache was refreshed at each step
|
||||
assert cache1 != cache2 # Changed after uninstall
|
||||
assert cache2 != cache3 # Changed after install
|
||||
assert cache3 != cache4 # Changed after restore
|
||||
216
tests/common/pip_util/test_pin_failure_retry.py
Normal file
216
tests/common/pip_util/test_pin_failure_retry.py
Normal file
@ -0,0 +1,216 @@
|
||||
"""
|
||||
Test pin failure and retry logic (Priority 1)
|
||||
|
||||
Tests that installation with pinned dependencies can retry without pins on failure
|
||||
"""
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def retry_policy(temp_policy_dir):
|
||||
"""Create policy with retry_without_pin"""
|
||||
policy_content = {
|
||||
"new-pkg": {
|
||||
"apply_all_matches": [
|
||||
{
|
||||
"type": "pin_dependencies",
|
||||
"pinned_packages": ["numpy", "pandas"],
|
||||
"on_failure": "retry_without_pin"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
policy_file = temp_policy_dir / "pip-policy.json"
|
||||
policy_file.write_text(json.dumps(policy_content, indent=2))
|
||||
return policy_file
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_retry_subprocess(monkeypatch):
|
||||
"""Mock subprocess that fails with pins, succeeds without"""
|
||||
call_sequence = []
|
||||
attempt_count = [0]
|
||||
|
||||
installed_packages = {
|
||||
"numpy": "1.26.0",
|
||||
"pandas": "2.0.0"
|
||||
}
|
||||
|
||||
def mock_run(cmd, **kwargs):
|
||||
call_sequence.append(cmd)
|
||||
|
||||
# pip freeze
|
||||
if "freeze" in cmd:
|
||||
output = "\n".join([f"{pkg}=={ver}" for pkg, ver in installed_packages.items()])
|
||||
return subprocess.CompletedProcess(cmd, 0, output, "")
|
||||
|
||||
# pip install
|
||||
if "install" in cmd and "new-pkg" in cmd:
|
||||
attempt_count[0] += 1
|
||||
|
||||
# First attempt with pins - FAIL
|
||||
if attempt_count[0] == 1 and "numpy==1.26.0" in cmd and "pandas==2.0.0" in cmd:
|
||||
raise subprocess.CalledProcessError(1, cmd, "", "Dependency conflict")
|
||||
|
||||
# Second attempt without pins - SUCCESS
|
||||
if attempt_count[0] == 2:
|
||||
installed_packages["new-pkg"] = "1.0.0"
|
||||
# Without pins, versions might change
|
||||
return subprocess.CompletedProcess(cmd, 0, "", "")
|
||||
|
||||
return subprocess.CompletedProcess(cmd, 0, "", "")
|
||||
|
||||
monkeypatch.setattr("subprocess.run", mock_run)
|
||||
return call_sequence, installed_packages, attempt_count
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
def test_pin_failure_retry_without_pin_succeeds(
|
||||
retry_policy,
|
||||
mock_manager_util,
|
||||
mock_context,
|
||||
mock_retry_subprocess,
|
||||
capture_logs
|
||||
):
|
||||
"""
|
||||
Test retry without pin succeeds after pin failure
|
||||
|
||||
Priority: 1 (Essential)
|
||||
|
||||
Purpose:
|
||||
Verify that when installation with pinned dependencies fails,
|
||||
the system automatically retries without pins and succeeds.
|
||||
"""
|
||||
import sys
|
||||
# Path setup handled by conftest.py
|
||||
|
||||
from comfyui_manager.common.pip_util import PipBatch
|
||||
|
||||
call_sequence, installed_packages, attempt_count = mock_retry_subprocess
|
||||
|
||||
with PipBatch() as batch:
|
||||
result = batch.install("new-pkg")
|
||||
|
||||
# Verify installation succeeded on retry
|
||||
assert result is True
|
||||
|
||||
# Verify two installation attempts were made
|
||||
install_calls = [cmd for cmd in call_sequence if "install" in cmd and "new-pkg" in cmd]
|
||||
assert len(install_calls) == 2
|
||||
|
||||
# First attempt had pins
|
||||
first_call = install_calls[0]
|
||||
assert "new-pkg" in first_call
|
||||
assert "numpy==1.26.0" in first_call
|
||||
assert "pandas==2.0.0" in first_call
|
||||
|
||||
# Second attempt had no pins (just new-pkg)
|
||||
second_call = install_calls[1]
|
||||
assert "new-pkg" in second_call
|
||||
assert "numpy==1.26.0" not in second_call
|
||||
assert "pandas==2.0.0" not in second_call
|
||||
|
||||
# Verify warning log
|
||||
assert any("retrying without pins" in record.message.lower() for record in capture_logs.records)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def fail_policy(temp_policy_dir):
|
||||
"""Create policy with on_failure: fail"""
|
||||
policy_content = {
|
||||
"pytorch-addon": {
|
||||
"apply_all_matches": [
|
||||
{
|
||||
"condition": {
|
||||
"type": "installed",
|
||||
"package": "torch",
|
||||
"spec": ">=2.0.0"
|
||||
},
|
||||
"type": "pin_dependencies",
|
||||
"pinned_packages": ["torch", "torchvision", "torchaudio"],
|
||||
"on_failure": "fail"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
policy_file = temp_policy_dir / "pip-policy.json"
|
||||
policy_file.write_text(json.dumps(policy_content, indent=2))
|
||||
return policy_file
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_fail_subprocess(monkeypatch):
|
||||
"""Mock subprocess that always fails"""
|
||||
call_sequence = []
|
||||
|
||||
installed_packages = {
|
||||
"torch": "2.1.0",
|
||||
"torchvision": "0.16.0",
|
||||
"torchaudio": "2.1.0"
|
||||
}
|
||||
|
||||
def mock_run(cmd, **kwargs):
|
||||
call_sequence.append(cmd)
|
||||
|
||||
# pip freeze
|
||||
if "freeze" in cmd:
|
||||
output = "\n".join([f"{pkg}=={ver}" for pkg, ver in installed_packages.items()])
|
||||
return subprocess.CompletedProcess(cmd, 0, output, "")
|
||||
|
||||
# pip install - ALWAYS FAIL
|
||||
if "install" in cmd and "pytorch-addon" in cmd:
|
||||
raise subprocess.CalledProcessError(1, cmd, "", "Installation failed")
|
||||
|
||||
return subprocess.CompletedProcess(cmd, 0, "", "")
|
||||
|
||||
monkeypatch.setattr("subprocess.run", mock_run)
|
||||
return call_sequence, installed_packages
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
def test_pin_failure_with_fail_raises_exception(
|
||||
fail_policy,
|
||||
mock_manager_util,
|
||||
mock_context,
|
||||
mock_fail_subprocess,
|
||||
capture_logs
|
||||
):
|
||||
"""
|
||||
Test exception is raised when on_failure is "fail"
|
||||
|
||||
Priority: 1 (Essential)
|
||||
|
||||
Purpose:
|
||||
Verify that when on_failure is set to "fail", installation
|
||||
failure with pinned dependencies raises an exception and
|
||||
does not retry.
|
||||
"""
|
||||
import sys
|
||||
# Path setup handled by conftest.py
|
||||
|
||||
from comfyui_manager.common.pip_util import PipBatch
|
||||
|
||||
call_sequence, installed_packages = mock_fail_subprocess
|
||||
|
||||
with PipBatch() as batch:
|
||||
# Should raise exception
|
||||
with pytest.raises(subprocess.CalledProcessError):
|
||||
batch.install("pytorch-addon")
|
||||
|
||||
# Verify only one installation attempt was made (no retry)
|
||||
install_calls = [cmd for cmd in call_sequence if "install" in cmd and "pytorch-addon" in cmd]
|
||||
assert len(install_calls) == 1
|
||||
|
||||
# Verify it had pins
|
||||
install_cmd = install_calls[0]
|
||||
assert "pytorch-addon" in install_cmd
|
||||
assert "torch==2.1.0" in install_cmd
|
||||
assert "torchvision==0.16.0" in install_cmd
|
||||
assert "torchaudio==2.1.0" in install_cmd
|
||||
139
tests/common/pip_util/test_platform_conditions.py
Normal file
139
tests/common/pip_util/test_platform_conditions.py
Normal file
@ -0,0 +1,139 @@
|
||||
"""
|
||||
Test platform-specific conditions (Priority 2)
|
||||
|
||||
Tests OS and GPU detection for conditional policies
|
||||
"""
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def platform_policy(temp_policy_dir):
|
||||
"""Create policy with platform conditions"""
|
||||
policy_content = {
|
||||
"onnxruntime": {
|
||||
"apply_first_match": [
|
||||
{
|
||||
"condition": {
|
||||
"type": "platform",
|
||||
"os": "linux",
|
||||
"has_gpu": True
|
||||
},
|
||||
"type": "replace",
|
||||
"replacement": "onnxruntime-gpu"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
policy_file = temp_policy_dir / "pip-policy.json"
|
||||
policy_file.write_text(json.dumps(policy_content, indent=2))
|
||||
return policy_file
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_platform_subprocess(monkeypatch):
|
||||
"""Mock subprocess for platform test"""
|
||||
call_sequence = []
|
||||
installed_packages = {}
|
||||
|
||||
def mock_run(cmd, **kwargs):
|
||||
call_sequence.append(cmd)
|
||||
|
||||
# pip freeze
|
||||
if "freeze" in cmd:
|
||||
output = "\n".join([f"{pkg}=={ver}" for pkg, ver in installed_packages.items()])
|
||||
return subprocess.CompletedProcess(cmd, 0, output, "")
|
||||
|
||||
# pip install
|
||||
if "install" in cmd:
|
||||
if "onnxruntime-gpu" in cmd:
|
||||
installed_packages["onnxruntime-gpu"] = "1.0.0"
|
||||
elif "onnxruntime" in cmd:
|
||||
installed_packages["onnxruntime"] = "1.0.0"
|
||||
return subprocess.CompletedProcess(cmd, 0, "", "")
|
||||
|
||||
return subprocess.CompletedProcess(cmd, 0, "", "")
|
||||
|
||||
monkeypatch.setattr("subprocess.run", mock_run)
|
||||
return call_sequence, installed_packages
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
def test_linux_gpu_uses_gpu_package(
|
||||
platform_policy,
|
||||
mock_manager_util,
|
||||
mock_context,
|
||||
mock_platform_subprocess,
|
||||
mock_platform_linux,
|
||||
mock_torch_cuda_available
|
||||
):
|
||||
"""
|
||||
Test GPU-specific package on Linux + GPU
|
||||
|
||||
Priority: 2 (Important)
|
||||
|
||||
Purpose:
|
||||
Verify that platform-conditional policies correctly detect
|
||||
Linux + GPU and install the appropriate package variant.
|
||||
"""
|
||||
import sys
|
||||
# Path setup handled by conftest.py
|
||||
|
||||
from comfyui_manager.common.pip_util import PipBatch
|
||||
|
||||
call_sequence, installed_packages = mock_platform_subprocess
|
||||
|
||||
with PipBatch() as batch:
|
||||
result = batch.install("onnxruntime")
|
||||
|
||||
# Verify installation succeeded
|
||||
assert result is True
|
||||
|
||||
# Verify GPU version was installed
|
||||
install_calls = [cmd for cmd in call_sequence if "install" in cmd]
|
||||
assert any("onnxruntime-gpu" in str(cmd) for cmd in install_calls)
|
||||
assert "onnxruntime-gpu" in installed_packages
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
def test_windows_no_gpu_uses_cpu_package(
|
||||
platform_policy,
|
||||
mock_manager_util,
|
||||
mock_context,
|
||||
mock_platform_subprocess,
|
||||
mock_platform_windows,
|
||||
mock_torch_cuda_unavailable
|
||||
):
|
||||
"""
|
||||
Test CPU package on Windows + No GPU
|
||||
|
||||
Priority: 2 (Important)
|
||||
|
||||
Purpose:
|
||||
Verify that when platform conditions are not met,
|
||||
the original package is installed without replacement.
|
||||
"""
|
||||
import sys
|
||||
# Path setup handled by conftest.py
|
||||
|
||||
from comfyui_manager.common.pip_util import PipBatch
|
||||
|
||||
call_sequence, installed_packages = mock_platform_subprocess
|
||||
|
||||
with PipBatch() as batch:
|
||||
result = batch.install("onnxruntime")
|
||||
|
||||
# Verify installation succeeded
|
||||
assert result is True
|
||||
|
||||
# Verify CPU version was installed (no GPU replacement)
|
||||
install_calls = [cmd for cmd in call_sequence if "install" in cmd]
|
||||
assert any("onnxruntime" in str(cmd) for cmd in install_calls)
|
||||
assert "onnxruntime-gpu" not in str(call_sequence)
|
||||
assert "onnxruntime" in installed_packages
|
||||
assert "onnxruntime-gpu" not in installed_packages
|
||||
180
tests/common/pip_util/test_policy_priority.py
Normal file
180
tests/common/pip_util/test_policy_priority.py
Normal file
@ -0,0 +1,180 @@
|
||||
"""
|
||||
Test policy priority and conflicts (Priority 2)
|
||||
|
||||
Tests that user policies override base policies correctly
|
||||
"""
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def conflicting_policies(temp_policy_dir, temp_user_policy_dir):
|
||||
"""Create conflicting base and user policies"""
|
||||
# Base policy
|
||||
base_content = {
|
||||
"numpy": {
|
||||
"apply_first_match": [
|
||||
{
|
||||
"type": "skip",
|
||||
"reason": "Base policy skip"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
base_file = temp_policy_dir / "pip-policy.json"
|
||||
base_file.write_text(json.dumps(base_content, indent=2))
|
||||
|
||||
# User policy (should override)
|
||||
user_content = {
|
||||
"numpy": {
|
||||
"apply_first_match": [
|
||||
{
|
||||
"type": "force_version",
|
||||
"version": "1.26.0",
|
||||
"reason": "User override"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
user_file = temp_user_policy_dir / "pip-policy.user.json"
|
||||
user_file.write_text(json.dumps(user_content, indent=2))
|
||||
|
||||
return base_file, user_file
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
def test_user_policy_overrides_base_policy(
|
||||
conflicting_policies,
|
||||
mock_manager_util,
|
||||
mock_context,
|
||||
mock_subprocess_success
|
||||
):
|
||||
"""
|
||||
Test user policy completely replaces base policy
|
||||
|
||||
Priority: 2 (Important)
|
||||
|
||||
Purpose:
|
||||
Verify that user policy completely overrides base policy
|
||||
at the package level (not section-level merge).
|
||||
"""
|
||||
import sys
|
||||
# Path setup handled by conftest.py
|
||||
|
||||
from comfyui_manager.common.pip_util import get_pip_policy
|
||||
|
||||
policy = get_pip_policy()
|
||||
|
||||
# Verify user policy replaced base policy
|
||||
assert "numpy" in policy
|
||||
assert "apply_first_match" in policy["numpy"]
|
||||
assert len(policy["numpy"]["apply_first_match"]) == 1
|
||||
|
||||
# Should be force_version (user), not skip (base)
|
||||
assert policy["numpy"]["apply_first_match"][0]["type"] == "force_version"
|
||||
assert policy["numpy"]["apply_first_match"][0]["version"] == "1.26.0"
|
||||
|
||||
# Base policy skip should be completely gone
|
||||
assert not any(
|
||||
item["type"] == "skip"
|
||||
for item in policy["numpy"]["apply_first_match"]
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def first_match_policy(temp_policy_dir):
|
||||
"""Create policy with multiple apply_first_match entries"""
|
||||
policy_content = {
|
||||
"pkg": {
|
||||
"apply_first_match": [
|
||||
{
|
||||
"condition": {
|
||||
"type": "installed",
|
||||
"package": "numpy"
|
||||
},
|
||||
"type": "force_version",
|
||||
"version": "1.0"
|
||||
},
|
||||
{
|
||||
"type": "force_version",
|
||||
"version": "2.0"
|
||||
},
|
||||
{
|
||||
"type": "skip"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
policy_file = temp_policy_dir / "pip-policy.json"
|
||||
policy_file.write_text(json.dumps(policy_content, indent=2))
|
||||
return policy_file
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_first_match_subprocess(monkeypatch):
|
||||
"""Mock subprocess for first match test"""
|
||||
call_sequence = []
|
||||
|
||||
installed_packages = {
|
||||
"numpy": "1.26.0"
|
||||
}
|
||||
|
||||
def mock_run(cmd, **kwargs):
|
||||
call_sequence.append(cmd)
|
||||
|
||||
# pip freeze
|
||||
if "freeze" in cmd:
|
||||
output = "\n".join([f"{pkg}=={ver}" for pkg, ver in installed_packages.items()])
|
||||
return subprocess.CompletedProcess(cmd, 0, output, "")
|
||||
|
||||
# pip install
|
||||
if "install" in cmd and "pkg" in cmd:
|
||||
if "pkg==1.0" in cmd:
|
||||
installed_packages["pkg"] = "1.0"
|
||||
return subprocess.CompletedProcess(cmd, 0, "", "")
|
||||
|
||||
return subprocess.CompletedProcess(cmd, 0, "", "")
|
||||
|
||||
monkeypatch.setattr("subprocess.run", mock_run)
|
||||
return call_sequence, installed_packages
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
def test_first_match_stops_at_first_satisfied(
|
||||
first_match_policy,
|
||||
mock_manager_util,
|
||||
mock_context,
|
||||
mock_first_match_subprocess
|
||||
):
|
||||
"""
|
||||
Test apply_first_match stops at first satisfied condition
|
||||
|
||||
Priority: 2 (Important)
|
||||
|
||||
Purpose:
|
||||
Verify that in apply_first_match, only the first policy
|
||||
with a satisfied condition is executed (exclusive execution).
|
||||
"""
|
||||
import sys
|
||||
# Path setup handled by conftest.py
|
||||
|
||||
from comfyui_manager.common.pip_util import PipBatch
|
||||
|
||||
call_sequence, installed_packages = mock_first_match_subprocess
|
||||
|
||||
with PipBatch() as batch:
|
||||
result = batch.install("pkg")
|
||||
|
||||
# Verify installation succeeded
|
||||
assert result is True
|
||||
|
||||
# First condition satisfied (numpy installed), so version 1.0 applied
|
||||
install_calls = [cmd for cmd in call_sequence if "install" in cmd and "pkg" in cmd]
|
||||
assert len(install_calls) > 0
|
||||
assert "pkg==1.0" in install_calls[0]
|
||||
assert "pkg==2.0" not in str(call_sequence) # Second policy not applied
|
||||
178
tests/common/pip_util/test_unit_parsing.py
Normal file
178
tests/common/pip_util/test_unit_parsing.py
Normal file
@ -0,0 +1,178 @@
|
||||
"""
|
||||
Unit tests for package spec parsing and condition evaluation
|
||||
|
||||
Tests core utility functions
|
||||
"""
|
||||
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
def test_parse_package_spec_name_only(mock_manager_util, mock_context):
|
||||
"""Test parsing package name without version"""
|
||||
import sys
|
||||
# Path setup handled by conftest.py
|
||||
|
||||
from comfyui_manager.common.pip_util import PipBatch
|
||||
|
||||
batch = PipBatch()
|
||||
name, spec = batch._parse_package_spec("numpy")
|
||||
|
||||
assert name == "numpy"
|
||||
assert spec is None
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
def test_parse_package_spec_exact_version(mock_manager_util, mock_context):
|
||||
"""Test parsing package with exact version"""
|
||||
import sys
|
||||
# Path setup handled by conftest.py
|
||||
|
||||
from comfyui_manager.common.pip_util import PipBatch
|
||||
|
||||
batch = PipBatch()
|
||||
name, spec = batch._parse_package_spec("numpy==1.26.0")
|
||||
|
||||
assert name == "numpy"
|
||||
assert spec == "==1.26.0"
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
def test_parse_package_spec_min_version(mock_manager_util, mock_context):
|
||||
"""Test parsing package with minimum version"""
|
||||
import sys
|
||||
# Path setup handled by conftest.py
|
||||
|
||||
from comfyui_manager.common.pip_util import PipBatch
|
||||
|
||||
batch = PipBatch()
|
||||
name, spec = batch._parse_package_spec("pandas>=2.0.0")
|
||||
|
||||
assert name == "pandas"
|
||||
assert spec == ">=2.0.0"
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
def test_parse_package_spec_hyphenated_name(mock_manager_util, mock_context):
|
||||
"""Test parsing package with hyphens"""
|
||||
import sys
|
||||
# Path setup handled by conftest.py
|
||||
|
||||
from comfyui_manager.common.pip_util import PipBatch
|
||||
|
||||
batch = PipBatch()
|
||||
name, spec = batch._parse_package_spec("scikit-learn>=1.0")
|
||||
|
||||
assert name == "scikit-learn"
|
||||
assert spec == ">=1.0"
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
def test_evaluate_condition_none(mock_manager_util, mock_context):
|
||||
"""Test None condition always returns True"""
|
||||
import sys
|
||||
# Path setup handled by conftest.py
|
||||
|
||||
from comfyui_manager.common.pip_util import PipBatch
|
||||
|
||||
batch = PipBatch()
|
||||
result = batch._evaluate_condition(None, "numpy", {})
|
||||
|
||||
assert result is True
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
def test_evaluate_condition_installed_package_exists(mock_manager_util, mock_context):
|
||||
"""Test installed condition when package exists"""
|
||||
import sys
|
||||
# Path setup handled by conftest.py
|
||||
|
||||
from comfyui_manager.common.pip_util import PipBatch
|
||||
|
||||
batch = PipBatch()
|
||||
condition = {"type": "installed", "package": "numpy"}
|
||||
installed = {"numpy": "1.26.0"}
|
||||
|
||||
result = batch._evaluate_condition(condition, "numba", installed)
|
||||
|
||||
assert result is True
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
def test_evaluate_condition_installed_package_not_exists(mock_manager_util, mock_context):
|
||||
"""Test installed condition when package doesn't exist"""
|
||||
import sys
|
||||
# Path setup handled by conftest.py
|
||||
|
||||
from comfyui_manager.common.pip_util import PipBatch
|
||||
|
||||
batch = PipBatch()
|
||||
condition = {"type": "installed", "package": "numpy"}
|
||||
installed = {}
|
||||
|
||||
result = batch._evaluate_condition(condition, "numba", installed)
|
||||
|
||||
assert result is False
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
def test_evaluate_condition_platform_os_match(
|
||||
mock_manager_util,
|
||||
mock_context,
|
||||
mock_platform_linux
|
||||
):
|
||||
"""Test platform OS condition matching"""
|
||||
import sys
|
||||
# Path setup handled by conftest.py
|
||||
|
||||
from comfyui_manager.common.pip_util import PipBatch
|
||||
|
||||
batch = PipBatch()
|
||||
condition = {"type": "platform", "os": "linux"}
|
||||
|
||||
result = batch._evaluate_condition(condition, "package", {})
|
||||
|
||||
assert result is True
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
def test_evaluate_condition_platform_gpu_available(
|
||||
mock_manager_util,
|
||||
mock_context,
|
||||
mock_torch_cuda_available
|
||||
):
|
||||
"""Test GPU detection when available"""
|
||||
import sys
|
||||
# Path setup handled by conftest.py
|
||||
|
||||
from comfyui_manager.common.pip_util import PipBatch
|
||||
|
||||
batch = PipBatch()
|
||||
condition = {"type": "platform", "has_gpu": True}
|
||||
|
||||
result = batch._evaluate_condition(condition, "package", {})
|
||||
|
||||
assert result is True
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
def test_evaluate_condition_platform_gpu_not_available(
|
||||
mock_manager_util,
|
||||
mock_context,
|
||||
mock_torch_cuda_unavailable
|
||||
):
|
||||
"""Test GPU detection when not available"""
|
||||
import sys
|
||||
# Path setup handled by conftest.py
|
||||
|
||||
from comfyui_manager.common.pip_util import PipBatch
|
||||
|
||||
batch = PipBatch()
|
||||
condition = {"type": "platform", "has_gpu": True}
|
||||
|
||||
result = batch._evaluate_condition(condition, "package", {})
|
||||
|
||||
assert result is False
|
||||
41
tests/pytest.ini
Normal file
41
tests/pytest.ini
Normal file
@ -0,0 +1,41 @@
|
||||
[pytest]
|
||||
# Global pytest configuration for comfyui-manager tests
|
||||
|
||||
# Test discovery
|
||||
python_files = test_*.py
|
||||
python_classes = Test*
|
||||
python_functions = test_*
|
||||
|
||||
# Add comfyui_manager to Python path
|
||||
pythonpath = ../comfyui_manager
|
||||
|
||||
# Output options
|
||||
addopts =
|
||||
# Verbose output
|
||||
-v
|
||||
# Show extra test summary info
|
||||
-ra
|
||||
# Show local variables in tracebacks
|
||||
--showlocals
|
||||
# Strict markers (fail on unknown markers)
|
||||
--strict-markers
|
||||
|
||||
# Markers for test categorization
|
||||
markers =
|
||||
unit: Unit tests for individual functions
|
||||
integration: Integration tests for policy application
|
||||
e2e: End-to-end workflow tests
|
||||
slow: Tests that take significant time
|
||||
requires_network: Tests that require network access
|
||||
|
||||
# Logging
|
||||
log_cli = false
|
||||
log_cli_level = INFO
|
||||
log_cli_format = %(asctime)s [%(levelname)8s] %(message)s
|
||||
log_cli_date_format = %Y-%m-%d %H:%M:%S
|
||||
|
||||
# Warnings
|
||||
filterwarnings =
|
||||
error
|
||||
ignore::DeprecationWarning
|
||||
ignore::PendingDeprecationWarning
|
||||
19
tests/requirements.txt
Normal file
19
tests/requirements.txt
Normal file
@ -0,0 +1,19 @@
|
||||
# Test Dependencies for pip_util.py
|
||||
# Install in isolated venv to prevent environment corruption
|
||||
|
||||
# Testing Framework
|
||||
pytest>=7.4.0
|
||||
pytest-cov>=4.1.0
|
||||
pytest-mock>=3.11.0
|
||||
|
||||
# Code Quality
|
||||
flake8>=6.0.0
|
||||
black>=23.0.0
|
||||
mypy>=1.5.0
|
||||
|
||||
# Dependencies from main project
|
||||
packaging>=23.0
|
||||
|
||||
# Mock and testing utilities
|
||||
responses>=0.23.0
|
||||
freezegun>=1.2.0
|
||||
75
tests/setup_test_env.sh
Executable file
75
tests/setup_test_env.sh
Executable file
@ -0,0 +1,75 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Test Environment Setup Script for pip_util.py
|
||||
# Creates isolated venv to prevent environment corruption
|
||||
|
||||
set -e
|
||||
|
||||
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
|
||||
VENV_DIR="${SCRIPT_DIR}/test_venv"
|
||||
|
||||
echo "=================================================="
|
||||
echo "pip_util.py Test Environment Setup"
|
||||
echo "=================================================="
|
||||
echo ""
|
||||
|
||||
# Check Python version
|
||||
PYTHON_CMD=""
|
||||
if command -v python3 &> /dev/null; then
|
||||
PYTHON_CMD="python3"
|
||||
elif command -v python &> /dev/null; then
|
||||
PYTHON_CMD="python"
|
||||
else
|
||||
echo "❌ Error: Python not found. Please install Python 3.8 or higher."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
PYTHON_VERSION=$($PYTHON_CMD --version 2>&1 | awk '{print $2}')
|
||||
echo "✓ Found Python: $PYTHON_VERSION"
|
||||
|
||||
# Remove existing venv if present
|
||||
if [ -d "$VENV_DIR" ]; then
|
||||
echo ""
|
||||
read -p "⚠️ Existing test venv found. Remove and recreate? (y/N): " -n 1 -r
|
||||
echo
|
||||
if [[ $REPLY =~ ^[Yy]$ ]]; then
|
||||
echo "🗑️ Removing existing venv..."
|
||||
rm -rf "$VENV_DIR"
|
||||
else
|
||||
echo "Keeping existing venv. Skipping creation."
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
|
||||
# Create venv
|
||||
echo ""
|
||||
echo "📦 Creating virtual environment..."
|
||||
$PYTHON_CMD -m venv "$VENV_DIR"
|
||||
|
||||
# Activate venv
|
||||
echo "🔌 Activating virtual environment..."
|
||||
source "${VENV_DIR}/bin/activate"
|
||||
|
||||
# Upgrade pip
|
||||
echo "⬆️ Upgrading pip..."
|
||||
pip install --upgrade pip
|
||||
|
||||
# Install test dependencies
|
||||
echo ""
|
||||
echo "📚 Installing test dependencies..."
|
||||
pip install -r "${SCRIPT_DIR}/requirements.txt"
|
||||
|
||||
echo ""
|
||||
echo "=================================================="
|
||||
echo "✅ Test environment setup complete!"
|
||||
echo "=================================================="
|
||||
echo ""
|
||||
echo "To activate the test environment:"
|
||||
echo " source ${VENV_DIR}/bin/activate"
|
||||
echo ""
|
||||
echo "To run tests:"
|
||||
echo " pytest"
|
||||
echo ""
|
||||
echo "To deactivate:"
|
||||
echo " deactivate"
|
||||
echo ""
|
||||
Loading…
Reference in New Issue
Block a user