ComfyUI-Manager/tests/glob/README.md

Glob API Endpoint Tests

This directory contains endpoint tests for the ComfyUI Manager glob API implementation.

Quick Navigation

• Running Tests: See Running Tests section below
• Test Coverage: See Test Coverage section
• Known Issues: See Known Issues and Fixes section
• Detailed Execution Guide: See TESTING_GUIDE.md
• Future Test Plans: See docs/internal/test_planning/

Test Files

• test_queue_task_api.py - Queue task API tests for install/uninstall/version switching operations (8 tests)
• test_enable_disable_api.py - Queue task API tests for enable/disable operations (5 tests)
• test_update_api.py - Queue task API tests for update operations (4 tests)
• test_complex_scenarios.py - Multi-version complex scenarios (10 tests) - Phase 1 + 3 + 4 + 5 + 6
• test_installed_api_original_case.py - Installed API case preservation tests (4 tests)
• test_version_switching_comprehensive.py - Comprehensive version switching tests (19 tests)
• test_case_sensitivity_integration.py - Full integration test for case sensitivity (1 test)

Total: 51 tests - All passing ✅ (+5 P1 tests: Phase 3.1, Phase 5.1, Phase 5.2, Phase 5.3, Phase 6)

Running Tests

Prerequisites

1. Install test dependencies:

pip install pytest requests

2. Start ComfyUI server with Manager:

cd tests/env
./run.sh

Run All Tests

# From project root
pytest tests/glob/ -v

# With coverage
pytest tests/glob/ -v --cov=comfyui_manager.glob --cov-report=html

Run Specific Tests

# Run specific test file
pytest tests/glob/test_queue_task_api.py -v

# Run specific test function
pytest tests/glob/test_queue_task_api.py::test_install_package_via_queue -v

# Run with output
pytest tests/glob/test_queue_task_api.py -v -s

Environment Variables

• COMFYUI_TEST_URL - Base URL for ComfyUI server (default: http://127.0.0.1:8188)
• TEST_SERVER_PORT - Server port (default: 8188, automatically used by conftest.py)
• COMFYUI_CUSTOM_NODES_PATH - Path to custom_nodes directory (default: tests/env/ComfyUI/custom_nodes)

Important: All tests now use the server_url fixture from conftest.py, which reads from these environment variables. This ensures compatibility with parallel test execution.

Example:

# Single test environment
COMFYUI_TEST_URL=http://localhost:8188 pytest tests/glob/ -v

# Parallel test environment (port automatically set)
TEST_SERVER_PORT=8189 pytest tests/glob/ -v

Test Coverage

The test suite covers:

1. Install Operations (test_queue_task_api.py)

   • Install package via queue task API
   • Version switching between CNR and Nightly
   • Case-insensitive package name handling
   • Queue multiple install tasks

2. Uninstall Operations (test_queue_task_api.py)

   • Uninstall package via queue task API
   • Complete install/uninstall cycle
   • Case-insensitive uninstall operations

3. Enable/Disable Operations (test_enable_disable_api.py) ✅ All via Queue Task API

   • Disable active package via queue task
   • Enable disabled package via queue task
   • Duplicate disable/enable handling via queue task
   • Complete enable/disable cycle via queue task
   • Marker file preservation (.tracking, .git)

4. Update Operations (test_update_api.py)

   • Update CNR package to latest version
   • Update Nightly package (git pull)
   • Skip update when already latest
   • Complete update workflow cycle

5. Complex Multi-Version Scenarios (test_complex_scenarios.py)

   • Phase 1: Enable from Multiple Disabled States
     • Enable CNR when both CNR and Nightly are disabled
     • Enable Nightly when both CNR and Nightly are disabled
   • Phase 3: Disable Complex Scenarios
     • Disable CNR when Nightly is disabled (both end up disabled)
   • Phase 4: Update with Other Versions Present
     • Update CNR with Nightly disabled (selective update)
     • Update Nightly with CNR disabled (selective update)
     • Update enabled package with multiple disabled versions
   • Phase 5: Install with Existing Versions (Complete) ✅
     • Install CNR when Nightly is enabled (automatic version switch)
     • Install Nightly when CNR is enabled (automatic version switch)
     • Install new version when both CNR and Nightly are disabled
   • Phase 6: Uninstall with Multiple Versions ✅
     • Uninstall removes all versions (enabled + all disabled) - default behavior
   • Version-specific enable with @version syntax
   • Multiple disabled versions management

6. Version Switching Comprehensive (test_version_switching_comprehensive.py)

   • Reverse scenario: Nightly → CNR → Nightly
   • Same version reinstall detection and skip

7. Case Sensitivity Integration (test_case_sensitivity_integration.py)

   • Full workflow: Install CNR → Verify lookup → Switch to Nightly
   • Directory naming convention verification
   • Marker file preservation (.tracking, .git)
   • Supports both pytest and standalone execution
   • Repeated version switching (4+ times)
   • Cleanup verification (no orphaned files)
   • Fresh install after complete uninstall

8. Queue Management

   • Queue multiple tasks
   • Start queue processing
   • Task execution order and completion

9. Integration Tests

   • Verify package in installed list
   • Verify filesystem changes
   • Version identification (.tracking vs .git)
   • .disabled/ directory mechanism

Known Issues and Fixes

Issue 1: Glob API Parameters

Important: Glob API does NOT support channel or mode parameters.

Note:

• channel and mode parameters are legacy-only features
• InstallPackParams data model includes these fields because it's shared between legacy and glob implementations
• Glob API implementation ignores these parameters
• Tests should NOT include channel or mode in request parameters

Issue 2: Case-Insensitive Package Operations (PARTIALLY RESOLVED)

Previous Problem: Operations failed when using different cases (e.g., "ComfyUI_SigmoidOffsetScheduler" vs "comfyui_sigmoidoffsetscheduler")

Current Status:

• Install: Requires exact package name due to CNR server limitations (case-sensitive)
• Uninstall/Enable/Disable: Works with any case variation using cnr_utils.normalize_package_name()

Normalization Function (cnr_utils.normalize_package_name()):

• Strips leading/trailing whitespace with .strip()
• Converts to lowercase with .lower()
• Accepts any case variation (e.g., "ComfyUI_SigmoidOffsetScheduler", "COMFYUI_SIGMOIDOFFSETSCHEDULER", " comfyui_sigmoidoffsetscheduler ")

Examples:

# Install - requires exact case
{"id": "ComfyUI_SigmoidOffsetScheduler"}  # ✓ Works
{"id": "comfyui_sigmoidoffsetscheduler"}  # ✗ Fails (CNR limitation)

# Uninstall - accepts any case
{"node_name": "ComfyUI_SigmoidOffsetScheduler"}  # ✓ Works
{"node_name": " ComfyUI_SigmoidOffsetScheduler "}  # ✓ Works (normalized)
{"node_name": "COMFYUI_SIGMOIDOFFSETSCHEDULER"}  # ✓ Works (normalized)
{"node_name": "comfyui_sigmoidoffsetscheduler"}  # ✓ Works (normalized)

Issue 3: .disabled/ Directory Mechanism

Critical Discovery: The .disabled/ directory is used by the disable operation to store disabled packages.

Implementation (manager_core.py:1115-1154):

def unified_disable(self, packname: str):
    # Disable moves package to .disabled/ with version suffix
    to_path = os.path.join(base_path, '.disabled', f"{folder_name}@{matched_active.version.replace('.', '_')}")
    shutil.move(matched_active.fullpath, to_path)

Directory Naming Format:

• CNR packages: .disabled/{package_name_normalized}@{version}
  • Example: .disabled/comfyui_sigmoidoffsetscheduler@1_0_2
• Nightly packages: .disabled/{package_name_normalized}@nightly
  • Example: .disabled/comfyui_sigmoidoffsetscheduler@nightly

Key Points:

• Package names are normalized (lowercase) in directory names
• Version dots are replaced with underscores (e.g., 1.0.2 → 1_0_2)
• Disabled packages preserve their marker files (.tracking for CNR, .git for Nightly)
• Enable operation moves packages back from .disabled/ to custom_nodes/

Testing Implications:

• Complex multi-version scenarios require install → disable sequences
• Fixture pattern: Install CNR → Disable → Install Nightly → Disable
• Tests must check .disabled/ with case-insensitive searches
• Directory format must match normalized names with version suffixes

Issue 4: Version Switch Mechanism

Behavior: Version switching uses a slot-based system with Nightly and Archive as separate slots.

Slot-Based System Concept:

• Nightly Slot: Git-based installation (one slot)
• Archive Slot: Registry-based installation (one slot)
• Only one slot is active at a time
• The inactive slot is stored in .disabled/
• Archive versions update within the Archive slot

Two Types of Version Switch:

1. Slot Switch: Nightly ↔ Archive (uses .disabled/ mechanism)

• Archive → Nightly:

  • Archive (any version) → moved to .disabled/ComfyUI_SigmoidOffsetScheduler
  • Nightly → active in custom_nodes/ComfyUI_SigmoidOffsetScheduler

• Nightly → Archive:

  • Nightly → moved to .disabled/ComfyUI_SigmoidOffsetScheduler
  • Archive (any version) → restored from .disabled/ and becomes active

2. Version Update: Archive ↔ Archive (in-place update within Archive slot)

• 1.0.1 → 1.0.2 (when Archive slot is active):
  • Directory contents updated in-place
  • pyproject.toml version updated: 1.0.1 → 1.0.2
  • .tracking file updated
  • NO .disabled/ directory used

3. Combined Operation: Nightly (active) + Archive 1.0 (disabled) → Archive 2.0

• Step 1 - Slot Switch: Nightly → .disabled/, Archive 1.0 → active
• Step 2 - Version Update: Archive 1.0 → 2.0 (in-place within Archive slot)
• Result: Archive 2.0 active, Nightly in .disabled/

Version Identification:

• Archive versions: Use pyproject.toml version field
• Nightly version: pyproject.toml ignored, Git commit SHA used instead

Key Points:

• Slot Switch (Nightly ↔ Archive): .disabled/ mechanism for enable/disable
• Version Update (Archive ↔ Archive): In-place content update within slot
• Archive installations have .tracking file
• Nightly installations have .git directory
• Only one slot is active at a time

Issue 5: Version Selection Logic (RESOLVED)

Problem: When enabling a package with both CNR and Nightly versions disabled, the system would always enable CNR instead of respecting the user's choice.

Root Cause (manager_server.py:876-919):

• do_enable() was parsing version_spec from cnr_id (e.g., packagename@nightly)
• But it wasn't passing version_spec to unified_enable()
• This caused unified_enable() to use default version selection (latest CNR)

Solution:

# Before (manager_server.py:876)
res = core.unified_manager.unified_enable(node_name)  # Missing version_spec!

# After (manager_server.py:876)
res = core.unified_manager.unified_enable(node_name, version_spec)  # ✅ Fixed

API Usage:

# Enable CNR version (default or latest)
{"cnr_id": "ComfyUI_SigmoidOffsetScheduler"}

# Enable specific CNR version
{"cnr_id": "ComfyUI_SigmoidOffsetScheduler@1.0.1"}

# Enable Nightly version
{"cnr_id": "ComfyUI_SigmoidOffsetScheduler@nightly"}

Version Selection Priority (manager_core.py:get_inactive_pack):

1. Explicit version in cnr_id (e.g., @nightly, @1.0.1)
2. Latest CNR version (if available)
3. Nightly version (if no CNR available)
4. Unknown version (fallback)

Files Modified:

• comfyui_manager/glob/manager_server.py - Pass version_spec to unified_enable
• comfyui_manager/common/node_package.py - Parse @version from disabled directory names
• comfyui_manager/glob/manager_core.py - Fix is_disabled() early-return bug

Status: ✅ Resolved - All 42 tests passing

Test Data

Test package: ComfyUI_SigmoidOffsetScheduler

• Package ID: ComfyUI_SigmoidOffsetScheduler
• CNR ID (lowercase): comfyui_sigmoidoffsetscheduler
• Version: 1.0.2
• Nightly: Git clone from main branch

Additional Documentation

Test Execution Guide

• TESTING_GUIDE.md - Detailed guide for running tests, updating OpenAPI schemas, and troubleshooting

Future Test Plans

• docs/internal/test_planning/ - Planned but not yet implemented test scenarios

---

Contributing

When adding new tests:

1. Follow pytest naming conventions (test_.py, test_)
2. Use fixtures for common setup/teardown
3. Add docstrings explaining test purpose
4. Update this README with test coverage information
5. For complex scenario tests, see docs/internal/test_planning/