wangbo/ComfyUI-Manager
1
0
Fork 0
mirror of https://github.com/Comfy-Org/ComfyUI-Manager.git synced 2025-12-16 18:02:58 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
43647249cf
ComfyUI-Manager/tests/README.md
Dr.Lt.Data 43647249cf refactor: remove package-level caching to support dynamic installation 
Remove package-level caching in cnr_utils and node_package modules to enable
proper dynamic custom node installation and version switching without ComfyUI
server restarts.

Key Changes:
- Remove @lru_cache decorators from version-sensitive functions
- Remove cached_property from NodePackage for dynamic state updates
- Add comprehensive test suite with parallel execution support
- Implement version switching tests (CNR ↔ Nightly)
- Add case sensitivity integration tests
- Improve error handling and logging

API Priority Rules (manager_core.py:1801):
- Enabled-Priority: Show only enabled version when both exist
- CNR-Priority: Show only CNR when both CNR and Nightly are disabled
- Prevents duplicate package entries in /v2/customnode/installed API
- Cross-match using cnr_id and aux_id for CNR ↔ Nightly detection

Test Infrastructure:
- 8 test files with 59 comprehensive test cases
- Parallel test execution across 5 isolated environments
- Automated test scripts with environment setup
- Configurable timeout (60 minutes default)
- Support for both master and dr-support-pip-cm branches

Bug Fixes:
- Fix COMFYUI_CUSTOM_NODES_PATH environment variable export
- Resolve test fixture regression with module-level variables
- Fix import timing issues in test configuration
- Register pytest integration marker to eliminate warnings
- Fix POSIX compliance in shell scripts (((var++)) → $((var + 1)))

Documentation:
- CNR_VERSION_MANAGEMENT_DESIGN.md v1.0 → v1.1 with API priority rules
- Add test guides and execution documentation (TESTING_PROMPT.md)
- Add security-enhanced installation guide
- Create CLI migration guides and references
- Document package version management

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-08 09:07:09 +09:00

4.5 KiB
Raw Blame History

ComfyUI Manager Test Suite

Comprehensive test suite for ComfyUI Manager with parallel execution support.

Quick Start

Fastest Way: Automated Testing

./tests/run_automated_tests.sh

What it does:

  • Cleans environment and stops old processes
  • Sets up 10 parallel test environments
  • Runs all 43 tests in ~2 minutes
  • Generates comprehensive report

Expected: 100% pass rate, ~140-160s execution time, 9x+ speedup

For Claude Code Users

Load the testing prompt:

@tests/TESTING_PROMPT.md

Claude Code will automatically execute tests and provide intelligent analysis.

Test Suite Overview

Coverage (54 Tests)

  • Queue Task API (8 tests) - Install, uninstall, version switching
  • Version Switching (19 tests) - CNR↔Nightly, upgrades, downgrades
  • Enable/Disable API (5 tests) - Package activation
  • Update API (4 tests) - Package updates
  • Installed API (4 tests) - Package listing, original case preservation
  • Case Sensitivity (2 tests) - Case-insensitive lookup, full workflow
  • Complex Scenarios (12 tests) - Multi-version state, automatic switching

Performance

  • Execution: ~140-160s (2.3-2.7 minutes)
  • Parallel: 10 environments
  • Speedup: 9x+ vs sequential
  • Load Balance: 1.2x variance (excellent)

Manual Execution

Parallel Testing (Recommended)

# Setup (one-time)
export NUM_ENVS=10
./tests/setup_parallel_test_envs.sh

# Run tests
./tests/run_parallel_tests.sh

Single Environment Testing

# Setup
./tests/setup_test_env.sh

# Run tests
cd tests/env
python ComfyUI/main.py --enable-manager &
sleep 20
pytest ../glob/

Adding New Tests

When adding 3+ new tests or modifying test execution time significantly:

# 1. Write your tests in tests/glob/

# 2. Run tests and check load balance
./tests/run_automated_tests.sh
# Look for "Load Balance: X.XXx variance" in report

# 3. If variance > 2.0x, update durations
./tests/update_test_durations.sh  # Takes ~15-20 min

# 4. Commit duration data
git add .test_durations
git commit -m "chore: update test duration data"

See: glob/TESTING_GUIDE.md for detailed workflow

Files

  • run_automated_tests.sh - One-command test execution
  • run_parallel_tests.sh - Parallel test runner
  • setup_parallel_test_envs.sh - Environment setup
  • update_test_durations.sh - Update load balancing data
  • TESTING_PROMPT.md - Claude Code automation
  • glob/ - Test implementations
  • glob/TESTING_GUIDE.md - Development workflow guide

Requirements

  • Python 3.12+
  • Virtual environment: /home/rho/venv
  • ComfyUI branch: ltdrdata/dr-support-pip-cm
  • Ports: 8188-8197 available

Troubleshooting

Tests Fail to Start

# Stop existing processes
pkill -f "ComfyUI/main.py"
sleep 2

# Re-run
./tests/run_automated_tests.sh

Slow Execution

If tests take >3 minutes, update duration data:

./tests/update_test_durations.sh

Environment Issues

Rebuild test environments:

rm -rf tests/env/ComfyUI_*
NUM_ENVS=10 ./tests/setup_parallel_test_envs.sh

Generated Files

  • Report: .claude/livecontext/automated_test_*.md
  • Logs: tests/tmp/test-results-[1-10].log
  • Server Logs: tests/tmp/comfyui-parallel-[1-10].log

CI/CD Integration

- name: Run Tests
  run: |
    source /home/rho/venv/bin/activate
    ./tests/run_automated_tests.sh

Exit code: 0 = pass, 1 = fail

Status: ✅ Production-ready (100% pass rate, <3min execution)

Recent Fixes (2025-11-06)

Fixed Test Failures

test_case_sensitivity_full_workflow

  • Issue: HTTP 405 error - incorrect API endpoint usage
  • Root Cause: Using non-existent /customnode/install endpoint
  • Fix: Migrated to queue API (/v2/manager/queue/task + /v2/manager/queue/start)
  • Changes: Updated request parameters ("id", "version", "selected_version")
  • Location: tests/glob/test_case_sensitivity_integration.py:65-200

test_enable_package

  • Issue: AssertionError - found 2 disabled packages instead of 1
  • Root Cause: Leftover disabled packages from previous parallel tests
  • Fix: Added cleanup before test execution (not just after)
  • Changes: Created _cleanup() helper, added filesystem sync delay
  • Location: tests/glob/test_enable_disable_api.py:56-111

Improvements

  • Increased wait times for parallel execution reliability (20s → 30s)
  • Added queue status checking for better debugging
  • Enhanced fixture cleanup with filesystem sync delays
  • Both tests now pass consistently in parallel execution