ComfyUI-Manager/tests

ComfyUI Manager Test Suite

Comprehensive test suite for ComfyUI Manager with parallel execution support.

Quick Start

Fastest Way: Automated Testing

./tests/run_automated_tests.sh

What it does:

• Cleans environment and stops old processes
• Sets up 10 parallel test environments
• Runs all 43 tests in ~2 minutes
• Generates comprehensive report

Expected: 100% pass rate, ~140-160s execution time, 9x+ speedup

For Claude Code Users

Load the testing prompt:

@tests/TESTING_PROMPT.md

Claude Code will automatically execute tests and provide intelligent analysis.

Test Suite Overview

Coverage (54 Tests)

• Queue Task API (8 tests) - Install, uninstall, version switching
• Version Switching (19 tests) - CNR↔Nightly, upgrades, downgrades
• Enable/Disable API (5 tests) - Package activation
• Update API (4 tests) - Package updates
• Installed API (4 tests) - Package listing, original case preservation
• Case Sensitivity (2 tests) - Case-insensitive lookup, full workflow
• Complex Scenarios (12 tests) - Multi-version state, automatic switching

Performance

• Execution: ~140-160s (2.3-2.7 minutes)
• Parallel: 10 environments
• Speedup: 9x+ vs sequential
• Load Balance: 1.2x variance (excellent)

Manual Execution

Parallel Testing (Recommended)

# Setup (one-time)
export NUM_ENVS=10
./tests/setup_parallel_test_envs.sh

# Run tests
./tests/run_parallel_tests.sh

Single Environment Testing

# Setup
./tests/setup_test_env.sh

# Run tests
cd tests/env
python ComfyUI/main.py --enable-manager &
sleep 20
pytest ../glob/

Adding New Tests

When adding 3+ new tests or modifying test execution time significantly:

# 1. Write your tests in tests/glob/

# 2. Run tests and check load balance
./tests/run_automated_tests.sh
# Look for "Load Balance: X.XXx variance" in report

# 3. If variance > 2.0x, update durations
./tests/update_test_durations.sh  # Takes ~15-20 min

# 4. Commit duration data
git add .test_durations
git commit -m "chore: update test duration data"

See: glob/TESTING_GUIDE.md for detailed workflow

Files

• run_automated_tests.sh - One-command test execution
• run_parallel_tests.sh - Parallel test runner
• setup_parallel_test_envs.sh - Environment setup
• update_test_durations.sh - Update load balancing data
• TESTING_PROMPT.md - Claude Code automation
• glob/ - Test implementations
• glob/TESTING_GUIDE.md - Development workflow guide

Requirements

• Python 3.12+
• Virtual environment: /home/rho/venv
• ComfyUI branch: ltdrdata/dr-support-pip-cm
• Ports: 8188-8197 available

Troubleshooting

Tests Fail to Start

# Stop existing processes
pkill -f "ComfyUI/main.py"
sleep 2

# Re-run
./tests/run_automated_tests.sh

Slow Execution

If tests take >3 minutes, update duration data:

./tests/update_test_durations.sh

Environment Issues

Rebuild test environments:

rm -rf tests/env/ComfyUI_*
NUM_ENVS=10 ./tests/setup_parallel_test_envs.sh

Generated Files

• Report: .claude/livecontext/automated_test_*.md
• Logs: tests/tmp/test-results-[1-10].log
• Server Logs: tests/tmp/comfyui-parallel-[1-10].log

CI/CD Integration

- name: Run Tests
  run: |
    source /home/rho/venv/bin/activate
    ./tests/run_automated_tests.sh    

Exit code: 0 = pass, 1 = fail

---

Status: ✅ Production-ready (100% pass rate, <3min execution)

Recent Fixes (2025-11-06)

Fixed Test Failures

test_case_sensitivity_full_workflow

• Issue: HTTP 405 error - incorrect API endpoint usage
• Root Cause: Using non-existent /customnode/install endpoint
• Fix: Migrated to queue API (/v2/manager/queue/task + /v2/manager/queue/start)
• Changes: Updated request parameters ("id", "version", "selected_version")
• Location: tests/glob/test_case_sensitivity_integration.py:65-200

test_enable_package

• Issue: AssertionError - found 2 disabled packages instead of 1
• Root Cause: Leftover disabled packages from previous parallel tests
• Fix: Added cleanup before test execution (not just after)
• Changes: Created _cleanup() helper, added filesystem sync delay
• Location: tests/glob/test_enable_disable_api.py:56-111

Improvements

• Increased wait times for parallel execution reliability (20s → 30s)
• Added queue status checking for better debugging
• Enhanced fixture cleanup with filesystem sync delays
• Both tests now pass consistently in parallel execution