ComfyUI/DEBUGGING_NOTES.md

Debugging Notes - Inference Scaling Implementation

Issues Found and Fixed

1. Noise vs Latent Image Handling ✅ FIXED

Issue: Was incorrectly extracting samples from latent_image dict and passing it as noise.

Fix:

• Now properly extracts latent_image_tensor from dict
• Uses comfy.sample.prepare_noise() to generate noise from latent_image and seed (same as common_ksampler)
• Passes both noise and latent_image_tensor to comfy.sample.sample()

2. Sampler Usage ✅ FIXED

Issue: Was creating KSampler instance but should use comfy.sample.sample() directly.

Fix:

• Now uses comfy.sample.sample() which handles all the setup properly
• This matches how common_ksampler works in nodes.py

3. VAE Decoding ✅ FIXED

Issue: Needed to ensure proper tensor format and device handling.

Fix:

• VAE.decode expects [B, C, H, W] tensor and returns [B, H, W, C]
• Added proper error handling and tensor validation
• Moves decoded images to CPU for quality checking to avoid GPU memory issues

4. Callback Scope Issues ✅ FIXED

Issue: Variables like steps not accessible in callback closure.

Fix:

• Removed reference to steps in logging (not needed)
• Added nonlocal declarations for variables modified in callback
• Added check for check_interval > 0 to avoid division by zero

5. Error Handling ✅ IMPROVED

Issue: Errors could break the sampling process.

Fix:

• Added comprehensive try/except blocks
• Quality check failures don't stop sampling
• Better logging with appropriate log levels
• Traceback only in debug mode to avoid spam

6. Result Format ✅ FIXED

Issue: Need to preserve all keys from input latent dict.

Fix:

• Uses latent_image.copy() to preserve all keys (batch_index, noise_mask, etc.)
• Only updates samples key with result

Remaining Limitations

CFG Adjustment

• Status: Cannot be dynamically adjusted during sampling
• Reason: CFG is set at the start of sampling and used throughout
• Workaround: Quality checking still works and provides valuable feedback
• Future: Could implement adaptive step count or early stopping

Performance

• VAE Decoding: Adds overhead during sampling (decodes at intervals)
• Mitigation: Only checks at specified intervals (default: every 5 steps)
• Future: Could optimize by using TAESD for faster preview decoding

Testing Checklist

• Syntax check passes (py_compile)
• No linter errors
• Import test (requires Python 3.8+ for get_origin)
• Runtime test with actual ComfyUI
• Test with different samplers/schedulers
• Test with different VAE models
• Test quality verification logic
• Test error handling (invalid verifier_id, etc.)

Code Quality Improvements Made

1. Better Error Messages: More descriptive exceptions with tracebacks
2. Logging: Added info/warning logs for quality checks
3. Type Safety: Added isinstance checks for tensors
4. Memory Management: Moves decoded images to CPU
5. Code Organization: Follows ComfyUI patterns (common_ksampler style)

Files Modified

• comfy_api_nodes/nodes_inference_scaling.py - Main implementation
• DEBUGGING_NOTES.md - This file

Next Steps for Full Testing

1. Test in actual ComfyUI environment
2. Verify callback is called correctly
3. Test VAE decoding with different models
4. Verify quality metrics are reasonable
5. Test edge cases (empty latents, different batch sizes, etc.)