mirror of
https://github.com/comfyanonymous/ComfyUI.git
synced 2026-05-06 07:12:30 +08:00
c254fa10d3
- Implement VerifierSelectionNode for creating quality verifiers - Implement InferenceScalingNode that wraps KSampler with quality checks - Add VAE decoding during sampling steps for quality assessment - Include quality verification logic using variance and edge detection - Fix noise/latent handling to match ComfyUI patterns - Add comprehensive error handling and logging - Include documentation and debugging notes
96 lines
3.5 KiB
Markdown
96 lines
3.5 KiB
Markdown
# Debugging Notes - Inference Scaling Implementation
|
|
|
|
## Issues Found and Fixed
|
|
|
|
### 1. **Noise vs Latent Image Handling** ✅ FIXED
|
|
**Issue:** Was incorrectly extracting `samples` from `latent_image` dict and passing it as `noise`.
|
|
|
|
**Fix:**
|
|
- Now properly extracts `latent_image_tensor` from dict
|
|
- Uses `comfy.sample.prepare_noise()` to generate noise from latent_image and seed (same as `common_ksampler`)
|
|
- Passes both `noise` and `latent_image_tensor` to `comfy.sample.sample()`
|
|
|
|
### 2. **Sampler Usage** ✅ FIXED
|
|
**Issue:** Was creating `KSampler` instance but should use `comfy.sample.sample()` directly.
|
|
|
|
**Fix:**
|
|
- Now uses `comfy.sample.sample()` which handles all the setup properly
|
|
- This matches how `common_ksampler` works in `nodes.py`
|
|
|
|
### 3. **VAE Decoding** ✅ FIXED
|
|
**Issue:** Needed to ensure proper tensor format and device handling.
|
|
|
|
**Fix:**
|
|
- VAE.decode expects `[B, C, H, W]` tensor and returns `[B, H, W, C]`
|
|
- Added proper error handling and tensor validation
|
|
- Moves decoded images to CPU for quality checking to avoid GPU memory issues
|
|
|
|
### 4. **Callback Scope Issues** ✅ FIXED
|
|
**Issue:** Variables like `steps` not accessible in callback closure.
|
|
|
|
**Fix:**
|
|
- Removed reference to `steps` in logging (not needed)
|
|
- Added `nonlocal` declarations for variables modified in callback
|
|
- Added check for `check_interval > 0` to avoid division by zero
|
|
|
|
### 5. **Error Handling** ✅ IMPROVED
|
|
**Issue:** Errors could break the sampling process.
|
|
|
|
**Fix:**
|
|
- Added comprehensive try/except blocks
|
|
- Quality check failures don't stop sampling
|
|
- Better logging with appropriate log levels
|
|
- Traceback only in debug mode to avoid spam
|
|
|
|
### 6. **Result Format** ✅ FIXED
|
|
**Issue:** Need to preserve all keys from input latent dict.
|
|
|
|
**Fix:**
|
|
- Uses `latent_image.copy()` to preserve all keys (batch_index, noise_mask, etc.)
|
|
- Only updates `samples` key with result
|
|
|
|
## Remaining Limitations
|
|
|
|
### CFG Adjustment
|
|
- **Status:** Cannot be dynamically adjusted during sampling
|
|
- **Reason:** CFG is set at the start of sampling and used throughout
|
|
- **Workaround:** Quality checking still works and provides valuable feedback
|
|
- **Future:** Could implement adaptive step count or early stopping
|
|
|
|
### Performance
|
|
- **VAE Decoding:** Adds overhead during sampling (decodes at intervals)
|
|
- **Mitigation:** Only checks at specified intervals (default: every 5 steps)
|
|
- **Future:** Could optimize by using TAESD for faster preview decoding
|
|
|
|
## Testing Checklist
|
|
|
|
- [x] Syntax check passes (`py_compile`)
|
|
- [x] No linter errors
|
|
- [ ] Import test (requires Python 3.8+ for `get_origin`)
|
|
- [ ] Runtime test with actual ComfyUI
|
|
- [ ] Test with different samplers/schedulers
|
|
- [ ] Test with different VAE models
|
|
- [ ] Test quality verification logic
|
|
- [ ] Test error handling (invalid verifier_id, etc.)
|
|
|
|
## Code Quality Improvements Made
|
|
|
|
1. **Better Error Messages:** More descriptive exceptions with tracebacks
|
|
2. **Logging:** Added info/warning logs for quality checks
|
|
3. **Type Safety:** Added isinstance checks for tensors
|
|
4. **Memory Management:** Moves decoded images to CPU
|
|
5. **Code Organization:** Follows ComfyUI patterns (`common_ksampler` style)
|
|
|
|
## Files Modified
|
|
|
|
- `comfy_api_nodes/nodes_inference_scaling.py` - Main implementation
|
|
- `DEBUGGING_NOTES.md` - This file
|
|
|
|
## Next Steps for Full Testing
|
|
|
|
1. Test in actual ComfyUI environment
|
|
2. Verify callback is called correctly
|
|
3. Test VAE decoding with different models
|
|
4. Verify quality metrics are reasonable
|
|
5. Test edge cases (empty latents, different batch sizes, etc.)
|