Commit Graph

2291 Commits

Author SHA1 Message Date
John Pollock
093d17c587 Add SeedVR2 temporal chunk and merge nodes
## Summary

Sampling a long SeedVR2 video in one pass runs out of VRAM: the DiT working set grows linearly with latent frames times pixel area, so a 100 frame clip at 4x upscale needs more memory than any consumer card has. This PR adds two workflow-level nodes that split the latent into overlapping temporal chunks and recombine the sampled chunks with a Hann crossfade. The executor's list handling runs the stock KSampler once per chunk, so the sampler itself is untouched.

- **Chunk SeedVR2 Latent** splits the latent on the temporal axis. `frames_per_chunk` is in pixel frames on the 4n+1 grid, `temporal_overlap` sets how many latent frames adjacent chunks share, and `chunking_mode=auto` solves the chunk size from measured free VRAM and the latent's own dimensions. The node outputs the effective overlap so the merge is wired, not typed.
- **Merge SeedVR2 Latent Chunks** recombines the sampled chunks in order, crossfading each shared region with a Hann window (flat shoulders on the outer thirds, fade across the middle third). Zero overlap is a plain concatenation, bit-identical to `torch.cat`.

## Changes

- Added 2 nodes and 1 crossfade helper to `comfy_extras/nodes_seedvr.py` (+201 lines).
- Added 4 chunk-law constants to `comfy/ldm/seedvr/constants.py` (+10 lines).
- Added pytest unit tests in `tests-unit/comfy_extras_test/test_seedvr2_temporal_chunk.py` (+66 lines): chunk geometry, 4n+1 and mode validation, overlap clamping, temporal noise mask slicing (5-D and 4-D), the auto law including batch scaling, crossfade weights and blend direction, round trip, and metadata handling.

Everything outside the two new nodes is byte-identical to the base branch. The new constants are read only by the chunk node, and workflows that do not use these nodes take no new code path.

## Auto mode calibration

The auto law is `max_latent_frames = (free_GiB - reserved - margin) / (0.30 * megapixels)`, calibrated on an RTX 5090 (32 GB) with the 3b fp16 model: a 17-cell resolution sweep plus temporal bisection located the activation wall and confirmed it is linear in latent frames times pixel area (the same total-voxel budget holds from 1.5:1 through 24:1 aspect ratios and under transposition). The margin is four standard deviations of the measured run-to-run spread, which costs about one latent frame of chunk size and makes an out-of-memory failure a lottery ticket rather than a coin flip. Manual mode bypasses the law entirely.

On a 32 GB card, a 640x480x100 input at 4x upscale sampled as a single chunk allocates past 31 GiB and dies; auto mode picks 49 frame chunks (three chunks with overlap) and the full pipeline completes in about 240 seconds. The same law stands down on small inputs: a 320x240x100 clip runs as a single chunk because it fits.

## Example workflow

Load a video, 4x upscale, auto chunking, temporal overlap 3. Expected output for a 640x480x100 input: 2560x1920, 100 frames, seams invisible at the default overlap.

<details>
<summary>API workflow JSON</summary>

```json
{
  "14": {"inputs": {"vae_name": "ema_vae_fp16.safetensors"}, "class_type": "VAELoader"},
  "17": {"inputs": {"video": ["24", 0]}, "class_type": "GetVideoComponents"},
  "20": {"inputs": {"fps": ["17", 2], "images": ["30", 0], "audio": ["17", 1]}, "class_type": "CreateVideo"},
  "21": {"inputs": {"tile_size": 192, "overlap": 64, "temporal_size": 64, "temporal_overlap": 8, "pixels": ["27", 0], "vae": ["14", 0]}, "class_type": "VAEEncodeTiled"},
  "22": {"inputs": {"tile_size": 256, "overlap": 32, "temporal_size": 64, "temporal_overlap": 8, "samples": ["33", 0], "vae": ["14", 0]}, "class_type": "VAEDecodeTiled"},
  "23": {"inputs": {"unet_name": "seedvr2_3b_fp16.safetensors", "weight_dtype": "default"}, "class_type": "UNETLoader"},
  "24": {"inputs": {"file": "input.mp4", "video-preview": ""}, "class_type": "LoadVideo"},
  "25": {"inputs": {"filename_prefix": "video/seedvr2_upscale", "format": "auto", "codec": "auto", "video": ["20", 0]}, "class_type": "SaveVideo"},
  "26": {"inputs": {"resize_type": "scale by multiplier", "resize_type.multiplier": 4, "scale_method": "bicubic", "input": ["17", 0]}, "class_type": "ResizeImageMaskNode"},
  "27": {"inputs": {"resized_images": ["26", 0]}, "class_type": "SeedVR2Preprocess"},
  "28": {"inputs": {"model": ["23", 0], "vae_conditioning": ["32", 0]}, "class_type": "SeedVR2Conditioning"},
  "29": {"inputs": {"seed": 5770521, "steps": 1, "cfg": 1, "sampler_name": "euler", "scheduler": "simple", "denoise": 1, "model": ["23", 0], "positive": ["28", 0], "negative": ["28", 1], "latent_image": ["32", 0]}, "class_type": "KSampler"},
  "30": {"inputs": {"color_correction_method": "lab", "images": ["22", 0], "original_resized_images": ["26", 0]}, "class_type": "SeedVR2PostProcessing"},
  "32": {"inputs": {"frames_per_chunk": 21, "temporal_overlap": 3, "chunking_mode": "auto", "latent": ["21", 0]}, "class_type": "SeedVR2TemporalChunk"},
  "33": {"inputs": {"temporal_overlap": ["32", 1], "latent_chunks": ["29", 0]}, "class_type": "SeedVR2TemporalMerge"}
}
```

</details>

## Prior art

- Reference implementation: https://github.com/ByteDance-Seed/SeedVR
- Community precedent for temporal chunking with blended reassembly: https://github.com/numz/ComfyUI-SeedVR2_VideoUpscaler

Chunk boundaries are a mathematical compromise: the model attends within a chunk, so different chunkings produce different outputs. The overlap crossfade hides the seam; power users can widen or zero the overlap from the workflow.
2026-07-04 13:05:19 -05:00
comfyanonymous
cb16da18f9
Fix nvfp4 model detection. 2026-07-03 19:00:04 -07:00
comfyanonymous
d3f5ae56b3
Fix cast issue. 2026-07-03 01:37:29 -04:00
Alexis Rolland
e06033fb97
Merge branch 'master' into seedvr2-native-support-v5 2026-07-03 12:36:42 +08:00
comfyanonymous
6d72960989
Fix ruff. 2026-07-02 23:04:52 -04:00
comfyanonymous
c7b2c3b569 Refactors and cleanups. 2026-07-02 22:59:38 -04:00
comfyanonymous
77d42ed7e9 Remove SeedVR2ProgressiveSampler. 2026-07-01 22:19:37 -04:00
comfyanonymous
f437d87155 Cleanups using AGENTS.md 2026-07-01 22:17:51 -04:00
John Pollock
e595965392 Remove SeedVR2 VAE memory convolution workaround 2026-07-01 15:27:21 -05:00
Silver
2c935de1b1
Fix Qwen3-VL tokenizer crash with custom embeddings (#14713)
Some checks are pending
Detect Unreviewed Merge / detect (push) Waiting to run
Python Linting / Run Ruff (push) Waiting to run
Python Linting / Run Pylint (push) Waiting to run
Full Comfy CI Workflow Runs / test-stable (12.1, , linux, 3.10, [self-hosted Linux], stable) (push) Waiting to run
Full Comfy CI Workflow Runs / test-stable (12.1, , linux, 3.11, [self-hosted Linux], stable) (push) Waiting to run
Full Comfy CI Workflow Runs / test-stable (12.1, , linux, 3.12, [self-hosted Linux], stable) (push) Waiting to run
Full Comfy CI Workflow Runs / test-unix-nightly (12.1, , linux, 3.11, [self-hosted Linux], nightly) (push) Waiting to run
Execution Tests / test (macos-latest) (push) Waiting to run
Execution Tests / test (ubuntu-latest) (push) Waiting to run
Execution Tests / test (windows-latest) (push) Waiting to run
Test server launches without errors / test (push) Waiting to run
Unit Tests / test (macos-latest) (push) Waiting to run
Unit Tests / test (ubuntu-latest) (push) Waiting to run
Unit Tests / test (windows-2022) (push) Waiting to run
2026-07-01 21:15:07 +03:00
Matt Miller
1c59659a2f
feat: make asset hashing opt-in via --enable-asset-hashing, off by default (#14663)
Add a --enable-asset-hashing CLI flag (action=store_true, default False)
and plumb it into the two asset-seeder call sites in main.py that
previously hardcoded compute_hashes=True (the startup scan and the
post-job output enqueue). Local runs now skip blake3 hashing unless the
user opts in, avoiding the startup/per-output cost on large models
directories while keeping hashing available for asset-portability
features.

Co-authored-by: Alexis Rolland <alexisrolland@hotmail.com>
2026-06-30 14:13:20 -07:00
comfyanonymous
79c555ce6b
Fix int8 mm being skipped on offloaded lora weights. (#14669) 2026-06-28 23:52:36 -04:00
comfyanonymous
470ac36a0a
Fix int8 loras causing lower quality requant with wrong settings. (#14650)
* Update comfy-kitchen

* Support requantizing with same settings as orig quant.
2026-06-26 16:41:29 -07:00
comfyanonymous
1a510f0423
Support int8 models. (#14636) 2026-06-25 11:23:58 -07:00
comfyanonymous
b910f4fa2a
More accurate memory usage factor for krea 2. (#14594) 2026-06-23 16:50:48 +08:00
Jukka Seppänen
2a61015582
feat: Support Krea2 (#14589)
Some checks are pending
Detect Unreviewed Merge / detect (push) Waiting to run
Python Linting / Run Ruff (push) Waiting to run
Python Linting / Run Pylint (push) Waiting to run
Full Comfy CI Workflow Runs / test-stable (12.1, , linux, 3.10, [self-hosted Linux], stable) (push) Waiting to run
Full Comfy CI Workflow Runs / test-stable (12.1, , linux, 3.11, [self-hosted Linux], stable) (push) Waiting to run
Full Comfy CI Workflow Runs / test-stable (12.1, , linux, 3.12, [self-hosted Linux], stable) (push) Waiting to run
Full Comfy CI Workflow Runs / test-unix-nightly (12.1, , linux, 3.11, [self-hosted Linux], nightly) (push) Waiting to run
Execution Tests / test (ubuntu-latest) (push) Waiting to run
Execution Tests / test (macos-latest) (push) Waiting to run
Execution Tests / test (windows-latest) (push) Waiting to run
Test server launches without errors / test (push) Waiting to run
Unit Tests / test (macos-latest) (push) Waiting to run
Unit Tests / test (ubuntu-latest) (push) Waiting to run
Unit Tests / test (windows-2022) (push) Waiting to run
2026-06-22 14:35:00 -07:00
comfyanonymous
e00b55631a
Small anima optimization. (#14557) 2026-06-20 08:05:28 +08:00
Barish Ozbay
cd77c551d6
feat: Context Windows sampling with LTX2 models and IC-LoRa guides (CORE-3) (#13325) 2026-06-20 07:47:31 +08:00
John Pollock
ad04a6199e Merge branch 'master' into seedvr2-native-support-v5 2026-06-18 09:58:04 -05:00
comfyanonymous
52257bb435
Add negative prompt to boogu edit node and set min images to 0. (#14529) 2026-06-17 15:42:29 -07:00
Jukka Seppänen
e25c391888
feat: Support Boogu-Image (CORE-308) (#14523)
Some checks are pending
Unit Tests / test (windows-2022) (push) Waiting to run
Detect Unreviewed Merge / detect (push) Waiting to run
Python Linting / Run Ruff (push) Waiting to run
Python Linting / Run Pylint (push) Waiting to run
Full Comfy CI Workflow Runs / test-stable (12.1, , linux, 3.10, [self-hosted Linux], stable) (push) Waiting to run
Full Comfy CI Workflow Runs / test-stable (12.1, , linux, 3.11, [self-hosted Linux], stable) (push) Waiting to run
Full Comfy CI Workflow Runs / test-stable (12.1, , linux, 3.12, [self-hosted Linux], stable) (push) Waiting to run
Full Comfy CI Workflow Runs / test-unix-nightly (12.1, , linux, 3.11, [self-hosted Linux], nightly) (push) Waiting to run
Execution Tests / test (macos-latest) (push) Waiting to run
Execution Tests / test (ubuntu-latest) (push) Waiting to run
Execution Tests / test (windows-latest) (push) Waiting to run
Test server launches without errors / test (push) Waiting to run
Unit Tests / test (macos-latest) (push) Waiting to run
Unit Tests / test (ubuntu-latest) (push) Waiting to run
2026-06-17 14:22:36 -07:00
Jukka Seppänen
ca3dbe206c
Allow using Qwen3-VL as flux2 klein text encoder (again) (#14526) 2026-06-17 08:45:06 -07:00
Jukka Seppänen
a590d60bb1
feat: SCAIL-2 multireference (CORE-310) (#14509)
Some checks are pending
Detect Unreviewed Merge / detect (push) Waiting to run
Python Linting / Run Ruff (push) Waiting to run
Python Linting / Run Pylint (push) Waiting to run
Full Comfy CI Workflow Runs / test-stable (12.1, , linux, 3.10, [self-hosted Linux], stable) (push) Waiting to run
Full Comfy CI Workflow Runs / test-stable (12.1, , linux, 3.11, [self-hosted Linux], stable) (push) Waiting to run
Full Comfy CI Workflow Runs / test-stable (12.1, , linux, 3.12, [self-hosted Linux], stable) (push) Waiting to run
Full Comfy CI Workflow Runs / test-unix-nightly (12.1, , linux, 3.11, [self-hosted Linux], nightly) (push) Waiting to run
Execution Tests / test (ubuntu-latest) (push) Waiting to run
Execution Tests / test (macos-latest) (push) Waiting to run
Execution Tests / test (windows-latest) (push) Waiting to run
Test server launches without errors / test (push) Waiting to run
Unit Tests / test (macos-latest) (push) Waiting to run
Unit Tests / test (ubuntu-latest) (push) Waiting to run
Unit Tests / test (windows-2022) (push) Waiting to run
* SCAIl-2: support multiref
2026-06-17 16:21:23 +03:00
Jukka Seppänen
fc964047e7
feat: Support text generation with Qwen3-VL (CORE-276) (#14298) 2026-06-17 08:12:44 +08:00
rattus
ec4dec93d2
Comfy Aimdo 0.4.10 + Dynamic --reserve-vram + --vram-headroom (#14480)
* main: implement --vram-headroom

Implement --vram-headroom for dynamic vram as a hybrid debug/diagnostic
option that can be used for people who still report shared VRAM spills.
They can trial and error the setting to maintain a bit more headroom to
avoid shared VRAM spills.

* main: implement --reserve-vram

Implement --reserve-vram as extra headroom on the simple method which
is semantically as close as possible to the stated functionality and
formet behaviour of non-dynamic VRAM.
2026-06-15 07:54:36 -07:00
comfyanonymous
7277d99d3a
Use comfy kitchen apply rope in omnigen2 model. (#14442) 2026-06-13 09:38:39 +08:00
John Pollock
d54ce3d781 Add SeedVR2 workflow nodes 2026-06-12 11:19:35 -05:00
John Pollock
a7ea0c2773 Add SeedVR2 VAE support 2026-06-12 11:19:34 -05:00
John Pollock
cd18c4460a Add SeedVR2 model support 2026-06-12 11:19:26 -05:00
rattus
d7a552720c
add --high-ram option (#14437)
Add this option for users who know they have so much ram they want
to pin everything or have a pagefile that outruns their disk speed.

The removes the RAM pressure caps completely and pins behind the
primary model load forcing all models to be permanently comitted
to RAM.
2026-06-12 07:53:33 -07:00
comfyanonymous
02656ea0bb
Fix potential dtype issue with ideogram 4. (#14436) 2026-06-12 07:51:12 -07:00
Jedrzej Kosinski
befc321438
Make --enable-manager-legacy-ui imply --enable-manager (#14421) 2026-06-12 06:45:22 +08:00
Barish Ozbay
91187c58d9
Improve context window resizing for SCAIL2 (CORE-286) (#14394) 2026-06-11 13:37:43 +08:00
rattus
bda19b2604
ops: tolerate already force casted dynamic weight (#14410)
Some custom nodes .to weights completely out of load context which
can wreak havoc if its for a model that is not active. Detect this
condition and just let it fall-through to the non-dynamic loader
straight up.
2026-06-10 20:32:57 -07:00
rattus
6d18f4adac
main: force cudnn.benchmark to false (#14390)
Some custom nodes try to set this true globally. It messes with dynamic
VRAM with one-off spikes that can OOM but this is also very high risk
for windows where such allocations might get serviced by shared memory
fallback.

Trump it.
2026-06-10 13:54:32 -04:00
Kohaku-Blueleaf
f350acdf21
[Trainer/bug] Ensure model is not inference mode (CORE-72) (#13400)
* Ensure model is not inference mode

* force clone inside training mode to avoid inference tensor

* Allow force deepcopy for model patcher
2026-06-09 23:07:47 -04:00
Talmaj
5ece24e73c
Depth anything 3 (Core-135) (#13853)
Co-authored-by: Alexis Rolland <alexisrolland@hotmail.com>
2026-06-10 09:28:24 +08:00
Jukka Seppänen
f8e51b674c
feat: Add Bernini-R model support (Wan video) (CORE-279) (#14216)
Some checks are pending
Detect Unreviewed Merge / detect (push) Waiting to run
Python Linting / Run Ruff (push) Waiting to run
Python Linting / Run Pylint (push) Waiting to run
Full Comfy CI Workflow Runs / test-stable (12.1, , linux, 3.10, [self-hosted Linux], stable) (push) Waiting to run
Full Comfy CI Workflow Runs / test-stable (12.1, , linux, 3.11, [self-hosted Linux], stable) (push) Waiting to run
Full Comfy CI Workflow Runs / test-stable (12.1, , linux, 3.12, [self-hosted Linux], stable) (push) Waiting to run
Full Comfy CI Workflow Runs / test-unix-nightly (12.1, , linux, 3.11, [self-hosted Linux], nightly) (push) Waiting to run
Execution Tests / test (macos-latest) (push) Waiting to run
Execution Tests / test (ubuntu-latest) (push) Waiting to run
Execution Tests / test (windows-latest) (push) Waiting to run
Test server launches without errors / test (push) Waiting to run
Unit Tests / test (macos-latest) (push) Waiting to run
Unit Tests / test (ubuntu-latest) (push) Waiting to run
Unit Tests / test (windows-2022) (push) Waiting to run
2026-06-10 07:47:34 +08:00
rattus
6f01b244a2
mm: dont reset cast buffers in cleanup_models_gc() (#14372)
cleanup_models_gc can be called once per load_models_gpu via
free_memory, which in turn can de-activate an active model via
this reset_cast_buffers.

cleanup_models_gc() could also come via obscure garbage collector
paths so limit reset_cast_buffers to the post-node callsite instead.
2026-06-09 13:57:04 -04:00
Jukka Seppänen
184009c2f6
feat: Add model support for SCAIL-2 (#14373)
* initial SCAIL2 support
2026-06-09 18:24:09 +03:00
kelseyee
07c53f8f0f
Add LoRA key mapping for LTXV/LTXAV models (#14349) 2026-06-09 09:57:58 -04:00
rattus
1639dc7a70
main/server: Add --debug-hang (#14371)
Add an option to debug a hang with ctrl-C, dumping the backtraces to
see where its stuck or slow.
2026-06-09 09:55:00 -04:00
Jukka Seppänen
8ed7f458d0
Allow custom templates with Ideogram4 TE (#14374) 2026-06-09 21:11:05 +08:00
Alexis Rolland
f89999289a
fix: Add back apply_rotary_emb for Qwen Image (#14364)
Some checks are pending
Detect Unreviewed Merge / detect (push) Waiting to run
Python Linting / Run Ruff (push) Waiting to run
Python Linting / Run Pylint (push) Waiting to run
Full Comfy CI Workflow Runs / test-stable (12.1, , linux, 3.10, [self-hosted Linux], stable) (push) Waiting to run
Full Comfy CI Workflow Runs / test-stable (12.1, , linux, 3.11, [self-hosted Linux], stable) (push) Waiting to run
Full Comfy CI Workflow Runs / test-stable (12.1, , linux, 3.12, [self-hosted Linux], stable) (push) Waiting to run
Full Comfy CI Workflow Runs / test-unix-nightly (12.1, , linux, 3.11, [self-hosted Linux], nightly) (push) Waiting to run
Execution Tests / test (macos-latest) (push) Waiting to run
Execution Tests / test (ubuntu-latest) (push) Waiting to run
Execution Tests / test (windows-latest) (push) Waiting to run
Test server launches without errors / test (push) Waiting to run
Unit Tests / test (macos-latest) (push) Waiting to run
Unit Tests / test (ubuntu-latest) (push) Waiting to run
Unit Tests / test (windows-2022) (push) Waiting to run
2026-06-09 11:55:49 +08:00
comfyanonymous
00b633f368
Revert "Add SeedVR2 support (CORE-6) (#14110)" (#14359)
Some checks are pending
Detect Unreviewed Merge / detect (push) Waiting to run
Python Linting / Run Ruff (push) Waiting to run
Python Linting / Run Pylint (push) Waiting to run
Full Comfy CI Workflow Runs / test-stable (12.1, , linux, 3.10, [self-hosted Linux], stable) (push) Waiting to run
Full Comfy CI Workflow Runs / test-stable (12.1, , linux, 3.11, [self-hosted Linux], stable) (push) Waiting to run
Full Comfy CI Workflow Runs / test-stable (12.1, , linux, 3.12, [self-hosted Linux], stable) (push) Waiting to run
Full Comfy CI Workflow Runs / test-unix-nightly (12.1, , linux, 3.11, [self-hosted Linux], nightly) (push) Waiting to run
Execution Tests / test (macos-latest) (push) Waiting to run
Execution Tests / test (ubuntu-latest) (push) Waiting to run
Execution Tests / test (windows-latest) (push) Waiting to run
Test server launches without errors / test (push) Waiting to run
Unit Tests / test (macos-latest) (push) Waiting to run
Unit Tests / test (ubuntu-latest) (push) Waiting to run
Unit Tests / test (windows-2022) (push) Waiting to run
This reverts commit 7863cf0e53.
2026-06-08 18:00:20 -04:00
John Pollock
7863cf0e53
Add SeedVR2 support (CORE-6) (#14110) 2026-06-08 18:15:05 +08:00
rattus
410df27253
Fix interoperation with external source of pinned memory pressure (#14252)
* mm: split off registration helper to doer and headroom calc

* pinned_memory: implement registration comfy side

Move away from Aimdo buffer registrations which seem fraught with
danger and do it comfy side. Just start with the basic move.

* pinned_memory: do registrations as portable memory

* pinned_memory: discard async errors on registration fail

Like the good ol days.

* pinned_memory: implement abs shortfall retry

If pinned registration happens to fail despite the previous budget
ensures, consider the allocation shortfall, ensure it again, and
try again. This allows comfy pins to interoperate with other software
that might be doing substantive pinning.
2026-06-05 08:39:35 -07:00
comfyanonymous
514bb8ba21
Fix ideogram if model dtype gets set to fp8. (#14291) 2026-06-04 19:20:22 -07:00
comfyanonymous
8e3045a90b
Memory usage factor for ideogram 4 on non dynamic vram. (#14264) 2026-06-03 12:19:18 -04:00
Jukka Seppänen
24f9a020ce
Support Ideogram4 (#14259) 2026-06-03 08:41:44 -07:00