1849 Commits

Author	SHA1	Message	Date
kijai	1a88fde6d9	Update kandinsky5.py	2025-12-03 16:08:48 +02:00
kijai	54b77e140f	Add image model's own chat template, remove unused image2video template	2025-12-03 16:08:47 +02:00
kijai	dafa2695d4	Code cleanup, don't force the fp32 layers as it has minimal effect	2025-12-03 16:08:47 +02:00
kijai	c14bfb0554	Update memory_usage_factor	2025-12-03 16:08:47 +02:00
kijai	b24bac604a	Update chunking	2025-12-03 16:08:47 +02:00
kijai	a3ce1e02d7	Further reduce peak VRAM consumption by chunking ffn	2025-12-03 16:08:47 +02:00
kijai	1baf5ec4af	Reduce peak VRAM usage a bit	2025-12-03 16:08:47 +02:00
kijai	4acd51abab	Handle latent dim difference for image model in the VAE instead	2025-12-03 16:08:47 +02:00
kijai	653ad84203	Fix typo	2025-12-03 16:08:47 +02:00
kijai	f1a5f6f5b3	Rever RoPE scaling to simpler one	2025-12-03 16:08:47 +02:00
kijai	e74db2404f	Update supported_models.py	2025-12-03 16:08:47 +02:00
kijai	68c712ef7e	Don't scale RoPE for lite model as that just doesn't work...	2025-12-03 16:08:46 +02:00
kijai	a72c7edd3c	Support official LoRAs	2025-12-03 16:08:46 +02:00
kijai	dd318ada2f	Support block replace patches (SLG mostly)	2025-12-03 16:08:46 +02:00
kijai	ccfc3ab7cf	Support text to image model	2025-12-03 16:08:46 +02:00
kijai	292c3576c2	Fix I2V, add necessary latent post process nodes	2025-12-03 16:08:46 +02:00
kijai	8f02217f85	Code cleanup, optimizations, use fp32 for all layers originally at fp32	2025-12-03 16:08:46 +02:00
kijai	0920cdcf63	Add transformer_options for attention	2025-12-03 16:08:46 +02:00
kijai	8fd49be716	Fix fp8_scaled text encoder	2025-12-03 16:08:46 +02:00
kijai	5f1346ccd1	Fix fp8	2025-12-03 16:08:46 +02:00
kijai	8c9f7cc781	Update kandinsky5.py	2025-12-03 16:08:46 +02:00
kijai	530df3d710	Add Kandinsky5 model support ... lite and pro T2V tested to work	2025-12-03 16:08:46 +02:00
rattus	519c941165	Prs/lora reservations (reduce massive Lora reservations especially on Flux2) (#11069) ... * mp: only count the offload cost of math once This was previously bundling the combined weight storage and computation cost * ops: put all post async transfer compute on the main stream Some models have massive weights that need either complex dequantization or lora patching. Don't do these patchings on the offload stream, instead do them on the main stream to syncrhonize the potentially large vram spikes for these compute processes. This avoids having to assume a worst case scenario of multiple offload streams all spiking VRAM is parallel with whatever the main stream is doing.	2025-12-03 02:28:45 -05:00
rattus	73f5649196	Implement temporal rolling VAE (Major VRAM reductions in Hunyuan and Kandinsky) (#10995) ... * hunyuan upsampler: rework imports Remove the transitive import of VideoConv3d and Resnet and takes these from actual implementation source. * model: remove unused give_pre_end According to git grep, this is not used now, and was not used in the initial commit that introduced it (see below). This semantic is difficult to implement temporal roll VAE for (and would defeat the purpose). Rather than implement the complex if, just delete the unused feature. (venv) rattus@rattus-box2:~/ComfyUI$ git log --oneline 220afe33 (HEAD) Initial commit. (venv) rattus@rattus-box2:~/ComfyUI$ git grep give_pre comfy/ldm/modules/diffusionmodules/model.py: resolution, z_channels, give_pre_end=False, tanh_out=False, use_linear_attn=False, comfy/ldm/modules/diffusionmodules/model.py: self.give_pre_end = give_pre_end comfy/ldm/modules/diffusionmodules/model.py: if self.give_pre_end: (venv) rattus@rattus-box2:~/ComfyUI$ git co origin/master Previous HEAD position was 220afe33 Initial commit. HEAD is now at 9d8a8179 Enable async offloading by default on Nvidia. (#10953) (venv) rattus@rattus-box2:~/ComfyUI$ git grep give_pre comfy/ldm/modules/diffusionmodules/model.py: resolution, z_channels, give_pre_end=False, tanh_out=False, use_linear_attn=False, comfy/ldm/modules/diffusionmodules/model.py: self.give_pre_end = give_pre_end comfy/ldm/modules/diffusionmodules/model.py: if self.give_pre_end: * move refiner VAE temporal roller to core Move the carrying conv op to the common VAE code and give it a better name. Roll the carry implementation logic for Resnet into the base class and scrap the Hunyuan specific subclass. * model: Add temporal roll to main VAE decoder If there are no attention layers, its a standard resnet and VideoConv3d is asked for, substitute in the temporal rolloing VAE algorithm. This reduces VAE usage by the temporal dimension (can be huge VRAM savings). * model: Add temporal roll to main VAE encoder If there are no attention layers, its a standard resnet and VideoConv3d is asked for, substitute in the temporal rolling VAE algorithm. This reduces VAE usage by the temporal dimension (can be huge VRAM savings).	2025-12-02 22:49:29 -05:00
comfyanonymous	b94d394a64	Support Z Image alibaba pai fun controlnets. (#11062) ... These are not actual controlnets so put it in the models/model_patches folder and use the ModelPatchLoader + QwenImageDiffsynthControlnet node to use it.	2025-12-02 21:38:31 -05:00
rattus	277237ccc1	attention: use flag based OOM fallback (#11038) ... Exception ref all local variables for the lifetime of exception context. Just set a flag and then if to dump the exception before falling back.	2025-12-02 17:24:19 -05:00
comfyanonymous	daaceac769	Hack to make zimage work in fp16. (#11057)	2025-12-02 17:11:58 -05:00
Dr.Lt.Data	b4a20acc54	feat: Support ComfyUI-Manager for pip version (#7555)	2025-12-01 22:32:52 -05:00
comfyanonymous	878db3a727	Implement the Ovis image model. (#11030)	2025-12-01 20:56:17 -05:00
comfyanonymous	2640acb31c	Update qwen tokenizer to add qwen 3 tokens. (#11029) ... Doesn't actually change anything for current workflows because none of the current models have a template with the think tokens.	2025-12-01 17:13:48 -05:00
comfyanonymous	0a6746898d	Make the ScaleRope node work on Z Image and Lumina. (#10994)	2025-11-29 18:00:55 -05:00
comfyanonymous	5151cff293	Add some missing z image lora layers. (#10980)	2025-11-28 23:55:00 -05:00
comfyanonymous	52a32e2b32	Support some z image lora formats. (#10978)	2025-11-28 21:12:42 -05:00
Jukka Seppänen	b907085709	Support video tiny VAEs (#10884) ... * Support video tiny VAEs * lighttaew scaling fix * Also support video taes in previews Only first frame for now as live preview playback is currently only available through VHS custom nodes. * Support Wan 2.1 lightVAE * Relocate elif block and set Wan VAE dim directly without using pruning rate for lightvae	2025-11-28 19:40:19 -05:00
rattus	0ff0457892	mm: wrap the raw stream in context manager (#10958) ... The documentation of torch.foo.Stream being usable with with: suggests it starts at version 2.7. Use the old API for backwards compatibility.	2025-11-28 16:38:12 -05:00
Urle Sistiana	6484ac89dc	fix QuantizedTensor.is_contiguous (#10956) (#10959)	2025-11-28 16:33:07 -05:00
comfyanonymous	f55c98a89f	Disable offload stream when torch compile. (#10961)	2025-11-28 16:16:46 -05:00
comfyanonymous	9d8a817985	Enable async offloading by default on Nvidia. (#10953) ... Add --disable-async-offload to disable it. If this causes OOMs that go away when you --disable-async-offload please report it.	2025-11-27 17:46:12 -05:00
rattus	3f382a4f98	quant ops: Dequantize weight in-place (#10935) ... In flux2 these weights are huge (200MB). As plain_tensor is a throw-away deep copy, do this multiplication in-place to save VRAM.	2025-11-27 08:06:30 -08:00
rattus	f17251bec6	Account for the VRAM cost of weight offloading (#10733) ... * mm: default to 0 for NUM_STREAMS Dont count the compute stream as an offload stream. This makes async offload accounting easier. * mm: remove 128MB minimum This is from a previous offloading system requirement. Remove it to make behaviour of the loader and partial unloader consistent. * mp: order the module list by offload expense Calculate an approximate offloading temporary VRAM cost to offload a weight and primary order the module load list by that. In the simple case this is just the same as the module weight, but with Loras, a weight with a lora consumes considerably more VRAM to do the Lora application on-the-fly. This will slightly prioritize lora weights, but is really for proper VRAM offload accounting. * mp: Account for the VRAM cost of weight offloading when checking the VRAM headroom, assume that the weight needs to be offloaded, and only load if it has space for both the load and offload * the number of streams. As the weights are ordered from largest to smallest by offload cost this is guaranteed to fit in VRAM (tm), as all weights that follow will be smaller. Make the partial unload aware of this system as well by saving the budget for offload VRAM to the model state and accounting accordingly. Its possible that partial unload increases the size of the largest offloaded weights, and thus needs to unload a little bit more than asked to accomodate the bigger temp buffers. Honor the existing codes floor on model weight loading of 128MB by having the patcher honor this separately withough regard to offloading. Otherwise when MM specifies its 128MB minimum, MP will see the biggest weights, and budget that 128MB to only offload buffer and load nothing which isnt the intent of these minimums. The same clamp applies in case of partial offload of the currently loading model.	2025-11-27 01:03:03 -05:00
Haoming	c38e7d6599	block info (#10841)	2025-11-26 20:28:44 -08:00
comfyanonymous	eaf68c9b5b	Make lora training work on Z Image and remove some redundant nodes. (#10927)	2025-11-26 19:25:32 -05:00
comfyanonymous	f16219e3aa	Add cheap latent preview for flux 2. (#10907) ... Thank you to the person who calculated them. You saved me a percent of my time.	2025-11-26 04:00:43 -05:00
comfyanonymous	58b8574661	Fix Flux2 reference image mem estimation. (#10905)	2025-11-26 02:36:19 -05:00
comfyanonymous	bdb10a583f	Fix loras not working on mixed fp8. (#10899)	2025-11-26 00:07:58 -05:00
comfyanonymous	0e24dbb19f	Adjustments to Z Image. (#10893)	2025-11-25 19:02:51 -05:00
comfyanonymous	e9aae31fa2	Z Image model. (#10892)	2025-11-25 18:41:45 -05:00
comfyanonymous	d196a905bb	Lower vram usage for flux 2 text encoder. (#10887)	2025-11-25 14:58:39 -05:00
comfyanonymous	dff996ca39	Fix crash. (#10885)	2025-11-25 14:30:24 -05:00
comfyanonymous	6b573ae0cb	Flux 2 (#10879)	2025-11-25 10:50:19 -05:00