1998 Commits

Author	SHA1	Message	Date
comfyanonymous	53e762a3af	Print memory summary on OOM to help with debugging. (#11613)	2026-01-03 22:28:38 -05:00
comfyanonymous	9a552df898	Remove leftover scaled_fp8 key. (#11603)	2026-01-02 17:28:10 -08:00
comfyanonymous	65cfcf5b1b	New Year ruff cleanup. (#11595)	2026-01-01 22:06:14 -05:00
comfyanonymous	1bdc9a947f	Remove duplicate import of model_management (#11587)	2025-12-31 19:29:55 -05:00
comfyanonymous	d622a61874	Refactor: move clip_preprocess to comfy.clip_model (#11586)	2025-12-31 17:38:36 -05:00
mengqin	0357ed7ec4	Add support for sage attention 3 in comfyui, enable via new cli arg (#11026) ... * Add support for sage attention 3 in comfyui, enable via new cli arg --use-sage-attiention3 * Fix some bugs found in PR review. The N dimension at which Sage Attention 3 takes effect is reduced to 1024 (although the improvement is not significant at this scale). * Remove the Sage Attention3 switch, but retain the attention function registration. * Fix a ruff check issue in attention.py	2025-12-30 22:53:52 -05:00
drozbay	178bdc5e14	Add handling for vace_context in context windows (#11386) ... Co-authored-by: ozbayb <17261091+ozbayb@users.noreply.github.com>	2025-12-30 14:40:42 -08:00
comfyanonymous	0e6221cc79	Add some warnings for pin and unpin errors. (#11561)	2025-12-29 18:26:42 -05:00
rattus	9ca7e143af	mm: discard async errors from pinning failures (#10738) ... Pretty much every error cudaHostRegister can throw also queues the same error on the async GPU queue. This was fixed for repinning error case, but there is the bad mmap and just enomem cases that are harder to detect. Do some dummy GPU work to clean the error state.	2025-12-29 18:19:34 -05:00
comfyanonymous	8fd07170f1	Comment out unused norm_final in lumina/z image model. (#11545)	2025-12-28 22:07:25 -05:00
comfyanonymous	2943093a53	Enable async offload by default for AMD. (#11534)	2025-12-27 18:54:15 -05:00
comfyanonymous	1e4e342f54	Fix noise with ancestral samplers when inferencing on cpu. (#11528)	2025-12-26 22:03:01 -05:00
comfyanonymous	fb478f679a	Only apply gemma quant config to gemma model for newbie. (#11436)	2025-12-20 01:02:43 -05:00
woctordho	4c432c11ed	Implement Jina CLIP v2 and NewBie dual CLIP (#11415) ... * Implement Jina CLIP v2 * Support quantized Gemma in NewBie dual CLIP	2025-12-20 00:57:22 -05:00
comfyanonymous	31e961736a	Fix issue with batches and newbie. (#11435)	2025-12-20 00:23:51 -05:00
comfyanonymous	3ab9748903	Disable prompt weights on newbie te. (#11434)	2025-12-20 00:19:47 -05:00
woctordho	0aa7fa464e	Implement sliding attention in Gemma3 (#11409)	2025-12-20 00:16:46 -05:00
drozbay	514c24d756	Fix error from logging line (#11423) ... Co-authored-by: ozbayb <17261091+ozbayb@users.noreply.github.com>	2025-12-19 20:22:45 -08:00
comfyanonymous	809ce68749	Support nested tensor denoise masks. (#11431)	2025-12-19 19:59:25 -05:00
BradPepersAMD	cc4ddba1b6	Allow enabling use of MIOpen by setting COMFYUI_ENABLE_MIOPEN=1 as an env var (#11366)	2025-12-19 17:01:50 -05:00
comfyanonymous	28eaab608b	Diffusion model part of Qwen Image Layered. (#11408) ... Only thing missing after this is some nodes to make using it easier.	2025-12-18 20:21:14 -05:00
comfyanonymous	6a2678ac65	Trim/pad channels in VAE code. (#11406)	2025-12-18 18:22:38 -05:00
comfyanonymous	e4fb3a3572	Support loading Wan/Qwen VAEs with different in/out channels. (#11405)	2025-12-18 17:45:33 -05:00
comfyanonymous	bf7dc63bd6	skip_load_model -> force_full_load (#11390) ... This should be a bit more clear and less prone to potential breakage if the logic of the load models changes a bit.	2025-12-17 23:29:32 -05:00
Kohaku-Blueleaf	86dbb89fc9	Resolution bucketing and Trainer implementation refactoring (#11117)	2025-12-17 22:15:27 -05:00
chaObserv	5d9ad0c6bf	Fix the last step with non-zero sigma in sa_solver (#11380)	2025-12-17 13:57:40 -05:00
chaObserv	827bb1512b	Add exp_heun_2_x0 sampler series (#11360)	2025-12-16 23:35:43 -05:00
comfyanonymous	ffdd53b327	Check state dict key to auto enable the index_timestep_zero ref method. (#11362)	2025-12-16 17:03:17 -05:00
comfyanonymous	bc606d7d64	Add a way to set the default ref method in the qwen image code. (#11349)	2025-12-16 01:26:55 -05:00
comfyanonymous	683569de55	Only enable fp16 on ZImage on newer pytorch. (#11344)	2025-12-15 22:33:27 -05:00
Haoming	ea2c117bc3	[BlockInfo] Wan (#10845) ... * block info * animate * tensor * device * revert	2025-12-15 17:59:16 -08:00
Haoming	fc4af86068	[BlockInfo] Lumina (#11227) ... * block info * device * Make tensor int again --------- Co-authored-by: Jedrzej Kosinski <kosinkadink1@gmail.com>	2025-12-15 17:57:28 -08:00
comfyanonymous	70541d4e77	Support the new qwen edit 2511 reference method. (#11340) ... index_timestep_zero can be selected in the FluxKontextMultiReferenceLatentMethod now with the display name set to the more generic "Edit Model Reference Method" node.	2025-12-15 19:20:34 -05:00
drozbay	77b2f7c228	Add context windows callback for custom cond handling (#11208) ... Co-authored-by: ozbayb <17261091+ozbayb@users.noreply.github.com>	2025-12-15 16:06:32 -08:00
Dr.Lt.Data	dbd330454a	feat(preview): add per-queue live preview method override (#11261) ... - Add set_preview_method() to override live preview method per queue item - Read extra_data.preview_method from /prompt request - Support values: taesd, latent2rgb, none, auto, default - "default" or unset uses server's CLI --preview-method setting - Add 44 tests (37 unit + 7 E2E)	2025-12-15 15:57:39 -08:00
comfyanonymous	5ac3b26a7d	Update warning for old pytorch version. (#11319) ... Versions below 2.4 are no longer supported. We will not break support on purpose but will not fix it if we do.	2025-12-14 04:02:50 -05:00
chaObserv	6592bffc60	seeds_2: add phi_2 variant and sampler node (#11309) ... * Add phi_2 solver type to seeds_2 * Add sampler node of seeds_2	2025-12-14 00:03:29 -05:00
comfyanonymous	971cefe7d4	Fix pytorch warnings. (#11314)	2025-12-13 18:45:23 -05:00
comfyanonymous	da2bfb5b0a	Basic implementation of z image fun control union 2.0 (#11304) ... The inpaint part is currently missing and will be implemented later. I think they messed up this model pretty bad. They added some control_noise_refiner blocks but don't actually use them. There is a typo in their code so instead of doing control_noise_refiner -> control_layers it runs the whole control_layers twice. Unfortunately they trained with this typo so the model works but is kind of slow and would probably perform a lot better if they corrected their code and trained it again.	2025-12-13 01:39:11 -05:00
comfyanonymous	c5a47a1692	Fix bias dtype issue in mixed ops. (#11293)	2025-12-12 11:49:35 -05:00
comfyanonymous	5495589db3	Respect the dtype the op was initialized in for non quant mixed op. (#11282)	2025-12-11 23:32:27 -05:00
comfyanonymous	eeb020b9b7	Better chroma radiance and other models vram estimation. (#11278)	2025-12-11 17:33:09 -05:00
comfyanonymous	ae65433a60	This only works on radiance. (#11277)	2025-12-11 17:15:00 -05:00
comfyanonymous	fdebe18296	Fix regular chroma radiance (#11276)	2025-12-11 17:09:35 -05:00
comfyanonymous	f8321eb57b	Adjust memory usage factor. (#11257)	2025-12-11 01:30:31 -05:00
Farshore	e711aaf1a7	Lower VAE loading requirements：Create a new branch for GPU memory calculations in qwen-image vae (#11199)	2025-12-10 22:02:26 -05:00
comfyanonymous	17c92a9f28	Tweak Z Image memory estimation. (#11254)	2025-12-10 19:59:48 -05:00
comfyanonymous	791e30ff50	Fix nan issue when quantizing fp16 tensor. (#11213)	2025-12-09 17:03:21 -05:00
Jukka Seppänen	e2a800e7ef	Fix for HunyuanVideo1.5 meanflow distil (#11212)	2025-12-09 16:59:16 -05:00
rattus	9d252f3b70	ops: delete dead code (#11204) ... This became dead code in https://github.com/comfyanonymous/ComfyUI/pull/11069	2025-12-09 00:55:13 -05:00
Lodestone	b9fb542703	add chroma-radiance-x0 mode (#11197)	2025-12-08 23:33:29 -05:00
rattus	e136b6dbb0	dequantization offload accounting (fixes Flux2 OOMs - incl TEs) (#11171) ... * make setattr safe for non existent attributes Handle the case where the attribute doesnt exist by returning a static sentinel (distinct from None). If the sentinel is passed in as the set value, del the attr. * Account for dequantization and type-casts in offload costs When measuring the cost of offload, identify weights that need a type change or dequantization and add the size of the conversion result to the offload cost. This is mutually exclusive with lowvram patches which already has a large conservative estimate and wont overlap the dequant cost so\ dont double count. * Set the compute type on CLIP MPs So that the loader can know the size of weights for dequant accounting.	2025-12-08 23:21:31 -05:00
comfyanonymous	d50f342c90	Fix potential issue. (#11201)	2025-12-08 23:20:04 -05:00
comfyanonymous	3b0368aa34	Fix regression. (#11194)	2025-12-08 17:38:36 -05:00
rattus	60ee574748	retune lowVramPatch VRAM accounting (#11173) ... In the lowvram case, this now does its math in the model dtype in the post de-quantization domain. Account for that. The patching was also put back on the compute stream getting it off-peak so relax the MATH_FACTOR to only x2 so get out of the worst-case assumption of everything peaking at once.	2025-12-08 15:18:06 -05:00
dxqb	8e889c535d	Support "transformer." LoRA prefix for Z-Image (#11135)	2025-12-08 15:17:26 -05:00
comfyanonymous	56fa7dbe38	Properly load the newbie diffusion model. (#11172) ... There is still one of the text encoders missing and I didn't actually test it.	2025-12-07 07:44:55 -05:00
comfyanonymous	329480da5a	Fix qwen scaled fp8 not working with kandinsky. Make basic t2i wf work. (#11162)	2025-12-06 17:50:10 -08:00
rattus	4086acf3c2	Fix on-load VRAM OOM (#11144) ... slow down the CPU on model load to not run ahead. This fixes a VRAM on flux 2 load. I went to try and debug this with the memory trace pickles, which needs --disable-cuda-malloc which made the bug go away. So I tried this synchronize and it worked. The has some very complex interactions with the cuda malloc async and I dont have solid theory on this one yet. Still debugging but this gets us over the OOM for the moment.	2025-12-06 18:42:09 -05:00
comfyanonymous	50ca97e776	Speed up lora compute and lower memory usage by doing it in fp16. (#11161)	2025-12-06 18:36:20 -05:00
comfyanonymous	ae676ed105	Fix regression. (#11137)	2025-12-05 23:01:19 -05:00
Jukka Seppänen	fd109325db	Kandinsky5 model support (#10988) ... * Add Kandinsky5 model support lite and pro T2V tested to work * Update kandinsky5.py * Fix fp8 * Fix fp8_scaled text encoder * Add transformer_options for attention * Code cleanup, optimizations, use fp32 for all layers originally at fp32 * ImageToVideo -node * Fix I2V, add necessary latent post process nodes * Support text to image model * Support block replace patches (SLG mostly) * Support official LoRAs * Don't scale RoPE for lite model as that just doesn't work... * Update supported_models.py * Rever RoPE scaling to simpler one * Fix typo * Handle latent dim difference for image model in the VAE instead * Add node to use different prompts for clip_l and qwen25_7b * Reduce peak VRAM usage a bit * Further reduce peak VRAM consumption by chunking ffn * Update chunking * Update memory_usage_factor * Code cleanup, don't force the fp32 layers as it has minimal effect * Allow for stronger changes with first frames normalization Default values are too weak for any meaningful changes, these should probably be exposed as advanced node options when that's available. * Add image model's own chat template, remove unused image2video template * Remove hard error in ReplaceVideoLatentFrames -node * Update kandinsky5.py * Update supported_models.py * Fix typos in prompt template They were now fixed in the original repository as well * Update ReplaceVideoLatentFrames Add tooltips Make source optional Better handle negative index * Rename NormalizeVideoLatentFrames -node For bit better clarity what it does * Fix NormalizeVideoLatentStart node out on non-op	2025-12-05 22:20:22 -05:00
comfyanonymous	092ee8a500	Fix some custom nodes. (#11134)	2025-12-05 18:25:31 -05:00
Jukka Seppänen	79d17ba233	Context windows fixes and features (#10975) ... * Apply cond slice fix * Add FreeNoise * Update context_windows.py * Add option to retain condition by indexes for each window This allows for example Wan/HunyuanVideo image to video to "work" by using the initial start frame for each window, otherwise windows beyond first will be pure T2V generations. * Update context_windows.py * Allow splitting multiple conds into different windows * Add handling for audio_embed * whitespace * Allow freenoise to work on other dims, handle 4D batch timestep Refactor Freenoise function. And fix batch handling as timesteps seem to be expanded to batch size now. * Disable experimental options for now So that the Freenoise and bugfixes can be merged first --------- Co-authored-by: Jedrzej Kosinski <kosinkadink1@gmail.com> Co-authored-by: ozbayb <17261091+ozbayb@users.noreply.github.com>	2025-12-05 12:42:46 -08:00
comfyanonymous	6fd463aec9	Fix regression when text encoder loaded directly on GPU. (#11129)	2025-12-05 15:33:16 -05:00
comfyanonymous	43071e3de3	Make old scaled fp8 format use the new mixed quant ops system. (#11000)	2025-12-05 14:35:42 -05:00
Jedrzej Kosinski	0ec05b1481	Remove line made unnecessary (and wrong) after transformer_options was added to NextDiT's _forward definition (#11118)	2025-12-05 14:05:38 -05:00
rattus	9bc893c5bb	sd: bump HY1.5 VAE estimate (#11107) ... Im able to push vram above estimate on partial unload. Bump the estimate. This is experimentally determined with a 720P and 480P datapoint calibrating for 24GB VRAM total.	2025-12-04 09:50:36 -08:00
rattus	f4bdf5f830	sd: revise hy VAE VRAM (#11105) ... This was recently collapsed down to rolling VAE through temporal. Clamp The time dimension.	2025-12-04 09:50:04 -08:00
rattus	6be85c7920	mp: use look-ahead actuals for stream offload VRAM calculation (#11096) ... TIL that the WAN TE has a 2GB weight followed by 16MB as the next size down. This means that team 8GB VRAM would fully offload the TE in async offload mode as it just multiplied this giant size my the num streams. Do the more complex logic of summing up the upcoming to-load weight sizes to avoid triple counting this massive weight. partial unload does the converse of recording the NS most recent unloads as they go.	2025-12-03 23:28:44 -05:00
comfyanonymous	ea17add3c6	Fix case where text encoders where running on the CPU instead of GPU. (#11095)	2025-12-03 23:15:15 -05:00
rattus	519c941165	Prs/lora reservations (reduce massive Lora reservations especially on Flux2) (#11069) ... * mp: only count the offload cost of math once This was previously bundling the combined weight storage and computation cost * ops: put all post async transfer compute on the main stream Some models have massive weights that need either complex dequantization or lora patching. Don't do these patchings on the offload stream, instead do them on the main stream to syncrhonize the potentially large vram spikes for these compute processes. This avoids having to assume a worst case scenario of multiple offload streams all spiking VRAM is parallel with whatever the main stream is doing.	2025-12-03 02:28:45 -05:00
rattus	73f5649196	Implement temporal rolling VAE (Major VRAM reductions in Hunyuan and Kandinsky) (#10995) ... * hunyuan upsampler: rework imports Remove the transitive import of VideoConv3d and Resnet and takes these from actual implementation source. * model: remove unused give_pre_end According to git grep, this is not used now, and was not used in the initial commit that introduced it (see below). This semantic is difficult to implement temporal roll VAE for (and would defeat the purpose). Rather than implement the complex if, just delete the unused feature. (venv) rattus@rattus-box2:~/ComfyUI$ git log --oneline 220afe33 (HEAD) Initial commit. (venv) rattus@rattus-box2:~/ComfyUI$ git grep give_pre comfy/ldm/modules/diffusionmodules/model.py: resolution, z_channels, give_pre_end=False, tanh_out=False, use_linear_attn=False, comfy/ldm/modules/diffusionmodules/model.py: self.give_pre_end = give_pre_end comfy/ldm/modules/diffusionmodules/model.py: if self.give_pre_end: (venv) rattus@rattus-box2:~/ComfyUI$ git co origin/master Previous HEAD position was 220afe33 Initial commit. HEAD is now at 9d8a8179 Enable async offloading by default on Nvidia. (#10953) (venv) rattus@rattus-box2:~/ComfyUI$ git grep give_pre comfy/ldm/modules/diffusionmodules/model.py: resolution, z_channels, give_pre_end=False, tanh_out=False, use_linear_attn=False, comfy/ldm/modules/diffusionmodules/model.py: self.give_pre_end = give_pre_end comfy/ldm/modules/diffusionmodules/model.py: if self.give_pre_end: * move refiner VAE temporal roller to core Move the carrying conv op to the common VAE code and give it a better name. Roll the carry implementation logic for Resnet into the base class and scrap the Hunyuan specific subclass. * model: Add temporal roll to main VAE decoder If there are no attention layers, its a standard resnet and VideoConv3d is asked for, substitute in the temporal rolloing VAE algorithm. This reduces VAE usage by the temporal dimension (can be huge VRAM savings). * model: Add temporal roll to main VAE encoder If there are no attention layers, its a standard resnet and VideoConv3d is asked for, substitute in the temporal rolling VAE algorithm. This reduces VAE usage by the temporal dimension (can be huge VRAM savings).	2025-12-02 22:49:29 -05:00
comfyanonymous	b94d394a64	Support Z Image alibaba pai fun controlnets. (#11062) ... These are not actual controlnets so put it in the models/model_patches folder and use the ModelPatchLoader + QwenImageDiffsynthControlnet node to use it.	2025-12-02 21:38:31 -05:00
rattus	277237ccc1	attention: use flag based OOM fallback (#11038) ... Exception ref all local variables for the lifetime of exception context. Just set a flag and then if to dump the exception before falling back.	2025-12-02 17:24:19 -05:00
comfyanonymous	daaceac769	Hack to make zimage work in fp16. (#11057)	2025-12-02 17:11:58 -05:00
Dr.Lt.Data	b4a20acc54	feat: Support ComfyUI-Manager for pip version (#7555)	2025-12-01 22:32:52 -05:00
comfyanonymous	878db3a727	Implement the Ovis image model. (#11030)	2025-12-01 20:56:17 -05:00
comfyanonymous	2640acb31c	Update qwen tokenizer to add qwen 3 tokens. (#11029) ... Doesn't actually change anything for current workflows because none of the current models have a template with the think tokens.	2025-12-01 17:13:48 -05:00
comfyanonymous	0a6746898d	Make the ScaleRope node work on Z Image and Lumina. (#10994)	2025-11-29 18:00:55 -05:00
comfyanonymous	5151cff293	Add some missing z image lora layers. (#10980)	2025-11-28 23:55:00 -05:00
comfyanonymous	52a32e2b32	Support some z image lora formats. (#10978)	2025-11-28 21:12:42 -05:00
Jukka Seppänen	b907085709	Support video tiny VAEs (#10884) ... * Support video tiny VAEs * lighttaew scaling fix * Also support video taes in previews Only first frame for now as live preview playback is currently only available through VHS custom nodes. * Support Wan 2.1 lightVAE * Relocate elif block and set Wan VAE dim directly without using pruning rate for lightvae	2025-11-28 19:40:19 -05:00
rattus	0ff0457892	mm: wrap the raw stream in context manager (#10958) ... The documentation of torch.foo.Stream being usable with with: suggests it starts at version 2.7. Use the old API for backwards compatibility.	2025-11-28 16:38:12 -05:00
Urle Sistiana	6484ac89dc	fix QuantizedTensor.is_contiguous (#10956) (#10959)	2025-11-28 16:33:07 -05:00
comfyanonymous	f55c98a89f	Disable offload stream when torch compile. (#10961)	2025-11-28 16:16:46 -05:00
comfyanonymous	9d8a817985	Enable async offloading by default on Nvidia. (#10953) ... Add --disable-async-offload to disable it. If this causes OOMs that go away when you --disable-async-offload please report it.	2025-11-27 17:46:12 -05:00
rattus	3f382a4f98	quant ops: Dequantize weight in-place (#10935) ... In flux2 these weights are huge (200MB). As plain_tensor is a throw-away deep copy, do this multiplication in-place to save VRAM.	2025-11-27 08:06:30 -08:00
rattus	f17251bec6	Account for the VRAM cost of weight offloading (#10733) ... * mm: default to 0 for NUM_STREAMS Dont count the compute stream as an offload stream. This makes async offload accounting easier. * mm: remove 128MB minimum This is from a previous offloading system requirement. Remove it to make behaviour of the loader and partial unloader consistent. * mp: order the module list by offload expense Calculate an approximate offloading temporary VRAM cost to offload a weight and primary order the module load list by that. In the simple case this is just the same as the module weight, but with Loras, a weight with a lora consumes considerably more VRAM to do the Lora application on-the-fly. This will slightly prioritize lora weights, but is really for proper VRAM offload accounting. * mp: Account for the VRAM cost of weight offloading when checking the VRAM headroom, assume that the weight needs to be offloaded, and only load if it has space for both the load and offload * the number of streams. As the weights are ordered from largest to smallest by offload cost this is guaranteed to fit in VRAM (tm), as all weights that follow will be smaller. Make the partial unload aware of this system as well by saving the budget for offload VRAM to the model state and accounting accordingly. Its possible that partial unload increases the size of the largest offloaded weights, and thus needs to unload a little bit more than asked to accomodate the bigger temp buffers. Honor the existing codes floor on model weight loading of 128MB by having the patcher honor this separately withough regard to offloading. Otherwise when MM specifies its 128MB minimum, MP will see the biggest weights, and budget that 128MB to only offload buffer and load nothing which isnt the intent of these minimums. The same clamp applies in case of partial offload of the currently loading model.	2025-11-27 01:03:03 -05:00
Haoming	c38e7d6599	block info (#10841)	2025-11-26 20:28:44 -08:00
comfyanonymous	eaf68c9b5b	Make lora training work on Z Image and remove some redundant nodes. (#10927)	2025-11-26 19:25:32 -05:00
comfyanonymous	f16219e3aa	Add cheap latent preview for flux 2. (#10907) ... Thank you to the person who calculated them. You saved me a percent of my time.	2025-11-26 04:00:43 -05:00
comfyanonymous	58b8574661	Fix Flux2 reference image mem estimation. (#10905)	2025-11-26 02:36:19 -05:00
comfyanonymous	bdb10a583f	Fix loras not working on mixed fp8. (#10899)	2025-11-26 00:07:58 -05:00
comfyanonymous	0e24dbb19f	Adjustments to Z Image. (#10893)	2025-11-25 19:02:51 -05:00
comfyanonymous	e9aae31fa2	Z Image model. (#10892)	2025-11-25 18:41:45 -05:00
comfyanonymous	d196a905bb	Lower vram usage for flux 2 text encoder. (#10887)	2025-11-25 14:58:39 -05:00
comfyanonymous	dff996ca39	Fix crash. (#10885)	2025-11-25 14:30:24 -05:00
comfyanonymous	6b573ae0cb	Flux 2 (#10879)	2025-11-25 10:50:19 -05:00
comfyanonymous	015a0599d0	I found a case where this is needed (#10875)	2025-11-25 03:23:19 -05:00
comfyanonymous	acfaa5c4a1	Don't try fp8 matrix mult in quantized ops if not supported by hardware. (#10874)	2025-11-25 02:55:49 -05:00
comfyanonymous	b6805429b9	Allow pinning quantized tensors. (#10873)	2025-11-25 02:48:20 -05:00
comfyanonymous	25022e0b09	Cleanup and fix issues with text encoder quants. (#10872)	2025-11-25 01:48:53 -05:00
Haoming	b2ef58e2b1	block info (#10844)	2025-11-24 10:40:09 -08:00
Haoming	6a6d456c88	block info (#10842)	2025-11-24 10:38:38 -08:00
Haoming	3d1fdaf9f4	block info (#10843)	2025-11-24 10:30:40 -08:00
comfyanonymous	cbd68e3d58	Add better error message for common error. (#10846)	2025-11-23 04:55:22 -05:00
comfyanonymous	532938b16b	--disable-api-nodes now sets CSP header to force frontend offline. (#10829)	2025-11-21 17:51:55 -05:00
comfyanonymous	943b3b615d	HunyuanVideo 1.5 (#10819) ... * init * update * Update model.py * Update model.py * remove print * Fix text encoding * Prevent empty negative prompt Really doesn't work otherwise * fp16 works * I2V * Update model_base.py * Update nodes_hunyuan.py * Better latent rgb factors * Use the correct sigclip output... * Support HunyuanVideo1.5 SR model * whitespaces... * Proper latent channel count * SR model fixes This also still needs timesteps scheduling based on the noise scale, can be used with two samplers too already * vae_refiner: roll the convolution through temporal Work in progress. Roll the convolution through time using 2-latent-frame chunks and a FIFO queue for the convolution seams. * Support HunyuanVideo15 latent resampler * fix * Some cleanup Co-Authored-By: comfyanonymous <121283862+comfyanonymous@users.noreply.github.com> * Proper hyvid15 I2V channels Co-Authored-By: comfyanonymous <121283862+comfyanonymous@users.noreply.github.com> * Fix TokenRefiner for fp16 Otherwise x.sum has infs, just in case only casting if input is fp16, I don't know if necessary. * Bugfix for the HunyuanVideo15 SR model * vae_refiner: roll the convolution through temporal II Roll the convolution through time using 2-latent-frame chunks and a FIFO queue for the convolution seams. Added support for encoder, lowered to 1 latent frame to save more VRAM, made work for Hunyuan Image 3.0 (as code shared). Fixed names, cleaned up code. * Allow any number of input frames in VAE. * Better VAE encode mem estimation. * Lowvram fix. * Fix hunyuan image 2.1 refiner. * Fix mistake. * Name changes. * Rename. * Whitespace. * Fix. * Fix. --------- Co-authored-by: kijai <40791699+kijai@users.noreply.github.com> Co-authored-by: Rattus <rattus128@gmail.com>	2025-11-20 22:44:43 -05:00
comfyanonymous	cb96d4d18c	Disable workaround on newer cudnn. (#10807)	2025-11-19 23:56:23 -05:00
comfyanonymous	17027f2a6a	Add a way to disable the final norm in the llama based TE models. (#10794)	2025-11-18 22:36:03 -05:00
comfyanonymous	d526974576	Fix hunyuan 3d 2.0 (#10792)	2025-11-18 16:46:19 -05:00
comfyanonymous	bd01d9f7fd	Add left padding support to tokenizers. (#10753)	2025-11-15 06:54:40 -05:00
comfyanonymous	443056c401	Fix custom nodes import error. (#10747) ... This should fix the import errors but will break if the custom nodes actually try to use the class.	2025-11-14 03:26:05 -05:00
comfyanonymous	f60923590c	Use same code for chroma and flux blocks so that optimizations are shared. (#10746)	2025-11-14 01:28:05 -05:00
rattus	94c298f962	flux: reduce VRAM usage (#10737) ... Cleanup a bunch of stack tensors on Flux. This take me from B=19 to B=22 for 1600x1600 on RTX5090.	2025-11-13 16:02:03 -08:00
contentis	3b3ef9a77a	Quantized Ops fixes (#10715) ... * offload support, bug fixes, remove mixins * add readme	2025-11-12 18:26:52 -05:00
rattus	1c7eaeca10	qwen: reduce VRAM usage (#10725) ... Clean up a bunch of stacked and no-longer-needed tensors on the QWEN VRAM peak (currently FFN). With this I go from OOMing at B=37x1328x1328 to being able to succesfully run B=47 (RTX5090).	2025-11-12 16:20:53 -05:00
rattus	18e7d6dba5	mm/mp: always unload re-used but modified models (#10724) ... The partial unloader path in model re-use flow skips straight to the actual unload without any check of the patching UUID. This means that if you do an upscale flow with a model patch on an existing model, it will not apply your patchings. Fix by delaying the partial_unload until after the uuid checks. This is done by making partial_unload a model of partial_load where extra_mem is -ve.	2025-11-12 16:19:53 -05:00
comfyanonymous	1199411747	Don't pin tensor if not a torch.nn.parameter.Parameter (#10718)	2025-11-11 19:33:30 -05:00
rattus	c350009236	ops: Put weight cast on the offload stream (#10697) ... This needs to be on the offload stream. This reproduced a black screen with low resolution images on a slow bus when using FP8.	2025-11-09 22:52:11 -05:00
comfyanonymous	dea899f221	Unload weights if vram usage goes up between runs. (#10690)	2025-11-09 18:51:33 -05:00
comfyanonymous	e632e5de28	Add logging for model unloading. (#10692)	2025-11-09 18:06:39 -05:00
comfyanonymous	2abd2b5c20	Make ScaleROPE node work on Flux. (#10686)	2025-11-08 15:52:02 -05:00
comfyanonymous	a1a70362ca	Only unpin tensor if it was pinned by ComfyUI (#10677)	2025-11-07 11:15:05 -05:00
rattus	cf97b033ee	mm: guard against double pin and unpin explicitly (#10672) ... As commented, if you let cuda be the one to detect double pin/unpinning it actually creates an asyc GPU error.	2025-11-06 21:20:48 -05:00
comfyanonymous	09dc24c8a9	Pinned mem also seems to work on AMD. (#10658)	2025-11-05 19:11:15 -05:00
comfyanonymous	1d69245981	Enable pinned memory by default on Nvidia. (#10656) ... Removed the --fast pinned_memory flag. You can use --disable-pinned-memory to disable it. Please report if it causes any issues.	2025-11-05 18:08:13 -05:00
comfyanonymous	97f198e421	Fix qwen controlnet regression. (#10657)	2025-11-05 18:07:35 -05:00
comfyanonymous	c4a6b389de	Lower ltxv mem usage to what it was before previous pr. (#10643) ... Bring back qwen behavior to what it was before previous pr.	2025-11-04 22:47:35 -05:00
contentis	4cd881866b	Use single apply_rope function across models (#10547)	2025-11-04 20:10:11 -05:00
comfyanonymous	7f3e4d486c	Limit amount of pinned memory on windows to prevent issues. (#10638)	2025-11-04 17:37:50 -05:00
comfyanonymous	af4b7b5edb	More fp8 torch.compile regressions fixed. (#10625)	2025-11-03 22:14:20 -05:00
comfyanonymous	0f4ef3afa0	This seems to slow things down slightly on Linux. (#10624)	2025-11-03 21:47:14 -05:00
comfyanonymous	6b88478f9f	Bring back fp8 torch compile performance to what it should be. (#10622)	2025-11-03 19:22:10 -05:00
comfyanonymous	e199c8cc67	Fixes (#10621)	2025-11-03 17:58:24 -05:00
comfyanonymous	0652cb8e2d	Speed up torch.compile (#10620)	2025-11-03 17:37:12 -05:00
comfyanonymous	958a17199a	People should update their pytorch versions. (#10618)	2025-11-03 17:08:30 -05:00
comfyanonymous	97ff9fae7e	Clarify help text for --fast argument (#10609) ... Updated help text for the --fast argument to clarify potential risks.	2025-11-02 13:14:04 -05:00
rattus	135fa49ec2	Small speed improvements to --async-offload (#10593) ... * ops: dont take an offload stream if you dont need one * ops: prioritize mem transfer The async offload streams reason for existence is to transfer from RAM to GPU. The post processing compute steps are a bonus on the side stream, but if the compute stream is running a long kernel, it can stall the side stream, as it wait to type-cast the bias before transferring the weight. So do a pure xfer of the weight straight up, then do everything bias, then go back to fix the weight type and do weight patches.	2025-11-01 18:48:53 -04:00
comfyanonymous	44869ff786	Fix issue with pinned memory. (#10597)	2025-11-01 17:25:59 -04:00
comfyanonymous	c58c13b2ba	Fix torch compile regression on fp8 ops. (#10580)	2025-11-01 00:25:17 -04:00
comfyanonymous	7f374e42c8	ScaleROPE now works on Lumina models. (#10578)	2025-10-31 15:41:40 -04:00
comfyanonymous	27d1bd8829	Fix rope scaling. (#10560)	2025-10-30 22:51:58 -04:00
comfyanonymous	614cf9805e	Add a ScaleROPE node. Currently only works on WAN models. (#10559)	2025-10-30 22:11:38 -04:00
rattus	513b0c46fb	Add RAM Pressure cache mode (#10454) ... * execution: Roll the UI cache into the outputs Currently the UI cache is parallel to the output cache with expectations of being a content superset of the output cache. At the same time the UI and output cache are maintained completely seperately, making it awkward to free the output cache content without changing the behaviour of the UI cache. There are two actual users (getters) of the UI cache. The first is the case of a direct content hit on the output cache when executing a node. This case is very naturally handled by merging the UI and outputs cache. The second case is the history JSON generation at the end of the prompt. This currently works by asking the cache for all_node_ids and then pulling the cache contents for those nodes. all_node_ids is the nodes of the dynamic prompt. So fold the UI cache into the output cache. The current UI cache setter now writes to a prompt-scope dict. When the output cache is set, just get this value from the dict and tuple up with the outputs. When generating the history, simply iterate prompt-scope dict. This prepares support for more complex caching strategies (like RAM pressure caching) where less than 1 workflow will be cached and it will be desirable to keep the UI cache and output cache in sync. * sd: Implement RAM getter for VAE * model_patcher: Implement RAM getter for ModelPatcher * sd: Implement RAM getter for CLIP * Implement RAM Pressure cache Implement a cache sensitive to RAM pressure. When RAM headroom drops down below a certain threshold, evict RAM-expensive nodes from the cache. Models and tensors are measured directly for RAM usage. An OOM score is then computed based on the RAM usage of the node. Note the due to indirection through shared objects (like a model patcher), multiple nodes can account the same RAM as their individual usage. The intent is this will free chains of nodes particularly model loaders and associate loras as they all score similar and are sorted in close to each other. Has a bias towards unloading model nodes mid flow while being able to keep results like text encodings and VAE. * execution: Convert the cache entry to NamedTuple As commented in review. Convert this to a named tuple and abstract away the tuple type completely from graph.py.	2025-10-30 17:39:02 -04:00
Jedrzej Kosinski	998bf60beb	Add units/info for the numbers displayed on 'load completely' and 'load partially' log messages (#10538)	2025-10-29 19:37:06 -04:00
comfyanonymous	906c089957	Fix small performance regression with fp8 fast and scaled fp8. (#10537)	2025-10-29 19:29:01 -04:00
comfyanonymous	25de7b1bfa	Try to fix slow load issue on low ram hardware with pinned mem. (#10536)	2025-10-29 17:20:27 -04:00
rattus	ab7ab5be23	Fix Race condition in --async-offload that can cause corruption (#10501) ... * mm: factor out the current stream getter Make this a reusable function. * ops: sync the offload stream with the consumption of w&b This sync is nessacary as pytorch will queue cuda async frees on the same stream as created to tensor. In the case of async offload, this will be on the offload stream. Weights and biases can go out of scope in python which then triggers the pytorch garbage collector to queue the free operation on the offload stream possible before the compute stream has used the weight. This causes a use after free on weight data leading to total corruption of some workflows. So sync the offload stream with the compute stream after the weight has been used so the free has to wait for the weight to be used. The cast_bias_weight is extended in a backwards compatible way with the new behaviour opt-in on a defaulted parameter. This handles custom node packs calling cast_bias_weight and defeatures async-offload for them (as they do not handle the race). The pattern is now: cast_bias_weight(... , offloadable=True) #This might be offloaded thing(weight, bias, ...) uncast_bias_weight(...) * controlnet: adopt new cast_bias_weight synchronization scheme This is nessacary for safe async weight offloading. * mm: sync the last stream in the queue, not the next Currently this peeks ahead to sync the next stream in the queue of streams with the compute stream. This doesnt allow a lot of parallelization, as then end result is you can only get one weight load ahead regardless of how many streams you have. Rotate the loop logic here to synchronize the end of the queue before returning the next stream. This allows weights to be loaded ahead of the compute streams position.	2025-10-29 17:17:46 -04:00