wangbo/ComfyUI
1
0
Fork 0
mirror of https://github.com/comfyanonymous/ComfyUI.git synced 2026-07-18 20:38:15 +08:00
Code Issues Actions 27 Packages Projects Releases Wiki Activity

force LowVramPatch for quantized patched weights

Browse Source 
hen using an FP8 / quantized checkpoint in FP8 mixed mode, applying a LoRA can cause an avoidable CUDA OOM during sampling, even though the same checkpoint works fine without LoRA.

The OOM appears to happen because LoRA weights are eagerly merged into the quantized model weight, then the patched float weight is immediately requantized back to FP8 on the GPU. This creates a large temporary allocation spike.

A practical fix is to force the existing LowVramPatch path for quantized weights that have LoRA patches, instead of eagerly calling patch_weight_to_device() and requantizing the patched weight.
This commit is contained in:
Neuroger 2026-07-09 20:03:45 +02:00
parent 04a30fb375
commit 59ac04ce2d
1 changed files with 18 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

20
comfy/model_patcher.py
View File

@ -189,6 +189,15 @@ def get_key_weight(model, key):
return weight, set_func, convert_func
def is_quantized_lora_target(model, key):
try:
weight, set_func, convert_func = get_key_weight(model, key)
except Exception:
return False
return isinstance(weight, QuantizedTensor)
def key_param_name_to_key(key, param):
if len(key) == 0:
return param
@ -950,8 +959,15 @@ class ModelPatcher:
weight_key = "{}.weight".format(n)
bias_key = "{}.bias".format(n)
if not full_load and hasattr(m, "comfy_cast_weights"):
if not lowvram_fits:
force_lazy_lora_patch = False
if not force_patch_weights:
if weight_key in self.patches and is_quantized_lora_target(self.model, weight_key):
force_lazy_lora_patch = True
if bias_key in self.patches and is_quantized_lora_target(self.model, bias_key):
force_lazy_lora_patch = True
if hasattr(m, "comfy_cast_weights"):
if force_lazy_lora_patch or (not full_load and not lowvram_fits):
offload_buffer = potential_offload
lowvram_weight = True
lowvram_counter += 1