Added support for --lowvram argument for 'mps' devices. If passed weights would not be loaded to VRAM completely. With modified GGUF loader it allows to keep memory-mapped weights on disk.

This commit is contained in:
fromfirstbyte 2026-06-25 06:06:26 +03:00
parent b22d0fb9c0
commit abbe30c49e

View File

@ -938,7 +938,7 @@ def load_models_gpu(models, memory_required=0, force_patch_weights=False, minimu
if lowvram_model_memory == 0:
lowvram_model_memory = 0.1
if vram_set_state == VRAMState.NO_VRAM:
if vram_set_state == VRAMState.NO_VRAM or (set_vram_to == VRAMState.LOW_VRAM and vram_state == VRAMState.SHARED and not force_full_load):
lowvram_model_memory = 0.1
loaded_model.model_load(lowvram_model_memory, force_patch_weights=force_patch_weights)