mirror of
https://github.com/comfyanonymous/ComfyUI.git
synced 2026-07-08 07:31:20 +08:00
c00d1d7a74
On Apple Silicon vram_state is VRAMState.SHARED (unified memory), but text_encoder_device() only returned the GPU for HIGH_VRAM/NORMAL_VRAM, so text encoders ran on the CPU. For LM-style encoders like ACE-Step 1.5 the text encode stage dominates generation time on Mac. This re-lands #12809, which was reverted in #13070 because it broke quantized text encoders on Mac: the MPS backend cannot cast float8 dtypes (pytorch/pytorch#132624), and the existing supports_cast() fallback in CLIP.__init__ only inspects declared model dtypes, which never reflect fp8 weights behind comfy_quant quantization metadata (QuantizedTensor reports the compute dtype, not the fp8 storage dtype). To keep quantized text encoders working, CLIP.__init__ now decides the device before any weights are allocated: it checks supports_cast() on the resolved dtype, and when the load device cannot cast fp8 it scans the incoming state dict for fp8 tensors or comfy_quant markers and keeps those text encoders on the offload device. Devices that can cast fp8 (cuda etc.) skip the scan entirely. The pre-existing shift-back path now also updates the current device so the load log stays accurate. Adds MPS unit tests: fp16 placement on the GPU, fp8 fallback via declared dtype, secondary dtype (dtype_llama style), state-dict fp8 weights, and comfy_quant markers, plus a canary that fails when a torch release adds fp8 casts on MPS so the fallback can be relaxed. |
||
|---|---|---|
| .. | ||
| audio_encoders | ||
| background_removal | ||
| cldm | ||
| comfy_types | ||
| extra_samplers | ||
| image_encoders | ||
| k_diffusion | ||
| ldm | ||
| sd1_tokenizer | ||
| t2i_adapter | ||
| taesd | ||
| text_encoders | ||
| weight_adapter | ||
| bg_removal_model.py | ||
| cli_args.py | ||
| clip_config_bigg.json | ||
| clip_model.py | ||
| clip_vision_config_g.json | ||
| clip_vision_config_h.json | ||
| clip_vision_config_vitl_336_llava.json | ||
| clip_vision_config_vitl_336.json | ||
| clip_vision_config_vitl.json | ||
| clip_vision_siglip2_base_naflex.json | ||
| clip_vision_siglip_384.json | ||
| clip_vision_siglip_512.json | ||
| clip_vision.py | ||
| conds.py | ||
| context_windows.py | ||
| controlnet.py | ||
| deploy_environment.py | ||
| diffusers_convert.py | ||
| diffusers_load.py | ||
| float.py | ||
| gligen.py | ||
| hooks.py | ||
| latent_formats.py | ||
| lora_convert.py | ||
| lora.py | ||
| memory_management.py | ||
| model_base.py | ||
| model_detection.py | ||
| model_management.py | ||
| model_patcher.py | ||
| model_prefetch.py | ||
| model_sampling.py | ||
| multigpu.py | ||
| nested_tensor.py | ||
| ops.py | ||
| options.py | ||
| patcher_extension.py | ||
| pinned_memory.py | ||
| pixel_space_convert.py | ||
| quant_ops.py | ||
| rmsnorm.py | ||
| sample.py | ||
| sampler_helpers.py | ||
| samplers.py | ||
| sd1_clip_config.json | ||
| sd1_clip.py | ||
| sd.py | ||
| sdxl_clip.py | ||
| supported_models_base.py | ||
| supported_models.py | ||
| utils.py | ||