mirror of
https://github.com/comfyanonymous/ComfyUI.git
synced 2026-07-08 23:47:24 +08:00
c00d1d7a74
On Apple Silicon vram_state is VRAMState.SHARED (unified memory), but text_encoder_device() only returned the GPU for HIGH_VRAM/NORMAL_VRAM, so text encoders ran on the CPU. For LM-style encoders like ACE-Step 1.5 the text encode stage dominates generation time on Mac. This re-lands #12809, which was reverted in #13070 because it broke quantized text encoders on Mac: the MPS backend cannot cast float8 dtypes (pytorch/pytorch#132624), and the existing supports_cast() fallback in CLIP.__init__ only inspects declared model dtypes, which never reflect fp8 weights behind comfy_quant quantization metadata (QuantizedTensor reports the compute dtype, not the fp8 storage dtype). To keep quantized text encoders working, CLIP.__init__ now decides the device before any weights are allocated: it checks supports_cast() on the resolved dtype, and when the load device cannot cast fp8 it scans the incoming state dict for fp8 tensors or comfy_quant markers and keeps those text encoders on the offload device. Devices that can cast fp8 (cuda etc.) skip the scan entirely. The pre-existing shift-back path now also updates the current device so the load log stays accurate. Adds MPS unit tests: fp16 placement on the GPU, fp8 fallback via declared dtype, secondary dtype (dtype_llama style), state-dict fp8 weights, and comfy_quant markers, plus a canary that fails when a torch release adds fp8 casts on MPS so the fallback can be relaxed. |
||
|---|---|---|
| .. | ||
| folder_path_test.py | ||
| model_detection_test.py | ||
| text_encoder_mps_test.py | ||