Merge fa71050a07 into de9ada6a41

* mp: fix full dynamic unloading

This was not unloading dynamic models when requesting a full unload via
the unpatch() code path.

This was ok, i your workflow was all dynamic models but fails with big
VRAM leaks if you need to fully unload something for a regular ModelPatcher

It also fices the "unload models" button.

* mm: load models outside of Aimdo Mempool

In dynamic_vram mode, escape the Aimdo mempool and load into the regular
mempool. Use a dummy thread to do it.

.dockerignore

@@ -0,0 +1,31 @@
+# This file should remain in sync with .gitignore. If you need to make changes,
+# please add a comment explaining why. For items that must be removed, comment
+# them out instead of deleting them.
+__pycache__/
+*.py[cod]
+/output/
+/input/
+# This file prevents the image from building and would be overwritten by the
+# /data volume in any case.
+#!/input/example.png
+/models/
+/temp/
+/custom_nodes/
+!custom_nodes/example_node.py.example
+extra_model_paths.yaml
+/.vs
+.vscode/
+.idea/
+venv/
+.venv/
+/web/extensions/*
+!/web/extensions/logging.js.example
+!/web/extensions/core/
+/tests-ui/data/object_info.json
+/user/
+*.log
+web_custom_versions/
+.DS_Store
+openapi.yaml
+filtered-openapi.yaml
+uv.lock

.gitattributes

@@ -1,3 +1,6 @@
 /web/assets/** linguist-generated
 /web/** linguist-vendored
 comfy_api_nodes/apis/__init__.py linguist-generated
+# Force LF eol for Docker entrypoint (fix "exec: no such file or directory"
+# error with CRLF checkouts)
+entrypoint.sh text eol=lf

.gitignore

@@ -1,3 +1,4 @@
+# If you modify this file, remember to update .dockerignore as well.
 __pycache__/
 *.py[cod]
 /output/

Dockerfile

@@ -0,0 +1,85 @@
+# Docker buildfile for the ComfyUI image, with support for hardware
+# acceleration, file ownership synchronization, custom nodes, and custom node
+# managers.
+
+# While Python 3.13 is well supported by ComfyUI, some older custom node packs
+# may not work correctly with this version, which is why we're staying on Python
+# 3.12 for now.
+#
+# Users are free to try different base Python image tags (e.g., 3.13, alpine,
+# *-slim), but for maintainability, only one base version is officially
+# supported at a time.
+FROM python:3.12.12-trixie
+
+# Install cmake, which is an indirect installation dependencies
+RUN apt-get update && apt-get install -y --no-install-recommends cmake
+
+# Create a regular user whose UID and GID will match the host user's at runtime.
+# Also create a home directory for this user (-m), as some common Python tools
+# (such as uv) interact with the user’s home directory.
+RUN useradd -m comfyui
+
+# Install ComfyUI under /comfyui and set folder ownership to the comfyui user.
+# With the legacy Docker builder (DOCKER_BUILDKIT=0), WORKDIR always creates missing
+# directories as root (even if a different USER is active). To ensure the comfyui user
+# can write inside, ownership must be fixed manually.
+WORKDIR /comfyui
+RUN chown comfyui:comfyui .
+
+# Install ComfyUI as ComfyUI
+USER comfyui
+
+# Set up a Python virtual environment and configure it as the default Python.
+#
+# Reasons for using a virtual environment:
+# - Some custom nodes use third-party tools like uv, which do not support
+#   user-level installations.
+# - Custom node managers may install or update dependencies as the regular user,
+#   so a global installation is not an option.
+# This leaves virtual environments as the only viable choice.
+RUN python -m venv .venv
+ENV PATH="/comfyui/.venv/bin:$PATH"
+
+# Install ComfyUI's Python dependencies. Although dependency keeping is also
+# performed at startup, building ComfyUI's base dependencies into the image
+# significantly speeds up each containers' first run.
+#
+# Since this step takes a long time to complete, it's performed early to take
+# advantage of Docker's build cache, thereby accelerating subsequent builds.
+COPY requirements.txt manager_requirements.txt ./
+RUN pip install --no-cache-dir --disable-pip-version-check \
+    -r requirements.txt
+
+# Install ComfyUI
+COPY . .
+
+# Purely declarative: inform Docker and image users that this image is designed
+# to listen on port 8188 for the web GUI.
+EXPOSE 8188
+
+# Declare persistent volumes. We assign one volume per data directory to match
+# ComfyUI’s natural file layout and to let users selectively choose which
+# directories they want to mount.
+VOLUME /comfyui/.venv
+VOLUME /comfyui/custom_nodes
+VOLUME /comfyui/input
+VOLUME /comfyui/models
+VOLUME /comfyui/output
+VOLUME /comfyui/temp
+VOLUME /comfyui/user
+VOLUME /home/comfyui
+
+# Switch back to root to run the entrypoint and to install additional system
+# dependencies
+USER root
+
+# Configure entrypoint
+RUN chmod +x entrypoint.sh
+ENTRYPOINT [ "./entrypoint.sh" ]
+CMD [ "python", "./main.py" ]
+
+# Install additional system dependencies
+ARG APT_EXTRA_PACKAGES
+RUN apt-get install -y --no-install-recommends $APT_EXTRA_PACKAGES \
+	&& apt-get clean                                               \
+	&& rm -rf /var/lib/apt/lists/*

README.md

@@ -46,6 +46,12 @@ ComfyUI lets you design and execute advanced stable diffusion pipelines using a
 - Get the latest commits and completely portable.
 - Available on Windows.
 
+#### [Docker Install](#running-with-docker)
+- Run ComfyUI inside an isolated Docker container
+- Most secure way to run ComfyUI and custom node packs
+- Requires Docker and Docker Compose
+- Supports NVIDIA GPUs (Not tested on other hardware.)
+
 #### [Manual Install](#manual-install-windows-linux)
 Supports all operating systems and GPU types (NVIDIA, AMD, Intel, Apple Silicon, Ascend).
 
@@ -350,6 +356,28 @@ For models compatible with Iluvatar Extension for PyTorch. Here's a step-by-step
 | `--enable-manager-legacy-ui` | Use the legacy manager UI instead of the new UI (requires `--enable-manager`) |
 | `--disable-manager-ui` | Disable the manager UI and endpoints while keeping background features like security checks and scheduled installation completion (requires `--enable-manager`) |
 
+## Running with Docker
+
+Start by installing Docker, Docker Compose, and the NVIDIA Container Toolkit on
+your host. Next, edit `compose.yaml` and update the `UID` and `GID` variables to
+match your user. Additional fields are documented in the file for further
+customization.
+
+Once ready, build and run the image locally:
+
+```shell
+# (Re)build the Docker image. Run this before the first start, after updating
+# ComfyUI, or after changing any build arguments in `compose.yaml`.
+docker compose build
+# Start ComfyUI. This reuses the most recently built image.
+docker compose up
+```
+
+To stop and remove the container along with its volumes, run:
+
+```shell
+docker compose down -v
+```
 
 # Running
 

comfy/model_management.py

@@ -19,7 +19,8 @@
 import psutil
 import logging
 from enum import Enum
-from comfy.cli_args import args, PerformanceFeature
+from comfy.cli_args import args, PerformanceFeature, enables_dynamic_vram
+import threading
 import torch
 import sys
 import platform
@@ -650,7 +651,7 @@ def free_memory(memory_required, device, keep_loaded=[], for_dynamic=False, ram_
                 soft_empty_cache()
     return unloaded_models
 
-def load_models_gpu(models, memory_required=0, force_patch_weights=False, minimum_memory_required=None, force_full_load=False):
+def load_models_gpu_orig(models, memory_required=0, force_patch_weights=False, minimum_memory_required=None, force_full_load=False):
     cleanup_models_gc()
     global vram_state
 
@@ -746,8 +747,25 @@ def load_models_gpu(models, memory_required=0, force_patch_weights=False, minimu
         current_loaded_models.insert(0, loaded_model)
     return
 
-def load_model_gpu(model):
-    return load_models_gpu([model])
+def load_models_gpu_thread(models, memory_required, force_patch_weights, minimum_memory_required, force_full_load):
+    with torch.inference_mode():
+        load_models_gpu_orig(models, memory_required, force_patch_weights, minimum_memory_required, force_full_load)
+        soft_empty_cache()
+
+def load_models_gpu(models, memory_required=0, force_patch_weights=False, minimum_memory_required=None, force_full_load=False):
+    #Deliberately load models outside of the Aimdo mempool so they can be retained accross
+    #nodes. Use a dummy thread to do it as pytorch documents that mempool contexts are
+    #thread local. So exploit that to escape context
+    if enables_dynamic_vram():
+        t = threading.Thread(
+            target=load_models_gpu_thread,
+            args=(models, memory_required, force_patch_weights, minimum_memory_required, force_full_load)
+        )
+        t.start()
+        t.join()
+    else:
+        load_models_gpu_orig(models, memory_required=memory_required, force_patch_weights=force_patch_weights,
+                             minimum_memory_required=minimum_memory_required, force_full_load=force_full_load)
 
 def loaded_models(only_currently_used=False):
     output = []
@@ -1112,11 +1130,11 @@ def get_cast_buffer(offload_stream, device, size, ref):
             return None
         if cast_buffer is not None and cast_buffer.numel() > 50 * (1024 ** 2):
             #I want my wrongly sized 50MB+ of VRAM back from the caching allocator right now
-            torch.cuda.synchronize()
+            synchronize()
             del STREAM_CAST_BUFFERS[offload_stream]
             del cast_buffer
             #FIXME: This doesn't work in Aimdo because mempool cant clear cache
-            torch.cuda.empty_cache()
+            soft_empty_cache()
         with wf_context:
             cast_buffer = torch.empty((size), dtype=torch.int8, device=device)
             STREAM_CAST_BUFFERS[offload_stream] = cast_buffer
@@ -1132,9 +1150,7 @@ def reset_cast_buffers():
     for offload_stream in STREAM_CAST_BUFFERS:
         offload_stream.synchronize()
     STREAM_CAST_BUFFERS.clear()
-    if comfy.memory_management.aimdo_allocator is None:
-        #Pytorch 2.7 and earlier crashes if you try and empty_cache when mempools exist
-        torch.cuda.empty_cache()
+    soft_empty_cache()
 
 def get_offload_stream(device):
     stream_counter = stream_counters.get(device, 0)
@@ -1284,7 +1300,7 @@ def discard_cuda_async_error():
         a = torch.tensor([1], dtype=torch.uint8, device=get_torch_device())
         b = torch.tensor([1], dtype=torch.uint8, device=get_torch_device())
         _ = a + b
-        torch.cuda.synchronize()
+        synchronize()
     except torch.AcceleratorError:
         #Dump it! We already know about it from the synchronous return
         pass
@@ -1688,6 +1704,12 @@ def lora_compute_dtype(device):
     LORA_COMPUTE_DTYPES[device] = dtype
     return dtype
 
+def synchronize():
+    if is_intel_xpu():
+        torch.xpu.synchronize()
+    elif torch.cuda.is_available():
+        torch.cuda.synchronize()
+
 def soft_empty_cache(force=False):
     global cpu_state
     if cpu_state == CPUState.MPS:
@@ -1713,9 +1735,6 @@ def debug_memory_summary():
         return torch.cuda.memory.memory_summary()
     return ""
 
-#TODO: might be cleaner to put this somewhere else
-import threading
-
 class InterruptProcessingException(Exception):
     pass
 

comfy/model_patcher.py

@@ -1597,7 +1597,7 @@ class ModelPatcherDynamic(ModelPatcher):
 
         if unpatch_weights:
             self.partially_unload_ram(1e32)
-            self.partially_unload(None)
+            self.partially_unload(None, 1e32)
 
     def partially_load(self, device_to, extra_memory=0, force_patch_weights=False):
         assert not force_patch_weights #See above

compose.yaml

@@ -0,0 +1,46 @@
+# Docker Compose file to run ComfyUI locally using Docker.
+
+services:
+  comfyui:
+    container_name: comfyui
+    build:
+      context: .
+      args:
+        # Declare additional system dependencies for custom nodes
+        APT_EXTRA_PACKAGES:
+
+    ports:
+      - 8188:8188
+
+    # Optional: enable GPU access for hardware acceleration.
+    deploy:
+      resources:
+        reservations:
+          devices:
+            - capabilities: [gpu]
+    volumes:
+      - ./custom_nodes:/comfyui/custom_nodes
+      - ./models:/comfyui/models
+
+      # (Optional) Mount host ComfyUI data directories
+      #
+      #- ./input:/comfyui/input
+      #- ./output:/comfyui/output
+      #- ./temp:/comfyui/temp
+      #- ./user:/comfyui/user
+
+    environment:
+      # Overwrite the container user's UID and GID to match the host's. This
+      # allows files created by ComfyUI to be mounted on the host without
+      # permission issues.
+      UID: 1000
+      GID: 1000
+      # Declare additional Python packages to install. Useful when a custom node
+      # pack does not properly specify all its dependencies or relies on
+      # optional dependencies.
+      PIP_EXTRA_PACKAGES:
+
+    # Optional: Override the default command. In this case, configure ComfyUI to
+    # listen on all network interfaces (which is required when not using
+    # `network_mode=host`.)
+    command: python ./main.py --listen 0.0.0.0

entrypoint.sh

@@ -0,0 +1,62 @@
+#!/bin/sh
+
+# Entrypoint script for the ComfyUI Docker image.
+
+set -e
+
+user="comfyui"
+user_group="$user"
+
+# Allow users to specify a UID and GID matching their own, so files created
+# inside the container retain the same numeric ownership when mounted on the
+# host.
+if [ -n "$UID" ] && [ -n "$GID" ]; then
+    echo "[entrypoint] Setting user UID and GID..."
+    usermod  -u "$UID" "$user" > /dev/null
+    groupmod -g "$GID" "$user_group"
+else
+    echo "[entrypoint] Missing UID or GID environment variables; keeping default values."
+fi
+
+# Changing a user's UID and GID revokes that user's access to files owned by the
+# original UID/GID. To preserve access to runtime data, the ownership of those
+# directories must be updated recursively so that their numeric owner matches
+# the user's new UID and GID.
+echo "[entrypoint] Changing directory ownership..."
+chown -R "$user:$user_group" \
+    /comfyui                 \
+    /home/comfyui
+
+# To use CUDA and other NVIDIA features, regular users must belong to the group
+# that owns the /dev/nvidia* device files -- typically the video group.
+#
+# Known issue: Because these device files are mounted from the host system,
+# there's no guarantee that the device's group ID will match the intended group
+# inside the container. For example, the video group might be mapped to GID 27
+# on the host, which corresponds to the sudo group in the python:3.12 image.
+# This shouldn't cause major problems, and given the lack of a universal
+# standard for system GIDs, there isn't much we can realistically change to
+# address this issue.
+echo "[entrypoint] Adding user to GPU device groups..."
+for dev in /dev/nvidia*; do
+    group=$(ls -ld "$dev" | awk '{print $4}')
+    usermod -aG "$group" "$user"
+done
+
+# Install or update the Python dependencies defined by ComfyUI (or any installed
+# custom node) and also install any user-defined dependencies specified in
+# PIP_EXTRA_PACKAGES.
+echo "[entrypoint] Updating Python dependencies..."
+su -c "
+   pip install                      \\
+        --no-cache-dir              \\
+        --disable-pip-version-check \\
+        -r requirements.txt         \\
+        $(find custom_nodes -mindepth 2 -maxdepth 2 -type f -name requirements.txt -printf "-r '%p' ") \\
+        $PIP_EXTRA_PACKAGES
+" comfyui \
+    || echo "[entrypoint] Failed to install dependencies, starting anyway" >&2
+
+# Run command as comfyui
+echo "[entrypoint] Running command"
+exec su -c "$*" comfyui