* execution: Roll the UI cache into the outputs
Currently the UI cache is parallel to the output cache with
expectations of being a content superset of the output cache.
At the same time the UI and output cache are maintained completely
seperately, making it awkward to free the output cache content without
changing the behaviour of the UI cache.
There are two actual users (getters) of the UI cache. The first is
the case of a direct content hit on the output cache when executing a
node. This case is very naturally handled by merging the UI and outputs
cache.
The second case is the history JSON generation at the end of the prompt.
This currently works by asking the cache for all_node_ids and then
pulling the cache contents for those nodes. all_node_ids is the nodes
of the dynamic prompt.
So fold the UI cache into the output cache. The current UI cache setter
now writes to a prompt-scope dict. When the output cache is set, just
get this value from the dict and tuple up with the outputs.
When generating the history, simply iterate prompt-scope dict.
This prepares support for more complex caching strategies (like RAM
pressure caching) where less than 1 workflow will be cached and it
will be desirable to keep the UI cache and output cache in sync.
* sd: Implement RAM getter for VAE
* model_patcher: Implement RAM getter for ModelPatcher
* sd: Implement RAM getter for CLIP
* Implement RAM Pressure cache
Implement a cache sensitive to RAM pressure. When RAM headroom drops
down below a certain threshold, evict RAM-expensive nodes from the
cache.
Models and tensors are measured directly for RAM usage. An OOM score
is then computed based on the RAM usage of the node.
Note the due to indirection through shared objects (like a model
patcher), multiple nodes can account the same RAM as their individual
usage. The intent is this will free chains of nodes particularly
model loaders and associate loras as they all score similar and are
sorted in close to each other.
Has a bias towards unloading model nodes mid flow while being able
to keep results like text encodings and VAE.
* execution: Convert the cache entry to NamedTuple
As commented in review.
Convert this to a named tuple and abstract away the tuple type
completely from graph.py.
* Change bf16 check and switch non-blocking to off default with option to force to regain speed on certain classes of iGPUs and refactor xpu check.
* Turn non_blocking off by default for xpu.
* Update README.md for Intel GPUs.
* feat: “--whitelist-custom-nodes” args for comfy core to go with “--disable-all-custom-nodes” for development purposes
* feat: Simplify custom nodes whitelist logic to use consistent code paths
* Add Ideogram generate node.
* Add staging api.
* Add API_NODE and common error for missing auth token (#5)
* Add Minimax Video Generation + Async Task queue polling example (#6)
* [Minimax] Show video preview and embed workflow in ouput (#7)
* Remove uv.lock
* Remove polling operations.
* Revert "Remove polling operations."
* Update stubs.
* Added Ideogram and Minimax back in.
* Added initial BFL Flux 1.1 [pro] Ultra node (#11)
* Add --comfy-api-base launch arg (#13)
* Add instructions for staging development. (#14)
* remove validation to make it easier to run against LAN copies of the API
* Manually add BFL polling status response schema (#15)
* Add function for uploading files. (#18)
* Add Luma nodes (#16)
* Refactor util functions (#20)
* Add VIDEO type (#21)
* Add rest of Luma node functionality (#19)
* Fix image_luma_ref not working (#28)
* [Bug] Remove duplicated option T2V-01 in MinimaxTextToVideoNode (#31)
* Add utils to map from pydantic model fields to comfy node inputs (#30)
* add veo2, bump av req (#32)
* Add Recraft nodes (#29)
* Add Kling Nodes (#12)
* Add Camera Concepts (luma_concepts) to Luma Video nodes (#33)
* Add Runway nodes (#17)
* Convert Minimax node to use VIDEO output type (#34)
* Standard `CATEGORY` system for api nodes (#35)
* Set `Content-Type` header when uploading files (#36)
* add better error propagation to veo2 (#37)
* Add Realistic Image and Logo Raster styles for Recraft v3 (#38)
* Fix runway image upload and progress polling (#39)
* Fix image upload for Luma: only include `Content-Type` header field if it's set explicitly (#40)
* Moved Luma nodes to nodes_luma.py (#47)
* Moved Recraft nodes to nodes_recraft.py (#48)
* Add Pixverse nodes (#46)
* Move and fix BFL nodes to node_bfl.py (#49)
* Move and edit Minimax node to nodes_minimax.py (#50)
* Add Minimax Image to Video node + Cleanup (#51)
* Add Recraft Text to Vector node, add Save SVG node to handle its output (#53)
* Added pixverse_template support to Pixverse Text to Video node (#54)
* Added Recraft Controls + Recraft Color RGB nodes (#57)
* split remaining nodes out of nodes_api, make utility lib, refactor ideogram (#61)
* Add types and doctstrings to utils file (#64)
* Fix: `PollingOperation` progress bar update progress by absolute value (#65)
* Use common download function in kling nodes module (#67)
* Fix: Luma video nodes in `api nodes/image` category (#68)
* Set request type explicitly (#66)
* Add `control_after_generate` to all seed inputs (#69)
* Fix bug: deleting `Content-Type` when property does not exist (#73)
* Add preview to Save SVG node (#74)
* change default poll interval (#76), rework veo2
* Add Pixverse and updated Kling types (#75)
* Added Pixverse Image to VIdeo node (#77)
* Add Pixverse Transition Video node (#79)
* Proper ray-1-6 support as fix has been applied in backend (#80)
* Added Recraft Style - Infinite Style Library node (#82)
* add ideogram v3 (#83)
* [Kling] Split Camera Control config to its own node (#81)
* Add Pika i2v and t2v nodes (#52)
* Temporary Fix for Runway (#87)
* Added Stability Stable Image Ultra node (#86)
* Remove Runway nodes (#88)
* Fix: Prompt text can't be validated in Kling nodes when using primitive nodes (#90)
* Fix: typo in node name "Stabiliy" => "Stability" (#91)
* Add String (Multiline) node (#93)
* Update Pika Duration and Resolution options (#94)
* Change base branch to master. Not main. (#95)
* Fix UploadRequest file_name param (#98)
* Removed Infinite Style Library until later (#99)
* fix ideogram style types (#100)
* fix multi image return (#101)
* add metadata saving to SVG (#102)
* Bump templates version to include API node template workflows (#104)
* Fix: `download_url_to_video_output` return type (#103)
* fix 4o generation bug (#106)
* Serve SVG files directly (#107)
* Add a bunch of nodes, 3 ready to use, the rest waiting for endpoint support (#108)
* Revert "Serve SVG files directly" (#111)
* Expose 4 remaining Recraft nodes (#112)
* [Kling] Add `Duration` and `Video ID` outputs (#105)
* Fix: datamodel-codegen sets string#binary type to non-existent `bytes_aliased` variable (#114)
* Fix: Dall-e 2 not setting request content-type dynamically (#113)
* Default request timeout: one hour. (#116)
* Add Kling nodes: camera control, start-end frame, lip-sync, video extend (#115)
* Add 8 nodes - 4 BFL, 4 Stability (#117)
* Fix error for Recraft ImageToImage error for nonexistent random_seed param (#118)
* Add remaining Pika nodes (#119)
* Make controls input work for Recraft Image to Image node (#120)
* Use upstream PR: Support saving Comfy VIDEO type to buffer (#123)
* Use Upstream PR: "Fix: Error creating video when sliced audio tensor chunks are non-c-contiguous" (#127)
* Improve audio upload utils (#128)
* Fix: Nested `AnyUrl` in request model cannot be serialized (Kling, Runway) (#129)
* Show errors and API output URLs to the user (change log levels) (#131)
* Fix: Luma I2I fails when weight is <=0.01 (#132)
* Change category of `LumaConcepts` node from image to video (#133)
* Fix: `image.shape` accessed before `image` is null-checked (#134)
* Apply small fixes and most prompt validation (if needed to avoid API error) (#135)
* Node name/category modifications (#140)
* Add back Recraft Style - Infinite Style Library node (#141)
* Fixed Kling: Check attributes of pydantic types. (#144)
* Bump `comfyui-workflow-templates` version (#142)
* [Kling] Print response data when error validating response (#146)
* Fix: error validating Kling image response, trying to use `"key" in` on Pydantic class instance (#147)
* [Kling] Fix: Correct/verify supported subset of input combos in Kling nodes (#149)
* [Kling] Fix typo in node description (#150)
* [Kling] Fix: CFG min/max not being enforced (#151)
* Rebase launch-rebase (private) on prep-branch (public copy of master) (#153)
* Bump templates version (#154)
* Fix: Kling image gen nodes don't return entire batch when `n` > 1 (#152)
* Remove pixverse_template from PixVerse Transition Video node (#155)
* Invert image_weight value on Luma Image to Image node (#156)
* Invert and resize mask for Ideogram V3 node to match masking conventions (#158)
* [Kling] Fix: image generation nodes not returning Tuple (#159)
* [Bug] [Kling] Fix Kling camera control (#161)
* Kling Image Gen v2 + improve node descriptions for Flux/OpenAI (#160)
* [Kling] Don't return video_id from dual effect video (#162)
* Bump frontend to 1.18.8 (#163)
* Use 3.9 compat syntax (#164)
* Use Python 3.10
* add example env var
* Update templates to 0.1.11
* Bump frontend to 1.18.9
---------
Co-authored-by: Robin Huang <robin.j.huang@gmail.com>
Co-authored-by: Christian Byrne <cbyrne@comfy.org>
Co-authored-by: thot experiment <94414189+thot-experiment@users.noreply.github.com>
This should speed up the lowvram mode a bit. It currently is only enabled when --async-offload is used but it will be enabled by default in the future if there are no problems.
* add dependency aware cache that removed a cached node as soon as all of its decendents have executed. This allows users with lower RAM to run workflows they would otherwise not be able to run. The downside is that every workflow will fully run each time even if no nodes have changed.
* remove test code
* tidy code
* Better argument handling of front-end-root
Improves handling of front-end-root launch argument. Several instances where users have set it and ComfyUI launches as normal and completely disregards the launch arg which doesn't make sense. Better to indicate to user that something is incorrect.
* Removed unused import
There was no real reason to use "Optional" typing in ther front-end-root argument.
The idea is that you can indicate how much quality vs speed you want.
At the moment:
--fast 2 enables fp16 accumulation if your pytorch supports it.
--fast 5 enables fp8 matrix mult on fp8 models and the optimization above.
--fast without a number enables all optimizations.
