Load the projector.safetensors file with the ModelPatchLoader node and use
the siglip_vision_patch14_384.safetensors "clip vision" model and the
USOStyleReferenceNode.
* Attempting a universal implementation of EasyCache, starting with flux as test; I screwed up the math a bit, but when I set it just right it works.
* Fixed math to make threshold work as expected, refactored code to use EasyCacheHolder instead of a dict wrapped by object
* Use sigmas from transformer_options instead of timesteps to be compatible with a greater amount of models, make end_percent work
* Make log statement when not skipping useful, preparing for per-cond caching
* Added DIFFUSION_MODEL wrapper around forward function for wan model
* Add subsampling for heuristic inputs
* Add subsampling to output_prev (output_prev_subsampled now)
* Properly consider conds in EasyCache logic
* Created SuperEasyCache to test what happens if caching and reuse is moved outside the scope of conds, added PREDICT_NOISE wrapper to facilitate this test
* Change max reuse_threshold to 3.0
* Mark EasyCache/SuperEasyCache as experimental (beta)
* Make Lumina2 compatible with EasyCache
* Add EasyCache support for Qwen Image
* Fix missing comma, curse you Cursor
* Add EasyCache support to AceStep
* Add EasyCache support to Chroma
* Added EasyCache support to Cosmos Predict t2i
* Make EasyCache not crash with Cosmos Predict ImagToVideo latents, but does not work well at all
* Add EasyCache support to hidream
* Added EasyCache support to hunyuan video
* Added EasyCache support to hunyuan3d
* Added EasyCache support to LTXV (not very good, but does not crash)
* Implemented EasyCache for aura_flow
* Renamed SuperEasyCache to LazyCache, hardcoded subsample_factor to 8 on nodes
* Eatra logging when verbose is true for EasyCache
These are not real controlnets but actually a patch on the model so they
will be treated as such.
Put them in the models/model_patches/ folder.
Use the new ModelPatchLoader and QwenImageDiffsynthControlnet nodes.
* P2 of qwen edit model.
* Typo.
* Fix normal qwen.
* Fix.
* Make the TextEncodeQwenImageEdit also set the ref latent.
If you don't want it to set the ref latent and want to use the
ReferenceLatent node with your custom latent instead just disconnect the
VAE.
This node is only useful if someone trains the kontext model to properly
use multiple reference images via the index method.
The default is the offset method which feeds the multiple images like if
they were stitched together as one. This method works with the current
flux kontext model.
* Added initial support for basic context windows - in progress
* Add prepare_sampling wrapper for context window to more accurately estimate latent memory requirements, fixed merging wrappers/callbacks dicts in prepare_model_patcher
* Made context windows compatible with different dimensions; works for WAN, but results are bad
* Fix comfy.patcher_extension.merge_nested_dicts calls in prepare_model_patcher in sampler_helpers.py
* Considering adding some callbacks to context window code to allow extensions of behavior without the need to rewrite code
* Made dim slicing cleaner
* Add Wan Context WIndows node for testing
* Made context schedule and fuse method functions be stored on the handler instead of needing to be registered in core code to be found
* Moved some code around between node_context_windows.py and context_windows.py
* Change manual context window nodes names/ids
* Added callbacks to IndexListContexHandler
* Adjusted default values for context_length and context_overlap, made schema.inputs definition for WAN Context Windows less annoying
* Make get_resized_cond more robust for various dim sizes
* Fix typo
* Another small fix
* ComfyAPI Core v0.0.2
* Respond to PR feedback
* Fix Python 3.9 errors
* Fix missing backward compatibility proxy
* Reorganize types a bit
The input types, input impls, and utility types are now all available in
the versioned API. See the change in `comfy_extras/nodes_video.py` for
an example of their usage.
* Remove the need for `--generate-api-stubs`
* Fix generated stubs differing by Python version
* Fix ruff formatting issues
* Add factorization utils for lokr
* Add lokr train impl
* Add loha train impl
* Add adapter map for algo selection
* Add optional grad ckpt and algo selection
* Update __init__.py
* correct key name for loha
* Use custom fwd/bwd func and better init for loha
* Support gradient accumulation
* Fix bugs of loha
* use more stable init
* Add OFT training
* linting
* [feat] Add GetImageSize node to return image dimensions
Added a simple GetImageSize node in comfy_extras/nodes_images.py that returns width and height of input images. The node displays dimensions on the UI via PromptServer and provides width/height as outputs for further processing.
* add display name mapping
* [fix] Add server module mock to unit tests for PromptServer import
Updated test to mock server module preventing import errors from the new PromptServer usage in GetImageSize node. Uses direct import pattern consistent with rest of codebase.
* [feat] Add ImageStitch node for concatenating images with borders
Add ImageStitch node that concatenates images in four directions with optional borders and intelligent size handling. Features include optional second image input, configurable borders with color selection, automatic batch size matching, and dimension alignment via padding or resizing.
Upstreamed from https://github.com/kijai/ComfyUI-KJNodes with enhancements for better error handling and comprehensive test coverage.
* [fix] Fix CI issues with CUDA dependencies and linting
- Mock CUDA-dependent modules in tests to avoid CI failures on CPU-only runners
- Fix ruff linting issues for code style compliance
* [fix] Improve CI compatibility by mocking nodes module import
Prevent CUDA initialization chain by mocking the nodes module at import time,
which is cleaner than deep mocking of CUDA-specific functions.
* [refactor] Clean up ImageStitch tests
- Remove unnecessary sys.path manipulation (pythonpath set in pytest.ini)
- Remove metadata tests that test framework internals rather than functionality
- Rename complex scenario test to be more descriptive of what it tests
* [refactor] Rename 'border' to 'spacing' for semantic accuracy
- Change border_width/border_color to spacing_width/spacing_color in API
- Update all tests to use spacing terminology
- Update comments and variable names throughout
- More accurately describes the gap/separator between images
* Make torch compile node use wrapper instead of object_patch for the entire diffusion_models object, allowing key assotiations on diffusion_models to not break (loras, getting attributes, etc.)
* Moved torch compile code into comfy_api so it can be used by custom nodes with a degree of confidence
* Refactor set_torch_compile_wrapper to support a list of keys instead of just diffusion_model, as well as additional torch.compile args
* remove unused import
* Moved torch compile kwargs to be stored in model_options instead of attachments; attachments are more intended for things to be 'persisted', AKA not deepcopied
* Add some comments
* Remove random line of code, not sure how it got there
* support wan camera models
* fix by ruff check
* change camera_condition type; make camera_condition optional
* support camera trajectory nodes
* fix camera direction
---------
Co-authored-by: Qirui Sun <sunqr0667@126.com>
* first pass at opus and mp3 as well as migrating flac to pyav
* minor mp3 encoding fix
* fix ruff
* delete dead code
* split out save audio to separate nodes per filetype
* fix ruff
* Add Ideogram generate node.
* Add staging api.
* Add API_NODE and common error for missing auth token (#5)
* Add Minimax Video Generation + Async Task queue polling example (#6)
* [Minimax] Show video preview and embed workflow in ouput (#7)
* Remove uv.lock
* Remove polling operations.
* Revert "Remove polling operations."
* Update stubs.
* Added Ideogram and Minimax back in.
* Added initial BFL Flux 1.1 [pro] Ultra node (#11)
* Add --comfy-api-base launch arg (#13)
* Add instructions for staging development. (#14)
* remove validation to make it easier to run against LAN copies of the API
* Manually add BFL polling status response schema (#15)
* Add function for uploading files. (#18)
* Add Luma nodes (#16)
* Refactor util functions (#20)
* Add VIDEO type (#21)
* Add rest of Luma node functionality (#19)
* Fix image_luma_ref not working (#28)
* [Bug] Remove duplicated option T2V-01 in MinimaxTextToVideoNode (#31)
* Add utils to map from pydantic model fields to comfy node inputs (#30)
* add veo2, bump av req (#32)
* Add Recraft nodes (#29)
* Add Kling Nodes (#12)
* Add Camera Concepts (luma_concepts) to Luma Video nodes (#33)
* Add Runway nodes (#17)
* Convert Minimax node to use VIDEO output type (#34)
* Standard `CATEGORY` system for api nodes (#35)
* Set `Content-Type` header when uploading files (#36)
* add better error propagation to veo2 (#37)
* Add Realistic Image and Logo Raster styles for Recraft v3 (#38)
* Fix runway image upload and progress polling (#39)
* Fix image upload for Luma: only include `Content-Type` header field if it's set explicitly (#40)
* Moved Luma nodes to nodes_luma.py (#47)
* Moved Recraft nodes to nodes_recraft.py (#48)
* Add Pixverse nodes (#46)
* Move and fix BFL nodes to node_bfl.py (#49)
* Move and edit Minimax node to nodes_minimax.py (#50)
* Add Minimax Image to Video node + Cleanup (#51)
* Add Recraft Text to Vector node, add Save SVG node to handle its output (#53)
* Added pixverse_template support to Pixverse Text to Video node (#54)
* Added Recraft Controls + Recraft Color RGB nodes (#57)
* split remaining nodes out of nodes_api, make utility lib, refactor ideogram (#61)
* Add types and doctstrings to utils file (#64)
* Fix: `PollingOperation` progress bar update progress by absolute value (#65)
* Use common download function in kling nodes module (#67)
* Fix: Luma video nodes in `api nodes/image` category (#68)
* Set request type explicitly (#66)
* Add `control_after_generate` to all seed inputs (#69)
* Fix bug: deleting `Content-Type` when property does not exist (#73)
* Add preview to Save SVG node (#74)
* change default poll interval (#76), rework veo2
* Add Pixverse and updated Kling types (#75)
* Added Pixverse Image to VIdeo node (#77)
* Add Pixverse Transition Video node (#79)
* Proper ray-1-6 support as fix has been applied in backend (#80)
* Added Recraft Style - Infinite Style Library node (#82)
* add ideogram v3 (#83)
* [Kling] Split Camera Control config to its own node (#81)
* Add Pika i2v and t2v nodes (#52)
* Temporary Fix for Runway (#87)
* Added Stability Stable Image Ultra node (#86)
* Remove Runway nodes (#88)
* Fix: Prompt text can't be validated in Kling nodes when using primitive nodes (#90)
* Fix: typo in node name "Stabiliy" => "Stability" (#91)
* Add String (Multiline) node (#93)
* Update Pika Duration and Resolution options (#94)
* Change base branch to master. Not main. (#95)
* Fix UploadRequest file_name param (#98)
* Removed Infinite Style Library until later (#99)
* fix ideogram style types (#100)
* fix multi image return (#101)
* add metadata saving to SVG (#102)
* Bump templates version to include API node template workflows (#104)
* Fix: `download_url_to_video_output` return type (#103)
* fix 4o generation bug (#106)
* Serve SVG files directly (#107)
* Add a bunch of nodes, 3 ready to use, the rest waiting for endpoint support (#108)
* Revert "Serve SVG files directly" (#111)
* Expose 4 remaining Recraft nodes (#112)
* [Kling] Add `Duration` and `Video ID` outputs (#105)
* Fix: datamodel-codegen sets string#binary type to non-existent `bytes_aliased` variable (#114)
* Fix: Dall-e 2 not setting request content-type dynamically (#113)
* Default request timeout: one hour. (#116)
* Add Kling nodes: camera control, start-end frame, lip-sync, video extend (#115)
* Add 8 nodes - 4 BFL, 4 Stability (#117)
* Fix error for Recraft ImageToImage error for nonexistent random_seed param (#118)
* Add remaining Pika nodes (#119)
* Make controls input work for Recraft Image to Image node (#120)
* Use upstream PR: Support saving Comfy VIDEO type to buffer (#123)
* Use Upstream PR: "Fix: Error creating video when sliced audio tensor chunks are non-c-contiguous" (#127)
* Improve audio upload utils (#128)
* Fix: Nested `AnyUrl` in request model cannot be serialized (Kling, Runway) (#129)
* Show errors and API output URLs to the user (change log levels) (#131)
* Fix: Luma I2I fails when weight is <=0.01 (#132)
* Change category of `LumaConcepts` node from image to video (#133)
* Fix: `image.shape` accessed before `image` is null-checked (#134)
* Apply small fixes and most prompt validation (if needed to avoid API error) (#135)
* Node name/category modifications (#140)
* Add back Recraft Style - Infinite Style Library node (#141)
* Fixed Kling: Check attributes of pydantic types. (#144)
* Bump `comfyui-workflow-templates` version (#142)
* [Kling] Print response data when error validating response (#146)
* Fix: error validating Kling image response, trying to use `"key" in` on Pydantic class instance (#147)
* [Kling] Fix: Correct/verify supported subset of input combos in Kling nodes (#149)
* [Kling] Fix typo in node description (#150)
* [Kling] Fix: CFG min/max not being enforced (#151)
* Rebase launch-rebase (private) on prep-branch (public copy of master) (#153)
* Bump templates version (#154)
* Fix: Kling image gen nodes don't return entire batch when `n` > 1 (#152)
* Remove pixverse_template from PixVerse Transition Video node (#155)
* Invert image_weight value on Luma Image to Image node (#156)
* Invert and resize mask for Ideogram V3 node to match masking conventions (#158)
* [Kling] Fix: image generation nodes not returning Tuple (#159)
* [Bug] [Kling] Fix Kling camera control (#161)
* Kling Image Gen v2 + improve node descriptions for Flux/OpenAI (#160)
* [Kling] Don't return video_id from dual effect video (#162)
* Bump frontend to 1.18.8 (#163)
* Use 3.9 compat syntax (#164)
* Use Python 3.10
* add example env var
* Update templates to 0.1.11
* Bump frontend to 1.18.9
---------
Co-authored-by: Robin Huang <robin.j.huang@gmail.com>
Co-authored-by: Christian Byrne <cbyrne@comfy.org>
Co-authored-by: thot experiment <94414189+thot-experiment@users.noreply.github.com>
* Upload files for Chroma Implementation
* Remove trailing whitespace
* trim more trailing whitespace..oops
* remove unused imports
* Add supported_inference_dtypes
* Set min_length to 0 and remove attention_mask=True
* Set min_length to 1
* get_mdulations added from blepping and minor changes
* Add lora conversion if statement in lora.py
* Update supported_models.py
* update model_base.py
* add uptream commits
* set modelType.FLOW, will cause beta scheduler to work properly
* Adjust memory usage factor and remove unnecessary code
* fix mistake
* reduce code duplication
* remove unused imports
* refactor for upstream sync
* sync chroma-support with upstream via syncbranch patch
* Update sd.py
* Add Chroma as option for the OptimalStepsScheduler node
* Add basic support for videos as types
This PR adds support for VIDEO as first-class types. In order to avoid
unnecessary costs, VIDEO outputs must implement the `VideoInput` ABC,
but their implementation details can vary. Included are two
implementations of this type which can be returned by other nodes:
* `VideoFromFile` - Created with either a path on disk (as a string) or
a `io.BytesIO` containing the contents of a file in a supported format
(like .mp4). This implementation won't actually load the video unless
necessary. It will also avoid re-encoding when saving if possible.
* `VideoFromComponents` - Created from an image tensor and an optional
audio tensor.
Currently, only h264 encoded videos in .mp4 containers are supported for
saving, but the plan is to add additional encodings/containers in the
near future (particularly .webm).
* Add optimization to avoid parsing entire video
* Improve type declarations to reduce warnings
* Make sure bytesIO objects can be read many times
* Fix a potential issue when saving long videos
* Fix incorrect type annotation
* Add a `LoadVideo` node to make testing easier
* Refactor new types out of the base comfy folder
I've created a new `comfy_api` top-level module. The intention is that
anything within this folder would be covered by semver-style versioning
that would allow custom nodes to rely on them not introducing breaking
changes.
* Fix linting issue
* draft pass at a native comfy implementation of Lotus-D depth and normal est
* fix model_sampling kludges
* fix ruff
---------
Co-authored-by: comfyanonymous <121283862+comfyanonymous@users.noreply.github.com>
This commit relaxes divisibility constraint for single-frame
conditionings. For single frames, the index can be arbitrary, while
multi-frame conditionings (>= 9 frames) must still be aligned to 8
frames.
Co-authored-by: Andrew Kvochko <a.kvochko@lightricks.com>
