* Apply cond slice fix
* Add FreeNoise
* Update context_windows.py
* Add option to retain condition by indexes for each window
This allows for example Wan/HunyuanVideo image to video to "work" by using the initial start frame for each window, otherwise windows beyond first will be pure T2V generations.
* Update context_windows.py
* Allow splitting multiple conds into different windows
* Add handling for audio_embed
* whitespace
* Allow freenoise to work on other dims, handle 4D batch timestep
Refactor Freenoise function. And fix batch handling as timesteps seem to be expanded to batch size now.
* Disable experimental options for now
So that the Freenoise and bugfixes can be merged first
---------
Co-authored-by: Jedrzej Kosinski <kosinkadink1@gmail.com>
Co-authored-by: ozbayb <17261091+ozbayb@users.noreply.github.com>
* chore(api-nodes): applied ruff's pyupgrade(python3.10) to api-nodes client's to folder
* chore(api-nodes): add validate_video_frame_count function from LTX PR
* chore(api-nodes): replace deprecated V1 imports
* fix(api-nodes): the types returned by the "poll_op" function are now correct.
Im able to push vram above estimate on partial unload. Bump the
estimate. This is experimentally determined with a 720P and 480P
datapoint calibrating for 24GB VRAM total.
TIL that the WAN TE has a 2GB weight followed by 16MB as the next size
down. This means that team 8GB VRAM would fully offload the TE in async
offload mode as it just multiplied this giant size my the num streams.
Do the more complex logic of summing up the upcoming to-load weight
sizes to avoid triple counting this massive weight.
partial unload does the converse of recording the NS most recent
unloads as they go.
* mp: only count the offload cost of math once
This was previously bundling the combined weight storage and computation
cost
* ops: put all post async transfer compute on the main stream
Some models have massive weights that need either complex
dequantization or lora patching. Don't do these patchings on the offload
stream, instead do them on the main stream to syncrhonize the
potentially large vram spikes for these compute processes. This avoids
having to assume a worst case scenario of multiple offload streams
all spiking VRAM is parallel with whatever the main stream is doing.
This should fix the problem with the portable updater not working with portables created from a separate branch on the repo.
This does not affect any current portables who were all created on the master branch.
* Added output_matchtypes to generated json for v3, initial backend support for MatchType, created nodes_logic.py and added SwitchNode
* Fixed providing list of allowed_types
* Add workaround in validation.py for V3 Combo outputs not working as Combo inputs
* Make match type receive_type pass validation
* Also add MatchType check to input_type in validation - will likely trigger when connecting to non-lazy stuff
* Make sure this PR only has MatchType stuff
* Initial work on DynamicCombo
* Add get_dynamic function, not yet filled out correctly
* Mark Switch node as Beta
* Make sure other unfinished dynamic types are not accidentally used
* Send DynamicCombo.Option inputs in the same format as normal v1 inputs
* add dynamic combo test node
* Support validation of inputs and outputs
* Add missing input params to DynamicCombo.Input
* Add get_all function to inputs for id validation purposes
* Fix imports for v3 returning everything when doing io/ui/IO/UI instead of what is in __all__ of _io.py and _ui.py
* Modifying behavior of get_dynamic in V3 + serialization so can be used in execution code
* Fix v3 schema validation code after changes
* Refactor hidden_values for v3 in execution.py to be more general v3_data, add helper functions for dynamic behavior, preparing for restructuring dynamic type into object (not finished yet)
* Add nesting of inputs on DynamicCombo during execution
* Work with latest frontend commits
* Fix cringe arrows
* frontend will no longer namespace dynamic inputs widgets so reflect that in code, refactor build_nested_inputs
* Prepare Autogrow support for the love of the game
* satisfy ruff
* Create test nodes for Autogrow to collab with frontend development
* Add nested combo to DCTestNode
* Remove array support from build_nested_inputs, properly handle missing expected values
* Make execution.validate_inputs properly validate required dynamic inputs, renamed dynamic_data to dynamic_paths for clarity
* MatchType does not need any DynamicInput/Output features on backend; will increase compatibility with dynamic types
* Probably need this for ruff check
* Change MatchType to have template be the first and only required param; output id's do nothing right now, so no need
* Fix merge regression with LatentUpscaleModel type not being put in __all__ for _io.py, fix invalid type hint for validate_inputs
* Make Switch node inputs optional, disallow both inputs from being missing, and still work properly with lazy; when one input is missing, use the other no matter what the switch is set to
* Satisfy ruff
* Move MatchType code above the types that inherit from DynamicInput
* Add DynamicSlot type, awaiting frontend support
* Make curr_prefix creation happen in Autogrow, move curr_prefix in DynamicCombo to only be created if input exists in live_inputs
* I was confused, fixing accidentally redundant curr_prefix addition in Autogrow
* Make sure Autogrow inputs are force_input = True when WidgetInput, fix runtime validation by removing original input from expected inputs, fix min/max bounds, change test nodes slightly
* Remove unnecessary id usage in Autogrow test node outputs
* Commented out Switch node + test nodes
* Remove commented out code from Autogrow
* Make TemplatePrefix max more clear, allow max == 1
* Replace all dict[str] with dict[str, Any]
* Renamed add_to_dict_live_inputs to expand_schema_for_dynamic
* Fixed typo in DynamicSlot input code
* note about live_inputs not being present soon in get_v1_info (internal function anyway)
* For now, hide DynamicCombo and Autogrow from public interface
* Removed comment
* hunyuan upsampler: rework imports
Remove the transitive import of VideoConv3d and Resnet and takes these
from actual implementation source.
* model: remove unused give_pre_end
According to git grep, this is not used now, and was not used in the
initial commit that introduced it (see below).
This semantic is difficult to implement temporal roll VAE for (and would
defeat the purpose). Rather than implement the complex if, just delete
the unused feature.
(venv) rattus@rattus-box2:~/ComfyUI$ git log --oneline
220afe33 (HEAD) Initial commit.
(venv) rattus@rattus-box2:~/ComfyUI$ git grep give_pre
comfy/ldm/modules/diffusionmodules/model.py: resolution, z_channels, give_pre_end=False, tanh_out=False, use_linear_attn=False,
comfy/ldm/modules/diffusionmodules/model.py: self.give_pre_end = give_pre_end
comfy/ldm/modules/diffusionmodules/model.py: if self.give_pre_end:
(venv) rattus@rattus-box2:~/ComfyUI$ git co origin/master
Previous HEAD position was 220afe33 Initial commit.
HEAD is now at 9d8a8179 Enable async offloading by default on Nvidia. (#10953)
(venv) rattus@rattus-box2:~/ComfyUI$ git grep give_pre
comfy/ldm/modules/diffusionmodules/model.py: resolution, z_channels, give_pre_end=False, tanh_out=False, use_linear_attn=False,
comfy/ldm/modules/diffusionmodules/model.py: self.give_pre_end = give_pre_end
comfy/ldm/modules/diffusionmodules/model.py: if self.give_pre_end:
* move refiner VAE temporal roller to core
Move the carrying conv op to the common VAE code and give it a better
name. Roll the carry implementation logic for Resnet into the base
class and scrap the Hunyuan specific subclass.
* model: Add temporal roll to main VAE decoder
If there are no attention layers, its a standard resnet and VideoConv3d
is asked for, substitute in the temporal rolloing VAE algorithm. This
reduces VAE usage by the temporal dimension (can be huge VRAM savings).
* model: Add temporal roll to main VAE encoder
If there are no attention layers, its a standard resnet and VideoConv3d
is asked for, substitute in the temporal rolling VAE algorithm. This
reduces VAE usage by the temporal dimension (can be huge VRAM savings).
Added PATCH http method to access-control-allow-header-methods header because there are now PATCH endpoints exposed in the API.
See 277237ccc1/api_server/routes/internal/internal_routes.py (L34) for an example of an API endpoint that uses the PATCH method.
These are not actual controlnets so put it in the models/model_patches
folder and use the ModelPatchLoader + QwenImageDiffsynthControlnet node to
use it.
RAMPressure caching may ned to purge the same model that you are
currently trying to offload for VRAM freeing. In this case, RAMPressure
cache takes priority and needs to be able to pull the trigger on dumping
the whole model and freeing the ModelPatcher in question. To do this,
defer the actual tranfer of model weights from GPU to RAM to
model_management state and not as part of ModelPatcher. This is dones as
a list of weakrefs.
If RAM cache decides to free to model you are currently unloading, then
the ModelPatcher and refs simply dissappear in the middle of the
unloading process, and both RAM and VRAM will be freed.
The unpatcher now queues the individual leaf modules to be offloaded
one-by-one so that RAM levels can be monitored.
Note that the UnloadPartially that is potentially done as part of a
load will not be freeable this way, however it shouldn't be anyway as
that is the currently active model and RAM cache cannot save you if
you cant even fit the one model you are currently trying to use.
This is currently put together as a list of indexes assuming the
current_loaded_models doesn't change. However we might need to pruge a
model as part of the offload process which means this list can change in
the middle of the freeing process. handle by taking independent refs to
the LoadedModel objects and dong safe by-value deletion of
current_loaded_models.
currently this hard assumes that the caller of model_unload will keep
current_loaded_models in sync. With RAMPressureCache its possible to
have the garbage collector occur in the middle of the model free process
which can split these two steps.
move the headroom logic into the RAM cache to make this a little easier
to call to "free me some RAM".
Rename the API to free_ram().
Split off the clean_list creation to a completely separate function to
avoid any stray strong reference to the content-to-be-freed on the
stack.
* feat(security): add System User protection with `__` prefix
Add protected namespace for custom nodes to store sensitive data
(API keys, licenses) that cannot be accessed via HTTP endpoints.
Key changes:
- New API: get_system_user_directory() for internal access
- New API: get_public_user_directory() with structural blocking
- 3-layer defense: header validation, path blocking, creation prevention
- 54 tests covering security, edge cases, and backward compatibility
System Users use `__` prefix (e.g., __system, __cache) following
Python's private member convention. They exist in user_directory/
but are completely blocked from /userdata HTTP endpoints.
* style: remove unused imports
* Support video tiny VAEs
* lighttaew scaling fix
* Also support video taes in previews
Only first frame for now as live preview playback is currently only available through VHS custom nodes.
* Support Wan 2.1 lightVAE
* Relocate elif block and set Wan VAE dim directly without using pruning rate for lightvae
