patientx
a779e34c5b
Merge branch 'comfyanonymous:master' into master
2025-01-16 11:29:26 +03:00
comfyanonymous
6320d05696
Slightly lower hunyuan video memory usage.
2025-01-16 00:23:01 -05:00
comfyanonymous
25683b5b02
Lower cosmos diffusion model memory usage.
2025-01-15 23:46:42 -05:00
comfyanonymous
4758fb64b9
Lower cosmos VAE memory usage by a bit.
2025-01-15 22:57:52 -05:00
comfyanonymous
008761166f
Optimize first attention block in cosmos VAE.
2025-01-15 21:48:46 -05:00
patientx
fb0d92b160
Merge branch 'comfyanonymous:master' into master
2025-01-15 23:33:05 +03:00
comfyanonymous
cba58fff0b
Remove unsafe embedding load for very old pytorch.
2025-01-15 04:32:23 -05:00
patientx
3ece3091d4
Merge branch 'comfyanonymous:master' into master
2025-01-15 11:55:19 +03:00
comfyanonymous
2feb8d0b77
Force safe loading of files in torch format on pytorch 2.4+
...
If this breaks something for you make an issue.
2025-01-15 03:50:27 -05:00
patientx
104e6a9685
Merge branch 'comfyanonymous:master' into master
2025-01-15 03:59:33 +03:00
Pam
c78a45685d
Rewrite res_multistep sampler and implement res_multistep_cfg_pp sampler. ( #6462 )
2025-01-14 18:20:06 -05:00
patientx
4f01f72bed
Update zluda.py
2025-01-14 20:03:55 +03:00
patientx
c4861c74d4
UPDATED ZLUDA PATCHING METHOD
2025-01-14 19:57:22 +03:00
patientx
c3fc894ce2
Add files via upload
2025-01-14 19:54:44 +03:00
patientx
c7ebd121d6
Merge branch 'comfyanonymous:master' into master
2025-01-14 15:50:05 +03:00
comfyanonymous
3aaabb12d4
Implement Cosmos Image/Video to World (Video) diffusion models.
...
Use CosmosImageToVideoLatent to set the input image/video.
2025-01-14 05:14:10 -05:00
patientx
aa2a83ec35
Merge branch 'comfyanonymous:master' into master
2025-01-13 14:46:19 +03:00
comfyanonymous
1f1c7b7b56
Remove useless code.
2025-01-13 03:52:37 -05:00
patientx
b03621d13b
Merge branch 'comfyanonymous:master' into master
2025-01-12 14:31:19 +03:00
comfyanonymous
90f349f93d
Add res_multistep sampler from the cosmos code.
...
This sampler should work with all models.
2025-01-12 03:10:07 -05:00
patientx
d319705d78
Merge branch 'comfyanonymous:master' into master
2025-01-12 00:39:02 +03:00
Jedrzej Kosinski
6c9bd11fa3
Hooks Part 2 - TransformerOptionsHook and AdditionalModelsHook ( #6377 )
...
* Add 'sigmas' to transformer_options so that downstream code can know about the full scope of current sampling run, fix Hook Keyframes' guarantee_steps=1 inconsistent behavior with sampling split across different Sampling nodes/sampling runs by referencing 'sigmas'
* Cleaned up hooks.py, refactored Hook.should_register and add_hook_patches to use target_dict instead of target so that more information can be provided about the current execution environment if needed
* Refactor WrapperHook into TransformerOptionsHook, as there is no need to separate out Wrappers/Callbacks/Patches into different hook types (all affect transformer_options)
* Refactored HookGroup to also store a dictionary of hooks separated by hook_type, modified necessary code to no longer need to manually separate out hooks by hook_type
* In inner_sample, change "sigmas" to "sampler_sigmas" in transformer_options to not conflict with the "sigmas" that will overwrite "sigmas" in _calc_cond_batch
* Refactored 'registered' to be HookGroup instead of a list of Hooks, made AddModelsHook operational and compliant with should_register result, moved TransformerOptionsHook handling out of ModelPatcher.register_all_hook_patches, support patches in TransformerOptionsHook properly by casting any patches/wrappers/hooks to proper device at sample time
* Made hook clone code sane, made clear ObjectPatchHook and SetInjectionsHook are not yet operational
* Fix performance of hooks when hooks are appended via Cond Pair Set Props nodes by properly caching between positive and negative conds, make hook_patches_backup behave as intended (in the case that something pre-registers WeightHooks on the ModelPatcher instead of registering it at sample time)
* Filter only registered hooks on self.conds in CFGGuider.sample
* Make hook_scope functional for TransformerOptionsHook
* removed 4 whitespace lines to satisfy Ruff,
* Add a get_injections function to ModelPatcher
* Made TransformerOptionsHook contribute to registered hooks properly, added some doc strings and removed a so-far unused variable
* Rename AddModelsHooks to AdditionalModelsHook, rename SetInjectionsHook to InjectionsHook (not yet implemented, but at least getting the naming figured out)
* Clean up a typehint
2025-01-11 12:20:23 -05:00
patientx
8ac79a7563
Merge branch 'comfyanonymous:master' into master
2025-01-11 15:04:10 +03:00
comfyanonymous
ee8a7ab69d
Fast latent preview for Cosmos.
2025-01-11 04:41:24 -05:00
patientx
7d45042e2e
Merge branch 'comfyanonymous:master' into master
2025-01-10 18:31:48 +03:00
comfyanonymous
2ff3104f70
WIP support for Nvidia Cosmos 7B and 14B text to world (video) models.
2025-01-10 09:14:16 -05:00
patientx
00cf1206e8
Merge branch 'comfyanonymous:master' into master
2025-01-10 15:44:49 +03:00
comfyanonymous
129d8908f7
Add argument to skip the output reshaping in the attention functions.
2025-01-10 06:27:37 -05:00
patientx
c37b5ccf29
Merge branch 'comfyanonymous:master' into master
2025-01-09 18:15:43 +03:00
comfyanonymous
ff838657fa
Cleaner handling of attention mask in ltxv model code.
2025-01-09 07:12:03 -05:00
patientx
0b933e1634
Merge branch 'comfyanonymous:master' into master
2025-01-09 12:57:31 +03:00
comfyanonymous
2307ff6746
Improve some logging messages.
2025-01-08 19:05:22 -05:00
patientx
867659b035
Merge branch 'comfyanonymous:master' into master
2025-01-08 01:49:22 +03:00
comfyanonymous
d0f3752e33
Properly calculate inner dim for t5 model.
...
This is required to support some different types of t5 models.
2025-01-07 17:33:03 -05:00
patientx
e1aa83d068
Merge branch 'comfyanonymous:master' into master
2025-01-07 13:57:13 +03:00
comfyanonymous
4209edf48d
Make a few more samplers deterministic.
2025-01-07 02:12:32 -05:00
patientx
58cfa6b7f3
Merge branch 'comfyanonymous:master' into master
2025-01-07 09:25:41 +03:00
Chenlei Hu
d055325783
Document get_attr and get_model_object ( #6357 )
...
* Document get_attr and get_model_object
* Update model_patcher.py
* Update model_patcher.py
* Update model_patcher.py
2025-01-06 20:12:22 -05:00
patientx
6c1e37c091
Merge branch 'comfyanonymous:master' into master
2025-01-06 14:50:48 +03:00
comfyanonymous
916d1e14a9
Make ancestral samplers more deterministic.
2025-01-06 03:04:32 -05:00
Jedrzej Kosinski
c496e53519
In inner_sample, change "sigmas" to "sampler_sigmas" in transformer_options to not conflict with the "sigmas" that will overwrite "sigmas" in _calc_cond_batch ( #6360 )
2025-01-06 01:36:47 -05:00
patientx
a5a09e45dd
Merge branch 'comfyanonymous:master' into master
2025-01-04 15:42:12 +03:00
comfyanonymous
d45ebb63f6
Remove old unused function.
2025-01-04 07:20:54 -05:00
patientx
c523c36aef
Merge branch 'comfyanonymous:master' into master
2025-01-02 19:55:49 +03:00
comfyanonymous
9e9c8a1c64
Clear cache as often on AMD as Nvidia.
...
I think the issue this was working around has been solved.
If you notice that this change slows things down or causes stutters on
your AMD GPU with ROCm on Linux please report it.
2025-01-02 08:44:16 -05:00
patientx
bfc4fb0efb
Merge branch 'comfyanonymous:master' into master
2025-01-02 00:37:45 +03:00
Andrew Kvochko
0f11d60afb
Fix temporal tiling for decoder, remove redundant tiles. ( #6306 )
...
This commit fixes the temporal tile size calculation, and removes
a redundant tile at the end of the range when its elements are
completely covered by the previous tile.
Co-authored-by: Andrew Kvochko <a.kvochko@lightricks.com>
2025-01-01 16:29:01 -05:00
patientx
0dbf8238af
Merge branch 'comfyanonymous:master' into master
2025-01-01 15:55:00 +03:00
comfyanonymous
79eea51a1d
Fix and enforce all ruff W rules.
2025-01-01 03:08:33 -05:00
patientx
5c8d73f4b4
Merge branch 'comfyanonymous:master' into master
2025-01-01 02:14:54 +03:00
blepping
c0338a46a4
Fix unknown sampler error handling in calculate_sigmas function ( #6280 )
...
Modernize calculate_sigmas function
2024-12-31 17:33:50 -05:00
patientx
5e9aa9bb9e
Merge branch 'comfyanonymous:master' into master
2024-12-31 23:34:57 +03:00
Jedrzej Kosinski
1c99734e5a
Add missing model_options param ( #6296 )
2024-12-31 14:46:55 -05:00
patientx
419df8d958
Merge branch 'comfyanonymous:master' into master
2024-12-31 12:22:04 +03:00
filtered
67758f50f3
Fix custom node type-hinting examples ( #6281 )
...
* Fix import in comfy_types doc / sample
* Clarify docstring
2024-12-31 03:41:09 -05:00
comfyanonymous
b7572b2f87
Fix and enforce no trailing whitespace.
2024-12-31 03:16:37 -05:00
patientx
cbcd4aa616
Merge branch 'comfyanonymous:master' into master
2024-12-30 15:02:02 +03:00
blepping
a90aafafc1
Add kl_optimal scheduler ( #6206 )
...
* Add kl_optimal scheduler
* Rename kl_optimal_schedule to kl_optimal_scheduler to be more consistent
2024-12-30 05:09:38 -05:00
patientx
349ddc0b92
Merge branch 'comfyanonymous:master' into master
2024-12-30 12:21:28 +03:00
comfyanonymous
d9b7cfac7e
Fix and enforce new lines at the end of files.
2024-12-30 04:14:59 -05:00
patientx
cd869622df
Merge branch 'comfyanonymous:master' into master
2024-12-30 11:54:26 +03:00
Jedrzej Kosinski
3507870535
Add 'sigmas' to transformer_options so that downstream code can know about the full scope of current sampling run, fix Hook Keyframes' guarantee_steps=1 inconsistent behavior with sampling split across different Sampling nodes/sampling runs by referencing 'sigmas' ( #6273 )
2024-12-30 03:42:49 -05:00
patientx
cd502aa403
Merge branch 'comfyanonymous:master' into master
2024-12-29 13:09:24 +03:00
comfyanonymous
a618f768e0
Auto reshape 2d to 3d latent for single image generation on video model.
2024-12-29 02:26:49 -05:00
patientx
9f71405928
Merge branch 'comfyanonymous:master' into master
2024-12-28 14:38:54 +03:00
comfyanonymous
b504bd606d
Add ruff rule for empty line with trailing whitespace.
2024-12-28 05:23:08 -05:00
patientx
b7c91dd68c
Merge branch 'comfyanonymous:master' into master
2024-12-28 12:02:44 +03:00
comfyanonymous
d170292594
Remove some trailing white space.
2024-12-27 18:02:30 -05:00
filtered
9cfd185676
Add option to log non-error output to stdout ( #6243 )
...
* nit
* Add option to log non-error output to stdout
- No change to default behaviour
- Adds CLI argument: --log-stdout
- With this arg present, any logging of a level below logging.ERROR will be sent to stdout instead of stderr
2024-12-27 14:40:05 -05:00
patientx
b2a9683d75
Merge branch 'comfyanonymous:master' into master
2024-12-27 15:38:26 +03:00
comfyanonymous
4b5bcd8ac4
Closer memory estimation for hunyuan dit model.
2024-12-27 07:37:00 -05:00
comfyanonymous
ceb50b2cbf
Closer memory estimation for pixart models.
2024-12-27 07:30:09 -05:00
patientx
4590f75633
Merge branch 'comfyanonymous:master' into master
2024-12-27 09:59:51 +03:00
comfyanonymous
160ca08138
Use python 3.9 in launch test instead of 3.8
...
Fix ruff check.
2024-12-26 20:05:54 -05:00
Huazhong Ji
c4bfdba330
Support ascend npu ( #5436 )
...
* support ascend npu
Co-authored-by: YukMingLaw <lymmm2@163.com>
Co-authored-by: starmountain1997 <guozr1997@hotmail.com>
Co-authored-by: Ginray <ginray0215@gmail.com>
2024-12-26 19:36:50 -05:00
patientx
1f9acbbfca
Merge branch 'comfyanonymous:master' into master
2024-12-26 23:22:48 +03:00
comfyanonymous
ee9547ba31
Improve temporal VAE Encode (Tiled) math.
2024-12-26 07:18:49 -05:00
patientx
49fa16cc7a
Merge branch 'comfyanonymous:master' into master
2024-12-25 14:05:18 +03:00
comfyanonymous
19a64d6291
Cleanup some mac related code.
2024-12-25 05:32:51 -05:00
comfyanonymous
b486885e08
Disable bfloat16 on older mac.
2024-12-25 05:18:50 -05:00
comfyanonymous
0229228f3f
Clean up the VAE dtypes code.
2024-12-25 04:50:34 -05:00
comfyanonymous
99a1fb6027
Make fast fp8 take a bit less peak memory.
2024-12-24 18:05:19 -05:00
patientx
077ebf7b17
Merge branch 'comfyanonymous:master' into master
2024-12-24 16:53:56 +03:00
comfyanonymous
73e04987f7
Prevent black images in VAE Decode (Tiled) node.
...
Overlap should be minimum 1 with tiling 2 for tiled temporal VAE decoding.
2024-12-24 07:36:30 -05:00
comfyanonymous
5388df784a
Add temporal tiling to VAE Encode (Tiled) node.
2024-12-24 07:10:09 -05:00
patientx
0f7b4f063d
Merge branch 'comfyanonymous:master' into master
2024-12-24 15:06:02 +03:00
comfyanonymous
bc6dac4327
Add temporal tiling to VAE Decode (Tiled) node.
...
You can now do tiled VAE decoding on the temporal direction for videos.
2024-12-23 20:03:37 -05:00
patientx
00afa8b34f
Merge branch 'comfyanonymous:master' into master
2024-12-23 11:36:49 +03:00
comfyanonymous
15564688ed
Add a try except block so if torch version is weird it won't crash.
2024-12-23 03:22:48 -05:00
Simon Lui
c6b9c11ef6
Add oneAPI device selector for xpu and some other changes. ( #6112 )
...
* Add oneAPI device selector and some other minor changes.
* Fix device selector variable name.
* Flip minor version check sign.
* Undo changes to README.md.
2024-12-23 03:18:32 -05:00
patientx
403a081215
Merge branch 'comfyanonymous:master' into master
2024-12-23 10:33:06 +03:00
comfyanonymous
e44d0ac7f7
Make --novram completely offload weights.
...
This flag is mainly used for testing the weight offloading, it shouldn't
actually be used in practice.
Remove useless import.
2024-12-23 01:51:08 -05:00
comfyanonymous
56bc64f351
Comment out some useless code.
2024-12-22 23:51:14 -05:00
zhangp365
f7d83b72e0
fixed a bug in ldm/pixart/blocks.py ( #6158 )
2024-12-22 23:44:20 -05:00
comfyanonymous
80f07952d2
Fix lowvram issue with ltxv vae.
2024-12-22 23:20:17 -05:00
patientx
757335d901
Update supported_models.py
2024-12-23 02:54:49 +03:00
patientx
713eca2176
Update supported_models.py
2024-12-23 02:32:50 +03:00
patientx
e9d8cad2f0
Merge branch 'comfyanonymous:master' into master
2024-12-22 16:56:29 +03:00
comfyanonymous
57f330caf9
Relax minimum ratio of weights loaded in memory on nvidia.
...
This should make it possible to do higher res images/longer videos by
further offloading weights to CPU memory.
Please report an issue if this slows down things on your system.
2024-12-22 03:06:37 -05:00
patientx
88ae56dcf9
Merge branch 'comfyanonymous:master' into master
2024-12-21 15:52:28 +03:00
comfyanonymous
da13b6b827
Get rid of meshgrid warning.
2024-12-20 18:02:12 -05:00
comfyanonymous
c86cd58573
Remove useless code.
2024-12-20 17:50:03 -05:00
comfyanonymous
b5fe39211a
Remove some useless code.
2024-12-20 17:43:50 -05:00
patientx
4d64ade41f
Merge branch 'comfyanonymous:master' into master
2024-12-21 01:30:32 +03:00
comfyanonymous
e946667216
Some fixes/cleanups to pixart code.
...
Commented out the masking related code because it is never used in this
implementation.
2024-12-20 17:10:52 -05:00
patientx
37fc9a3ff2
Merge branch 'comfyanonymous:master' into master
2024-12-21 00:37:40 +03:00
Chenlei Hu
d7969cb070
Replace print with logging ( #6138 )
...
* Replace print with logging
* nit
* nit
* nit
* nit
* nit
* nit
2024-12-20 16:24:55 -05:00
City
bddb02660c
Add PixArt model support ( #6055 )
...
* PixArt initial version
* PixArt Diffusers convert logic
* pos_emb and interpolation logic
* Reduce duplicate code
* Formatting
* Use optimized attention
* Edit empty token logic
* Basic PixArt LoRA support
* Fix aspect ratio logic
* PixArtAlpha text encode with conds
* Use same detection key logic for PixArt diffusers
2024-12-20 15:25:00 -05:00
patientx
07ea41ecc1
Merge branch 'comfyanonymous:master' into master
2024-12-20 13:06:18 +03:00
comfyanonymous
418eb7062d
Support new LTXV VAE.
2024-12-20 04:38:29 -05:00
patientx
ebf13dfe56
Merge branch 'comfyanonymous:master' into master
2024-12-20 10:05:56 +03:00
comfyanonymous
cac68ca813
Fix some more video tiled encode issues.
...
The downscale_ratio formula for the temporal had issues with some frame
numbers.
2024-12-19 23:14:03 -05:00
comfyanonymous
52c1d933b2
Fix tiled hunyuan video VAE encode issue.
...
Some shapes like 1024x1024 with tile_size 256 and overlap 64 had issues.
2024-12-19 22:55:15 -05:00
comfyanonymous
2dda7c11a3
More proper fix for the memory issue.
2024-12-19 16:21:56 -05:00
comfyanonymous
3ad3248ad7
Fix lowvram bug when using a model multiple times in a row.
...
The memory system would load an extra 64MB each time until either the
model was completely in memory or OOM.
2024-12-19 16:04:56 -05:00
patientx
778005af3d
Merge branch 'comfyanonymous:master' into master
2024-12-19 14:51:33 +03:00
comfyanonymous
c441048a4f
Make VAE Encode tiled node work with video VAE.
2024-12-19 05:31:39 -05:00
comfyanonymous
9f4b181ab3
Add fast previews for hunyuan video.
2024-12-18 18:24:23 -05:00
comfyanonymous
cbbf077593
Small optimizations.
2024-12-18 18:23:28 -05:00
patientx
43a0204b07
Merge branch 'comfyanonymous:master' into master
2024-12-18 15:17:15 +03:00
comfyanonymous
ff2ff02168
Support old diffusion-pipe hunyuan video loras.
2024-12-18 06:23:54 -05:00
patientx
947aba46c3
Merge branch 'comfyanonymous:master' into master
2024-12-18 12:33:32 +03:00
comfyanonymous
4c5c4ddeda
Fix regression in VAE code on old pytorch versions.
2024-12-18 03:08:28 -05:00
patientx
c062723ca5
Merge branch 'comfyanonymous:master' into master
2024-12-18 10:16:43 +03:00
comfyanonymous
37e5390f5f
Add: --use-sage-attention to enable SageAttention.
...
You need to have the library installed first.
2024-12-18 01:56:10 -05:00
comfyanonymous
a4f59bc65e
Pick attention implementation based on device in llama code.
2024-12-18 01:30:20 -05:00
patientx
0e5fa013b2
Merge branch 'comfyanonymous:master' into master
2024-12-18 00:43:24 +03:00
comfyanonymous
ca457f7ba1
Properly tokenize the template for hunyuan video.
2024-12-17 16:22:02 -05:00
comfyanonymous
cd6f615038
Fix tiled vae not working with some shapes.
2024-12-17 16:22:02 -05:00
patientx
4ace4e9ecb
Merge branch 'comfyanonymous:master' into master
2024-12-17 19:55:56 +03:00
comfyanonymous
e4e1bff605
Support diffusion-pipe hunyuan video lora format.
2024-12-17 07:14:21 -05:00
patientx
dc574cdc47
Merge branch 'comfyanonymous:master' into master
2024-12-17 13:57:30 +03:00
comfyanonymous
d6656b0c0c
Support llama hunyuan video text encoder in scaled fp8 format.
2024-12-17 04:19:22 -05:00
comfyanonymous
f4cdedea62
Fix regression with ltxv VAE.
2024-12-17 02:17:31 -05:00
comfyanonymous
39b1fc4ccc
Adjust used dtypes for hunyuan video VAE and diffusion model.
2024-12-16 23:31:10 -05:00
comfyanonymous
bda1482a27
Basic Hunyuan Video model support.
2024-12-16 19:35:40 -05:00
comfyanonymous
19ee5d9d8b
Don't expand mask when not necessary.
...
Expanding seems to slow down inference.
2024-12-16 18:22:50 -05:00
Raphael Walker
61b50720d0
Add support for attention masking in Flux ( #5942 )
...
* fix attention OOM in xformers
* allow passing attention mask in flux attention
* allow an attn_mask in flux
* attn masks can be done using replace patches instead of a separate dict
* fix return types
* fix return order
* enumerate
* patch the right keys
* arg names
* fix a silly bug
* fix xformers masks
* replace match with if, elif, else
* mask with image_ref_size
* remove unused import
* remove unused import 2
* fix pytorch/xformers attention
This corrects a weird inconsistency with skip_reshape.
It also allows masks of various shapes to be passed, which will be
automtically expanded (in a memory-efficient way) to a size that is
compatible with xformers or pytorch sdpa respectively.
* fix mask shapes
2024-12-16 18:21:17 -05:00
patientx
9704e3e617
Merge branch 'comfyanonymous:master' into master
2024-12-14 14:24:39 +03:00
comfyanonymous
e83063bf24
Support conv3d in PatchEmbed.
2024-12-14 05:46:04 -05:00
patientx
fd2eeb5e30
Merge branch 'comfyanonymous:master' into master
2024-12-13 16:09:33 +03:00
comfyanonymous
4e14032c02
Make pad_to_patch_size function work on multi dim.
2024-12-13 07:22:05 -05:00
patientx
3218ed8559
Merge branch 'comfyanonymous:master' into master
2024-12-13 11:21:47 +03:00
Chenlei Hu
563291ee51
Enforce all pyflake lint rules ( #6033 )
...
* Enforce F821 undefined-name
* Enforce all pyflake lint rules
2024-12-12 19:29:37 -05:00
Chenlei Hu
2cddbf0821
Lint and fix undefined names (1/N) ( #6028 )
2024-12-12 18:55:26 -05:00
Chenlei Hu
60749f345d
Lint and fix undefined names (3/N) ( #6030 )
2024-12-12 18:49:40 -05:00
Chenlei Hu
d9d7f3c619
Lint all unused variables ( #5989 )
...
* Enable F841
* Autofix
* Remove all unused variable assignment
2024-12-12 17:59:16 -05:00
patientx
5d059779d3
Merge branch 'comfyanonymous:master' into master
2024-12-12 15:26:42 +03:00
comfyanonymous
fd5dfb812c
Set initial load devices for te and model to mps device on mac.
2024-12-12 06:00:31 -05:00
patientx
0759f96414
Merge branch 'comfyanonymous:master' into master
2024-12-11 23:05:47 +03:00
