wangbo/ComfyUI
1
0
Fork 0
mirror of https://github.com/comfyanonymous/ComfyUI.git synced 2026-01-23 04:40:15 +08:00
Code Issues Actions 15 Packages Projects Releases Wiki Activity
5,479 Commits 67 Branches 111 Tags 1.3 GiB
397fb62df5
Commit Graph

2548 Commits

Author SHA1 Message Date
Benjamin Berman
480b59b913 Migrate to pure pyproject.toml and uv for significantly faster and correct installation, add model paths for flux all in ones, and fix PathsList bug 2025-04-24 10:01:36 -07:00
thot experiment
e2eed9eb9b
throw away alpha channel in clip vision preprocessor (#7769) 
saves users having to explicitly discard the channel
 2025-04-23 21:28:36 -04:00
comfyanonymous
552615235d
Fix for dino lowvram. (#7748) 2025-04-23 04:12:52 -04:00
Robin Huang
0738e4ea5d
[API nodes] Add backbone for supporting api nodes in ComfyUI (#7745) 
* Add Ideogram generate node.

* Add staging api.

* COMFY_API_NODE_NAME node property

* switch to boolean flag and use original node name for id

* add optional to type

* Add API_NODE and common error for missing auth token (#5)

* Add Minimax Video Generation + Async Task queue polling example (#6)

* [Minimax] Show video preview and embed workflow in ouput (#7)

* [API Nodes] Send empty request body instead of empty dictionary. (#8)

* Fixed: removed function from rebase.

* Add pydantic.

* Remove uv.lock

* Remove polling operations.

* Update stubs workflow.

* Remove polling comments.

* Update stubs.

* Use pydantic v2.

* Use pydantic v2.

* Add basic OpenAITextToImage node

* Add.

* convert image to tensor.

* Improve types.

* Ruff.

* Push tests.

* Handle multi-form data.

- Don't set content-type for multi-part/form
- Use data field instead of JSON

* Change to api.comfy.org

* Handle error code 409.

* Remove nodes.

---------

Co-authored-by: bymyself <cbyrne@comfy.org>
Co-authored-by: Yoland Y <4950057+yoland68@users.noreply.github.com>
 2025-04-23 02:18:08 -04:00
comfyanonymous
2d6805ce57
Add option for using fp8_e8m0fnu for model weights. (#7733) 
Seems to break every model I have tried but worth testing?
 2025-04-22 06:17:38 -04:00
Kohaku-Blueleaf
a8f63c0d5b
Support dora_scale on both axis (#7727) 2025-04-22 05:01:27 -04:00
Kohaku-Blueleaf
966c43ce26
Add OFT/BOFT algorithm in weight adapter (#7725) 2025-04-22 04:59:47 -04:00
comfyanonymous
3ab231f01f
Fix issue with WAN VACE implementation. (#7724) 2025-04-21 23:36:12 -04:00
Kohaku-Blueleaf
1f3fba2af5
Unified Weight Adapter system for better maintainability and future feature of Lora system (#7540) 2025-04-21 20:15:32 -04:00
comfyanonymous
5d0d4ee98a
Add strength control for vace. (#7717) 2025-04-21 19:36:20 -04:00
doctorpangloss
17b14110ab Update to latest ComfyUI 2025-04-21 14:11:56 -07:00
doctorpangloss
5823497d55 Merge branch 'master' of github.com:comfyanonymous/ComfyUI 2025-04-21 13:14:36 -07:00
filtered
5d51794607
Add node type hint for socketless option (#7714) 
* Add node type hint for socketless option

* nit - Doc
 2025-04-21 16:13:00 -04:00
comfyanonymous
ce22f687cc
Support for WAN VACE preview model. (#7711) 
* Support for WAN VACE preview model.

* Remove print.
 2025-04-21 14:40:29 -04:00
comfyanonymous
2c735c13b4
Slightly better fix for #7687 2025-04-20 11:33:27 -04:00
comfyanonymous
fd27494441 Use empty t5 of size 128 for hidream, seems to give closer results. 2025-04-19 19:49:40 -04:00
power88
f43e1d7f41
Hidream: Allow loading hidream text encoders in CLIPLoader and DualCLIPLoader (#7676) 
* Hidream: Allow partial loading text encoders

* reformat code for ruff check.
 2025-04-19 19:47:30 -04:00
comfyanonymous
636d4bfb89 Fix hard crash when the spiece tokenizer path is bad. 2025-04-19 15:55:43 -04:00
comfyanonymous
dbcfd092a2 Set default context_img_len to 257 2025-04-17 12:42:34 -04:00
comfyanonymous
c14429940f Support loading WAN FLF model. 2025-04-17 12:04:48 -04:00
comfyanonymous
0d720e4367 Don't hardcode length of context_img in wan code. 2025-04-17 06:25:39 -04:00
comfyanonymous
1fc00ba4b6 Make hidream work with any latent resolution. 2025-04-16 18:34:14 -04:00
comfyanonymous
9899d187b1 Limit T5 to 128 tokens for HiDream: #7620 2025-04-16 18:07:55 -04:00
comfyanonymous
f00f340a56 Reuse code from flux model. 2025-04-16 17:43:55 -04:00
Chenlei Hu
cce1d9145e
[Type] Mark input options NotRequired (#7614) 2025-04-16 15:41:00 -04:00
comfyanonymous
b4dc03ad76 Fix issue on old torch. 2025-04-16 04:53:56 -04:00
comfyanonymous
9ad792f927 Basic support for hidream i1 model. 2025-04-15 17:35:05 -04:00
comfyanonymous
6fc5dbd52a Cleanup. 2025-04-15 12:13:28 -04:00
comfyanonymous
3e8155f7a3 More flexible long clip support. 
Add clip g long clip support.

Text encoder refactor.

Support llama models with different vocab sizes.
 2025-04-15 10:32:21 -04:00
comfyanonymous
8a438115fb add RMSNorm to comfy.ops 2025-04-14 18:00:33 -04:00
chaObserv
e51d9ba5fc
Add SEEDS (stage 2 & 3 DP) sampler (#7580) 
* Add seeds stage 2 & 3 (DP) sampler

* Change the name to SEEDS in comment
 2025-04-12 18:36:08 -04:00
catboxanon
1714a4c158
Add CublasOps support (#7574) 
* CublasOps support

* Guard CublasOps behind --fast arg
 2025-04-12 18:29:15 -04:00
Chargeuk
ed945a1790
Dependency Aware Node Caching for low RAM/VRAM machines (#7509) 
* add dependency aware cache that removed a cached node as soon as all of its decendents have executed. This allows users with lower RAM to run workflows they would otherwise not be able to run. The downside is that every workflow will fully run each time even if no nodes have changed.

* remove test code

* tidy code
 2025-04-11 06:55:51 -04:00
Chenlei Hu
98bdca4cb2
Deprecate InputTypeOptions.defaultInput (#7551) 
* Deprecate InputTypeOptions.defaultInput

* nit

* nit
 2025-04-10 06:57:06 -04:00
Jedrzej Kosinski
e346d8584e
Add prepare_sampling wrapper allowing custom nodes to more accurately report noise_shape (#7500) 2025-04-09 09:43:35 -04:00
comfyanonymous
70d7242e57 Support the wan fun reward loras. 2025-04-07 05:01:47 -04:00
comfyanonymous
3bfe4e5276 Support 512 siglip model. 2025-04-05 07:01:01 -04:00
Raphael Walker
89e4ea0175
Add activations_shape info in UNet models (#7482) 
* Add activations_shape info in UNet models

* activations_shape should be a list
 2025-04-04 21:27:54 -04:00
comfyanonymous
3a100b9a55 Disable partial offloading of audio VAE. 2025-04-04 21:24:56 -04:00
doctorpangloss
ffc1912eff Fix issues with tests 2025-04-04 08:27:33 -07:00
BiologicalExplosion
2222cf67fd
MLU memory optimization (#7470) 
Co-authored-by: huzhan <huzhan@cambricon.com>
 2025-04-02 19:24:04 -04:00
BVH
301e26b131
Add option to store TE in bf16 (#7461) 2025-04-01 13:48:53 -04:00
comfyanonymous
a3100c8452 Remove useless code. 2025-03-29 20:12:56 -04:00
doctorpangloss
040a324346 Merge branch 'master' of github.com:comfyanonymous/ComfyUI 2025-03-29 15:57:24 -07:00
comfyanonymous
2d17d8910c Don't error if wan concat image has extra channels. 2025-03-28 08:49:29 -04:00
comfyanonymous
0a1f8869c9 Add WanFunInpaintToVideo node for the Wan fun inpaint models. 2025-03-27 11:13:27 -04:00
comfyanonymous
3661c833bc Support the WAN 2.1 fun control models. 
Use the new WanFunControlToVideo node.
 2025-03-26 19:54:54 -04:00
comfyanonymous
8edc1f44c1 Support more float8 types. 2025-03-25 05:23:49 -04:00
comfyanonymous
e471c726e5 Fallback to pytorch attention if sage attention fails. 2025-03-22 15:45:56 -04:00
comfyanonymous
d9fa9d307f Automatically set the right sampling type for lotus. 2025-03-21 14:19:37 -04:00
thot experiment
83e839a89b
Native LotusD Implementation (#7125) 
* draft pass at a native comfy implementation of Lotus-D depth and normal est

* fix model_sampling kludges

* fix ruff

---------

Co-authored-by: comfyanonymous <121283862+comfyanonymous@users.noreply.github.com>
 2025-03-21 14:04:15 -04:00
comfyanonymous
3872b43d4b A few fixes for the hunyuan3d models. 2025-03-20 04:52:31 -04:00
comfyanonymous
32ca0805b7 Fix orientation of hunyuan 3d model. 2025-03-19 19:55:24 -04:00
comfyanonymous
11f1b41bab Initial Hunyuan3Dv2 implementation. 
Supports the multiview, mini, turbo models and VAEs.
 2025-03-19 16:52:58 -04:00
comfyanonymous
3b19fc76e3 Allow disabling pe in flux code for some other models. 2025-03-18 05:09:25 -04:00
comfyanonymous
50614f1b79 Fix regression with clip vision. 2025-03-17 13:56:11 -04:00
comfyanonymous
6dc7b0bfe3 Add support for giant dinov2 image encoder. 2025-03-17 05:53:54 -04:00
comfyanonymous
e8e990d6b8 Cleanup code. 2025-03-16 06:29:12 -04:00
Jedrzej Kosinski
2e24a15905
Call unpatch_hooks at the start of ModelPatcher.partially_unload (#7253) 
* Call unpatch_hooks at the start of ModelPatcher.partially_unload

* Only call unpatch_hooks in partially_unload if lowvram is possible
 2025-03-16 06:02:45 -04:00
chaObserv
fd5297131f
Guard the edge cases of noise term in er_sde (#7265) 2025-03-16 06:02:25 -04:00
comfyanonymous
55a1b09ddc Allow loading diffusion model files with the "Load Checkpoint" node. 2025-03-15 08:27:49 -04:00
comfyanonymous
3c3988df45 Show a better error message if the VAE is invalid. 2025-03-15 08:26:36 -04:00
comfyanonymous
a2448fc527 Remove useless code. 2025-03-14 18:10:37 -04:00
comfyanonymous
6a0daa79b6 Make the SkipLayerGuidanceDIT node work on WAN. 2025-03-14 10:55:19 -04:00
FeepingCreature
9c98c6358b
Tolerate missing @torch.library.custom_op (#7234) 
This can happen on Pytorch versions older than 2.4.
 2025-03-14 09:51:26 -04:00
FeepingCreature
7aceb9f91c
Add --use-flash-attention flag. (#7223) 
* Add --use-flash-attention flag.
This is useful on AMD systems, as FA builds are still 10% faster than Pytorch cross-attention.
 2025-03-14 03:22:41 -04:00
comfyanonymous
35504e2f93 Fix. 2025-03-13 15:03:18 -04:00
comfyanonymous
299436cfed Print mac version. 2025-03-13 10:05:40 -04:00
Chenlei Hu
9b6cd9b874
[NodeDef] Add documentation on multi_select input option (#7212) 2025-03-12 17:29:39 -04:00
chaObserv
3fc688aebd
Ensure the extra_args in dpmpp sde series (#7204) 2025-03-12 17:28:59 -04:00
chaObserv
01015bff16
Add er_sde sampler (#7187) 2025-03-12 02:42:37 -04:00
comfyanonymous
ca8efab79f Support control loras on Wan. 2025-03-10 17:23:13 -04:00
comfyanonymous
9aac21f894 Fix issues with new hunyuan img2vid model and bumb version to v0.3.26 2025-03-09 05:07:22 -04:00
Jedrzej Kosinski
528d1b3563
When cached_hook_patches contain weights for hooks, only use hook_backup for unused keys (#7067) 2025-03-09 04:26:31 -04:00
comfyanonymous
7395b0c0d1 Support new hunyuan video i2v model. 
Use the new "v2 (replace)" guidance type in HunyuanImageToVideo and set
image_interleave to 4 on the "Text Encode Hunyuan Video" node.
 2025-03-08 20:34:47 -05:00
comfyanonymous
0952569493 Fix stable cascade VAE on some lowvram machines. 2025-03-08 20:24:04 -05:00
doctorpangloss
e068c4c920 Improved support for Wan features 
- Wan and Cosmos prompt upsamplers
 - Fixed torch.compile issues
 - Known models added
 - Cosmos, Wan and Hunyuan Video resolutions now supported by Fit Image
   to Diffusion Size.
 - Better error messages for Ampere and Triton interactions
 2025-03-08 15:12:28 -08:00
comfyanonymous
be4e760648 Add an image_interleave option to the Hunyuan image to video encode node. 
See the tooltip for what it does.
 2025-03-07 19:56:26 -05:00
comfyanonymous
11b1f27cb1 Set WAN default compute dtype to fp16. 2025-03-07 04:52:36 -05:00
comfyanonymous
70e15fd743 No need for scale_input when fp8 matrix mult is disabled. 2025-03-07 04:49:20 -05:00
comfyanonymous
e1474150de Support fp8_scaled diffusion models that don't use fp8 matrix mult. 2025-03-07 04:39:21 -05:00
JettHu
e62d72e8ca
Typo in node_typing.py (#7092) 2025-03-06 15:24:04 -05:00
comfyanonymous
dfa36e6855 Fix some things breaking when embeddings fail to apply. 2025-03-06 13:31:55 -05:00
doctorpangloss
8b7c247be3 Known WAN and Hunyuan I2V checkpoints 2025-03-06 07:49:17 -08:00
doctorpangloss
0f85e7d2b0 Merge branch 'master' of github.com:comfyanonymous/ComfyUI 2025-03-06 07:41:41 -08:00
doctorpangloss
83948cafd1 WAN 2.1 support 2025-03-06 07:32:04 -08:00
comfyanonymous
29a70ca101 Support HunyuanVideo image to video model. 2025-03-06 03:07:15 -05:00
comfyanonymous
0bef826a98 Support llava clip vision model. 2025-03-06 00:24:43 -05:00
doctorpangloss
3c82be86d1 Merge branch 'master' of github.com:comfyanonymous/ComfyUI 2025-03-05 14:38:50 -08:00
comfyanonymous
85ef295069 Make applying embeddings more efficient. 
Adding new tokens no longer makes a whole copy of the embeddings weight
which can be massive on certain models.
 2025-03-05 17:34:38 -05:00
doctorpangloss
810c303210 Remove web directory from package 2025-03-05 14:05:18 -08:00
Chenlei Hu
5d84607bf3
Add type hint for FileLocator (#6968) 
* Add type hint for FileLocator

* nit
 2025-03-05 15:35:26 -05:00
Silver
c1909f350f
Better argument handling of front-end-root (#7043) 
* Better argument handling of front-end-root

Improves handling of front-end-root launch argument. Several instances where users have set it and ComfyUI launches as normal and completely disregards the launch arg which doesn't make sense. Better to indicate to user that something is incorrect.

* Removed unused import

There was no real reason to use "Optional" typing in ther front-end-root argument.
 2025-03-05 15:34:22 -05:00
Chenlei Hu
52b3469606
[NodeDef] Explicitly add control_after_generate to seed/noise_seed (#7059) 
* [NodeDef] Explicitly add control_after_generate to seed/noise_seed

* Update comfy/comfy_types/node_typing.py

Co-authored-by: filtered <176114999+webfiltered@users.noreply.github.com>

---------

Co-authored-by: filtered <176114999+webfiltered@users.noreply.github.com>
 2025-03-05 15:33:23 -05:00
doctorpangloss
8d2d613f95 Move language extensions in order to support packagized frontend 2025-03-05 10:39:31 -08:00
comfyanonymous
369b079ff6 Fix lowvram issue with ltxv vae. 2025-03-05 05:26:08 -05:00
comfyanonymous
9c9a7f012a Adjust ltxv memory factor. 2025-03-05 05:16:05 -05:00
comfyanonymous
93fedd92fe Support LTXV 0.9.5. 
Credits: Lightricks team.
 2025-03-05 00:13:49 -05:00
comfyanonymous
65042f7d39 Make it easier to set a custom template for hunyuan video. 2025-03-04 09:26:05 -05:00
comfyanonymous
7c7c70c400 Refactor skyreels i2v code. 2025-03-04 00:15:45 -05:00
doctorpangloss
d82261485f Prompt upsampling, better torch.compile support for language models 2025-03-03 18:36:47 -08:00
doctorpangloss
c6111fae7d Fix Pixtral 12b compatibility 2025-03-03 13:07:36 -08:00
comfyanonymous
f86c724ef2 Temporal area composition. 
New ConditioningSetAreaPercentageVideo node.
 2025-03-03 06:50:31 -05:00
comfyanonymous
9af6320ec9 Make 2d area composition nodes work on video models. 2025-03-02 08:19:16 -05:00
comfyanonymous
4dc6709307 Rename argument in last commit and document the options. 2025-03-01 02:43:49 -05:00
Chenlei Hu
4d55f16ae8
Use enum list for --fast options (#7024) 2025-03-01 02:37:35 -05:00
comfyanonymous
cf0b549d48 --fast now takes a number as argument to indicate how fast you want it. 
The idea is that you can indicate how much quality vs speed you want.

At the moment:

--fast 2 enables fp16 accumulation if your pytorch supports it.
--fast 5 enables fp8 matrix mult on fp8 models and the optimization above.

--fast without a number enables all optimizations.
 2025-02-28 02:48:20 -05:00
comfyanonymous
eb4543474b Use fp16 for intermediate for fp8 weights with --fast if supported. 2025-02-28 02:17:50 -05:00
comfyanonymous
1804397952 Use fp16 if checkpoint weights are fp16 and the model supports it. 2025-02-27 16:39:57 -05:00
comfyanonymous
f4dac8ab6f Wan code small cleanup. 2025-02-27 07:22:42 -05:00
BiologicalExplosion
89253e9fe5
Support Cambricon MLU (#6964) 
Co-authored-by: huzhan <huzhan@cambricon.com>
 2025-02-26 20:45:13 -05:00
comfyanonymous
3ea3bc8546 Fix wan issues when prompt length is long. 2025-02-26 20:34:02 -05:00
comfyanonymous
0270a0b41c Reduce artifacts on Wan by doing the patch embedding in fp32. 2025-02-26 16:59:26 -05:00
comfyanonymous
c37f15f98e Add fast preview support for Wan models. 2025-02-26 08:56:23 -05:00
comfyanonymous
4bca7367f3 Don't try to use clip_fea on t2v model. 2025-02-26 08:38:09 -05:00
comfyanonymous
b6fefe686b Better wan memory estimation. 2025-02-26 07:51:22 -05:00
comfyanonymous
fa62287f1f More code reuse in wan. 
Fix bug when changing the compute dtype on wan.
 2025-02-26 05:22:29 -05:00
comfyanonymous
0844998db3 Slightly better wan i2v mask implementation. 2025-02-26 03:49:50 -05:00
comfyanonymous
4ced06b879 WIP support for Wan I2V model. 2025-02-26 01:49:43 -05:00
comfyanonymous
cb06e9669b Wan seems to work with fp16. 2025-02-25 21:37:12 -05:00
comfyanonymous
9a66bb972d Make wan work with all latent resolutions. 
Cleanup some code.
 2025-02-25 19:56:04 -05:00
comfyanonymous
ea0f939df3 Fix issue with wan and other attention implementations. 2025-02-25 19:13:39 -05:00
comfyanonymous
f37551c1d2 Change wan rope implementation to the flux one. 
Should be more compatible.
 2025-02-25 19:11:14 -05:00
comfyanonymous
63023011b9 WIP support for Wan t2v model. 2025-02-25 17:20:35 -05:00
comfyanonymous
f40076096e Cleanup some lumina te code. 2025-02-25 04:10:26 -05:00
doctorpangloss
048746f58b Update to 0.3.15 and improve models 
- Cosmos now fully tested
 - Preliminary support for essential Cosmos prompt "upsampler"
 - Lumina tests
 - Tweaks to language and image resizing nodes
 - Fix for #31 all the samplers are now present again
 2025-02-24 21:27:15 -08:00
doctorpangloss
693038738a Merge branch 'master' of github.com:comfyanonymous/ComfyUI 2025-02-24 09:39:26 -08:00
comfyanonymous
96d891cb94 Speedup on some models by not upcasting bfloat16 to float32 on mac. 2025-02-24 05:41:32 -05:00
comfyanonymous
ace899e71a Prioritize fp16 compute when using allow_fp16_accumulation 2025-02-23 04:45:54 -05:00
comfyanonymous
aff16532d4 Remove some useless code. 2025-02-22 04:45:14 -05:00
comfyanonymous
072db3bea6 Assume the mac black image bug won't be fixed before v16. 2025-02-21 20:24:07 -05:00
comfyanonymous
a6deca6d9a Latest mac still has the black image bug. 2025-02-21 20:14:30 -05:00
comfyanonymous
41c30e92e7 Let all model memory be offloaded on nvidia. 2025-02-21 06:32:21 -05:00
comfyanonymous
12da6ef581 Apparently directml supports fp16. 2025-02-20 09:30:24 -05:00
Silver
c5be423d6b
Fix link pointing to non-exisiting docs (#6891) 
* Fix link pointing to non-exisiting docs

The current link is pointing to a path that does not exist any longer.
I changed it to point to the currect correct path for custom nodes datatypes.

* Update node_typing.py
 2025-02-20 07:07:07 -05:00
maedtb
5715be2ca9
Fix Hunyuan unet config detection for some models. (#6877) 
The change to support 32 channel hunyuan models is missing the `key_prefix` on the key.

This addresses a complain in the comments of acc152b674.
 2025-02-19 07:14:45 -05:00
Benjamin Berman
83ae94b96c Fix absolute import 2025-02-18 20:09:56 -08:00
Benjamin Berman
1e74a4cf08 Fix absolute imports, fix linting issue with dataclass 2025-02-18 19:59:09 -08:00
Benjamin Berman
ffc6a7fd38 Use spawn multiprocessing context to fix Linux ProcessPool issues 2025-02-18 19:46:57 -08:00
doctorpangloss
e65faca817 Distributed setup now defaults to panicking when out of memory now, to facilitate graceful recovery 2025-02-18 15:07:02 -08:00
bymyself
afc85cdeb6
Add Load Image Output node (#6790) 
* add LoadImageOutput node

* add route for input/output/temp files

* update node_typing.py

* use literal type for image_folder field

* mark node as beta
 2025-02-18 17:53:01 -05:00
doctorpangloss
3ddec8ae90 Better support for process pool executors 
- --panics-when=torch.cuda.OutOfMemory will now correctly panic and
   exit the worker, giving it time to reply that the execution failed
   and better dealing with irrecoverable out-of-memory errors
 - --executor-factory=ProcessPoolExecutor will use a process instead of
   a thread to execute comfyui workflows when using the worker. When
   this process panics and exits, it will be correctly replaced, making
   a more robust worker
 2025-02-18 14:37:20 -08:00
Jukka Seppänen
acc152b674
Support loading and using SkyReels-V1-Hunyuan-I2V (#6862) 
* Support SkyReels-V1-Hunyuan-I2V

* VAE scaling

* Fix T2V

oops

* Proper latent scaling
 2025-02-18 17:06:54 -05:00
doctorpangloss
684d180446 Users can now configure their workers to panic if they have out of memory exceptions, which occur due to complex failures in custom nodes 2025-02-18 10:57:23 -08:00
comfyanonymous
b07258cef2 Fix typo. 
Let me know if this slows things down on 2000 series and below.
 2025-02-18 07:28:33 -05:00
comfyanonymous
31e54b7052 Improve AMD arch detection. 2025-02-17 04:53:40 -05:00
comfyanonymous
8c0bae50c3 bf16 manual cast works on old AMD. 2025-02-17 04:42:40 -05:00
comfyanonymous
530412cb9d Refactor torch version checks to be more future proof. 2025-02-17 04:36:45 -05:00
doctorpangloss
d04288ce8d ImagePadForOutpaint now correctly returns a MaskBatch 2025-02-16 15:39:36 -08:00
comfyanonymous
e2919d38b4 Disable bf16 on AMD GPUs that don't support it. 2025-02-16 05:46:10 -05:00
doctorpangloss
0ca30c3c87 export_custom_nodes now handles abstract base classes better 2025-02-14 15:36:51 -08:00
doctorpangloss
f4e65590b8 Fix subfolder being None when images are viewed 2025-02-14 07:20:58 -08:00
comfyanonymous
1cd6cd6080 Disable pytorch attention in VAE for AMD. 2025-02-14 05:42:14 -05:00
comfyanonymous
d7b4bf21a2 Auto enable mem efficient attention on gfx1100 on pytorch nightly 2.7 
I'm not not sure which arches are supported yet. If you see improvements in
memory usage while using --use-pytorch-cross-attention on your AMD GPU let
me know and I will add it to the list.
 2025-02-14 04:18:14 -05:00
comfyanonymous
019c7029ea Add a way to set a different compute dtype for the model at runtime. 
Currently only works for diffusion models.
 2025-02-13 20:34:03 -05:00
comfyanonymous
8773ccf74d Better memory estimation for ROCm that support mem efficient attention. 
There is no way to check if the card actually supports it so it assumes
that it does if you use --use-pytorch-cross-attention with yours.
 2025-02-13 08:32:36 -05:00
doctorpangloss
31b6b53236 Quality of life improvements 
- export_custom_nodes() finds all the classes that inherit from
   CustomNode and exports them correctly for custom node discovery to
   find
 - regular expressions
 - additional string formatting and parsing nodes
 2025-02-12 14:12:10 -08:00
comfyanonymous
1d5d6586f3 Fix ruff. 2025-02-12 06:49:16 -05:00
zhoufan2956
35740259de
mix_ascend_bf16_infer_err (#6794) 2025-02-12 06:48:11 -05:00
comfyanonymous
ab888e1e0b Add add_weight_wrapper function to model patcher. 
Functions can now easily be added to wrap/modify model weights.
 2025-02-12 05:55:35 -05:00
comfyanonymous
d9f0fcdb0c Cleanup. 2025-02-11 17:17:03 -05:00
HishamC
b124256817
Fix for running via DirectML (#6542) 
* Fix for running via DirectML

Fix DirectML empty image generation issue with Flux1. add CPU fallback for unsupported path. Verified the model works on AMD GPUs

* fix formating

* update casual mask calculation
 2025-02-11 17:11:32 -05:00
comfyanonymous
af4b7c91be Make --force-fp16 actually force the diffusion model to be fp16. 2025-02-11 08:33:09 -05:00
comfyanonymous
4027466c80 Make lumina model work with any latent resolution. 2025-02-10 00:24:20 -05:00
comfyanonymous
095d867147 Remove useless function. 2025-02-09 07:02:57 -05:00
Pam
caeb27c3a5
res_multistep: Fix cfgpp and add ancestral samplers (#6731) 2025-02-08 19:39:58 -05:00
comfyanonymous
3d06e1c555 Make error more clear to user. 2025-02-08 18:57:24 -05:00
catboxanon
43a74c0de1
Allow FP16 accumulation with --fast (#6453) 
Currently only applies to PyTorch nightly releases. (>=20250208)
 2025-02-08 17:00:56 -05:00
comfyanonymous
079eccc92a Don't compress http response by default. 
Remove argument to disable it.

Add new --enable-compress-response-body argument to enable it.
 2025-02-07 03:29:21 -05:00
doctorpangloss
ef74b9fdda More graceful health check handling of this connection not being ready 2025-02-06 11:08:09 -08:00
doctorpangloss
5b3eb2e51c Fix torch.zeroes error 2025-02-06 09:00:10 -08:00
comfyanonymous
14880e6dba Remove some useless code. 2025-02-06 05:00:37 -05:00
doctorpangloss
3f1f427ff4 Distinct Seed and Seed64 input specs. numpy only supports 32 bit seeds 2025-02-05 14:08:09 -08:00
doctorpangloss
6ab1aa1e8a Improving MLLM/VLLM support and fixing bugs 
- fix #29 str(model) no longer raises exceptions like with
   HyVideoModelLoader
 - don't try to format CUDA tensors because that can sometimes raise
   exceptions
 - cudaAllocAsync has been disabled for now due to 2.6.0 bugs
 - improve florence2 support
 - add support for paligemma 2. This requires the fix for transformers
   that is currently staged in another repo, install with
   `uv pip install --no-deps "transformers@git+https://github.com/zucchini-nlp/transformers.git#branch=paligemma-fix-kwargs"`
 - triton has been updated
 - fix missing __init__.py files
 2025-02-05 14:02:28 -08:00
comfyanonymous
37cd448529 Set the shift for Lumina back to 6. 2025-02-05 14:49:52 -05:00
comfyanonymous
94f21f9301 Upcasting rope to fp32 seems to make no difference in this model. 2025-02-05 04:32:47 -05:00
comfyanonymous
60653004e5 Use regular numbers for rope in lumina model. 2025-02-05 04:17:25 -05:00
comfyanonymous
a57d635c5f Fix lumina 2 batches. 2025-02-04 21:48:11 -05:00
doctorpangloss
dcac115f68 Revert "Update logging when models are loaded" 
This reverts commit 0d15a091c2.
 2025-02-04 15:18:00 -08:00
doctorpangloss
80db9a8e25 Florence2 2025-02-04 15:17:14 -08:00
doctorpangloss
1a24ceef79 Updates for torch 2.6.0, prepare Anthropic nodes, accept multiple logging levels 2025-02-04 09:27:18 -08:00
Rafał Leszko
0d15a091c2
Update logging when models are loaded 
The "Loaded " log was logged even if no model were actually loaded into VRAM
 2025-02-04 14:44:12 +01:00
comfyanonymous
8ac2dddeed Lower the default shift of lumina to reduce artifacts. 2025-02-04 06:50:37 -05:00
comfyanonymous
3e880ac709 Fix on python 3.9 2025-02-04 04:20:56 -05:00
comfyanonymous
e5ea112a90 Support Lumina 2 model. 2025-02-04 04:16:30 -05:00
comfyanonymous
44e19a28d3 Use maximum negative value instead of -inf for masks in text encoders. 
This is probably more correct.
 2025-02-02 09:46:00 -05:00
Dr.Lt.Data
0a0df5f136
better guide message for sageattention (#6634) 2025-02-02 09:26:47 -05:00
KarryCharon
24d6871e47
add disable-compres-response-body cli args; add compress middleware; (#6672) 2025-02-02 09:24:55 -05:00
comfyanonymous
9e1d301129 Only use stable cascade lora format with cascade model. 2025-02-01 06:35:22 -05:00
doctorpangloss
1488f2c59b Logger should check attributes on current sys.stdout, which may have been overwritten 2025-01-31 10:58:11 -08:00
comfyanonymous
8d8dc9a262 Allow batch of different sigmas when noise scaling. 2025-01-30 06:49:52 -05:00
filtered
222f48c0f2
Allow changing folder_paths.base_path via command line argument. (#6600) 
* Reimpl. CLI arg directly inside folder_paths.

* Update tests to use CLI arg mocking.

* Revert last-minute refactor.

* Fix test state polution.
 2025-01-29 08:06:28 -05:00
doctorpangloss
95a12f42e2 Fix pylint errors (they were real, as they usually are) 2025-01-28 17:16:15 -08:00
doctorpangloss
d24098cd9b Fix mask uploads 2025-01-28 16:34:59 -08:00
doctorpangloss
044dff6887 Updates and fixes 
- Update to latest triton
 - Fix huggingface hub automatic downloads
 - Latest transformers may require updating huggingface llava models
 - Compiling flux with fp8 weights is not supported
 2025-01-28 16:22:09 -08:00
doctorpangloss
a3452f6e6a Merge branch 'master' of github.com:comfyanonymous/ComfyUI 2025-01-28 13:45:51 -08:00
comfyanonymous
13fd4d6e45 More friendly error messages for corrupted safetensors files. 2025-01-28 09:41:09 -05:00
comfyanonymous
255edf2246 Lower minimum ratio of loaded weights on Nvidia. 2025-01-27 05:26:51 -05:00
comfyanonymous
67feb05299 Remove redundant code. 2025-01-25 19:04:53 -05:00
comfyanonymous
14ca5f5a10 Remove useless code. 2025-01-24 06:15:54 -05:00
comfyanonymous
96e2a45193 Remove useless code. 2025-01-23 05:56:23 -05:00
Chenlei Hu
dfa2b6d129
Remove unused function lcm in conds.py (#6572) 2025-01-23 05:54:09 -05:00
comfyanonymous
d6bbe8c40f Remove support for python 3.8. 2025-01-22 17:04:30 -05:00
doctorpangloss
e5044799da Fix forward versus backward slash in these params 2025-01-22 13:58:24 -08:00
doctorpangloss
2d4e579503 Fix pylint error 2025-01-22 10:41:23 -08:00
doctorpangloss
b1bcf082af Native Ideogram support 2025-01-22 10:32:04 -08:00
chaObserv
e857dd48b8
Add gradient estimation sampler (#6554) 2025-01-22 05:29:40 -05:00
comfyanonymous
fb2ad645a3 Add FluxDisableGuidance node to disable using the guidance embed. 2025-01-20 14:50:24 -05:00
comfyanonymous
d8a7a32779 Cleanup old TODO. 2025-01-20 03:44:13 -05:00
Sergii Dymchenko
ebf038d4fa
Use torch.special.expm1 (#6388) 
* Use `torch.special.expm1`

This function provides greater precision than `exp(x) - 1` for small values of `x`.

Found with TorchFix https://github.com/pytorch-labs/torchfix/

* Use non-alias
 2025-01-19 04:54:32 -05:00
catboxanon
b1a02131c9
Remove comfy.samplers self-import (#6506) 2025-01-18 17:49:51 -05:00
comfyanonymous
507199d9a8 Uni pc sampler now works with audio and video models. 2025-01-18 05:27:58 -05:00
comfyanonymous
2f3ab40b62 Add warning when using old pytorch versions. 2025-01-17 18:47:27 -05:00
comfyanonymous
0aa2368e46 Fix some cosmos fp8 issues. 2025-01-16 17:45:37 -05:00
doctorpangloss
a9347c6713 Fix pylint error 2025-01-16 14:09:37 -08:00
comfyanonymous
cca96a85ae Fix cosmos VAE failing with videos longer than 121 frames. 2025-01-16 16:30:06 -05:00
doctorpangloss
005459ee77 Release version 0.3.11 2025-01-16 12:40:34 -08:00
doctorpangloss
cf3c96e593 Cosmos support 2025-01-16 12:39:05 -08:00
doctorpangloss
631d9e44c6 Merge branch 'master' of github.com:comfyanonymous/ComfyUI 2025-01-16 09:58:02 -08:00
comfyanonymous
31831e6ef1 Code refactor. 2025-01-16 07:23:54 -05:00
comfyanonymous
88ceb28e20 Tweak hunyuan memory usage factor. 2025-01-16 06:31:03 -05:00
comfyanonymous
23289a6a5c Clean up some debug lines. 2025-01-16 04:24:39 -05:00
comfyanonymous
9d8b6c1f46 More accurate memory estimation for cosmos and hunyuan video. 2025-01-16 03:48:40 -05:00
comfyanonymous
6320d05696 Slightly lower hunyuan video memory usage. 2025-01-16 00:23:01 -05:00
comfyanonymous
25683b5b02 Lower cosmos diffusion model memory usage. 2025-01-15 23:46:42 -05:00
comfyanonymous
4758fb64b9 Lower cosmos VAE memory usage by a bit. 2025-01-15 22:57:52 -05:00
comfyanonymous
008761166f Optimize first attention block in cosmos VAE. 2025-01-15 21:48:46 -05:00
comfyanonymous
cba58fff0b Remove unsafe embedding load for very old pytorch. 2025-01-15 04:32:23 -05:00
comfyanonymous
2feb8d0b77 Force safe loading of files in torch format on pytorch 2.4+ 
If this breaks something for you make an issue.
 2025-01-15 03:50:27 -05:00
Pam
c78a45685d
Rewrite res_multistep sampler and implement res_multistep_cfg_pp sampler. (#6462) 2025-01-14 18:20:06 -05:00
comfyanonymous
3aaabb12d4 Implement Cosmos Image/Video to World (Video) diffusion models. 
Use CosmosImageToVideoLatent to set the input image/video.
 2025-01-14 05:14:10 -05:00
comfyanonymous
1f1c7b7b56 Remove useless code. 2025-01-13 03:52:37 -05:00
comfyanonymous
90f349f93d Add res_multistep sampler from the cosmos code. 
This sampler should work with all models.
 2025-01-12 03:10:07 -05:00
Jedrzej Kosinski
6c9bd11fa3
Hooks Part 2 - TransformerOptionsHook and AdditionalModelsHook (#6377) 
* Add 'sigmas' to transformer_options so that downstream code can know about the full scope of current sampling run, fix Hook Keyframes' guarantee_steps=1 inconsistent behavior with sampling split across different Sampling nodes/sampling runs by referencing 'sigmas'

* Cleaned up hooks.py, refactored Hook.should_register and add_hook_patches to use target_dict instead of target so that more information can be provided about the current execution environment if needed

* Refactor WrapperHook into TransformerOptionsHook, as there is no need to separate out Wrappers/Callbacks/Patches into different hook types (all affect transformer_options)

* Refactored HookGroup to also store a dictionary of hooks separated by hook_type, modified necessary code to no longer need to manually separate out hooks by hook_type

* In inner_sample, change "sigmas" to "sampler_sigmas" in transformer_options to not conflict with the "sigmas" that will overwrite "sigmas" in _calc_cond_batch

* Refactored 'registered' to be HookGroup instead of a list of Hooks, made AddModelsHook operational and compliant with should_register result, moved TransformerOptionsHook handling out of ModelPatcher.register_all_hook_patches, support patches in TransformerOptionsHook properly by casting any patches/wrappers/hooks to proper device at sample time

* Made hook clone code sane, made clear ObjectPatchHook and SetInjectionsHook are not yet operational

* Fix performance of hooks when hooks are appended via Cond Pair Set Props nodes by properly caching between positive and negative conds, make hook_patches_backup behave as intended (in the case that something pre-registers WeightHooks on the ModelPatcher instead of registering it at sample time)

* Filter only registered hooks on self.conds in CFGGuider.sample

* Make hook_scope functional for TransformerOptionsHook

* removed 4 whitespace lines to satisfy Ruff,

* Add a get_injections function to ModelPatcher

* Made TransformerOptionsHook contribute to registered hooks properly, added some doc strings and removed a so-far unused variable

* Rename AddModelsHooks to AdditionalModelsHook, rename SetInjectionsHook to InjectionsHook (not yet implemented, but at least getting the naming figured out)

* Clean up a typehint
 2025-01-11 12:20:23 -05:00
comfyanonymous
ee8a7ab69d Fast latent preview for Cosmos. 2025-01-11 04:41:24 -05:00
comfyanonymous
2ff3104f70 WIP support for Nvidia Cosmos 7B and 14B text to world (video) models. 2025-01-10 09:14:16 -05:00
comfyanonymous
129d8908f7 Add argument to skip the output reshaping in the attention functions. 2025-01-10 06:27:37 -05:00
doctorpangloss
1f69d3ec0b Fix directory creation 2025-01-09 18:19:50 -08:00
comfyanonymous
ff838657fa Cleaner handling of attention mask in ltxv model code. 2025-01-09 07:12:03 -05:00
comfyanonymous
2307ff6746 Improve some logging messages. 2025-01-08 19:05:22 -05:00
comfyanonymous
d0f3752e33 Properly calculate inner dim for t5 model. 
This is required to support some different types of t5 models.
 2025-01-07 17:33:03 -05:00
comfyanonymous
4209edf48d Make a few more samplers deterministic. 2025-01-07 02:12:32 -05:00
Chenlei Hu
d055325783
Document get_attr and get_model_object (#6357) 
* Document get_attr and get_model_object

* Update model_patcher.py

* Update model_patcher.py

* Update model_patcher.py
 2025-01-06 20:12:22 -05:00
comfyanonymous
916d1e14a9 Make ancestral samplers more deterministic. 2025-01-06 03:04:32 -05:00
Jedrzej Kosinski
c496e53519
In inner_sample, change "sigmas" to "sampler_sigmas" in transformer_options to not conflict with the "sigmas" that will overwrite "sigmas" in _calc_cond_batch (#6360) 2025-01-06 01:36:47 -05:00
doctorpangloss
12082d877d Fix linting issues 2025-01-04 14:07:19 -08:00
doctorpangloss
f8a00af0d2 Fix issues with paths 2025-01-04 13:52:32 -08:00
comfyanonymous
d45ebb63f6 Remove old unused function. 2025-01-04 07:20:54 -05:00
comfyanonymous
9e9c8a1c64 Clear cache as often on AMD as Nvidia. 
I think the issue this was working around has been solved.

If you notice that this change slows things down or causes stutters on
your AMD GPU with ROCm on Linux please report it.
 2025-01-02 08:44:16 -05:00
Andrew Kvochko
0f11d60afb
Fix temporal tiling for decoder, remove redundant tiles. (#6306) 
This commit fixes the temporal tile size calculation, and removes
a redundant tile at the end of the range when its elements are
completely covered by the previous tile.

Co-authored-by: Andrew Kvochko <a.kvochko@lightricks.com>
 2025-01-01 16:29:01 -05:00
comfyanonymous
79eea51a1d Fix and enforce all ruff W rules. 2025-01-01 03:08:33 -05:00
blepping
c0338a46a4
Fix unknown sampler error handling in calculate_sigmas function (#6280) 
Modernize calculate_sigmas function
 2024-12-31 17:33:50 -05:00
Jedrzej Kosinski
1c99734e5a
Add missing model_options param (#6296) 2024-12-31 14:46:55 -05:00
filtered
67758f50f3
Fix custom node type-hinting examples (#6281) 
* Fix import in comfy_types doc / sample

* Clarify docstring
 2024-12-31 03:41:09 -05:00
comfyanonymous
b7572b2f87 Fix and enforce no trailing whitespace. 2024-12-31 03:16:37 -05:00
blepping
a90aafafc1
Add kl_optimal scheduler (#6206) 
* Add kl_optimal scheduler

* Rename kl_optimal_schedule to kl_optimal_scheduler to be more consistent
 2024-12-30 05:09:38 -05:00
comfyanonymous
d9b7cfac7e Fix and enforce new lines at the end of files. 2024-12-30 04:14:59 -05:00
Jedrzej Kosinski
3507870535
Add 'sigmas' to transformer_options so that downstream code can know about the full scope of current sampling run, fix Hook Keyframes' guarantee_steps=1 inconsistent behavior with sampling split across different Sampling nodes/sampling runs by referencing 'sigmas' (#6273) 2024-12-30 03:42:49 -05:00
comfyanonymous
a618f768e0 Auto reshape 2d to 3d latent for single image generation on video model. 2024-12-29 02:26:49 -05:00
doctorpangloss
a811336e58 remove unneeded test, make these fields optional 2024-12-28 14:46:15 -08:00
doctorpangloss
9d5a5dd533 Merge branch 'master' of github.com:comfyanonymous/ComfyUI 2024-12-28 14:24:27 -08:00
comfyanonymous
b504bd606d Add ruff rule for empty line with trailing whitespace. 2024-12-28 05:23:08 -05:00
comfyanonymous
d170292594 Remove some trailing white space. 2024-12-27 18:02:30 -05:00
filtered
9cfd185676
Add option to log non-error output to stdout (#6243) 
* nit

* Add option to log non-error output to stdout

- No change to default behaviour
- Adds CLI argument: --log-stdout
- With this arg present, any logging of a level below logging.ERROR will be sent to stdout instead of stderr
 2024-12-27 14:40:05 -05:00
comfyanonymous
4b5bcd8ac4 Closer memory estimation for hunyuan dit model. 2024-12-27 07:37:00 -05:00
comfyanonymous
ceb50b2cbf Closer memory estimation for pixart models. 2024-12-27 07:30:09 -05:00
comfyanonymous
160ca08138 Use python 3.9 in launch test instead of 3.8 
Fix ruff check.
 2024-12-26 20:05:54 -05:00
Huazhong Ji
c4bfdba330
Support ascend npu (#5436) 
* support ascend npu

Co-authored-by: YukMingLaw <lymmm2@163.com>
Co-authored-by: starmountain1997 <guozr1997@hotmail.com>
Co-authored-by: Ginray <ginray0215@gmail.com>
 2024-12-26 19:36:50 -05:00
comfyanonymous
ee9547ba31 Improve temporal VAE Encode (Tiled) math. 2024-12-26 07:18:49 -05:00
doctorpangloss
7655be873c Updates to support Hunyuan Video 2024-12-25 22:39:12 -08:00
comfyanonymous
19a64d6291 Cleanup some mac related code. 2024-12-25 05:32:51 -05:00
comfyanonymous
b486885e08 Disable bfloat16 on older mac. 2024-12-25 05:18:50 -05:00
comfyanonymous
0229228f3f Clean up the VAE dtypes code. 2024-12-25 04:50:34 -05:00
doctorpangloss
0fd407ae87 Merge branch 'master' of github.com:comfyanonymous/ComfyUI 2024-12-24 16:48:03 -08:00
comfyanonymous
99a1fb6027 Make fast fp8 take a bit less peak memory. 2024-12-24 18:05:19 -05:00
comfyanonymous
73e04987f7 Prevent black images in VAE Decode (Tiled) node. 
Overlap should be minimum 1 with tiling 2 for tiled temporal VAE decoding.
 2024-12-24 07:36:30 -05:00
comfyanonymous
5388df784a Add temporal tiling to VAE Encode (Tiled) node. 2024-12-24 07:10:09 -05:00
comfyanonymous
bc6dac4327 Add temporal tiling to VAE Decode (Tiled) node. 
You can now do tiled VAE decoding on the temporal direction for videos.
 2024-12-23 20:03:37 -05:00
comfyanonymous
15564688ed Add a try except block so if torch version is weird it won't crash. 2024-12-23 03:22:48 -05:00
Simon Lui
c6b9c11ef6
Add oneAPI device selector for xpu and some other changes. (#6112) 
* Add oneAPI device selector and some other minor changes.

* Fix device selector variable name.

* Flip minor version check sign.

* Undo changes to README.md.
 2024-12-23 03:18:32 -05:00
comfyanonymous
e44d0ac7f7 Make --novram completely offload weights. 
This flag is mainly used for testing the weight offloading, it shouldn't
actually be used in practice.

Remove useless import.
 2024-12-23 01:51:08 -05:00
comfyanonymous
56bc64f351 Comment out some useless code. 2024-12-22 23:51:14 -05:00
zhangp365
f7d83b72e0
fixed a bug in ldm/pixart/blocks.py (#6158) 2024-12-22 23:44:20 -05:00
comfyanonymous
80f07952d2 Fix lowvram issue with ltxv vae. 2024-12-22 23:20:17 -05:00
comfyanonymous
57f330caf9 Relax minimum ratio of weights loaded in memory on nvidia. 
This should make it possible to do higher res images/longer videos by
further offloading weights to CPU memory.

Please report an issue if this slows down things on your system.
 2024-12-22 03:06:37 -05:00
comfyanonymous
da13b6b827 Get rid of meshgrid warning. 2024-12-20 18:02:12 -05:00
comfyanonymous
c86cd58573 Remove useless code. 2024-12-20 17:50:03 -05:00
comfyanonymous
b5fe39211a Remove some useless code. 2024-12-20 17:43:50 -05:00
comfyanonymous
e946667216 Some fixes/cleanups to pixart code. 
Commented out the masking related code because it is never used in this
implementation.
 2024-12-20 17:10:52 -05:00
Chenlei Hu
d7969cb070
Replace print with logging (#6138) 
* Replace print with logging

* nit

* nit

* nit

* nit

* nit

* nit
 2024-12-20 16:24:55 -05:00
City
bddb02660c
Add PixArt model support (#6055) 
* PixArt initial version

* PixArt Diffusers convert logic

* pos_emb and interpolation logic

* Reduce  duplicate code

* Formatting

* Use optimized attention

* Edit empty token logic

* Basic PixArt LoRA support

* Fix aspect ratio logic

* PixArtAlpha text encode with conds

* Use same detection key logic for PixArt diffusers
 2024-12-20 15:25:00 -05:00
comfyanonymous
418eb7062d Support new LTXV VAE. 2024-12-20 04:38:29 -05:00
comfyanonymous
cac68ca813 Fix some more video tiled encode issues. 
The downscale_ratio formula for the temporal had issues with some frame
numbers.
 2024-12-19 23:14:03 -05:00
comfyanonymous
52c1d933b2 Fix tiled hunyuan video VAE encode issue. 
Some shapes like 1024x1024 with tile_size 256 and overlap 64 had issues.
 2024-12-19 22:55:15 -05:00
comfyanonymous
2dda7c11a3 More proper fix for the memory issue. 2024-12-19 16:21:56 -05:00
comfyanonymous
3ad3248ad7 Fix lowvram bug when using a model multiple times in a row. 
The memory system would load an extra 64MB each time until either the
model was completely in memory or OOM.
 2024-12-19 16:04:56 -05:00
comfyanonymous
c441048a4f Make VAE Encode tiled node work with video VAE. 2024-12-19 05:31:39 -05:00
doctorpangloss
86b15084d5 Fix issues with directories and running on macOS 
- include detailed runtime instructions for Windows and macOS
 - include instructions for agreeing to use Hugging Face repositories
 - always create directories by default when run interactively
 - model downloader now supports multiple folder names for known models
 - improve logging in sd.py
 2024-12-18 15:37:16 -08:00
comfyanonymous
9f4b181ab3 Add fast previews for hunyuan video. 2024-12-18 18:24:23 -05:00
comfyanonymous
cbbf077593 Small optimizations. 2024-12-18 18:23:28 -05:00
comfyanonymous
ff2ff02168 Support old diffusion-pipe hunyuan video loras. 2024-12-18 06:23:54 -05:00
comfyanonymous
4c5c4ddeda Fix regression in VAE code on old pytorch versions. 2024-12-18 03:08:28 -05:00
comfyanonymous
37e5390f5f Add: --use-sage-attention to enable SageAttention. 
You need to have the library installed first.
 2024-12-18 01:56:10 -05:00
comfyanonymous
a4f59bc65e Pick attention implementation based on device in llama code. 2024-12-18 01:30:20 -05:00
comfyanonymous
ca457f7ba1 Properly tokenize the template for hunyuan video. 2024-12-17 16:22:02 -05:00
comfyanonymous
cd6f615038 Fix tiled vae not working with some shapes. 2024-12-17 16:22:02 -05:00
comfyanonymous
e4e1bff605 Support diffusion-pipe hunyuan video lora format. 2024-12-17 07:14:21 -05:00
comfyanonymous
d6656b0c0c Support llama hunyuan video text encoder in scaled fp8 format. 2024-12-17 04:19:22 -05:00
comfyanonymous
f4cdedea62 Fix regression with ltxv VAE. 2024-12-17 02:17:31 -05:00
comfyanonymous
39b1fc4ccc Adjust used dtypes for hunyuan video VAE and diffusion model. 2024-12-16 23:31:10 -05:00
comfyanonymous
bda1482a27 Basic Hunyuan Video model support. 2024-12-16 19:35:40 -05:00
comfyanonymous
19ee5d9d8b Don't expand mask when not necessary. 
Expanding seems to slow down inference.
 2024-12-16 18:22:50 -05:00
Raphael Walker
61b50720d0
Add support for attention masking in Flux (#5942) 
* fix attention OOM in xformers

* allow passing attention mask in flux attention

* allow an attn_mask in flux

* attn masks can be done using replace patches instead of a separate dict

* fix return types

* fix return order

* enumerate

* patch the right keys

* arg names

* fix a silly bug

* fix xformers masks

* replace match with if, elif, else

* mask with image_ref_size

* remove unused import

* remove unused import 2

* fix pytorch/xformers attention

This corrects a weird inconsistency with skip_reshape.
It also allows masks of various shapes to be passed, which will be
automtically expanded (in a memory-efficient way) to a size that is
compatible with xformers or pytorch sdpa respectively.

* fix mask shapes
 2024-12-16 18:21:17 -05:00
comfyanonymous
e83063bf24 Support conv3d in PatchEmbed. 2024-12-14 05:46:04 -05:00
comfyanonymous
4e14032c02 Make pad_to_patch_size function work on multi dim. 2024-12-13 07:22:05 -05:00
Chenlei Hu
563291ee51
Enforce all pyflake lint rules (#6033) 
* Enforce F821 undefined-name

* Enforce all pyflake lint rules
 2024-12-12 19:29:37 -05:00
Chenlei Hu
2cddbf0821
Lint and fix undefined names (1/N) (#6028) 2024-12-12 18:55:26 -05:00
Chenlei Hu
60749f345d
Lint and fix undefined names (3/N) (#6030) 2024-12-12 18:49:40 -05:00
Chenlei Hu
d9d7f3c619
Lint all unused variables (#5989) 
* Enable F841

* Autofix

* Remove all unused variable assignment
 2024-12-12 17:59:16 -05:00
comfyanonymous
fd5dfb812c Set initial load devices for te and model to mps device on mac. 2024-12-12 06:00:31 -05:00
comfyanonymous
7a7efe8424 Support loading some checkpoint files with nested dicts. 2024-12-11 08:04:54 -05:00
comfyanonymous
44db978531 Fix a few things in text enc code for models with no eos token. 2024-12-10 23:07:26 -05:00
doctorpangloss
65f3be4f8f Fix incorrect use of internal API 2024-12-10 13:32:21 -08:00
comfyanonymous
1c8d11e48a Support different types of tokenizers. 
Support tokenizers without an eos token.

Pass full sentences to tokenizer for more efficient tokenizing.
 2024-12-10 15:03:39 -05:00
doctorpangloss
d989e65fde Update ComfyUI and fix tests 2024-12-09 19:45:17 -08:00
catboxanon
23827ca312
Add cond_scale to sampler_post_cfg_function (#5985) 2024-12-09 20:13:18 -05:00
doctorpangloss
2d1676c717 Merge branch 'master' of github.com:comfyanonymous/ComfyUI 2024-12-09 15:54:37 -08:00
Chenlei Hu
0fd4e6c778
Lint unused import (#5973) 
* Lint unused import

* nit

* Remove unused imports

* revert fix_torch import

* nit
 2024-12-09 15:24:39 -05:00
comfyanonymous
e2fafe0686 Make CLIP set last layer node work with t5 models. 2024-12-09 03:57:14 -05:00
Haoming
fbf68c4e52
clamp input (#5928) 2024-12-07 14:00:31 -05:00
doctorpangloss
a52ffe2aff Fix supported model extensions being passed as a frozen set, making them impossible to mutate later 2024-12-06 11:22:39 -08:00
doctorpangloss
778541755d Fix addresses being saved with commas 2024-12-06 11:22:20 -08:00
doctorpangloss
1a2c4b4cc6 Improve response semantics 2024-12-06 11:22:09 -08:00
comfyanonymous
8af9a91e0c A few improvements to #5937. 2024-12-06 05:49:15 -05:00
Michael Kupchick
005d2d3a13
ltxv: add noise to guidance image to ensure generated motion. (#5937) 2024-12-06 05:46:08 -05:00
doctorpangloss
4f085c4d58 Fix typename here 2024-12-05 19:12:42 -08:00
doctorpangloss
68900be89e Fix operation ids in openapi 2024-12-05 18:57:04 -08:00
doctorpangloss
8c0ece1b8c Creating all known paths now correctly creates paths for custom nodes' models 2024-12-05 14:11:41 -08:00
comfyanonymous
1e21f4c14e Make timestep ranges more usable on rectified flow models. 
This breaks some old workflows but should make the nodes actually useful.
 2024-12-05 16:40:58 -05:00
Chenlei Hu
48272448ad
[Developer Experience] Add node typing (#5676) 
* [Developer Experience] Add node typing

* Shim StrEnum

* nit

* nit

* nit
 2024-12-04 15:01:00 -05:00
comfyanonymous
452179fe4f Make ModelPatcher class clone function work with inheritance. 2024-12-03 13:57:57 -05:00
comfyanonymous
c1b92b719d Some optimizations to euler a. 2024-12-03 06:11:52 -05:00
comfyanonymous
57e8bf6a9f Fix case where a memory leak could cause crash. 
Now the only symptom of code messing up and keeping references to a model
object when it should not will be endless prints in the log instead of the
next workflow crashing ComfyUI.
 2024-12-02 19:49:49 -05:00
Jedrzej Kosinski
0ee322ec5f
ModelPatcher Overhaul and Hook Support (#5583) 
* Added hook_patches to ModelPatcher for weights (model)

* Initial changes to calc_cond_batch to eventually support hook_patches

* Added current_patcher property to BaseModel

* Consolidated add_hook_patches_as_diffs into add_hook_patches func, fixed fp8 support for model-as-lora feature

* Added call to initialize_timesteps on hooks in process_conds func, and added call prepare current keyframe on hooks in calc_cond_batch

* Added default_conds support in calc_cond_batch func

* Added initial set of hook-related nodes, added code to register hooks for loras/model-as-loras, small renaming/refactoring

* Made CLIP work with hook patches

* Added initial hook scheduling nodes, small renaming/refactoring

* Fixed MaxSpeed and default conds implementations

* Added support for adding weight hooks that aren't registered on the ModelPatcher at sampling time

* Made Set Clip Hooks node work with hooks from Create Hook nodes, began work on better Create Hook Model As LoRA node

* Initial work on adding 'model_as_lora' lora type to calculate_weight

* Continued work on simpler Create Hook Model As LoRA node, started to implement ModelPatcher callbacks, attachments, and additional_models

* Fix incorrect ref to create_hook_patches_clone after moving function

* Added injections support to ModelPatcher + necessary bookkeeping, added additional_models support in ModelPatcher, conds, and hooks

* Added wrappers to ModelPatcher to facilitate standardized function wrapping

* Started scaffolding for other hook types, refactored get_hooks_from_cond to organize hooks by type

* Fix skip_until_exit logic bug breaking injection after first run of model

* Updated clone_has_same_weights function to account for new ModelPatcher properties, improved AutoPatcherEjector usage in partially_load

* Added WrapperExecutor for non-classbound functions, added calc_cond_batch wrappers

* Refactored callbacks+wrappers to allow storing lists by id

* Added forward_timestep_embed_patch type, added helper functions on ModelPatcher for emb_patch and forward_timestep_embed_patch, added helper functions for removing callbacks/wrappers/additional_models by key, added custom_should_register prop to hooks

* Added get_attachment func on ModelPatcher

* Implement basic MemoryCounter system for determing with cached weights due to hooks should be offloaded in hooks_backup

* Modified ControlNet/T2IAdapter get_control function to receive transformer_options as additional parameter, made the model_options stored in extra_args in inner_sample be a clone of the original model_options instead of same ref

* Added create_model_options_clone func, modified type annotations to use __future__ so that I can use the better type annotations

* Refactored WrapperExecutor code to remove need for WrapperClassExecutor (now gone), added sampler.sample wrapper (pending review, will likely keep but will see what hacks this could currently let me get rid of in ACN/ADE)

* Added Combine versions of Cond/Cond Pair Set Props nodes, renamed Pair Cond to Cond Pair, fixed default conds never applying hooks (due to hooks key typo)

* Renamed Create Hook Model As LoRA nodes to make the test node the main one (more changes pending)

* Added uuid to conds in CFGGuider and uuids to transformer_options to allow uniquely identifying conds in batches during sampling

* Fixed models not being unloaded properly due to current_patcher reference; the current ComfyUI model cleanup code requires that nothing else has a reference to the ModelPatcher instances

* Fixed default conds not respecting hook keyframes, made keyframes not reset cache when strength is unchanged, fixed Cond Set Default Combine throwing error, fixed model-as-lora throwing error during calculate_weight after a recent ComfyUI update, small refactoring/scaffolding changes for hooks

* Changed CreateHookModelAsLoraTest to be the new CreateHookModelAsLora, rename old ones as 'direct' and will be removed prior to merge

* Added initial support within CLIP Text Encode (Prompt) node for scheduling weight hook CLIP strength via clip_start_percent/clip_end_percent on conds, added schedule_clip toggle to Set CLIP Hooks node, small cleanup/fixes

* Fix range check in get_hooks_for_clip_schedule so that proper keyframes get assigned to corresponding ranges

* Optimized CLIP hook scheduling to treat same strength as same keyframe

* Less fragile memory management.

* Make encode_from_tokens_scheduled call cleaner, rollback change in model_patcher.py for hook_patches_backup dict

* Fix issue.

* Remove useless function.

* Prevent and detect some types of memory leaks.

* Run garbage collector when switching workflow if needed.

* Moved WrappersMP/CallbacksMP/WrapperExecutor to patcher_extension.py

* Refactored code to store wrappers and callbacks in transformer_options, added apply_model and diffusion_model.forward wrappers

* Fix issue.

* Refactored hooks in calc_cond_batch to be part of get_area_and_mult tuple, added extra_hooks to ControlBase to allow custom controlnets w/ hooks, small cleanup and renaming

* Fixed inconsistency of results when schedule_clip is set to False, small renaming/typo fixing, added initial support for ControlNet extra_hooks to work in tandem with normal cond hooks, initial work on calc_cond_batch merging all subdicts in returned transformer_options

* Modified callbacks and wrappers so that unregistered types can be used, allowing custom_nodes to have their own unique callbacks/wrappers if desired

* Updated different hook types to reflect actual progress of implementation, initial scaffolding for working WrapperHook functionality

* Fixed existing weight hook_patches (pre-registered) not working properly for CLIP

* Removed Register/Direct hook nodes since they were present only for testing, removed diff-related weight hook calculation as improved_memory removes unload_model_clones and using sample time registered hooks is less hacky

* Added clip scheduling support to all other native ComfyUI text encoding nodes (sdxl, flux, hunyuan, sd3)

* Made WrapperHook functional, added another wrapper/callback getter, added ON_DETACH callback to ModelPatcher

* Made opt_hooks append by default instead of replace, renamed comfy.hooks set functions to be more accurate

* Added apply_to_conds to Set CLIP Hooks, modified relevant code to allow text encoding to automatically apply hooks to output conds when apply_to_conds is set to True

* Fix cached_hook_patches not respecting target_device/memory_counter results

* Fixed issue with setting weights from hooks instead of copying them, added additional memory_counter check when caching hook patches

* Remove unnecessary torch.no_grad calls for hook patches

* Increased MemoryCounter minimum memory to leave free by *2 until a better way to get inference memory estimate of currently loaded models exists

* For encode_from_tokens_scheduled, allow start_percent and end_percent in add_dict to limit which scheduled conds get encoded for optimization purposes

* Removed a .to call on results of calculate_weight in patch_hook_weight_to_device that was screwing up the intermediate results for fp8 prior to being passed into stochastic_rounding call

* Made encode_from_tokens_scheduled work when no hooks are set on patcher

* Small cleanup of comments

* Turn off hook patch caching when only 1 hook present in sampling, replace some current_hook = None with calls to self.patch_hooks(None) instead to avoid a potential edge case

* On Cond/Cond Pair nodes, removed opt_ prefix from optional inputs

* Allow both FLOATS and FLOAT for floats_strength input

* Revert change, does not work

* Made patch_hook_weight_to_device respect set_func and convert_func

* Make discard_model_sampling True by default

* Add changes manually from 'master' so merge conflict resolution goes more smoothly

* Cleaned up text encode nodes with just a single clip.encode_from_tokens_scheduled call

* Make sure encode_from_tokens_scheduled will respect use_clip_schedule on clip

* Made nodes in nodes_hooks be marked as experimental (beta)

* Add get_nested_additional_models for cases where additional_models could have their own additional_models, and add robustness for circular additional_models references

* Made finalize_default_conds area math consistent with other sampling code

* Changed 'opt_hooks' input of Cond/Cond Pair Set Default Combine nodes to 'hooks'

* Remove a couple old TODO's and a no longer necessary workaround
 2024-12-02 14:51:02 -05:00
comfyanonymous
79d5ceae6e
Improved memory management. (#5450) 
* Less fragile memory management.

* Fix issue.

* Remove useless function.

* Prevent and detect some types of memory leaks.

* Run garbage collector when switching workflow if needed.

* Fix issue.
 2024-12-02 14:39:34 -05:00
comfyanonymous
2d5b3e0078 Remove useless code. 2024-12-02 06:49:55 -05:00
comfyanonymous
8e4118c0de make dpm_2_ancestral work with rectified flow. 2024-12-01 07:37:41 -05:00
comfyanonymous
26fb2c68e8 Add a way to disable cropping in the CLIPVisionEncode node. 2024-11-28 20:24:47 -05:00
comfyanonymous
bf2650a80e Fast previews for ltxv. 2024-11-28 06:46:15 -05:00
comfyanonymous
b666539595 Remove print. 2024-11-27 20:28:39 -05:00
comfyanonymous
95d8713482 Missing parentheses. 2024-11-27 13:45:32 -05:00
comfyanonymous
497db6212f Alternative fix for #5767 2024-11-26 17:53:04 -05:00
comfyanonymous
4c82741b54 Support official SD3.5 Controlnets. 2024-11-26 11:31:25 -05:00
comfyanonymous
15c39ea757 Support for the official mochi lora format. 2024-11-26 03:34:36 -05:00
comfyanonymous
b7143b74ce Flux inpaint model does not work in fp16. 2024-11-26 01:33:01 -05:00
comfyanonymous
61196d8857 Add option to inference the diffusion model in fp32 and fp64. 2024-11-25 05:00:23 -05:00
comfyanonymous
b4526d3fc3 Skip layer guidance now works on hydit model. 2024-11-24 05:54:30 -05:00
comfyanonymous
ab885b33ba Skip layer guidance node now works on LTX-Video. 2024-11-23 10:33:05 -05:00
doctorpangloss
b1ad9cad37 Known Flux controlnet models 2024-11-22 18:00:29 -08:00
comfyanonymous
839ed3368e Some improvements to the lowvram unloading. 2024-11-22 20:59:15 -05:00
doctorpangloss
4b77c4941c LTXV tests 2024-11-22 17:13:19 -08:00
doctorpangloss
fe64070b41 Fix bad merge of terminal_service 2024-11-22 15:55:51 -08:00
doctorpangloss
f39b8dfebc Merge branch 'master' of github.com:comfyanonymous/ComfyUI 2024-11-22 15:50:23 -08:00
comfyanonymous
6e8cdcd3cb Fix some tiled VAE decoding issues with LTX-Video. 2024-11-22 18:00:34 -05:00
comfyanonymous
e5c3f4b87f LTXV lowvram fixes. 2024-11-22 17:17:11 -05:00
comfyanonymous
bc6be6c11e Some fixes to the lowvram system. 2024-11-22 16:40:04 -05:00
comfyanonymous
5818f6cf51 Remove print. 2024-11-22 10:49:15 -05:00
comfyanonymous
5e16f1d24b Support Lightricks LTX-Video model. 2024-11-22 08:46:39 -05:00
comfyanonymous
2fd9c1308a Fix mask issue in some attention functions. 2024-11-22 02:10:09 -05:00
comfyanonymous
8f0009aad0 Support new flux model variants. 2024-11-21 08:38:23 -05:00
comfyanonymous
41444b5236 Add some new weight patching functionality. 
Add a way to reshape lora weights.

Allow weight patches to all weight not just .weight and .bias

Add a way for a lora to set a weight to a specific value.
 2024-11-21 07:19:17 -05:00
comfyanonymous
07f6eeaa13 Fix mask issue with attention_xformers. 2024-11-20 17:07:46 -05:00
comfyanonymous
22535d0589 Skip layer guidance now works on stable audio model. 2024-11-20 07:33:06 -05:00
doctorpangloss
9d20de6462 Merge branch 'improved_memory' of github.com:comfyanonymous/ComfyUI 2024-11-19 11:06:27 -08:00
comfyanonymous
b699a15062 Refactor inpaint/ip2p code. 2024-11-19 03:25:25 -05:00
doctorpangloss
8ba412897e Mochi and SageAttention improvements 2024-11-18 15:40:15 -08:00
doctorpangloss
264d84db39 Fix Pylint warnings 2024-11-18 14:10:58 -08:00
doctorpangloss
fb7a3f9386 Update ComfyUI 
- use their logger when running interactively
 - move the extra nodes files to where this fork expects them
 - add the mochi checkpoints to known models
 - add a mochi workflow test
 2024-11-18 13:58:24 -08:00
doctorpangloss
c0f072ee0f Merge branch 'master' of github.com:comfyanonymous/ComfyUI 2024-11-18 13:12:31 -08:00
comfyanonymous
d9f90965c8 Support block replace patches in auraflow. 2024-11-17 08:19:59 -05:00
comfyanonymous
41886af138 Add transformer options blocks replace patch to mochi. 2024-11-16 20:48:14 -05:00
doctorpangloss
4150dbbbe5 Tweaks to distributed queueing 
- Do not auto delete the queue
 - Make the queue durable
 - Progress notifications expire
 - Deprecation fix
 2024-11-14 15:08:59 -08:00
comfyanonymous
3b9a6cf2b1 Fix issue with 3d masks. 2024-11-13 07:18:30 -05:00
comfyanonymous
8ebf2d8831 Add block replace transformer_options to flux. 2024-11-12 08:00:39 -05:00
doctorpangloss
44be2591df Fix broken create-directories command 2024-11-11 16:21:16 -08:00
doctorpangloss
228794d835 Fix missing folder paths, fix #26 the protobuf compatibility issue manifests in 1.28 2024-11-11 13:35:57 -08:00
comfyanonymous
eb476e6ea9 Allow 1D masks for 1D latents. 2024-11-11 14:44:52 -05:00
comfyanonymous
8b275ce5be Support auto detecting some zsnr anime checkpoints. 2024-11-11 05:34:11 -05:00
comfyanonymous
2a18e98ccf Refactor so that zsnr can be set in the sampling_settings. 2024-11-11 04:55:56 -05:00
comfyanonymous
bdeb1c171c Fast previews for mochi. 2024-11-10 03:39:35 -05:00
comfyanonymous
8b90e50979 Properly handle and reshape masks when used on 3d latents. 2024-11-09 15:30:19 -05:00
comfyanonymous
2865f913f7 Free memory before doing tiled decode. 2024-11-07 04:01:24 -05:00
comfyanonymous
b49616f951 Make VAEDecodeTiled node work with video VAEs. 2024-11-07 03:47:12 -05:00
comfyanonymous
5e29e7a488 Remove scaled_fp8 key after reading it to silence warning. 2024-11-06 04:56:42 -05:00
comfyanonymous
8afb97cd3f Fix unknown VAE being detected as the mochi VAE. 2024-11-05 03:43:27 -05:00
contentis
69694f40b3
fix dynamic shape export (#5490) 2024-11-04 14:59:28 -05:00
doctorpangloss
772e768fe8 Merge branch 'master' of github.com:comfyanonymous/ComfyUI 2024-11-04 10:17:26 -08:00
doctorpangloss
cde95eb71d Improve logging and typing information for LoRA patches in ComfyUI 2024-11-04 09:38:13 -08:00
comfyanonymous
95972bab86 Fix issue. 2024-11-04 05:07:07 -05:00
comfyanonymous
6c9dbde7de Fix mochi all in one checkpoint t5xxl key names. 2024-11-03 01:40:42 -05:00
comfyanonymous
fabf449feb Mochi VAE encoder. 2024-11-01 17:33:09 -04:00
doctorpangloss
021d0d4f57 Fix #25 custom nodes which have input paths set at import time will now correctly see a models directory (or similar) that respects the configuration intended by the user 2024-11-01 13:40:03 -07:00
doctorpangloss
31eacb6ac9 Improve compilation of models, adding support for triton 2024-11-01 10:40:58 -07:00
comfyanonymous
bd5d8f150f Prevent and detect some types of memory leaks. 2024-11-01 06:55:42 -04:00
comfyanonymous
975927cc79 Remove useless function. 2024-11-01 04:40:33 -04:00
comfyanonymous
1735d4fb01 Fix issue. 2024-11-01 04:25:27 -04:00
comfyanonymous
d8bd2a9baa Less fragile memory management. 2024-11-01 02:41:51 -04:00
Aarni Koskela
1c8286a44b
Avoid SyntaxWarning in UniPC docstring (#5442) 2024-10-31 15:17:26 -04:00
comfyanonymous
1af4a47fd1 Bump up mac version for attention upcast bug workaround. 2024-10-31 15:15:31 -04:00
comfyanonymous
daa1565b93 Fix diffusers flux controlnet regression. 2024-10-30 13:11:34 -04:00
comfyanonymous
09fdb2b269 Support SD3.5 medium diffusers format weights and loras. 2024-10-30 04:24:00 -04:00
doctorpangloss
a5467b897d Fix pylint error / 3.10 missing add_node 2024-10-29 19:37:06 -07:00
doctorpangloss
45299987f3 Mochi variable now correctly ferenced 2024-10-29 19:24:17 -07:00
doctorpangloss
b3ceeebf94 Fix bugs in folder paths 
- Adding the output paths now correctly registers a relative path,
   i.e., outputs/loras and models/lorals will now be searched on all
   your base paths
 - Adding absolute paths with models/ works better
 - All the base paths and directories are queried better
 2024-10-29 19:22:51 -07:00
doctorpangloss
a8d8bff548 Improve support for torch compilation and sage attention 2024-10-29 19:22:26 -07:00
doctorpangloss
76a80a65ea Merge branch 'master' of github.com:comfyanonymous/ComfyUI 2024-10-29 15:35:39 -07:00
doctorpangloss
e6bddb4a9c Fix pylint errors 2024-10-29 14:27:14 -07:00
doctorpangloss
b42e59d602 Fix tests 2024-10-29 14:27:14 -07:00
doctorpangloss
4a13766d14 --base-paths argument adds additional paths to search for models/checkpoints, models/loras, etc. directories, including directories specified in this pattern by custom nodes 2024-10-29 14:27:14 -07:00
comfyanonymous
30c0c81351 Add a way to patch blocks in SD3. 2024-10-29 00:48:32 -04:00
comfyanonymous
13b0ff8a6f Update SD3 code. 2024-10-28 21:58:52 -04:00
comfyanonymous
c320801187 Remove useless line. 2024-10-28 17:41:12 -04:00
comfyanonymous
669d9e4c67 Set default shift on mochi to 6.0 2024-10-27 22:21:04 -04:00
comfyanonymous
9ee0a6553a float16 inference is a bit broken on mochi. 2024-10-27 04:56:40 -04:00
comfyanonymous
5cbb01bc2f Basic Genmo Mochi video model support. 
To use:
"Load CLIP" node with t5xxl + type mochi
"Load Diffusion Model" node with the mochi dit file.
"Load VAE" with the mochi vae file.

EmptyMochiLatentVideo node for the latent.
euler + linear_quadratic in the KSampler node.
 2024-10-26 06:54:00 -04:00
comfyanonymous
c3ffbae067 Make LatentUpscale nodes work on 3d latents. 2024-10-26 01:50:51 -04:00
comfyanonymous
d605677b33 Make euler_ancestral work on flow models (credit: Ashen). 2024-10-25 19:53:44 -04:00
PsychoLogicAu
af8cf79a2d
support SimpleTuner lycoris lora for SD3 (#5340) 2024-10-24 01:18:32 -04:00
comfyanonymous
66b0961a46 Fix ControlLora issue with last commit. 2024-10-23 17:02:40 -04:00
comfyanonymous
754597c8a9 Clean up some controlnet code. 
Remove self.device which was useless.
 2024-10-23 14:19:05 -04:00
comfyanonymous
915fdb5745 Fix lowvram edge case. 2024-10-22 16:34:50 -04:00
contentis
5a8a48931a
remove attention abstraction (#5324) 2024-10-22 14:02:38 -04:00
comfyanonymous
8ce2a1052c Optimizations to --fast and scaled fp8. 2024-10-22 02:12:28 -04:00
comfyanonymous
f82314fcfc Fix duplicate sigmas on beta scheduler. 2024-10-21 20:19:45 -04:00
comfyanonymous
0075c6d096 Mixed precision diffusion models with scaled fp8. 
This change allows supports for diffusion models where all the linears are
scaled fp8 while the other weights are the original precision.
 2024-10-21 18:12:51 -04:00
comfyanonymous
83ca891118 Support scaled fp8 t5xxl model. 2024-10-20 22:27:00 -04:00
comfyanonymous
f9f9faface Fixed model merging issue with scaled fp8. 2024-10-20 06:24:31 -04:00
comfyanonymous
471cd3eace fp8 casting is fast on GPUs that support fp8 compute. 2024-10-20 00:54:47 -04:00
comfyanonymous
a68bbafddb Support diffusion models with scaled fp8 weights. 2024-10-19 23:47:42 -04:00
comfyanonymous
73e3a9e676 Clamp output when rounding weight to prevent Nan. 2024-10-19 19:07:10 -04:00
comfyanonymous
67158994a4 Use the lowvram cast_to function for everything. 2024-10-17 17:25:56 -04:00
comfyanonymous
0bedfb26af Revert "Fix Transformers FutureWarning (#5140)" 
This reverts commit 95b7cf9bbe.
 2024-10-16 12:36:19 -04:00
doctorpangloss
a83b561ea7 Follow symlinks for statics so that packages can correctly serve files when installed with uv. Update version. 2024-10-15 11:01:46 -07:00
doctorpangloss
5412451def Handle custom_nodes returning None responses more gracefully 2024-10-15 11:01:21 -07:00
doctorpangloss
995807b4be Improve custom node compatibility by including this stub symbol 2024-10-15 10:13:28 -07:00
doctorpangloss
40902acc28 Use the HuggingFace file for dreamshaper 2024-10-15 10:13:13 -07:00
Benjamin Berman
e5fc19a25b Improve vanilla node importing and fix CUDA on CPU devices bug 2024-10-15 00:02:06 -07:00
Benjamin Berman
9c9df424b4 Fix CUDA package with no drivers 2024-10-14 22:56:21 -07:00
comfyanonymous
f584758271 Cleanup some useless lines. 2024-10-14 21:02:39 -04:00
svdc
95b7cf9bbe
Fix Transformers FutureWarning (#5140) 
* Update sd1_clip.py

Fix Transformers FutureWarning

* Update sd1_clip.py

Fix comment
 2024-10-14 20:12:20 -04:00
doctorpangloss
b0d606a282 Improve installation instructions with non-deprecated messaging. 0.2.3 is now directly written as the server version. 2024-10-14 15:54:21 -07:00
doctorpangloss
8512f361fe Merge branch 'master' of github.com:comfyanonymous/ComfyUI 2024-10-14 15:26:27 -07:00
comfyanonymous
3c60ecd7a8 Fix fp8 ops staying enabled. 2024-10-12 14:10:13 -04:00
comfyanonymous
7ae6626723 Remove useless argument. 2024-10-12 07:16:21 -04:00
comfyanonymous
6632365e16 model_options consistency between functions. 
weight_dtype -> dtype
 2024-10-11 20:51:19 -04:00
Kadir Nar
ad07796777
🐛 Add device to variable c (#5210) 2024-10-11 20:37:50 -04:00
doctorpangloss
c0d1c9f96d Improve OpenAPI spec 2024-10-11 14:46:26 -07:00
doctorpangloss
ed078c2f1f Update web content 2024-10-11 14:00:16 -07:00
doctorpangloss
b5df6c64fa Update OpenAPI spec to be more accurate 2024-10-11 13:59:57 -07:00
doctorpangloss
79b465faf2 Increase server response timeouts 2024-10-11 13:52:17 -07:00
doctorpangloss
caa6a37936 Fix pylint error 2024-10-11 13:51:13 -07:00
doctorpangloss
1cc637cb4f Fix SDXL clip issue, fix website header issue 2024-10-10 22:46:52 -07:00
doctorpangloss
f3da381869 Fix inference mode execution issues 2024-10-10 21:00:09 -07:00
doctorpangloss
a38968f098 Improvements to execution 
- Validation errors that occur early in the lifecycle of prompt
   execution now get propagated to their callers in the
   EmbeddedComfyClient. This includes error messages about missing node
   classes.
 - The execution context now includes the node_id and the prompt_id
 - Latent previews are now sent with a node_id. This is not backwards
   compatible with old frontends.
 - Dependency execution errors are now modeled correctly.
 - Distributed progress encodes image previews with node and prompt IDs.
 - Typing for models
 - The frontend was updated to use node IDs with previews
 - Improvements to torch.compile experiments
 - Some controlnet_aux nodes were upstreamed
 2024-10-10 19:30:18 -07:00
doctorpangloss
69e523b89d Experimental quantization support. Only Linux is meaningfully supported 2024-10-10 13:43:06 -07:00
comfyanonymous
1b80895285 Make clip loader nodes support loading sd3 t5xxl in lower precision. 
Add attention mask support in the SD3 text encoder code.
 2024-10-10 15:06:15 -04:00
doctorpangloss
5f26b76f59 Gracefully handle running with cuda torch on CPU only devices 2024-10-10 10:42:22 -07:00
Dr.Lt.Data
5f9d5a244b
Hotfix for the div zero occurrence when memory_used_encode is 0 (#5121) 
https://github.com/comfyanonymous/ComfyUI/issues/5069#issuecomment-2382656368
 2024-10-09 23:34:34 -04:00
Jonathan Avila
4b2f0d9413
Increase maximum macOS version to 15.0.1 when forcing upcast attention (#5191) 2024-10-09 22:21:41 -04:00
comfyanonymous
e38c94228b Add a weight_dtype fp8_e4m3fn_fast to the Diffusion Model Loader node. 
This is used to load weights in fp8 and use fp8 matrix multiplication.
 2024-10-09 19:43:17 -04:00
doctorpangloss
c34403b574 Fix invalid device here 2024-10-09 11:21:19 -07:00
comfyanonymous
7ea7b2e77f Slightly improve the fast previews for flux by adding a bias. 2024-10-09 09:48:18 -07:00
comfyanonymous
9786ea4a17 Use torch.nn.functional.linear in RGB preview code. 
Add an optional bias to the latent RGB preview code.
 2024-10-09 09:48:17 -07:00
comfyanonymous
91f458061c Fix flux doras with diffusers keys. 2024-10-09 09:48:16 -07:00
City
7d1c420d19 Flux torch.compile fix (#5082) 2024-10-09 09:47:46 -07:00
doctorpangloss
99f0fa8b50 Enable sage attention autodetection 2024-10-09 09:27:05 -07:00
doctorpangloss
388dad67d5 Fix pylint errors in attention 2024-10-09 09:26:02 -07:00
doctorpangloss
bbe2ed330c Memory management and compilation improvements 
- Experimental support for sage attention on Linux
 - Diffusers loader now supports model indices
 - Transformers model management now aligns with updates to ComfyUI
 - Flux layers correctly use unbind
 - Add float8 support for model loading in more places
 - Experimental quantization approaches from Quanto and torchao
 - Model upscaling interacts with memory management better

This update also disables ROCm testing because it isn't reliable enough
on consumer hardware. ROCm is not really supported by the 7600.
 2024-10-09 09:13:47 -07:00
comfyanonymous
203942c8b2 Fix flux doras with diffusers keys. 2024-10-08 19:03:40 -04:00
comfyanonymous
8dfa0cc552 Make SD3 fast previews a little better. 2024-10-07 09:19:59 -04:00
comfyanonymous
e5ecdfdd2d Make fast previews for SDXL a little better by adding a bias. 2024-10-06 19:27:04 -04:00
comfyanonymous
7d29fbf74b Slightly improve the fast previews for flux by adding a bias. 2024-10-06 17:55:46 -04:00
comfyanonymous
7d2467e830 Some minor cleanups. 2024-10-05 13:22:39 -04:00
Benjamin Berman
0a25b67ff8 Fix pylint errors 2024-10-04 21:12:37 -07:00
Benjamin Berman
afbb8aa154 Fix #23 2024-10-04 21:10:19 -07:00
doctorpangloss
de45dd50c5 Improve vanilla node importing for execution nodes 2024-10-04 10:56:43 -07:00
comfyanonymous
6f021d8aa0 Let --verbose have an argument for the log level. 2024-10-04 10:05:34 -04:00
comfyanonymous
d854ed0bcf Allow using SD3 type te output on flux model. 2024-10-03 09:44:54 -04:00
comfyanonymous
abcd006b8c Allow more permutations of clip/t5 in dual clip loader. 2024-10-03 09:26:11 -04:00
comfyanonymous
d985d1d7dc CLIP Loader node now supports clip_l and clip_g only for SD3. 2024-10-02 04:25:17 -04:00
comfyanonymous
d1cdf51e1b Refactor some of the TE detection code. 2024-10-01 07:08:41 -04:00
doctorpangloss
144fe6c421 Fix aiohttp bugs 2024-09-30 13:12:53 -07:00
comfyanonymous
b4626ab93e Add simpletuner lycoris format for SD unet. 2024-09-30 06:03:27 -04:00
comfyanonymous
a9e459c2a4 Use torch.nn.functional.linear in RGB preview code. 
Add an optional bias to the latent RGB preview code.
 2024-09-29 11:27:49 -04:00
comfyanonymous
3bb4dec720 Fix issue with loras, lowvram and --fast fp8. 2024-09-28 14:42:32 -04:00
City
8733191563
Flux torch.compile fix (#5082) 2024-09-27 22:07:51 -04:00
doctorpangloss
6ef2d534b6 Fix polling for history too quickly. This will need an alternative approach so that readiness is immediate 2024-09-27 12:46:28 -07:00
doctorpangloss
d25394d386 API now supports fire-and-forget, checking on queue status; prefetch_count now expressly set to 1 for workers 2024-09-27 12:07:54 -07:00
doctorpangloss
a664a1fbc9 Add Flux inpainting model 2024-09-27 12:06:58 -07:00
doctorpangloss
667b77149e Improve scaling and fit for diffusion 2024-09-26 18:08:34 -07:00
doctorpangloss
dbc8ee92a5 Add method to make this congruent with aio client 2024-09-26 18:08:15 -07:00
doctorpangloss
ab1a1de7a4 Fix missing arg to add_model_folder_path 2024-09-26 13:26:52 -07:00
doctorpangloss
a78f20178d Fix linking error 2024-09-25 10:16:56 -07:00
doctorpangloss
8f58242c91 Fix frozenset v set issue in folder_paths 2024-09-24 20:36:50 -07:00
comfyanonymous
bdd4a22a2e Fix flux TE not loading t5 embeddings. 2024-09-24 22:57:22 -04:00
chaObserv
479a427a48
Add dpmpp_2m_cfg_pp (#4992) 2024-09-24 02:42:56 -04:00
doctorpangloss
fa3176f96f Merge branch 'master' of github.com:comfyanonymous/ComfyUI 2024-09-23 12:50:31 -07:00
doctorpangloss
4bdc208f29 Add noise to specific channels in a latent 2024-09-23 08:51:48 -07:00
comfyanonymous
3a0eeee320 Make --listen listen on both ipv4 and ipv6 at the same time by default. 2024-09-23 04:38:19 -04:00
comfyanonymous
9c41bc8d10 Remove useless line. 2024-09-23 02:32:29 -04:00
comfyanonymous
7a415f47a9 Add an optional VAE input to the ControlNetApplyAdvanced node. 
Deprecate the other controlnet nodes.
 2024-09-22 01:24:52 -04:00
comfyanonymous
dc96a1ae19 Load controlnet in fp8 if weights are in fp8. 2024-09-21 04:50:12 -04:00
comfyanonymous
2d810b081e Add load_controlnet_state_dict function. 2024-09-21 01:51:51 -04:00
comfyanonymous
9f7e9f0547 Add an error message when a controlnet needs a VAE but none is given. 2024-09-21 01:33:18 -04:00
comfyanonymous
70a708d726 Fix model merging issue. 2024-09-20 02:31:44 -04:00
yoinked
e7d4782736
add laplace scheduler [2407.03297] (#4990) 
* add laplace scheduler [2407.03297]

* should be here instead lol

* better settings
 2024-09-19 23:23:09 -04:00
comfyanonymous
ad66f7c7d8 Add model_options to load_controlnet function. 2024-09-19 08:23:35 -04:00
Simon Lui
de8e8e3b0d
Fix xpu Pytorch nightly build from calling optimize which doesn't exist. (#4978) 2024-09-19 05:11:42 -04:00
doctorpangloss
e820a5de20 Revert "Reduce repeated calls of get_immediate_node_signature for ancestors in cache (#4871)" 
This reverts commit f6b7194f64.
 2024-09-17 16:54:55 -07:00
doctorpangloss
d30f15ed09 Fix caching issues with text nodes when working with the UI 2024-09-17 16:09:47 -07:00
pharmapsychotic
0b7dfa986d
Improve tiling calculations to reduce number of tiles that need to be processed. (#4944) 2024-09-17 03:51:10 -04:00
comfyanonymous
d514bb38ee Add some option to model_options for the text encoder. 
load_device, offload_device and the initial_device can now be set.
 2024-09-17 03:49:54 -04:00
comfyanonymous
0849c80e2a get_key_patches now works without unloading the model. 2024-09-17 01:57:59 -04:00
comfyanonymous
e813abbb2c Long CLIP L support for SDXL, SD3 and Flux. 
Use the *CLIPLoader nodes.
 2024-09-15 07:59:38 -04:00
comfyanonymous
f48e390032 Support AliMama SD3 and Flux inpaint controlnets. 
Use the ControlNetInpaintingAliMamaApply node.
 2024-09-14 09:05:16 -04:00
doctorpangloss
83b2f0174c Fix tests, improve distributed worker health check, add torch compile options 2024-09-13 18:10:11 -07:00
doctorpangloss
ffb4ed9cf2 Merge branch 'master' of github.com:comfyanonymous/ComfyUI 2024-09-13 12:45:23 -07:00
comfyanonymous
cf80d28689 Support loading controlnets with different input. 2024-09-13 09:54:37 -04:00
Robin Huang
b962db9952
Add cli arg to override user directory (#4856) 
* Override user directory.

* Use overridden user directory.

* Remove prints.

* Remove references to global user_files.

* Remove unused replace_folder function.

* Remove newline.

* Remove global during get_user_directory.

* Add validation.
 2024-09-12 08:10:27 -04:00
comfyanonymous
9d720187f1 types -> comfy_types to fix import issue. 2024-09-12 03:57:46 -04:00
comfyanonymous
9f4daca9d9 Doesn't really make sense for cfg_pp sampler to call regular one. 2024-09-11 02:51:36 -04:00
yoinked
b5d0f2a908
Add CFG++ to DPM++ 2S Ancestral (#3871) 
* Update sampling.py

* Update samplers.py

* my bad

* "fix" the sampler

* Update samplers.py

* i named it wrong

* minor sampling improvements

mainly using a dynamic rho value (hey this sounds a lot like smea!!!)

* revert rho change

rho? r? its just 1/2
 2024-09-11 02:49:44 -04:00
comfyanonymous
9c5fca75f4 Fix lora issue. 2024-09-08 10:10:47 -04:00
comfyanonymous
32a60a7bac Support onetrainer text encoder Flux lora. 2024-09-08 09:31:41 -04:00
Jim Winkens
bb52934ba4
Fix import issue (#4815) 2024-09-07 05:28:32 -04:00
comfyanonymous
ea77750759 Support a generic Comfy format for text encoder loras. 
This is a format with keys like:
text_encoders.clip_l.transformer.text_model.encoder.layers.9.self_attn.v_proj.lora_up.weight

Instead of waiting for me to add support for specific lora formats you can
convert your text encoder loras to this format instead.

If you want to see an example save a text encoder lora with the SaveLora
node with the commit right after this one.
 2024-09-07 02:20:39 -04:00
doctorpangloss
25e636fb65 Qwen2 2024-09-06 17:44:08 -07:00
doctorpangloss
e8eab4dbc6 Fix tensor types 2024-09-06 11:04:32 -07:00
comfyanonymous
c27ebeb1c2 Fix onnx export not working on flux. 2024-09-06 03:21:52 -04:00
doctorpangloss
a4fb34a0b8 Improve language and compositing nodes 2024-09-05 21:56:04 -07:00
doctorpangloss
7e1201e777 Merge branch 'master' of github.com:hiddenswitch/ComfyUI 2024-09-05 09:30:45 -07:00
doctorpangloss
0ba08f273a Move comfy_extras nodes, fix pylint errors 2024-09-05 09:29:26 -07:00
doctorpangloss
db423f8013 Merge branch 'master' of github.com:comfyanonymous/ComfyUI 2024-09-05 09:23:00 -07:00
Benjamin Berman
fc02cd8373 Add fine tuned CLIP checkpoint 2024-09-05 01:24:17 -07:00
comfyanonymous
5cbaa9e07c Mistoline flux controlnet support. 2024-09-05 00:05:17 -04:00
doctorpangloss
ed33ab1e7d Support ProcessPoolExecutor to improve memory management 2024-09-04 17:03:22 -07:00
comfyanonymous
c7427375ee Prioritize freeing partially offloaded models first. 2024-09-04 19:47:32 -04:00
Jedrzej Kosinski
f04229b84d
Add emb_patch support to UNetModel forward (#4779) 2024-09-04 14:35:15 -04:00
doctorpangloss
c75b9964ab Fix Never on python 3.10 2024-09-04 09:35:10 -07:00
Silver
f067ad15d1
Make live preview size a configurable launch argument (#4649) 
* Make live preview size a configurable launch argument

* Remove import from testing phase

* Update cli_args.py
 2024-09-03 19:16:38 -04:00
doctorpangloss
38bcd9fcbd Merge branch 'master' of github.com:comfyanonymous/ComfyUI 2024-09-03 15:28:52 -07:00
comfyanonymous
483004dd1d Support newer glora format. 2024-09-03 17:02:19 -04:00
comfyanonymous
00a5d08103 Lower fp8 lora memory usage. 2024-09-03 01:25:05 -04:00
comfyanonymous
d043997d30 Flux onetrainer lora. 2024-09-02 08:22:15 -04:00
comfyanonymous
8d31a6632f Speed up inference on nvidia 10 series on Linux. 2024-09-01 17:29:31 -04:00
comfyanonymous
b643eae08b Make minimum_inference_memory() depend on --reserve-vram 2024-09-01 01:18:34 -04:00
comfyanonymous
935ae153e1 Cleanup. 2024-08-30 12:53:59 -04:00
Chenlei Hu
e91662e784
Get logs endpoint & system_stats additions (#4690) 
* Add route for getting output logs

* Include ComfyUI version

* Move to own function

* Changed to memory logger

* Unify logger setup logic

* Fix get version git fallback

---------

Co-authored-by: pythongosssss <125205205+pythongosssss@users.noreply.github.com>
 2024-08-30 12:46:37 -04:00
comfyanonymous
63fafaef45 Fix potential issue with hydit controlnets. 2024-08-30 04:58:41 -04:00
doctorpangloss
3f88282b6a Fix absolute imports 2024-08-29 18:38:58 -07:00
doctorpangloss
52230c24f2 Fix runwayml removing their huggingface repositories 2024-08-29 18:14:24 -07:00
doctorpangloss
1bc96a7a1b Fix #20 base path can now be set before folder paths are initialized, although all of this really has to be reworked 2024-08-29 18:02:36 -07:00
doctorpangloss
fd503d8a96 Merge branch 'master' of github.com:comfyanonymous/ComfyUI 2024-08-29 16:37:30 -07:00
comfyanonymous
6eb5d64522 Fix glora lowvram issue. 2024-08-29 19:07:23 -04:00
comfyanonymous
10a79e9898 Implement model part of flux union controlnet. 2024-08-29 18:41:22 -04:00
comfyanonymous
ea3f39bd69 InstantX depth flux controlnet. 2024-08-29 02:14:19 -04:00
comfyanonymous
b33cd61070 InstantX canny controlnet. 2024-08-28 19:02:50 -04:00
doctorpangloss
ccdbd957ef Fix pylint issues 2024-08-28 15:48:47 -07:00
doctorpangloss
9e8bb0b297 Add image tracing to SVG support using vtrace, python skia. The Skia library can be used for additional drawing tasks 2024-08-28 14:49:19 -07:00
doctorpangloss
46ffaa2f0d Fix Flux controlnets 2024-08-28 14:48:42 -07:00
comfyanonymous
d31e226650 Unify RMSNorm code. 2024-08-28 16:56:38 -04:00
comfyanonymous
38c22e631a Fix case where model was not properly unloaded in merging workflows. 2024-08-27 19:03:51 -04:00
doctorpangloss
54740d99d6 Upstream the chat templates 2024-08-27 12:58:40 -07:00
Chenlei Hu
6bbdcd28ae
Support weight padding on diff weight patch (#4576) 2024-08-27 13:55:37 -04:00
comfyanonymous
ab130001a8 Do RMSNorm in native type. 2024-08-27 02:41:56 -04:00
doctorpangloss
8615c86722 Merge branch 'master' of github.com:comfyanonymous/ComfyUI 2024-08-26 16:59:38 -07:00
doctorpangloss
27f4d70904 Fix pylint 2024-08-26 16:56:27 -07:00
doctorpangloss
f49bcd4f3c Upstream InstantX Union ControlNet support for Flux 2024-08-26 16:54:29 -07:00
comfyanonymous
2ca8f6e23d Make the stochastic fp8 rounding reproducible. 2024-08-26 15:12:06 -04:00
comfyanonymous
7985ff88b9 Use less memory in float8 lora patching by doing calculations in fp16. 2024-08-26 14:45:58 -04:00
comfyanonymous
c6812947e9 Fix potential memory leak. 2024-08-26 02:07:32 -04:00
doctorpangloss
48ca1a4910 Include Kijai fp8 nodes. LoRAs are not supported by nf4 2024-08-25 22:41:10 -07:00
doctorpangloss
69e6d52301 Fix tests 2024-08-25 19:55:18 -07:00
doctorpangloss
c4fe16252b Fix imports 2024-08-25 18:56:47 -07:00
doctorpangloss
7100603016 Register moves 2024-08-25 18:53:50 -07:00
doctorpangloss
5155a3e248 Merge WIP 2024-08-25 18:52:29 -07:00
doctorpangloss
d7b65c9f55 Add flux controlnet to known controlnets 2024-08-25 15:24:46 -07:00
Benjamin Berman
ad9c4a7237 Upstream nf4 nodes 2024-08-25 15:23:14 -07:00
comfyanonymous
9230f65823 Fix some controlnets OOMing when loading. 2024-08-25 05:54:29 -04:00
comfyanonymous
8ae23d8e80 Fix onnx export. 2024-08-23 17:52:47 -04:00
comfyanonymous
7df42b9a23 Fix dora. 2024-08-23 04:58:59 -04:00
comfyanonymous
5d8bbb7281 Cleanup. 2024-08-23 04:06:27 -04:00
comfyanonymous
2c1d2375d6 Fix. 2024-08-23 04:04:55 -04:00
Simon Lui
64ccb3c7e3
Rework IPEX check for future inclusion of XPU into Pytorch upstream and do a bit more optimization of ipex.optimize(). (#4562) 2024-08-23 03:59:57 -04:00
Scorpinaus
9465b23432
Added SD15_Inpaint_Diffusers model support for unet_config_from_diffusers_unet function (#4565) 2024-08-23 03:57:08 -04:00
comfyanonymous
c0b0da264b Missing imports. 2024-08-22 17:20:51 -04:00
comfyanonymous
c26ca27207 Move calculate function to comfy.lora 2024-08-22 17:12:00 -04:00
comfyanonymous
7c6bb84016 Code cleanups. 2024-08-22 17:05:12 -04:00
comfyanonymous
c54d3ed5e6 Fix issue with models staying loaded in memory. 2024-08-22 15:58:20 -04:00
comfyanonymous
c7ee4b37a1 Try to fix some lora issues. 2024-08-22 15:32:18 -04:00
David
7b70b266d8
Generalize MacOS version check for force-upcast-attention (#4548) 
This code automatically forces upcasting attention for MacOS versions 14.5 and 14.6. My computer returns the string "14.6.1" for `platform.mac_ver()[0]`, so this generalizes the comparison to catch more versions.

I am running MacOS Sonoma 14.6.1 (latest version) and was seeing black image generation on previously functional workflows after recent software updates. This PR solved the issue for me.

See comfyanonymous/ComfyUI#3521
 2024-08-22 13:24:21 -04:00
comfyanonymous
8f60d093ba Fix issue. 2024-08-22 10:38:24 -04:00
comfyanonymous
843a7ff70c fp16 is actually faster than fp32 on a GTX 1080. 2024-08-21 23:23:50 -04:00
comfyanonymous
a60620dcea Fix slow performance on 10 series Nvidia GPUs. 2024-08-21 16:39:02 -04:00
comfyanonymous
015f73dc49 Try a different type of flux fp16 fix. 2024-08-21 16:17:15 -04:00
comfyanonymous
904bf58e7d Make --fast work on pytorch nightly. 2024-08-21 14:01:41 -04:00
Svein Ove Aas
5f50263088
Replace use of .view with .reshape (#4522) 
When generating images with fp8_e4_m3 Flux and batch size >1, using --fast, ComfyUI throws a "view size is not compatible with input tensor's size and stride" error pointing at the first of these two calls to view.

As reshape is semantically equivalent to view except for working on a broader set of inputs, there should be no downside to changing this. The only difference is that it clones the underlying data in cases where .view would error out. I have confirmed that the output still looks as expected, but cannot confirm that no mutable use is made of the tensors anywhere.

Note that --fast is only marginally faster than the default.
 2024-08-21 11:21:48 -04:00
comfyanonymous
76369e991c Indentation. 2024-08-20 23:02:45 -07:00
Xrvk
bd18041d25 Add Flux model support for InstantX style controlnet residuals (#4444) 
* Add Flux model support for InstantX style controlnet residuals

* Refactor Flux controlnet residual step to a separate method

* Rollback minor change

* New format for applying controlnet residuals: input->double_blocks, output->single_blocks

* Adjust XLabs Flux controlnet to fit new syntax of applying Flux controlnet residuals

* Remove unnecessary import and minor style change
 2024-08-20 23:02:45 -07:00
doctorpangloss
3e54f9da36 Fix torch_dtype issues, missing DualCLIPLoader known model support 2024-08-20 23:00:12 -07:00
comfyanonymous
03ec517afb Remove useless line, adjust windows default reserved vram. 2024-08-21 00:47:19 -04:00
doctorpangloss
540c43fae7 Typings 2024-08-20 21:25:16 -07:00
comfyanonymous
510f3438c1 Speed up fp8 matrix mult by using better code. 2024-08-20 22:53:26 -04:00
comfyanonymous
ea63b1c092 Simpletrainer lycoris format. 2024-08-20 12:05:13 -04:00
comfyanonymous
9953f22fce Add --fast argument to enable experimental optimizations. 
Optimizations that might break things/lower quality will be put behind
this flag first and might be enabled by default in the future.

Currently the only optimization is float8_e4m3fn matrix multiplication on
4000/ADA series Nvidia cards or later. If you have one of these cards you
will see a speed boost when using fp8_e4m3fn flux for example.
 2024-08-20 11:55:51 -04:00
comfyanonymous
d1a6bd6845 Support loading long clipl model with the CLIP loader node. 2024-08-20 10:46:36 -04:00
comfyanonymous
83dbac28eb Properly set if clip text pooled projection instead of using hack. 2024-08-20 10:46:36 -04:00
comfyanonymous
538cb068bc Make cast_to a nop if weight is already good. 2024-08-20 10:46:36 -04:00
comfyanonymous
1b3eee672c Fix potential issue with multi devices. 2024-08-20 10:46:36 -04:00
comfyanonymous
9eee470244 New load_text_encoder_state_dicts function. 
Now you can load text encoders straight from a list of state dicts.
 2024-08-19 17:36:35 -04:00
comfyanonymous
045377ea89 Add a --reserve-vram argument if you don't want comfy to use all of it. 
--reserve-vram 1.0 for example will make ComfyUI try to keep 1GB vram free.

This can also be useful if workflows are failing because of OOM errors but
in that case please report it if --reserve-vram improves your situation.
 2024-08-19 17:16:18 -04:00
comfyanonymous
4d341b78e8 Bug fixes. 2024-08-19 16:28:55 -04:00
comfyanonymous
6138f92084 Use better dtype for the lowvram lora system. 2024-08-19 15:35:25 -04:00
comfyanonymous
be0726c1ed Remove duplication. 2024-08-19 15:26:50 -04:00
comfyanonymous
4506ddc86a Better subnormal fp8 stochastic rounding. Thanks Ashen. 2024-08-19 13:38:03 -04:00
comfyanonymous
20ace7c853 Code cleanup. 2024-08-19 12:48:59 -04:00
comfyanonymous
22ec02afc0 Handle subnormal numbers in float8 rounding. 2024-08-19 05:51:08 -04:00
comfyanonymous
39f114c44b Less broken non blocking? 2024-08-18 16:53:17 -04:00
comfyanonymous
6730f3e1a3
Disable non blocking. 
It fixed some perf issues but caused other issues that need to be debugged.
 2024-08-18 14:38:09 -04:00
comfyanonymous
73332160c8 Enable non blocking transfers in lowvram mode. 2024-08-18 10:29:33 -04:00
comfyanonymous
2622c55aff Automatically use RF variant of dpmpp_2s_ancestral if RF model. 2024-08-18 00:47:25 -04:00
Ashen
1beb348ee2 dpmpp_2s_ancestral_RF for rectified flow (Flux, SD3 and Auraflow). 2024-08-18 00:33:30 -04:00
comfyanonymous
d31df04c8a Indentation. 2024-08-17 23:00:44 -04:00
Xrvk
e68763f40c
Add Flux model support for InstantX style controlnet residuals (#4444) 
* Add Flux model support for InstantX style controlnet residuals

* Refactor Flux controlnet residual step to a separate method

* Rollback minor change

* New format for applying controlnet residuals: input->double_blocks, output->single_blocks

* Adjust XLabs Flux controlnet to fit new syntax of applying Flux controlnet residuals

* Remove unnecessary import and minor style change
 2024-08-17 22:58:23 -04:00
comfyanonymous
4f7a3cb6fb unet -> diffusion_models. 2024-08-17 21:31:04 -04:00
comfyanonymous
bb222ceddb Fix loras having a weak effect when applied on fp8. 2024-08-17 15:20:17 -04:00
comfyanonymous
fca42836f2 Add model_options for text encoder. 2024-08-17 11:17:20 -04:00
comfyanonymous
cd5017c1c9 calculate_weight function to use a different dtype. 2024-08-17 01:06:08 -04:00
doctorpangloss
870297a2ed Fix StringEnumRequestParameter 2024-08-16 15:55:06 -07:00
doctorpangloss
f1a096b3e1 Merges new frontend 
- fixes bfloat16 on cpu to numpy issues
 - extensions should go into comfy/web/extensions/javascript
 2024-08-16 15:46:11 -07:00
doctorpangloss
527ddb5ac8 Move model_filemanager 2024-08-16 14:32:13 -07:00
doctorpangloss
24a9eb2600 Update with our changes 2024-08-16 14:31:26 -07:00
doctorpangloss
f04b582744 Move inverse execution stuff 2024-08-16 14:31:00 -07:00
doctorpangloss
fb1feed1a2 Move commit registration 2024-08-16 14:30:27 -07:00
doctorpangloss
8284ea2fca WIP merge 2024-08-16 14:25:06 -07:00
comfyanonymous
83f343146a Fix potential lowvram issue. 2024-08-16 17:12:42 -04:00
doctorpangloss
a6a080487f Fix pylint issue with hydit, fix absolute versus relative imports 2024-08-16 13:06:33 -07:00
Matthew Turnshek
1770fc77ed
Implement support for taef1 latent previews (#4409) 
* add taef1 handling to several places

* remove guess_latent_channels and add latent_channels info directly to flux model

* remove TODO

* fix numbers
 2024-08-16 12:53:13 -04:00
doctorpangloss
7500d02af5 Improve language models and performance, adding a translation workflow example 2024-08-15 11:09:55 -07:00
comfyanonymous
5960f946a9 Move a few files from comfy -> comfy_execution. 
Python code in the comfy folder should not import things from outside it.
 2024-08-15 11:21:14 -04:00