wangbo/ComfyUI
1
0
Fork 0
mirror of https://github.com/comfyanonymous/ComfyUI.git synced 2026-01-19 19:00:21 +08:00
Code Issues Actions 9 Packages Projects Releases Wiki Activity
4,341 Commits 62 Branches 110 Tags 1.3 GiB
cc299b83a3
Commit Graph

136 Commits

Author SHA1 Message Date
doctorpangloss
3d1d833e6f Merge branch 'master' of github.com:comfyanonymous/ComfyUI 2024-07-15 14:22:49 -07:00
comfyanonymous
391c1046cf More flexibility with text encoder return values. 
Text encoders can now return other values to the CONDITIONING than the cond
and pooled output.
 2024-07-10 20:06:50 -04:00
comfyanonymous
e44fa5667f Support returning text encoder attention masks. 2024-07-10 19:31:22 -04:00
comfyanonymous
bb663bcd6c Rename clip_t5base to t5base for stable audio text encoder. 2024-07-08 08:53:55 -04:00
comfyanonymous
80c4590998 Allow specifying the padding token for the tokenizer. 2024-07-06 00:06:49 -04:00
comfyanonymous
ce649d61c0 Allow zeroing out of embeds with unused attention mask. 2024-07-05 23:48:17 -04:00
doctorpangloss
a13088ccec Merge upstream 2024-07-04 11:58:55 -07:00
Mario Klingemann
eee815ec99
Update sd1_clip.py (#3684) 
Made token instance check more flexible so it also works with integers from numpy arrays or long tensors
 2024-06-19 16:42:41 -04:00
Max Tretikov
cbe69364db Lowercase T for torch.tensor 2024-06-14 15:58:59 -06:00
Max Tretikov
61b484d546 Fix lack of NotImplemented exceptions in model_base 2024-06-13 23:24:41 -06:00
Max Tretikov
d7f7d81b8a Change tokens to tensor of type long 2024-06-13 15:25:15 -06:00
doctorpangloss
cac6690481 Add known SD3 model files, merge branch 'master' of github.com:comfyanonymous/ComfyUI 2024-06-12 10:56:41 -07:00
comfyanonymous
0e49211a11 Load the SD3 T5xxl model in the same dtype stored in the checkpoint. 2024-06-11 17:03:26 -04:00
doctorpangloss
d778277a68 Merge with upstream and fix tests 2024-06-10 10:01:08 -07:00
comfyanonymous
742d5720d1 Support zeroing out text embeddings with the attention mask. 2024-06-09 16:51:58 -04:00
comfyanonymous
56333d4850 Use the end token for the text encoder attention mask. 2024-06-07 03:05:23 -04:00
doctorpangloss
3125366eda Improve compatibility with comfyui-extra-models, improve API 2024-05-30 16:50:34 -07:00
doctorpangloss
5a9055fe05 Tokenizers are now shallow cloned when CLIP is cloned. This allows nodes to add vocab to the tokenizer, as some checkpoints and LoRAs may require. 2024-05-16 12:39:19 -07:00
doctorpangloss
8741cb3ce8 LLM support in ComfyUI 
- Currently uses `transformers`
 - Supports model management and correctly loading and unloading models
   based on what your machine can support
 - Includes a Text Diffusers 2 workflow to demonstrate text rendering in
   SD1.5
 2024-05-14 17:30:23 -07:00
doctorpangloss
00728eb20f Merge upstream 2024-03-11 09:32:57 -07:00
comfyanonymous
65397ce601 Replace prints with logging and add --verbose argument. 2024-03-10 12:14:23 -04:00
doctorpangloss
272e3ee357 (broken) Merge branch 'master' of github.com:comfyanonymous/ComfyUI 2024-02-27 16:58:34 -08:00
comfyanonymous
03c47fc0f2 Add a min_length property to tokenizer class. 2024-02-26 21:36:37 -05:00
doctorpangloss
bd5073caf2 Merge branch 'master' of github.com:comfyanonymous/ComfyUI 2024-02-26 08:51:07 -08:00
comfyanonymous
8ac69f62e5 Make return_projected_pooled setable from the __init__ 2024-02-25 14:49:13 -05:00
comfyanonymous
c2cb8e889b Always return unprojected pooled output for gligen. 2024-02-25 07:33:13 -05:00
comfyanonymous
1cb3f6a83b Move text projection into the CLIP model code. 
Fix issue with not loading the SSD1B clip correctly.
 2024-02-25 01:41:08 -05:00
doctorpangloss
7520691021 Merge with master 2024-02-19 10:55:22 -08:00
comfyanonymous
97d03ae04a StableCascade CLIP model support. 2024-02-16 13:29:04 -05:00
doctorpangloss
82edb2ff0e Merge with latest upstream. 2024-01-29 15:06:31 -08:00
comfyanonymous
4871a36458 Cleanup some unused imports. 2024-01-21 21:51:22 -05:00
doctorpangloss
369aeb598f Merge upstream, fix 3.12 compatibility, fix nightlies issue, fix broken node 2024-01-03 16:00:36 -08:00
comfyanonymous
57926635e8 Switch text encoder to manual cast. 
Use fp16 text encoder weights for CPU inference to lower memory usage.
 2023-12-10 23:00:54 -05:00
comfyanonymous
9ac0b487ac Make --gpu-only put intermediate values in GPU memory instead of cpu. 2023-12-08 02:35:45 -05:00
comfyanonymous
fbdb14d4c4 Cleaner CLIP text encoder implementation. 
Use a simple CLIP model implementation instead of the one from
transformers.

This will allow some interesting things that would too hackish to implement
using the transformers implementation.
 2023-12-06 23:50:03 -05:00
doctorpangloss
3fd5de9784 fix nodes 2023-12-06 17:30:33 -08:00
comfyanonymous
be3468ddd5 Less useless downcasting. 2023-12-04 12:53:46 -05:00
Benjamin Berman
01312a55a4 merge upstream 2023-12-03 20:41:13 -08:00
comfyanonymous
728613bb3e Fix last pr. 2023-11-14 14:41:31 -05:00
Jianqi Pan
f2e49b1d57 fix: adaptation to older versions of pytroch 2023-11-14 14:32:05 +09:00
comfyanonymous
656c0b5d90 CLIP code refactor and improvements. 
More generic clip model class that can be used on more types of text
encoders.

Don't apply weighting algorithm when weight is 1.0

Don't compute an empty token output when it's not needed.
 2023-11-06 14:17:41 -05:00
comfyanonymous
b3fcd64c6c Make SDTokenizer class work with more types of tokenizers. 2023-11-06 01:09:18 -05:00
comfyanonymous
2a134bfab9 Fix checkpoint loader with config. 2023-10-27 22:13:55 -04:00
comfyanonymous
e60ca6929a SD1 and SD2 clip and tokenizer code is now more similar to the SDXL one. 2023-10-27 15:54:04 -04:00
comfyanonymous
434ce25ec0 Restrict loading embeddings from embedding folders. 2023-10-27 02:54:13 -04:00
doctorpangloss
e8b60dfc6e merge upstream 2023-10-06 15:02:31 -07:00
comfyanonymous
44361f6344 Support for text encoder models that need attention_mask. 2023-09-15 02:02:05 -04:00
comfyanonymous
fb3b728203 Fix issue where autocast fp32 CLIP gave different results from regular. 2023-09-11 21:49:56 -04:00
doctorpangloss
db673f7728 merge upstream 2023-08-29 13:36:53 -07:00
comfyanonymous
ec96f6d03a Move text_projection to base clip model. 2023-08-24 23:43:48 -04:00
comfyanonymous
e3d0a9a490 Fix potential issue with text projection matrix multiplication. 2023-08-24 00:54:16 -04:00
comfyanonymous
00c0b2c507 Initialize text encoder to target dtype. 2023-08-23 21:01:15 -04:00
comfyanonymous
f081017c1a Save memory by storing text encoder weights in fp16 in most situations. 
Do inference in fp32 to make sure quality stays the exact same.
 2023-08-23 01:08:51 -04:00
Benjamin Berman
d25d5e75f0 wip make package structure coherent 2023-08-22 11:35:20 -07:00
Benjamin Berman
dc4289dbb9 working install from git repo 2023-08-04 15:44:02 -07:00
comfyanonymous
c99d8002f8 Make sure the pooled output stays at the EOS token with added embeddings. 2023-08-03 20:27:50 -04:00
comfyanonymous
50b1180dde Fix CLIPSetLastLayer not reverting when removed. 2023-07-15 01:41:21 -04:00
comfyanonymous
46dc050c9f Fix potential tensors being on different devices issues. 2023-07-12 19:29:27 -04:00
comfyanonymous
606a537090 Support SDXL embedding format with 2 CLIP. 2023-07-10 10:34:59 -04:00
comfyanonymous
608fcc2591 Fix bug with weights when prompt is long. 2023-07-06 02:43:40 -04:00
comfyanonymous
ce35d8c659 Lower latency by batching some text encoder inputs. 2023-07-01 15:07:39 -04:00
comfyanonymous
b6a60fa696 Try to keep text encoders loaded and patched to increase speed. 
load_model_gpu() is now used with the text encoder models instead of just
the unet.
 2023-07-01 13:28:07 -04:00
comfyanonymous
97ee230682 Make highvram and normalvram shift the text encoders to vram and back. 
This is faster on big text encoder models than running it on the CPU.
 2023-07-01 12:37:23 -04:00
comfyanonymous
9920367d3c Fix embeddings not working with --gpu-only 2023-06-29 20:43:06 -04:00
comfyanonymous
20f579d91d Add DualClipLoader to load clip models for SDXL. 
Update LoadClip to load clip models for SDXL refiner.
 2023-06-25 01:40:38 -04:00
comfyanonymous
f87ec10a97 Support base SDXL and SDXL refiner models. 
Large refactor of the model detection and loading code.
 2023-06-22 13:03:50 -04:00
comfyanonymous
f7edcfd927 Add a --gpu-only argument to keep and run everything on the GPU. 
Make the CLIP model work on the GPU.
 2023-06-15 15:38:52 -04:00
comfyanonymous
bb1f45d6e8 Properly disable weight initialization in clip models. 2023-06-14 20:13:08 -04:00
comfyanonymous
0c7cad404c Don't initialize clip weights to default values. 2023-06-14 12:47:36 -04:00
comfyanonymous
23cf8ca7c5 Fix bug when embedding gets ignored because of mismatched size. 2023-06-08 23:48:14 -04:00
comfyanonymous
af9cc1fb6a Search recursively in subfolders for embeddings. 2023-05-05 01:28:48 -04:00
comfyanonymous
81d1f00df3 Some refactoring: from_tokens -> encode_from_tokens 2023-04-15 18:46:58 -04:00
comfyanonymous
719c26c3c9 Merge branch 'master' of https://github.com/BlenderNeko/ComfyUI 2023-04-15 14:16:50 -04:00
BlenderNeko
d0b1b6c6bf fixed improper padding 2023-04-15 19:38:21 +02:00
comfyanonymous
04d9bc13af Safely load pickled embeds that don't load with weights_only=True. 2023-04-14 15:33:43 -04:00
BlenderNeko
da115bd78d ensure backwards compat with optional args 2023-04-14 21:16:55 +02:00
BlenderNeko
752f7a162b align behavior with old tokenize function 2023-04-14 21:02:45 +02:00
comfyanonymous
334aab05e5 Don't stop workflow if loading embedding fails. 2023-04-14 13:54:00 -04:00
BlenderNeko
8489cba140 add unique ID per word/embedding for tokenizer 2023-04-13 22:01:01 +02:00
comfyanonymous
1718730e80 Ignore embeddings when sizes don't match and print a WARNING. 2023-04-04 11:49:29 -04:00
comfyanonymous
50099bcd96 Support multiple paths for embeddings. 2023-03-18 03:08:43 -04:00
comfyanonymous
00a9189e30 Support old pytorch. 2023-02-19 16:59:03 -05:00
comfyanonymous
137ae2606c Support people putting commas after the embedding name in the prompt. 2023-02-19 02:50:48 -05:00
comfyanonymous
324273fff2 Fix embedding not working when on new line. 2023-02-09 14:12:02 -05:00
comfyanonymous
f73e57d881 Add support for textual inversion embedding for SD1.x CLIP. 2023-01-29 18:46:44 -05:00
comfyanonymous
220afe3310 Initial commit. 2023-01-16 22:37:14 -05:00