ComfyUI/comfy/ldm
Jukka Seppänen be95871adc
feat: Gemma4 text generation support (CORE-30) (#13376)
* initial gemma4 support

* parity with reference implementation

outputs can 100% match transformers with same sdpa flags, checkpoint this and then optimize

* Cleanup, video fixes

* cleanup, enable fused rms norm by default

* update comment

* Cleanup

* Update sd.py

* Various fixes

* Add fp8 scaled embedding support

* small fixes

* Translate think tokens

* Fix image encoder attention mask type

So it works with basic attention

* Handle thinking tokens different only for Gemma4

* Code cleanup

* Update nodes_textgen.py

* Use embed scale class instead of buffer

Slight difference to HF, but technically more accurate and simpler code

* Default to fused rms_norm

* Update gemma4.py
2026-05-02 22:46:15 -04:00
..
ace Support Ace Step 1.5 XL model. (#13317) 2026-04-07 03:13:47 -04:00
anima
audio
aura
cascade cascade: remove dead weight init code (#13026) 2026-03-17 20:59:10 -04:00
chroma
chroma_radiance
cogvideo Cogvideox (#13402) 2026-04-29 19:30:08 -04:00
cosmos
ernie Some optimizations to make Ernie inference a bit faster. (#13472) 2026-04-18 23:02:29 -04:00
flux Add a supports_fp64 function. (#13368) 2026-04-11 21:06:36 -04:00
genmo
hidream
hunyuan3d
hunyuan3dv2_1
hunyuan_video
hydit
kandinsky5
lightricks Implement block prefetch + Lora Async load + and adopt in LTX (Speedup!) (CORE-111) (#13618) 2026-05-02 19:23:24 -04:00
lumina
mmaudio/vae
models Add support for small flux.2 decoder (#13314) 2026-04-07 03:44:18 -04:00
modules feat: Gemma4 text generation support (CORE-30) (#13376) 2026-05-02 22:46:15 -04:00
omnigen
pixart
qwen_image
rt_detr CORE-13 feat: Support RT-DETRv4 detection model (#12748) 2026-03-28 23:34:10 -04:00
sam3 Disable sageattention for SAM3 (#13529) 2026-04-23 11:14:42 -07:00
supir feat: SUPIR model support (CORE-17) (#13250) 2026-04-18 23:02:01 -04:00
wan wan: vae: Fix light/color change (#13101) 2026-03-21 18:44:35 -04:00
common_dit.py
util.py