17 Commits

Author	SHA1	Message	Date
Krishna Chaitanya	b53b10ea61	Fix Train LoRA crash when training_dtype is "none" with bfloat16 LoRA weights (#13145) ... When training_dtype is set to "none" and the model's native dtype is float16, GradScaler was unconditionally enabled. However, GradScaler does not support bfloat16 gradients (only float16/float32), causing a NotImplementedError when lora_dtype is "bf16" (the default). Fix by only enabling GradScaler when LoRA parameters are not in bfloat16, since bfloat16 has the same exponent range as float32 and does not need gradient scaling to avoid underflow. Fixes #13124	2026-03-24 23:53:44 -04:00
Kohaku-Blueleaf	5ebb0c2e0b	FP8 bwd training (#13121)	2026-03-24 20:39:04 -04:00
Kohaku-Blueleaf	20561aa919	[Trainer] FP4, 8, 16 training by native dtype support and quant linear autograd function (#12681)	2026-03-16 21:31:50 -04:00
rattus	8902907d7a	dynamic_vram: Training fixes (#12442)	2026-02-13 15:29:37 -05:00
Kohaku-Blueleaf	cdcf4119b3	[Trainer] training with proper offloading (#12189) ... * Fix bypass dtype/device moving * Force offloading mode for training * training context var * offloading implementation in training node * fix wrong input type * Support bypass load lora model, correct adapter/offloading handling	2026-02-10 21:45:19 -05:00
Kohaku-Blueleaf	a97c98068f	[Weight-adapter/Trainer] Bypass forward mode in Weight adapter system (#11958) ... * Add API of bypass forward module * bypass implementation * add bypass fwd into nodes list/trainer	2026-01-24 22:56:22 -05:00
Christian Byrne	bbb8864778	add search aliases to all nodes (#12035) ... * feat: Add search_aliases field to node schema Adds `search_aliases` field to improve node discoverability. Users can define alternative search terms for nodes (e.g., "text concat" → StringConcatenate). Changes: - Add `search_aliases: list[str]` to V3 Schema - Add `SEARCH_ALIASES` support for V1 nodes - Include field in `/object_info` response - Add aliases to high-priority core nodes V1 usage: ```python class MyNode: SEARCH_ALIASES = ["alt name", "synonym"] ``` V3 usage: ```python io.Schema( node_id="MyNode", search_aliases=["alt name", "synonym"], ... ) ``` ## Related PRs - Frontend: Comfy-Org/ComfyUI_frontend#XXXX (draft - merge after this) - Docs: Comfy-Org/docs#XXXX (draft - merge after stable) * Propagate search_aliases through V3 Schema.get_v1_info to NodeInfoV1 * feat: add SEARCH_ALIASES for core nodes (#12016) Add search aliases to 22 core nodes in nodes.py to improve node discoverability: - Checkpoint/model loaders: CheckpointLoader, DiffusersLoader - Conditioning nodes: ConditioningAverage, ConditioningSetArea, ConditioningSetMask, ConditioningZeroOut - Style nodes: StyleModelApply - Image nodes: LoadImageMask, LoadImageOutput, ImageBatch, ImageInvert, ImagePadForOutpaint - Latent nodes: LoadLatent, SaveLatent, LatentBlend, LatentComposite, LatentCrop, LatentFlip, LatentFromBatch, LatentUpscale, LatentUpscaleBy, RepeatLatentBatch * feat: add SEARCH_ALIASES for image, mask, and string nodes (#12017) Add search aliases to nodes in comfy_extras for better discoverability: - nodes_mask.py: mask manipulation nodes - nodes_images.py: image processing nodes - nodes_post_processing.py: post-processing effect nodes - nodes_string.py: string manipulation nodes - nodes_compositing.py: compositing nodes - nodes_morphology.py: morphological operation nodes - nodes_latent.py: latent space nodes Uses search_aliases parameter in io.Schema() for v3 nodes. * feat: add SEARCH_ALIASES for audio and video nodes (#12018) Add search aliases to audio and video nodes for better discoverability: - nodes_audio.py: audio loading, saving, and processing nodes - nodes_video.py: video loading and processing nodes - nodes_wan.py: WAN model nodes Uses search_aliases parameter in io.Schema() for v3 nodes. * feat: add SEARCH_ALIASES for model and misc nodes (#12019) Add search aliases to model-related and miscellaneous nodes: - Model nodes: nodes_model_merging.py, nodes_model_advanced.py, nodes_lora_extract.py - Sampler nodes: nodes_custom_sampler.py, nodes_align_your_steps.py - Control nodes: nodes_controlnet.py, nodes_attention_multiply.py, nodes_hooks.py - Training nodes: nodes_train.py, nodes_dataset.py - Utility nodes: nodes_logic.py, nodes_canny.py, nodes_differential_diffusion.py - Architecture-specific: nodes_sd3.py, nodes_pixart.py, nodes_lumina2.py, nodes_kandinsky5.py, nodes_hidream.py, nodes_fresca.py, nodes_hunyuan3d.py - Media nodes: nodes_load_3d.py, nodes_webcam.py, nodes_preview_any.py, nodes_wanmove.py Uses search_aliases parameter in io.Schema() for v3 nodes, SEARCH_ALIASES class attribute for legacy nodes.	2026-01-22 18:36:58 -08:00
comfyanonymous	bf7dc63bd6	skip_load_model -> force_full_load (#11390) ... This should be a bit more clear and less prone to potential breakage if the logic of the load models changes a bit.	2025-12-17 23:29:32 -05:00
Kohaku-Blueleaf	86dbb89fc9	Resolution bucketing and Trainer implementation refactoring (#11117)	2025-12-17 22:15:27 -05:00
comfyanonymous	ecdc8697d5	Qwen Image Lora training fix from #11090 (#11094)	2025-12-03 22:49:28 -05:00
Kohaku-Blueleaf	cc6a8dcd1a	Dataset Processing Nodes and Improved LoRA Trainer Nodes with multi resolution supports. (#10708) ... * Create nodes_dataset.py * Add encoded dataset caching mechanism * make training node to work with our dataset system * allow trainer node to get different resolution dataset * move all dataset related implementation to nodes_dataset * Rewrite dataset system with new io schema * Rewrite training system with new io schema * add ui pbar * Add outputs' id/name * Fix bad id/naming * use single process instead of input list when no need * fix wrong output_list flag * use torch.load/save and fix bad behaviors	2025-11-26 19:18:08 -05:00
Kohaku-Blueleaf	7be2b49b6b	Fix LoRA Trainer bugs with FP8 models. (#9854) ... * Fix adapter weight init * Fix fp8 model training * Avoid inference tensor	2025-09-20 21:24:48 -04:00
Kohaku-Blueleaf	eb2f78b4e0	[Training Node] algo support, grad acc, optional grad ckpt (#9015) ... * Add factorization utils for lokr * Add lokr train impl * Add loha train impl * Add adapter map for algo selection * Add optional grad ckpt and algo selection * Update __init__.py * correct key name for loha * Use custom fwd/bwd func and better init for loha * Support gradient accumulation * Fix bugs of loha * use more stable init * Add OFT training * linting	2025-07-23 20:57:27 -04:00
Kohaku-Blueleaf	1205afc708	Better training loop implementation (#8820)	2025-07-09 11:41:22 -04:00
Kohaku-Blueleaf	181a9bf26d	Support Multi Image-Caption dataset in lora training node (#8819) ... * initial impl of multi img/text dataset * Update nodes_train.py * Support Kohya-ss structure	2025-07-08 20:18:04 -04:00
comfyanonymous	ae75a084df	SaveLora now saves in the same filename format as all the other nodes. (#8538)	2025-06-15 03:44:59 -04:00
Kohaku-Blueleaf	520eb77b72	LoRA Trainer: LoRA training node in weight adapter scheme (#8446)	2025-06-13 19:25:59 -04:00