12 Commits

Author	SHA1	Message	Date
Kohaku-Blueleaf	cdcf4119b3	[Trainer] training with proper offloading (#12189) ... * Fix bypass dtype/device moving * Force offloading mode for training * training context var * offloading implementation in training node * fix wrong input type * Support bypass load lora model, correct adapter/offloading handling	2026-02-10 21:45:19 -05:00
Kohaku-Blueleaf	a97c98068f	[Weight-adapter/Trainer] Bypass forward mode in Weight adapter system (#11958) ... * Add API of bypass forward module * bypass implementation * add bypass fwd into nodes list/trainer	2026-01-24 22:56:22 -05:00
comfyanonymous	bdb10a583f	Fix loras not working on mixed fp8. (#10899)	2025-11-26 00:07:58 -05:00
Kohaku-Blueleaf	7be2b49b6b	Fix LoRA Trainer bugs with FP8 models. (#9854) ... * Fix adapter weight init * Fix fp8 model training * Avoid inference tensor	2025-09-20 21:24:48 -04:00
Kohaku-Blueleaf	b20ba1f27c	Fix #9537 (#9576)	2025-08-27 12:45:02 -04:00
flybirdxx	4c3e57b0ae	Fixed an issue where qwenLora could not be loaded properly. (#9208)	2025-08-06 13:23:11 -04:00
Kohaku-Blueleaf	eb2f78b4e0	[Training Node] algo support, grad acc, optional grad ckpt (#9015) ... * Add factorization utils for lokr * Add lokr train impl * Add loha train impl * Add adapter map for algo selection * Add optional grad ckpt and algo selection * Update __init__.py * correct key name for loha * Use custom fwd/bwd func and better init for loha * Support gradient accumulation * Fix bugs of loha * use more stable init * Add OFT training * linting	2025-07-23 20:57:27 -04:00
Kohaku-Blueleaf	520eb77b72	LoRA Trainer: LoRA training node in weight adapter scheme (#8446)	2025-06-13 19:25:59 -04:00
Kohaku-Blueleaf	2ab9618732	Fix the bugs in OFT/BOFT moule (#7909) ... * Correct calculate_weight and load for OFT * Correct calculate_weight and loading for BOFT	2025-05-02 13:12:37 -04:00
Kohaku-Blueleaf	a8f63c0d5b	Support dora_scale on both axis (#7727)	2025-04-22 05:01:27 -04:00
Kohaku-Blueleaf	966c43ce26	Add OFT/BOFT algorithm in weight adapter (#7725)	2025-04-22 04:59:47 -04:00
Kohaku-Blueleaf	1f3fba2af5	Unified Weight Adapter system for better maintainability and future feature of Lora system (#7540)	2025-04-21 20:15:32 -04:00