all the tests i made so far :) I have been using a machine on RunPod that has 8x RTX A6000 over a week now. I have done all the following experiments with each one being 3000 steps. I will hopefully update this post as I do more. All findings, grids, and best configs and 1-click installers for Windows, RunPod and Massed Compute shared here : https://www.patreon.com/posts/kohya-flux-lora-110293257 You can see saved safetensors files in below repos. Click load more button at the end to see entire repo files https://huggingface.co/MonsterMMORPG/FLUX_Kohya_SS_Massive_Research/tree/main https://huggingface.co/MonsterMMORPG/FLUX_Kohya_SS_Massive_Research_Part2/tree/main https://huggingface.co/MonsterMMORPG/FLUX_Kohya_SS_Massive_Research_Part3/tree/main https://huggingface.co/MonsterMMORPG/FLUX_Kohya_SS_Massive_Research_Part4/tree/main https://huggingface.co/MonsterMMORPG/FLUX_Kohya_SS_Massive_Research_Part4/tree/main All Tested Experiments List So Far 1) 16GB_Config.safetensors 2) 1e_04_bf16_128_rank-000170.safetensors 3) 1e_04_bf16_128_rank_full_bf16.safetensors 4) 1e_04_fp16_128_rank-000180.safetensors 5) 1e_04_fp16_128_rank_full_fp16-000180.safetensors 6) 1e_04_fp8_32_rank-000170.safetensors 7) 1e_04_fp8_bf16_accelerate_32_rank-000160.safetensors 8) 1e_04_fp8_bf16_accelerate_full_bf16_32_rank-000160.safetensors 9) 1e_04_fp8_full_fp16_32_rank-000170.safetensors 10) 2e_04_fp8_bf16_accelerate_full_bf16_32_rank.safetensors 11) 3e_04_fp8_bf16_accelerate_full_bf16_32_rank.safetensors 12) 6e_05_best_additive_sigma.safetensors 13) 6e_05_best_additive_uniform.safetensors 14) 6e_05_best_max_grad_norm_0.safetensors 15) 6e_05_best_raw_sigma.safetensors 16) 6e_05_best_raw_sigmoid.safetensors 17) 6e_05_best_sigma_scaled_sigma.safetensors 18) 6e_05_best_with_reg_images.safetensors 19) 6e_05_bf16_128_rank_full_bf16.safetensors 20) 7e_05_bf16_128_rank_full_bf16.safetensors 21) 8e_05_bf16_128_rank_full_bf16.safetensors 22) 8e_05_fp8_bf16_accelerate_full_bf16_32_rank.safetensors 23) 9e_05_bf16_128_rank_full_bf16.safetensors 24) 9e_05_fp8_bf16_accelerate_full_bf16_32_rank.safetensors 25) Best_v1.safetensors 26) Best_v1_3e_5.safetensors 27) Best_v1_4e_5.safetensors 28) Best_v1_512px.safetensors 29) Best_v1_512px_Lowest_VRAM.safetensors 30) Best_v1_5e_5.safetensors 31) Best_v1_5e_5_max_grad_norm_0.safetensors 32) Best_v1_Lowest_VRAM.safetensors 33) Best_v1_T5_attention_mask.safetensors 34) Best_v1_cfg_3_5.safetensors 35) Best_v1_max_grad_norm_0.safetensors 36) Best_v1_reg_0_1_percent.safetensors 37) Best_v1_reg_prior_lost_0_3.safetensors 38) Best_v1_reg_prior_lost_0_6.safetensors 39) Best_v1_reg_prior_lost_1_0_apply_t5_attention_mask.safetensors 40) Best_v2.safetensors 41) Best_v2_1280.safetensors 42) Best_v2_1280_256rank.safetensors 43) Best_v2_1536.safetensors 44) Best_v2_1536_256rank.safetensors 45) Best_v2_FP8.safetensors 46) Best_v2_FP8_LowVRAM.safetensors 47) Best_v2_Rank16.safetensors 48) Best_v2_Rank256.safetensors 49) Best_v2_Rank32.safetensors 50) Best_v2_Rank4.safetensors 51) Best_v2_Rank64.safetensors 52) Best_v2_Rank8.safetensors 53) Best_v2_Split_QKV.safetensors 54) Best_v2_joycaption.safetensors 55) Best_v2_joycaption_Split_QKV.safetensors 56) CivitAI_FLUX.safetensors