Dreambooth

[ehowton]

The below is my configuration with RTX 4090

Dreambooth LoRA --> Configuarion file --> Open --> [confignew.json] --> Load

mkdir -p img/##_loraname [number of training images x folder repeats (##) x Epochs / batch size = < 3k]

Utilities --> WD14 Captioning --> Image folder to caption --> [00_loraname]
Utilities --> Blip Captioning --> Image folder to caption --> [00_loraname] (trigger word suffix)

CAPTION IMAGES

(not needed if "Blip" above)

for x in `ls *.txt`; do dos2unix $x; done
for x in `ls *.txt`; do sed 's/$/, loratrigger/' $x > new$x; done
for x in `ls *.txt | awk '!/^new/ {print}'` ; do rm $x ;done
rename 's/new//g' *.txt


Dreambooth LoRA --> Source model --> Pretrained model name or path -->

animefull-final-pruned_(NovelAI) [for rpg/anime/fantasy]
v1-5-pruned [for realistic models/people]

Dreambooth LoRA --> Folders --> Image folder --> [img]
Dreambooth LoRA --> Folders --> Output folder --> [../]
Dreambooth LoRA --> Folders --> Model output name --> loraname

Dreambooth LoRA --> Training parameters

Train batch size --> 4 (six if recently rebooted and vRAM hasn't been touched)
Epoch --> 20
Save every N epochs --> 2
Mixed precision --> bf16 (nVidia only)
Save precision --> bf16 (nVidia only)
Number of CPU threads per core --> 2
Cache latents --> CHECKED
LR Scheduler --> polynomial
Optimizer --> AdamW8bit
Text Encoder learning rate --> 0.000045
Unet learning rate --> 0.0002
Network Rank (Dimension) --> 96
Network Alpha --> 192
Max resolution --> 768,768
Enable buckets --> CHECKED

Advanced configuration --> Clip Skip --> 1 (for v1-5-pruned)
Advanced configuration --> Clip Skip --> 2 (for NovelAI)
Gradient checkpointing --> UNCHECKED
Use xformers --> CHECKED
Don't upscale bucket resolution --> CHECKED
Noise offset --> 0.05

Sample images config --> Sample every n epochs --> 1
Sample prompts --> (loratrigger) --n low quality, worst quality, bad anatomy, --w 512 --h 512 --d 1 --l 7.5 --s 20

TRAIN MODEL

Reference articles:
https://aituts.com/stable-diffusion-lora/
https://www.reddit.com/r/StableDiffusion/comments/11r2shu/i_made_a_style_lora_from_a_photoshop_action_i/
https://www.zoomyizumi.com/lora-experiment-8/