conceptmod tutorial - fire (updated ++ term) - train any lora with just text, no data required

Trained for 2000 steps on:
#|fire%{random_prompt}, fire:-0.1|fire++:guidance=2

animations use lora strength 0.0 to 1.1 with the trigger word fire

Here's how:
Use the runpod at https://runpod.io/gsc?template=8y3jhbola2&ref=xf9c949d

1. Create a pod

I chose a 3090. It needs > 20 GB ram.
Don't encrypt your volume, container disk defaults to 5 gb and volume disk to 50 gb, these are fine
click Continue

It will take a few minutes to download conceptmod, once Connect becomes enabled, you are ready to continue.

2. Login to the web console

"Connect", SSH or "Start web console" and connect to it.

Once you log in, it will install dependencies (take a minute) then output a welcome message.

3. Send over your the base model checkpoint that you want to train on:

Note: be sure to use a safetensors checkpoint.

Using https://github.com/runpod/runpodctl

on local:

runpodctl send mycheckpoint.safetensors

on pod:

cd /workspace/stable-diffusion-webui/models/Stable-diffusion/
runpodctl receive <code from send>

Note: You can also use scp, wget or a cloud storage attachment to transfer your model

4. Train on your phrase (takes 3 hours for 1000 steps)

cd /workspace/sd-scripts

python3 train-scripts/train-esd.py --prompt "#|fire%{random_prompt}, fire:-0.1|fire++:guidance=2" --train_method selfattn --ckpt_path /workspace/stable-diffusion-webui/models/Stable-diffusion/mycheckpoint.safetensors

It saves every 300 steps, which is about an hour.

Selecting a phrase

Look at the models here and find one to modify: https://civitai.com/tag/conceptmod?model=58873&sort=Newest

5. Extract the lora
bash /workspace/conceptmod/docker/extract_lora.sh /workspace/stable-diffusion-webui/models/Stable-diffusion/<mycheckpoint>.safetensors

The argument is your base checkpoint from (3).

6. (optional) Test the lora in webui

Your model (and the training intermediates) will now be available in webui as a lora. Select your base model from (3) and apply the lora to figure out what the strength should be.

Freeze the seed to manually see how lora strength changes the model.

For ease of use:

cd /workspace/stable-diffusion-webui/Lora
mv compvis-word_firefire%\{random_prompt\}-0.1-metho.safetensors fire.safetensors

7. (optional) Create animations to show how your lora changes the images


choice a) To create an animation on your prompt:

python3 lora_anim.py -s 0.0 -e 0.7 -l "fire" -p "fire prompt"

-s is starting lora strength

-e is ending lora strength

-l "fire" is your lora

-p "fire prompt" is your prompt

choice b) To create one animation using a top 80k prompt appended with your trigger (like these previews):

python3 lora_anim.py -s 0.0 -e 0.7 -l "fire" -lp ", fire"

prompt defaults to https://huggingface.co/datasets/Gustavosta/Stable-Diffusion-Prompts

-lp ", fire" is your trigger.

Run continuously for many videos

while true; do python3 lora_anim.py -s 0.0 -e 0.7 -l "fire" -lp ", fire"; done

Break with ctrl-c. The animations are in the v4 directory as mpv files.

Transfer your videos:

on pod:

runpodctl send v4

on local:
runpodctl receive <code from send>

8. Download the lora

on pod:

runpodctl send /workspace/stable-diffusion-webui/models/Lora/

on local:

runpodctl receive <code from send>

9. Stop and terminate the pod to stop paying money

on https://runpod.io/console.pods , stop and terminate the running pod

10. Post the lora to civitai with the tag conceptmod.

Include your training phrase for a 5 star review.

https://civitai.com/tag/conceptmod?model=58873&sort=Newest