Navigating Through Stable Diffusion Tools - by EDG

Welcome to Stable Diffusion.

Whether you are a beginner or an experienced user looking to study the classics, you are in the right place. Sit back and enjoy reading this article whose purpose is to cover the essential tools needed to achieve satisfaction during your Stable Diffusion experience.

Of course this article is based on the premise that you already have installed a Stable Diffusion webui such as this one here.

INTRODUCTION

There is certainly no shortage of content available on Civitai... Checkpoints, Textual Inversions, LoRAs, Lycoris, ControlNet poses, and many more...

I can imagine how a new user would feel overwhelmed by all the tera bytes of tools available! "What do I download first?" and "Should I buy an external disk drive?" are question that I can picture newbies asking themselves. It's like your first day at the gym, so many machines, so many ways to use them...

Without acquiring knowledge you will be at best, lost, at worst, disappointed and will uninstall SD (Stable Diffusion) missing out on all the fun and the utilities that it can bring to your life. Knowledge acquisition is a prerequisite to being a power user of anything, and I dare say this goes doubly for AI and bleeding edge technology.

It is incredibly fun and challenging. Many of us are losing sleep over questions such as. "How can I refine my prompt?", "How do I inpaint better?", "How do I use a LoRA?", "How do I train or improve the training of a LoRA?" Questions will bring other questions and before you know it SD will become a hobby that will take you value greatly in your life. I am here to help you achieve that through this article. I do not have the presumption to be able to give an answer to all your questions, but at the very least, I think I can spark something inside you that will make you want to know more and do more.

1- CHECKPOINTS: It all starts here!

Choosing your checkpoints are the basis of a good SD experience. It all starts here, you cannot do anything without checkpoints. This needs to be highlighted, they are the most basic tools in your SD user arsenal, therefore this is where we will start.

A checkpoint is a large file (coming into .ckpt or .safetensors) that weighs several giga bytes. It is a relatively large AI model from which you will generate the images that you want. Checkpoints are like people. They have their strong suits, their limitations, and their biases. Since downloading a lot of checkpoints would be very heavy on your hard drive, I would recommend you to pick your checkpoints like you would pick a team. Pick checkpoints that complement each other and whose style your like! Pick checkpoints you like to work with, test them, review them, and see how they respond to your prompting.

1a- Prompting with your checkpoints

As I said earlier, you cannot do ANYTHING without checkpoints, and certainly, you cannot do anything satisfactory without putting in time to test and choose your checkpoints. Some users like me have spent so much time testing checkpoints that we have even made our own custom mix! My mix can be found here. Yes, you can even make your own mix of checkpoints you like, further customizing your experience! This will be covered later in the guide.

The way you test your checkpoint and make the most of it is through prompting. This is a very important word in the AI word. It is so important that it has even become a paid job. Companies are starting to recruit Prompt Engineers. Look it up! I want you to take prompting seriously and learn to refine it. You may browse Civitai's images section to get inspiration and knowledge on prompting.

You cannot do anything without checkpoints, and you cannot do anything with checkpoints if you do not know how to prompt. I will therefore insist heavily that you learn to prompt and learn how the checkpoint you are using responds to your prompting. Indeed, you must be aware of the fact that there are different prompting styles and that some prompting styles are better suited to some checkpoints. You will be able to tell that with experience. Much trial and error awaits you!

1b- Refining a prompt with a checkpoint

Once you learn how to prompt, you may get pleasant images and interesting results. However you may not be completely done yet. If there are imperfections on your image, you might like to click the [Send to inpaint] and use the inpainting brush to modify your image. Note that this is best done with an "inpainting checkpoint" which is a checkpoint designed to make the most of the inpainting function. Ask other users for details or help if you struggle with inpainting.

There is also the possibility of locking the seed of your generation and trying to add or delete some words in your prompts to see if you can get an even better result! The very nature of AI generation is random, and those that have the most fun with it embrace the randomness. But, if it is a problem, you can lock the seed you liken play around with it, or slightly increase the variation of the seed with the extra option.

All in all, refining your prompt requires you to be knowledgeable and creative. People jump on LoRAs quite fast without pushing the limits of what checkpoints alone can do. Before you move on to these tools, make sure to be well acquainted with your checkpoint!

1c- Mixing your own checkpoint

At this stage, you have tested many checkpoints on civitai and have probably found some you like. But you feel like it's not enough. How can I have the most complete and customized checkpoint? This is where we start diving into advanced knowledge and notions such as "Checkpoint Merging". Note that you are perfectly able to merge checkpoints on the Checkpoint Merger tab of your SD webui. But if you want more control and customization possibilities, this tool is available to you.

Mixing your own checkpoints will require to select how much of each checkpoint you want to put into the final product. At this point you have to think like a cook in a kitchen. You have many different ingredients in front of you, each with its own distinct flavor and aftertaste. You will have to dose each ingredient carefully in order to come up with a pleasant result. Needless to say that finding the right mix may take hours, but those will be hours well spent. Instead of being random user #561265 of a popular checkpoint, you will have your own distinct style and that is very rewarding!

Once you have your own merged checkpoint, it does not necessarily end there. You can add LoRAs to it with the Supermerger tool that I linked you before hand to increase the customization even further. At this point, you should be a power user to make the most of it.

Speaking or LoRA, they will be at the core of the next article...

Stay tuned for more, and if you want to learn more about LoRA before the next article drops, feel free to check my tutorials, also consider commissioning me for a custom model or supporting me on ko-fi!

The work that I deliver is high quality, check my model page on civitai for more.

See you soon,