If you find our articles informative, please follow me to receive updates. It would be even better if you could also follow our ko-fi, where there are many more articles and tutorials that I believe would be very beneficial for you!
如果你觉得我们的文章有料，请关注我获得更新通知，
如果能同时关注我们的 ko-fi 就更好了，
那里有多得多的文章和教程! 相信能使您获益良多.
For collaboration and article reprint inquiries, please send an email to [email protected]
合作和文章转载 请发送邮件至 [email protected]
by: Mazz
There are many ways to precisely control the shape of objects in Stable Diffusion, but most of them require some extensions or external software. Additionally, for objects with more random shapes, such as water, fire, dirt, or smoke, the generated results are often not realistic enough. This article will introduce a simple and effective method to generate objects of any shape at any position in the picture using inpaint within Stable Diffusion.
In many drawing software, various types of "brushes" can be selected. With the development of AI drawing technology, I believe that the types of "brushes" in the future will not only be limited to a few dozen or a few hundred options. Through AI technology, any brush that the artist wants can be generated in real-time. Although this technology has not yet appeared, we can control the stroke generation effect of
the inpaint in Stable Diffusion using prompts. Perhaps we can call it a "Prompt Brush”. This article will use various objects as brushes to generate text and demonstrate the effect of the "Prompt Brush”.
Let's start with "water."
Water
First, we need a base map. Our goal is to turn the letter "W" in the image into a water effect.
Place the image in img2img > inpaint. Set the following parameters:
prompt: masterpiece, best quality, realistic, water, waterdrop
Size: 768x768
Steps: 30, Sampler: DPM++ 2M Karras, CFG scale: 7,
Model: lyriel_v15,
Denoising strength: 0.89,
Mask blur: 4, Noise multiplier:1.05
Select [original] for Masked
content and [Only masked] forInpaint area.
If you don't want the generated effect to be related to the original image, then select [latent noise] for Inpaint area.I choose to generate four images at once for more options.Adjust the brush size and draw the first stroke to generate the image.
Select [Send to inpaint], put the generated result into inpaint without changing the parameters, continue with the previous steps for several times until the letter W is completely drawn.
After finishing, we get a picture like this:
We can see that the generated result is not perfect. At this point, we need to take the final and most important step. Select [img2img], adjust the generation size to 1024x1024, and adjust the denoising to around 0.5 for overall generation. (The denoising value is closely related to the generated size. If your image size is different, please adjust it according to the actual situation.)
Generated result:
n the end, we used a "Prompt Brush" to splash water and draw the letter W.
You might wonder why we didn't just draw the whole letter at once and then generate it directly. I don't recommend doing so because if the mask in inpaint is too complicated and messy, it often generates chaotic results. We need to draw stroke by stroke to achieve the best result, much like painting. Of course, you can also try to draw it with one stroke and see how the result turns out.
Next, let's try using prompt to turn the "brush" into other objects to generate letters.
Fire
masterpiece, best quality, realistic, floating (((fire))), ultra detailed
Negative prompt: brazier, bubble, bad_prompt_version2, illustration
Model: rmadaMergeSD21768_v50
Smoke
masterpiece, best quality, realistic, (((close up))), white smoke
Negative prompt: black, colorful, shadow, bad_prompt_version2, illustration, human,
face
Model: realismEngine_v10
Aurora
masterpiece, best quality, realistic, aurora
Negative prompt: bad_prompt_version2, illustration
Model: artius_v1
Dirt
masterpiece, best quality, realistic, (((dirt))), dust, macro photography, top view
Negative prompt: human, animal, bad_prompt_version2
Model: v1-5-pruned
Meat
masterpiece, best quality, realistic, meat, top view
Negative prompt: human, animal, bad_prompt_version2
Model: v1-5-pruned
Cloth
masterpiece, best quality, realistic, (white cloth), orange silk, detailed (((texture))),
(((no_human)))
Negative prompt: human, animal, bad_prompt_version2
Model: neurogenV11_v11
TIPS
> The process of generating text effects in this article is only an example. In fact, the "Prompt Brush" method can be applied in various situations. Text effects can be applied to various commercial graphics, and you can use this method to add various special effects to your work.
>In the case of having a base map, the choice of colors is closely related to the object you want to generate. This affects the effect you generate. If you want to generate "black flames" or "green meat", you may not have a good effect.
>The final step of generating the entire image again is very important, and your image can get the most satisfying effect at this step. This also means that you do not need to pay too much attention to details during the inpainting process.
> In inpainting, the size of the brush or the mask range is also related to the object properties. For objects with sharp edges, such as water and meat, the mask range can be smaller, and for smoke, the mask can be larger. For flames, auroras, etc., the lower edge of the mask can be contracted while the upper edge needs to be expanded, because these two objects spread from bottom to top. Therefore, when controlling the mask range, you need to think more to get better and more realistic effects.
>Writing a prompt requires some skills. For example, when depicting water, just using "water" is not enough. Adding "water drop" can generate the details of water instead
of lake surfaces and giant waves; when generating smoke, if you do not add "close up", you may generate more clouds and exploding smoke.
>This article recommends corresponding models for the mentioned objects. Each model will be better at describing certain objects, and you may notice that I used v1-5
primitive models when drawing soil and meat. So don't ignore every model and try more, and you will find the most suitable model.