This LoRA was inspired by a contest, related to the upcoming Year of the Snake.
(More details about the contest here: “Call for Submissions: Year of the Snake Resources!”)
Inspirations and main idea
I was wondered with art deco illustrations, when I was generating my images, using exLlama prompt enhancer, and OneButtonPrompt. This particular LoRA was created by crafting a prompt on a topic, “a woman with the snake”. I then used this prompt to generate 20 training images, that I tested on the site to verify whether they comply with the requirements of the contest. While the LoRA is in beta version, I may still try to fix it, whenever is possible.
Training data:
TBD.
Technical setup
Now technical details. I used a simple workflow with the following model:
and positive/negative promt fror CLIP as well.
I generated around 50 images, but only 20 of them passed the filter on the moment of uploading. I will upload the training data images once the LoRA is uploaded to the site. Please note that while I tried my best to not pass any red lines, in terms of the content, as it was required by the contest, still some images may have other rating. Nevertheless, I am uploading them as well, to be transparent. So potentially I may reconsider make a few more runs to get better results.
I used LoRA tagging workflow with Florence 2 tagger, and resized images to 512x768 (WxH).
Training workflow
Now to the training workflow. I used the official workflow from Kijai (GitHub - kijai/ComfyUI-FluxTrainer), based on Kohya script. I trained the LoRA with such settings — 20 images, number of steps — 1000 (I found that the best result was in between 200 and 400 steps, all others were seem to overfit the model (i.e., 500, 600 steps etc.). So, based on my observations, these values translate to 10 and 20 epochs. This number may vary from dataset to a dataset, but for my dataset it was around this value.
Now regarding the checkpoints used. I used Atomix FLUX Unet (v.1.0) for training, well, because it was only checkpoint in photorealistic style, I had in Unet format and FP8, so they may not fit training for photorealistic style. Regarding the training parameters — I used fp8 training format without offloading and b. More details are provided in the training workflow.
I generated the LoRA in few intervals — 200, 400, 500, 600, 800, 1000 steps. Steps over 400 were glitchy and deformed, so as I decided to remove them. The 200 steps LoRA looks promising because of more fine face details, but it had too many problems with hands.
LoRA deployment and testing
Now to deployment of the model. I tested it (and still testing to check any issues) using same Unet and Text encoder I used during training:
clip-L from Hugging Face from Flux-dev repository: black-forest-labs/FLUX.1-dev at main
T5xxl fp8 encoder FLUX.1 T5 Text Encoder
The best results so far I got with the following parameters:
Lora model weight — 1.0
Lora CLIP weight — 1.0
Steps — 15
CFG — 1.5
Sampler: Euler
Scheduler: simple
Since the LoRA was trained with tags from the initial training images, instead of trigger words you may use the tags section from the example prompt:
"A photo-realistic shoot from a front camera angle about a woman sitting in a chair with a snake coiled around her legs, wearing a black strapless dress and multiple necklaces. the image also shows a dark-skinned woman with dark hair, sitting in the middle of the image, with her upper body facing the viewer and her eyes looking directly at the camera. she has a confident expression and is wearing red lipstick. her body is slim and she has bare shoulders. she is sitting on a wooden chair in a dimly lit room with a brown curtain and an ornate archway in the background. the lighting is dramatic, casting shadows on her body and highlighting her curves. the woman is wearing black jewelry, including a necklace and earrings, and her feet are bare. the snake is wrapped around her body, with its scales visible. the overall mood is dark and mysterious, with a focus on the woman and the snake.
score_9, score_8_up, score_7_up, Stylized reinterpretation of Orientalist photography, masterpiece, best quality, 1girl, solo, breasts, looking at viewer, black hair, dress, jewelry, bare shoulders, sitting, full body, earrings, black eyes, necklace, barefoot, spread legs, feet, lips, toes, chair, black dress, curly hair, arm support, snake, interspecies"
Credits
Thanks to the developers of mentioned models and ComfyUI nodes, for inspiration in prompting and workflows. All credits for used models and workflows left for the respective authors (AlexLai, JayNL, kijai). Thanks to authors of other awesome nodes, models and tools not mentioned here, but which were essential to create this image.
Special thanks. I also want to say thanks to JayNL since his checkpoint was used to generate images for my LoRA and I give credits for creation of the LoRA to him as well.
Disclaimer on content
Since the checkpoint is in early beta stage, it can generate some content, that is not for all audiences, if used alonlside a checkpoint (e.g. dedistilled), if prompted. The LoRA does not depict a real person and serves only for testing purposes only.
License
The LoRA inherits the license from Atomix Flux (used in training workflow as Unet):
FLUX.1 [dev] Non-Commercial License .
The FLUX.1 [dev] Model is licensed by Black Forest Labs. Inc. under the FLUX.1 [dev] Non-Commercial License. Copyright Black Forest Labs. Inc.
IN NO EVENT SHALL BLACK FOREST LABS, INC. BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH USE OF THIS MODEL.