[change logs]
25.04.18/v1.0 for start/end
Resolved an issue resulting in excessively lengthy final prompts; improved the coherence and visual connectivity for transitions between start and end frames, and added a translation node.
25.04.18/v1.0 for FramePack
Create a very simple prompt.
https://github.com/lllyasviel/FramePack
25.04.14/v1.1
Fixed an issue caused by an overly long and unnecessary final prompt, and adjusted to avoid consecutive API calls.
*25.04.15/v1.1a - Add translation node
25.03.19/v1.0
Fixed an issue where a single incorrect symbol was present in the LLM prompt. This is a minor change, but it could slightly improve issues that may occur when inputting text in languages other than English. Additionally, the default setting for the stream option has been changed from ON to OFF.
25.03.25/for start-end frame(beta) -> beta+ (Improved results by modifying some of the prompts)
kijai workflow
Analyses the start and end images and ultimately generates an appropriate prompt for use in the i2v start-end workflow. However, depending on the image or motion, the end frame may not work properly. (If you can input the additional motion correctly, you can reinforce the intermediate movement using the existing v1.0 workflow.)
Regarding censorship:
Gemini has image analysis censorship by default settings. However, by adding a special part to the prompt to bypass this, I've been able to force the analysis of quite a few images. But if you get an error, try deleting the ComfyUI cache (the default location is in the top right of the screen) and try again.
Using a custom LLM prompt, analyze the image and output the structure as a prompt suitable for the wan2.1 i2v model.
+While it can also be used in Hunyuan, it is recommended to exclude prompts related to camera motion.
Analysis Results - This is a prompt generated by analyzing an image. It signifies the meaning of the image rather than motion.
Image Analysis with Motion - This adds appropriate motion to the generated prompt based on the image.
[Optional] Motion Input - Enter custom motion. The LLM model will refer to what you input.
*You can enter anything as long as it's a language Gemini can understand.
A Gemini API key is required. (FREE)
Also, enter the API key issued in the JSON file located at ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui-ollamagemini\config.json
Test Generation (v1.0)
Test setting
-Model) wan 2.1 i2v 480p GGUF Q6_K
-LoRA 1) https://civitai.com/models/929497/aesthetic-quality-modifiers-masterpiece?modelVersionId=1498121
-LoRA 2) https://civitai.com/models/1264662/live-wallpaper-style
-480*701, 20steps, 64fps(4sec)
+LoRA was used with trigger words to assist with movement
Analysis Results
A digital painting illustration of an anime pale girl with short white hair decorated with a dark rose on her head and dark angel wings behind her back. She has a red halo with spikes floating above her. She wears a long black dress with gold trim and holds out her hand, with a ring on her finger, as she prepares to cast a spell. The background is a dark gradient of black and red tones. A full shot from a medium perspective.
[Optional] Motion Input
The surrounding blood effects spin around the girl. Her wings gently flutter. Her arms gently wrap around the effects.
Image Analysis with Motion (with Optional Input)
A digital painting of a pale woman with short white hair and piercing red eyes, adorned with a black rose. She stands with a confident posture, her left hand elegantly raised, displaying a delicate gold ring. The woman is dressed in a tight-fitting black dress with gold trim, complemented by matching gold bracelets and thigh-high stockings with floral prints. The surrounding blood effects spin around her. Her wings gently flutter. Her arms gently wrap around the effects. The background features a dark, blurry ambiance with a subtle red glow, enhancing the mysterious atmosphere. The image is a medium shot from a slightly high angle, giving her an imposing look.
Image Analysis with Motion (w/o Optional Input)
A digital painting illustration of an anime pale girl with short white hair decorated with a dark rose on her head and dark angel wings behind her back. She has a red halo with spikes floating above her head. She wears a long black dress with gold trim, a gold bracelet, and long black gloves, revealing a white tattoo on her leg. She holds out her hand, with a gold ring on her finger, as she prepares to cast a spell, her fingers slightly curved and radiating a faint glow. The background is a dark gradient of black and red tones with red energy lines. A full shot from a medium perspective.
Prompts with Motion (w/o optional input)
Prompts with Motion (with optional input)
Base Prompts (Analysis Results)
*The output is more significantly affected by the motion LoRA due to the simplified prompt.
Test Generation (for i2v start-end frame_Beta+)
Test setting
workflow default setting (480p model)
[Start Image]
An anime-style full body shot of a young girl with snow-white hair pulled into two pigtails tied with red ribbons. Her fair skin is complemented by her fully open, bright red eyes and a cheerful smile. She wears a traditional Japanese miko outfit, consisting of a white top with red accents and a matching red skirt, complete with decorative knots and a large bow. Her right hand is raised in a waving gesture. The background features a vividly colored Shinto shrine with red pillars and traditional lanterns, creating a festive atmosphere with stone pavement. A medium shot captures the scene.
[End Image]
An anime-style digital painting of a girl with fair skin and a slight smile with closed eyes, standing in front of a Shinto shrine. She has long, white hair styled in low pigtails adorned with red ribbons. Her traditional red and white "miko" outfit includes a "chihaya" top with red and white trim, and a matching red "hakama" skirt, accented by a bold red bow. She stands with her hands on her hips, exuding calm and confidence. The background features a traditional Japanese Shinto shrine with red pillars, a dark tiled roof, and hanging lanterns, with blurred figures in the distance. A medium shot captures the full composition of the scene.
[Final Prompts]
An anime-style video prompt portraying a young girl in a traditional Japanese miko outfit transitioning smoothly from a cheerful greeting to a state of serene contemplation. Initially, the girl stands with a bright, open smile and waving right hand, her snow-white hair in pigtails tied with red ribbons, set against a vividly colored Shinto shrine with red pillars and festive lanterns. Gradually, her waving hand lowers gently as her expression softens into a slight smile, and her eyes close serenely. She then places both hands on her hips, exuding calm confidence. Throughout the transition, the background remains a detailed Shinto shrine with red pillars, a dark tiled roof, and hanging lanterns, with blurred figures adding depth. The camera remains fixed in a medium shot, capturing the nuanced changes in expression and posture as she gracefully shifts from an active, welcoming pose to a peaceful, contemplative one.