Getting VERY sick of my videos being removed for no reason. If anyone knows a decent ai community that are not retarded. Please let me know... this website is on its last legs.
Info here.
https://civitai.com/articles/10440
https://civitai.com/articles/11073
The blur noise group node in the flow for both images can be changed.
Sigma x is left right blur
sigma y is up down blur.
x seemed to work similar to noise if you want to try using only it. 0.25-.75
V7
Change to how things render.
- New option to use inserted latents or empty
when using empty (new default) This unlocks
- Clearer Base videos
- Clearer upscales
- easier extends (still not perfect, but the clearer base/upscales helps) color match is less needed unless above 1024
- Slightly larger upscales above 1152 i had slight blur issues. This may be able to be solved with upscale settings but i have not found anything that can clear up past this point.
- Smaller videos work well. 320x256 animates fine, and upscales 2x without problems and can flirt with doing 3x.
- Cleaned up the last of the node line spam.
All options still work with default empty latents, noise, first/last except add new frames. You need to insert to do that. And it works the same as before.
First last options seem to works better without latent inserts.
Same image in first/last generates loops, ususally needs noise to force it to move reliably. Backgrounds sometimes dont animate properly with this
If you want an idle animations and they wont stand still for you. first/last as same image works well if you add enough noise to the first image to animate between the 2 frames.
- added image selecter to image load to allow animated gif loading.
- this can be used as text to video without the inserts enabled.
6.1
Separated Perturbed layers.
Made 4 flows
main, manual prompt, florence only, and extrasfull
7 png example flows. These were made during setup of v6.1 They will probably not be setup properly, but they show settings i used for each.
Few notes for perturbed attention
V6
Removed notes
Added a very simple settings tester workflow you can use to understand the settings better.
If you want to try LTX but dont want to get all the extra stuff in the workflow, you can test with the test settings workflow, it needs only ltx nodes really.
Note : default override is set at 14.
10 works best IMO. set at 1.2 scale. 0 rescale. But can do rescale 0.2 to 0.5
10 holds animations better, while 14 can make them more fluid.
Both can be used with 10,14
I forgot to set that up properly.
Example video shows the diff between them.
New LLM Prompting
https://github.com/SeargeDP/ComfyUI_Searge_LLM
Model
Fine tuned LTX model has been paywalled with bait and switch bs. Its out there if you want to find it... but i cant support the bs. I am no longer going to be using the fine tuned model for my uses and will no longer link to his bullshit.
Also FYI i have tested the fine tune vs normal 9.1... i cant even say there is a difference. Like at all...
There is an issue when trying to extend anything. The size of the upscale is changed if not matching the proper sizes. Eg. 720 is changed to 704. This causes extends to have issues. Working on a mehood to just input your upscale size you want and then it will set base /upscale sizes.
Anyone interested in searge LLM prompting? I am experimenting with it and it seems very good.
V5.1
- Fixed the damn restart bug. Finally. 3 things...
- Added blur noise option (bug as its in the group with noise, its meant to be enabled with a noise option to dull the noise)
- added superprompt rewording for prompts.
- Added custom masked noise. I dont think it works well.
- Finished Sd1.5 frame fix, delete if you dont want it.
- made the perturbed attention optional both for base and upscale.
- added feta enhance node. I think it helps, but not really sure yet.
V5 - Fully redid interface now that i know main settings
- All options are the same, just now its much easier to follow and change.
- Pretty sure defaults now just work with noise/prompt/seed being main options for change.
Extends Beta
- Features 3 clip extends from base for total of 4 clips.
- Tail to head processing.
- Modular interface for expanding.
May be bugs, not many notes atm.
The Following node can help lower vram needed for the flow to work well, i will not be adding it to the flow as its a one off node that is not in the manager. Install only if you have issues with vram.
https://gist.github.com/city96/30743dfdfe129b331b5676a79c3a8a39
This node allows the offloading of the clip to cpu/system ram instead of fighting with the ltx model for room in your vram. I went from using 91% of my vram for 512x768 to using 53 and 61%. The text encode of the prompt is slower but the render works just as fast, and moreso if you were maxing out your ram before.
Sorry for the frequent changes, i wanted to get this out before i started on an extend workflow.
4.3 - small update
- changed vae decodes/encodes to tiled i dont notice any diff between this and normal besides it does not lock my pc when decoding large videos. You can change this if you prefer normal ones. (note - temporal size is max. It causes a skip in the video if not)
- Researched Perturbed attention - new base settings and explanation
- added color match node at end of base and upscale to attempt color match the video properly back to the original image.
- added main ref tagging option for extends.
- Fixed vae swap i did not know did not work. Added VAE decoder noise node.
4.2 - tried not to change or add much
- Added second add noise option (mtb nodes)
- removed apply stg node. either i was using it wrong or it was overridden by other nodes as it changed nothing in the renders for me.
- Changed upscaler Now upscales in 3 - 5 steps and is better (uncoupled CFG for upscale)
- fixed any scaling issues that could happen when the the input image is not divisible by 8.
- moved fps to settings area.
Note - I have tried the tiled new tiled vae, It works, you need to set your temporal size to max, or at least your frame number.
Note 2 - Try setting rescale to 0 (off) i have read details on how perturbed attention works and tested 0 and got better results. Will add more to notes next update.
4.1 bugfixes
- Fixed florence bypass to work with it disabled
- Fixed default upscale setting to work with defaults
- Fixed updscale(added note to explain what its doing and what can be changed)
If you have not updated your python lately and would like to have latest torch and such.
Inside comfy ui main folder is an update folder. This has update comfy and python dependencies. This updates everything.
V4 - useability Update
- Cleaned up the interface to make it simplified
- removed upscale options that should have been copied from base.
- Change how extend works
- Replaced all notes
- changed many small things with the latents for upscale
tips
Noise (blur) is the main factor in getting movement
Change seed if you dont like the movement you are getting
Inserts(addnewframes) / Stretch change how it is rendered, if one does not work try another.
Scale up gives more movement / Scale down holds to the image ref more.
Rescale up holds to the image ref more / down allows more movement
CFG is how much it tries to do your movement, raise this if you get movement but not what you asked for and down if the video is being burnt.
Number of frames changes settings needed
10 steps is enough... 20-30 are ok but can change the video content.
If upscaling SGM uniform 0.52 denoise works mostly when not complex movements, you can go higher here but it can change things.
How placed at option works ... i think
Latents placed are 8 latents long, meaning
0 is frame 1-8 are replaced with the image latents.
-1 is last frame meaning last 8 frames are replaced by the image latents.
Now... if you have 49 frames, you can use 49/8=6 (rounded down)
This means you can insert the frames into the middle of the video at 8 frame intervals at 6 points.
3 is mid way, 1-2 are frames 9-17 and 18-25 and same for 4-5-6.
V3
-changed interface
-added many examples for all settings
Have fun :)
https://github.com/Lightricks/LTX-Video
Search LTX in manager
Note to users. Previewing the video as its made is possible with an option in the comfy ui settings under VHS settings, Latent preview can be enabled. This shows animated previews as its rendering. Not sure if it slows things down,
Also note this needs to be disabled to use any of my other workflows as this setting makes them crash.
Confirmed working resolutions
512x360 wide and tall
360x360
V2
-added tools group with cropper/removebg groups
-changed prompt to allow pre/app text
-extended the replace functionality
-added upscale group
-added remove/replace after upscale
-added clip extend functionality from upscale or base video gens.
-added many test videos to show settings
-changed default attentions for many things
v1.1
-added video frame extract(request)
-fixed group enablers to properly turn things off
-added apply STG node
-florence prompt now optional can be turned off
-added on/off switch for sampler group to allow image prep.
-added more notes on settings
-included a few test videos with settings to show what they do better.
This is my first delve into LTX
This flow uses Florence and LTX to do single image to video and 2 image frame interpolation.
Few bugs i will fix tomorrow
All option boxes are set to max one, should be default, right click on the nodes and properties to change this
Tips -
Try turning off stretch in the sampler group, It seems to allow much longer videos.
with this disabled i was able to do 500 frames, 600 caused the video to turn brown...
Features
2 image interpolation
1 image to video
THis works, probably better than sparse controls for animate diff for 2 image interp. But its not perfect, the images need to be close and the same person/scene for the effect to work properly.
Many notes in the flow
Will require the 2 extra ltx nodes from the manager to use this.