Sign In

Sparse Controlnet Animate Diff Video to Video +extras

228
2.8k
77
Updated: Jan 14, 2024
assets
Type
Workflows
Stats
2,129
Reviews
Published
Jan 14, 2024
Base Model
SD 1.5
Hash
AutoV2
6CDB1A79F9
default creator card background decoration
Gold Assets Badge
rocky533's Avatar
rocky533

Beta 3 - extended

replaced the color of the padding to black to stop coco from detecting nothing.

Added cropping to end

added load video group to just before the upscaling to allow the reloading of previous videos or anything this skips

more minor fixes

Beta 3

Fixed a lot with the first masking and proper background removal skip, it really frustrated me as nothing I rendered was nearly as good as when I first started. Now it's back to proper clarity.

Masking combined for upscales

few other small fixes.

Beta 2 - needs more testing but throwing it out as i think its better atm than the first and it fixes a few bugs.

Added

-3 mask options, coco is great but slow. Can use any of the three mask options at all steps they are needed. Not sure if they help with memory.  Bypass masking groups you are not needing.

If you're smart and know a better masking solution, you can insert your own group to mask off things

-added anime/cartoon mask creation option

-added face numbered preview to the reactor area to find your needed images

-Added lora groups to all model areas

-Added second save option

-Added Cutoff nodes.  - allows better color control stability in general.

-Many more notes - too many notes... I talk too much.

Edit - Updated to include reference videos used for samples in the main download with the workflow

This is a beta of my sparse control workflow for video to video.

Features

Sparse RGB or Scribble (any line art works) control nets

Full subject replacement

Full background replacement (both animated and static)

Face/body enhancements

Full upscale option

Cropping/padding allowing subject placements of subjects within the video.

Notes in the flow

Adding a few of the ref videos i used to the assets

Will add the requirements when I can, too tired tonight. :P