supplemental.zip

<h1 id="heading-302">What is EbSynth?</h1><code>a portrait of a beautiful blonde american woman in all blue NASA astronaut suit holding an (ice cream cone:1.1) &lt;lora:add_detail:1&gt;</code>Download the attachments for the source video and some finished examples with different blend rates.Original (lowres gif, I could only fit part of the video in the size limit:)<img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/d0784d24-54fc-4701-bd22-aa7d2f04bed0/width=525/d0784d24-54fc-4701-bd22-aa7d2f04bed0.jpeg" />EbSynth is a software package for "Fast Example-based Image Synthesis and Style Transfer". It is a technique dating to 2018. The original github page for EbSynth is available here: <a target="_blank" rel="ugc" href="https://github.com/jamriska/ebsynth">https://github.com/jamriska/ebsynth</a>.By combining EbSynth with modern diffusion models, the process of creating stylized video clips can be streamlined. With EbSynth, frames around each keyframe are interpolated, instead of each frame being a seperate diffusion image like <a target="_blank" rel="ugc" href="https://civitai.com/articles/84/ai-videos-in-mov2mov-with-stable-diffusion">mov2mov</a>.EbSynth works best for short video clips with less motion. The EbSynth pipeline works by masking and selecting keyframes from the original video, applying a new style to the keyframes in img2img, then interpolating nearby frames with EbSynth. The most challenging aspect of working with EbSynth is managing the transition between keyframes. Download the supplemental materials for the source video and some finished examples with different blend rates.Selecting a video that requires fewer keyframes is better, because each key frame is likely to result in significant changes to the overall scene. Consider instead using <a target="_blank" rel="ugc" href="https://civitai.com/articles/84/ai-videos-in-mov2mov-with-stable-diffusion">mov2mov </a>for scenes with high motion.It's recommended to have completed the <a target="_blank" rel="ugc" href="https://civitai.com/articles/88/using-img2img-effectively-with-canny-controlnets">canny img2img </a>guide before using EbSynth. You must have canny controlnet installed for this guide.This guide is targeted at Windows, using the AUTOMATIC1111 webUI.<h1 id="heading-303">Installing:</h1><ol><li>Download and extract the <a target="_blank" rel="ugc" href=" https://ebsynth.com/">EbSynth program</a>. Make note of where you extract the files.</li><li>Install the EbSynth WebUi extension<ul><li>Extensions tab -&gt; install from URL -&gt; <a target="_blank" rel="ugc" href="https://github.com/s9roll7/ebsynth_utility">https://github.com/s9roll7/ebsynth_utility</a></li></ul></li><li>Install <a target="_blank" rel="ugc" href="https://ffmpeg.org/download.html">FFmpeg.</a></li><li>In Settings-&gt;ControlNet-&gt;Allow other script to control this extension<ul><li>apply and reload the webUI. You should have a new tab for "Ebsynth Utility".</li></ul></li></ol><h1 id="heading-304">Getting ready:</h1><ol><li>Downscale the video<ul><li>If the input video is too high resolution for your GPU, downscale the video. 720p works well if you have the VRAM and patience for it.</li><li>You can use FFmpeg to downscale a video with the following command:<pre><code>ffmpeg -i input.mp4 -vf "scale=-1:720" output.mp4</code></pre>In this example:<ul><li><code>ffmpeg</code> is the command to start the FFmpeg tool.</li><li><code>-i input.mp4</code> specifies the input file, in this case <code>input.mp4</code>.</li><li><code>-vf "scale=-1:720"</code> adjusts video's height to 720 pixels and will automatically adjust the width to maintain the aspect ratio. For vertical videos, use 1280 pixels for the height.</li><li><code>output.mp4</code> is the output file.</li></ul>Remember to replace <code>input.mp4</code> and <code>output.mp4</code> with your actual file names</li></ul></li><li>Select the EbSynth Utility tab.<img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/a29e6645-7473-4737-b01a-3e93c06e12b1/width=525/a29e6645-7473-4737-b01a-3e93c06e12b1.jpeg" /><ul><li>"Project Directory" is where the output for each step of the pipeline is created.</li><li>Original movie path doesn't have to be in project directory</li></ul></li></ol><h1 id="heading-653">Project pipeline steps:</h1>Every pipeline step has additional options under "configuration". Some of those are covered in this guide.<h3 id="heading-5274">1. Extract and mask video frames</h3><ul><li>extract frames</li><li>After extracting frames, it uses a model to create masks for each image</li><li>Don't interrupt the masking process or it will restart from 0.</li><li>Default settings work fine for most videos.--- GPU BOUND: TAKE A BREAK WHILE FRAMES ARE MASKED ---</li></ul><h3 id="heading-5275">2. Select keyframes for EbSynth</h3><ul><li>The utility will attempt to automatically select keyframes. Change the "Threshold of delta frame edge" to higher values to allow more varience between frames before it selects a new keyframe. Start with <code>Threshold of delta frame edge: 35</code>. If it outputs too many keyframes, try increasing the threshold and/or minimum keyframe gap</li><li>MAKE SURE to select "STEP 2" radio button to avoid re-running step 1.</li><li>Minimum Keyframe Gap: 10</li><li>Maximum Keyframe Gap: 1000 (33 seconds at 30FPS)</li><li>You can select keyframes manually by copying files from <code>video_frame</code> to <code>video_key</code></li><li>This step can be EXTREMELY TEDIOUS. For many videos, this process does not work very well regardless of what keyframes are selected. Your time is better spent selecting a video that needs only a few key frames, instead of selecting individual keyframes for EBsynth.</li><li>Approximately 0.3-2s per keyframe works well. Less than that causes too much flicker, and more than that causes too many video artifacts as EbSynth fails to interpolate over too many frames.</li></ul><h3 id="heading-5276">3. Generate img2img keyframes</h3><ol><li>Click "generate" to print instructions for img2img.<ul><li>Instructions are also printed in the terminal for easy reference</li><li>I use the following settings:<ul><li>For steps 5 and 6 I prefer to use the resolution of the input video</li><li>For step 7, follow my canny img2img guide to use higher denoising values <a target="_blank" rel="ugc" href="https://civitai.com/articles/88/using-img2img-effectively-with-canny-controlnets">https://civitai.com/articles/88/using-img2img-effectively-with-canny-controlnets</a></li><li>Select Checkpoint</li><li>Select sampler and # steps</li><li>Set canvas size</li><li>ControlNet settings (<a target="_blank" rel="ugc" href="https://civitai.com/articles/88/using-img2img-effectively-with-canny-controlnets">canny</a>)</li><li>Denoising: 0.75</li><li>Set weight to 1 in both the controlnet and ebsynth script "controlnet option weight"</li></ul></li></ul></li><li>Put one of the keyframes in the img2img box</li><li>Check the "Generation TEST" box and generate until you're happy with the test keyframe<ul><li>Note that the test generation does not use the mask from step 1, but it's still a good indication of how your prompt will perform over the set of keyframes.</li></ul></li><li>Uncheck the "TEST" box and "Generate". If you have a lot of keyframes, this will take a while.<ul><li>if you get <code>AttributeError: 'NoneType' object has no attribute 'convert'</code>, place any image in the img2img field and try again. It's a bug.</li></ul></li></ol>--- GPU BOUND: TAKE A BREAK WHILE KEYFRAMES GENERATE ---<ul><li>Inspect the keyframe output. You want the keyframes to look pretty similar, without major changes in outfit, accessories, shapes, background, etc. This might take multiple tries, don't rush this step.</li><li>For best results, target minimal variation between generated keyframes</li></ul><h3 id="heading-5277">3.5 Optional: Perform color correction on the keyframes</h3><ul><li>Go to Ebsynth Utility -&gt; configuration tab -&gt; Stage 3.5</li><li>Choose the best img2img keyframe for color matching and drop it in the window</li><li>Generate. A backup of the img2img keys is created at <code>st3_5_backup_img2img_key</code>. The color corrected images are placed in the <code>img2img_key</code> folder</li><li>Sometimes this makes the result worse, so only do it if there's a lot of color variation in the img2img keyframes</li><li>If you have a good monitor please comment on the differences between the color correction algorithms</li></ul><h3 id="heading-5278">4. Rescale img2img output</h3><ul><li>Skip this step if your img2img keyframes are the same resolution as the input video. Otherwise, click "Generate" and follow the printed instructions</li></ul><h3 id="heading-5279">5. Generate .ebs files.</h3><ul><li>Just click generate</li><li>These are used with the Ebsynth program</li><li>Due to GUI limitation of the EBsynth program only ~15 keyframes are contained in each .ebs file</li></ul><h3 id="heading-5280">6. Run EbSynth</h3><ol><li>Double click the .ebs file and select "open with"... and choose the Ebsynth.exe application. It will open with the keyframe points prefilled.</li><li>Click "Run All".<img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/35b4908f-f574-40f8-8dd8-ac61a09fe82e/width=525/35b4908f-f574-40f8-8dd8-ac61a09fe82e.jpeg" /><ul><li>If you generated multiple .ebs files in step 5, you will need run each file individually. It's possible to run multiple windows at a time, but beware that you can overload your system and invoke a BSOD this way, so don't go overboard.</li></ul></li></ol>--- CPU BOUND: TAKE A BREAK WHILE KEYFRAMES GENERATE ---<h3 id="heading-5281">7. Final video render</h3>Now pick the "blend rate" between each keyframe.<ul><li>Blend rate of 1 is often too high and causes "smeary" transitions.</li><li>Blend rate of 0 will perform 0 blending between keyframes and there will be an abrupt transition between each keyframe</li><li>Find a blend rate that keeps the most detail while minimizing judder between keyframes</li><li>The supplemental materials includes the original video, and blend rates 0, 0.5, and 1.</li><li>Please share tips in the comments!</li></ul>

25391f12-63b2-4b80-9ed2-111044344b24

Ebsynth Movie Tutorial: Interpolated img2img keyframes (Advanced)

sexual situations

physical violence

disturbing

male nudity

hanging

hate symbols

nazi party

revealing clothes

weapon violence

female swimwear or underwear

male swimwear or underwear

partial nudity

white supremacy

adult toys

graphic male nudity

illustrated explicit nudity

nudity

graphic violence or gore

graphic female nudity

pg-13

corpses

wide hips

convenient censoring

peeing

oral invitation

emaciated bodies

exposed female nipple

blowjob

female nudity

sexual activity

sexual intent

undressed

male underwear

female swimwear

genitals

female underwear

thick thighs

breasts out

strapless leotard

vore

breast out

one breast out

huge breasts

gigantic breasts

huge butt

covered nipples

hair over breasts

no panties

sitting on face

anal

dildo riding

downblouse

oral

porn

futanari

hentai

nude

lingerie

sexy

nsfw

suggestive

child on child

self injury

extremist

hate speech

diapers

urine

incest

scat

latex clothing

swimwear

bukkake

fellatio

cumshot

implied fellatio

eat_cum

cumdrip

cum in pussy

cum on face

after fellatio

cum on hair

cum on body

cum on tongue

cum on hands

cum in mouth

triple fellatio

autofellatio

fucked silly

cum on pussy

pov fellatio