This works great. This is how you use it:
Follow this video like most people did -

https://www.youtube.com/watch?v=OxFcIv8Gq8o

Then instead of downloading the 50GB of models, download the 8 models from this page and use those instead.

I did that and am not using controlnet, it's awesome and big big thanks to uploader.