Okay, so hopefully this is actually the final version! After reading a few different papers on the subject, I've completely changed my LoRA baking process, including switching back to NAI from AnyLoRA, changing several different settings which influence the learning rate, lowering the total steps to be closer to 1k, and going through and editing the tags for my dataset manually!
Use this version at a weight of 1.0, it should just work for the most part, as I have also now used multiple checkpoints to generate images and ensure it works well on a variety of them. Sometimes other LoRA make the hair different colors so, as always, adjust tags as necessary.
All images for this version are generated using HiRes-Fix and ADetailer.