A lot of models struggle with the lip color, especially when not focusing on the face. When using anything but AOM3 you might want to inpaint them in/color manually and send through img2img.