We were not blessed with images of too many minor characters, so we had to recreate them four times. We compromised.
It stabilizes at about 0.8 influence, but the pose is inflexible, as if it has been over-learned.
It seems the only way is to force the control net to change the pose.
I've only uploaded unrelated pictures, but I'd appreciate it if you could follow me on Twitter.