If I ran stable diffusion I would controlnet loss ever so slightly into every image