CLIP guided style-gan can be a little wonky compared to diffusion. Try different learning rates and be sure to tinker with the prompt - adding/removing details that are distracting.

afiaka87/mannequin-gan-3-electric-boogaloo-2

Guide a StyleGAN3 trained on pictures of mannequins with CLIP.
Public
850
runs