HiDream Img2Img ComfyUI Workflow with settings that worked for me
This workflow enables advanced image-to-image generation using the HiDream model suite and Florence-2 prompt generator, designed for use with ComfyUI and Replicate.
Overview
- Image-to-image generation with HiDream diffusion model
- Florence-2 for prompt generation and captioning
- VAE encoding/decoding and advanced CLIP-based text encoding
- Negative prompt support for artifact reduction
- LOW VRAM MODE
Required Models & Credits
Diffusion Model
- hidream_i1_full_fp16.safetensors
Place in:ComfyUI/models/diffusion_models
Download
Thanks to HiDream.ai for the model!
For low VRAM user - GPU with less than 24GB VRAM:
- You can replace the standard Diffusion Model Loader with the custom node Unet LOADER from city96/ComfyUI-GGUF by city96.
- Download the HiDream-I1 Full GGUF model from: https://huggingface.co/city96/HiDream-I1-Full-gguf/tree/main Place all in: ComfyUI/models/unet
Text Encoders
Place all in: ComfyUI/models/text_encoders
- clip_g_hidream.safetensors
Download
- clip_l_hidream.safetensors
Download
- llama_3.1_8b_instruct_fp8_scaled.safetensors
Download
- t5xxl_fp8_e4m3fn_scaled.safetensors
Download
VAE
- ae.safetensors
Place in:ComfyUI/models/vae
Download
Florence-2 Prompt Generator (NO need to download - will be downloaded automatacally at runtime)
- Florence-2-large
Microsoft Florence-2
Thanks to MiaoshouAI for the correct implementation!
Usage
- Download all required models and place them in the correct directories as listed above.
- Drag the workflow image in ComfyUI
- Use the workflow to generate new images from your input images and prompts.
For low VRAM user - GPU with less than 24GB VRAM:
- You can replace the standard Diffusion Model Loader with the custom node Unet LOADER from city96/ComfyUI-GGUF by city96.
- Download the HiDream-I1 Full GGUF model from: https://huggingface.co/city96/HiDream-I1-Full-gguf/tree/main
- Follow the instructions in the custom node repository to set up and use the GGUF model with the Unet LOADER node.
Workflow Diagram
See the full workflow structure here:
WORKFLOW-HIDREAM-IMG2IMG.png
Acknowledgements
- HiDream.ai for the diffusion model and encoders
- Microsoft for Florence-2
- MiaoshouAI for the Florence-2 prompt generator implementation
- ComfyUI team for the UI and workflow engine
Thank you to all model creators and contributors!