When depth and canny used at the same time it appears to ignore canny
#26
by john-glodon - opened
I've experimented with using both depth and canny at the same time [depth, canny] in that order. When I set the weights for depth to 0 it seems it completely ignores canny. Is this expected? Individually when I only pass one, it works properly. I wonder if this is an issue with diffusers or something else. Anyone else see this?
I also encountered this issue.
It seems to work if you load the controlnet union twice. Seems like that wasn't the way it was meant to be used, but maybe it is!
controlnet_model_union = 'Shakker-Labs/FLUX.1-dev-ControlNet-Union-Pro'
controlnet_depth = FluxControlNetModel.from_pretrained(controlnet_model_union, torch_dtype=torch.bfloat16)
controlnet_canny = FluxControlNetModel.from_pretrained(controlnet_model_union, torch_dtype=torch.bfloat16)
controlnet = FluxMultiControlNetModel([controlnet_depth, controlnet_canny])