Style and Composition transfer

[cubiq]

I'll try to use the Discussions to post about IPAdapter updates.

I just pushed an update to transfer Style only and Composition only. You find the new option in the weight_type of the advanced node. The style option (that is more solid) is also accessible through the Simple IPAdapter node. It works only with SDXL due to its architecture.

For the composition try to use a reference that has something to do with what you are trying to generate (eg: from a tiger to a dog), but it seems to be working well with pretty much anything

The style transfer works very well and can be improved with accurate prompting. In the corner you can see the reference image.

When I have time I'll try to add a node that lets you pick both the style and the composition image.

[zhangn77]

It's a wonderful work! Can I ask how to only refer the composition from the target image by IP-Adapter?

[cubiq]

set the weight type to "composition"

[zhangn77]

Sorry, I want to ask that the algorithms behind this component, could you please explain it? Because as I know, the IP-Adapter cannot decompose the composition from the target image.

[cubiq]

In SDXL the unet layers are very specialized, so I basically apply the weights only to specific layers

[zhangn77]

thank you! wonderful idea! Can it apply to sd 1.5 unet?

[xiaohu2015]

Well done, you made IP-Adapter great again!

[cubiq]

hehehe IPAdapter has always been great 😄

Have you tried to play with the blocks and the Face models? I don't seem to be able to find the right ones.

Here more examples of style and composition

[xiaohu2015]

face is only part of the image, maybe not work, maybe need to combine different blocks

[cubiq]

It seems to be possible for SD1.5 too. I need more time for testing (images upscaled by 1.5)

[benj-pan]

This feature excites me. However, I installed the latest version as you provided, but I don't find SDXL in weight_type.

[benj-pan]

May I ask how to solve this?