[https://github.com/ramyma/A8R8\_ComfyUI\_nodes](https://github.com/ramyma/A8R8_ComfyUI_nodes)
This set of nodes were initially created to be used with [A8R8](https://github.com/ramyma/a8r8) as demonstrated [in this post](https://www.reddit.com/r/StableDiffusion/comments/1diug6c/a8r8_adding_attention_coupling_and_controlnet/), I've created a set of nodes to help with regional prompting and decided to make it usable from within Comfy to give back to the awesome community that helped creating it.
Acknowledgment: core implementation is derived from [https://github.com/laksjdjf/cgem156-ComfyUI/tree/main/scripts/attention\_couple](https://github.com/laksjdjf/cgem156-ComfyUI/tree/main/scripts/attention_couple) and https://github.com/Haoming02/sd-forge-couple.
Cheers!
Great idea of making regional prompting in Comfy more intuitive, I've used similar nodes but found they left quite a bit to be desired in regardless to readability.
As much as downloading nodes these days feels like russian roulette (with the recent drama), I've checked the python and as of commit 18a8564, nothing looks suspect - so I'll check these out at least.
Please continue to do this. The culture online in general now is just to blindly download. I try to audit code especially after XY Tools but my eyes are... untrained
Works pretty good, but somewhat hit and miss for me on getting the night if changing models. I'm also having more trouble getting it to do two characters standing next to each other. When comparing to [https://github.com/Davemane42/ComfyUI\_Dave\_CustomNode](https://github.com/Davemane42/ComfyUI_Dave_CustomNode) 's MultiAreaConditioning - I have less of a hit rate - posting my attempts below here
A pro tip is to use it with a scribble CN [https://huggingface.co/xinsir/controlnet-scribble-sdxl-1.0](https://huggingface.co/xinsir/controlnet-scribble-sdxl-1.0)
It will help you guide the composition.
When applying CN, how do you go about it in Comfy? I haven't had much luck with Apply ControlNet output conditioning feeding into Attention Couple Region. Do you just do one Controlnet condioning for the whole image and then pass it into the KSampler positive?
https://preview.redd.it/z924t7wnk09d1.png?width=1920&format=png&auto=webp&s=1b661e2395c9581610a54a723c58cc8b1ca1f5b2
How about that? :D
Global prompt is: photo in a golf court
I'm using A8R8 interface here, but it plugs into the same nodes.
Adding a scribble CN makes it a lot more steerable.
I'll post another image with the masks and scribble layer.
nice nice - definitely progress over mine, I had a prompt more focused on the scenery and with no implied interaction, can you make them consistently also? - like what is the hit rate in a batch of 9 for you? - and is it possible to still get them to be in a specific setting also - like under a streetlight late at night type of thing?
I feel like if we could consistently get to two characters, a specific setting and a specific interaction we are in a new category. We're slowly getting there.
Here is my test, using the prompt you also used, but a batch of 9 at 1152x896 and using newrealitiesv2, 30 steps, 7 cfg, dpmpp\_2m karras
Hit rate about half the time, but when it does it usually includes the added interaction. Which is neat, so I'll give you the same grade as Dave's solution now :)
https://preview.redd.it/v5ccw331709d1.png?width=1395&format=png&auto=webp&s=5314c2f17a2f31057317a73d2ca8487a58389e3e
https://preview.redd.it/k4denjv8q09d1.png?width=1336&format=png&auto=webp&s=f60ec930b2d24f4bd5be64896b0f20cbb5390970
With some detailing; same attention masks and prompts
Does this work if you have the two characters coming from two different Lora’s? I have been able to prompt for characters, but once I need them to come from Lora’s, they get mixed.
I haven't personally found a way to mask loras - I had more succes for all this using faceid in the ipadapter though it is not perfect either when starting to pile on multiple characters, actions, locations and loras for style - it'll do one character in a specific location and in any visual style brilliantly though and with relative ease through various resamplings and so on.
Thanks! That makes me feel like I am not crazy.
Now I am working on a solution based on the A8R8 nodes then using the LayerStyle nodes to do some compositing. The last step is a final low cfg to get some interaction between the two characters. Thats the plan anyway, currently its not working, but showing promise.
I can't determine from the sample images, but does this version of regional prompting tend to lower the quality and detail, possibly art style?
Experience with regional prompting in Auto1111, InvokeAI, and forge couple shows that this is usually the case. Only the latent couple (two-shot) in A1111 doesn't have this problem for whatever reason.
An example with anime images is that they tend to be smoothed of details.
It doesn't affect the quality in my tests, works as it normally would with a regular prompt.
Assisting it with a scribble CN helps guide the generation further as well.
Not a node per se, but Lt. Data's Workflow Component adds the Image Refiner feature to Comfy. You can use it to paint different colors on the canvas and then use the Mask From RGB/CMY/BW node (which is part of ComfyUI Essentials node package) to extract a mask from each color.
This is the most elegant solution for multi area prompting to date! Thank you.
QQ -
In the case of multiple LORAs or ControlNets, is there a way to keep their influence limited to their attention area? It seems when you go to do model merge and the like, the influence of the LORA and ControlNet get applied to the whole image.
Thanks!
You can use it with [https://github.com/Kosinkadink/ComfyUI-Advanced-ControlNet](https://github.com/Kosinkadink/ComfyUI-Advanced-ControlNet) to add attention masks for the CN nodes.
For multiple loras it's a bit trickier, it might need a special ksampler node AFAIK.
Let me know if you stumble upon something interesting.
You can also try A8R8 to use it with an easier interface than Comfy's.
[https://www.reddit.com/r/StableDiffusion/comments/1diug6c/a8r8\_adding\_attention\_coupling\_and\_controlnet/](https://www.reddit.com/r/StableDiffusion/comments/1diug6c/a8r8_adding_attention_coupling_and_controlnet/)
[https://www.reddit.com/r/StableDiffusion/comments/1dom7ds/a8r8\_new\_release\_v070\_with\_comfy\_integration/](https://www.reddit.com/r/StableDiffusion/comments/1dom7ds/a8r8_new_release_v070_with_comfy_integration/)
I added some use anywhere nodes and a different mask generation method and it helped clean things up.
https://preview.redd.it/ui28ssa5fz8d1.png?width=2631&format=png&auto=webp&s=33fba54ce6ca4c9e1d47086196a44e98b7eb4b0f
Thanks for the response. Even with the Advanced control net, my characters are getting mashed into one. I will keep working on it. The character is cute but should be two
https://preview.redd.it/vide1jaidz8d1.png?width=1536&format=png&auto=webp&s=1b57073c21ce51942c2d81cd2ca6b7308f6c8016
Actually that's all the model; no loras.
I was just showing how to get more than one character in the same generation.
I believe I came by a custom ksampler that allows using multiple loras at the same time, but perhaps the simpler solution would be to merge the loras into one and use the different trigger words in each region prompt separately.
[https://github.com/ramyma/A8R8\_ComfyUI\_nodes](https://github.com/ramyma/A8R8_ComfyUI_nodes) This set of nodes were initially created to be used with [A8R8](https://github.com/ramyma/a8r8) as demonstrated [in this post](https://www.reddit.com/r/StableDiffusion/comments/1diug6c/a8r8_adding_attention_coupling_and_controlnet/), I've created a set of nodes to help with regional prompting and decided to make it usable from within Comfy to give back to the awesome community that helped creating it. Acknowledgment: core implementation is derived from [https://github.com/laksjdjf/cgem156-ComfyUI/tree/main/scripts/attention\_couple](https://github.com/laksjdjf/cgem156-ComfyUI/tree/main/scripts/attention_couple) and https://github.com/Haoming02/sd-forge-couple. Cheers!
Beautiful, Ty!
Great idea of making regional prompting in Comfy more intuitive, I've used similar nodes but found they left quite a bit to be desired in regardless to readability. As much as downloading nodes these days feels like russian roulette (with the recent drama), I've checked the python and as of commit 18a8564, nothing looks suspect - so I'll check these out at least.
It's all open source, so feel free to audit as you please, thanks!
Exactly, I just wanted to help alleviate some community fears given I could give it another set of trained eyes.
Please continue to do this. The culture online in general now is just to blindly download. I try to audit code especially after XY Tools but my eyes are... untrained
Thank you!
Works pretty good, but somewhat hit and miss for me on getting the night if changing models. I'm also having more trouble getting it to do two characters standing next to each other. When comparing to [https://github.com/Davemane42/ComfyUI\_Dave\_CustomNode](https://github.com/Davemane42/ComfyUI_Dave_CustomNode) 's MultiAreaConditioning - I have less of a hit rate - posting my attempts below here
A pro tip is to use it with a scribble CN [https://huggingface.co/xinsir/controlnet-scribble-sdxl-1.0](https://huggingface.co/xinsir/controlnet-scribble-sdxl-1.0) It will help you guide the composition.
When applying CN, how do you go about it in Comfy? I haven't had much luck with Apply ControlNet output conditioning feeding into Attention Couple Region. Do you just do one Controlnet condioning for the whole image and then pass it into the KSampler positive?
https://preview.redd.it/80aiq0otgs9d1.png?width=1172&format=png&auto=webp&s=bf71cf15cbf118deae162b5651a4751c46f7e37c There you go.
Thanks! So basically use controlnet to drive the whole image guidance. Makes sense.
Yeah, in a sense. Keep in mind that Attention Couple patches the model directly, so the regional prompts are preserved.
You can also play with the global prompt weight, increasing it helps sometimes.
https://preview.redd.it/6nhkxz6dzy8d1.png?width=1206&format=png&auto=webp&s=e7ec3946d989ead97e043238ccca722dcf7e2256 A8R8
https://preview.redd.it/4w8473cxyz8d1.png?width=1425&format=png&auto=webp&s=d4b9503af5ae3266671831baddee8b9695aba4db
https://preview.redd.it/97lypuipzz8d1.png?width=1425&format=png&auto=webp&s=62fe947e1f77f01fad0ff3996139efc26e35b693
https://preview.redd.it/z924t7wnk09d1.png?width=1920&format=png&auto=webp&s=1b661e2395c9581610a54a723c58cc8b1ca1f5b2 How about that? :D Global prompt is: photo in a golf court I'm using A8R8 interface here, but it plugs into the same nodes. Adding a scribble CN makes it a lot more steerable. I'll post another image with the masks and scribble layer.
u/Emperorof_Antarctica
https://preview.redd.it/carsqfn1l09d1.png?width=1920&format=png&auto=webp&s=6a5c6cebbb8c5019c02c76feb4700c6dca242de4
https://preview.redd.it/5kcn2fl7l09d1.png?width=1920&format=png&auto=webp&s=86a82d4780d70e910334426e95d8c1d8ed2d2cc7
nice nice - definitely progress over mine, I had a prompt more focused on the scenery and with no implied interaction, can you make them consistently also? - like what is the hit rate in a batch of 9 for you? - and is it possible to still get them to be in a specific setting also - like under a streetlight late at night type of thing? I feel like if we could consistently get to two characters, a specific setting and a specific interaction we are in a new category. We're slowly getting there. Here is my test, using the prompt you also used, but a batch of 9 at 1152x896 and using newrealitiesv2, 30 steps, 7 cfg, dpmpp\_2m karras Hit rate about half the time, but when it does it usually includes the added interaction. Which is neat, so I'll give you the same grade as Dave's solution now :) https://preview.redd.it/v5ccw331709d1.png?width=1395&format=png&auto=webp&s=5314c2f17a2f31057317a73d2ca8487a58389e3e
https://preview.redd.it/k4denjv8q09d1.png?width=1336&format=png&auto=webp&s=f60ec930b2d24f4bd5be64896b0f20cbb5390970 With some detailing; same attention masks and prompts
haha thats great, but I need some more details on that and how high the hit rate is etc.
Hit rate is pretty high with a CN to guide it.
Check my other comment, I added some screenshots.
https://preview.redd.it/g68fhluizy8d1.png?width=1790&format=png&auto=webp&s=37cafecbd7c04bc5ba0920414f041e481f5272f1 Davemane42 Multiareaconditioning
Does this work if you have the two characters coming from two different Lora’s? I have been able to prompt for characters, but once I need them to come from Lora’s, they get mixed.
I haven't personally found a way to mask loras - I had more succes for all this using faceid in the ipadapter though it is not perfect either when starting to pile on multiple characters, actions, locations and loras for style - it'll do one character in a specific location and in any visual style brilliantly though and with relative ease through various resamplings and so on.
Thanks! That makes me feel like I am not crazy. Now I am working on a solution based on the A8R8 nodes then using the LayerStyle nodes to do some compositing. The last step is a final low cfg to get some interaction between the two characters. Thats the plan anyway, currently its not working, but showing promise.
What’s this software ?
It's a ComfyUI set of nodes
I can't determine from the sample images, but does this version of regional prompting tend to lower the quality and detail, possibly art style? Experience with regional prompting in Auto1111, InvokeAI, and forge couple shows that this is usually the case. Only the latent couple (two-shot) in A1111 doesn't have this problem for whatever reason. An example with anime images is that they tend to be smoothed of details.
It doesn't affect the quality in my tests, works as it normally would with a regular prompt. Assisting it with a scribble CN helps guide the generation further as well.
Are there any nodes that allow you to easily drag and drop regions or draw them in comfyui to assist with this?
Not a node per se, but Lt. Data's Workflow Component adds the Image Refiner feature to Comfy. You can use it to paint different colors on the canvas and then use the Mask From RGB/CMY/BW node (which is part of ComfyUI Essentials node package) to extract a mask from each color.
You could theoretically use mask editor to draw in your mask areas
This is the most elegant solution for multi area prompting to date! Thank you. QQ - In the case of multiple LORAs or ControlNets, is there a way to keep their influence limited to their attention area? It seems when you go to do model merge and the like, the influence of the LORA and ControlNet get applied to the whole image.
Thanks! You can use it with [https://github.com/Kosinkadink/ComfyUI-Advanced-ControlNet](https://github.com/Kosinkadink/ComfyUI-Advanced-ControlNet) to add attention masks for the CN nodes. For multiple loras it's a bit trickier, it might need a special ksampler node AFAIK. Let me know if you stumble upon something interesting. You can also try A8R8 to use it with an easier interface than Comfy's. [https://www.reddit.com/r/StableDiffusion/comments/1diug6c/a8r8\_adding\_attention\_coupling\_and\_controlnet/](https://www.reddit.com/r/StableDiffusion/comments/1diug6c/a8r8_adding_attention_coupling_and_controlnet/) [https://www.reddit.com/r/StableDiffusion/comments/1dom7ds/a8r8\_new\_release\_v070\_with\_comfy\_integration/](https://www.reddit.com/r/StableDiffusion/comments/1dom7ds/a8r8_new_release_v070_with_comfy_integration/)
I added some use anywhere nodes and a different mask generation method and it helped clean things up. https://preview.redd.it/ui28ssa5fz8d1.png?width=2631&format=png&auto=webp&s=33fba54ce6ca4c9e1d47086196a44e98b7eb4b0f
Thanks for the response. Even with the Advanced control net, my characters are getting mashed into one. I will keep working on it. The character is cute but should be two https://preview.redd.it/vide1jaidz8d1.png?width=1536&format=png&auto=webp&s=1b57073c21ce51942c2d81cd2ca6b7308f6c8016
https://preview.redd.it/1w873i6lxz8d1.png?width=1479&format=png&auto=webp&s=f3413b671f038816236f75fca6e2371e844de251
https://preview.redd.it/cupdcg5eyz8d1.png?width=1362&format=png&auto=webp&s=ff3a4a93d79ebef6e78c684537cfc3c846374495
That looks perfect! Are you doing a model merge after you add the loras to the workflow?
Actually that's all the model; no loras. I was just showing how to get more than one character in the same generation. I believe I came by a custom ksampler that allows using multiple loras at the same time, but perhaps the simpler solution would be to merge the loras into one and use the different trigger words in each region prompt separately.
What's the name of that KSampler?
Regardless, the new nodes are great, thank you for supporting the community. If I find a solution, I will post about it.