In today’s digital landscape, the ability to connect and communicate seamlessly between applications and AI models has become increasingly valuable. Comfy UI offers a user-friendly interface that enables the creation of API surfers, facilitating the interaction with other applications and AI models to generate images or videos. While this process may initially seem daunting, it is relatively straightforward with a basic understanding of programming skills. In this blog post, we will explore the steps involved in setting up an API surfer using Comfy UI and delve into the potential it holds for expanding your creative possibilities. By the way, Stability Matrix uses ComfyUI as a backend system!
Enabling Developer Mode and Saving API Format
To begin creating your API surfer, you will need to install the Comfy UI manager. Once installed, access the settings menu by clicking on the gear icon. Within the settings, enable the developer mode option. This enables the functionality to save your workflows as API formats. By saving your workflow diagrams in this format, Comfy UI can run them from the backend of the surfer, allowing for seamless integration with other applications.
Comfy UI as a Web Server
Comfy UI acts as a web server, providing a platform to utilize its features. Upon starting Comfy UI, various command prompts and files are loaded, configuring your system and installing custom notes. The web interface is accessed through a localhost link, enabling users to interact with Comfy UI via their web browsers. As we proceed, we will connect and utilize the provided ComfyUI script examples:
Directory of D:\ComfyUI\script_examples
basic_api_example.py
websockets_api_example.py
More scripts about ComfyUI API you can take from here https://github.com/yushan777?tab=repositories
If you will use API via websocket so you have to install websocket-client from https://github.com/websocket-client/websocket-client before.
Customizing the API Clients
To communicate with the Comfy UI web server, we need to create API client apps or machines. Although the server may be running on one machine, the Python code can be transferred to other client apps or machines. This allows for remote connections and interaction with the Comfy UI server. We will focus on a basic text-to-image prompt to demonstrate the process, utilizing the provided websocket API examples.
Editing Workflow and Prompts
Within the Comfy UI script examples, we locate the workflow JSON format. However, we can discard the hard-coded JSON format and instead load our own workflow JSON files. By opening the saved workflow API JSON file, we gain access to our customized workflow. It is essential to ensure the correct titles are assigned to the custom notes for easy management and identification. In our case, we modify the positive and negative text prompts to suit our requirements.
Expanding Functionality and Saving the Workflow
While our current focus is on text-to-image prompts, Comfy UI offers flexibility for various applications, such as video generation or enhancing image details. By saving our modified workflow as an API call, we ensure the proper format for future use. Remember to use the designated button for saving API files rather than the regular save button.
The actual format of the API file is the same ComfyUI json, but without the XY-coordinates of the nodes that belong to this workflow. And it can also be loaded easy into the ComfyUI GUI.
Integrating API Clients with Workflow
Returning to the code editor, we can now establish the connection between the API clients and the workflow. By referencing the saved workflow API JSON file, we load the workflow data. This step eliminates the need for hard-coded JSON format and allows for customization. We modify the text prompts and other variables to align with our workflow requirements, ensuring a seamless integration between the API clients and Comfy UI server.
As it turned out, you can create your own node code just as easily. There is even an example, a template file called example_node.py.examples.
Here's an example of a simple node ImageSizer (script from the Internet):
import math
class ImageSizer:
def __init__(self):
pass
@classmethod
def INPUT_TYPES(s):
return {
"required": {
"model_type": (["SD","SDXL"],),
"aspect_ratio_width": ("INT",{
"default": 1,
"step":1,
"display": "number"
}),
"aspect_ratio_height": ("INT",{
"default": 1,
"step":1,
"display": "number"
})
}
}
RETURN_TYPES = ("INT","INT")
RETURN_NAMES = ("Width", "Height")
FUNCTION = "run"
CATEGORY = "CodyCustom"
def run(self, model_type, aspect_ratio_width, aspect_ratio_height):
# Define the total pixel counts for SD and SDXL
total_pixels = {
'SD': 512 * 512,
'SDXL': 1024 * 1024
}
# Calculate the number of total pixels based on model type
pixels = total_pixels.get(model_type, 0)
# Calculate the aspect ratio decimal
aspect_ratio_decimal = aspect_ratio_width / aspect_ratio_height
# Calculate width and height
width = math.sqrt(pixels * aspect_ratio_decimal)
height = pixels / width
# Return the width and height as a tuple of integers
return (int(round(width)), int(round(height)))
NODE_CLASS_MAPPINGS = {
"ImageSizer": ImageSizer
}
NODE_DISPLAY_NAME_MAPPINGS = {
"ImageSizer": "Image Sizer"
}
Conclusion
Comfy UI provides a powerful platform for creating API surfers, enabling seamless communication and integration between applications and AI models. By following the steps outlined in this guide, you can harness the potential of Comfy UI to expand your creative possibilities. Whether it’s generating images, videos, or automating complex workflows, Comfy UI empowers you to explore new horizons in the world of digital interaction.
Good luck.
1 comment:
# __init__.py
from .ImageSizer import NODE_CLASS_MAPPINGS, NODE_DISPLAY_NAME_MAPPINGS
__all__ = ['NODE_CLASS_MAPPINGS', 'NODE_DISPLAY_NAME_MAPPINGS']
Post a Comment
А что вы думаете по этому поводу?