About ComfyUI API.

Hi all.

In today’s digital landscape, the ability to connect and communicate seamlessly between  applications and AI models has become increasingly valuable. Comfy  UI offers a  user-friendly interface that enables the creation of  API surfers, facilitating the interaction with other  applications and AI models to generate images or videos. While this process may initially seem daunting, it is relatively straightforward with a basic understanding of programming skills. In this blog post, we will explore the steps involved in setting up an API surfer using Comfy UI and delve into the potential it holds for expanding your creative possibilities. By the way, Stability Matrix uses ComfyUI as a backend system!

Enabling Developer Mode and Saving API Format

To begin creating your  API surfer, you will need to install the Comfy  UI manager. Once installed, access the settings menu by clicking on the gear icon. Within the settings, enable the developer mode option. This enables the functionality to save your  workflows as  API formats. By saving your  workflow diagrams in this format, Comfy UI can run them from the backend of the surfer, allowing for seamless integration with other applications.

Comfy UI as a Web Server

Comfy UI acts as a web server, providing a platform to utilize its features. Upon starting Comfy UI, various command prompts and files are loaded, configuring your system and installing custom notes. The web interface is accessed through a localhost link, enabling users to interact with Comfy  UI via their web browsers. As we proceed, we will connect and utilize the provided ComfyUI script examples:

Directory of D:\ComfyUI\script_examples

basic_api_example.py

websockets_api_example.py

More scripts about ComfyUI API you can take from here https://github.com/yushan777?tab=repositories

If you will use API via websocket so you have to install websocket-client  from https://github.com/websocket-client/websocket-client before.

Customizing the API Clients

To communicate with the Comfy UI  web server, we need to create  API  client apps or machines. Although the server may be running on one machine, the Python code can be transferred to other client apps or machines. This allows for remote connections and interaction with the Comfy UI server. We will focus on a basic text-to-image prompt to demonstrate the process, utilizing the provided websocket API examples.

Editing Workflow and Prompts

Within the Comfy UI script examples, we locate the  workflow JSON format. However, we can discard the hard-coded JSON format and instead load our own  workflow JSON files. By opening the saved workflow API JSON file, we gain access to our customized workflow. It is essential to ensure the correct titles are assigned to the custom notes for easy management and identification. In our case, we modify the positive and negative text prompts to suit our requirements.

Expanding Functionality and Saving the Workflow

While our current focus is on text-to-image prompts, Comfy  UI offers flexibility for various  applications, such as video generation or enhancing image details. By saving our modified  workflow as an  API call, we ensure the proper format for future use. Remember to use the designated button for saving  API files rather than the regular save button.

The actual format of the API file is the same ComfyUI json, but without the XY-coordinates of the nodes that belong to this workflow. And it can also be loaded easy into the ComfyUI GUI.

Integrating API Clients with Workflow

Returning to the code editor, we can now establish the connection between the API clients and the workflow. By referencing the saved workflow API JSON file, we load the workflow data. This step eliminates the need for hard-coded JSON format and allows for customization. We modify the text prompts and other variables to align with our  workflow requirements, ensuring a seamless integration between the API clients and Comfy UI  server.

As it turned out, you can create your own node code just as easily. There is even an example, a template file called example_node.py.examples.

Here's an example of a simple node ImageSizer (script from the Internet):

import math

class ImageSizer:

    def __init__(self):

        pass

   @classmethod

    def INPUT_TYPES(s):

        return {

            "required": {

                "model_type": (["SD","SDXL"],),

                "aspect_ratio_width": ("INT",{

                    "default": 1,

                    "step":1,

                    "display": "number"

                }),

                "aspect_ratio_height": ("INT",{

                    "default": 1,

                    "step":1,

                    "display": "number"

                })

            }

        }

    RETURN_TYPES = ("INT","INT")

    RETURN_NAMES = ("Width", "Height")

    FUNCTION = "run"

    CATEGORY = "CodyCustom"

    def run(self, model_type, aspect_ratio_width, aspect_ratio_height):

        # Define the total pixel counts for SD and SDXL

        total_pixels = {

            'SD': 512 * 512,

            'SDXL': 1024 * 1024

        }

        # Calculate the number of total pixels based on model type

        pixels = total_pixels.get(model_type, 0)

        # Calculate the aspect ratio decimal

        aspect_ratio_decimal = aspect_ratio_width / aspect_ratio_height

        # Calculate width and height

        width = math.sqrt(pixels * aspect_ratio_decimal)

        height = pixels / width

        # Return the width and height as a tuple of integers

        return (int(round(width)), int(round(height)))

NODE_CLASS_MAPPINGS = {

    "ImageSizer": ImageSizer

}

NODE_DISPLAY_NAME_MAPPINGS = {

    "ImageSizer": "Image Sizer"

}

Conclusion

Comfy  UI provides a powerful platform for creating API surfers, enabling seamless communication and integration between applications and AI models. By following the steps outlined in this guide, you can harness the potential of Comfy UI to expand your creative possibilities. Whether it’s generating images, videos, or automating complex  workflows, Comfy UI empowers you to explore new horizons in the world of digital interaction.

Good luck.