Week 8 · Lesson 2 of 10

Understanding the Node Graph and the Text-to-Image Pipeline

0% Complete

Core Idea

ComfyUI represents image generation as a graph of connected nodes. Each node performs one part of the process, and each connection passes a specific type of data to the next stage. The interface becomes manageable once the graph is understood as a visible pipeline rather than a collection of unrelated boxes.

The standard text-to-image pipeline has six functional stages:

  1. Load a checkpoint.
  2. Encode a positive prompt.
  3. Encode a negative prompt.
  4. Create an empty latent image.
  5. Sample the latent representation with a KSampler.
  6. Decode the latent result and preview or save the image.

How It Works

The Load Checkpoint node supplies three important outputs used in the workflow:

Model: The image-generation model used by the sampler.

CLIP: The component used by the text-encoding nodes to convert positive and negative prompt text into conditioning.

VAE: The component used to convert between a latent representation and a visible image.

Two CLIP Text Encode nodes are connected to the checkpoint's CLIP output. One holds the positive prompt: the visual content that should appear. The other holds the negative prompt: the content or qualities that should be avoided. Renaming them Positive Prompt and Negative Prompt makes their roles explicit.

The KSampler receives the model, positive conditioning, negative conditioning, and a latent image. It is the algorithmic stage that transforms the starting latent representation according to the prompts and sampling settings.

For text-to-image generation, the input is an Empty Latent Image. Despite the name, it is not a blank visible canvas. It represents random noise. The generation process begins with that noise and removes portions of it over a sequence of steps until the latent representation corresponds to the requested image.

The KSampler output is still latent data and cannot be viewed directly. A VAE Decode node converts it into an image. The VAE input normally comes from the checkpoint because most checkpoints in this workflow include a VAE. The decoded image is then connected to either Preview Image or Save Image.

Preview Image displays the output without automatically writing every generation to disk. A previewed image can still be saved manually by right-clicking it and selecting Save Image. Save Image both displays and automatically saves the result.

Nodes can be added in several ways. Right-clicking the canvas exposes the complete node menu, but the menu becomes difficult to navigate because of the number of available nodes. Double-clicking the canvas and searching by node name is faster. Dragging outward from an existing connector is also useful because ComfyUI proposes nodes that accept the relevant data type. Dragging from the checkpoint's CLIP connector, for example, proposes a CLIP Text Encode node.

Why It Matters

Advanced ComfyUI workflows are extensions of the same pipeline. Image-to-image replaces the empty latent input with a VAE-encoded image. ControlNet modifies the positive conditioning before it reaches the KSampler. InstantID also adds model and conditioning logic before sampling. Ultimate SD Upscale replaces the ordinary sampling stage with a tiled image-to-image process.

Without a clear model of the baseline graph, advanced workflows appear to be arbitrary wiring. With the baseline understood, each extension can be identified by the stage it modifies.

Practical Application

Build a text-to-image graph from an empty canvas:

  1. Double-click the canvas, search for Load Checkpoint, and select the required checkpoint.
  2. Drag from the CLIP output and create a CLIP Text Encode node.
  3. Create a second CLIP Text Encode node for the negative prompt.
  4. Connect both text encoders to the checkpoint's CLIP output.
  5. Rename the nodes Positive Prompt and Negative Prompt.
  6. Add a KSampler.
  7. Connect the checkpoint's Model output to the KSampler's Model input.
  8. Connect the positive text encoder to Positive and the negative text encoder to Negative.
  9. Add an Empty Latent Image and connect it to Latent Image.
  10. Add a VAE Decode node and connect the KSampler's latent output to Samples.
  11. Connect the checkpoint's VAE output to the VAE Decode node.
  12. Add Preview Image and connect the decoded image.
  13. Enter a positive prompt such as “medieval warrior, realistic, 8K, masterpiece, ultra detailed.”
  14. Enter negative terms such as “painting, cartoon, anime, watermark.”
  15. Queue the prompt and watch the active nodes highlight as execution moves through the graph.

A node can be cloned by right-clicking it and selecting Clone. Standard copy and paste also creates a duplicate. Ctrl+Shift+V is more useful when the copy should retain the same incoming connections as the original node. This is especially efficient when duplicating prompt encoders or later stages that share the same model, VAE, or conditioning inputs.

Trade-Offs and Limitations

The node graph exposes control but creates visual complexity. Searching for nodes, renaming them, and understanding connector types are necessary skills; otherwise, the graph becomes difficult to modify safely.

Preview Image reduces unwanted files but adds a manual save step for good outputs. Save Image is convenient when every output should be retained, but it also saves failed or experimental generations automatically.

The graph must remain type-correct. A visible image cannot be connected directly to a latent input, and latent data cannot be previewed without VAE decoding. These conversions are not optional visual details; they are required transformations between data representations.

Key Takeaway

The standard ComfyUI pipeline is checkpoint and prompts to KSampler, then latent output to VAE Decode, then image preview or saving. Mastering that data flow makes every later workflow easier to construct and troubleshoot.

Back to top