Run ComfyUI Online:

ComfyUI in the Cloud with (up to) 8 x GPU's

Comfy UI Animation Install Guide For Vast.ai (v3)

Rent storage, gpu, install comfy, comfy manager, custom nodes, models & controlnet.
ComfyUI on Vast.ai
Sign up to Vast.ai here.


Setting up vast.ai template:

(Or Use AI Animated V3 Premade Template - here)

1. Select CUDA Template on Vast.ai:

  • Press edit.
  • Change to Version 11.8 of Cuda.

2. Docker Options:

-e JUPYTER_DIR=/ -p 3000:3000

3. On Start Script:

mkdir /workspace; env | grep _ >> /etc/environment; touch ~/.no_auto_tmux; sleep 1;
/start.sh

4. Open Jupyter, then open terminal.

5. Run below code blocks.


EXTRA INFO:

Find Extra Models here:
https://civitai.com/

Upscaling Models:

https://openmodeldb.info/

Sign up to Vast.ai here.

Block 1: System Setup

# Update the system
apt update -y
apt upgrade -y
# Install Python and venv apt-get install python3.11 -y apt install python3.10-venv -y
# Set up virtual environment
cd /workspace git clone https://github.com/comfyanonymous/ComfyUI cd /workspace/ComfyUI python3 -m venv venv source venv/bin/activate

Block 2: Dependency Installation

**The '--no-deps' line of code ensures the requirements.txt file that comes with Comfy does'nt install the newer (incompatible version of Pytorch, this may change in the future. Alteratively you could update the requirements text file to ensure it uses Pytorch 2.0.1)

# Install System-wide Dependencies
apt-get install ffmpeg -y apt-get install -y libgl1-mesa-glx
# Install Python Libraries in Virtual Environment (**Pytorch 2.1 is not yet compatible so use 2.0.1) pip install torch==2.0.1+cu118 torchvision==0.15.2+cu118 torchaudio==2.0.2 --index-url https://download.pytorch.org/whl/cu118
pip install xformers==0.0.20
pip install opencv-python
pip install safetensors
pip install trampoline
pip install -r requirements.txt --no-deps
# Explicitly install dependencies for other packages, skipping torch, torchvision, and torchaudio pip install einops transformers>=4.25.1 safetensors>=0.3.0 aiohttp accelerate pyyaml Pillow scipy tqdm psutil

Block 3: Add checkpoints2 folder and models i.e. Disney, realism etc. (find more on civitai.com).

Create a folder called checkpoints2 (**Jupyter can't open the default folder called 'checkpoints' as that name is reserved by the system)

mkdir -p /workspace/ComfyUI/models/checkpoints2

The below code tells terminal to download a model from CivitAI to the checkpoints2 folder and ensure it has the correct file name for the model, in this case it is 'disneyPixarCartoon_v10.safetensors'. Plus a dinsey style model as well.

cd /workspace/ComfyUI/models/checkpoints2
# wget -O disneyPixarCartoon_v10.safetensors https://civitai.com/api/download/models/69832

# wget -O epicrealism_naturalSinRC1VAE.safetensors https://civitai.com/api/download/models/143906? wget -O dreamshaper_8.safetensors https://civitai.com/api/download/models/128713

cd /workspace/ComfyUI/models/loras

wget -O pytorch_lora_weights.safetensors https://huggingface.co/latent-consistency/lcm-lora-sdv1-5/resolve/main/pytorch_lora_weights.safetensors?download=true

Block 4: Create a file to tell  ComfyUI to use the checkpoints2 folder created earlier in the code, instead of the standard 'checkpoints' folder.

# Create extra_model_paths.yaml
echo -e "comfyui:\n\n      checkpoints: models/checkpoints2/" > /workspace/ComfyUI/extra_model_paths.yaml

Block 4: Custom Nodes and related model installation

# Navigate to the custom_nodes directory
cd /workspace/ComfyUI/custom_nodes

# Clone the GitHub repositories for custom nodes
git clone https://github.com/ltdrdata/ComfyUI-Manager.git
git clone https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved.git
git clone https://github.com/Kosinkadink/ComfyUI-Advanced-ControlNet.git
git clone https://github.com/Kosinkadink/ComfyUI-VideoHelperSuite.git
git clone https://github.com/cubiq/ComfyUI_IPAdapter_plus.git
git clone https://github.com/aianimation55/ComfyUI-FatLabels.git
# Install models for AnimatedDiff
wget -P /workspace/ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved/models https://huggingface.co/guoyww/animatediff/resolve/main/mm_sd_v15_v2.ckpt
wget -P /workspace/ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved/models https://huggingface.co/guoyww/animatediff/resolve/main/mm_sd_v14.ckpt
# Install models for IPAdapter Plus
wget -P /workspace/ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus/models https://huggingface.co/h94/IP-Adapter/resolve/main/models/ip-adapter_sd15.bin
#Use the below model for IPAdator when the text prompt should have much more bearing on the generation.
wget -P /workspace/ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus/models https://huggingface.co/h94/IP-Adapter/blob/main/models/ip-adapter_sd15_light.bin
#IPAdaptor needs a Clipvision model to run successfully.
wget -P /workspace/ComfyUI/models/clip_vision https://huggingface.co/h94/IP-Adapter/resolve/main/models/image_encoder/model.safetensors

Block 5: Install control net models

# Code to add ControlNet models
wget -P /workspace/ComfyUI/models/controlnet https://huggingface.co/lllyasviel/ControlNet-v1-1/resolve/main/control_v11p_sd15_canny.pth
wget -P /workspace/ComfyUI/models/controlnet https://huggingface.co/lllyasviel/ControlNet-v1-1/resolve/main/control_v11p_sd15_canny.yaml
wget -P /workspace/ComfyUI/models/controlnet https://huggingface.co/lllyasviel/ControlNet-v1-1/resolve/main/control_v11f1p_sd15_depth.pth
wget -P /workspace/ComfyUI/models/controlnet https://huggingface.co/lllyasviel/ControlNet-v1-1/resolve/main/control_v11f1p_sd15_depth.yaml
wget -P /workspace/ComfyUI/models/controlnet https://huggingface.co/lllyasviel/ControlNet/resolve/main/models/control_sd15_hed.pth
wget -P /workspace/ComfyUI/models/controlnet https://huggingface.co/lllyasviel/ControlNet-v1-1/resolve/main/control_v11p_sd15_openpose.pth
wget -P /workspace/ComfyUI/models/controlnet https://huggingface.co/lllyasviel/ControlNet-v1-1/resolve/main/control_v11p_sd15_openpose.yaml
wget -P /workspace/ComfyUI/models/controlnet https://huggingface.co/lllyasviel/ControlNet-v1-1/resolve/main/control_v11p_sd15_lineart.pth
wget -P /workspace/ComfyUI/models/controlnet https://huggingface.co/lllyasviel/ControlNet-v1-1/resolve/main/control_v11p_sd15_lineart.yaml
#SEG CONTROLNET
wget -P /workspace/ComfyUI/models/controlnet https://huggingface.co/lllyasviel/ControlNet-v1-1/resolve/main/control_v11p_sd15_seg.pth
wget -P /workspace/ComfyUI/models/controlnet https://huggingface.co/lllyasviel/ControlNet-v1-1/resolve/main/control_v11p_sd15_seg.yaml

Block 6: Add upscale models i.e. Ersgan.

# Install upscale models
# Navigate to the custom_nodes directory
cd /workspace/ComfyUI/models/upscale_models
wget https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.1/RealESRGAN_x2plus.pth

Block 8: Start Script Setup For Easy Use:

The below code creates a start.sh file in the system root folder, so Comfy UI is loaded automatically when you start your instance on Vast.ai.

# Create and setup start.sh script
echo -e '#!/bin/bash\n\nsleep 10\napt update\napt install psmisc\nfuser -k 3000/tcp\ncd /workspace/ComfyUI/venv\nsource bin/activate\ncd /workspace/ComfyUI\npython main.py --listen 0.0.0.0 --port 3000' > /start.sh
chmod +x /start.sh

Block 9: Start (or restart Comfy UI).

# Start ComfyUI on port 3000
apt update
apt install psmisc
fuser -k 3000/tcp
cd /workspace/ComfyUI/venv
source bin/activate
cd /workspace/ComfyUI
python main.py --listen 0.0.0.0 --port 3000


Extra Custom Nodes:


MTB (Collection of mixed useful nodes including Background Remover).

Find out more about it here: https://github.com/melMass/comfy_mtb

Install MTB: Custom Nodes and related model installation

# Navigate to the custom_nodes directory
cd /workspace/ComfyUI/custom_nodes

# Clone the GitHub repositories for custom nodes
git clone https://github.com/melMass/comfy_mtb.git

Another Cool Model:
Ether Real Mix - Via Civit AI here.

cd /workspace/ComfyUI/models/checkpoints2
wget -O etherREALMIX_etherREALMIX31.safetensors https://civitai.com/api/download/models/156263?type=Model&format=SafeTensor&size=pruned&fp=fp16
cd /workspace/ComfyUI/models/vae
wget -O kl-f8-anime2.safetensors https://civitai.com/api/download/models/156263?type=VAE&format=Other

Restart Comfy UI:

# Start ComfyUI on port 3000
apt update
apt install psmisc
fuser -k 3000/tcp
cd /workspace/ComfyUI/venv
source bin/activate
cd /workspace/ComfyUI
python main.py --listen 0.0.0.0 --port 3000

COMFY UI & MODEL LINKS

Checkout the latest ComfyUI information about installation and compatibility.
ComfyUI Github Page.

Find Extra Models and more here:
https://civitai.com/

Upscaling Models:
https://openmodeldb.info/

EXAMPLE COMFY UI WORKFLOW

My basic Controlnet Workflow for ComfyUI, featuring Checkpoint, Model, Pytorhc Lora, IPAdaptor (for image reference), 2x Controlnets, Animatediff, KSampler and Video Conversion.

Download Zipped JSON file here. To then load into ComfyUI.



Example Workflow:

AI Animation Tutorial 2x Controlnet ComfyUI Workflow

EXAMPLE COMFY UI WORKFLOW

My basic Controlnet Workflow for ComfyUI, featuring Checkpoint, Model, Pytorhc Lora, IPAdaptor (for image reference), 2x Controlnets, Animatediff, KSampler and Video Conversion.

Download Zipped JSON file here. To then load into ComfyUI.