Proxmox LCX + ComfyUI + SDV IMG2VID XT1.1

Using a Proxmox Virtual Enviroment Linux Container (PVE LCX) I got Stable Diffusion Video creation working with a Nvidia Tesla P4 with only 8Gb of VRAM.

Created an unprivileged LXC with the Debian 12 image, and add Nvidia cgroup2 and mount entries into LXC config file:

arch: amd64
cores: 16
features: nesting=1
memory: 16384
ostype: debian
rootfs: local-zfs:subvol-7011-disk-0,size=32G
swap: 512
unprivileged: 1
lxc.cgroup2.devices.allow: c 195:0 rwm      # /dev/nvidia0
lxc.cgroup2.devices.allow: c 195:255 rwm    # /dev/nvidiactl
lxc.cgroup2.devices.allow: c 195:254 rwm    # /dev/nvidia-modeset
lxc.cgroup2.devices.allow: c 510:0 rwm      # /dev/nvidia-uvm
lxc.cgroup2.devices.allow: c 510:1 rwm      # /dev/nvidia-uvm-tools
lxc.cgroup2.devices.allow: c 226:0 rwm      # /dev/dri/card0
lxc.cgroup2.devices.allow: c 226:128 rwm    # /dev/dri/renderD128
lxc.cgroup2.devices.allow: c 235:1 rwm      # /dev/nvidia-caps/nvidia-cap1
lxc.cgroup2.devices.allow: c 235:2 rwm      # /dev/nvidia-caps/nvidia-cap2
lxc.mount.entry: /dev/nvidia0 dev/nvidia0 none bind,optional,create=file
lxc.mount.entry: /dev/nvidiactl dev/nvidiactl none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-modeset dev/nvidia-modeset none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-uvm dev/nvidia-uvm none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-uvm-tools dev/nvidia-uvm-tools none bind,optional,create=file
lxc.mount.entry: /dev/dri/card0 dev/dri/card0 none bind,optional,create=file
lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-caps/nvidia-cap1 dev/nvidia-caps/nvidia-cap1 none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-caps/nvidia-cap2 dev/nvidia-caps/nvidia-cap2 none bind,optional,create=file

Update and install dependencies inside the LXC:

apt update && apt upgrade -y

apt install -y build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev xz-utils tk-dev libffi-dev liblzma-dev python3-openssl git libdb-dev gdbm-dev

Installed the Nvidia Linux drivers and made sure it sees using the nvidia-smi command inside the LXC:

./NVIDIA-Linux-x86_64-570.86.15.run --no-kernel-modules

reboot

nvidia-smi

Removed the Python 3.11 inside the LXC:

apt remove python3.11-minimal

apt autoremove

Build and install Python 3.12 from source inside the LXC:

cd /opt/

wget wget https://www.python.org/ftp/python/3.12.0/Python-3.12.0.tgz

tar zxvf Python-3.12.0.tgz

rm Python-3.12.0.tgz

cd /opt/Python-3.12.0/

./configure --enable-optimizations

make -j 16

make altinstall

pip3.12 install --upgrade pip

Clone ComfyUI and get it going inside the LXC:

cd /opt/

git clone https://github.com/comfyanonymous/ComfyUI.git

cd /opt/ComfyUI

pip3.12 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu126

pip3.12 install -r requirements.txt

cd /opt/ComfyUI/models/checkpoints

wget https://huggingface.co/genai-archive/stable-diffusion-v1-5/resolve/main/v1-5-pruned-emaonly.fp16.safetensors

cd /opt/ComfyUI/

/usr/local/bin/python3.12 /opt/ComfyUI/main.py --port 80 --lowvram --listen

Test and check it can run the shipped workflow.

Now add the SDV Xt1.1 model, load the workflow and add your image of 1024x576 dimensions. If you get out of resources errors, bring the "video_frames" down until it runs. This ran 1200 seconds.

Bonus information

magick .\sdv_img2vid__00001_.webp -coalesce -gravity center -crop 600x600+0+0 +repage output.webp

magick .\output.webp .\sdv_img2vid__00046_.gif

Hope this helps someone else.