Posted: . At: 9:05 AM. This was 1 year ago. Post ID: 17762
Page permalink. WordPress uses cookies, or tiny pieces of information stored on your computer, to verify who you are. There are cookies for logged in users and for commenters.
These cookies expire two weeks after they are set.


A nice text to Video synthesis model is out now.

by

A very nice text-to-video synthesis model is out now. This can take a text prompt and then generate a good-quality video from it. and it only requires 16 Gigabytes of VRAM. This is very promising. If the user had an Nvidia A-100 they would be able to generate very good quality video footage. This is the future of Youtube, you could easily generate a training video and you would not need to pay any actors for their time. Could you possibly create a movie starring your favorite actors and not need to actually hire them? This would be very interesting for sure.

Git clone https://www.modelscope.cn/damo/text-to-video-synthesis.git to clone the repo and run it locally.

This is the model: https://modelscope.cn/models/damo/text-to-video-synthesis/files. This is required to get this running. And of course, you would require training the AI as well. But this could work very well. This GPU would work very well to run this model. AORUS GeForce RTX™ 4090 XTREME WATERFORCE 24G (rev. 1.1).

See it here: https://www.gigabyte.com/Graphics-Card/GV-N4090AORUSX-W-24GD-rev-11#kf. Or any other GPU card with more than 24 GiB of VRAM. or even this one if you have the spare cash.

NVIDIA Tesla V100 Volta GPU Accelerator 32GB Graphics Card: https://www.amazon.com/NVIDIA-Tesla-Volta-Accelerator-Graphics/dp/B07JVNHFFX

Install the Python package.

pip install modelscope
pip install open_clip_torch

  • Getting this running locally is pretty straightforward:
  • clone sd-video repo and clone model from chinese huggingface (modelscope)
  • go to the sd-video folder
  • conda create -n videogen python=3.10.9
  • conda activate videogen
  • conda install cuda -c nvidia/label/cuda-11.7.0 -c nvidia/label/cuda-11.7.1
  • pip install -r requirements.txt
  • conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia
  • follow the instructions to load the model and generate it in sd-video readme

Uses about 18 Gigabytes of VRAM max, but I’m sure this will get optimized over time.

Here is the translated README: https://0bin.net/paste/JOQEp4si#ZOyD8zEMBBMkFhPm5ha3SR8YosQmI1mT4PLQQOM4IT0

Nice Rentry page explaining all of the installation and setup steps.

https://rentry.org/f34hy.


Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.