GPUS with 6GB Vram can now generate AI videos

Your GPU that has 6GB of RAM can now generate AI videos thanks to a new implementation presented in the Girub.

O FramePack was created by Lvmin Zhang of Github, in collaboration with Manesh Agrawala, from Stanford University. The solution uses video diffusion through fixed length temporal context for more efficient processing, allowing longer and higher quality videos. For example, a 13 billion parameter model built using the Framepack architecture can generate a 60 second clip with only 6 GB of video memory of your video card.

Framepack is a neural network architecture that uses multi -stages optimization techniques to allow local IA video generation. Currently Framepack’s graphic interface allegedly performs a Custom model based on Hunyuan, although the research article mentions that existing pre-trained models can be adjusted with Framepack.

Typical diffusion models process data from previously generated noise frames to predict the next table with less noise. The number of input frames considered for each forecast is called the temporal context length, which increases with the size of the video. Standard video diffusion models require a more robust Vram amount, with 12 GB being a common starting point. But with Framepack it changes figure.

Framepack is a new architecture that compact input frames, based on its importance, a fixed -size context length, dramatically reducing GPU memory overload. All frames must be compacted to converge at a desired upper limit for context length. The authors describe computational costs as similar to those of image diffusion.

This new architecture offers a longer video generation without significantly compromising fidelity, coupled with techniques to mitigate the drifting. Framepack requires a GPU of the NVIDIA RTX series 30/40/50 with support for FP16 and BF16 data formats. Support to turing and previous architectures was not verified, without mention of Hardware AMD or Intel. Linux is also among supported operating systems.

The model used probably has a limit of 30 fps. Framepack can make video generation more accessible to the average consumer, enabling content creation as a tool for creating gifs, memes and other things. Remembering that this is one of your first versions and these solutions are always constantly evolving.

FONTE: GitHub