Intro
I recently made this video and I posted it on Twitter and I got a lot of people asking, how the heck did you make that video? Well, I did the whole thing with AI and in this video, I'm gonna walk you through exactly how you can make videos just like the one you're seeing right now. Let's get into it. All right, so in this video, we're going to install Deforum, which is a really cool add-on to Stable Diffusion, which allows you to make really crazy animations. Now, if you haven't already, you need to have Automatic 11.11, the Stable Diffusion user interface installed on your computer or running in a cloud server like Rundefusion already. If you don't have the Automatic 11.11 Stable Diffusion interface installed, I did make another video about it. I'll make sure it's linked up below.
If you're watching this in a browser, it should also pop up somewhere above me. Installing Deforum is really easy. Once you've already got Automatic 11.11 installed, you're gonna come to github.com. Up in the search, we're gonna type deforum, D-E-F-O-R-U-M, and then we're going to click on Deforum for Automatic 11.11 Web UI. We're gonna come over here to this green code button, click on the little arrow to drop down, and we're going to download the zip file. We're going to grab this zip file, and we're going to move it to where we have Stable Diffusion installed. So on my computer, that's under the C drive, under the AI folder, and then under Stable Diffusion Web UI. I'm going to double-click into extensions here, and then I am going to paste this zip file inside of extensions, and we will go ahead and unzip this zip file right here to this folder. We no longer need this zip file, so I'm gonna go ahead and delete it, and then we have this new folder called Deforum for Automatic 11.11. I'm going to double-click on that so I can rename it, and I am just going to delete all of this so it just says Deforum, and once you've done that, we're gonna click back to come to our Stable Diffusion Web UI folder, and we are going to run Stable Diffusion the way we normally would.
We're gonna come down to webui-user.bat, double-click on that, and fire up Stable Diffusion Automatic 11.11 like we normally would. Once this is done doing its thing, we're just gonna double-click on this URL here, Control + C to copy it, jump over to our browser, paste this URL into our browser, and fire up Automatic 11.11, and once you've done that, you'll notice that you have a new tab here that says Deforum. Now, if you're running a cloud version on Run Diffusion, we're just gonna go ahead and boot up our run diffusion server here the way we normally would.
And would you take a look at that? The good folks over at RunDiffusion already pre-installed Deforum for you. So you don't have to do anything. You just got to log in. Okay. So here's the thing with Deforum. There are a lot of various buttons and knobs that you can turn on this thing. There are so many settings and so many different things that you can tweak with it that I am nowhere near an expert on exactly what everything in Deforum does. I'm going to give you a very basic overview of how to play with some of the settings and how to get something that looks fairly cool.
I'm kind of figuring a lot of this stuff out as I go myself. So I'll probably be making a lot of future tutorials as I discover cool things about Deforum that I'm not going to show you in this video. This is just meant to be kind of the basics and show you what it's capable of and open your eyes to something that you can do with stable diffusion. Pretty much for free or very inexpensively with like a cloud service. I just want to show off cool tools and nerd out with you. I may not always be a hundred percent accurate with my descriptions of how some of this stuff works.
I'm just going to explain it from what I've sort of found through my own experimentation. All right. So let's go through some of the tabs and settings that are available here. So on our run tab here, we've got our sampler and it's just like the text to image. You've got all of these various sampling options that you can play around with. And I highly recommend generating different videos with different samplers and seeing what kind of results you get from my own experimentation, I've had the best results from this Euler A, but definitely just experiment and try different samplers because different samplers are going to give you different results in the same way that when you're doing text to image, different samplers are going to give you different results. Now the steps here, these work just like the steps that you'd find in the sampling steps and text to image. The more steps, the more it will sort of improve the quality of the output, but also the longer it's going to take to render out the final product. You've got your width and height. It works pretty much as you would expect it to. So you set the dimensions that you want your video output to be. I'm going to leave it at 512 by 512. You've got your seed. It works just like a text to image seed. If it says negative one, it's going to generate a random seed. Every time you make a new generation, if you got a video that you really, really liked and you want to try to recreate something that looks similar, you can try to reuse the same seed from that video that you used in the past. The batch name deform. This is basically, I believe the folder that it's going to save it into. And pretty much all this stuff down here. I just don't mess with. Now this key frames tab here is where all the knobs and buttons are that I was talking about.
This is where you're going to tweak all of the various settings and stuff that you're going to do in the video. And we'll skip this for a second. Cause there's a lot here. This is where there's honestly things that I don't even know what they do. I haven't played with them yet, but I'm going to go over some of the basics with you. And then you've got your prompt area. This is where you actually write your prompt that you want to generate here. And quite honestly, I don't really mess with the rest of this. I haven't really played around under this tab yet. I haven't really played around under hybrid video, and there's not a whole lot that you need to mess with on output, leaving this CRF at 17. Seems to work pretty well. This is your frames per second. Leaving at 15 has always done a good job for me. You can add a soundtrack to your video as well. I haven't even done that myself yet, but that's something that you can play with. So let's jump back over to the prompts real quick. And let me explain this. If you click this little arrow here, there is some instructions that you can follow that will help you with your prompt, but here's the basics of it.
So you've got these numbers here in quotation marks. That's the starting frame of where you want that specific generation to start. So you can see here in the sort of pre-filled prompt here that they've created for us, starting at frame zero, it's going to generate a tiny, cute swamp bunny, highly detailed, et cetera, et cetera. You'd use all the same kind of image prompting that you would use with text to image, starting at frame 30, it's going to change into an anthropomorphic, clean cat surrounded by fractals and all of this stuff, starting at frame 60, a beautiful coconut, starting at frame 90, a beautiful durian, trending on art station. You can see right here, starting at frame 60, they also added this dash dash neg photo and realistic. They're basically adding a negative prompt here. They don't want this beautiful coconut to look photo realistic. You can see right here, it gives some instructions about the negative prompts. It says for negative prompts, please write your positive prompt, then dash dash neg, and then ugly text, asymmetric, et cetera, whatever you want to make sure is negative. So again, these are the starting frame of what you want this to change to. And this actually relates over here on the key frames. If I jump over to key frames real quick, you can see this is the max frames. So right now you can see that this video is going to be 120 frames total. So if we look at the prompts here from zero to 29, it's going to be our tiny bunny from frames 30 to 59, it'll be our clean cat from frame 61 to 89. It will be this beautiful coconut. And from frame 90 through frame 120, which is the max frames that we set. So the remainder from 90 to one 20 will be this beautiful durian. Now, if we change this key frames, cause we want our video to be longer. Let's say I want it to have 200 frames in our video instead of 120 frames.
Now, when I come back to the prompts, if I want each of these prompts to be equal amount throughout the video, I would need to adjust these numbers. So now from zero to 50, we want it to be our cute bunny from 50 to 100. We want it to be our cat from 100 to one 50. We want it to be our beautiful coconut. And from one 50 to frame 200, we want it to be our beautiful durian. If you do change the amount of key frames, make sure you also make the proper adjustment to your prompts so that it changes when you want it to change. So let's start by just leaving everything on the default, but tweaking some of the prompts so that we can see it just kind of move through these various images. So let's do a realistic gray wolf in the forest. I always like to add trending on art station because it seems to make some pretty decent images when I do that. Massive red fire breathing dragon trending on art station.
A realistic Viking warrior wearing war paint trending on our station. And then a shiny metallic humanoid robot. So ideally it'll be kind of a robot shaped like a human. And so there we go from zero to 50 frames, we'll have a gray wolf from 50 to a hundred frames, we'll have a fire breathing dragon from a hundred to 150 frames. We have a Viking warrior and from one 50 up to 200 frames, we have a shiny metallic.
Robot. If we go ahead and generate this, let's see what it gives us before we even mess with any of the default key frames here.
Demo
All right. So it's finished processing. You can see there's a little preview image here, and then there's a gray button up here that says click here after the generation to show the video. So let's see what it made for us. Again, it should start with a wolf, go into a fire breathing dragon, show a Viking warrior, and then a metallic humanoid robot. Let's see if it cycles through those various images. There's our wolf.
Now it kind of morphed into this dragon. There's our Viking. And then there was our metallic robot. Looked like it had some sort of kitchen colander or something.
And you can see just by kind of leaving everything on the default, it's just sort of cycling through these images. It kind of has a slow zoom in on it as it's doing it, but it's just cycling through these images, really. So now let's jump over to our key frames here and explore some of what's available. So you can see right here, it's got this sort of long value here for the payset, which it's zooming in, which I don't totally understand what all that means. I think it's just a very, very, very subtle zoom in.
Now, I believe if you just set this to one and leave it at one, it doesn't really affect the zoom. It just stays stable. Up here, you've got your angle. The angle is sort of the rotation. If you put a negative number in there, it's going to rotate clockwise. If you put a positive number in there, it's going to rotate counterclockwise. Translation X, that's the left to right sort of value, and translation Y, that's your sort of up and down value.
Now, one of the best ways to test how these motions work here is to come up to your prompts, just get rid of all of the rest of the prompts, just have kind of one image. And if you only have one image, you want to get rid of this comma at the end. Come to your key frames and under max frames, just set it to like 15 frames. Just set it to something really short. And now we can play around with the angles and the zooms and the translations and see what effect they have on a real short video before we start making longer videos.
Let's go ahead and just set the angle on this one to two and see what happens when we generate something real quick here. Now we can go ahead and preview this, and you can see that it rotates counterclockwise with a positive number here. Now, if I change this angle down here to a negative two and generate it, we can see that now it's rotating clockwise. So let's go ahead and set this angle back at zero here. Now, the zoom number, it can't be zero. If we just leave it at one, that's just stationary. It's not zooming. If we set it to two and we generate, you can see that it zooms in, but it's zooming in really, really fast. That's why when we first opened it, it had that long fraction. So really for a zoom, you probably want to go more like a 1.02 or something like that. And if we generate that, you'll see it'll be a much less aggressive zoom. You can see the zoom isn't quite as crazy with a 1.02 there. Go ahead and set that back to one.
Now, if we mess with the translation X, that should shift it from left to right based on whether you give it a positive or a negative number. So let's go ahead and just set this as a two and let's see what happens when we generate that. And you can see that now it's shifting to the right. Set this as a negative two, take a peeksy. And now it's shifting to the left. Translation Y does the same thing, just up and down. So now that we know what each one of these things does, we can start to tie it all together.
Adding More Prompts
So if we come back to our prompt, let's go ahead and add some more prompts in here. Again, we'll go ahead and generate a 200 frame image this time. So I'll go ahead and start with the realistic gray wolf. I'm gonna go ahead and paste that in here again, but this time starting at frame 50, let's go a realistic lion on the plains. And then let's say starting at frame 100, we can do a realistic tiger in the jungle.
And then starting at frame 150, let's do a realistic great white shark in the ocean. We want this to be 200 frames. So let's make sure we set our max frames up to 200 here, and then we can start to mess with some of the motion. So it's going to zoom in very slowly on the animals. So let's say we just want it to kind of stay stationary for the first 50 frames. We can set the translation X to zero. And then so from frame zero to frame 50, it'll stay stable.
And then starting on frame 50, it will actually start to pivot to one side. Let's go ahead and add a two there. So that motion will start at frame 50. And then let's say once it gets to frame 100, we want it to stabilize again and stay in place. So we'll add frame 100 and then we'll make this zero again. So now from frame zero to 50, it's going to stay still. Once it hits frame 50, it's going to start to move. Once it hits frame 100, it's going to stay still again.
And then let's say from frame 150 on, it's going to start to move up and down. So let's go ahead and starting on frame 150, we'll start to translate it on the Y axis. All right. So it's going to zoom in slowly through the whole thing until frame 50, it's going to stay stationary. Once it hits frame 50, the camera is going to start to move on the X axis. Once it hits frame 100, it's going to stop moving on the X axis. Once it hits frame 150, it's going to start moving on the Y axis. All right. And as it goes through these prompts, we're going to see a gray wolf, a realistic lion, a tiger, and then a shark.
Testing
So let's go ahead and generate this. All right. So let's see what it generated for us. So remember from frame zero to 50, we have a wolf from 50 to 100. We have a lion from 100 to 150. We have a tiger and from 150 on, we have a white shark, a great white shark. So let's see what this looks like.
we got that lion in there you can see that the camera is shifting around now it's shifting to a tiger and now it's shifted to a shark with a lot of extra teeth. If we look at our key frames again oh I forgot to put a colon in in this one so we didn't get our translation on the y-axis but you can see we've got a very slight zoom in throughout the camera is sort of shifted and then stopped I think I want to regenerate it without the translation on the x-axis here and let's get rid of the translation on the y-axis let's actually have the angle changing as it's going and let's have the zoom be a little bit more dramatic here and let's go ahead and generate this one more time and see what we get.
so there's our wolf zooming in now we've got our lion and now it's starting to shift into a tiger and now it's shifting into a great white shark let's go try one more but let's use some different prompts this time let's start with a realistic monkey on an alien planet an astronaut on an alien planet a little green alien on an alien planet a metal robot shooting laser beams from its eyes let's do that and then for our key frames i'm going to get rid of the rotation angle here and i'm going to actually up the rate of the zoom a little bit here so 1.08 and let's generate this one with a new set of images all right let's see what our short science fiction movie looks like you you you you
It's actually zooming in quite a bit faster than I thought, but it still looks pretty cool. There's our alien, our robot shooting laser beams. Starts back over with the monkey. I wish the alien and the astronaut stuck out a little bit more. I mean, the monkey's very,
You can kind of see the astronaut on the edge there. The alien is like practically non-existent. And then there's our robot with the laser beams. All right, I'm going to generate one more and I'm just going to slow down the zoom a little bit. Make it 1.03. Let's generate one more time. Can already tell it's going to be interesting because we got a two faced monkey here.
The astronaut looks a lot better in this second version.
I saw our alien for a second and then there was our robot.
Now, obviously, if you wanted to, you can get a lot more detailed with these prompts, but for the sake of this demonstration, I was kind of trying to keep the pace up. I only scratched the surface of what you can do with the forum here, because there's the 2D animation mode, the 3D animation mode, interpolation, and then video input where you can actually upload a video and put some effects over it as well. There's also the replicate border. There's the wrap border, which does different things.
And there is just a ton of other settings that we didn't even touch on. I just hit the basics. So as I learn more about it, I'm going to make more videos. I actually have some calls coming up with some experts in the forum. I'm going to try to record some of those calls and share some of the insights that they give with you. And we're going to deep dive into the forum and figure out how to make some really, really awesome videos in some of my upcoming YouTube videos. So if you haven't already, make sure you subscribe to make sure you see those when they come up. Also make sure you like this video too, because if you like this video, you'll see more like it in your feed. And finally, if you haven't already check out future tools.io. This is where I curate all of the cool tools that I come across in the AI space. Right now there's almost 900 of them. And if 900 is too much and you're just like, Oh, my brain is, can't handle that many AI tools. That's just too many AI tools for me. Join the free newsletter. Cause every Friday, I just send you the five coolest tools that I come across over the week. So if I come across 150 tools this week, I'll tell you the five coolest that you should pay attention to. I send that email out every Friday, along with some cool, interesting news articles, a handful of YouTube videos, and one cool way to make money with AI, so you're going to want to be on it, you can find it over at future tools.io click this button here and I will hook you up it's totally free. And thanks again for tuning in. Hopefully you enjoyed this one. Thanks for nerding out. See you guys in the next video.