 Hey, it's Anfa! Today I want to talk about Blender video sequence editor and its performance problems. Let's kick it off! So a few weeks ago I've crossed the line of 1,000 subscribers and I consider this to be a milestone. YouTube is almost paying for my phone bill right now monthly, which is kind of cool. And I want to talk about something that is a big problem that I encountered in the past but even more recently and that is the performance of video sequence editor in Blender. Blender is a very capable program that can do 3D modeling, animation, rendering, physics, simulation and also video editing. And actually its video editor is very capable and I really like it. It is one of the most capable tools available in the open source world. The other comparatively capable program is called Kden Live, but both suffer from the same issue I think and that is performance. And the big problem is that they don't use the full potential of today's PCs. So my main problem is that when Blender video sequence editor is processing frames, it's doing that on CPU. That is one thing, not GPU, and it's using a single thread processing. So right now I have a desktop PC that has Ryzen 7 1700 with eight cores, 16 threads, I have 16 gigs of RAM and I have a GTX 1060 to give me the computational power to do these things, to do videos, to capture high resolution, high quality, high-framed video, to edit that. And I'm using OBS, open broadcaster software, to capture the video and it is capturing two full HD 60 frames per second images, my desktop and the webcam. It's composing them into a single image and coding this into H24 Matryoshka, which is MKV file container. So I can edit this later. It's doing this with no problem. I can load a heavy outdoor session. I can do CPU intensive work. I can perform audio DSP in the background with no problems and no dropouts. This is why I built this PC, the desktop and the webcam in full resolution, full frame rate, and then compose them onto each other. The problem is now with Blender VSE. I'm using Blender to edit these videos from the beginning, the first few videos of ANFA vlog, the first episode, I guess, yes, it wasn't edited at all. I just pressed record, I did my thing, I've stopped the recording, I uploaded it. That's it. Because I had troubles with Blender VSE before, I couldn't get video and audio to sync properly. They drifted apart and I was unable to edit this. So my second most viewed video is called How to make drum and bass and it suffers from bad video and audio desync. And after this video, I got so frustrated with these problems that I stopped making videos for a few years. And just this year, I've returned to making videos. And I bought myself a new microphone, I bought myself a new camera, I invested in a new PC. I did all these things to start doing these videos better because I want to get serious and I want to really push this forward. And I see a potential in growing here and doing some very useful content for the open source community. Also as a thank you for all the developers who create software that makes my dreams possible. So my criticism of open source software Blender is not because I hate it, I love it, it's fantastic. It's the most capable piece of open source software there. And it allows me to do all these things. I'm using it right now, I'm using it for every video. So I started editing videos with Blender when I captured full HD 30 frames per second video. And after calculating proxy for half the resolution, I was able to edit this video with no problem on a laptop with i7, not the best of i7's Intel processors. Right now I have Ryzen 7 1700, it's orders of magnitude more powerful, but I'm also capturing two full HD images and 60 frames per second instead of 30. So it's four times the volume. And I'm composing two images using also a mask to overlay them and apply a soft edge to my face in the corner. And the performance is that without using proxy, and I had problems with using proxy and I reported bug, and it seems to be kind of fixed, but not necessarily there is another problem, like they are generated and then they disappear. So I can't use them. So I need to use a full source material, which is hard to process. And it takes a lot of resources to process. So when I'm doing this, I have like one, two frames per second. And you can imagine that's difficult to edit. So I'm effectively editing just by audio and Blender is not responsive. It's very laggy and it's very hard to edit these videos. So the recent few videos like UnfaVlog 13, 14, 15, I guess these three, they were made like this with this workflow with separate two full HD images overlaid in editing. So I can do a cut to my face full screen without blurry pixels, without unneeded stuff, and it's clean, it's nice, it's good quality, but it's very slow. How slow? When I did my first video, I guess it was UV 13, it was around like an hour or maybe 45 minutes. I was rendering the video in one Blender instance and it took 12 hours, 12 hours to render my CPU utilization is about 20% total all the time. So it's basically idle. And my GPU utilization is zero. So why is that? Because Blender VSE is using CPU in a single thread, it's not using my 16 threads on the CPU to process the frames because it could do that. In fact, there is a script called Pulverize. It was initially written in PHP and then someone did a Python version and I'm using that. It is basically splitting the video into several chunks and then running Blender with with command line parameters to determine the start and end frame to render parts of the video separately. And then it puts them together without re-encoding using FFMPG. So that allows me to render things a bit faster and I'm finally capable to saturate my CPU usage to 100%. But I can't do more than six instances at once because I have 16 gigs of RAM and if I do more for an hour long video, there's quite a lot of stuff happening. And it uses RAM and each Blender instance of these six is using independent memory. And each one is reading the input files from the disk separately. The system IO cache probably is helping, but when my RAM is saturated and even I'm swapping, there is no free memory to do the cache. So I can also be doing much more hard disk work than I should. It's very, very unoptimal. And even with six instances through pulverize, rendering one hour video takes like two, even four hours if I'm doing some more advanced stuff. Not to mention that editing this is a pain and I actually can't do anything sophisticated than just cutting because if I want to do some zoom in animations and masking and stuff and effects, I just can't because I have two frames per second preview. It's so laggy, I don't feel what I'm doing. And also there are some errors that are left over in the video that I just can't verify because I can't view the video in smoothly. I reported a bug that made me unable to use the proxy because when I use the proxy, the function in Blender that is called image offset or image crop produced incorrect results. So I couldn't preview my videos with my face overlaid in the corner properly. It was like messed up, so I had to edit the full resolution video. Recently in a daily build, I did a test, a quick test with a short video, like 25 second snippet, and I managed to generate the proxy and I managed to use it and edit with 25% resolution of a proxy and it worked and I was able to cut the video. So I'm very glad, but it's not consistent. I was doing this again for a bigger file, like 1.6 gigabyte file and I wasn't able to generate the proxy files. They were generating, they were done and then they disappeared. I tried seeing if Blender dropped some information in the terminal, but no go. I don't know what's happening. Sometimes it happens, sometimes it doesn't. So onto the main dish. The biggest problem with Blender VSE and it's not only about the VSE, it's about all the open source video editors is the performance and that is because they're not using the hardware we currently have to their maximum. Take a look at OBS. Why OBS is able to process to full HD images at 60 frames per second, compose them, encode them, save them to file and I can also do a lot of other stuff because I have plenty of free CPU power to do things. Why? Because it's using GPU. It's composing all the images using the graphics processor. Graphics processors are for this and GLSL is a very capable framework and I'm pretty sure that you can do Gaussian blur and different things using GLSL shaders. I know because Blender is doing these things in Blender game engine in real time. That's right, in real time. 60 frames per second, no problem. You know, if Blender main rendering engine cycles was limited to one thread on CPU, nobody would be using it, but it's using all the CPU power and the GPU power and that is what we need. That's basically what you need. We don't buy hardware to make it idle. We buy hardware to make it sweat to see the results. We can put the hardware to the maximum load and see how hard it can work to get stuff done and hardware can do get stuff done but the software doesn't know how to do it. So I have no idea how Blender is coded internally, how the VSE is working but I feel like there is a big chunk of code that is basically very old that isn't optimized to use a GPU that could be replaced and I think that the Blender video sequence editor needs a general re-haul. It needs to be reworked to be using all the CPU and GPU power that the modern machines can deliver so it can be competitive and it can get the stuff done in reasonable time. If Blender VSE was using GLSL to compose the frames and use all the CPU power and all the CPU threads to stream and decode the video from the disk, it could be very fast. You wouldn't probably even need to use proxy most of the time. Like, we would use proxy if we were shooting 4k video. You know, I can't imagine editing 4k video with Blender right now. I would shoot myself. I considered to buy a 4k monitor mostly for Blender and for Ardor. I finally decided to buy two Full HD monitors instead, one and two, they're in front of me right now, because I get two screens Full HD for half the price than one 4k monitor and also if I were to capture 4k video for my screencasts, I wouldn't be able to edit this in Blender. I would kill myself. Like, if I would have to spend 24 hours waiting for one hour video to be rendered from Blender VSE, that wouldn't make any sense. I don't know what I would do. I'm glad I didn't buy a 4k monitor. That would be a pain in the ass. And also, you know, having two separate displays is much convenient because I can capture just one, use the other one for monitoring my capture, for watching my audio levels, to making sure that it's recording properly, show notes in front of me, and have the other screen to do the screencasting, to show you things that I need to show you. So, OBS is using the GPU. It's open source. It's on the free platforms. Blender itself in the game engine is using GPU for different things. It can also apply GLSL shader filters in real time to the image. And people do things like high dynamic range effects with the bloom and stuff, like screen space, ambient occlusion. I know there is an option to use OpenCL for Blender Compositor. Blender Compositor is a very powerful tool, and I use it a lot. And there is a dispute around the internet. I was asking this myself. Could Blender expose a way to use the Blender Compositor in the video sequence editor? So, we could do green screen keying inside the video sequence editor instead of importing a movie clip, processing with Compositor, exporting it, and then using it in the video sequence editor. Could we do this all at once? In one session, without re-exporting, re-encoding the video, like every time we re-encode, we use more hard disk space. We stack video compression artifacts on top of each other. Even if they are very minute, they sum up. Like, you don't do music mixes, you don't master albums from MP3s because you're going to just expose the compression artifacts. You need lossless files. Of course, dealing with lossless video is totally impractical. So, we all use compressed video. So, if Blender Compositor could also be rewritten to entirely used OpenGL for the processing, then maybe the video sequence editor could even use the Compositor in real time. But that's a different topic. My main question for Blender developers. Modern CPUs have at least two cores for threads. Even the mobile devices like phones and tablets have two or even four core CPUs in them. And with Ryzen CPUs, even the prosumers can get eight core 16 thread CPUs like I did. And Blender VSE seems to still be in the Pentium era of one thread CPU. That is screaming to heavens for a change. Please, Tom, have mercy on the VSE. Make it multi-threaded. Make it GPU aware. It will make such a difference. For example, I once taught Blender video sequence editor to a friend who was working in a professional film studio and who built it. And we run Blender on a server machine that had like, you know, 64 CPU cores. That was probably some Xeon processor. Like, I don't know, 20, 128 gigs of RAM. They had like, I don't know, several gigabit lands in there. It was like a data, like it was like a rendering server. And Blender couldn't play back a full HD video with full frame rate. It was skipping frames. Like, and that guy was like, why is it doing this? Because he wanted to do workshops and teach kids how to do video editing. But when he saw that Blender has so much performance problems, he just dropped it. And I think that this is a big issue because I really would like to sell and advertise Blender's video sequence editor as a professional tool to more people. You know, many, many YouTubers are already using OBS for video capture because it's capable. It can deliver the performance. Blender could do this too, but they're not using Blender to edit the video because it doesn't have the performance because to edit a video, you most of the time, you need to kind of calculate a proxy and the proxy can take very long time to calculate because the proxy generation also isn't multi threaded. It's taking one proxy at a time and 15 of my CPU threads are just idle. And I'm waiting like an hour to calculate proxies so I can start my editing. And then I realize that the proxy files are gone. So I can't do it. Anyway, so I have to edit with one or two frames per second, or just disable the video at all and just edit by ear by the waveforms and by the audio. I know it sounds ridiculous. Sorry, I don't want to be harsh. I really love Blender and I'm really glad that you're doing what you're doing. I really want to say thank you Blender developers. You've done amazing work and this is an absolutely fantastic program, but it has some issues. So you know, I think this needs to be done. You're getting amazing updates, amazing new features into Blender, the upcoming 2.8 release with EV real time PBR rendering is great. I know you can use this GPU to its limits. You can please cut out some space to put video sequence editing to make it competitive with professional great tools because it is possible. It needs a rehaul to get the performance. A second issue is I would say that the audio editing aspect of Blender VSE is very, very minimal. It's very limited. There is no audio mixer. There's no way to preview the levels. There's no way to apply a limiter at least or a compressor and stuff, but that's second there. We can deal with that. Okay, that's a separate issue. Don't care about this right now. Please care about Blender's video sequence editor performance and using all the CPU threads you can and all the GPU power you can to process the frames as quickly as possible to make editing a breeze and rendering also because rendering an hour video for 12 hours sucks and nobody wants to wait that long. So yeah, also another problem with VSE performance is how it handles audio waveform previews. For example, if you edit video and you use waveform previews, which is kind of mandatory to be able to see what your audio is doing, if you delete one strip and then undo all of the waveform previews disappear and they are recalculated from scratch. And this happens many times and it can bug down the performance because again, it's not multi-threaded. So second, it's not cached. The waveform previews should be stored in a file on the disc and then could be just reread whenever you need it. So for example, Ardor, an open source digital audio workstation that I use frequently, is generating audio waveform previews and it's saving them to disk and these can be regenerated again if needed, but they are generated once usually per session, per wave file, per audio file and they're stored there and it can handle tens, dozens, really, of tracks with audio strips all over the place and it can read that from the disk and efficiently handle this and draw all these waveform previews on screen quickly, really. Back to the topic. I'm really glad for all the work you did and I'm continuing to use Blender for video editing. I try to use Cayden Live, by the way, and Cayden Live is based upon the Melt framework, which has a few effects like blurring or some noise generation or something, I don't know, that can be processed on the GPU but that's just like a tiny piece of the functionality and most of that is being processed on the CPU and the biggest problem with Cayden Live I had is you guessed it, rendering times. Rendering 20 minute video was estimated to take first two hours, then four hours and then I didn't want to wait. Yeah, so here's my rant. Please put a little more focus on the video sequence editor. There are a lot of people doing video and the open source community is growing. There are more and more YouTubers that are doing videos and they're using Blender and if we can get Blender videos to consider the performance on par with professional packages that would really be amazing. Thanks for watching. I hope I didn't bug you down too much. Keep up your work and I'll see you in the next video. Bye!