Post subject: AviSynth+ and multithreading for official encoding package
Zinfidel
He/Him
Player (200)
Joined: 11/21/2019
Posts: 247
Location: Washington
UPDATE: These changes have been integrated into the official 64-bit TASVideos encoding package! Just download the newest zip of the package at https://github.com/TASVideos/TASEncodingPackage if you don't want to modify anything. The instructions will be left here for posterity. Original post below
This post is a guide to modifying the official encoding package scripts to use 64-bit programs and plugins, and to take advantage of the multithreading capabilities of AviSynth+. Moving the entire pipeline over to 64-bit programs has the advantage of allowing the encoding script to be previewed in x64 VirtualDub, which handles 4K content much better than its x86 counterpart. Multithreading modifications improve encoding performance in general (especially for 4K encodes). In my case, my encoding speed improved by 4X for the HD encode after these changes. AviSynth+ For any of this stuff to work, you will need to be using AviSynth+. You can download the installer from here: https://github.com/AviSynth/AviSynthPlus/releases/ I don't know how AviSynth works if you use a "files only" package, so this guide is going to assume that you install AviSynth+. You probably need to uninstall other versions of AviSynth you might have. Back up your plugins directories before you uninstall! 64-bit Conversion Downloads You will need to get 64-bit versions of all the plugins that are used in the encode.avs script. For the global.bat batch script, you will need the following 64-bit versions of programs: Installation You can just throw the 64-bit plugins into the programs folder and reference them with the LoadPlugin lines in encode.avs, or even better, use AviSynth+'s autoloading feature and instead put your x86 and x64 plugins in C:\Program Files (x86)\AviSynth+\plugins64+ and C:\Program Files (x86)\AviSynth+\plugins+ directories, respectively, and delete the LoadPlugin lines. For global.bat, the only change(s) that need to be made are to change each instance of avs2pipemod to avs2pipemod64, or just rename the 64-bit version to avs2pipemod and replace the 32-bit version with it. Change the reference to ffmpeg (for VirtualBoy) to point to the 64-bit executable. Multithreading Read this wiki for more information on these changes: http://avisynth.nl/index.php/AviSynth+#MT_Notes Add this line to the top of encode.avs:
Language: avisynth

SetFilterMTMode("DEFAULT_MT_MODE", MT_MULTI_INSTANCE)
This automatically sets all plugins that can use multithreading modes to use a safe multithreading mode. As you might have read in the linked wiki article, to actually make AviSynth+ use multithreading, you need to add a call to Prefetch to the bottom of your script as well, but there's a complication for us. The ExactDedup plugin requires strictly serial playback of video during its analysis pass, and multithreading will break this in strange and mysterious ways. So, to account for this, add this line to the bottom of the script instead:
Language: avisynth

pass != 1 ? Prefetch(7) : last
This will prevent multithreaded execution for the modern encode that uses ExactDedup during the analysis phase. (NOTE: this issue can also probably be solved using the MT_SERIALIZED mode for ExactDeDup but I have not tested this). Change the argument for Prefetch to a number that makes sense for your system!. Read the wiki for more information on sane values. My system has 8 logical cores, but setting the value to use all logical cores actually causes performance to take a hit, and Windows gets really upset. Setting it to logical cores - 1 gave me the best results for my system. That's all it takes. I got a very noticeable performance boost for 4K encodes by doing this, and a smaller boost to the SD encodes. VDub64 handles previews for HD encodes a lot better as well.
Darth_Marios
He/Him
Joined: 5/11/2015
Posts: 106
I have an i5-9600k (6cores, 6threads, 3.7GHz - Turbo 4.7GHz) i will get good improvements with this method? (normally i get 120fps for 720p encoding and 30fps for 1080p)
Zinfidel
He/Him
Player (200)
Joined: 11/21/2019
Posts: 247
Location: Washington
Darth Marios wrote:
I have an i5-9600k (6cores, 6threads, 3.7GHz - Turbo 4.7GHz) i will get good improvements with this method? (normally i get 120fps for 720p encoding and 30fps for 1080p)
I honestly have no idea. Possibly. My very amateur understanding of what's going on here is that if the AviSynth script itself is your bottleneck, this should help. The Official Encoding Package is already using multi-threaded x264 to do the actual encoding work, so if your bottleneck is there, this probably won't help. My laptop has an i7-8550U mobile chip @ 1.8ghz with 4 cores, significantly slower than yours. It might be that your processor is fast enough that the AviSynth script isn't a problem, in which case this won't help. You'll have to try these modifications and see if something changes. It's worth noting that if you are using very intensive filters in your AviSynth script, like QTGMC for deinterlacing, these changes will have a much more profound effect.
Publisher
Joined: 4/23/2009
Posts: 1283
Great work! We should really be moving towards AviSynth+ or VapourSynth. Glad someone wrote a guide and even added multithreading!