Site Admin, Skilled player (1236)
Joined: 4/17/2010
Posts: 11268
Location: RU
https://github.com/vadosnaprimer/jpcrr/releases Release 0: https://github.com/vadosnaprimer/jpcrr/releases/tag/segmentedavi-0 Dump the JMD as usual (no extension required, it will append it automatically). Drag the JMD onto dumpconvert. Voila, segmented lossless encode, split by resolution and framerate changes. Check segment framerates using this tool: https://github.com/vadosnaprimer/avifps Note that you don't have to set framerate that's too high. Usually gameplay segments gave framerate that you can just force for other segments, without losing much data. If framerate changes in actual gameplay, use the lowest common framerate for gameplay segments and force it for other segments, but still limit 512kb and Youtube encode to 60fps.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Post subject: Re: New DOS encoding workflow
creaothceann
He/Him
Editor
Joined: 4/7/2005
Posts: 1874
Location: Germany
feos wrote:
Now we need to figure out how to properly process cases with different framerates.
Least common multiple + null frames
Post subject: Re: New DOS encoding workflow
Joined: 10/14/2013
Posts: 335
Location: Australia
Testing commenced! I’m glad to see this finally released. Hopefully it entices more encoders to try DOS, now that it’s more user-friendly. I’ve spoken to a few people about handling VFR already, but I’ll post my thoughts here so it’s public.
creaothceann wrote:
Least common multiple + null frames
I agree, that was my first thought too. I’m in the process of drafting a few ideas in AVISynth that can handle this and preserve the correct frame times. For the MKV, the rate can be pushed ridiculously high with no ill side-effects in the resulting output file, as it gets dedupped either way. The resulting clip is VFR and only swaps between the frame rates of the input files when done correctly, preserving the correct rate of the segments and the associated frame timings. If we're using the current encoding package and AVISynth, I think this is the best option. The down side to this method is that it delays the first pass, due to needing to analyze all of the duplicate frames. It processes at a much faster fps, but because the frame rate is increased, the overall process is slower (and it gets increasingly slow with more framerate variations). Given that we’re using x264 placebo settings for encoding, in most situations (allowing for 1-3 frame rates) this shouldn't extend the encoding process too much, as the bulk of the MKV encoding time is spent in the second pass and this does not affect that. VFR cases are currently not common, but at least this method means that when they are encountered they can be handled correctly for the time being. For the Youtube and compatibility MP4 encode (where CFR is required), either the game’s most common (or most significant) rate, or the highest available rate can be forced. Forcing the game's most common / most significant rate would lead to more consistent motion in the majority of the footage, but may have the unintended side effect of dropped frames (Youtube's 60fps cap already does this anyway, so perhaps this is the best option). Forcing the highest available rate would avoid the dropped frames, but would also introduce duplicate frames that would interrupt the motion and lead to stuttering, which is very visually distracting (and could be further hindered by decimation to the 60fps cap, leading to dropped frames anyway, in a less consistent pattern). Neither is ideal, but as CFR is required in these situations a standard should probably be known as the preferred method. I believe the encoding package is currently set up to use a user-picked segment from the dump to set the rate of the output video, which allows easily for either of the above situations, or even a case-by-case subjective judgement. The same method could also be applied to the MKV encode and it would be both faster and an easier implementation, but I strongly disagree with forcing CFR on VFR content for the reasons mentioned in the previous paragraph. This feels like we'd be taking the easy option, when there is a slower but much more accurate option available. Still, it stands to reason that at some point in the future there may be more situations that result in VFR dumps, potentially even true VFR (rather than just several framerates) that the first method won't handle properly. I know we were intending on moving from AVS to AVS+, but perhaps it's time to look for other more modern alternatives that don't rely on the AVI format. Vapoursynth comes to mind as a potential option (and it's been briefly mentioned before), but I don't know enough about it to give a personal recommendation, nor do I know if it does what we would want it to. I do know it was created with the intention to be a cross-platform successor to AVS, it's Python based, boasts multithreading, and has support for format changes and multiple frame sizes (which if the container supports it, would benefit DOS as well as other consoles that change resolution throughout the dump). There might be other candidates too, although in any case it would require a major rewrite of the script and package, but maybe now's the time. Anyway, I've written enough for now. These changes to the DOS dumping process were well overdue and are extremely welcome. Thanks for all the hard work, feos!
I'm not as active as I once was, but I can be reached here if I should be needed.
Site Admin, Skilled player (1236)
Joined: 4/17/2010
Posts: 11268
Location: RU
Before I read the longpost, I can reduce num and denom before writing them to the avi dump, so they're a bit easier to handle.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Joined: 10/14/2013
Posts: 335
Location: Australia
Reduce how? AVS currently detects the FPS num and denom as being already simplified when the clips are loaded.
I'm not as active as I once was, but I can be reached here if I should be needed.
Site Admin, Skilled player (1236)
Joined: 4/17/2010
Posts: 11268
Location: RU
Wow! Didn't know that. BTW, D/Generation has no fps changes, so I'll publish it ASAP.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Site Admin, Skilled player (1236)
Joined: 4/17/2010
Posts: 11268
Location: RU
Release 1: https://github.com/vadosnaprimer/jpcrr/releases/tag/segmentedavi-1 Attenuate pcspeaker by default Simplify avi segment numeration
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Site Admin, Skilled player (1236)
Joined: 4/17/2010
Posts: 11268
Location: RU
Release 2: https://github.com/vadosnaprimer/jpcrr/releases/tag/segmentedavi-2 Dump correct divider for 60fps mode It is known that 60fps is forced by the movie and never changes during it, so such cases won't need to hack around fps changes.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Site Admin, Skilled player (1236)
Joined: 4/17/2010
Posts: 11268
Location: RU
Release 3: https://github.com/vadosnaprimer/jpcrr/releases/tag/segmentedavi-3 cscd: stop resizing height to mod4 (width is fine since dos seems to always use mod4 for it) show frame number in segment messages dumpresolution.exe is now included too.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.