This might be old or irrelevant news, but quite a while ago I mentioned FFVideoSource("movie.avi", fpsnum=60, fpsden=1) loaded Dolphin dumps incorrectly. This is still the case, as it drops several frames, best explained by ABCCEFG (in this example, "C" is duplicated and "D" is entirely absent). Using the old M/D method with calculations was my original work around this, but gives the wrong output frame rate and duplicates or deletes a frame every thousand or so. It was better than the alternative, but it's still incorrect.
The latest LSMASHSource has been updated to fix this. LWLibAVVideoSource("movie.avi", fpsnum=60, fpsden=1) loads the source correctly, giving ABCDEFG in the above situation, although it needs to load twice to be correct. The first load builds the index and loads the clip (this takes significant time, depending on the dump size and the program appears to be frozen for this duration). Virtualdub must then be quit and re-opened to give the correct frame timings and results (this only happens on the initial load, subsequent loads once the index is created are fine).
This is definitely an improvement over the alternatives mentioned above, or using ffmpeg to convert the FFV1 dumps into a more usable format.
Also, it's worth noting that due to the nature of 4K dumps and the resolution size, Virtualdub may freeze or crash during seek (or if you try and save directly from Virtualdub). Closing the program and reloading the script fixes this and the encoding package deals with this without issue.
I'm not as active as I once was, but I can be reached here if I should be needed.
The biggest problem with that was related to the dump size. A 775gb dump meant it took quite a while and took up a fair bit of extra space. While that's not an issue for me personally, I was hoping to find a way to load the dumps up directly without first needing to re-encode or convert them.
What's the problem with LWLibAVVideoSource? Is there's a frame inaccuracy I didn't notice somewhere during my tests?
I'm not as active as I once was, but I can be reached here if I should be needed.
I have not tested it so I can't comment, honestly. I just know of FFMS2 problems.
Edit: I think a good test is to do a compare of the output from LSMASHSource and FFMPEG with lossless FFV1 level 1. If they match completely, then it would be pretty safe to say LSMASHSource is a good idea.
My tests with libav and ffms2 show that it always drops or adds frames. For reliable comparison I have a cfr lossless dump that I can load normally (even though it's segmented), and then with ffmpeg and compare. With dolphin, you need some absolutely accurate source to compare to.
Post #459368
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
I have never seen FFMS2 drop a frame or add a frame. Stop using it incorrectly: Do not give it timecodes, and do not ask it to perform any framerate conversions.
Then it completely desync sound, even if I only import a single segment.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
I'm doing it for the purpose discussed in the thread I linked, multiple segments, like a lot. Normal avisource dies after 70 segments for me, ffmpeg filters don't.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
I'm doing it for the purpose discussed in the thread I linked, multiple segments, like a lot. Normal avisource dies after 70 segments for me, ffmpeg filters don't.
You had me confused, as the title of this thread, of course, is Dolphin dumps, which are always one segment (unless it's more than 3 hours long...)
I always thought the best way to handle multisegment would be to dump 1:1 at the highest resolution in the video as a single segment, then send metadata in a separate file that could be used in Avisynth to selectively resize in different ways whatever parts needed to be resized. I started working on something like that when I did an FF9 encode, but it was a screwball process that I never quite perfected.
@Aktan, could I have the command line for ffmpeg to do FFV1 level 1 to compare?
In my tests against the x264rgb file I created with ffmpeg I found that file loaded up with AVISource and the original dump loaded up with LWLibAVVideoSource to be identical when loaded using 60/1, although this comparison was sound, I did only thoroughly examine (frame by frame) known problem areas using FFMS2 as the dump I was using was excessive in size (250,000+ frames).
I'd like to examine this further in direct comparison to what you suggested, with the latest ffmpeg and AVISynth plugins. I'll happily provide files for these tests, too (scripts, plugins and dumps, etc) so everyone can see the results and compare them for themselves.
Also, to elaborate and clarify the problem with FFMS2 with loading Dolphin dumps outright (for anyone not aware), the issue is that it loads the clip at the incorrect rate when no parameters are specified, eg: FFVideoSource("clip.avi") - causing frame times to be off by a large amount in certain circumstances (leading to desync in audio as feos has pointed out) and potentially absent frames. When the parameters were set to 60/1, it skips and duplicates frames as necessary to achieve the target rate rate causing a lot of unnecessary stuttering and missing frames. This outlines "do not ask it to perform any framerate conversions", as natt said. The issue is that without those parameters, the rate it gives is not the correct one. When loading up using LSMASH and the same 60/1 rate, I get a completely smooth video with those frames still in tact, with no dropped frames in comparison to the ffmpeg output file I produced prior (therefore, performing how I believe it should).*
*Although as I mentioned earlier, this was done through tests I'm not thoroughly satisfied with and I'll be conducting more in the very near future.
Edit: As a second note, if LSMASH doesn't have a rate specified it works the same was FFMS2 without a rate specified, loading the crip at an incorrect rate and causing frame timing to be off. 60/1 has to be specified to properly circumvent this.
I'm not as active as I once was, but I can be reached here if I should be needed.
Aktan and I had a long talk in IRC and did some thorough tests of the different methods regarding loading Dolphin sources. I'll try and cover everything in this post as best I can. If you just want the conclusions drawn from these tests, they will be at the bottom of this post, otherwise here are the test files.
The first thing to note is that a lot of the conversions have already been done. Most of the remaining batch files and scripts are present to show exactly what was done in order to get these output files prepared. I'll go over that first. If you want to do these tests in their entirety yourself (and not use the pre-prepared files), start at step 1. If you just want to compare the pre-prepared files, skip straight to step 7.
1 - Delete the contents of the "lagarith_output" folder, and the contents of the "dumps" folder, with the exception of framedump0.avi and dspdump.wav. These two files are dumps done with Dolphin 5.0's release and were the dumps used for testing. These can be replaced with your own Dolphin dumps if necessary, provided they overwrite the original files.
2 - Running the batch file "make-ffmpeg-ffv1" will create a new file in "dumps" using the FFV1 codec. The point of this conversion is to change some of the codec settings and correct the colorspace into something AVISynth will accept.
3 - Running the batch file "make-ffmpeg-x264rgb" will create a new file in "dumps" using the x264rgb codec.
4 - Now that the dump files are prepared, source loading can begin. Load "load_dolphin_ffms2.avs" in Virtualdub. The application may appear to have frozen - this is normal. The bigger the dump, the longer this process will take.
5 - Once you see the video has loaded, quit out of the program. Load "load_dolphin_lsmash.avs" in Virtualdub. Once you see the video has loaded, quit out of the program. The reason behind doing this (and the previous) step is that these source filters create index files. In certain circumstances (particularly with LSMASH) on the first load (the one that creates the index), the video output will be incorrect, having absent frames and incorrect timings.
6 - Load up each of the six AVISynth scripts beginning with "load_" and save them using a lossless codec (I recommend Lagarith) to the Lagarith output directory using their default file names. If "load_ffmpeg_ffv1_avisource.avs" or load_x264rgb_avisource.avs" give you any trouble or do not display the correct output, and installer for a compatible version of ffdshow has been provided in the prereq folder. Installing this should fix the problem. The files appended with "_60" have been given parameters for a forced frame rate of 60fps. You can examine the scripts as necessary.
7 - Edit "compare.avs". The first few lines load up all of the Lagarith output files created in the previous steps (or pre-prepared, if you've skipped straight to this step) and assign them various names. Method1, Method2, Method3 and Method 4 are all methods of loading up the source directly in AVISynth. Compare1 and Compare2 have been losslessly converted using FFMPEG prior to their loading and make good reference clips for this reason. Set Clip1 and Clip2 to whichever two of these you wish to compare. By default, "Clip1 = Compare1" and "Clip1 = Compare2". This compares both of the FFMPEG converted files. Save the script.
8 - Load the script in Virtualdub. The video should appear grey. You can seek through the video - any moments that aren't identical in the two clips will become visible.
9 - To ensure the clips are completely identical, set the codec to Lagarith and save the video file anywhere you can find. For this example, I'll call it "compare.avi" and it'll be saved to my desktop.
10 - Load the file you just saved in Virtualdub. Hit File > File Information. We want "Number of Keyframes:" to be 1. If it's higher, this means there was a difference between the two compared video files, potentially one too subtle to make out at first. If it is 1, ensure the keyframe is the first frame of the video. Repeat steps 7-10 for as many different comparisons as you wish to do. If the files are identical, particularly to Compare1 and Compare2 (which are identical to each other), then they have been loaded up correctly.
These tests confirmed what I originally thought: LSMASH does load the Dolphin dumps correctly when given a forced frame rate. A few other things worth mentioning are that that FFMS2 gives an incorrect result both with and without the forced rate, and that LSMASH gives an incorrect result without the forced rate. This means that as far as I can see, the best way to load Dolphin dumps without converting or re-encoding prior to AVISynth usage is using the following:
That being said, transcoding with ffmpeg to both FFV1 and x264rgb is just as accurate if you'd prefer operate with AVISource and don't mind the extra disk space usage or waiting time.
The command to convert to a readable FFV1 stream using ffmpeg is as follows:
It looks like in addition to the necessity of level 1, another big issue with the original Dolphin FFV1 dump is the colorspace being bgra. It looks like some sort of incompatibility with the alpha channel is part of the problem and as you'll see above in the ffmpeg codec for converting to FFV1, the line "-pix_fmt bgr0" fixes this. It might be worth trying to get the additional parameters ("-pix_fmt bgr0 -level 1 -g 1 -coder 1 -context 1") implemented into Dolphin directly, so in the future dumps can be loaded directly with AVISource.
Anyway, I think I've spent far too much time typing this thing up already so I'll draw the post to a close here. A lot of the technical work would not have been possible without Aktan, who was responsible for a vast majority of the problem solving, parameters and,well.. everything.
There is one outstanding problem here still, however. feos wrote:
My tests with libav and ffms2 show that it always drops or adds frames. For reliable comparison I have a cfr lossless dump that I can load normally (even though it's segmented), and then with ffmpeg and compare. With dolphin, you need some absolutely accurate source to compare to.
Post #459368
Given it's CFR, were you forcing the CFR rate? And did you let the index build and then reload the clip to ensure you were getting the correct timings?
I'm not as active as I once was, but I can be reached here if I should be needed.
Joined: 11/21/2019
Posts: 247
Location: Washington
Given that the last post here was made nearly 3 years ago, and some things have apparently changed, I'd like to provide an update on the ffvs vs libav thing. At least for Resident Evil 4, ffvs (using fpsnum=60) now syncs up to the audio, sort of. Take a look at this comparison video:
Link to video
Interestingly, while the encode that used ffvs does sync up to the audio nowadays, the actual video does not run at the correct frame rate some of the time. This is particularly noticeable at the very beginning during the memory card read and the title screen, where the two videos are noticeably running at different rates. However, once actual gameplay starts, the videos line up almost exactly. I say almost because inspecting the comparison frame by frame reveals that the two videos will be out of sync by exactly 1 frame every now and then, and not necessarily in the same order at any given time (sometimes libav is ahead, sometimes behind).
At sections of the game where the resolution changes (any time the player brings up the inventory screen), the ffvs and libav versions diverge slightly. You can see that the bright magenta flash lasts longer on the ffvs side, and it's harder to notice this, but the ffvs side fades to black more quickly than the libav side. Encodes of this entire TAS (#6832: Ubercapitalist's GC Resident Evil 4 "The Mercenaries: Waterworld" in 08:36.57) sync up with the audio regardless of the import function used, but the LSMASHSource plugin is still the only one that outputs video that matches the emulator output exactly. I will note that feeding the raw output into FFMPEG directly (and muxing with audio there if you want) will output correct video as well that matches up with LSMASHSource output. It's just the ffvs plugin that seems to have issues.
While I have been trying to educate myself with the finer details of how encoding timebases work, I am not entirely sure how it is that this difference in decoding occurs. I think it has something to do with FFV1 encodes actually having timestamps for individual frames rather than a particular framerate, thus effectively producing variable frame rate video. Something about the way AviSynth (and the plugins) handle this falls apart if a constant frame rate conversion is not baked in to the script.
Data for the encode I posted:
Note that StackHorizontal does automatically adjust the second video to fit the parameters of the first video (w/ regards to framerate and some other properties) so I checked to make sure that it wasn't affecting output. Switching which video comes first does not affect the output of each individual video at all, and individually encoded videos line up with what's in the comparison video.
Dolphin's dumper only writes unique frames with their timewise positions, but the timebase is 1/60, so the result can be loaded by VFW importers without additional framerate info. Dolphin rounds up timing of every frame to fit into this 60fps grid, even though originally they may not be aligning with it. It's constant fps, but at the same time dedupped.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Joined: 11/21/2019
Posts: 247
Location: Washington
feos wrote:
Dolphin's dumper only writes unique frames with their timewise positions, but the timebase is 1/60, so the result can be loaded by VFW importers without additional framerate info. Dolphin rounds up timing of every frame to fit into this 60fps grid, even though originally they may not be aligning with it. It's constant fps, but at the same time dedupped.
Ah, thanks for the breakdown. That is in line with what I thought might be going on in the FFV1 encode. A possibly interesting observation on the "framerate" of the FFV1 encode is that using FFVideoSource without parameters will cause VirtualDub to report the framerate as 27.479, whereas LWLibavSource without parameters will report as 60.0 fps. I know from experience that if you do a segmented dump of the game, each segment will have a slightly different reported framerate as well.
I guess this means that FFVideoSource is trying to do some sort of framerate heuristics on the clip to guess a possible/average framerate, given however the frames happen to line up. Maybe it's trying to be too smart or something.
It is known in the past that FFMS2 way of converting VFR to CFR is wonky. I don't know if it's still like that now as I have not kept up, but it's why I wrote my own plugin here:
https://forum.doom9.org/showthread.php?t=165045
Joined: 11/21/2019
Posts: 247
Location: Washington
That thread was helpful in understanding more about how this timecode situation works, thanks for sharing Aktan. I was not using ffvs back when you wrote your plugin, so I don't know how the plugin has changed if at all. Based on the discussion in this thread, it seems like things have changed though, as in the comparison I did, I did not find dropped frames in either video. Where frames actually showed up relative to the other video shifted around as I mentioned, but each video had all of the same unique frames as far as I could tell (I did not do a proper, exhaustive search however).
Based on the discussion in that thread, I think I have a better understanding of why two different plugins might produce these results. What I'm still struggling with though is why the output from Dolphin has so many issues with playback in all of the playback software I have. Avisynth, VirtualDub, MPC-HD, and even Windows' built-in players all play the FFV1 encode incorrectly. Based on what I know about the FFV1 encodes now, it seems like all of the information to play the video back correctly is present in the file, yet this conversion step has to happen first still.
Since Dolphin doesn't dump a timecodes file anymore, I didn't figure the VFRtoCFR plugins would be useful anymore, so I never investigated them. Perhaps I'm wrong on that account?
What I'm still struggling with though is why the output from Dolphin has so many issues with playback in all of the playback software I have. Avisynth, VirtualDub, MPC-HD, and even Windows' built-in players all play the FFV1 encode incorrectly.
Incorrectly how? What about my 5.0 build?
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Joined: 11/21/2019
Posts: 247
Location: Washington
Alright, strike the comment about it not playing back correctly on my media players. I compared playback of the ffv1 framedump to my converted encodes and they match up exactly. What I had been doing as a shortcut is comparing my encodes against Ubercapitalist's temp encode since I could access and play it quickly, and there's something about the way his encode is done that throws off the timing compared to mine.
The quick map restart that is done right at the beginning is much faster on his encode, though the rest of the encode then lines up exactly, including the audio. It looks like my ffv1 dumps match up with the emulation though, so maybe he did some editing for his temp.
What I had been doing as a shortcut is comparing my encodes against Ubercapitalist's temp encode since I could access and play it quickly, and there's something about the way his encode is done that throws off the timing compared to mine.
The quick map restart that is done right at the beginning is much faster on his encode, though the rest of the encode then lines up exactly, including the audio. It looks like my ffv1 dumps match up with the emulation though, so maybe he did some editing for his temp.
My temp encode was recorded with xbox gamebar, no other editing was done.
