Posted: 8/27/2024 1:55 AM
Quote
Sand
He/Him
Player (143)
Joined: 6/26/2018
Posts: 174
I tried encoding a video from a bk2 file, and while the video file has the expected total duration and what seems to be a correct framerate, the total number of frames is smaller than the number of frames in the bk2 file. Take User movie #638529018325843149 for example. The bk2 contains 90741 frames (which, given the duration of 1517.80 s, implies a framerate of 59.7845 fps). The encoded video, however, contains 90655 frames for the same total duration, leading to a framerate of 59.7275 fps. Is this expected? I'm using BizHawk 2.9.1 on Linux, Gambatte core, FFmpeg lossless Matroska encoding. Reproduction instructions and more analysis follow.
  1. Download User movie #638529018325843149.
  2. Run BizHawk as ./EmuHawkMono.sh --movie=kwirk-headingout-90741.bk2 kwirk.gb.
  3. Emulation→Pause and File→Movie→Play from Beginning.
  4. File→Movie→On Movie End...→✓Pause.
  5. File→AVI/WAV→Config and Record AVI/WAV....
  6. Click OK at "Most of these options will cause crashes on Linux."
  7. Select FFmpeg writer and leave Sync to Audio checked. Click OK.
  8. Select Matroska Lossless. Command should be -c:a pcm_s16le -c:v libx264rgb -qp 0 -pix_fmt rgb24 -f matroska. Click OK.
  9. Choose an output filename (I used tmp.mkv) and click Save.
  10. Uncheck Config→Speed/Skip→Clock Throttle, then uncheck Emulation→Pause. Let the movie play until it auto-pauses at frame 90741.
  11. File→AVI/WAV→Stop AVI/WAV
These are the stats from Header.txt in the bk2 file: 
CycleCount 3183059634
ClockRate 2097152
3183059634 / 2097152 yields a total duration of 1517.80 s, which matches what's on the userfile page. 90741 frames divided by that duration gives a framerate of 59.7845 fps. The encoded video file has the expected duration of 1517.80 s, but counting frames reveals that the video stream has just 90655 frames, and a framerate of 59.7275 fps (some ffprobe output omitted): 
$ ffprobe -show_streams -count_frames tmp.mkv
Input #0, matroska,webm, from 'tmp.mkv':
  Metadata:
    ENCODER         : Lavf59.27.100
  Duration: 00:25:17.81, start: 0.000000, bitrate: 1514 kb/s
  Stream #0:0: Video: h264 (High 4:4:4 Predictive), gbrp(pc, gbr/unknown/unknown, progressive), 160x144 [SAR 1:1 DAR 10:9], 59.73 fps, 59.73 tbr, 1k tbn
    Metadata:
      ENCODER         : Lavc59.37.100 libx264rgb
      DURATION        : 00:25:17.810000000
  Stream #0:1: Audio: pcm_s16le, 44100 Hz, 2 channels, s16, 1411 kb/s
    Metadata:
      ENCODER         : Lavc59.37.100 pcm_s16le
      DURATION        : 00:25:17.801000000
[STREAM]
index=0
r_frame_rate=23891/400
avg_frame_rate=23891/400
nb_read_frames=90655
[/STREAM]
(Note, however, that the audio stream has the expected 90741 frames): 
[STREAM]
index=1
r_frame_rate=0/0
avg_frame_rate=0/0
nb_read_frames=90741
[/STREAM]
I am not sure where the 59.7845 fps (derived from the cycle counts in Header.txt) originally stems from, but the 59.7275 fps looks to be attributable directly to Gambatte.cs, as well as being attested in online documentation about Game Boy hardware. https://github.com/TASEmulators/BizHawk/blob/3c1248547f5c8116152a041a43d8e806419dc0fe/src/BizHawk.Emulation.Cores/Consoles/Nintendo/Gameboy/Gambatte.cs#L267-L275 
Language: csharp
/// <summary>
/// the nominal length of one frame
/// </summary>
private const uint TICKSINFRAME = 35112;

/// <summary>
/// number of ticks per second
/// </summary>
private const uint TICKSPERSECOND = 2097152;
2097152 / 35112 = 59.72750057..., which as a ratio also matches https://mgba-emu.github.io/gbdoc/#hardware's 4194304 cycles per second divided by 70224 cycles per frame. In summary, with lossless video encoding, I expected there to be exactly one video frame for each frame in the bk2 file. Instead, the encoded video file has fewer frames for the same total duration (which implies a somewhat lower framerate and that some frames must have been discarded). I am not sure if this is expected behavior or not, and I would appreciate some guidance before doing a lot of experiments with different cores, etc. Any hints? The reason I noticed this at all is that I had a Lua script to write out the values of some status variables on each frame, with the intention of adding a tracker UI in postprocessing by matching up the status variable with the video, frame by frame. But because the video did not represent every frame in the input file, the tracker very slowly drifted out of sync with the game.
Posted: 8/27/2024 2:26 AM
Quote
CasualPokePlayer
Emulator Coder, Judge, Experienced player (728)
Joined: 2/26/2020
Posts: 774
Location: California
Arcade TAS of 2023 TASer of 2021 Gameboy TASer of 2021 Rookie of 2020 Gameboy TASer of 2020 Lucky TAS of 2020
The Game Boy effectively has a variable framerate. The ~59.7275 FPS only applies while the LCD is on. The game can turn the LCD off, at which point, there is no concept of "framerate" anymore. Gambatte by default runs for 35112 samples (i.e. the amount of audio in a nominal 59.7275 FPS frame) or when VBlank occurs, whichever comes first. As such, you will generally end up getting a frame much shorter than expected when the LCD is switched from off to on. If you were to say remove all the "short" frames, you would return back to an actual ~59.7275 FPS. BizHawk will encode a CFR video file, which uses the nominal framerate reported by the core (which case is ~59.7275 FPS). Since cores might not have perfect correspondence with the video rate and audio rate (whether through natural issues such as with Game Boy with variable framerates, or just severe core deficiencies such as with Mupen), BizHawk will attempt to keep video and audio in sync by either sacrificing video or audio. With "Sync To Audio", video is sacrificed, skipping or duping frames when needed. If you uncheck Sync To Audio, you will instead have audio sacrificed, leading to various pitch issues throughout the video. Audio issues are much more noticable than a few video frames being skipped/duped occasionally, hence why Sync To Audio is the default. In practice too, this doesn't actually affect GB encoding, since the few frames which get duped would be identical to the ""correct"" frame anyways, leading to no actual sacrifice of video output.
Posted: 8/27/2024 6:41 AM
Quote
Sand
He/Him
Player (143)
Joined: 6/26/2018
Posts: 174
Thank you. That explanation helps me understand. If I run an emu.totalexecutedcycles() each frame, I can see that the difference between consecutive frames is not constant. What BizHawk is doing is totally reasonable, given the constraints you've outlined. I reworked my status variable logger to store a timestamp, rather than a frame number, with each set of values. When postprocessing the encoded video, I can take the video decoder's timestamp for each frame, and use that to look up the set of values with the closest timestamp. It's usually not an exact match, but the difference is less than 1 frame, which is fine for what I'm doing.