Topic by Bisqwit

Posted: 3/30/2005 9:11 AM

Quote

Bisqwit

Editor, Active player (296)

Joined: 3/8/2004
Posts: 7469
Location: Arzareth

Well, the exact FPS the emulator uses is important, because otherwise the audio in the AVI will drift out of sync with the video.

Posted: 3/30/2005 9:12 AM

Quote

Bag_of_Magic_Food

He/Him

Active player (277)

Joined: 5/29/2004
Posts: 5712

Oh, right, of course it depends on the emulator. Do YOU know where to find what speed it uses?

put yourself in my rocketpack if that poochie is one outrageous dude

Posted: 3/30/2005 7:48 PM
(Edited: 1/23/2022 7:40 PM)

Quote

Bisqwit

Editor, Active player (296)

Joined: 3/8/2004
Posts: 7469
Location: Arzareth

Nitsuja, there is some problem with the video recording. I'm getting video frames of very strange size. Were there warnings in compiling nesvideos-piece.cpp ? Because it's writing data that doesn't make sense. The first four video frames are 7976, 8192, 8192, 8192 bytes long respectively, but it writes to the header that each of them are 46080 bytes long. It's as if the call "fwrite(vidbuf, 1, framesize, avifp);" somehow failed to write the entire buffer at once, but it doesn't care. Edit: Additionally, there seems to be some other problem regarding the video resolution and the sound in beginning. See this sample: https://files.tasvideos.org/bisqwit/vba-problem1.avi - The video buffer is not 160x144 - it contains a marginal that skews the image. The marginal seems to be always 4 pixels wide. It would be nice if you can be so kind as to pass a marginal-removed-buffer to NESVideoLoggingVideo() by creating a temporary buffer prior the call. - There's noise in the beginning. This is probably my error: I'm assuming that memory returned by malloc() is blank, when it's not really so. I fixed this problem at http://bisqwit.iki.fi/kala/nesvideos-piece.cc - please update. - The startup logo is of wrong resolution. My error. Please use the updated source code. - There's big delay between video and audio. This is what I noticed also when I tried to playback a movie without AVI recording. Please resolve. :) I also attempted to write explicit error handling to the buffer writing in the updated code. I hope it works better now. Windows apparently never would have guessed that a file is actually a pipe.

Posted: 3/30/2005 8:09 PM

Quote

Phil

Active player (410)

Joined: 3/16/2004
Posts: 2623
Location: America, Québec

lmao what a big problem here

Posted: 3/31/2005 12:52 AM

Quote

Bag_of_Magic_Food

He/Him

Active player (277)

Joined: 5/29/2004
Posts: 5712

I don't know if this is related, but I've noticed that in Nitsuja's patch, the sound seems to be a bit behind the visuals, like a few extra frames, compared to older patches. This is even causing sounds from the present to be heard a while after a save state is loaded, which is annoying when I want to be able to tell what's going on based on the sound. Is this fixable?

put yourself in my rocketpack if that poochie is one outrageous dude

Posted: 3/31/2005 3:49 AM

Quote

nitsuja

Emulator Coder, Skilled player (1300)

Joined: 12/21/2004
Posts: 2687

OK, I figured there will be problems as I don't know how to test whether the output is correct - I don't know what you use to read the raw data in and convert it to AVI but it would help if I was able to do that to see if the output makes sense. I'll see if I can find any of these problems at the moment, though. About the sound lagging, that's because of the "Synchronize Emulation Speed" option that causes emulation speed to wait for the sound to play for its buffer length so it can proceed at a speed that lets the sound play perfectly smoothly. Even official versions do this. (Setting the throttle to something else than 100% used to disable synchronize which is probably why you didn't notice it happening before.) It's not related because he was most likely using -synchronize 0 when recording. (EDIT: err, actually it is related, he was just talking about playback, I didn't read carefully enough. This is how the emulator behaves normally, I don't know how to fix it besides turning off synchronize which has even worse side-effects.) (another EDIT: I just found why the frame size would be screwed up, I thought I was passing the exact pixel data that was being passed to the AVI output, but I missed that it was calling another helper function to strip off black pixels and convert the bit depth first. It doesn't appear to be 4 pixels though, it looks to be 2 on the right and 1 on the top, from the code.)

Posted: 4/1/2005 4:12 AM

Quote

nitsuja

Emulator Coder, Skilled player (1300)

Joined: 12/21/2004
Posts: 2687

Here's another version: http://nvdata.pilif.ch/VBA-rerecording-new4.zip I tried fixing the video skew and un-offsetting the sound offset, but right now I can only guess that these work as intended. (For when you're not recording, any delay is probably due to latency in emulating DirectSound or something. I've never experienced a sound delay that could be noticed when listening at 100% speed, and of the few frames of delay there are, I think some/most of that is really supposed to be there. EDIT: OK, perhaps I'm wrong about that, I just tried Shanedudddy's version instead of the official version and noticed the sound delay of 6 frames or so is gone, I'll take another look at what changes he made... EDIT2: On further investigation, the only reason you could hear sound effects at exactly the right frame before was because the sound kept playing in a loop on frame advance. There is no way around the fact that the sound that plays from a sound buffer must be delayed by the sound buffer length, which is currently about 6 frames. I could make the buffer length configurable, however, so making it shorter would reduce sound latency, but for now that's not necessary, it can't affect the AVI anyway.) Also, in case you still wanted to know this: The sound is always 44100 Hz, 16-bit, Stereo. And I just verified that VBA emulates all games at 60 fps.

Posted: 4/2/2005 1:44 AM
(Edited: 1/23/2022 7:41 PM)

Quote

Bisqwit

Editor, Active player (296)

Joined: 3/8/2004
Posts: 7469
Location: Arzareth

nitsuja wrote:

Here's another version: http://nvdata.pilif.ch/VBA-rerecording-new4.zip

Much better, but still not quite there! :) https://files.tasvideos.org/bisqwit/vba-problem2.avi https://files.tasvideos.org/bisqwit/vba-problem2b.avi It seems that Windows has a _popen() function, by the way. I fixed the noticed problems with nesvideos-piece and created the proper header file for it. http://bisqwit.iki.fi/kala/nesvideos-piece.cc http://bisqwit.iki.fi/kala/nesvideos-piece.hh I don't understand what the offset parameter in NESVideoLoggingAudio call was for. Please don't add it there - add it to the calling mechanism instead. I'd like to keep the module as much unchanged between emulators as possible. As for A/V sync problems, I don't care about DirectSound buffer things as long as my only motive is to get AVI recordings of GBx games. But even considering that there seems to be a sync problem. Is it possible to force the audio generator to start at the same time as the video generator, not earlier, so that the audio generator won't take a headstart?

Nitsuja wrote:

Also, in case you still wanted to know this: The sound is always 44100 Hz, 16-bit, Stereo. And I just verified that VBA emulates all games at 60 fps.

Good!

Posted: 4/2/2005 2:25 AM

Quote

nitsuja

Emulator Coder, Skilled player (1300)

Joined: 12/21/2004
Posts: 2687

Haha! Sorry, I know exactly why it does that. Their code that strips away the black pixels also flips the whole thing upside-down as a Windows image formatting thing, I'd forgotten about that. The thing that worries me more is the audio offset. The offset I added is completely arbitrary, because there is nothing to correct, no internal source of sound-to-video lag anywhere that I can find. In fact, I output the sound at the exact same time as it would get laced into the AVI, and that stays perfectly synchronized. EDIT: actually, if you could test this with the -outputAVI flag and see if it also works that way for you, that would help a lot. The delay seems to be almost exactly 0.5 seconds, minus the 1/3 of a second I arbitrarily subtracted to try to fix it, which as you point out isn't the right way to fix it anyway. But I don't think I can output sound 0.5 seconds ahead of when I do without predicting the future. I really think it has to do with the 1 second of audio delay that the audio logging function adds the first time it's called. That's the only real difference I can find between the sound logging and the sound+video AVI output. (Perhaps a test of what happens when no audio or video delay is added would clarify things.) A miscalculation of the bytes_per_second variable would explain it completely, but I've looked many times and haven't found anything wrong with that.

Posted: 4/2/2005 2:42 AM

Quote

nitsuja

Emulator Coder, Skilled player (1300)

Joined: 12/21/2004
Posts: 2687

Uh, wait a sec, is it just me or does the sound in vba-problem2.avi come too late in the video, and the sound in vba-problem2b.avi comes too EARLY the video? I thought it was off by a constant amount, do you mean it actually changes per game?

Posted: 4/2/2005 2:54 AM

Quote

ventuz

He/Him

Player (123)

Joined: 10/4/2004
Posts: 940

aw man, it made my movie looks bad, pick up sample run instead ;) uh yea it sound like came EARLY than on actually handheld/console.

Posted: 4/2/2005 3:29 AM

Quote

nitsuja

Emulator Coder, Skilled player (1300)

Joined: 12/21/2004
Posts: 2687

It's coming out early just because I subtracted from the delay time at the beginning, not because the sound generation is preceeding the video. What I'm wondering about is why the sound in Mario Land didn't come out too early. Is there a difference because one is SGB and the other is GB? Do other games follow the same pattern, or was the sound in one manually put at a different position to show just one problem (upside-down-ness) at a time? Also, the code to add the logo seems to completely drop missing frames without adjusting the audio delay time, and it looked like the Mario Land one was missing logo frames (or it could just be my AVI player skipping).

Posted: 4/2/2005 11:36 AM

Quote

Bisqwit

Editor, Active player (296)

Joined: 3/8/2004
Posts: 7469
Location: Arzareth

nitsuja wrote:

Also, the code to add the logo seems to completely drop missing frames without adjusting the audio delay time, and it looked like the Mario Land one was missing logo frames (or it could just be my AVI player skipping).

It drops them quietly without adjusting, if the frames are not found on disk, but in these samples, there were no dropped frames. I also encoded it with "-mc 0 -noskip" options so no frames were dropped/duplicated. I used the same settings for both AVIs.

nitsuja wrote:

if you could test this with the -outputAVI flag and see if it also works that way for you, that would help a lot.

I tried -outputAVI. It prompted for a filename, but when I selected an AVI and pressed ok, it ended up not actually producing anything. It ran the game normally (with unlimited speed), but a file was not written. By the way, when I select 1x view scaling at Options | Video | 1x, here's what it does:

That is, it's not really 1x (it's smaller), because it miscalculates the window size because of the menu that wraps. It doesn't matter in AVI creation, but it's an inconvenience otherwise.

Posted: 4/2/2005 12:15 PM

Quote

Bag_of_Magic_Food

He/Him

Active player (277)

Joined: 5/29/2004
Posts: 5712

Well, once you finally get it working, don't forget to take that announcement off the GB board!

put yourself in my rocketpack if that poochie is one outrageous dude

Posted: 4/3/2005 1:46 AM

Quote

nitsuja

Emulator Coder, Skilled player (1300)

Joined: 12/21/2004
Posts: 2687

OK, outputAVI probably relies on DLLs and codecs being loaded into Windows so I'm not too surprised it didn't work. I have a better idea of what's happening now after doing some more testing and putting in better debug checks, but the (pre-existing) problem is pretty major so it might take a little while longer to fix. Basically, the video isn't always output at a constant rate; sometimes the GB switches into a mode where it doesn't output video data or accept input, and that means frames don't get added into the AVI during that time even though time is passing by. It's only a few frames here and there but it causes the audio and video to drift apart in some games, and certain actions (such as leaving the Mario Land title screen) cause much more drift than others. The easiest way to fix this would be to insert dummy frames into the AVI when the emulated system is stalling its video like this. My only concern with this is that it will make the AVI slightly longer than the time calculated from the number of emulated frames, so I'm not sure if the movie file should store extra information of how many dummy frames there are, or if the time difference is within the acceptable range attributable to quirks in emulation timing and conversion to AVI. EDIT: hmm, maybe this is not as big a problem as I thought. The test AVI's I'm getting look/sound great now, with no data needing to be dropped or noticeable differences from emulation speed. Now to make the video right-side-up...

Posted: 4/3/2005 8:35 AM

Quote

nitsuja

Emulator Coder, Skilled player (1300)

Joined: 12/21/2004
Posts: 2687

Here is yet another version, I hope this one works acceptably well: http://nvdata.pilif.ch/VBA-rerecording-new5.zip Video should be right-side-up (I should've claimed that was an april fool's joke), and sound should be synchronized with video. And I didn't have to change either emulator core's timing as I'd thought might be necessary, so the movies people are working on now should still work. Also, I added the flags -videoScale. -hideMenu, and -pauseAt. So to fix your problem with the 1x scale not really being 1x because of the menu, you can try using -videoscale 1 -hidemenu, if you're sure you won't need to access the menu directly that session. (It actually properly detects the menu's height for me, so it's probably a Windows version difference problem.) By the way, _popen() is not what it seems like it should be (it tries to run the file in a new console window), so I had to change it back to fwrite().

Posted: 4/3/2005 12:25 PM
(Edited: 1/23/2022 7:42 PM)

Post subject: new nesvideos-piece

Quote

Bisqwit

Editor, Active player (296)

Joined: 3/8/2004
Posts: 7469
Location: Arzareth

nitsuja wrote:

By the way, _popen() is not what it seems like it should be (it tries to run the file in a new console window), so I had to change it back to fwrite().

Maybe it would have worked properly in Wine. Now I never know. Or maybe it would have been possible to do it with CreatePipe and such calls. You wrote a comment: // windows has _popen but that actually tries to RUN the file, not open it for output which sounds exactly like what it was _supposed_ to do. The idea is that I want to execute the program that handles the movie stream directly instead of having to pass it via an intermediate file. With the fopen technique, I can still use a "fifo" - that is, a file-looking entity (that can be fopened) that actually is a pipe (the data written to the file is immediately read by the program that opened the pipe for reading, without the data ever visiting the filesystem). The problem with "fifo" is that I need to launch the readers before launching the emulator, and if the emulator never actually opens the pipe(s) for writing (because of a parameter error for example), the readers will be left running in a waiting state even after the emulator exits, everafter waiting until someone finally opens the fifo for writing. ------------

nitsuja wrote:

http://nvdata.pilif.ch/VBA-rerecording-new5.zip

I tested this one... After great studies, it was revealed that when there are onscreen texts, such as the input display or frame counter, the emulator can only run at ~3% speed and eats 70-90 % of system resources. TERRIBLE lag. When there are no onscreen texts, it runs fine. Do you have an idea why it happens? ------------ I also noticed that wine hogs about 1.3 GB of virtual memory upon start, no matter what its executing. :o I have 1 GB of RAM in this machine - maybe it's related. ------------ When running VBA, wine says a lot of: err:dsound:DSOUND_MixOne underrun on sound buffer 0x77c4ae90 err:dsound:DSOUND_MixOne underrun on sound buffer 0x77c4ae90 err:dsound:DSOUND_MixOne underrun on sound buffer 0x77c4ae90 err:dsound:DSOUND_MixOne underrun on sound buffer 0x77c4ae90 err:dsound:DSOUND_MixOne underrun on sound buffer 0x77c4ae90 and this has happened with all VBA versions so far. It happens very very rarely with Famtasia, even if it's ran on 900% speed. When it happens, sound chops and may sometimes turn completely mute, as if some error situation is never resolved. ------------ Then there was some audio desync problem. https://files.tasvideos.org/bisqwit/vba-problem3.avi In this AVI, it seems like mencoder is receiving only half a second of silence. https://files.tasvideos.org/bisqwit/vba-problem3b.avi In this AVI, the amount is different. _Something_ eats the silence... It could be some pessimization of LAME. In an attempt to solve it... ------------ I created an updated nesvideos-piece module again. News: It creates now a real interleaved AVI file instead of two separate streams. For this reason, it now only requires one Set...Cmd call. NESVideoSetAudioCmd() no longer exists. http://bisqwit.iki.fi/kala/nesvideos-piece.cc http://bisqwit.iki.fi/kala/nesvideos-piece.hh Also, the calling semantics of NESVideoLoggingAudio() and NESVideoLoggingVideo() have changed. Most importantly, NESVideoLoggingAudio() now requires specifying the exact format of the audio data. The relevance of this update is important: it resolves all the nasty deadlocking issues that happen when pipes are opened at different times and programs expect them in a certain order. It _may_ also fix the A/V sync problems. At least it works fine with FCEU :) However, for the logger to be usable for DeHackEd and others too, the logo creation would have to be sani...argh...ed a bit.

Posted: 4/4/2005 2:24 AM

Quote

nitsuja

Emulator Coder, Skilled player (1300)

Joined: 12/21/2004
Posts: 2687

I found a problem where I was assuming that audio and video are being logged at the same time. It should actually stay synchronized if you use -videolog with -audiolog simultaneously. But, I'll take a look at your update, that sounds like a better solution. About the text display lag - what it actually does (in all versions of VBA) is request the video memory and ask Windows to write directly onto that, drawing the text over the game and anything else that might be there on the screen. It runs pretty well for me, but maybe that's an extremely slow operation to emulate. If you're able to change the mode from DirectDraw to something else, maybe that will help, but I'm guessing not. This is a very fixable problem, though, I can put the text in the image that's being drawn to the screen instead of on top of the screen. And I was probably wrong about _popen, but the problem for me is that it always opens it in Windows Media Player which doesn't know what to do with the data because it apparently isn't in a valid AVI format, even after some of it's been written.

Posted: 4/4/2005 8:11 AM

Quote

Bisqwit

Editor, Active player (296)

Joined: 3/8/2004
Posts: 7469
Location: Arzareth

nitsuja wrote:

If you're able to change the mode from DirectDraw to something else, maybe that will help, but I'm guessing not.

GDI expects 32-bit surface, and I'm using 16-bit. No work. GL modes don't work, because I don't have opengl properly installed.

nitsuja wrote:

This is a very fixable problem, though, I can put the text in the image that's being drawn to the screen instead of on top of the screen.

If then please. :)

nitsuja wrote:

And I was probably wrong about _popen, but the problem for me is that it always opens it in Windows Media Player which doesn't know what to do with the data because it apparently isn't in a valid AVI format, even after some of it's been written.

WMP is very picky unlike MPlayer. AVI is not meant to be streamable, but it works very well with MPlayer. I want to try that _popen thing in Wine however. Or maybe if you can make it a commandline flag so I don't have to wait for the next day when you're next available in case it doesn't work :)

Posted: 4/4/2005 5:26 PM

Quote

TNSe

Emulator Coder

Joined: 10/9/2004
Posts: 453
Location: Norway

I suggest if you write it directly to the video surface, write it *after* screenshot code, and *after* giving it to AVI writer :P

Posted: 4/4/2005 7:29 PM

Quote

nitsuja

Emulator Coder, Skilled player (1300)

Joined: 12/21/2004
Posts: 2687

Bisqwit wrote:

GDI expects 32-bit surface, and I'm using 16-bit. No work.

That's odd - my monitor is always in 16-bit mode and GDI works for me. I don't use it since I find that mode pretty inefficient at rendering, though. I believe the text is currently drawn using GDI even when nothing else is, which may be why it's so slow for you.

nitsuja wrote:

AVI is not meant to be streamable, but it works very well with MPlayer. I want to try that _popen thing in Wine however. Or maybe if you can make it a commandline flag so I don't have to wait for the next day when you're next available in case it doesn't work :)

OK, I'll make it a flag. _popen does handle "wb" differently from "w", so it'll still always use "wb". I'm also adding general-purpose debugging functionality to verify in real-time that all the frames written and number of seconds it thinks have been written are matching up between audio and video. I'm unclear about what the audio logging parameters are - there seem to be 2 rate parameters, 1 called rate and 1 called something like samples. I think I've determined that rate is the number of samples per second (44100 for me) and samples is number of samples being written at the moment, but maybe this should be more explicit in the header (I thought one of them would be the number of bytes per second but neither seems to be).

TNSe wrote:

I suggest if you write it directly to the video surface, write it *after* screenshot code, and *after* giving it to AVI writer :P

I'll make it an option to draw text either "on top of screen" (like now), "on top of game" (after filters and AVI output), or "in game" (before filters and AVI output). They already have most of the code in for doing these, I think the only reason it's currently not done is that the GB screen isn't wide enough to fit much of a message in, and certain filters (2xSaI) selectively update parts of the screen which would leave chunks of text over places that haven't changed in-game when using those filters.

Posted: 4/4/2005 7:38 PM

Quote

TNSe

Emulator Coder

Joined: 10/9/2004
Posts: 453
Location: Norway

Another request, change the Cooperative level on the direct sound buffer, so that even if I switch away to another window, the gameboy keeps making sound... Edit: For some reason, with the input, VBA always receives input, even when inactive, but with sound, it doesn't give sound when inactive, I'd like this reversed. Keep on playing sound when not active, and stop receiving input when inactive Also please add an always-on-top option (I cant seem to find this option atm). Both would be win32 specific additions.

Posted: 4/4/2005 8:15 PM

Quote

nitsuja

Emulator Coder, Skilled player (1300)

Joined: 12/21/2004
Posts: 2687

TNSe wrote:

Another request, change the Cooperative level on the direct sound buffer, so that even if I switch away to another window, the gameboy keeps making sound...

I think I've tried all the possible cooperative levels and none of them had this effect, DirectSound seems to always insist the window is active to play anything, unless I'm forgetting about some setting or flag.

TNSe wrote:

For some reason, with the input, VBA always receives input, even when inactive.

Changing DirectInput's cooperative level from the default should fix this. (Will make this an option too.)

TNSe wrote:

Also please add an always-on-top option (I cant seem to find this option atm).

I saw some code to do this that was commented out, that probably means it didn't work right but I'll try it out...

Posted: 4/4/2005 8:23 PM

Quote

Bisqwit

Editor, Active player (296)

Joined: 3/8/2004
Posts: 7469
Location: Arzareth

nitsuja wrote:

I'm unclear about what the audio logging parameters are - there seem to be 2 rate parameters, 1 called rate and 1 called something like samples. I think I've determined that rate is the number of samples per second (44100 for me) and samples is number of samples being written at the moment, but maybe this should be more explicit in the header (I thought one of them would be the number of bytes per second but neither seems to be).

rate is the sampling rate, for example: 44100 bits is 8 or 16 chans is 1 or 2 nsamples is the number of samples being written in this call. As the comment says, "nsamples*chans*(bits/8) = bytes in *data." Which means, that the "data" buffer is expected to contain exactly nsamples samples, each of which is chans*(bits/8) bytes in size.

Posted: 4/4/2005 8:24 PM

Quote

TNSe

Emulator Coder

Joined: 10/9/2004
Posts: 453
Location: Norway

I believe what you want is DirectSound->SetCooperativeLevel(HWnd, DSSCL_EXCLUSIVE) (Before you create primary buffer, and set the format of it)

Locked