Editor, Experienced Forum User, Published Author, Expert player
(3632)
Joined: 11/30/2014
Posts: 2773
Location: US
Bigbass wrote:
SNES might become a lot more possible in the near future. rasteri has created a hardware mod (open source) that essentially synchronizes the APU (audio) clock to the CPU clock, which should improve how deterministic the console is. But more testing is needed.
Cool project! I'll definitely mod my SNES when its finished.
Editor, Experienced Forum User, Published Author, Expert player
(3632)
Joined: 11/30/2014
Posts: 2773
Location: US
TiKevin83 wrote:
I would also argue the whole PC TASing scene is quite interesting as to how it questions the very idea of what a verification is - usually it applies only to a single SKU of a system, eg the GBA/GBP variant of the Game Boy Color, but my Backyard Baseball TAS for example is replicable across systems of wildly different hardware capabilities. There's not really any interesting accuracy insight that could be added by eg sending the inputs over mock mouse/keyboard over usb instead of through libTAS since you're then validating lag behavior of the OS layer much more than the hardware. So it kinda shows how our concept of verification is tightly tied to these earlier bare metal systems without an OS layer or ones where the OS is very meager and without any/many revisions.
It would be cool if more modern PC games offered built in TAS / replay tools, but its probably too niche for devs to bother.
Patashu wrote:
There are much harder N64 games to console verify, because we can't even emulate them accurately, because to emulate their lag requires cycle accuracy and their lag changes physics. Goldeneye being a good example of this (at least the last time I checked)
SNES is also very hard to console verify, IIRC the reason is because two different components have their own timers and in any actual SNES their timers are slightly desynced due to physical inaccuracies.
CasualPokePlayer wrote:
Even if you have some perfect accuracy, you still end up screwed due to non-deterministic behavior. You have the bootup time of the game being randomized with a hardware RNG (thanks Nintendo), and then you have the clock relationships between the RCP (i.e. what controls many different components of the N64) and CPU being dictated by a PLL clock multiplier (which ends up having a similar in concept issue with SNES, although it's still one clock, just that clock being multiplied cannot have the multiplication done deterministically, so you get a issue similar with having 2 separate clocks).
My opinion on this is that either a hardware mod is needed, or some other new form of TAS is necessary for these systems. Maybe something like a scripting language + machine vision can account for non-determinism in real time? At least for SNES I have personally seen that verification is just generally not possible with the same pipeline as NES.
Bigbass wrote:
NES:
[*]There's [2137] NES Spy vs. Spy by Sir VG in 08:46.52 which continuously polls for input. The current publication may not even be verifiable, idk. ViGrey and I both struggled to make sense of how to dump, let alone replay, inputs for that game.
[/list]
Oh this is a very interesting case that I was not aware of (or maybe just forgot.) You really never know which games will be troublesome until you look.
Of course accounting for everything that a console / game could do becomes a bottomless pit of time and effort, so maybe picking 'capstones' from amongst games that do 'reasonable' things only could tame the problem a bit.
This discussion reminds me how young a technology console verification still is.
Editor, Experienced Forum User, Published Author, Expert player
(3632)
Joined: 11/30/2014
Posts: 2773
Location: US
Dimon12321 wrote:
Oh. I though EEPROM is used for game saves and high score boards only, and I didn't make a single save in my run. It goes out of sync about half way through Level 1-3.
I assume, the reason is I changed the weapon auto-switching option and the emulator wrote to the wrong part of RAM which caused extra lag frames and/or RNG change. Am I on the right track? Just a guess.
My Doom TAS of Level 1 goes out of sync when I save the game at the end on v2.1.3. More time is spent saving
Yes, when you change weapon to auto-switch it writes to EEPROM, and that causes the desync.
Even with the correct EEPROM size, timing can still be variable. If you want a complete run to work on console, I recommend only TASing like one level / stage past the first EEPROM access / save, then test on console to make sure timing on that particular cart matches emulator before continuing.
Or maybe for Duke Nukem its not a big deal to avoid to weapon switching thing to preserve determinism.
Editor, Experienced Forum User, Published Author, Expert player
(3632)
Joined: 11/30/2014
Posts: 2773
Location: US
Dimon12321's comment: "When every known game can be played back, only then Duke Nukem can be played back too" got me thinking, what are the most challenging runs to console verify for each system? What would be the run that signifies pretty much any other run could be verified as well?
I guess there are a lot of factors to consider for such a question. Some games interface with special hardware so could be their own challenge, but aren't necessarily relevant to the rest of the game library. Some consoles / games have various levels of non-determinism, so could be difficult just in requiring extreme luck. In general you have to know a lot about the system and games library to really pick out the most challenging ones. Some games certainly require perfect emulation, but you have to find them case by case.
Anyway I thought the topic was interesting enough to write down my thoughts, might be interesting to have some goals to aim for.
NES: We might already be there with Bad Apple. It would hard to push both the system and the console verification tools harder than that. For a more standard TAS, there are many runs that require cycle accurate emulation of the entire system, so it's hard to pick just one, though Time Lord is particularly challenging.
GB/C: Here we have an interesting case of linked play. Pokemon Coop Diploma would probably the most complex case possible. For single system runs, the Pokemon yellow ACE might be it. For normal TASes, probably we are already there with Pokemon Crystal and Men in Black.
GBA: Currently the game that I know requires cycle accuracy for the entire run is Banko Kazooie: Grunty's Revenge. That game also needs quite a few difficult edge cases to be emulated to get that cycle accuracy. The GBA library is huge and I have tested very little of it, so there could be more challenging cases out there. Linked play is also something to explore.
Any other examples I missed?
These are the systems I have the most knowledge about. I'd be interested to hear from other people knowledgeable in other systems to learn about some of the challenging 'capstone' like examples there as well.
Editor, Experienced Forum User, Published Author, Expert player
(3632)
Joined: 11/30/2014
Posts: 2773
Location: US
CoolKirby wrote:
In Zebra Pen 1, did you skip a dance at the end? It sounds like Private's voice starts to play, then cuts out and the BGM restarts in the second room. Something similar happens at the end of Monkey Cages 2 with Alex.
At 1:27, you toboggan across part of that pond, but on revisits at 10:37 and 16:37 you don't. Could it also be faster to toboggan across at 13:50, 17:20, etc? Otherwise, looks good and it's great you shaved off almost a full minute.
That is just the notification for collecting 100 medals and earning an extra fish in your health bar.
I don't think that saves time, as far as I can tell it's the same speed as jumping.
Editor, Experienced Forum User, Published Author, Expert player
(3632)
Joined: 11/30/2014
Posts: 2773
Location: US
Dimon12321 wrote:
Alyosha wrote:
I was finally able to test the Duke Nukem test run that Dimon12321 made. Works fine on console.
Holy Father! I didn't expect it to sync at all, knowing it's unusual hardware utilization, 3D graphics and uncapped framerate, because all three FPS TAS tests I've provided are lag-sensitive.
Judging by verifications of side-scrollers, all those RNG troubles, I was convinced "When every known game can be played back, only then Duke Nukem can be played back too". Well, that feels different now.
I made a 100% low-effort TAS of Duke Nukem. In case you'd like to check it even further, here's the movie: User movie #638510452607262334
Regardless, you've influenced my TASing priorities!
Unfortunately the 100% run desync in v2.1.3 because 2.1.2 used the wrong EEPROM size.
If you plan on TASing Duke Nukem or Doom, be sure to use v2.1.3 where these EEPROM issues are fixed.
Editor, Experienced Forum User, Published Author, Expert player
(3632)
Joined: 11/30/2014
Posts: 2773
Location: US
Incorporating FriedPorkchop's improvements was easier than expected. I was able to resync the most recent WIP and made a couple minor improvements as well. I also improved the latter half of the run mainly by incorporating the new tech and by realizing the sailors can be attacked normally, they don't all need to be darted.
Please replace the movie file with this one:
https://tasvideos.org/UserFiles/Info/638534902598449672
it is console verified as well, I will update the temp encode.
Editor, Experienced Forum User, Published Author, Expert player
(3632)
Joined: 11/30/2014
Posts: 2773
Location: US
BigBoyAdvance wrote:
Sorry if posting in wrong thread.
Game & Watch Gallery 4 (USA) seems to fail linking together (Boxing 2P mode) in the newest GBAHawk. Is there anything I'm doing wrong? It's my first time using this emu for GBA link play.
I haven't worked on linking in some time, so it's probably just broken for that game. I'll make a github issue for it, thanks for the report.
Editor, Experienced Forum User, Published Author, Expert player
(3632)
Joined: 11/30/2014
Posts: 2773
Location: US
I was finally able to test the DKC 2 run that RetroEdit had resynced a while ago.
This run desyncs in Glimmer's Galleon. This appears to be a legitimate desync caused by emulation error. I checked trivial things already. Game does not seem sensitive to EEPROM timing. Inputs are checked at normal times, not close to frame boundaries which can sometimes effect sync.
I think I found where it desyncs, and there is a frame in the area that is only about 50 cycles away from being a lag frame.
It's not very likely I can track down the cause of the desync this deep into the run. I also currently don't have time to dive into trace logs and figure out what's happening. So, this one will probably remain unsolved for a while.
Some mysteries still remain!
Editor, Experienced Forum User, Published Author, Expert player
(3632)
Joined: 11/30/2014
Posts: 2773
Location: US
I tried resyncing the Snood "Puzzle' run but it desyncs in GBAHawk and on console the same way. The run only makes it to the third level before desyncing, and does not seem to be fixable.
Depending on luck, a new console accurate run may be slower. I don't have time to work on it right now, so for the time being I'll just add it to the bad RNG list.
Editor, Experienced Forum User, Published Author, Expert player
(3632)
Joined: 11/30/2014
Posts: 2773
Location: US
CoolKirby wrote:
Have you also read the forum thread? FriiedPorkchop discovered some glitches involving tobogganing mode that help preserve speed and can even clip through things, skipping part of Arctic Pen 3. The Giraffe Pen demo encode used the mGBA core, while the BK2 from the later post used VBA-Next in BizHawk 1.11.1, and I couldn't get it to sync in BizHawk 2.3.2 so it might require an older version. I did format and paste its input into a GBMV but only the very first level (Mr. Chew) syncs, the rest may need to be resynced for every room if you want to directly compare strategies.
Nope I definitely did not read the thread, rookie mistake me!
I'll try to implement this stuff in the next couple weeks.
Editor, Experienced Forum User, Published Author, Expert player
(3632)
Joined: 11/30/2014
Posts: 2773
Location: US
Actually with 2 reset manipulations i get an even better fight than the mgba version!
Sop i think I'll go with that one. Will post file once i verify it.
EDIT: new file: *supercded see below*
Please replace the movie file with this one.
updating temp encode, console verified
Editor, Experienced Forum User, Published Author, Expert player
(3632)
Joined: 11/30/2014
Posts: 2773
Location: US
SuperSqank wrote:
I have made an updated TAS. https://tasvideos.org/UserFiles/Info/638530054286630870
It saves approximately 21-22 seconds compared to Alyosha's update. I think it should be a sub 28 when resynced to GBAHawk but even if it isn't, it should be very close. The main time saves come from much heavier use of double/triple jump storage along with not having to reset in the final level. There are various other optimisations and time saves as well. The final level should end at 1:38 IGT. For reference, I'll also put up a video link to the run.
Link to video
That was fast, nice work!
Resyncing is not entirely successful though. I can get 10 foosas at the dance part, but 30 frames slower since their positions aren't optimal. More important though the arena doesn't sync at all and I can't seem to manipulate it? How do you manipulate that part besides the reset method in the original? If I just resync it as is it will be significantly slower.
It's kind of surprising the arena doesn't work even though it worked when resyncing from the original run on vba.
Here is what I have:
*removed*
EDIT: The arena also desyncs on console, so definitely needs to be redone.
Editor, Experienced Forum User, Published Author, Expert player
(3632)
Joined: 11/30/2014
Posts: 2773
Location: US
SuperSqank wrote:
Nice run. I’m not particularly familiar with this game but one thing I can say is that you can fully skip dialogue boxes with the start button. That’s the only improvement I can think of however, the rest of the run seems clean.
I didn't even think to try that, thanks for the tip! I'll work on implementing it.
Editor, Experienced Forum User, Published Author, Expert player
(3632)
Joined: 11/30/2014
Posts: 2773
Location: US
SuperSqank wrote:
I’m cool with looking at it. I was actually interested in TASing the game myself, willing to remake the whole thing. I will see what I can do and I will look into sending you an improved movie when I’m done. Could you provide a BK2 or TASPROJ file for the run?
Here is a bk2 but note that it will desync around frame 8000 due to incorrect lag emulation in mGBA. I can always resync a final movie back to GBAHawk again later anyway for console verification.
https://tasvideos.org/UserFiles/Info/638528620588612122
Editor, Experienced Forum User, Published Author, Expert player
(3632)
Joined: 11/30/2014
Posts: 2773
Location: US
^ Thanks for the detailed comments. Are you interested in improving the TAS? This run is short and sync friendly enough that it could be improved relatively easily so I don't mind just cancelling it.
Editor, Experienced Forum User, Published Author, Expert player
(3632)
Joined: 11/30/2014
Posts: 2773
Location: US
Bigbass wrote:
Nice job Alyosha! Also, if you count obsoleted TASes, we're currently at 521 verified publications. :D
As for the N64.. some in-depth testing needs to be performed to really know how viable the system is for verifications at this time. SM64 and a couple other games are verifiable already, even with the much less accurate mupen64 core. But this appears to be because the way those games poll and make use of controller input, makes them unaffected by timing inaccuracies, at least as far as the replay device's ability to stay in sync. If you actually measured the realtime it took to replay, I'm certain it would be different than the emulator (based on what Ridley has explained to me, through her efforts regularly verifying SM64 TASes.)
I suspect it might be possible to "scan" the N64 library of games with a Lua script in BizHawk, to try to predict games that are similar to SM64. But you'd have to manually play each game to induce some lag into the test.
RDRAM initialization shouldn't affect anything. Maybe if a game depends on the initial state somehow? but I haven't heard of any N64 game doing that, and the initial state of RDRAM may actually be deterministic (needs more research). If necessary, it may be possible to initialize at least most of the system to expected values before starting the game/TAS replay.
I was considering starting N64 testing myself with the Ares core, but my impression is that I should probably wait for a few more updates as it's not quite there yet.
For NES, GB/C, and GBA, pretty much every edge case emulation detail has been necessary for console verification for at least some games. Assuming this extends to other systems as well, it's hard to dedicate a lot of time to testing before a high degree of accuracy is guaranteed from the emulator. But not being an N64 dev myself it's also hard to judge when that point is reached.
Still maybe it would be a good idea to make some simple test runs for games like Goldeneye or Rogue Squadron to check if replay is at least deterministic for games developed by other publishers.
Editor, Experienced Forum User, Published Author, Expert player
(3632)
Joined: 11/30/2014
Posts: 2773
Location: US
We just hit 400 current console verified TASes! Another hundred milestone hit about 14 months after the 300 one, relatively quick progress.
Despite this being a big numerical milestone, it all is still coming from the same systems. There is however a lot of interesting development happening that could lead to some verifications on newer systems in the not so distant future.
There is a big PR open in melonDS ( https://github.com/melonDS-emu/melonDS/pull/1955 ) that implements a lot of the complicated cache stuff. Seems to fix some timing issues so it could be a big step forward in DS accuracy once it gets merged. Looks like it still needs a bit of work but pretty exciting to see such a complicated thing being worked on. DS seems like it is the next likely console where a mature console verification pipeline can be built up. I'm looking forward to seeing this happen.
The N64 situation in Ares is also looking good. N64 has RDRAM initialization though, which could impact determinism to some extent. I'm no expert so I don't know how bad it could be. Obviously some runs sync reliably on N64, but this might not extend to more of the library.
If anyone was watching the Team 0% stuff in Mario Maker, they know that the last level was uploaded using TAS playback tools for the Wii U. I found this really surprising and it's cool too see stuff like that come out of nowhere. I haven't researched any of the technical details, but presumably the inputs were hand crafted similar to the Mario Maker 2 demo by TASBot folks. Maybe this approach has some potential for longer runs with some more work to make it less painful?
I personally don't have time to start any new big projects, so I'll be grinding out GBA verifications for the foreseeable future. There are still a lot I want to do.
Onward to 500,
Editor, Experienced Forum User, Published Author, Expert player
(3632)
Joined: 11/30/2014
Posts: 2773
Location: US
v2.1.3 is now released, fixing the EEPROM size detection problem here (and in the other run.)
Resyncing is just a matter of adding lag frames, I can provide them tomorrow if desired.
Editor, Experienced Forum User, Published Author, Expert player
(3632)
Joined: 11/30/2014
Posts: 2773
Location: US
GBAHawk v2.1.3 is now released. This is a bug fix release which reworks EEPROM size detection (fixing probably ~1000 incorrectly entries based on MAME DB.) I probably should have done this from the start but didn't realize how bad the state of affairs was. This also re-balances audio between FIFO channels and GB style channels.
Next release will have manual Flash timings, which I wanted for this release but don't currently have the time to sort out properly, as there are a lot of variables.
I haven't encountered any core accuracy problems in a while which is promising. I know some of some very edgy edge cases that could cause problems, but they haven't popped up yet, so for now I am content to keep grinding out console verifications.
EDIT: There was am bug in audio mixing, I fixed it and updated the 2.1.3 executable / downloads.
Editor, Experienced Forum User, Published Author, Expert player
(3632)
Joined: 11/30/2014
Posts: 2773
Location: US
feos wrote:
Alyosha wrote:
What you want to do here is verify that the RAM state at the end of the SMB3 movie matches that at the start of the SMB1 movie. BizHawk lets you export RAM as a text file (from hex editor), which can then be used by online diff checker or something similar to verify they are the same.
I did so and they match, so it's good.
You mean even tho savestates don't match, RAM state inside them does?
Yeah. But really the savestate is an unnecessary complication here since NESHawk has the ability to load RAM via sync setting.
Here is a .bk2 version of the run that starts from power on and loads RAM in this way so the savestate is avoided. I played it back and it syncs just fine:
https://drive.google.com/file/d/1iss7PgrndTHwzg3UCJ9TBj29u6LxgTlV/view?usp=sharing
I would suggest using this version as the publication file actually, since savestate is not garaunteed forwards compatible, and the verification movie only really generates RAM anyway.
