Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
This game is pretty sync friendly, so here is a resynced level 1 which incorporates the cutscene skips as well as adds in a few time saving jumps from the RTA world record. EDIT: more levels done.
I'll implement andypanther's new route as I go.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
Really cool run, nice job figuring out a good route here. I'm surprised just how much faster this is than the old select glitch route. Also I think this game looks way better in color.
I think this run can obsolete the current select glitch run.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
Noxxa wrote:
Alyosha wrote:
This makes it seem like fixing emulation wasn't really a factor and the movie would have been changed regardless.
Does it really matter? The movie had multiple reasons to be resynced/adjusted, so both were done at the same time. Either change individually would've been a benefit to the product in the long run.
Yes it does matter, as this isn't the only movie that suffers from emulation issues, so i would like to know if that is now sufficient reason to make an official resync as was done here, or if it still isn't.
Thanks for the other answers.
This run was resynced to BizHawk 2.8 by DrD2k9 in order to fix the poor sound emulation from the original run, as well as to remove a piece of outdated language.
This makes it seem like fixing an emulation issue was a primary reason of changing the movie file, but then in the submission it says:
Samsara: The publication file has been replaced with a resynced version on BizHawk 2.8, courtesy of DrD2k9. This was done primarily to replace one piece of outdated language used in the original run, but also has the added benefit of more accurate sound emulation.
This makes it seem like fixing emulation wasn't really a factor and the movie would have been changed regardless.
Also did Heisanevilgenius agree to the change in language? That seems like an important thing to note considering authorship wasn't changed despite the content changing in a non-trivial way.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
On level 17 it is faster to jump on the right edge. The latter levels required some resyncing after this, but I still ended up ~10 frames faster overall.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
https://tasvideos.org/UserFiles/Info/638138923631475487
Here is a resync of this run that works on BizHawk 2.8 This version is console verified, so that question is resolved.
I didn't make an encode as this probably isn't the final version. I lost a bit of time (~15 frames) on level 6, mostly at the end of the level. Maybe Pankaj can improve it.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
Pankaj wrote:
I'm currently using NESHawk (BizHawk 2.8), will definately try to make a new TAS on BizHawk but not any sooner.
So far I tried converting this fm2 into bk2 and shifted the inputs 1 frame earlier (as BizHawk power on 1 frame sooner than FCEUX) and it desyncs because you get different grenade guy door in level 1.
There is an extra lag frame after pressing start to go to the password screen, so you even though you can delete a frame at the start, you have to add one in again around frame 55. This should get you through level 1. I was able to get up to level 4 just by comparing lag frames, but then it desyncs with the door to get the bomb being empty. I don't know how that RNG works so couldn't fix it, but resyncing seems doable.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
This game seems to keep popping up. Yeah coincidentally ThunderAxe31 sent me a short (completing 2 levels) test run of this game made in NESHawk about a week ago to try on console, and indeed it did work.
I tried this current run just now, and after 12 trials, the furthest it got was the end of level 4. So by that measure ThunderAxe31's test run wasn't long enough to say anything definitive.
One of the things to keep in mind with NES games, and especially MMC3 games like this one, is that console verification is probabilistic. The NES has several uncontrollable start up variables, not even including uninitialized RAM, so you might have to try numerous times to get a success. For timing sensitive games the probability can be around 1/8.
So far uninitialized RAM doesn't seem to be an issue here, ThunderAxe31's run behaves the same with initial RAM state all 0's or all F's.
I would need a complete run made in NESHawk (Bizhawk version 2.8) to say anything more definitive.
I think Bigbass' data suffers from using a lot of old runs. I only do console testing with BizHawk version 2.8, which contains numerous accuracy updates to NESHawk. My success rate for console verification of TASes in this setting is 58/63. Maybe I miscounted one or two but it's around 90%. Also worth noting that the failures aren't due to any known cause, or in other words not due to any known emulation deficiency (NESHawk does get some edge cases wrong, but I'm pretty sure these aren't the cause of the failures.) On the other hand there are numerous deficiencies in FCEUX that may result in failure on console. Obviously most games are still playable, so they aren't relying on careful timing to begin with, so many TASes made in FCEUX will still work on console.
Anyway, if you want your run to have the best chance of working on console, you should use NESHawk (on BizHawk version 2.8 or later.)
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
I decided to implement cycle accurate sprite emulation since I had some free time. As expected it wasn't that difficult or impactful on performance since it was only a small modification from what I already had. Now all the TTE demo profiles are timed correctly:
Additionally all the NBA RAM access tests work correctly. I fixed numerous other bugs along the way as well, so now some of the more complicated homebrews like Vroom3D work correctly.
Unfortunately, all of this still isn't enough to make Metroid Fusion sync on console. There are still a few more things I need to work on, so hopefully one of those fixes it.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
micro500 wrote:
The repo is a bit out of date and I definitely need to clean it up. I also have a decent amount of local work that isn't on github yet. I'm planning to tackle that cleanup in the next few weeks and make some more progress on this project,
Excited to hear you are still working on this. Good luck!
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
With some basic optimization I was able to recover about half of the performance I lost with the rendering rewrite, Still needs a lot of improvement needed, but its getting there. For now I'm taking a bit of a break from rendering stuff and working on some other features.
I added basic Flash RAM support. There is a lot of variability in Flash RAM, and what I have now is only rudimentary but it's a good start.
I also added in tilt control support, so now Yoshi Topsy Turvy and Koro Koro Puzzle work.
I'll probably add in solar sensor etc. for the Boktai games next.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
You can still find the code and some analysis archived here:
https://code.google.com/archive/p/treasure-master-hack/wikis
Certainly a good place to start would be figuring out a proper ending for the second prize world (if one exists.) A feasible path forward would be to map out the ROM and see if there are any subroutines that aren't called anywhere else in the game that might be used to create an ending for the second prize world.
Reverse engineering the algorithm has been done, but there are a lot of steps and a lot of one-way-ish functions so going backwards is also hard.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
After my previous post about VRAM access timings, it occurred to me that there were a lot of edge cases that my current code would not be able to handle. It seemed I would have to do a cycle accurate background pipeline sooner rather than later. So I went ahead and implemented it. Now every VRAM and palette fetch for backgrounds occurs exactly when it should. This allows me to be able to correctly display the 'mode 7 demo' ( https://github.com/ladystarbreeze/mode7demo ), which I think is pretty cool:
The resulting rendering code is considerably longer and more complicated than it was before though, and it comes at a pretty steep performance cost of 10% (or more for complicated scenes.) Some games now run under 60 fps at times. I still have to rewrite the sprite pipeline too, but I don't think that will have as big an impact, so things shouldn't get much slower. but I'll definitely need to focus a lot on optimization now to claw back some of that lost fps.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
GJTASer2018 wrote:
How would that even work in theory? Is it something that would happen online, or between two instances of BizHawk running on the same machine?
It would be the same as for Gameboy, just two copies of the system running at the same time, on the same instance, with some linking logic between them. Currently for very complicated games, such as the quake demo, my core only runs at 65 fps, so for two systems with linking logic it would probably be around 30 fps. Compare to mGBA which can run around 400 fps for the same game, it has plenty of room to run 2 systems.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
Making good progress towards accurate VRAM access timing. Backgrounds and palettes are pretty much done, so all of those access timing tests work now. This allows stuff like the infamous coin flip in the Madden NFL games to work. It also makes the Tonc tte demo profiling work correctly (after I fixed a bug in cpu timing that had me stuck for quite a while.) The only one not correct is the obj one, which I haven't gotten to yet:
With numerous other small bug fixes happening along the way, things are shaping up pretty well. I expect sprite access timing to be a fair bit more difficult than the others, so it will probably take a while. I also have a fair bit of refactoring to do before the ppu can be really considered cycle accurate, but I plan to that incrementally rather than with a sweeping re-write, it's been working out well so far.
PPU emulation aside, I still need to do things like Flash save ram, gyro controls, RTC, and possibly solar sensor, I might do some of these before tackling sprite timing just to change things up a bit.
Link cable emulation would also be cool, but I don't think I can make it run full speed.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
TheAmazingYucemu wrote:
Even if my nintendo switch syncs the arduino, no matter if it's sync controllers menu on Horizon OS, the emulated pro controller never appeared on this menu after bunch of button presses tries and even tried to run a dummy TAS file to the arduino and nothing worked so i'm pernamently ceased my open-source nintendo switch tasbot development forever.
Aww, that's too bad, I was really hoping to see some cool stuff come out of this. Better luck with your next project.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
One of the things that keeps me interested in emulation is just how close you can get to being right while still ultimately being wrong. I was recently working through the newer NBA ppu tests and somehow things just were not working out. It seemed like there was some off by one issue with DMA even though many other stringent DMA timing tests worked. Several hours of tinkering got me nowhere, then by happenstance I was scrolling through my code and noticed my execution loop, where it turns out I had a made a very basic error. Here is what things looked like before:
dma_Tick(); // DMA
pre_Tick(); // prefetcher
ser_Tick(); // serial port
tim_Tick(); // timers
cpu_Tick(); // CPU
The problem here is that both the CPU and the DMA channels can read and write to the timer registers and access ROM, but the DMA tick is happening before the timers and prefetcher while the CPU is happening after. Somehow I had gone through all the grueling timer and prefetcher tests (some of which do use DMA) without realizing this. It really is amazing that it all worked. Once I put things in the correct order, a cascade of other small errors showed up. Fixing everything up took a while, but now I can get through the NBA tests that were giving me trouble while everything else still works.
I still have a small list of things to do before tackling VRAM access timing, with the top of the list being video capture DMA, but I'm pretty sure most everything else is solid. There are also some untested cases that could effect TAS console sync, like exact timing of audio FIFO DMA and DMA IRQs, multiplication timing, and probably various horrible prefetcher edge cases, but those are things I'd worry about after VRAM. Still a lot to do.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
I did some tests with Metroid Fusion and Fire Emblem and they both desynced, apparently due to loading times, I believe because of VRAM access delays not implemented yet. The GBA has a feature called forced blank that allows you to access VRAM without any conflicts. A side effect of this though is that the screen fades to white. So, if you want to change scenes with a fade to black instead, which both of those game do, you have to deal with VRAM access delays as the PPU is still rendering.
Fusion made it the furthest so here is a video:
--superseded by more recent progfress--
So it looks like I'll have to implement this before further testing.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
Starting to test out GBA verifications. Decided to do a test run of a short homebrew game (Feline) to test out my workflow. This test run was made entirely in my new core to test out save states and TASing tools. Seems to be working well so far. Feline is also interesting from an emulation point of view by itself because if you get seen by the lab rats, the game attempts to change ARM mode in an architecturally unpredictable way (attempting to set the thumb bit by MSR instruction.) Turns out nothing bad happens on hardware, but it took me a while to figure out why it was crashing on me in emulator. EDIT: Old video replaced with published version
Link to video
The run plays out the same in mGBA, except you can start a frame earlier in my core (presumably due to pre-fetch but I haven't checked the details), so this was a relatively easy test. Once I fix a few more emulation bugs I'll try something more difficult.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
Made some good progress over the past few weeks. Most of the rendering effects are done now mainly thanks to the Tonc demos (although I probably missed some alpha blending cases somewhere.) EEPROM saving is done, and various bugs are fixed. Now I can pass the entire mGBA test suite (with the caveat that Misc edge tests -> H-Blank bit -> Flip #1 matches hardware but not what the test says) and all of the NBA tests.
The only major element left to work on is sound. In the past this has always given me the most trouble, but I already have the FIFO channels working and the others are just slightly modified GB audio channels, so I'm hopeful I can make relatively quick work of it. I also haven't done VRAM access timing stuff yet, I'll hold off on that until I see how many games really need it. Anyway, I know of a few bugs I need to sort out besides audio, after that console testing can start, hopefully in a month or two.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
Game: NES Batman
Emulator: BizHawk: 2.8 (NESHawk)
Console Verification Device: TAStm32
Movie:https://tasvideos.org/UserFiles/Info/638097680760634038Description of Desync: Desyncs shortly into stage 2, Batman jumps into the acid.
Research: This game uses an MMC3A and DMC channel. There are no other verifications of MMC3A games that require careful timing such as this, (there are others using MMC3A, but don't require careful timing, and there are verificaitons of games ex. SMB3 that require careful timing but use a later MMC variant) so may be related to that revision, but it's not clear why this would be.
Possible Next Steps: ???
Status: Open
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
feos wrote:
Levels 5 and 15 look slower than the human record. Is that correct, and can it be helped without breaking everything else?
Oh I missed checking a couple levels somehow. Unfortunate it's at the beginning. There is no RNG in this game, so it's definitely fixable, but then resyncing everything is tedious.
I'm not putting more hours into this game right now, so I'll just cancel.
If anyone wants to fix it I'll be happy to add a co-author, maybe another set of eyes can even find some more improvements.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
Wow new (and open source) switch verification work, cool stuff.
What does 'NO CFW required' mean?
Good luck in development, I'll be really interested in your progress.
Additionally all the NBA RAM access tests work correctly. I fixed numerous other bugs along the way as well, so now some of the more complicated homebrews like Vroom3D work correctly.
Unfortunately, all of this still isn't enough to make Metroid Fusion sync on console. There are still a few more things I need to work on, so hopefully one of those fixes it. 
I'll probably add in solar sensor etc. for the Boktai games next.
The resulting rendering code is considerably longer and more complicated than it was before though, and it comes at a pretty steep performance cost of 10% (or more for complicated scenes.) Some games now run under 60 fps at times. I still have to rewrite the sprite pipeline too, but I don't think that will have as big an impact, so things shouldn't get much slower. but I'll definitely need to focus a lot on optimization now to claw back some of that lost fps.
With numerous other small bug fixes happening along the way, things are shaping up pretty well. I expect sprite access timing to be a fair bit more difficult than the others, so it will probably take a while. I also have a fair bit of refactoring to do before the ppu can be really considered cycle accurate, but I plan to that incrementally rather than with a sweeping re-write, it's been working out well so far.
PPU emulation aside, I still need to do things like Flash save ram, gyro controls, RTC, and possibly solar sensor, I might do some of these before tackling sprite timing just to change things up a bit.
Link cable emulation would also be cool, but I don't think I can make it run full speed.
The only major element left to work on is sound. In the past this has always given me the most trouble, but I already have the FIFO channels working and the others are just slightly modified GB audio channels, so I'm hopeful I can make relatively quick work of it. I also haven't done VRAM access timing stuff yet, I'll hold off on that until I see how many games really need it. Anyway, I know of a few bugs I need to sort out besides audio, after that console testing can start, hopefully in a month or two.