Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2752
Location: US
Gamer Maiden Sonia wrote:
Alyosha wrote:
Down to just 2 tests, scnaline and TVpassfail. I'm not really interested in the TV one since it's just a filter, maybe I'll do it eventually.
So it's not important then? I'd like if you could tell me more about TVpassfail (what we'd be missing out if it's not accurate). I like learning about those little things whenever possible.
Well it's important for proper appearence, but it's basically a post processing feature so it doesn't effect core emulation.
The test is looking for proper emulation of cross talk between components of the output color singal. I'm no expert so here is a picture of the comparison between 2 filters in punes, the left emulates NTSC video signal output to a real life TV, the right is just raw video output.
Bizhawk looks like the right, so at some point a filter needs to be put in to make it look like the left.
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2752
Location: US
RaijinXBlade wrote:
Alyosha, I know of an NSF player by the name of NSFPlay which was reworked by someone called Rainwarrior. It is considered to be highly accurate, including for the mappers. Do you think that can be of some use to you?
Thanks for posting that, I looked over the code and even there they count up. Every over place I check also counts up, so I'm not really sure why the wiki says it counts down, counting up also gives correct test results, which is good enough for me!
Down to just 2 tests, scnaline and TVpassfail. I'm not really interested in the TV one since it's just a filter, maybe I'll do it eventually. Scanline is puzzling. Currently I get results similar to punes, and the test seems to rely a bit on start up state. I also don't see how FCEUX is able to pass it, I'm guessing a false positive.
EDIT: I finally fixed the Saint Seiya - Ougon Densetsu Kanketsu Hen glitch. This turned out to be a ppu $2007 write during rendering, for which you simply need to read from address the background renderer is using instead of the normal vram one. A simpler fix then I realized. I'm happy that this one is done, as it was one of the first things I tried fixing.
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2752
Location: US
hegyak wrote:
What is the white square doing? Is that normal?
Yup it's normal. That square is sprite zero, so presumably it's used for timing, although I don't know how exactly.
I got square channel mostly worked out, although something seems off:
NesDev Wiki wrote:
The reason for these odd sequences is that the sequence counter is initialized to zero but counts downward rather than upward
Right now it only works by counting up, not down as is clearly intended, so not sure what all that is about, but the test is much much closer to correct now.
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2752
Location: US
Thanks a lot! I really like accuracy too, and I was always curious why NES emulation never really made into the realm of bsnes in terms of accuracy, despite the wealth of research done by NesDEv folks over the past few years. I think we're getting closer now though (at least with NTSC variants.) BizHawk was already really powerful, and my hope is that the accuracy improvements , combined with feos' numerous improvements to TASStudio, might bring in some runs that can be console verified. My personal goal is to have verifications of MegaMan, Punch Out, and Battletoads.
I'm making progress on APU Mixer tests. I'm using the full nonlinear mixer now, and DMC and Noise sound perfect. Triangle sounds strange but roughly matches punes, so not sure what to make of that one. Square is a mess. Audio is not my strong suite so it might take a while to track down what's going on, but either way 4 tests to go!
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2752
Location: US
That's pretty cool. I'm always surprised how such tricks can go undiscovered for so long.
I think my name can probably be removed from the author list now. This current work certainly supercedes anything I had to add in the previous run.
Nice Work!
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2752
Location: US
Gamer Maiden Sonia wrote:
Just posting here to say I'm impressed with your work, Alyosha. A 100% NEShawk would be really, really awesome. Good luck with the remaining tests!
Thanks!
As I expected the remaining tests are going pretty smoothly, only 7 more to go!
@feos: oh wow, thanks for pointing that out, I guess the hard part is already done then!
EDIT: We're #1! Only 5 tests to go!
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2752
Location: US
Once I got DMC figured out it was relatively easy to sort out the frmae timer. I got it working correctly and cleaned up the regression is DMA_and_IRQ. In the process I also got several other tests passing that were part of the same test suite.
So with that, there are only 10 more tests to go! Some of them seem relatively easy, so we should be tied with punes in no time!
The only challenging ones here that i am worried about are scanline (which I have no idea how it's supposed to pass and punes doesn't pass either) and tvpassfail which seems like a great deal of work.
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2752
Location: US
With a little added logic the other DMA test variant, sprdma_and_dmc_dma_512, now passes as well. That's 2 of the most grueling tests passing, so if I can just work out frmae IRQ stuff there will be no more hurdles and only tuning the APU left to worry about.
I'm a little worried about the accruing performance hits though. I'm currently at about a 20% penalty over 1.11.6. On my laptop that equates to being able to run at about 140 fps. Well hopefully it will still be at over 2x speed once I'm done, I think it will need some serious optimization work though.
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2752
Location: US
After days and days of repeated failure, I finally found the last piece of the puzzle needed get sprdma_and_dmc_dma to pass in a cycle accurate way, with full RDY implementation and no hardcoded hacks.
Visual 2A03 saved the day here, and without it there is absolutely no way I could have figured this out. The key seems to be an overlooked feature of DMC DMA where if it is called from a write to $4015 and the buffer is empty, it will take one less cycle to complete. This 1 cycle is what had me stumped and there is no other conceivable way it could have worked out (I tried every imaginable alternative.)
There is still some work to be done to clean things up, but I doubt there will be any remaining tests as difficult as this one, so onward to 100%!
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2752
Location: US
^ glad to see a positive response to what must have been a lot of negative feedback after seeing that pricetag. I was definitely excited to see this done...but not $30 worth of excited, so glad it's free now!
(It's still an awesome project, nice work)
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2752
Location: US
cpu_exec_space/test_cpu_exec_space_apu now passes with a few simple fixes. That makes 10 new tests passed, up to 90%!
I did not include DMA timing fixes, as I still don't know the problem is, so that 90% can be taken with a grain of salt, but still this is progress.
Pretty much every remaining test involves the APU. This is good, since it means the rest of the core is really solid, but that is also the least developed part, so will take some effort to get in order.
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2752
Location: US
Pokota wrote:
Which tests are now failed?
cpu_interrupts_v2/rom_singles/4-irq_and_dma
cpu_interrupts_v2/rom_singles/5-branch_delays_irq
^ those tests fail directly as a result of fixing the timing bug, (which is running the CPU one extra cycle before starting OAM DMA.)
They are both related to IRQs caused by the frame timer at $4017. I tried repeatedly to fix it, but when I do then other tests fail (jitter, NMI and IRQ, even IRQ timing.) I can't get them all to pass at once, which makes me think there is something else going wrong.
Also, srp_and_dmc_dma is consistently off by 1 cycle too many for all 16 tests. Nothing I do makes that 1 cycle go away. I think all these issues are somehow related, but have no ideas.
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2752
Location: US
I finally got Time Lord working, Hurray! This is the only total incompatibility that I know of that remained so I'm quite happy to get it working. It was actually a simple fix, lucky the NesDev wiki already said what needed to be done, it was just a matter of implementing it.
Unfortunately things have gone awry elsewhere. NesHawk had a timing bug and apparently passed some rather important tests through shear luck and cancelation of errors. Removing this bug has led these tests to no longer pass , despite my repeated efforts to resolve them. So looks like a long road ahead.
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2752
Location: US
scrimpeh wrote:
Yo, I just wanted to drop in and say I'm amazed by all the work you've been doing for NEShawk, Alyosha.
Keep it up!
Thanks!
SaxxonPike wrote:
I noticed that the DMA work you've been doing relies on the RDY pin. I've put off working much on the 6502 CPU core until after I could get some automated testing going. However, the IncPc micro-op in the MOS6502X core was missing the dummy read (which would be stalled by RDY.) I've committed that fix today to master.
AHX/SHX/SHY illegal opcodes affected by the high byte of the effective address also behave differently based on the RDY pin. More info here: http://csdb.dk/forums/index.php?roomid=11&topicid=94460
If you read that, keep in mind that in the C64 context, RDY is controlled by the video chip and is made inactive when it needs the bus fully.
Oh good catch! After looking at that I noticed that the dummy read for various instructions also is not affected by the RDY pin as it should be, and I think that is part of what is keeping from passing some more of these test, so fortuitous timing.
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2752
Location: US
Aww, too bad, I hope this gets turned into a neat play around so it can get published.
I also hope the actual fastest run gets submitted. Anyway to ask the authors about it?
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2752
Location: US
thommy3 wrote:
Wow, it's crazy how fast you get these improvements out. Great work.
I've a question though, is an emulator passing all those above tests perfect in the sense that it behaves like real hardware?
Or is it just that he passes all pubished test roms but there might still be differences to real hardware because maybe those roms didn't cover every hardware aspect?
Thanks!
In the case of NesHawk specifically, yes there is a very real difference between the emulation architecture and the real hardware.
NesHawk runs at the the timing level of a PPU tick, when 3 of those ticks go by we run one CPU instruction. A real NES though runs at a timing level one step above this, where a master clock ticks off pulses and the PPU and CPU use their own divders to time execution. In particular the 6502 (or the NES variant in this case) actually runs 2 phases which make up one instruction clock, and this level of hardware detail is not emulated at all by NesHawk. The fact it can still pass all these edge case tests though is simply proof that this level of detail is not really needed almost all of the time.
If you are interested in how things really work, you can check out Visual 2A03 (the NES cpu) or Visual 2C02 (the ppu), pretty neat stuff!
____________________________
I finally got around to working on ppu open bus behaviour. It was pretty simple, now cpu_dummy_writes/cpu_dummy_writes_ppumem and ppu_open_bus/ppu_open_bus both pass. Scratch 2 more off the list.
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2752
Location: US
jlun2 wrote:
I got a suggestion: Try running the console verified runs on BizHawk and see if they sync. Most of them were done on FCEUX, but if for whatever reason it does not sync maybe something is wrong?
Good idea! I think I'll give it a try once I get DMA timings worked out. Only games that are very geenrous with timing alignments will sync without that.
Speaking of which, after several failed attempts and hours of debugging, dpcmletterbox/dpcmletterbox now works properly. This is a signifcant improvement to DMC DMA timing so I hope i can make more progress on OAM DMA now.
No where near that bad. The performance hit so far came from code that runs every pixel (3 times per cpu tick) but that is done now. I think only accurate APU emulation remains that will a signifcant negative impact on speed (my guess is another 2-3%.) After that I do plan to go over everything and make obvious optimizations to gain back some of the performance losses, but I doubt I can reach parity with 1.11.6 again in terms of speed.
I don't really understand what feos is suggesting with the cpu refactor, so I'll leave that up to him, seems like a huge undertaking.
zeromus wrote:
Sure thing I installed it, but I can't tell what, if anything, it is doing, how do I use it?
_____________________
I've been testing various games known to be tricky to emulate. The first game that was previously incompatible with BizHawk now works, Fire Hawk!
Curiously this game runs on FCEUX but went to black screen on previous versions of BizHawk.
If anyone knows other games which are just plain incompatible please let me know (I am aware of Time Lord currently) having games to test on really helps.
EDIT: oh, the recent fixes to OAM reads also fixes cpu_dummy_writes/cpu_dummy_writes_oam, so, scratch another one off the list!
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2752
Location: US
The version of AtariHawk in BizHawk 1.11.6 contains numerous, and some rather serious, bugs that have since been fixed. In partiuclar a bug in the 6532 causes measurably innacurate timing, which this game uses. This run does not sync in the current dev build (fails at level 1.)
I'm not sure what the rules are regarding this, are runs from dev builds submittable?
Still entertainging game though. Having to play with both joysticks must make it challenging.
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2752
Location: US
I really like these technical runs. Impressive result.
Pretty entertaining and good commentary. I'm also curious about those shaky portraits in the credits though, how strange.
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2752
Location: US
feos wrote:
What about attempting to speed up the core too?
Well, my current improvements to accuracy slow down the core by a noticable amount. At 400% speed I can run SMB3 on my laptop at about 180 fps on 1.11.6. The current Bizhawk master build can do about 160. My current test build can do about 152.
So... we're kind of going in the wrong direction there 8D
These represent almost entirely ppu changes. So that might also be a good place to look for performance improvements.
EDIT: after spending a long time to find a very small bug, oam_stress/oam_stress now passes.
I'm also making progress on sprdma_and_dmc_dma, I'm hoping maybe by the end of the month those tests will pass.
Bizhawk looks like the right, so at some point a filter needs to be put in to make it look like the left.
With a little added logic the other DMA test variant, sprdma_and_dmc_dma_512, now passes as well. That's 2 of the most grueling tests passing, so if I can just work out frmae IRQ stuff there will be no more hurdles and only tuning the APU left to worry about.
I'm a little worried about the accruing performance hits though. I'm currently at about a 20% penalty over 1.11.6. On my laptop that equates to being able to run at about 140 fps. Well hopefully it will still be at over 2x speed once I'm done, I think it will need some serious optimization work though.
I finally got Time Lord working, Hurray! This is the only total incompatibility that I know of that remained so I'm quite happy to get it working. It was actually a simple fix, lucky the NesDev wiki already said what needed to be done, it was just a matter of implementing it.
Unfortunately things have gone awry elsewhere. NesHawk had a timing bug and apparently passed some rather important tests through shear luck and cancelation of errors. Removing this bug has led these tests to no longer pass , despite my repeated efforts to resolve them. So looks like a long road ahead.
Curiously this game runs on FCEUX but went to black screen on previous versions of BizHawk.
If anyone knows other games which are just plain incompatible please let me know (I am aware of Time Lord currently) having games to test on really helps.
EDIT: oh, the recent fixes to OAM reads also fixes cpu_dummy_writes/cpu_dummy_writes_oam, so, scratch another one off the list!
I'm also making progress on sprdma_and_dmc_dma, I'm hoping maybe by the end of the month those tests will pass.