I can confirm it runs just fine after resyncing to the Mario + Duck Hunt cart. It would take a while for me to actually be able to upload any sort of video though.
SMS UPDATE
SMS now syncs on my Model 1, BIOS 1.3 console :)
HOWEVER, the game I am testing, Alex Kidd in Miracle World, has some sort of nondeterministic RNG where the moneybag drops are not always big bags, but sometimes small bags. So the midget doesn't have the money to buy the bike in the second stage. :( But Stage 1 syncs nearly every time (sometimes not because of vsync offset, but >80%) so progress is being made. Just need SMS games, and maybe someone to look into Miracle World...
EDIT:
Ordered 19 SMS games and I lucked out; now I have all 4 common external mapper types (5208/5235/5235+BBRAM/5365) so I should be able to attempt to verify every SMS game. None of the games I ordered have runs on this site yet so they will all have to be donors :( Technically only the 5235 type should be needed but I want to be as similar to original hardware as possible when original hardware isn't made available.
I tried Wonder Boy III (thank you Walter Payton) and it does not sync; both emulator and console act similarly (but not the same) in that they don't start a new game and hit Continue instead. I made a demo movie and it was wildly different with sync and start time. Site run probably needs old emulator? Other emu problems? From my experiences so far I don't think SMS emulator accuracy is even close to other consoles.
I will also probably need to either modify the console or build an actuator of some type to hit the Pause button for games that use it. I am pretty sure WB3 is one of these games.
So progress is being made for the first SMS verification, but nothing promising yet.
Been very busy with non-TAS projects. Here is the status of TAS projects.
MultiReplay: The first three recipients got theirs, and two have used it to verify games. The last finally got back to me and should be doing it soon. Will be making more when I get my oven controller board in and modify my oven. I also need to order more parts.
Verification videos: I have a large backlog. I will get to them when my visualization boards come in. These are boards that look like controllers and have LEDs to show input being pressed. They connect to my MultiReplay device, but the output pinout has been kept the same as console for NES and SNES so could technically be wired up to the controller output directly in parallel. Maybe speedrunners who run live on console would like this to visualize the buttons they are pushing on camera?
SMS: not much progress; this is the next thing to be completed before other major progress. Lots of stuff to do here. I will have time for this soon.
NES: Thanks for scrimpy's help, he wrote a PRG RAM eraser that can be booted from a cart, removed, the target game hot-inserted and a button press (or a few) later, erased. Has options for erasing MMC1 and MMC3-enabled carts and has been tested so far with MMC1.
My bot now has an option to slow down clock detection to work around emulator / console inconsistencies (generally fixes A Boy and His Blob playback).
SNES: Nothing interesting
Lynx: I have one, it doesn't boot
Game Gear: I have one, it boots but has a fucked up screen. I wish I had a dev board for this or more games.
Worked on SMS tonight. Wrote a dumper for BH. The way sync works is with vsync detection as I don't have any other kind of polling to work with. Unlike dega, I need to give the run ~627 frames lead time (dega with Alex Kidd in Miracle World was ~489 frames lead time) for the bios to complete.
Aladdin did not sync - it always trips on the barrel after the rock/thing on the floor and gets caught.
Wonder Boy ML did not sync - never goes through the second door. But like AKMW, the RNG for drops is different every time the game is run.
Can someone look at how SMS BIOS (specifically v1.3, I have 3 of them) works to see what it does to RAM and if it is responsible for weird RNG? Then take a look at the game to see what it is responsible for? Is RAM not initialized? Or is it and some hardware facility is being used to seed RNG? Is some hardware facility being used directly?
Hi, I was summoned here from SMS Power. Post there to get the world's greatest SMS minds to see...
I may not be the greatest but I can offer some advice and/or problems...
1. Dega is woefully inaccurate, I'd not bother with it.
2. SMS games generally read the inputs during the VBlank. That might be before or after the actual blanking signal, so ideally you'd need to try to sync to somewhere around the last line of the active display.
3. Most RNGs mix in entropy from user inputs and the r register. The latter is where you find trouble, what you get in there is going to be a bit dependent on some analogue characteristics of the initial console power on.
4. You could detect controller polling with a little hardware effort to monitor the CPU bus. This could be done over the expansion slot on an original model system. You might even attempt to control the r register that way, but that's getting very tricky.
1. Yes I understand Dega is inaccurate. Most runs here are using Dega so for the only game I had for a while (Alex Kidd MW) I used Dega input. And it works, up until the second level, because of the RNG... The other games I tried were with the other emulator, with similar failures.
2. I think I am updating immediately on vsync, so I need to add a changable delay so that I don't update during a read (probably not happening, but...). My point was that I need to put in a wide variation of blank frames during the BIOS routine before the game runs to get any kind of sync.
3. From register, ok. Nothing in the BIOS, this will be game dependent?
4. Vsync detect is probably OK with SMS, unless I can't really get input to sync up and input lag is involved and not emulated properly.
But you mention controlling registers this way...are you thinking an expansion ROM that sets the register to what the emulator expects, then sets the cart slot to active? (But this would require a delay to insert the cart since cart slot is checked before expansion?) This cart could even be used to signal to the replay device when it is OK to start sending inputs - when the !OE goes high after going low then the cart is selected and running...
Or maybe even a replacement BIOS although this is getting into HW mod territory? I have three original SMS and could always hack one up but an external removable mod used for determinism / verification is preferred over modding the console. Is the connector pitch 2.5mm or 2.54mm?
The BIOS really doesn't do much, it's about detecting a cartridge and enforcing region detection. Some emulators bypass the checks in order to avoid having to emulate the multi-slot nature of the system, and that would account for a lot of the time difference you see.
The r register is a bit odd, you can't set it in code. It's a counter that increments on every cycle, which provides a useful pattern for certain old memory chips, to help refresh their values and avoid data loss. The Z80 chip did this because it was useful at the time. Games use it as a random seed because its value is not deterministic, as the CPU clock runs for a while before the power stabilisation components allow it to start executing code. You could make a replacement BIOS, or a watchdog circuit on the expansion port using a CPLD or something, to watch the value and try to get the game start code to have a consistent value by halting the CPU or, in the BIOS case, executing code chunks that waste the right amount of time.
Some games may not use the r register for their RNG. There's no way way to be sure, except for reading the code.
Games use it as a random seed because its value is not deterministic, as the CPU clock runs for a while before the power stabilisation components allow it to start executing code.
Then would this be as simple as an RC circuit on the reset line to delay Z80 startup? The docs I see say when Z80 is reset, R is set to 0 - but I can't see if it is level or edge triggered? (does it increment while in reset?) Some docs say you can load R, some don't, so I'll assume not. Obviously I am not too familiar programming the Z80 =)
I have received two GBAs and some Spongebob games from someone who wants to remain anonymous; thanks! I can try verifying games as long as there is some way to detect input poll or screen update on the hardware, will need to look for that. Hopefully someone has already done the work.
---
Been discussing SMS stuff here, and it looks like SMS verification apart from emulator accuracy will take a while yet. And as far as emulator accuracy is concerned, I don't think we're there yet either, at least with Wonder Boy. Still need to work at this but it isn't hopeful.
Maxim wrote:
The BIOS really doesn't do much, it's about detecting a cartridge and enforcing region detection. Some emulators bypass the checks in order to avoid having to emulate the multi-slot nature of the system, and that would account for a lot of the time difference you see.
The time difference I was talking about was a variable difference depending on the game. I would imagine the BIOS execution until game instruction start would be the same? But it doesn't seem to be.
---
NES and SNES controller visualization boards should be here in a week or two. I will then resume recording verification videos for the backlog of NES games.
EDIT Shipping is slow as shit from China right now, from the latest holiday and maybe the protests in Hong Kong. My stuff is waiting to get on a plane. No idea how long this will take... I'm just starting to get other unrelated stuff shipped from Shenzhen and HK that was ordered a week prior, so maybe this is still on target for next week.
EDIT 2 Boards are in. Verification videos are being made.
I've built a custom bot for Genesis/MegaDrive from Teensy 3.1 and posted my first verification using it:
http://tasvideos.org/forum/viewtopic.php?p=391831#391831
It's using custom code and a loader program that runs on MD itself to try to deal with nondeterministic state that the hardware boots in. Unfortunately it's unable to perfectly sync with the VDP, it's only approximate, but it's enough to get sonic to sync reliably. There are multiple issues that I could find that prevent perfect sync even from a loader program:
- the horizontal pixel counter only exposes 8bits out of 9-bit internal counter, least significant bit is lost
- VDP runs from a different divider, so even if you manage to read a desired value in hvcounter register (which is difficult as between 2 fastest reads the 68k can do, the counter value will already be increased by 2 or 3), you can't know if it was "start" or "end" of that pair of pixels in VDP
- there are asynchronous refresh cycles generated by the memory controller for both RAM and cart slot. If the 68k memory accesses hits the refresh cycle, it will get a wait state and lose sync with the VDP.
That probably only scratches the surface as there is also Z80 with R register and ability to access the main bus (and thus halt the 68k and affect timing) and YM2612 with it's timers and stuff. I've only been able to get semi-stable VDP sync from my MD program as long as it doesn't access main RAM, Z80 and sound chips, as soon as RAM is accessed VDP sync is lost because of mentioned async refresh and I haven't even bothered trying to do this VDP sync with Z80 and friends active.
Interesting. At least it is progress.
I would like to avoid a loader, but maybe it won't be possible. I was going to work on a reset circuit which would reset everything when the clock dividers were in phase to fix this issue. From minor details in your post (either uninitialized or free-running even in reset register values?) it looks like this might not be enough, though it might be a help. Wish you were on IRC, we've been discussing this a bit.
As far as changed frames go, no in-level changes needed is a good thing. Glad to see that.
Most of the issues re: timing should be fixable in emulation. If it isn't emulated well enough, that is an inaccuracy of the emulator.
Re: R register on Z80, not sure how many Genesis games this might affect, but it is certainly an issue with SMS verification. There have been ideas posted about that earlier in this thread and at smspower. At least with a loader on SMS, we can still run the original software from the original cart without much hassle. Ideally though future TAS would support booting with a BIOS so the proper amount of frames pass and the register is set to a proper value, but right now rerecording emulators do not support BIOS for SMS, and I'm not sure when/if they will. Also need to decide which is the best BIOS to use - do we prefer speed at any cost, or prefer the more common ones (like v1.3)? Right now every SMS TAS on the site with games that use the R register is effectively unverifiable.
I would like to avoid a loader, but maybe it won't be possible. I was going to work on a reset circuit which would reset everything when the clock dividers were in phase to fix this issue. From minor details in your post (either uninitialized or free-running even in reset register values?) it looks like this might not be enough, though it might be a help. Wish you were on IRC, we've been discussing this a bit.
I really doubt you can reset everything, I'm not sure the VDP even cares about the reset signals. For example, it is well known that VDP keeps doing DMA after the user presses reset, so games need to be careful to check if VDP is not busy while they boot, as accessing busy VDP often results in a lockup. That is a common problem homebrew has (pressing reset locks up the machine randomly). It is not clear if memory controller (aka bus arbiter) can be reset, from my indirect tests it seems it doesn't care about reset button at least.
True wrote:
Most of the issues re: timing should be fixable in emulation. If it isn't emulated well enough, that is an inaccuracy of the emulator.
Yeah but currently there is no emulator that would do complete bus cycle emulation. This is quite hard to do actually and is pretty unrewarding to emulator authors as no game requires that level of accuracy to work properly. Currently Exodus is the closest one to achieving that, but even it is not quite there yet and there is not enough information available about some things like:
- How/when exactly 68k does it's bus accesses. There are ongoing attempts to read the microcode from die shots and some detailed patents have been found of pre-production 68k, but those efforts are still ongoing AFAIK.
- The mentioned memory refresh cycles are a mystery too, people are seeing them with logic analyzers but there is no understanding how exactly they work:
http://gendev.spritesmind.net/forum/viewtopic.php?t=1411
- bus sharing with Z80 needs to be worked out, I know Exodus attempts it at least, but I don't know how well it gets it.
Exodus also hasn't opened it's sources yet and can't be used for TAS related work, but that is supposed to change soon.
Gens is especially bad at timing as it doesn't even bother emulating VDP access related stalls except DMA, and those are currently quite well understood. It's quite unfortunate most runs are made with Gens, it means most of runs will likely never be hardware verified (not that there was much choice when people started doing Genesis TASes, if only at least Fusion was open source..). There are of course exceptions like Sonic, which is very forgiving to wrong timing, but any game that has a RNG that's seeded from VDP's hvcounter (or is just using it directly) will never sync. And unfortunately it was well known by game developers that hvcounter is good source of entropy, so it's widely used.
Joined: 12/8/2012
Posts: 706
Location: Missouri, USA
notaz wrote:
Gens is especially bad at timing as it doesn't even bother emulating VDP access related stalls except DMA, and those are currently quite well understood. It's quite unfortunate most runs are made with Gens, it means most of runs will likely never be hardware verified (not that there was much choice when people started doing Genesis TASes, if only at least Fusion was open source..). There are of course exceptions like Sonic, which is very forgiving to wrong timing, but any game that has a RNG that's seeded from VDP's hvcounter (or is just using it directly) will never sync. And unfortunately it was well known by game developers that hvcounter is good source of entropy, so it's widely used.
This is a good point. All the previous TASes I did with Gens11b desynced within 1 minute of converting to Bizhawk.
If you're a D&D fan, my Warriors of the Eternal Sun run was made with BizHawk :)
"But as it is written, Eye hath not seen, nor ear heard, neither have entered into the heart of man, the things which God hath prepared for them that love him." - 1 Corinthians 2:9
There are of course exceptions like Sonic, which is very forgiving to wrong timing,
Indeed, the Sonic games are forgiving enough that their runs can all probably be synched in hardware by adding or removing enough delays to compensate for lag differences, maybe adding occasional pauses. Even the animal manipulation (and other similar luck manipulation) can probably be made to work in this manner. It would be an ungrateful process, but it would be doable.
The synch process would go about much more smoothly, and would be much less painful, if there was a way to read some values from RAM near desynch points.
I would like to avoid a loader, but maybe it won't be possible. I was going to work on a reset circuit which would reset everything when the clock dividers were in phase to fix this issue. From minor details in your post (either uninitialized or free-running even in reset register values?) it looks like this might not be enough, though it might be a help. Wish you were on IRC, we've been discussing this a bit.
I really doubt you can reset everything, I'm not sure the VDP even cares about the reset signals.
My thought was to hold Z80 and 68K in reset at power on, then sync those to VDP clock phase. Hardware mod, yes, but only on the reset lines; we can probably look past this since it isn't on any bus and would only be at system start.
It's not as serious as what I need to do for SNES...
notaz wrote:
Yeah but currently there is no emulator that would do complete bus cycle emulation.
Yes, of this I am aware. TAS verification / legitimacy is a fairly new phenomenon, just add motivated archivists and we might get there eventually. Some quirks of these systems are known outside of this community but we're finding new ones still (like latest SMW not verifying - seems to be some unemulated register access locking up the system. Need a bigger LA to analyze that...)
notaz wrote:
Gens is especially bad at timing as it doesn't even bother emulating VDP access related stalls except DMA, and those are currently quite well understood. It's quite unfortunate most runs are made with Gens, it means most of runs will likely never be hardware verified (not that there was much choice when people started doing Genesis TASes, if only at least Fusion was open source..).
Was quite aware of the limitations of Gens. Not as much aware of limitations of gensgx core though it also isn't an accuracy emulator. Things will hopefully improve, eventually. They did for SNES after all. Hell, right now at this very moment there's heavy development on SGB integration and accuracy.
I would like to avoid a loader, but maybe it won't be possible. I was going to work on a reset circuit which would reset everything when the clock dividers were in phase to fix this issue.
Oh, I had forgotten to reply to this on my previous post: I had asked around about resets some time ago (for an issue in Gens), and what I got was this (reposting from the issue):
marzojr wrote:
Here is some information on soft resets I got from talking to Tiido/TmEE:
TmEE wrote:
The general process is that reset button goes to some ASIC where the signal is debounced and the ASIC will spit out the real reset signal (VRES).
There's also the Power On Reset (MRES, SRES) signal that does initial reset, that one also resets the VDP and IO chip, and is also fed to MegaCD in inverted form (FRES). MegaCD will not see further resets from MD side.
32X sees both SRES and VRES. The 32X generates its own power on reset and it overrides SRES on MD side, so 32X can extend power on reset until its own is ready, ensuring proper operation. 32X does react to VRES too, but what exctly happens I don't know.
I have not done a whole lot with 32X and done nothing with MCD as far as programming goes.
He gave me some links, one to a forum thread (http://gendev.spritesmind.net/forum/viewtopic.php?t=1262) and several for chip schematics.
The thread lists the time that VRES stays asserted and that there is an additional signal (ZRES) that is asserted at the same time as VRES and goes to the z80 and the ym2612. ZRES does not get deasserted on its own; the 68k needs to deassert it after the reset.
In a nutshell, for a soft reset:
* for plain Genesis: VRES is fed to 68000 for 128 VCLKs (16.7us); ZRES is fed to the z80 and ym2612, and remains asserted until the 68000 does something to deassert it; VDP and IO chip are unaffected.
* for Sega CD: soft resets reset only the Genesis side, as above, except for the initial power-on reset.
* for 32X, there is little information as far as soft resets go.
Researching based on this information, I came up with:
From the M68000UM.PDF document, the reset loads the longword at $000000 into the supervisor stack pointer, then reads the longword at $000004 into PC, then initializes the interrupt level in the status register to 7. No other registers are affected by a reset.
From the Z80UM.PDF document, a reset clears the interrupt enable, PC and registers I and R, then sets interrupt status to mode 0.
I couldn't find any solid information on the behavior of the ym2612 under reset, except for "System reset, initialize registers".
So no, neither the VDP nor the IO chip care about soft resets.
In fact, Sega's default initialization code even relies on this behavior from the IO chip: it uses the IO chip's state to determine if this is a soft or a hard reset, and sets the Genesis to a known state by clearing everything on the latter case.
Sega's default initialization code also accounts for the VDP's behavior by having an apparently useless read from the VDP control port: this is meant to clear the internal write-pending flag in the VDP. I would wager that it is this, not DMA, which causes issues in homebrew: the VDP will be expecting the second half of a command longword to trigger a DMA operation, and anything written to it will fit the bill. I would wager it is this because whenever the VDP is doing DMA from 68k to the VDP, it disconnects the 68k from the bus; any access by the 68k to its addressing space (say, fetching an instruction, or fetching the reset vector from ROM) causes it to lock up until the DMA is done. So a DMA that persists across reset must finish before the 68k proceeds (it is only on DMA copy and DMA fill that the 68k can go on).
OK, so something may be possible with Power On Reset signals, I just wonder if they also reset the phase of those refresh cycles that memory controller/bus arbiter generates.
True wrote:
It's not as serious as what I need to do for SNES...
Do you have that documented somewhere? Would be an interesting read.
True wrote:
notaz wrote:
Gens is especially bad at timing as it doesn't even bother emulating VDP access related stalls except DMA, and those are currently quite well understood. It's quite unfortunate most runs are made with Gens, it means most of runs will likely never be hardware verified (not that there was much choice when people started doing Genesis TASes, if only at least Fusion was open source..).
Was quite aware of the limitations of Gens. Not as much aware of limitations of gensgx core though it also isn't an accuracy emulator.
Well it has pretty good VDP and it's timing emulation, but doesn't deal with 68k/Z80 bus sharing or memory refresh related wait states, so runs made for it are somewhat more likely to sync on real hardware.
(not that there was much choice when people started doing Genesis TASes, if only at least Fusion was open source..).
I noticed this bit in your last reply. While it is true that Fusion is generally more accurate than Gens, it is nowhere near perfect and it does not emulate lots of things that woule be needed for a run to synch. Regen would be a better bet, as it is more accurate, but even it is not accurate enough. For example, neither of them emulate the z80 competing with the 68k for bus access, neither of them emulate the z80 being stopped during DMA when accessing 68k space, and a few other things. Some more info: http://sonicresearch.org/forums/index.php?showtopic=1335&p=38684 (interestingly, there is a quirk with z80 busreq that is correctly emulated on Gens but not on Kega that can lock up a real Genesis on some rare cases).
OK, so something may be possible with Power On Reset signals, I just wonder if they also reset the phase of those refresh cycles that memory controller/bus arbiter generates.
I don't think they do; they just go through a counter and are gated, IIRC. This is why I was going to make the reset phase circuit - at power-on, my module will hold the actual reset lines on 68K/Z80 in reset for some time (to allow caps to charge, POR circuit to stabilize if present and so on), then wait until correct phase is achieved before releasing reset.
notaz wrote:
True wrote:
It's not as serious as what I need to do for SNES...
Do you have that documented somewhere? Would be an interesting read.
Two words: clock injection. =)
More words: I will document this as I go. It is confirmed / verified that CPU and sound processor clock ratio matters for some games. As an emulator will assume perfect clock, I will be injecting phase-locked clocks into the console to see if I can get Super Metroid to sync, or at least get farther. If it gets farther with only this change, then we know emulation accuracy is good, and this could be the basis of a determinism-modified SNES being acceptable for verification.
I will photodocument the process. I also discuss this stuff regularly in #tasvideos.
Marble Madness syncs consistently without any loader programs or input modifications:
http://tasvideos.org/forum/viewtopic.php?p=392291#392291
Not surprising since it keeps CPU mostly idle and doesn't read hvcounter, so emu timing issues don't matter. Highly recommended as a test game for anyone making a Genesis bot.
I'll need to get a copy then, or finish my flashcart. My bot already plays back / interfaces with Genesis OK but it is hard to test with the games I do have. Thanks for finding that one out.
Edit: With Ecco verification, I need to get that game instead. It looks like it's time to hack in lua bindings for movie input reading and TH pin callbacks... if someone else doesn't do this I'll look into it early next year. Hopefully someone can add this before then.
