Alyosha
He/Him
Editor, Emulator Coder, Expert player (4727)
Location: US
Joined: 11/30/2014
Posts: 2983
Location: US
I am using this thread to keep track of my progress on a project to console verify TASes using console feedback to counter non-determinism. The plan is to use video output to time when to send inputs. Let's use SNES Battletoads as an example. Battletoads turns on automatic input polling before loading music for a level. Therefore the imprecise APU clock can change how long loading takes and thus how many input polls happen before the level starts. This happens on each loading screen, and even with careful tuning I just can't get close enough to get every one right. If we were able to ignore those early automatic polls, and only use the inputs needed to play the level, console playback would be pretty easy, as there isn't any RNG that requires cycle accurate execution. Fortunately, when loading is happening the screen is black, so if we say wait for a certain pixel on screen to be a certain color to trigger sending inputs, we could avoid the unused early automatic polls. The SNES in particular is perfectly suited as a test case for this plan as it sends out multiple video signals by default. I can do video capture with the composite signal and video compare using the RGBs signals. After looking around a bit online I think I can use a MC44250FN to convert RGB analog to digital, maybe with some voltage level shifting since the SNES only outputs 5 bits of color but that chip can handle 8. Various places online say the csync signal can be split into Hsync and Vsync with an LM1881, which should give me all the information I need to count pixels. I just need a programmable counter to latch the correct data. I think I can use an ESP32 to collect all the pixel data and send out the controller polls fast enough. So that's the plan anyway. It if works out maybe the same approach can be used for N64 (which doesn't output RGB but the pins are available on the encoder chip) or even disc based systems. The obvious downside here is that any TAS that requires cycle exact execution between emulator and console to sync will not work. I'll post again when I have the cable made and have good signals. Sidenote: Fiskbit on NesDev told me that the SNES actually has digital RGB outputs available on the ppu. It needs a bit of a mod to access but would save me a conversion step. I want to avoid touching the console if possible but I'll keep this in mind as a backup plan.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (4727)
Location: US
Joined: 11/30/2014
Posts: 2983
Location: US
I made a cable that has all the available signals from the SNES output on it connected. I was fortunate that the SNES RGB connector cable I bought had all the pins populated, so I just had to hack it up and solder some wires to it. I terminated the other video signals with an RJ-45 connector. Here is the description of what I have currently: EDIT: Removed no longer used. I also got myself a scope and tested the RGB output channels. I made a basic test program to test maximum output levels. It looks like max of 170 mV for full 5 bits set color, which seems pretty low (I see 660mV online for RGB signals) (EDIT: I had attenuation settings wrong, its actually ~1.5 volts.). For real world testing I captured some waveforms and the results are kind of interesting. Rise times are pretty slow compared to pixel clocking rate. One pixel lasts ~200 ns and rise times can be approximately that long. Obviously this still works as the RGB output channel is used regularly but this something to keep in mind when comparing pixel color emulator. Here is a sample: I still have the option to tap the digital RGB on the ppu, but I would really prefer if I didn't have to touch the console so I will press on with this for now. Now I need a clock source, split the csync signal, and to convert to digital color.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (4727)
Location: US
Joined: 11/30/2014
Posts: 2983
Location: US
I looked around a bit more and found the MAX1003 which is a 6 bit flash adc. I think this is a better fit than the MC44250FN. The 8 bits of the latter would be tricky to fit with the 5 bits of the SNES video output. With the MAX1003 I can attenuate the video signal to 125 mV from the current ~175 mV and then use the 250 mV peak to peak setting of the chip and simply ignore the top bit. I guess alternatively I can amplify to 250 mV and ignore the lower bit, but I would need to be careful about timing and distortions. Unfortunately cable testing has revealed significant noise on the RGB channels. Max red color for example can cause noise on the blue channel of the same magnitude as 1 bit would be. This noise doesn't exist when I measure from the video connector pin, so I guess that's what I get for getting cheap cables. I'll need to make something higher quality.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (4727)
Location: US
Joined: 11/30/2014
Posts: 2983
Location: US
Since clean signals are vital for this project to work, I redid the RGBs lines with RG174 coax. Soldering it was a huge pain but now I have no cross talk noise between the lines. Now I just need to attenuate to 125 mV peak to peak. Looking at the max1003 data sheet, I can use differential AC coupled input mode and the 500 mV peak to peak setting to give myself a little more leeway too (since voltage divisions will now be twice as much.) So I think the digitization has a good plan, now I need to start breadboarding stuff.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (4727)
Location: US
Joined: 11/30/2014
Posts: 2983
Location: US
Another digitization option is the AD9984A, which is a 10 bit ADC that can do all 3 RGB channels at once. This chip is a bit more complicated than the others in that it needs some programming to set up, but that wouldn't be too hard since I'll have an ESP32 running alongside it anyway. This also means I wouldn't be able to evaluate things until I have basically the whole system set up, which is a pretty big down side for a beginner like me. This chip only needs the hsync and vsync signals as inputs, which I will be generating anyway, and conveniently outputs a data clock along with the ADC conversion. Using this would save me the significant trouble of creating a clock source on my own. It also has programmable sampling time, so I can target towards the end of each pixel when the color intensity is actually reached. Pretty cool. Analog Devices has other ADCs and video decoders of various kinds, I just need to make a decision and go with it.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (4727)
Location: US
Joined: 11/30/2014
Posts: 2983
Location: US
After reading the data sheet a bit more carefully, I noticed that the AD9984A requires minimum 0.5V peak to peak for operation. So to use this I would need to fairly carefully amplify each color signal. I would also need some SMD soldering tools to use it which I currently don't have. So in the interest of simplicity I settled on the MAX1003 and got a couple of those along with some LMH1981's for sync separation. With the MAX1003 it's on me to set up clock and timing information, but once I manage that the operation is very simple. My biggest concern is drift between my clock and the SNES clock, but i think I can sync to hsync and that should be good enough for a single scanline. Another downside is that I need two of them for the 3 channels, but as long as sample timing doesn't vary too much between the two that should be fine. Probably the AD9984A is the best choice for someone who actually knows what they're doing, maybe if I learn enough from this process I can take another look at it. I'm happy with my current choice though, getting it working seems doable. I'll report back with some pictures as i test stuff and get things going.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (4727)
Location: US
Joined: 11/30/2014
Posts: 2983
Location: US
It turns out I made the most basic of basic mistakes and didn't have my probe attenuation matching my scope setting, so the actual RGB voltage peak to peak is ~1.5 volts not ~0.15 volts, oops! So now instead of attenuating to ~75% and being able to use differential input I have to attenuate to only ~33% to fit in the 500 mV peak to peak limit of the MAX1003, disaster! I tested this using basic potentiometers to divide the voltage down and got the above picture. The voltage signal is still technically DC so nothing weird happens, but the potentiometers must have some small internal capacitance because the voltage drop takes longer than for the original signal. I think this is still in time for pixel to pixel transitions, but it's pretty close. Aside from that the signals are still pretty clean after attenuating so I will continue with my original plan.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (4727)
Location: US
Joined: 11/30/2014
Posts: 2983
Location: US
I successfully managed to setup an lm1981 on a bread board and extract hsync and ysnc signals from the SNES. This is a very basic operation but was good practice for my first attempt at bread boarding stuff. Obviously I can use hsync and vsync to count scanlines, which is half the information I need. Now I need a clock source for generating ppu clocks, which are not available from the display port. If I sync the clocks (and clock generation) using hsync, Then sending this to the ADC I should be able to get pixel level resolution. It's slow going but so far so good.
Editor, Experienced player (708)
Joined: 11/8/2010
Posts: 4241
Great to see more progress on this! Good luck. I'll be following along (subscribed to topic) even if I don't comment.

1783008456