Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
I'm making steady progress on the new core. Input, sound, and savestates now work. If this were a production core it would basically be done.
Now it's time to optimize. I've already played around with a few things, and was happy to see that things I thought might improve performance actually did. This is very welcome considering how much I banged my head against the wall with C# trying to optimize things.
Once I'm comfortable with the level of optimization and can verify that nothing gets broken along the way, I can start the process of turning this into an MSX core. That part shouldn't be too hard hopefully. Having all of this infrastructure and the z80 in place is already at least 80% of the work.
For follow on work I'm leaning heavily toward an MC68000 based system. I think this will offer the most utility in the long run just having the cpu available. But I'm still not 100% sure yet.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
I'm making progress on a new C++ core. I made a game gear version as a proof of concept since it has mostly the same parts as MSX. It was more of struggle to convert things to work in C++ then I thought. Actually most of the problem was due to needing to use a C interface in the DLL. Things would be so much easier if there was a standard C++ interface, but oh well. This was a very worth while exercise because I was able to sort out most of the bugs just by comparison with the existing C# core.
As you can see in the screenshot, this core runs around 490 fps on my laptop. This is actually not very good, since this core in C# runs at 310 fps. This is no where near the performance I need, it should be up around 800 like gambatte. So I obviously have a long way to go in terms of optimization. The whole point of going to C++ though is that there are many possibilities to achieve those optimizations, so I'll be starting down that road once all other aspects of the core are done.
On the up side, starting from scratch has let me organize the core in very straightforward way that should hopefully be largely reusable as I build more systems. Also some of the more complicated things like memory domains and trace logger are done. The two major thins I have left to add are sound and savestates, but neither of those should be too tough.
After that I'll probably rework the CPU core to take advantage of C++ features. The Z80A is really slow, partly because I needed to capture all the signals for spectrum and CPC, but also just because it's poorly optimized. That was partly by design since I wanted maximum clarity in order to easily fix bugs in development, but by now it's quite well verified and I can sacrifice some clarity on the C++ side to go fast.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
Wow! Starting off 2020 strong with a awesome find, nice work The8bitbeast!
Although I would like to see this performed on console to ensure it's not a mupen artifact.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
feos wrote:
“arc” wrote:
They also suggested using some games that aren't so ancient. The zoomers watching were born around the year 2000 and didn't grow up with games from the 1980s/90s.
Do they also realize that GDQ runs also play “ancient” games? Because this isn’t a TASBot only thing. Do they even understand the point of GDQ?
I don't think they are really comparable. I think people expect novel things from TASBot that demonstrates advancement, it's why TASBot gets prime time spots and not early morning spots.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
Lobsterzelda wrote:
Just to double-check: could a movie made on BizHawk 2.3.2 obsolete this movie if it’s faster than this movie, or does it need to be done on BizHawk 2.3.3?
If you are using gambatte it doesn't really matter which version you use, it hasn't changed since 2.3.2.
Really though we should start disallowing older BizHawk builds, most of them are very obsolete.
I see you have a complete level 1 now with a 2 second save, so I'll cancel this one, there shouldn't be any hangups that cost 2 seconds. Feel free to use it as a baseline or just avoid redoing the tedious autoscrollers.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
Mario Maker was awesome!
That is the kind of innovative stuff that really fascinates me about TASBot, great work to all involved there.
For the other part of the block I agree with Arc's comments mostly, but am not really sure what to do about it. It's a tough spot to be in.
We have LibTAS now, but Celeste is done already so what comes next? I could make a SubGBHawk for GBC ACE, but that probably would still be seen as old news.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
Oops, yeah you are right it should say 'don't change from 8.'
Yeah I should really remove that as a sync setting since it's probably always 8 (actually it's likely just CPU reset vector processing.) It's more of a leftover from comparisons with gambatte earlier on in development.
I'll probably remove it as a sync setting for 2.4. EDIT: Done.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
This is GBHawk in 2.3.3. Every input frame is now exactly the same length. Gambatte still uses a variable length frame so comparisons are still not one to one.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
Yeah you need to adjust the movie if you are using gambatte because they are timed differently.
I tested gambatte as well and with a few adjustments it did sync just fine, but unfortunately I don't have that file anymore. It shouldn't be more difficault then adding / subtracting a fe blank frames here and there though.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
I didn't try to improve level 1, so it looks like you are ahead by a bit already. You would have to play through a bit more of the game though to make sure you don't have RNG problems.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
Wow finally done, another sorely needed update finally realized. Nice work sticking with this Arc. It's always a bit embarrassing to see how irrelevant a lot of the old TASes have become, at least this is one less. You're pretty committed to have done that luck manipulation by hand. I hope the next project you pick is something you can enjoy a little more! Yes vote from me.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
InfamousKnight wrote:
Maybe we should have a vote on desired core? Like poll results?
I would really love to see PCem ported to bizhawk. With waterbox there shouldnt be any desync issues. If PPSSPP is easier to integrate, I would vote on that instead.
Well I'll be on MSX for at least a month or 2, so there's no rush here. And also it's not really a vote, people just work on things that interest them and seem accomplishable with the amount of time they have.
Sonia wrote:
Can you give us some links containing the resources/materials necessary for studying then? Because for someone starting from zero like me, I'd appreciate some directions at least.
Just information that'd be relevant to fixing the core is enough.
Sure!
If you are totally new to programming in general, the best place to start is C++ tutorials here:
http://www.cplusplus.com/doc/tutorial/
In terms of program complexity, emulator cores are VERY simple. You don't really need to know anything past the Class tutorials there to fix bugs. C++ is a bit different from C#, but aside from syntax things are mostly the same for basic stuff.
Once you've gotten comfortable with basic programs, clone BizHawk and start Console.WriteLine()'ing around the PCE core to get an idea of the flow of the system. Experimenting is the best way to learn.
For PCE information, the best readily available resource is here:
http://www.archaicpixels.com/
It's not the best resource I've ever seen, but it's pretty tractable. Don't get overwhelmed in the details, try to understand this part first:
http://www.archaicpixels.com/Memory_Map
Staring at the emulator as a whole won't really help you learn too much though. I recommend looking at a simple, concrete problem and seeing if you can track down what's going wrong. This is a good place to start:
https://github.com/TASVideos/BizHawk/issues/1580
Splatterhouse is just an ordinary PCE title and doesn't use complicated CD stuff or supergrafx. I am pretty confident you should be able to figure this one out.
Alternatively, you can look into this: https://github.com/TASVideos/BizHawk/issues/1231
Crashes are pretty simple to spot in a trace log, should be fairly easy to analyze.
Good Luck, you can do it!
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
EZGames69 wrote:
Can we also look into porting Mednafen’s PCE core into bizhawk to replace PCEHawk?
Not me, maybe feos or SuuperW once they are done now that waterbox can be built.
Honestly though, starting from zero, you or anyone other interested person is only like 2 months of relatively simple study away from being 100% capable of fixing the existing core. It's really really not that hard. I would strongly encourage you or any other interested PCE folks to give it a try, you may find it rewarding to get a system working that you are very interested in.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
dekutony wrote:
I would like to see a new Genesis core that could hit cycle accuarcy, like a BlastEm port or something.
That's probably going to be difficult to do though.
What utility would you get out of this? Is there something you see that is holding back gensplusgx? I honestly don't see what is holding back that core.
Realistically I would probably need to make a basic genesis core anyway in order to test a MC68000 if I go that route, just so I can compare trace logs. Otherwise I would have to blind debug a new CPU core on a new system in an unfamiliar environment and that doesn't sound like a fun time for me. It wouldn't be a production core though.
If someone out there wanted to make a 32X core I would commit to making a cycle accurate genesis core with a clean interface to build off of, but I would need a solid commitment.
Al82 wrote:
Personally, I think a working TAS-capable Amiga emulator would be amazing; the workflow in recording games for the system is so long-winded without re-recording.
Even if I did make an Amiga core it would only be a baseline implementation (with easy expandability baked in.) It's just a stepping stone for me, but maybe having a baseline model would inspire someone else to jump in and try expanding it. And I don't mind duplicating effort if necessary, >99% of what I've done in BizHawk was already done somewhere else at least once.
For now I'll press ahead with MSX. I've almost got the Z80A core ported to C++.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
Sonia wrote:
Here are the systems currently missing in Bizhawk that I'm looking forward to the most:
...
MSX
OpenMSX/BlueMSX could be ported over, but since you want to develop something in-house I guess that's out of question. I remember you once said all the components for a MSX core are already there, and it's just a matter of stitching them together. So maybe this one wouldn't be super hard?
...
That's it. As a side note, I know this has already been said many times, but if you can reserve a bit of time to improve PCEHawk and bring it closer to Mednafen levels of accuracy, it'd be much appreciated. That core is in pretty bad shape.
xy2_ wrote:
+1 for X68000 and PC-98, these are systems that I would definitely use.
MSX is an excellent suggestion that I completely overlooked. That does sound like a good place to start, I think I'll go for it. Thanks! X68000 might be a good follow on, not sure yet.
As for PCE, I just don't have any interest in it. It would be great if someone who was actually interested in the system would try to fix the bugs, it really needs someone who can focus some time on it.
feos wrote:
Developing emulation for these from scratch looks like a waste of time to me if instead existing emulators could be improved and reused.
I wasn't aware that PCem and UAE were so highly developed. The gap between existing and actually being ported is quite large though, but I guess I'll just leave them for someone else.
EZGames69 wrote:
Some in the discord server agree that they want PSP support.
PSP would itself be of the same difficulty as N64, I don't see it happening, at least not form me.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
As I develop more cores for BizHawk, it's increasingly clear where the line is between what can be done in native C# and what can't. And as developing cores for 30+ year old systems has gotten kind of dull for me, I wanted to take some time to think about how I can contribute to more modern systems and where actual gaps are in Bizhawk that need filling.
My long term goal is to build an N64 core in C++, but it's too complicated to just drop everything and start doing. I need to get an idea on a simpler system of how to best organize a core in a C++ framework that accounts for both accuracy and performance. The current in-house core architecture for C# is very simple and very robust, I can (and have) added a blank core template in a matter of hours. But, it just won't work for modern systems with higher clock speeds. I need a new approach, but I also want something that's not just a one-off, a new architecture that's re-usable would be ideal. And since I learn nothing from looking at other people's code, I really need to dive in myself.
Looking at the current landscape in BizHawk, it seems like the 2 most important cases to consider would be IBM PC and Amiga. IBM would probably be simpler at first, but the IBM landscape is so broad this might be a very deep rabbit hole. It might get the most use though. Amiga would have the advantage of needing an in-house 68000 cpu which could be re-used on a lot of things. But I'm not sure how much those things would actually be used (a lot of people still just use gens even though we have gensplusgx for example.) Also instruction decoding for 68000 is a nightmare, but once that part is done it wouldn't be so bad.
I'm open to suggestions here though. Is there a system that has a currently ported core that would be better off with something in-house that has more control? Is there a system I'm overlooking that is more modern then old 8-bit stuff that someone would definitely use? What I don't want though is something like SNES where infinite effort would be expended in PPU details that have nothing to do with overall architecture.
Basically I want a roadmap for myself like:
Easy system, work out a modern C++ architecture and play around with optimization
|
|
v
Medium System, more modern CPU architecture that requires careful planning and optimization. Pipelines, cache misses, etc. but overall system still tractable.
|
|
v
N64
If anyone has any input here I would really like to hear it. Thanks!
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
Hey that's pretty cool. Nice job making sure things are realistic.
By console verification do you mean the entire TAS, or just verify that the trick is possible?
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
It's significantly slower then the one by FractalFusion and I don't really see the point of moving the goal posts here. It's an interesting glitch I guess though.
I'm making steady progress on the new core. Input, sound, and savestates now work. If this were a production core it would basically be done.
Now it's time to optimize. I've already played around with a few things, and was happy to see that things I thought might improve performance actually did. This is very welcome considering how much I banged my head against the wall with C# trying to optimize things.
Once I'm comfortable with the level of optimization and can verify that nothing gets broken along the way, I can start the process of turning this into an MSX core. That part shouldn't be too hard hopefully. Having all of this infrastructure and the z80 in place is already at least 80% of the work.
For follow on work I'm leaning heavily toward an MC68000 based system. I think this will offer the most utility in the long run just having the cpu available. But I'm still not 100% sure yet.
I'm making progress on a new C++ core. I made a game gear version as a proof of concept since it has mostly the same parts as MSX. It was more of struggle to convert things to work in C++ then I thought. Actually most of the problem was due to needing to use a C interface in the DLL. Things would be so much easier if there was a standard C++ interface, but oh well. This was a very worth while exercise because I was able to sort out most of the bugs just by comparison with the existing C# core.
As you can see in the screenshot, this core runs around 490 fps on my laptop. This is actually not very good, since this core in C# runs at 310 fps. This is no where near the performance I need, it should be up around 800 like gambatte. So I obviously have a long way to go in terms of optimization. The whole point of going to C++ though is that there are many possibilities to achieve those optimizations, so I'll be starting down that road once all other aspects of the core are done.
On the up side, starting from scratch has let me organize the core in very straightforward way that should hopefully be largely reusable as I build more systems. Also some of the more complicated things like memory domains and trace logger are done. The two major thins I have left to add are sound and savestates, but neither of those should be too tough.
After that I'll probably rework the CPU core to take advantage of C++ features. The Z80A is really slow, partly because I needed to capture all the signals for spectrum and CPC, but also just because it's poorly optimized. That was partly by design since I wanted maximum clarity in order to easily fix bugs in development, but by now it's quite well verified and I can sacrifice some clarity on the C++ side to go fast.