(Configured so the lower the "Main Value" becomes, the better the run attempt is.)
Hello everyone,
I have made a Bizhawk forked experimental build that includes a Genetic Algorithm Bot, based off of the Basic Bot that comes with Bizhawk.
I would like to request for anyone who uses this bot, to see:
* Is the bot helpful for you in creating TAS runs?
* Is there any issues with the bot?
* Any other feedback?
Testimonialsslamo wrote:
genetic algo bot has helped find a new clip (Game: Pac-in-Time (Game Boy))
bpm46 wrote:
In this game, you can do a midair attack that boosts you off enemies higher than if you jump on them. What I did here was jump from the top platform and attack-boost off the enemies, but the bot instead drops onto the ground then jumps on them normally. This seems to have slightly less lag and also lets the player land sooner and jump over the slope at the end.
There's also an interesting coincidence: jumping into the flower at the end doesn't save or lose time here, but if you don't jump, the next level loads several frames slower. The bot didn't know that, its input was set to stop before the flower. I'm lucky it happened to jump there and I noticed it.
Randomno wrote:
Gave it a quick try. I had no idea what to set the probability controls to. I wanted to use as few left presses as possible, so set right and B to 65, A to 25, and left to 5. Main value of the highest X position and 100 frames. It had good results after 75 generations. I kept it going until 180 generations and it hadn't improved. I was able to improve it by a few frames by manually editing the input afterwards to remove a few left presses. My initial attempt had 715 xpos, the bot found 730, and my changes afterwards got it to 741. So a good result for just a few minutes of work, and saved me figuring out a frustrating section to optimise.
How the Botting Works:
When you begin botting, the first thing you do is to set up the parameters and configurations for the bot to know about your game inputs. Things like, how long the input recording should take, how many attempts do you want to do per population generation, how likely it is for the bot to enter in the inputs you wanted during the botting process, and what RAM address values are you expecting by the end of the botting process.
The moment you press the "Run" button, the first thing the bot does is, it stores the current state of your specified RAM address value. This is called the "origin", or the "source attempt".
As it proceeds to enter in inputs, the game will handle what happens after those inputs are sent, and does things in the game. The RAM address values you specify to the bot will change (or not change) depending on those inputs.
By the time the bot reaches the end of the input recording session, which is on the last frame of the entire frame length you specified, it will then log the final RAM address values into a collection. This result is called the "destination" or the "target attempt". The Attempts counter will then increase, and the bot will then proceed to do a new Attempt within the current generation.
When all of the attempts in the current generation is done, and all of the results are collected, it will begin the "evaluation" process. When evaluating, it compares which attempt's results leads to a value that fits the conditions you have set to the bot, and then sets that as the "Best Attempt". if it finds a "Best Attempt", it will log it and displays the information to you, as well as which "Attempt" it was, and from which "Generation". Otherwise, if no results fit your conditions, the results will be discarded, and you would have a blank "Best Attempt". This is intended, so you don't get confused about the "Best Attempt", when it was clearly not the best one.
After the evaluation is complete, it will then begin the crossover and mutation phase, where some of the recordings are "breed" from the previous generation, and may mutate as a result of the crossover.
Finally, we proceed to the next generation, and the generation counter is incremented. This whole process repeats from the "origin", reach the "destination", and evaluate the results at the end of each generation, until you stop the bot.
Your best attempt will be the attempt that matches your set of conditions. If you aren't happy with it, you can restart over by clearing the attempted results with the "Clear" button, and modify the input probabilities (or called "weighted values"), so you can refine the best attempts better.
Changelogs:
v1.0.4 - 2023 August 5
Made a pre-release build.
Updated log entry generation class to reference the newer generator from BizHawk dev build, 2023 May 25.
v1.0.3 - 2023 August 2
Fixed issue related to loading a previously saved BOT file from failing.
v1.0.2 - 2023 April 8
Fixed "Turn Off Audio/Video" setting not reset back to the state it should be.
v1.0.1 - 2023 February 8
Updated the bot to accommodate new External Tools API changes for Bizhawk 2.9 (ETA pending release).
v1.0.0 - 2022 July 28
Used a workaround (thanks to adelikat for the suggestion) to resolve the configuration file issue.
All major bugs are now fixed. Consider this bot to be released as a stable build, and no longer be an experimental build.
v0.1.4 - 2022 July 28
Allows the user to modify the default mutation rate.
Improved tooltip description.
Modified the input control label text, so it is more meaningful.
Fixed Clear button state.
v0.1.3 - 2022 July 27
Fixed issue where you can't do negative values to do comparison checks.
Added upstream fixes with loading BOT files for wrong games.
Use unsigned long integers (unsigned 64-bit integers) for address values.
This should improve compatibility with emulators that requires 64-bit addresses.
v0.1.2 - 2022 July 26
Clear button is not enabled immediately after the first attempt was made.
Added missing Generation label.
Fixed crash when GeneticAlgorithmBot attempts to load BasicBot .BOT save files. It is now compatible with loading BasicBot files. However, it will save as GeneticAlgorithmBot file.
Fixed Generation counter not showing up and properly counting.
Fixed Attempt counter not using the correct variable.
Fixed Best Result not storing Generation counter.
Fixed issue where the Run/Play buttons can get stay stuck in disabled state when entering addresses and sliding the slider controls.
v0.1.1 - 2022 July 25
Fixed issue where the < and <= are not being evaluated correctly.
Increased BotAttempt values from "int" to "uint" type.
Changed value comparison checks to support "long" type as a result.
Restored text fields from being disabled all the time.
Restored functionality with comparison attempts.
Slightly tweaked the UI. Not sure if it will help.
v0.1.0 - 2022 July 25
First minor release.
Ported the experimental build into an external tool.
Removed unusable GeneticSharp library, improving its synchronizations as a result.
Can now set Population Size.
Now displays Generations and correct Attempts.
Now properly evaluates each attempt's fitnesses.
v0.0.4 - 2022 July 22
Fixed the Run/Stop button killing off the population and creating a new one. Made it so that only when you press the "Clear" button, will the current population then be wiped clean.
You will then start with a new population and begin from the initial generation after you press "Clear" and then "Run" button, strictly in this order.
If you click the "Run" button first, and press "Clear", the bot will continue to use the old population, and will continue using this population until you click "Stop". After that, you will no longer be able to recover this population.
Your Best Attempt will be cleared, and you will still be able to get a new Best Attempt while the bot is running.
Run/Stop button will now only advance to the next generation, and they will cause the bot to evaluate the previous generation after advancing.
Attempts will not increase as a result.
Current Best Attempt will also not be overwritten as a result.
Tweaked fitness evaluation.
v0.0.3 - 2022 July 22
Fixed input recording playback problems.
Fixed UI where the logs are not updated correctly.
Miscellaneous bugfixes.
v0.0.2 - 2022 July 21
Fixed issue where the bot was only producing 1 frame of inputs, rather than using the entire frame length (or the frame amount specified in the bot)
Added a bunch of tweaks to make the genetic algorithm work properly.
v0.0.1 - 2022 July 18
Initial release.
Known Issues:
NEW: On Windows 11, your UI may be misaligned. This is caused by Windows 11's new UI design changes, and that's something I cannot handle, since I'm on Windows 10.
[*]NEW: Currently waiting on this issue to be fixed. Github Issue Link (Fixed in v1.0.0)
The next time you open EmuHawk, it will corrupt your config.ini file. Please make sure to make backups of your config.ini file before attempting this bot.
This is the reason why this bot is still an experimental build.
There is an "off-by-one" error. (For example: If frame length is set to 200, recording only saves up to 199 frames of input. If set to 100, you get 99 frames. Etc.)
[*]Basic Bot and Genetic Algorithm Bot cannot be opened at the same time. (Because Genetic Algorithm Bot is a child class of Basic Bot parent class.) (Don't know if this still matters now.)
Put the DLL in the "ExternalTools" folder in BizHawk.
Launch EmuHawk.
Tools -> External Tools -> Genetic Algorithm Bot
The Genetic Algorithm Bot is used just like how you use Basic Bot. The only difference is that it should slowly evolve so that you would get the desired results as time goes on, in theory. However, practice doesn't always follow theories, and there are definitely some kinks in the build.
However, there are many things that I'm still learning and tweaking things as I go. I'm still testing and playing around with the code, and I do plan to continue improving upon it for the foreseeable future.
I do not know if there are any plans to put this bot into Bizhawk proper, but that really depends. We shall see.
Details
This Genetic Algorithm Bot inherits some features from the built-in Basic Bot.
What it does is, it takes in whatever probability you give it, to the control buttons, a determined probability for each button, and the max value you set for the bot.
Then, when it starts the botting process, it begins by setting up an "Adam Chromosome", and generates a pool of population. Each member of this pool is called a chromosome, and are entities that can produce offsprings. Just like in real life, chromosomes are based around the theory of evolution, where future generations should be better and fitter than the parents, as they evolve towards reaching the intended target you set in the bot.
Each chromosome stores an input recording of the play, up to the frame length you specify in the bot. The probability for each button is then used to calculate the likelihood the button is pressed in the duration of the input recording.
Each end of a cycle, it begins calculating and evaluates, out of all the generation cycles, which one has the better fitness. It then uses that to determine the parents of the next generation's offspring. While producing the offspring, the chromosome is then mutated and crossed over with other chromosomes, so that mutations in the chromosomes can become the key to surviving with a better fitness level.
You repeat the process until, in theory, the chromosome with the highest fitness is able to hit the intended target you specify to the bot.
This mean, you should get an optimal solution of input recordings for your TAS runs. (Again, in theory.)
Pretty neat idea, but why not make it an external tool? then it wouldn't need to be an experimental build
I have no idea what it means "to make an external tool". Also, I have no concrete picture of what an external tool should be like.
I know there are external projects, external core projects, and external tool projects in the Bizhawk source code, but I'm not that familiar with any of those.
I might have to create an external tool build instead of a Bizhawk integrated bot instead. I am starting to notice some inefficiencies with the GeneticSharp library I am using for this Genetic Algorithm Bot, especially where are multiple threads running concurrently and deadlocking itself, pausing/halting the Bizhawk emulator entirely without warning, and not having total control over the mutations.
I tried this out for a little bit but it doesn't seem to want to work correctly. I'm somewhat familiar with Basic Bot so I don't think I did anything too stupid, like I remembered to turn on recording mode. The problem is that the bot only writes an input on only one random frame, no matter what you set the # of frames to. It also doesn't adhere to the input probabilities you can set; if I have R set to 100%, it still only presses it for one frame. I'm trying this with Pac-In-Time for the GB, if that helps.
I tried this out for a little bit but it doesn't seem to want to work correctly. I'm somewhat familiar with Basic Bot so I don't think I did anything too stupid, like I remembered to turn on recording mode. The problem is that the bot only writes an input on only one random frame, no matter what you set the # of frames to. It also doesn't adhere to the input probabilities you can set; if I have R set to 100%, it still only presses it for one frame. I'm trying this with Pac-In-Time for the GB, if that helps.
Updated the bot. It's in the OP. Please check it out and see if you still encounter issues. Thanks for using the bot!
Thanks! It generates proper inputs now. After testing some more, there are some issues with playing back the best attempt. It seems like it's not being saved correctly. Only a portion of the input is played, for example if I set the frame amount to 50 it will stop short of that. Also if I try to play it back, the input roll in the Bot window doesn't match what's played back in TAStudio. The attempt # is also always at 0.
Otherwise, it looks like it's actually coming up with some good solutions. I just wish I could play them back!
I'm aware of the off-by-one issue with the frame numbers. (Like, if you set it to 200 frames, it will only log up to 199 frames). I'm trying to investigate how to solve this, and whether if it's better to just append "+ 1" when creating the frames. I'm wondering if that is related to what you're seeing where the recording stops short of it.
I'll aim to try to fix the recording for the next build. Great to see you are seeing some interesting things with it.
It's not just short by 1 for me, it's short by a lot. And I just figured something out - it only plays back the current attempt, even if it's not finished. That's why it's stopping short.
It's not just short by 1 for me, it's short by a lot. And I just figured something out - it only plays back the current attempt, even if it's not finished. That's why it's stopping short.
Made a new build.
EDIT: Nevermind.... just found the "Play back best attempt recording not syncing up" bug still persisted even after I thought it was fixed... Gah...
I can't believe it occurred after more than 20 tries. Then again, I was clicking on "Run" pretty quickly.
EDIT 2: Someone mentioned it might have been caused by bad save states, since the Genetic Algorithm Bot is using the latest dev builds, and not a stable release build.
It's behaving kind of strangely. If I clear the previous best, run the bot for one attempt, and then try to play back that "best" attempt, it looks completely different, often making up an attempt that I've never seen before.
It's behaving kind of strangely. If I clear the previous best, run the bot for one attempt, and then try to play back that "best" attempt, it looks completely different, often making up an attempt that I've never seen before.
I'm now starting to think it's not a good idea to use a genetic algorithm library that's usually used in academic studies and research papers.
Yesterday (more like early this morning in the 1AM ~ 2AM range), I did check and examined each line of code, line by line, trying to figure out how the genetic algorithm library I'm using determines the fitnesses and how it evaluates each input recording. To my amazement, and possibly an oversight on my part, is it evaluates "immediately" whenever I start or resume the genetic algorithm library. This library doesn't support "delayed evaluations" (in other words, it does not "let me evaluate the fitness of this chromosome at a later point in time"), and the library code doesn't even check to see whether we have obtained the final RAM address values. The library itself doesn't really have good information on how to really customize the data types it is using to do the number crunching, even though it's one of the more popular genetic algorithm libraries in C#. (It's called GeneticSharp, for those curious.)
I'm in the process of converting this genetic algorithm bot to an external tool, and in the process, completely gut the original genetic algorithm library I was using. I have the source code for the genetic algorithm library, so I can make some modifications to allow "delayed evaluations", and then include that new library as part of the external tool. This should allow more customizations for me, as well as making it more flexible enough so that it's more stable and less agonizing for everyone.
I probably will need to get some help from adelikat, CasualPokePlayer, and YoshiRulez for some API Hawk modifications.
Sorry for lowering your expectations. I'll be commiting the external tool's source code to this repository here:
https://github.com/tommai78101/Bizhawk-GeneticAlgorithmBot
But I wanted to clean up some things I messed up in the Basic Bot and the Genetic Algorithm Bot code base in my forked experimental build, before making my commits.
UPDATE 1: New build is out. Please see OP, and disregard the contents below.
Update on the progress.
I believed I have finished porting the bot code over to a new project as an external tool. As mentioned before, I gutted the genetic algorithm library completely from the bot, because it runs multithreaded tasks in the background to calculate the fitness values, and you know what they say how multithreading things can introduce desynchronizations in the TAS recording. Thus, this external tool is single-threaded.
The current progress is pushed to this repository here:
https://github.com/tommai78101/Bizhawk-GeneticAlgorithmBot
There are only 2 things to tackle, that are outside of my abilities:
1) Loading an external tool and being able to debug code requires the memory domains be loaded upon initialization. (Github issue here)
2) There seems to be an issue regarding saving the configuration files for the external tools. (Github issue here)
The first point is important, because it can help me troubleshoot some of the fitness evaluations and code logic bugs that aren't detected by the compiler's build errors.
The second point is important because it is a requirement for external tools when the tool opens and loads ROMs into the emulator.
I'll try to figure out any alternatives to test the latest external tool build if any opportunities arise.
I've tested this a bit and it does indeed save the correct attempt! I'll play around with this more in depth and see what I can come up with, thank you for your efforts.
I'm not sure if you know about this or if it looks normal for you, but the GUI looks fairly messed up to me:
Some of the objects are on top of each other. It's still completely usable though.
UI fixes will come in the next build. If the tweaks aren't working, you might have to increase the window's width and height (or maximize it).
OP is updated.
Adelikat proposed an A/B Test, using the Basic Bot to compete with my GA Bot.
The simple goal is to get Mario move/jump the farthest from its starting position within 30 frames, in Super Mario Bros. NES of World 1-1, using core NESHawk, with the following bot parameters:
Each test took around 15 minutes to do. Each test only checks the Main Value set to the address of 0x0086, using the comparison type >=. The test stops exactly at 30,000 frames.
A/B Test Results:
Basic Bot:
Genetic Algorithm Bot:
Thus far, I think this benchmark is the first bot comparison baseline. My bot moved Mario more than the Basic Bot by 5 more pixels to the right. I think it looks promising.
On an unrelated note, I found 3 UI bugs, so I gotta go fix those.
Still having some problems with the GUI. I forgot to mention last time that I've gotten this error ever since it got ported to an external tool:
Here's the full contents of the error window: https://pastebin.com/ZwEZ97cE
The GUI issue may be related to this, not sure. I can press continue and get a window identical to the one I posted yesterday - notice how it also has the wrong title on the window. Stretching the window out doesn't help. Maybe this is a Windows 11 problem.
I'm also still not getting any best results when I start with a cleared state and use < or <=. The value option works now, though.
Still having some problems with the GUI. I forgot to mention last time that I've gotten this error ever since it got ported to an external tool:
Here's the full contents of the error window: https://pastebin.com/ZwEZ97cE
The GUI issue may be related to this, not sure. I can press continue and get a window identical to the one I posted yesterday - notice how it also has the wrong title on the window. Stretching the window out doesn't help. Maybe this is a Windows 11 problem.
I'm also still not getting any best results when I start with a cleared state and use < or <=. The value option works now, though.
According to YoshiRulz,
OpposingDirPolicy
is a property that doesn't exist in 2.8 stable, and was added to 2.9 dev builds. (Which I also didn't know until now myself.) I can make a new build for 2.8 stable, and will add it to the OP later on.
UPDATE: Apparently, it was partially my fault for not explaining how the bot keeps track of the best attempts during the botting process. I'll be adding this info to the OP shortly.
New build is released.
Some notes:
The bot is not backwards-compatible with BizHawk v2.8 stable release and below, because of a change to services loader to skip loading services, if conditions aren't satisfied. (TASEmulators/BizHawk@d796210)
Unless this fix is backported to BizHawk v2.8, the bot is exclusive to the development builds and BizHawk v2.9 for the foreseeable future.
It is possible the issue you may be having with < and <= not evaluating correctly is due to this same Github issue mentioned earlier.
The configuration file stores the information of what data types you use when you attempt to bot your recordings. With no bugfix, the configuration file will always be corrupted, and therefore all saved user preferences will be reverted to the default settings. By default, the bot will assume you are looking at the address values of size 1 byte, and will always do comparison checks with that data size. You may need to manually (re)set the data size back to 2 or 4 bytes, and also toggle whether to use Little Endian or Big Endian, when doing comparison checks where the numbers are going to be higher than 128 or lower than -128.
I've tested this a bit and it does indeed save the correct attempt! I'll play around with this more in depth and see what I can come up with, thank you for your efforts.
I'm not sure if you know about this or if it looks normal for you, but the GUI looks fairly messed up to me:
Some of the objects are on top of each other. It's still completely usable though.
I decided to use my work computer which is running Windows 11, and see if I can fix the UI layout on that machine.
But I didn't see anything strange, nor change anything in the code.
Please check the release for v0.1.3, and see if the build I posted there helps solve the UI issue you're facing. Let me know if anything. There are 2 builds, one is built on Windows 10 (denoted as "win10"), and the other is built on Windows 11 (denoted as "win11").
Updated the bot v0.1.4 to allow anyone to modify the mutation rates. Also, I made some changes to explain how the mutation rates and the input weighted values work together.
Tomorrow is the deadline for me to pause development on the bot. Xenoblade Chronicles 3 comes out, and I'll be spending more time focused on that game.
Until I see big upstream changes requiring me to fix this bot, I probably won't come back to work on the bot, until I beat XB3.
(Configured so the lower the "Main Value" becomes, the better the run attempt is.)
Hello everyone,
I have made a Bizhawk forked experimental build that includes a Genetic Algorithm Bot, based off of the Basic Bot that comes with Bizhawk.
I would like to request for anyone who uses this bot, to see:
There's also an interesting coincidence: jumping into the flower at the end doesn't save or lose time here, but if you don't jump, the next level loads several frames slower. The bot didn't know that, its input was set to stop before the flower. I'm lucky it happened to jump there and I noticed it.
Some of the objects are on top of each other. It's still completely usable though.
Each test took around 15 minutes to do. Each test only checks the Main Value set to the address of 0x0086, using the comparison type >=. The test stops exactly at 30,000 frames.
A/B Test Results:
Basic Bot:
Genetic Algorithm Bot:
Thus far, I think this benchmark is the first bot comparison baseline. My bot moved Mario more than the Basic Bot by 5 more pixels to the right. I think it looks promising.
On an unrelated note, I found 3 UI bugs, so I gotta go fix those.
Here's the full contents of the error window: https://pastebin.com/ZwEZ97cE
The GUI issue may be related to this, not sure. I can press continue and get a window identical to the one I posted yesterday - notice how it also has the wrong title on the window. Stretching the window out doesn't help. Maybe this is a Windows 11 problem.
I'm also still not getting any best results when I start with a cleared state and use < or <=. The value option works now, though.
But I didn't see anything strange, nor change anything in the code.
Please check the release for v0.1.3, and see if the build I posted there helps solve the UI issue you're facing. Let me know if anything. There are 2 builds, one is built on Windows 10 (denoted as "win10"), and the other is built on Windows 11 (denoted as "win11").