Posts for Dwood15
1 2 3
Posted: 11/11/2016 2:54 AM
Quote
Dwood15
Experienced Forum User
Joined: 6/18/2015
Posts: 54
SavantAI wrote:
I ported the original LUA code from SethBling NEAT to native c# inside the source code for bizhawk, firstly because NEAT was something I tried for a long time, I changed many parts, but can run it at 4-5k fps without drawing anything. I sometimes stream it @ https://www.twitch.tv/savantai Come by for a visit some time, we can chat. I appreciate all suggestions. Thanks, SavantAI.
Can you post the code to github? I'm interested in how you set it up.
Posted: 4/25/2016 4:09 AM
Quote
Dwood15
Experienced Forum User
Joined: 6/18/2015
Posts: 54
Hey guys, Just so everyone's aware, I've moved my lua project to GitHub: https://github.com/Dwood15/LuaNEAT-rnn The smw-bizhawk codebase I was working with has long since been updated, and is nearly unrecognizable to the way it was before, so I'm not going to update that code to latest unless there are some new memory addresses I should care about. One thing I'd like to do is make some changes to the code to make the neural network more agnostic from SMW in the future. Hopefully next week I'll be able to update the code. This ai has some issues which are not present on my main machine, so somewhere along the lines I've updated the ai and broken something, so as of April 24th, 2016 the script is broken and will crash after the final species has been evaluated and a new generation is created. This will be fixed in the next couple of days. The readme of the project describes more in-depth what's going on, and the goals of my project. I work on it when I'm in the mood basically, so updates for the project are pretty sporadic, but I do have a vision of where I want my clone of MarI/O to go in the long run. Here's hoping I can make it so agnostic that we can get ai playing other games as well.
Posted: 11/2/2015 11:48 PM
Quote
Dwood15
Experienced Forum User
Joined: 6/18/2015
Posts: 54
That code has been completely stripped from my script, but sadly, I'm a simple programmer, not a reverse engineer. If you want more specific information, you're better off looking at how to read memory addresses and assembly. There's websites that have basically the roms of the games completely mapped out, so there is that.
Posted: 10/30/2015 12:40 PM
Quote
Dwood15
Experienced Forum User
Joined: 6/18/2015
Posts: 54
spankypants wrote:
righto, I'll put up the three files. Does that mean you are stopping development of *I/O AI's?
No, it's 2 files. Just two. One named neatevolve.lua and the other is smw-bizhawk.lua. and yes, very soon, my MarI/O version will no longer be under development. I've just got a couple of bugs I'd like to fix first, but I'm kind of sick of the lua programming language.
Posted: 10/29/2015 10:57 PM
Quote
Dwood15
Experienced Forum User
Joined: 6/18/2015
Posts: 54
spankypants wrote:
Dwood15, I edited your script to say that it needed the smw-bizhawk file as well, I didn't want to include it in the repo as it might get updated. I'll put in a README file to say that it needs to be downloaded along with your script.
Well, you need is the neatevolve script, which looks like SethBling's AND the smw-bizhawk script, which is different than the one at the guy's repository. The smw-bizhawk.lua script I edited probably won't be changing, and I am beginning to get real tired of lua.
Posted: 10/28/2015 2:27 AM
Quote
Dwood15
Experienced Forum User
Joined: 6/18/2015
Posts: 54
Thanks for adding my script spanky, but to let you know, my AI's lua script is missing from your repository. The ai needs both files.
Posted: 10/21/2015 1:42 AM
Quote
Dwood15
Experienced Forum User
Joined: 6/18/2015
Posts: 54
This is going to be my first time publishing this script in a few months, so please be kind to it! Short update, not that anyone cares: My main dev machine is still not up and running, so a c# port of the ai is still a long ways off. (VS 2010 is the latest version I have installed on this laptop, and VS 2013, which is on my new machine, is sooo nice...) I've briefly been looking into multi-threading in LUA. Thing is, there's a lot of functions even with in the script itself, which could easily be set to a multi-threaded state, but my knowledge on what it takes to thread in LUA is very limited. I may just wait it out- for the time being, anyway. At any rate, I've finally made a change I've meant to do for a very long time, and that's add a config file setup, so that you don't have to open the script file in order to make minor changes. All you need to do is stop the script, hit refresh, and restart it again. With a config system, I can have it run 24/7, log all the data, move it to a run #, and then archive it for later usage. I can also do things like modify the max number of nodes, the mutation rates, et cetera. Since the ai takes a lot of variables to get started, I figure this will be the most effective way of moving with my plans. I'm also going to add a "no-ai" mode, where a player can evaluate themselves against the ai. Please note a few things with this: Please, do not report to me a single bug from a modification you made. I do not have the time nor the desire to fix your problems. Secondly, On the first run, the script will generate a config file. The config setup still hasn't been worked out 100% yet, so I'm leaving it barebones, as-is. Many of the variables even a .01 change can make a large difference, so be careful! (300 is the # of individuals per generation, 255000 is the max number of nodes You can probably change this without too much difference in your AI's fitness, however, you may end up killing your performance and ram if you increase it by a lot. Remember that the number of nodes in each pool is a function of the number of individuals per round - this means you could hit Lua's 32bit memory limit quite easily (i've done it, hence the 255000, not 1 million)) I've been running this ai 24/7 for the past 2 months. It will actually beat Yoshi's Island 3 faster than YI 1, due to level differences. Also, please note that Staleness is broken, and I haven't had time to fix it. If anyone finds out what's wrong, let me know. Here's the script file: http://pastebin.com/8zqf6vnG Here's the required "smw-bizhawk.lua" file: http://pastebin.com/ZF0Mbf5j Once I get a few more features up and ready, I'll make public the lua file!
Posted: 10/7/2015 2:03 AM
Quote
Dwood15
Experienced Forum User
Joined: 6/18/2015
Posts: 54
NachtRaben wrote:
Ahh that makes sense. lol. Stats tracking is something I've wanted to see implemented for a while. Just for shits and giggles to know how many times the ai has spent jumping off that cliff at YI3 before realizing to go up. I'll gladly bench stuff you want some longer data on.
Would you be interested if I stripped the whole script of all AI and let people play with the fitness measuring?
Posted: 10/2/2015 11:30 PM
Quote
Dwood15
Experienced Forum User
Joined: 6/18/2015
Posts: 54
NachtRaben wrote:
What you had said earlier about an array of savestates just went right over my head. I haven't messed with much of the code as programming doesn't come as natively to me as it does others. I'd be curious to see how yours works though. I haven't seen YI3 beat in under 100 before.
I just moved to the tilesSeen approach, and it's bringing up its own set of issues, so my approach with this update may not be conducive to the 100 generations which I promised. For example, my ai is having difficulty beating YI 1, as it seems to be getting stuck, not realizing it needs to jump over some walls. hopefully I can start optimizing for the ai so it can start making more effective decisions. If I get the next stage up and running, I will be very happy. :) I should start tracking the stats on this guy! Basically the idea is to switch between levels for each run, since we're gunning for an ai that does well across many levels.
Posted: 10/2/2015 5:09 AM
Quote
Dwood15
Experienced Forum User
Joined: 6/18/2015
Posts: 54
Nevermind, found the bugs with my code. I made some pretty dumb mistakes. Anyway, I've got a good 50% of my goal up and running. It's pretty sick at this point. While mario doesn't beat certain levels more efficiently than he did before, but I can't help but feel like i'm making more progress! Now what's going on with my Ai, I've implemented the tilesSeen which Raben showed me, I've removed fitness from the staleness calculations, the AI now groups the genes in a slightly recalibrated, efficient fashion, i've reworked some of the evolution algorithms, and we log the finalstats for each and every run. I'm looking into a way I can more efficiently calculate some of this stuff, because there are a *lot* of for loops. Many of which I wonder if we can combine for efficiency. This may sacrifice readability, but I think it would be worth it for the efficiency gainz. Before that, I'm going to create a config file to store all the constants, so each time the emulator loads up, we load up the constants from the file. The next high priority is to begin discovering and prioritizing the overcoming of local minima.
Posted: 9/29/2015 12:51 AM
Quote
Dwood15
Experienced Forum User
Joined: 6/18/2015
Posts: 54
NachtRaben wrote:
I got it to beat YI3 twice in under 250 generations, and it runs incredibly fast.
I don't mean to boast, but YI3 should be beatable consistently in under 100 generations. Secondly, have you tried creating an array of save states, and having each ai run through each state before it was done with each run? That decreases overfitting quite a bit. You'll need to track and average the fitnesses for each run, but it should even out over time. My AI has some extra inputs and other changes, which are on the pages previous to this (nothing groundbreaking's changed in the code since the last pastebin i posted, imo, but seriously, your ai should be able to beat YI 3 in under 100 generations. If not, something's amiss.)
I wish there was more forks of MarI/O to test. or even adaptations of MarI/O for something other than winterbunny's MarI/O kart. Everyone who was playing with the concept has stopped streaming or making videos of it so this thread is my last chance to see what kind of cool things people can make out of these concepts.
Well, it seems i've hit a snag in my own code. This may force me to move to c# completely as I'm getting sick of debugging Seth's code, so if I can't get this fixed in the next couple weeks, you may get to look at my own additions anyways. Somewhere along the line my fitness values are being overwritten with nil. I think it has to deal with Lua's strange Scope system, but idk.
Posted: 9/24/2015 1:59 AM
Quote
Dwood15
Experienced Forum User
Joined: 6/18/2015
Posts: 54
Thanks Raben! That's actually a _really_ interesting way of doing it, and I hadn't thought of doing fitness using a hash table. I am ridiculously impressed.
Posted: 9/10/2015 5:00 AM
Quote
Dwood15
Experienced Forum User
Joined: 6/18/2015
Posts: 54
Ok then Tom, my guess is that you have had bad luck with your ai. Start the training over again. You've reached so many generations on one level, the way the culling and propagations work, I don't see any AI that hasn't made progress in 100-200 generations making effective progress in the long run, the algorithm doesn't encourage novelty well. The way the AI works, if a bad seed propagates (and they will) the problems will spread. You're better off restarting the trial runs after 100 generations if you can't get the ai to beat the level. On my project: I've gotten incredibly lazy (!!!) with my methodology. I don't want to modify the C# code in bizhawk without a full dev setup, so my code has lately just been Lua updates. I've setup a bitbucket repository for my Lua code. After a number of rewrites, I'm going to make it public. While I respect Sethbling, the code has been out there for so long and I never signed anything, I'm just going to put my modifications up publicly. The ai is in an interesting spot, because it has no effective novelty search. It has a kind-of-but-not-really novelty search (removeStaleSpecies(), not to be confused with removeWeakSpecies()) but it's not really an effective function besides culling - so there's about 3 rounds of culling which occurs in the code, 1. We just cull a set amount from any given species. 2. We cull all but the top members of the species, and 3rd. We cull all which have the same/similar fitness (removeStaleSpecies()). In the end the pressure for getting the highest fitness is far, far too high. Therefore, what I want to do is divide it up with a nearestNeighbor() search, where we create some arbitrary numbering system, and begin weighing ai against that for their breedability - The ai species which are far apart (but within the same 'species') get to breed together, while those that are too close will not get to 'breed' - in this way, we encourage novelty. On the other hand, then, we cull the species which consistently underperform. So, an average species fitness and a more specific per-organism novelty. In this case, novelty is the ai's ability to make progress where the other ai has failed. So ai that doesn't get stuck at a wall, while only walking through the level, can propagate vs a running one that can't figure out to jump over a ledge. By default the culling for the 3rd one without a real novelty search does _not_ take into account as many variables as they should. Secondly, the first two methods inadvertently remove species (albeit accidentally) which may or may not have traits we want. Sadly, even with these changes, due to the requirement of some randomness in the generation of the ai, one run could have God-status ai, while the next run could have trash ai that just memorizes the level.
Posted: 9/3/2015 4:42 AM
Quote
Dwood15
Experienced Forum User
Joined: 6/18/2015
Posts: 54
1. Make your image smaller tom 2. Have you modified the script at all? My Ai has been able to beat that level consistently.
Posted: 9/1/2015 5:10 AM
Quote
Dwood15
Experienced Forum User
Joined: 6/18/2015
Posts: 54
Hey Toolchild, so I took a look at biasCell in context of the whole script, and from what I can tell, biasCell doesn't actually cover any of the inputs into the neural network. Instead of that, biasCell is actually just something used for rendering. I didn't look up too much of it, however, because for me the network visualization is basically useless. In fact, it's on my list of things to rewrite, as I hate the way SethBling coded the display and the sprite readings. My own code is progressing smoothly, albeit slowly, hoping to have a proper c# implementation by the end of the week. If not, it should be next week. Secondly, i've realized something about fitness - that there's a flaw in the fitness function in that it considers time as a portion of the fitness. While it's a nice thought in theory, it takes too much hand tuning to get right, so i've removed time from the fitness function. Thirdly, one (small) way to speed lua up is to use multiplication of decimal values instead of division where you can. Over the course of thousands of iterations, m-plying can cost a noticeably less number of iterations than dividing touse. Fourthly, the real issues holding the ai back the most, toolchild, is definitely the fitness function. If you want to figure out how to help the ai succeed, you'll want to revisit fitness and add your own variables from ram from places like smw-central or the smw bizhawk lua script linked earlier in the thread.
Posted: 8/22/2015 4:55 PM
Quote
Dwood15
Experienced Forum User
Joined: 6/18/2015
Posts: 54
Gotta be honest, I thought this thread died. Anyway, I've been in limbo lately, and haven't had access to a good development computer setup for a while. I still don't but I'm in a situation where I can get working on my multihawk setup. Zeromus: I know it's not supported, but it looks like there's an issue with Multihawk's implementation of bsnes. Forcing snes9x in multihawk allows me to run multiple snes instances. I haven't decided whether I want to use bizhawk or multihawk as my emulator mnger or roll my own, as each one seems to have their own setups I'm going to want to configure, though as things have progressed I'm leaning towards my own control form which launches the emulator instances in a threaded state. toolchild: I'm not sure what you expect to get by removing bias cell, but I wish you the best in getting a smarter AI out of it. As far as the connections in your latest post child, don't worry about the layout of the neurons in the networks too much. The goal is to get the ai to work, not to audit the neurons. Survival of the fittest doesn't care how something gets done, just that it does. Lastly, the SethBling MarI/O ai has a major bug, in that it should allow for recurrent neural networks, but doesn't. It needs to allow full clocks to form. Amaraticando: It looks like there hasn't been much progress on the AI front for a while. I'm at the start of a pretty big change in mario nn's though. The focus after this is going to be on creating a more effective training environment, so mario won't just be memorizing.
Posted: 7/16/2015 8:13 PM
Quote
Dwood15
Experienced Forum User
Joined: 6/18/2015
Posts: 54
Zeromus: Then multihawk isn't successful if it won't run.
Samsara wrote:
Dwood15 wrote:
1 week ago (7/10/2014)
Helluva long week, isn't it?
It really is man. I feel like i've been doing this for a year long.
Posted: 7/16/2015 7:35 PM
Quote
Dwood15
Experienced Forum User
Joined: 6/18/2015
Posts: 54
Unmodified roms: Super Mario World (U) [!].smc or Super Mario World (USA).sfc C:\Users\Dwood\Desktop\Bizhawk Emulator - Copy\Roms Since I copied from the main repo 1 week ago (7/10/2014) it does this. Multihawk has never loaded any other SNES roms.
Posted: 7/16/2015 5:30 PM
Quote
Dwood15
Experienced Forum User
Joined: 6/18/2015
Posts: 54
Line 463, in RomLoader.cs exceptions out: When loading from multihawk. 
((CoreFileProvider)nextComm.CoreFileProvider).SubfileDirectory = Path.GetDirectoryName(path.Replace("|", String.Empty));
Posted: 7/16/2015 9:03 AM
Quote
Dwood15
Experienced Forum User
Joined: 6/18/2015
Posts: 54
I haven't looked into fixing it, but whenever I open a SNES rom in debug mode in Multihawk, I get a null object reference error. I'll edit this post later with more details.
Posted: 7/14/2015 4:21 PM
Quote
Dwood15
Experienced Forum User
Joined: 6/18/2015
Posts: 54
zeromus wrote:
The machine isn't as beast as you think. bsnes is a slow emulator. be sure youve picked the performance core..
That, and he needs to uncheck "clock throttling". That said, even my pc gets a minimum of 120 fps unthrottled, and it's crap.
Posted: 7/13/2015 8:25 PM
Quote
Dwood15
Experienced Forum User
Joined: 6/18/2015
Posts: 54
DarkZero, those are some interesting ideas, but I'm going to stick with C# unless I move to an all-C++ emulator. I think I have located the largest source of the problem we need to solve. As I was working on integrating sharpNEAT into the emulator, I was thinking about what the limitations of our experiments are, performance-wise. You aren't being limited by CPU DarkZero. I think you are being limited by concurrency. How much CPU is bizhawk using per core? What if we could spawn multiple emulators at once and train several Neural Networks at the same time? While I was trying to use sharpNEAT last night I discovered an inherent issue with the library. The evolution algorithms are extremely tightly coupled with the "Domain" library. Suffice it to say, it's (on the surface) going to be simpler for me to have the evolution program create the main emulator form, auto-load the Lua file to manage the fitness, and then kill the instance once fitness of the genome is measured, then load up the next neural ne then fire up another emulator session. This would have the benefit of allowing us to run X number of emulator instances all at once (where X is the number of species or genomes within a generation), and training them all at the same time... In that case we need to kill all other logic that we don't care about, essentially making the emulator a console. Then the only thing we'd need to script for are the fitness functions, which we can do with lua to make sure it works and then port it to c#, where we can strip the emulator to strictly logic. The Following are mostly notes for myself for the future, you guys can critique this if you want: 1) Have a top-level execution layer that packages up a thread with the neural net(s) (given the settings from a config file) and emulator. This way we can run multiple instances at once. a) Cut out all nonessential logic from the emulator. (bye video, bye sound!) 2) Use either C# or Lua to evaluate fitness. Lua for fast prototyping without having to rebuild the emu every time I want to test a fitness function, then once I have a good fitness function, cut out all the Lua and move it to C#. 3) The spawned thread(s) evaluate nets it holds for fitness and returns the fitness to the main thread. I'll look into ways I can integrate bizhawk with another program.
Posted: 7/11/2015 3:08 AM
Quote
Dwood15
Experienced Forum User
Joined: 6/18/2015
Posts: 54
zeromus wrote:
You'll find that theres no good place to actually put c# code right now to control the emulator with. Your best bet would probably be to extend the lua console system to be able to manage a list of c# coroutines activated by name via reflection, in addition to lua scripts, and have the c# coroutines run immediately after the lua coroutines, so that the means by which the main emulator frontend launches the coroutines is uniform.
I've finally taken a peek at the code. I've never messed with reflection in C#, so I'm going to have to read up about that, as I'm not sure what it's for/does. You have a good idea there about adding the C# function I want to run to the Lua coroutines. I'll take a look at it. To be honest, I kind of just want to call a function using C#, extensibility be damned. I am going to do what you said, but just hardcode it and not bother with a plugin/dll system. That said, I'm trying to track down the sections is that you're talking about for the LUA coroutines. So far I haven't seen any code that leads from a main loop. directly. Edit: Well I've decided what I'm going to do. I'm going to implement the NEAT library from sharpNEAT in C# so lua doesn't do that heavy lifting, and make a createNetworks(<params>), evaluateNetworks(<netInputs>) which returns the output neuron states. It would also allow us/me to focus on the fitness function and the inputs/outputs in lua.
Posted: 7/6/2015 10:47 PM
Quote
Dwood15
Experienced Forum User
Joined: 6/18/2015
Posts: 54
zeromus wrote:
no, that's not what he's suggesting. he proposed to make a dll that lua can consume. lua would still be involved, as would the bizhawk to lua bindings.
Ah, well I wouldn't want to do that. I'd rather bypass LUA entirely if I go the managed route. As far as that extra Lua stuff, I definitely wouldn't want to mess with any of that co-routine stuff. I'll take a look at the emulator later this week, I've downloaded the source to my PC, so i'm going to take a peek at what you mean in the code. I'm going to be sad if I couldn't call some function to read a byte directly using the c# code. If it was all loosely coupled and I could call the functions exposed to LUA directly as a normal C# function, that would be ideal, instead of any other stuff.
Posted: 7/6/2015 5:27 AM
Quote
Dwood15
Experienced Forum User
Joined: 6/18/2015
Posts: 54
zeromus wrote:
Well... that's a decent idea... it will be a bit odd due to needing to move a lot of data out of bizhawk, into lua, and from then to the dll, and the opposite: calling bizhawk (savestate, for example) methods curiously by passing delegates to do it out of lua and into c++. But--it might be easier overall than bolting a new controller onto a c++ core and you can do it piecemeal instead of taking the plunge all at once. Certainly worth considering.
From what I can tell, what he's suggesting would make it so we can just bypass lua entirely. I haven't even downloaded bizhawk's source to know how tightly coupled the lua functions bizhawk exposes are between the core and LUA the language. If it's not too tightly coupled, we could make a plugin API so people can write their own scripts using managed .dll's. At any rate, if I were to actually do it, that would be exactly the plan I'd try to do. Just build the whole thing out that way. It may actually be faster since there's about a bajillion NEAT implementations in C#/C++ we could take advantage of instead. None in C# use CUDA/OpenCl, but even C# would offer better memory mgmt than lua. I think I'll do it. I'll download the source and if I have time this week between classes and studying for tests, I'll try to implement at least a basic NEAT implementation using the C# functions that bizhawk exposes to Lua.
1 2 3