Post by Dwood15

Posted: 9/8/2015 12:24 AM

Quote

tom_mai78101

He/Him

Player (127)

Joined: 3/16/2015
Posts: 160

1. I don't know how to make the image smaller. 2. I only edited the part where it draws green and red lines. I haven't had a clue on what to modify in the core parts that determines what to press.

Posted: 9/8/2015 12:31 AM

Quote

Masterjun

He/Him

Site Developer, Expert player (2047)

Joined: 10/12/2010
Posts: 1185
Location: Germany

Just don't embed it! (As in, just post the link!)

Warning: Might glitch to credits I will finish this ACE soon as possible (or will I?)

Posted: 9/10/2015 5:00 AM

Quote

Ok then Tom, my guess is that you have had bad luck with your ai. Start the training over again. You've reached so many generations on one level, the way the culling and propagations work, I don't see any AI that hasn't made progress in 100-200 generations making effective progress in the long run, the algorithm doesn't encourage novelty well. The way the AI works, if a bad seed propagates (and they will) the problems will spread. You're better off restarting the trial runs after 100 generations if you can't get the ai to beat the level. On my project: I've gotten incredibly lazy (!!!) with my methodology. I don't want to modify the C# code in bizhawk without a full dev setup, so my code has lately just been Lua updates. I've setup a bitbucket repository for my Lua code. After a number of rewrites, I'm going to make it public. While I respect Sethbling, the code has been out there for so long and I never signed anything, I'm just going to put my modifications up publicly. The ai is in an interesting spot, because it has no effective novelty search. It has a kind-of-but-not-really novelty search (removeStaleSpecies(), not to be confused with removeWeakSpecies()) but it's not really an effective function besides culling - so there's about 3 rounds of culling which occurs in the code, 1. We just cull a set amount from any given species. 2. We cull all but the top members of the species, and 3rd. We cull all which have the same/similar fitness (removeStaleSpecies()). In the end the pressure for getting the highest fitness is far, far too high. Therefore, what I want to do is divide it up with a nearestNeighbor() search, where we create some arbitrary numbering system, and begin weighing ai against that for their breedability - The ai species which are far apart (but within the same 'species') get to breed together, while those that are too close will not get to 'breed' - in this way, we encourage novelty. On the other hand, then, we cull the species which consistently underperform. So, an average species fitness and a more specific per-organism novelty. In this case, novelty is the ai's ability to make progress where the other ai has failed. So ai that doesn't get stuck at a wall, while only walking through the level, can propagate vs a running one that can't figure out to jump over a ledge. By default the culling for the 3rd one without a real novelty search does _not_ take into account as many variables as they should. Secondly, the first two methods inadvertently remove species (albeit accidentally) which may or may not have traits we want. Sadly, even with these changes, due to the requirement of some randomness in the generation of the ai, one run could have God-status ai, while the next run could have trash ai that just memorizes the level.

Posted: 9/20/2015 8:18 PM

Quote

NachtRaben

He/Him

Joined: 9/20/2015
Posts: 4

Also, you guys should check out the code Firzen14 uses when he played with MarI/O. I've benched this thing for days before and the fitness function works very well. He livestreamed its progress on YI3 here. http://www.twitch.tv/firzen14/v/6378992 and the pastebin to his code here. pastebin.com/wqxNwEjh

Posted: 9/24/2015 1:59 AM

Quote

Dwood15

Joined: 6/18/2015
Posts: 54

Thanks Raben! That's actually a _really_ interesting way of doing it, and I hadn't thought of doing fitness using a hash table. I am ridiculously impressed.

Posted: 9/24/2015 5:16 AM

Quote

NachtRaben

He/Him

Joined: 9/20/2015
Posts: 4

I work with several developers on a daily basis. So far Firzen's is the best code I've come across. Very efficient, I got it to beat YI3 twice in under 250 generations, and it runs incredibly fast. I wish there was more forks of MarI/O to test. or even adaptations of MarI/O for something other than winterbunny's MarI/O kart. Everyone who was playing with the concept has stopped streaming or making videos of it so this thread is my last chance to see what kind of cool things people can make out of these concepts.

Posted: 9/29/2015 12:51 AM

Quote

Dwood15

Joined: 6/18/2015
Posts: 54

NachtRaben wrote:

I got it to beat YI3 twice in under 250 generations, and it runs incredibly fast.

I don't mean to boast, but YI3 should be beatable consistently in under 100 generations. Secondly, have you tried creating an array of save states, and having each ai run through each state before it was done with each run? That decreases overfitting quite a bit. You'll need to track and average the fitnesses for each run, but it should even out over time. My AI has some extra inputs and other changes, which are on the pages previous to this (nothing groundbreaking's changed in the code since the last pastebin i posted, imo, but seriously, your ai should be able to beat YI 3 in under 100 generations. If not, something's amiss.)

I wish there was more forks of MarI/O to test. or even adaptations of MarI/O for something other than winterbunny's MarI/O kart. Everyone who was playing with the concept has stopped streaming or making videos of it so this thread is my last chance to see what kind of cool things people can make out of these concepts.

Well, it seems i've hit a snag in my own code. This may force me to move to c# completely as I'm getting sick of debugging Seth's code, so if I can't get this fixed in the next couple weeks, you may get to look at my own additions anyways. Somewhere along the line my fitness values are being overwritten with nil. I think it has to deal with Lua's strange Scope system, but idk.

Posted: 10/2/2015 5:09 AM

Quote

Dwood15

Joined: 6/18/2015
Posts: 54

Nevermind, found the bugs with my code. I made some pretty dumb mistakes. Anyway, I've got a good 50% of my goal up and running. It's pretty sick at this point. While mario doesn't beat certain levels more efficiently than he did before, but I can't help but feel like i'm making more progress! Now what's going on with my Ai, I've implemented the tilesSeen which Raben showed me, I've removed fitness from the staleness calculations, the AI now groups the genes in a slightly recalibrated, efficient fashion, i've reworked some of the evolution algorithms, and we log the finalstats for each and every run. I'm looking into a way I can more efficiently calculate some of this stuff, because there are a *lot* of for loops. Many of which I wonder if we can combine for efficiency. This may sacrifice readability, but I think it would be worth it for the efficiency gainz. Before that, I'm going to create a config file to store all the constants, so each time the emulator loads up, we load up the constants from the file. The next high priority is to begin discovering and prioritizing the overcoming of local minima.

Posted: 10/2/2015 10:01 PM

Quote

NachtRaben

He/Him

Joined: 9/20/2015
Posts: 4

What you had said earlier about an array of savestates just went right over my head. I haven't messed with much of the code as programming doesn't come as natively to me as it does others. I'd be curious to see how yours works though. I haven't seen YI3 beat in under 100 before.

Posted: 10/2/2015 11:30 PM

Quote

Dwood15

Joined: 6/18/2015
Posts: 54

NachtRaben wrote:

What you had said earlier about an array of savestates just went right over my head. I haven't messed with much of the code as programming doesn't come as natively to me as it does others. I'd be curious to see how yours works though. I haven't seen YI3 beat in under 100 before.

I just moved to the tilesSeen approach, and it's bringing up its own set of issues, so my approach with this update may not be conducive to the 100 generations which I promised. For example, my ai is having difficulty beating YI 1, as it seems to be getting stuck, not realizing it needs to jump over some walls. hopefully I can start optimizing for the ai so it can start making more effective decisions. If I get the next stage up and running, I will be very happy. :) I should start tracking the stats on this guy! Basically the idea is to switch between levels for each run, since we're gunning for an ai that does well across many levels.

Posted: 10/3/2015 10:06 PM

Quote

NachtRaben

He/Him

Joined: 9/20/2015
Posts: 4

Ahh that makes sense. lol. Stats tracking is something I've wanted to see implemented for a while. Just for shits and giggles to know how many times the ai has spent jumping off that cliff at YI3 before realizing to go up. I'll gladly bench stuff you want some longer data on.

Posted: 10/7/2015 2:03 AM

Quote

Dwood15

Joined: 6/18/2015
Posts: 54

NachtRaben wrote:

Ahh that makes sense. lol. Stats tracking is something I've wanted to see implemented for a while. Just for shits and giggles to know how many times the ai has spent jumping off that cliff at YI3 before realizing to go up. I'll gladly bench stuff you want some longer data on.

Would you be interested if I stripped the whole script of all AI and let people play with the fitness measuring?

Posted: 10/21/2015 1:42 AM

Quote

Dwood15

Joined: 6/18/2015
Posts: 54

This is going to be my first time publishing this script in a few months, so please be kind to it! Short update, not that anyone cares: My main dev machine is still not up and running, so a c# port of the ai is still a long ways off. (VS 2010 is the latest version I have installed on this laptop, and VS 2013, which is on my new machine, is sooo nice...) I've briefly been looking into multi-threading in LUA. Thing is, there's a lot of functions even with in the script itself, which could easily be set to a multi-threaded state, but my knowledge on what it takes to thread in LUA is very limited. I may just wait it out- for the time being, anyway. At any rate, I've finally made a change I've meant to do for a very long time, and that's add a config file setup, so that you don't have to open the script file in order to make minor changes. All you need to do is stop the script, hit refresh, and restart it again. With a config system, I can have it run 24/7, log all the data, move it to a run #, and then archive it for later usage. I can also do things like modify the max number of nodes, the mutation rates, et cetera. Since the ai takes a lot of variables to get started, I figure this will be the most effective way of moving with my plans. I'm also going to add a "no-ai" mode, where a player can evaluate themselves against the ai. Please note a few things with this: Please, do not report to me a single bug from a modification you made. I do not have the time nor the desire to fix your problems. Secondly, On the first run, the script will generate a config file. The config setup still hasn't been worked out 100% yet, so I'm leaving it barebones, as-is. Many of the variables even a .01 change can make a large difference, so be careful! (300 is the # of individuals per generation, 255000 is the max number of nodes You can probably change this without too much difference in your AI's fitness, however, you may end up killing your performance and ram if you increase it by a lot. Remember that the number of nodes in each pool is a function of the number of individuals per round - this means you could hit Lua's 32bit memory limit quite easily (i've done it, hence the 255000, not 1 million)) I've been running this ai 24/7 for the past 2 months. It will actually beat Yoshi's Island 3 faster than YI 1, due to level differences. Also, please note that Staleness is broken, and I haven't had time to fix it. If anyone finds out what's wrong, let me know. Here's the script file: http://pastebin.com/8zqf6vnG Here's the required "smw-bizhawk.lua" file: http://pastebin.com/ZF0Mbf5j Once I get a few more features up and ready, I'll make public the lua file!

Posted: 10/24/2015 10:50 AM

Post subject: Mario repo on Github

Quote

spankypants

Joined: 10/24/2015
Posts: 3
Location: Chocolate

Hi, I've made a public code repository on github, ordered by: game/creator/ (Henke's code is orphaned because it is late and I want to go to bed :P) The repository is here: https://github.com/spankypants/MarI-O Its public, so if you want to take your code down, or post it up, you can. If you want your code posted, but dislike Github, I'll post it for you. If you don't want your code posted there at all, please PM me and I'll skip any code you post. There is a wiki page with links to other resources, eg the other variants (Pong, SMK so far) and some other stuff incl other AI's playing games. I'm also going to start on getting other games to work, eg Punchout, Ghouls and Ghosts, or Street Fighter. SP

Posted: 10/29/2015 8:38 AM

Quote

spankypants

Joined: 10/24/2015
Posts: 3
Location: Chocolate

Dwood15, I edited your script to say that it needed the smw-bizhawk file as well, I didn't want to include it in the repo as it might get updated. I'll put in a README file to say that it needs to be downloaded along with your script.

Posted: 10/29/2015 10:57 PM

Quote

Dwood15

Joined: 6/18/2015
Posts: 54

spankypants wrote:

Dwood15, I edited your script to say that it needed the smw-bizhawk file as well, I didn't want to include it in the repo as it might get updated. I'll put in a README file to say that it needs to be downloaded along with your script.

Well, you need is the neatevolve script, which looks like SethBling's AND the smw-bizhawk script, which is different than the one at the guy's repository. The smw-bizhawk.lua script I edited probably won't be changing, and I am beginning to get real tired of lua.

Posted: 10/30/2015 7:44 AM

Quote

spankypants

Joined: 10/24/2015
Posts: 3
Location: Chocolate

righto, I'll put up the three files. Does that mean you are stopping development of *I/O AI's?

Posted: 10/30/2015 12:40 PM

Quote

Dwood15

Joined: 6/18/2015
Posts: 54

spankypants wrote:

righto, I'll put up the three files. Does that mean you are stopping development of *I/O AI's?

No, it's 2 files. Just two. One named neatevolve.lua and the other is smw-bizhawk.lua. and yes, very soon, my MarI/O version will no longer be under development. I've just got a couple of bugs I'd like to fix first, but I'm kind of sick of the lua programming language.

Posted: 11/1/2015 7:01 PM
(Edited: 11/3/2015 10:48 AM)

Quote

vogosvagen

Joined: 11/1/2015
Posts: 2

Hey Dwood15, I found a comment of yours on reddit, and found my self stumbling here. I'm currently going through the source code, and I'm having a bit of trouble understanding what the getTile(dx,dy) function does. From what I've gathered, it determines where mario is (page o, or page 1. As in a level contains 2 pages?), and determines which Tile is active(?). I'm talking about this part (super mario bros. part of the code):

Language: lua
local subx = math.floor((x%256)/16)
		local suby = math.floor((y - 32)/16)
		local addr = 0x500 + page*13*16+suby*16+subx
		
		if suby >= 13 or suby < 0 then
			return 0
		end
		
		if memory.readbyte(addr) ~= 0 then
			return 1
		else
			return 0
		end

is there anyone who documented/commented over the source code?

Posted: 11/2/2015 11:48 PM

Quote

Dwood15

Joined: 6/18/2015
Posts: 54

That code has been completely stripped from my script, but sadly, I'm a simple programmer, not a reverse engineer. If you want more specific information, you're better off looking at how to read memory addresses and assembly. There's websites that have basically the roms of the games completely mapped out, so there is that.

Posted: 11/3/2015 1:46 AM

Quote

Amaraticando

It/Its

Editor, Player (162)

Joined: 1/10/2012
Posts: 673
Location: Brazil

I cannot answer much for SMB, but it should be similar to SMW. getTile(dx, dy) gets the 16x16 square-tile around Mario. The tiles are stored in the memory according to the x, y positions, according to some rule. ~~The offset should start at WRAM $0500 and go on, according to some rule, mapping all the level.~~ NES games have really small WRAM space, so it should store only nearby tiles (not the entire level, like in SMW).

Language: lua
local x = marioX + dx + 8  --> gets the relevant X, that is around Mario
local y = marioY + dy - 16  --> relevant Y
local page = math.floor(x/256)%2  --> x//256 is the screen number, so page is the screen parity

local subx = math.floor((x%256)/16)  --> horizontal tile (each 16 pixels)
local suby = math.floor((y - 32)/16)  --> vertical tile (each 16 pixels)
local addr = 0x500 + page*13*16+suby*16+subx  --> bijection between the pair (subx, suby) and the address it is stored, with offset $0500

if suby >= 13 or suby < 0 then --> ignore tiles far away
    return 0
end

if memory.readbyte(addr) ~= 0 then  --> the memory is non-zero iff there's a tile there. It treats all different tiles equally.
    return 1
else
    return 0
end

Edit: typos / HTML code screwing.

Posted: 11/9/2015 1:52 PM

Quote

vogosvagen

Joined: 11/1/2015
Posts: 2

Thank you both, I won't say I understand how it works 100% but what you said definitely helped. I'm trying to understand the algorithm now. Taking it nice and slow.

Posted: 2/22/2016 11:45 AM

Quote

skalarfeld

He/Him

Joined: 2/12/2016
Posts: 2
Location: Germany

Is there anyone still actively playing with this thing? I'd love to improve the fitness function; maybe even alter it to create 'flavours', i.e. speed optimized, score optimized, and 'most human like'. However, I lack the emulator knowlegde to make MarI/O run efficiently. Earlier in this thread I read about disabling graphical output but the author is no longer active here. I only have an old Core 2 Duo machine and I'm not able to achieve more than 80 fps. In any case, having to train MarI/O for 24h after every change in the fitness functions is too inefficient... anybody that could help? Furthermore, I read about breaking with lua and transferring the algorithm to C++. Maybe that's a nice long-term goal.

Posted: 2/22/2016 3:06 PM

Quote

Amaraticando

It/Its

Editor, Player (162)

Joined: 1/10/2012
Posts: 673
Location: Brazil

> Earlier in this thread I read about disabling graphical output but the author is no longer active here. BizHawk offers this option in the Display menu. > In any case, having to train MarI/O for 24h after every change in the fitness functions is too inefficient... anybody that could help? Maybe, using Snes9x 1.51 could be better. The emulation is like 10 times faster and, except for heavy glitch abuse that the bot will neve pull off, the movie can be translated into BizHawk/lsnes format without desyncs.

Forum BizHawk MarI/O thread

Forum

BizHawk

MarI/O thread