hahahaha that's funny. nope! Still lots of work to be done. The network is heavily limited by its inputs and the fitness function as well as the variables used for training it. TAS-ers don't have these limitations. That is, unless their tools are bad.
My ai after running it for 2 days non-stop couldn't even beat the level it was on in a consistent manner. Then again, i'm sure if it had all the inputs the ai could be as good, if not better. Just don't expect it to overflow the stack to create minigames for itself.
If someone wants to add the snes9x core to bizhawk, I'd be down to use that. If you want to craft a neural net bot in c++ for max performance, you'll need a C/C++ based Open Source emulator. There's: higan, which is GPLv3, my cmd prompt isn't (ugh) reading the PATH variable to run make, so it's being a pain in general. If you get higan to build let me know if you had to do anything special.
Askye:
I think we can get by using C# instead of Lua by using the C# interface of the scripts. It would probably also make bizhawk more extensible if we could use .NET languages to create scripts instead of Lua by making our own plugin interface. This way, we could write a .dll in C++ or C# that uses functions we expose. It gives us way more control and better debugging if we can debug our plugins and scripts using Visual Studio. C# will be faster than LUA, but it also could be used to create an API that C++ implementations of any of these languages. If we go with C# there's a library that allows us to program like we're using CUDA but build to OpenCL or CUDA depending on the targeted gfx card called cudafy. That would be nice for me because i'm running AMD in my PC, which obviously doesn't support CUDA.
Also, using Visual Studio to create these allows us to navigate other's code so much faster and easier.
I'm also thinking that we create a more abstract set of functions for getting the neural net's inputs so we can train the network using different mario games easier, by pre-defining the memory space in the inputs we use. On another note, if there's a NEAT/Neural network where someone's done the legwork for us, we should use it. There's es-hyperneat in c#, website, based on: sharpneat. These places kind of just throw software to the wild without documentation, however.(lol)
I'm going to take a look at these different NEAT implementations. My real question is: what does it mean to train a network on static inputs/outputs, and how is that different from a standard network?
Askye: I was actually thinking about doing the same thing where we dump the state every so often and save the inputs and the outputs, and 'train' the network based on those inputs and outputs. I think that would be a faster and a smarter way of doing things.
It would also allow us to use OpenCL and take advantage of gfx processing. This would make the training 10x+ faster I'm sure.
At that point, however, I wonder if we would be better off training a neural network on images instead? We could prt scr every 5th frame and dump the inputs, then train the network based on that.
Then, we could use our RAM addresses to further evaluate fitness functions.
And yeah, I've restarted every time i've made a major fitness formula adjustment. It's a pain, but when the formula is bad, Mario has a tendency to oscillate and not progress in the level. So i've made sure to add a fitness = -1 if the game timer runs out. :P
That's... really clever askye. Mind sharing the exact details on how you did that? I'm interested in squeezing more fps out of my emulator, since it is using up 90% of my cpu (it's kind of a cheapo cpu)
On another note, i'm now forcing Lua to garbage collect every time initializeRun is called, using the generic collectgarbage(). It has dropped the working memory from 400 average to 300 MB. Considering that it's ballooned out of control but not any more, I'm very hopeful I can just leave this ai running over the next 2-3 days and have it putting out some good results.
Next up there's going to be a few problems I need to fix or add:
1. Fitness function is completely borked, is less than half as useful as it should be, so I need to rethink what failure and success means for the ai. This is despite my fitness function and timeout functions being 10x better than the default version, imo.
Along with that
2. Need to account for Mario in the overworld. what i've got so far is much, much better than any other person has for fitness in SMW, that I am 100% certain. On the other hand, It still has some painful points where the second Mario enters teh overworld, it's jacked up. I've got to get all the values (x, y, etc) for the overworld
3. Missing sprites. I have all the sprite info for the basic characters and the extended sprites, but I don't think I have any for the bosses or any castle specific sprites.
4. Various other inputs: help boxes, save menus, yoshi, better/more fine-grained enemy sensing, etc etc, slope detection, timer, and the direction, mario is facing, as well as velocity are all inputs the neural net needs.
Here's an open invitation, however: To any person that wants to see MarI/O in another game, if you get me the sufficient documentation of a game and its ram addresses I will port MarI/O over to that game so you can watch the ai play, or even play with it if it's a Multiplayer game of some kind.
Ah. Well I don't know how emulators work so it's good to know. I'll just stick with what it's at now then, and add a check to make sure cpu on the emu isn't being throttled.
I still think you're confused. If you set the speed, you're doing it wrong. Setting the speed sets the throttle speed. Disable the throttle, and the speed you set it to is irrelevant.
All right, I removed the throttling. (that gave a significant performance boost, nice!)
I just nopped the whole frameadvance on quickNES and got 27000fps.
What do you mean? Like if i run my script without frameadvance in lua to move the gameforward, how am I supposed to tell the emu my script is done, and pick up where it left off?
zeromus, this script is heavily dependent on the speed of the emulator, because it uses frameadvance and not some other method. He's definitely hit the speedup setting cap at 6400%, because he can't set the speed faster than that. I have hit that as well.
Askye, if you get a custom build going to remove the cap and adding other 'headless' features, i'd be down on working with you on it, as I'm interested in running the simulations much faster.
Also askye, if you want to REALLY speed it up, remove all console.writeline calls, and all gui graphics drawing calls. I have been unable to ascertain whether or not the script is too slow at 6400% or what.
Also you could look into modding the rom to nop out all the draw calls used by the rom.This may reduce the computational load by a lot, but I'm not sure how much.
Thanks for the notice on the client.speedmode() function, I'm going to use that from now on. I've been running at the default 200%, but it's too slow :P.
BWAHAHAHAA I found my issue. Apparently I accidentally removed "removeWeakSpecies()" from my script. It's back now and the script works better. I'm also going to try forcing a garbage collect every time there's a new generation. Maybe that will cut down on memory usage in general. With removeWeakSpecies() back, it seems to hover around 600 MB and then drop to 500 or so. I'll leave it running and see if it keeps going higher and higher. If the memory balloons out of control I'll be sad.
I've also found that for a starting generation we should specify an equivalent number to the number of inputs at least.
What would be REALLY interesting to me is if we had a computer vision script running which would then output to the various controls instead of this specialized set of inputs. That way our game playing neural net would be able to be trained on other games in a more generic manner as well.
We fixed memory leaks in lua at some point. Probably after 1.9.4
1.9.4? I'm still on 1.1.0 and the download links are still from the 15th.
Edit: oh, i see. (1.10.0 ~= 1.1.0)
Yeah, it's definitely the script. After the first generation it seems to spam making generations for a period of time. that has to be fixed.
EDIT x 3+: really, really, really, strange bug. The script is adding text into the script:
This is an example backup.state.pool file. All those words in it, you know? Those are supposed to be numbers. Somehow those variables are being set to those words.
WTF? how is it that this is happening?
I suspect that's the problem in the last code I've posted as well, but just in case it's not, here: http://pastebin.com/eCBRbcMT PLEASE HELP! I have only the slightest clue on how to debug, and I'm running low on time to get this in a working state!
On that note, I think it may be best to change from x.xRates["SOMESTRING"] to x.xRates.SOMESTRING....
Edit: changing them to other names doesn't work. I cannot even find those names in the script as strings any more and it's still doing that. wtf? This occurs only after the first (0) generation, so set the population to something low like 10 and you'll see what i mean. then check for the backup state files.
Ok, so it's not the text in the script. huh. Well i'm going to have to do a line-by-line comparison i guess....
Trust me, if you want to do him a courtesy, go immediately kick his script's ass with your own script and write 'based on sethbling's cool work' at the top of it
Thanks for the motivation! Out of respect, until I get a reply it's pastebin for public use of my version of the script. Then again, if he doesn't reply for too long I won't bother.
I'm working on kicking his script's ass for sure, haha. I've made some recent kick-ass changes too. Now, Mario has a vague sense of his surroundings.
And one more thing I 'spose. So i've increased the number of inputs, now mario has 3500+ inputs. This, this is a problem, because I've hit Lua's memory limit. What does this mean? This mean that Bizhawk(?) is not supplying the amount of RAM that the script needs to run and produce a reliable output. It seems that the libsneshawk is at 114MB. The first generation runs fine, but then when it requests more memory it throws an error.
If someone who knows how to fix this could help me out i'd appreciate it. Then again, my script might be having a memory leak... Actually, I may have caused it to leak. dammit. These kinds of things are really hard to track down. Maybe I should just build my own bizhawk and debug that if that's easier. iirc it runs out of memory on a new generation.
Edit: yep bizhawk is climbing from 200 MB to 450+ loool.
why are you still using pastebin anyway, someone make a github already
I have pm'ed SethBling to ask permission to set up a public github with no response on Reddit. Until I get permission for doing so, if I do set one up (and I get that I should) it will be private and I'll continue with pastebin. If Seth ever makes a github, I will proudly fork it and add my changes to his.
Secondly, I have tried using github, and I hate using it. Almost as much as I hate using lua.
All right! I've made some progress! Mario now can tell the difference between every single normal sprite! Along with that, he now knows the vague direction a sprite is. Despite this, however, we still have one major issue: Mario is getting stuck at walls and cannot seem to figure out he needs to jump around them.
This script is 100% incompatible with your other pools. I recommend starting the script on a brand new pool. Also some changes: Timer changes every time mario changes a screen. Along with that, i've reduced default population size by 250 so there is more generational advancement.
There is also a strange bug where mario advances in generations of 16. Not sure what's going on there.
You'll see mario jump and do crazy
smw-bizhawk(required by the script below)mari/o, has a few bugs, see edit for a fix
edit: just discovered a bug, go to line 203 in the second pastebin and change: inputs[inputs] to inputs[#inputs]
Also, change population to 150 or more, that seems to fix the creation of 16 unnecessary generarions each time we hit the end of the species.
To see the distance from exit for example you could make a script to read the sprite data from the level that gives you the x position of the goal tape, and make a counter that decreases as Mario is going right (if only the x distance matters, if you want the real distance is the diagonal to the exit). I ~guess~ it could work.
Doesn't mario's x-coordinate change based on the 'room' he's in? So let's say there are two exits in a level, like in many portions of the game, or mario has to go through a door or pipe to get to that particular exit. How would I account for that?
Is there a simple way to get what room a door connects to <from> ?
Double posts probably aren't allowed, but I just wanted to share with you guys something I've done, i've figured out how we need to add a sprite to the Input array, but i'm afraid to inform you that it will come at a massive cost to performance without some changes. We need to modify the sigmoid function in order for the performance not to kill us, though i'm not sure how we should do it to get what we want. Basically from what I can tell is that the sigmoid function takes the activation of a neuron, * the 'weight' of that neuron and then averages that out to be between -1 or 1 no matter what.
That, or I'll have to increase the size of the input array from 169 inputs to 14,656 inputs + 169 + any extra inputs we want to add. This would decrease performance by... a factor of 86 times.
That's only getting the 6 by 6 tile range that mario can 'see' already. http://www.smwiki.net/wiki/Sprite
function getSprites()
local sprites = {}
for slot= 0, SMW.sprite_max - 1 do
local status = memory.readbyte(0x14C8+slot)
if status > 7 then -- 228 + 5320
spritex = memory.readbyte(0xE4+slot) + memory.readbyte(0x14E0+slot)*256 --5344
spritey = memory.readbyte(0xD8+slot) + memory.readbyte(0x14D4+slot)*256
spriteNo = u8(WRAM.sprite_number + slot)
sprites[#sprites+1] = {["number"]= spriteNo, ["x"]=spritex, ["y"]=spritey}
end
end
return sprites
end
This would allow him to interact better with the world, and to interact directly with these sprites.
The sigmoid function is here:
function sigmoid(x)
return 2/(1+math.exp(-4.9*x))-1
end
I'm not sure if what I want to do is possible though, because afaict a neuron is either firing or it isn't.
EDIT: I'm wrong, in that the number of input neurons does not have to be that much higher. I can convert the hex values into a binary array where each digit represents a neuron. Then, I can be sure that the section of memory actually turns out to be only 8 extra inputs (0 - FF)
Quick question team, is there a way to read the level files from memory or ram? I'd like to get things like the distance from exits, etc.
Not sure what you mean, but we can:
1) Once inside a level, read from WRAM the content of each tile and the position of the goal tape;
2) Outside a level, only reading from the ROM, as far as I know.
If I only care about inside the level how would I do # 1 to get the location of a tile that's not on-screen. (say, the goal-tape or a key and keyhole)?
I'm thinking I would search the whole ram memory for the id of the sprite I want and just assume that's what it is?
No support for SMB, only SMW, so all the filename checks are gone.
Path for where the files should go.
Expose mario's speed and a few more state details as inputs
Expose the actual id of the tiles and objects
MOD EDIT: Pastebinned lengthy code segment. -Mothrayas
I tried running it and it gave me errors. I tried fixing it but my lua skills are very lacking.
His code isn't fixable. You're better off trying to read it and get his intentions than to fix it. While my code isn't much better, my pastebin from the last page is actually runnable, so use that as a basis. Also note that the additions to the inputs don't actually work, so you're better off just using the base script.
Do note that the fittness criteria needs to be rather fine grained. Counting rooms won't do.
Yeah I've been keeping that in mind. I just realized I've been doing my inputs all wrong. A neuron's value should be firing or not firing, and therefore on my inputs, they need to be either -1 or 1, and the network should decide the weights on its own, therefore I need to find a way to set this whole thing up so that all the numbers in the game are -1 or 1, which clarifies things as I try to update the network.
The more I look at this code, the more I realize there's limitations, if you will, to when a certain action should be taken. Secondly, this also means that mario is uncapable of differentiating between what is an enemy, friend, or powerup. All it knows is that there is something nearby.
Therefore, this mario ai cannot, by its nature, beat a level without memorizing it. There are a couple of things here which makes things difficult for us then. In order to get an accurate sense of the game to the ai, we need better values. We need to have sprite types passed to mario. We also need to change it from mario's x and y to a vector distance between the closest point on mario's collision box to the enemy or to the other sprites' collision boxes, and have x number of neurons firing when an enemy is close, varying by distance from mario to the other sprite.
The getInputs() function is where the magic happens, and I am ashamed of myself for not seeing this sooner than I have. I had even watched a video describing this - neurons are either firing or they aren't. There's nothing else to it.
Then the inputs from there are passed in and the controller outputs are spat right back out. It's way simple, and I guess I got lost by lua's loop form.
On that note, this is a week from Hell for me. We'll see if I can get anything added to it, but I'm not sure. If someone could help me out, I would love to have a reliable way of getting the exact locations each object on the map is going to be. not enemies, but pow, end level, keyholes, keys, et cetera.
If there was a way to get that location on the map, I would be severely impressed.
OR if you want these neural nets in other games (legend of zelda, super smash, advance wars, et cetera), i'd love to see if we can get interesting things from this script.
I'm presently engaged in a long-running process to evaluate squirrel as a possible replacement. Doesn't help you any directly, but reading about a language designed by people who hated the same things about lua as you do might help you feel better while venting.
Thanks Zeromus, if you ever started with it, I could help out with the implementations of it, as I also understand a bit of C# and having worked on an emulator would be good for my resume. I'll look into squirrel later as well, but for now I gotta add stuff to the script!
As a side note I am also working in my off time on a compiler for a lisp-styled scripting language. It's not nearly as featureful as lua but it's a fun project. :)
Back on topic. That code above has become this:
local level = {} --used for object creation, don't modify
level.__index = level --used for object creation, don't modify
function level.new(init) --used for object creation, don't modify
local self = setmetatable({}, level)
self.lvlID = init[1]
self.Room_index = init[2]
self.x = init[3]
self.y = init[4]
return self
end
function level.set_value(self, newval) -- used for object manipulation
self.lvlID = newval.lvlID
self.x = newval.x
self.y = newval.y
end
function getVScreenCount() -- zero-indexed
return u8(WRAM.vscreen_number)
end
function getHScreenCount()
return u8(WRAM.hscreen_number) -1
end
function getCurrentVScreen()
s8(WRAM.y + 1)
end
function getCurrentHScreen()
s8(WRAM.x + 1)
end
function getCurrentRoom()
return bit.lshift(u8(WRAM.room_index), 16) + bit.lshift(u8(WRAM.room_index + 1), 8) + u8(WRAM.room_index + 2)
end
local current_level = level.new({getCurrentRoom(), getCurrentVScreen(), getCurrentHScreen()})
i'm shamelessly stealing (thanks henke!) from the smw project from earlier. One thing i'm having trouble with though, is the size of the script file, so i'm going to try and separate it between the neural network and the main file with the main, executed bits. I hope I don't slow down LUA too much if I have a getMemoryValue() function like above.
Yeah, it's an issue with the way it's been programmed. SethBling uses the game screenspace (for showing it off, presumably) whereas we should be using the emulator's screen space, imo.
At any rate, have I mentioned that i hate lua enough? I want types and structs back. This programming language sucks. I'll take a language that requires an interpreter sure, but hot damn how they handle everything is sooo unintuitive.
I'm almost apt to write my own scripting interpreter for the emulator, though it wouldn't be so fully featured. Who uses 1-based indexing by default?
And going over the lua manuals:
With those two functions, it is straightforward to implement
Could they be any more pretentious?
For example, i'm trying to migrate fitness from a right-based movement approach to a discovery-based focus, thus using the rooms and current level to indicate fitness, so that the ai is more apt to explore.
So, to you guys out there that know more about lua than me. Am I doing this right so far?
local levelsPlayed = { local LevelIdx = nil, local RoomsVisited = { x = nil, y = nil} }
local currentLevelIndex = 1
function isOldLevelAddtoLevel(lvl, lvlsPlayed)
for i, #lvlsPlayed do
if lvlsPlayed[i].LevelIndex == lvl.LevelIndex then return true end
end
lvlsPlayed[#lvlsPlayed + 1] = lvl
return false
end
-- rooms are an x and y value sooo
function isNewRoom(testRoom)
local currentLevelPlayed = lvlsPlayed[currentLevelIndex]
for i, currentLevelPlayed.#RoomsVisited
if testRoom.x == currentLevelPlayed[i].x and testRoom.y == currentLevelPlayed[i].y then
return false
else
currentLevelPlayed[i] = testRoom
return true
end
end
end
Thanks for the input. I've been using Notepad++, but without being able to map out variables and such easily i'm kind of stuck doing it the hard way. Oh well.
This is partly why I hate lua, without at least a 'var' before a variable being required (yes, i know about local) it can be hard to trace the flow of a program by eyeballing it.
Do we know how to get that kind of stuff? What about deaths on-death event?
Edit: i've been playing with the script a bit, so here's a pastebin:
http://pastebin.com/Xvr5LQEP
This is still really crummy, but have a look at it. Please read the top notes before using it.
As it stands, you need a DP1, DP2, and DP3.state to run this or it will error out on you. Along with that, don't click the "show map" box, there's a bug and idk how to track that one down.
Edit: if someone could find something to show level victory, that would be pretty sweet.