Edit: adelikat informs me that the slowdown was due to the backup feature in FCEUX. This can be disabled through config -> enable -> backup savestates. That should render the script below unnecessary, but perhaps someone will still find a use for it. Oh well.
Edit 2: After some quick testing, it seems that backup savestates are not related to the slowdown.
* * *
I'm working on a bot for a game that, as yet, shall go unmentioned. The game is rather long (about 400,000 frames or two hours) and this has caused problems.
Whenever I use savestate.load (which is often, because it's a bot) it creates a new copy of the movie file. I believe this is because Lua cannot know if I have a savestate on the branch I'm leaving, so it preserves the old movie file in case savestate.load needs it at a later time. As the movie grew in size, I was losing about a second every time I opened an old state. This was unacceptable, as my bot took some eight hours to produce 250,000 frames. My goal was to create "quicksave" and "quickload" functions that instead of saving a branch, just goes back and rewrites over the movie file.
I seem to have accomplished just that, although not as smoothly as possible. Here is my Lua script, called savetools.lua:
Language: lua
dofile("savetable.lua")
function table.copy(t)
local t2 = {}
for k,v in pairs(t) do
t2[k] = v
end
return t2
end
savetools={}
function savetools.quickcreate()
state={}
state[1]=savestate.create()
state[2]=0
return state
end
function savetools.quicksave(state)
savestate.save(state[1])
state[2]=frameindex
return state
end
function savetools.quickload(state)
savestate.load(state[1])
frameindex=state[2]
end
function savetools.create()
state={}
state[1]=savestate.create()
state[2]=0
state[3]={}
return state
end
function savetools.save(state)
savestate.save(state[1])
state[2]=frameindex
state[3]=table.copy(movievector)
return state
end
function savetools.load(state)
savestate.load(state[1])
frameindex=state[2]
movievector=table.copy(state[3])
end
function savetools.frameadvance(buttons)
if not(buttons) then
buttons={}
end
joypad.set(1,buttons)
movievector[frameindex]=buttons
emu.frameadvance()
frameindex=frameindex+1
end
function savetools.savemovie(filename)
assert( table.save( movievector, filename ) == 1 )
end
function savetools.openmovie(filename)
movievector = table.load( filename )
return movievector
end
function savetools.playmovie(filename)
movievector = savetools.openmovie(filename)
for i=0,#movievector do
joypad.set(1,movievector[i])
emu.frameadvance()
end
end
frameindex=0
movievector={}
There are nine notable functions in this script:
•
savetools.create,
savetools.save, and
savetools.load all work almost exactly the same way that the savestate.create, savestate.save, and savestate.load do. The only difference is that savetools.save returns the state, which has been updated, so the syntax is something like state1=savetools.save(state1) instead of the old savestate.save(state1). Someone with a little more experience with Lua can probably rewrite it to match the old syntax. I've tried to make it so that you can just do a search for "savestate" and replace all instances with "savetools" and it will (almost) still work.
•
savetools.quickcreate,
savetools.quicksave, and
savetools.quickload all work like their non-quick counterparts, except that they just backtrack and rewrite over the old movie file rather than saving it. This saves a
lot of time with intensive botting because the movie file doesn't need to be copied. The state you quickload should be strictly in the past, however. I don't know exactly what would happen if you quickload a state on an alternate branch, but I'm certain that you'd end up producing garbage.
•
savetools.frameadvance works as a combination of joypad.set and emu.frameadvance. With no input arguments, it just advances a frame without pressing any buttons. It also takes an argument of the same format as the tables accepted by joypad.set. For example, savetools.frameadvance({A=1}) is equivalent to joypad.set({A=1}) emu.frameadvance().
•
savetools.savemovie saves the movie to file. Note that the movie format is different from .fm2. I'll explain that in a second, but for now I'd just like to reassure you that you can fairly easily convert a movie of this format to .fm2 format.
•
savetools.playmovie plays the movie from the current point.
•(
savetools.openmovie is used by savetools.playmovie, but it otherwise isn't too useful. You can use it to read the movie file as a table directly, should you want to. Perhaps that can be useful as a crude form of hex editing...)
There are just two important variables in the script: frameindex and movievector. The frameindex variable should always match the frame that the emulator is currently on while movievector is just a vector of button presses from the start of recording. Also of note is the dofile("savetable.lua") line at the top of the script. That's a script I downloaded from
here and allows you to save and load Lua tables to file. In this case, it saves the movie file. I'll include a copy of that script shortly.
To implement this file, include the line dofile("savetools.lua") (or whatever you call it) at the top of your bot.
To create a .fm2 movie file, pause the game, power on, and run the bot, being sure to include the line savetools.savemovie("mymovie.tbl") (or whatever your movie's name) at the end of your script. Then write a second script that only executes savetools.playmovie("mymovie.tbl"). Pause the game, record a new movie, and then run the new script. The .tbl movie should execute flawlessly, producing a .fm2 movie. If movie.record worked, you would be able to do this all internally without writing a second script.
The script could perhaps use a little bit of improvement. For one, I should probably make it so that it deletes (rather than just overwrites) the button presses in the old movie file. Second, I could probably make it so that for the save and load functions, it only records button presses after the save.
This script should be extremely easy to port to other emulators. I would also like it to act directly on the native movie file. If any emulator developers would like to implement these features directly, I'd be quite honored.
Here is the savetable.lua script:
Language: lua
do
local function exportstring( s )
s = string.format( "%q",s )
s = string.gsub( s,"\\\n","\\n" )
s = string.gsub( s,"\r","\\r" )
s = string.gsub( s,string.char(26),"\"..string.char(26)..\"" )
return s
end
function table.save( tbl,filename )
local charS,charE = " ","\n"
local file,err
if not filename then
file = { write = function( self,newstr ) self.str = self.str..newstr end, str = "" }
charS,charE = "",""
elseif filename == true or filename == 1 then
charS,charE,file = "","",io.tmpfile()
else
file,err = io.open( filename, "w" )
if err then return _,err end
end
local tables,lookup = { tbl },{ [tbl] = 1 }
file:write( "return {"..charE )
for idx,t in ipairs( tables ) do
if filename and filename ~= true and filename ~= 1 then
file:write( "-- Table: {"..idx.."}"..charE )
end
file:write( "{"..charE )
local thandled = {}
for i,v in ipairs( t ) do
thandled[i] = true
if type( v ) ~= "userdata" then
if type( v ) == "table" then
if not lookup[v] then
table.insert( tables, v )
lookup[v] = #tables
end
file:write( charS.."{"..lookup[v].."},"..charE )
elseif type( v ) == "function" then
file:write( charS.."loadstring("..exportstring(string.dump( v )).."),"..charE )
else
local value = ( type( v ) == "string" and exportstring( v ) ) or tostring( v )
file:write( charS..value..","..charE )
end
end
end
for i,v in pairs( t ) do
if (not thandled[i]) and type( v ) ~= "userdata" then
if type( i ) == "table" then
if not lookup[i] then
table.insert( tables,i )
lookup[i] = #tables
end
file:write( charS.."[{"..lookup[i].."}]=" )
else
local index = ( type( i ) == "string" and "["..exportstring( i ).."]" ) or string.format( "[%d]",i )
file:write( charS..index.."=" )
end
if type( v ) == "table" then
if not lookup[v] then
table.insert( tables,v )
lookup[v] = #tables
end
file:write( "{"..lookup[v].."},"..charE )
elseif type( v ) == "function" then
file:write( "loadstring("..exportstring(string.dump( v )).."),"..charE )
else
local value = ( type( v ) == "string" and exportstring( v ) ) or tostring( v )
file:write( value..","..charE )
end
end
end
file:write( "},"..charE )
end
file:write( "}" )
if not filename then
return file.str.."--|"
elseif filename == true or filename == 1 then
file:seek ( "set" )
return file:read( "*a" ).."--|"
else
file:close()
return 1
end
end
function table.load( sfile )
if string.sub( sfile,-3,-1 ) == "--|" then
tables,err = loadstring( sfile )
else
tables,err = loadfile( sfile )
end
if err then return _,err
end
tables = tables()
for idx = 1,#tables do
local tolinkv,tolinki = {},{}
for i,v in pairs( tables[idx] ) do
if type( v ) == "table" and tables[v[1]] then
table.insert( tolinkv,{ i,tables[v[1]] } )
end
if type( i ) == "table" and tables[i[1]] then
table.insert( tolinki,{ i,tables[i[1]] } )
end
end
for _,v in ipairs( tolinkv ) do
tables[idx][v[1]] = v[2]
end
for _,v in ipairs( tolinki ) do
tables[idx][v[2]],tables[idx][v[1]] = tables[idx][v[1]],nil
end
end
return tables[1]
end
end
And finally, here's a function at the heart of my bots. All it does is execute "action" at different times until it finds the earliest it can do so to obtain the desired results. I'll explain it in more detail later if anyone is interested.
Language: lua
local function guessandcheck(action,condition,framedelay,lowguess,highguess,executeaction)
worstpossible=highguess
local state2=savetools.quickcreate()
local oldstate={}
for i=1,#condition.adr do
oldstate[i]=memory.readbyte(condition.adr[i])
end
local foundit=false
state2=savetools.quicksave(state2)
while not(foundit) do
savetools.quickload(state2)
guess=math.floor((highguess+lowguess)/2)
for i=1,guess do
savetools.frameadvance()
end
for i=1,#action do
savetools.frameadvance(action[i])
end
for i=1,framedelay do
savetools.frameadvance()
end
if condition.fn(unpack(oldstate),unpack(condition)) then
highguess=guess
else
lowguess=guess+1
end
foundit = (highguess==lowguess)
end
savetools.quickload(state2)
for i=1,lowguess do
savetools.frameadvance()
end
if executeaction then
for i=1,#action do
savetools.frameadvance(action[i])
end
end
if highguess==worstpossible then
print("BAD!")
end
end
Hope this helps others as much as it's helping me!