Edit: adelikat informs me that the slowdown was due to the backup feature in FCEUX. This can be disabled through config -> enable -> backup savestates. That should render the script below unnecessary, but perhaps someone will still find a use for it. Oh well.
Edit 2: After some quick testing, it seems that backup savestates are not related to the slowdown.
* * *
I'm working on a bot for a game that, as yet, shall go unmentioned. The game is rather long (about 400,000 frames or two hours) and this has caused problems.
Whenever I use savestate.load (which is often, because it's a bot) it creates a new copy of the movie file. I believe this is because Lua cannot know if I have a savestate on the branch I'm leaving, so it preserves the old movie file in case savestate.load needs it at a later time. As the movie grew in size, I was losing about a second every time I opened an old state. This was unacceptable, as my bot took some eight hours to produce 250,000 frames. My goal was to create "quicksave" and "quickload" functions that instead of saving a branch, just goes back and rewrites over the movie file.
I seem to have accomplished just that, although not as smoothly as possible. Here is my Lua script, called savetools.lua:
Language: lua
dofile("savetable.lua")
function table.copy(t)
local t2 = {}
for k,v in pairs(t) do
t2[k] = v
end
return t2
end
savetools={}
function savetools.quickcreate()
state={}
state[1]=savestate.create()
state[2]=0
return state
end
function savetools.quicksave(state)
savestate.save(state[1])
state[2]=frameindex
return state
end
function savetools.quickload(state)
savestate.load(state[1])
frameindex=state[2]
end
function savetools.create()
state={}
state[1]=savestate.create()
state[2]=0
state[3]={}
return state
end
function savetools.save(state)
savestate.save(state[1])
state[2]=frameindex
state[3]=table.copy(movievector)
return state
end
function savetools.load(state)
savestate.load(state[1])
frameindex=state[2]
movievector=table.copy(state[3])
end
function savetools.frameadvance(buttons)
if not(buttons) then
buttons={}
end
joypad.set(1,buttons)
movievector[frameindex]=buttons
emu.frameadvance()
frameindex=frameindex+1
end
--[[ --movie.record doesn't work
function savetools.record(filename,startpoint,players,options)
movie.record(filename,startpoint,players,options)
for i=0,#movievector do
joypad.set(1,movievector[i])
emu.frameadvance()
end
end
]]--
function savetools.savemovie(filename)
assert( table.save( movievector, filename ) == 1 )
end
function savetools.openmovie(filename)
movievector = table.load( filename )
return movievector
end
function savetools.playmovie(filename)
movievector = savetools.openmovie(filename)
for i=0,#movievector do
joypad.set(1,movievector[i])
emu.frameadvance()
end
end
frameindex=0
movievector={}
There are nine notable functions in this script:
•
savetools.create,
savetools.save, and
savetools.load all work almost exactly the same way that the savestate.create, savestate.save, and savestate.load do. The only difference is that savetools.save returns the state, which has been updated, so the syntax is something like state1=savetools.save(state1) instead of the old savestate.save(state1). Someone with a little more experience with Lua can probably rewrite it to match the old syntax. I've tried to make it so that you can just do a search for "savestate" and replace all instances with "savetools" and it will (almost) still work.
•
savetools.quickcreate,
savetools.quicksave, and
savetools.quickload all work like their non-quick counterparts, except that they just backtrack and rewrite over the old movie file rather than saving it. This saves a
lot of time with intensive botting because the movie file doesn't need to be copied. The state you quickload should be strictly in the past, however. I don't know exactly what would happen if you quickload a state on an alternate branch, but I'm certain that you'd end up producing garbage.
•
savetools.frameadvance works as a combination of joypad.set and emu.frameadvance. With no input arguments, it just advances a frame without pressing any buttons. It also takes an argument of the same format as the tables accepted by joypad.set. For example, savetools.frameadvance({A=1}) is equivalent to joypad.set({A=1}) emu.frameadvance().
•
savetools.savemovie saves the movie to file. Note that the movie format is different from .fm2. I'll explain that in a second, but for now I'd just like to reassure you that you can fairly easily convert a movie of this format to .fm2 format.
•
savetools.playmovie plays the movie from the current point.
•(
savetools.openmovie is used by savetools.playmovie, but it otherwise isn't too useful. You can use it to read the movie file as a table directly, should you want to. Perhaps that can be useful as a crude form of hex editing...)
There are just two important variables in the script: frameindex and movievector. The frameindex variable should always match the frame that the emulator is currently on while movievector is just a vector of button presses from the start of recording. Also of note is the dofile("savetable.lua") line at the top of the script. That's a script I downloaded from
here and allows you to save and load Lua tables to file. In this case, it saves the movie file. I'll include a copy of that script shortly.
To implement this file, include the line dofile("savetools.lua") (or whatever you call it) at the top of your bot.
To create a .fm2 movie file, pause the game, power on, and run the bot, being sure to include the line savetools.savemovie("mymovie.tbl") (or whatever your movie's name) at the end of your script. Then write a second script that only executes savetools.playmovie("mymovie.tbl"). Pause the game, record a new movie, and then run the new script. The .tbl movie should execute flawlessly, producing a .fm2 movie. If movie.record worked, you would be able to do this all internally without writing a second script.
The script could perhaps use a little bit of improvement. For one, I should probably make it so that it deletes (rather than just overwrites) the button presses in the old movie file. Second, I could probably make it so that for the save and load functions, it only records button presses after the save.
This script should be extremely easy to port to other emulators. I would also like it to act directly on the native movie file. If any emulator developers would like to implement these features directly, I'd be quite honored.
Here is the savetable.lua script:
Language: lua
--[[
Save Table to File/Stringtable
Load Table from File/Stringtable
v 0.94
Lua 5.1 compatible
Userdata and indices of these are not saved
Functions are saved via string.dump, so make sure it has no upvalues
References are saved
----------------------------------------------------
table.save( table [, filename] )
Saves a table so it can be called via the table.load function again
table must a object of type 'table'
filename is optional, and may be a string representing a filename or true/1
table.save( table )
on success: returns a string representing the table (stringtable)
(uses a string as buffer, ideal for smaller tables)
table.save( table, true or 1 )
on success: returns a string representing the table (stringtable)
(uses io.tmpfile() as buffer, ideal for bigger tables)
table.save( table, "filename" )
on success: returns 1
(saves the table to file "filename")
on failure: returns as second argument an error msg
----------------------------------------------------
table.load( filename or stringtable )
Loads a table that has been saved via the table.save function
on success: returns a previously saved table
on failure: returns as second argument an error msg
----------------------------------------------------
chillcode, http://lua-users.org/wiki/SaveTableToFile
Licensed under the same terms as Lua itself.
]]--
do
-- declare local variables
--// exportstring( string )
--// returns a "Lua" portable version of the string
local function exportstring( s )
s = string.format( "%q",s )
-- to replace
s = string.gsub( s,"\\\n","\\n" )
s = string.gsub( s,"\r","\\r" )
s = string.gsub( s,string.char(26),"\"..string.char(26)..\"" )
return s
end
--// The Save Function
function table.save( tbl,filename )
local charS,charE = " ","\n"
local file,err
-- create a pseudo file that writes to a string and return the string
if not filename then
file = { write = function( self,newstr ) self.str = self.str..newstr end, str = "" }
charS,charE = "",""
-- write table to tmpfile
elseif filename == true or filename == 1 then
charS,charE,file = "","",io.tmpfile()
-- write table to file
-- use io.open here rather than io.output, since in windows when clicking on a file opened with io.output will create an error
else
file,err = io.open( filename, "w" )
if err then return _,err end
end
-- initiate variables for save procedure
local tables,lookup = { tbl },{ [tbl] = 1 }
file:write( "return {"..charE )
for idx,t in ipairs( tables ) do
if filename and filename ~= true and filename ~= 1 then
file:write( "-- Table: {"..idx.."}"..charE )
end
file:write( "{"..charE )
local thandled = {}
for i,v in ipairs( t ) do
thandled[i] = true
-- escape functions and userdata
if type( v ) ~= "userdata" then
-- only handle value
if type( v ) == "table" then
if not lookup[v] then
table.insert( tables, v )
lookup[v] = #tables
end
file:write( charS.."{"..lookup[v].."},"..charE )
elseif type( v ) == "function" then
file:write( charS.."loadstring("..exportstring(string.dump( v )).."),"..charE )
else
local value = ( type( v ) == "string" and exportstring( v ) ) or tostring( v )
file:write( charS..value..","..charE )
end
end
end
for i,v in pairs( t ) do
-- escape functions and userdata
if (not thandled[i]) and type( v ) ~= "userdata" then
-- handle index
if type( i ) == "table" then
if not lookup[i] then
table.insert( tables,i )
lookup[i] = #tables
end
file:write( charS.."[{"..lookup[i].."}]=" )
else
local index = ( type( i ) == "string" and "["..exportstring( i ).."]" ) or string.format( "[%d]",i )
file:write( charS..index.."=" )
end
-- handle value
if type( v ) == "table" then
if not lookup[v] then
table.insert( tables,v )
lookup[v] = #tables
end
file:write( "{"..lookup[v].."},"..charE )
elseif type( v ) == "function" then
file:write( "loadstring("..exportstring(string.dump( v )).."),"..charE )
else
local value = ( type( v ) == "string" and exportstring( v ) ) or tostring( v )
file:write( value..","..charE )
end
end
end
file:write( "},"..charE )
end
file:write( "}" )
-- Return Values
-- return stringtable from string
if not filename then
-- set marker for stringtable
return file.str.."--|"
-- return stringttable from file
elseif filename == true or filename == 1 then
file:seek ( "set" )
-- no need to close file, it gets closed and removed automatically
-- set marker for stringtable
return file:read( "*a" ).."--|"
-- close file and return 1
else
file:close()
return 1
end
end
--// The Load Function
function table.load( sfile )
-- catch marker for stringtable
if string.sub( sfile,-3,-1 ) == "--|" then
tables,err = loadstring( sfile )
else
tables,err = loadfile( sfile )
end
if err then return _,err
end
tables = tables()
for idx = 1,#tables do
local tolinkv,tolinki = {},{}
for i,v in pairs( tables[idx] ) do
if type( v ) == "table" and tables[v[1]] then
table.insert( tolinkv,{ i,tables[v[1]] } )
end
if type( i ) == "table" and tables[i[1]] then
table.insert( tolinki,{ i,tables[i[1]] } )
end
end
-- link values, first due to possible changes of indices
for _,v in ipairs( tolinkv ) do
tables[idx][v[1]] = v[2]
end
-- link indices
for _,v in ipairs( tolinki ) do
tables[idx][v[2]],tables[idx][v[1]] = tables[idx][v[1]],nil
end
end
return tables[1]
end
-- close do
end
And finally, here's a function at the heart of my bots. All it does is execute "action" at different times until it finds the earliest it can do so to obtain the desired results. I'll explain it in more detail later if anyone is interested.
Language: lua
local function guessandcheck(action,condition,framedelay,lowguess,highguess,executeaction)
worstpossible=highguess
local state2=savetools.quickcreate()
local oldstate={}
for i=1,#condition.adr do
oldstate[i]=memory.readbyte(condition.adr[i])
end
local foundit=false
state2=savetools.quicksave(state2)
while not(foundit) do
savetools.quickload(state2)
guess=math.floor((highguess+lowguess)/2)
for i=1,guess do
savetools.frameadvance()
end
for i=1,#action do
savetools.frameadvance(action[i])
end
for i=1,framedelay do
savetools.frameadvance()
end
if condition.fn(unpack(oldstate),unpack(condition)) then
highguess=guess
else
lowguess=guess+1
end
foundit = (highguess==lowguess)
end
savetools.quickload(state2)
for i=1,lowguess do
savetools.frameadvance()
end
if executeaction then
for i=1,#action do
savetools.frameadvance(action[i])
end
end
if highguess==worstpossible then
print("BAD!")
end
--while not(condition["fn"](oldstate,unpack(condition))) do
-- emu.frameadvance()
--end
end
Hope this helps others as much as it's helping me!