Joined: 4/17/2010
Posts: 11475
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
I still don't understand what exactly is wrong in TAS sound plugin? All the difference I hear is clipping here and there.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Try Azure Dreams to see just how bad TAS SPU is. I think the game was not publishable until I used Eternal SPU.
The sound of this game is simply chaotic without Eternal SPU, but since feos seem to be curious... there is worst (warning, I'm not responsible of any seizure or other problem for you accessing to this content) :p
Anyway, I wanted to give a big shot this weekend to make some serious progress and finaly I was able to automate at 95% the whole process for encoding with the double pcsxrr instance trick as explained previously. In the end I got a nice video, without (or almost) any pain.
The only change I made from the initial plan is that the Eternal SPU plugin have to run "from" kkapture, since as I said before, pcsxrr might desync more or less often with kkapture.
Now, while I like the final result, there is still a lot place for improving the script like fixing some bugs and optimizing the "checkpoint list". For info, see the azure dreams checkpoint list. Some frame weren't probably necessary to keep the run in sync and since some loadstate might cause a (very) small audio glitch... I was also thinking about capturing the checkpoint in mute mode to avoid this problem. But I guess I should experiment this later.
So yeah, even though this workflow project need more testing, basicly the script work. All you need to do is customize the settings in the "pcsxrrWorkflowConfig.lua", then run an instance of pcsxrr(I assume you already configured the graphical plugin correctly) with the detectCheckpoint.lua and the TAS sound plugin for processing the screenshot. After the screenshot are made, just leave this instance play in loop with the script.
Then use kkapture(as a dummy or a NUL output for this test) to launch an another instance of pcsxrr with the Eternal SPU and run the synchCheckpoint.lua. If everything is going fine, magic should happen and the checkpoint list will appear in the detectInfoStatus.xml.
Once synchCheckpoint.lua has finished you can close all the pcsxrr instance and launch kkapture for doing a real encode with the synchCheckpoint.lua.
While working on this, make sure there is enough RAM for pcsxrr and space storage for the savestate to avoid any crash. If some crash happen, you can try to rollback a previous savestate by using the config.oldStateLoader(frame) or config.stateLoader(frame) functions and reloading your current script.
Overall, while the workflow might seem a bit complex, this is definitly more conveniant than encoding manually.
Later this month or next month, I'm willing to check if the Umihara Kawase Shun run work fine, even if it desync at the beginning this shouldn't be a problem.
For now, here's PcsxrrEncodeWofkflowV4.08 ! Enjoy.
edith: Still some bugs, I'll post v4.09 soon.
Joined: 4/17/2010
Posts: 11475
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Joined: 4/17/2010
Posts: 11475
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Same error if I move it there. "Module not found"
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Joined: 4/17/2010
Posts: 11475
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Same error if I move all of them to pcsx folder.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Seem your missing some random dll. Try this to unpack this new version instead(the is some unused dll there, but at least you won't get this error anymore)
Once you get the script running, you might have to change some setting in the "pcsxrrWorkflowConfig.lua" depending on you want to do.
If you don't understand anything, you can try to catch me on IRC.
edit: here a new version (I added few more stuff to prevent crash)
So, right now the issue pointed by feors is fixed, so we could try to get the Bushido Blade 2 run working. So far the first pass for getting the hashframe went fine... thought I should add some code to automate some kind of backup :)
Then I decided to go ahead with the second pass(with the double instance) and I stumbled on an another weird problem
Yet, as we can see with the "Process Explorer" program, I have plenty of memory on my system(6gb used of 12gb)
In anycase, I think the pcsxrr memory grow quickly only when it get some heavy desync with a 3D game or maybe pcsxrr didn't like I was multitasking while the movie processed? Anyway, I'll try to find out some way to monitor this either with the garbage collector or some other way around.
It seem that this could be a memory leak issue by pcsxrr. So I guess the patch for 4gb won't help.
I'll try to make a new version for this weekend that should fix some other issues and request suggested by feos on irc.
Sometime, a movie can desync at the very last frame of a movie(this happen in Bushido Blade 2). So we have to check the frame after a movie as well. Also there is still some random stuff that might happen causing crash(don't know what I can do about this) or even a corrupted savestate that desync. I'll have to look at this again and check if I can make a "second level" of rollback rules... which is somewhat annoying.
Usually when a crash happen, I just reload the nearest savestate and I let's the process going.
The request is about making a separate folder for the whole resync tool instead of been in the pcsxrr root folder.
Joined: 4/17/2010
Posts: 11475
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
BadPotato wrote:
It seem that this could be a memory leak issue by pcsxrr. So I guess the patch for 4gb won't help.
I expect the hole to be in the scripts somewhere, but the problem needs some debugging (I'm not the one to do it though).
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Sorry to be a bit slow for this new update. So far, from my previous test with feos, I think at some point, there is no way to prevent pcsxrr from crashing or getting some problem(thought, I still think there something I could do to prevent some weirdness).
When a crash happen for wathever reason, I want the encoder to still been able to recover what he did and restart the process at the nearest checkpoint. To do this you can manually change some value in the script, but this isn't very userfriendly.
Then I was thinking about doing some hud, but I believe that pcsxrr have enough stuff to handle on his UI. Then I found wxLua and I ran some sample code and it seem to work(edit: actually I only tested vclua).
Basicly, I would make a "small pannel control" for each script/instance. So the encoder can control his workflow, load a particuliar savestate, show progress of hashframe or the status of the xml, etc.
So yeah, this means more experimentation... but I believe this is the best way to do it.
Joined: 4/17/2010
Posts: 11475
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Do you anywhere kill the savestates you create during the process of finding the best savestate frame that fixes desync? The first instance seems to roll back every 11 frames or so at desync points, and check if the new savestate (being farther back) fixes desync. It creates new savestates during this process, but once the earlier one gets created, you do not need to store the later ones (as they don't fix desync). I don't know if they are saved to disc or kept in memory, if the latter, it's most likely the memory leak. However, I don't see any way to kill the savestate in pcsx lua.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Yes, when a desync happen I make sure the desync happen last at least 11-15 frame. Then I ask for the first instance(detection script) to make a savestate at the desync point.
The way I handle the savestate is by using the normal "slot 1-2-3-4-5-6-7-8-9" of a movie. So they're wrote on disc. I usualy use only the slot 1 (that use the .000 extension instead of the .001).
creating a new savestate(used only the dectection script, so it should alway be "safe")
local saveStateFile = io.open(pathSaveState .. "-" .. checkPointToLoad, "rb")
local save = saveStateFile:read("*all")
local loadStateFile = io.open (movieSaveStateFolder .. "/" .. movieName .. ".000", "wb")
loadStateFile:write(save)
io.close(loadStateFile)
local newCheckPoint = savestate.create(1)
savestate.load(newCheckPoint)
savestate.save(newCheckPoint) --save at slot1
So far this trick alway worked. But as we experienced, at some point pcsx will restore a corrupted savestate and then I believe this when pcsxrr begin to get some memory leak issue. There is no reason why the emulator would use suddently that much of memory anyway.
Though, I just noticed that the last line of "loading savestate" could maybe cause some kind of deadlock issue : "savestate.save(newCheckPoint)". So maybe this is the problem? I'll check this out.
Also, I know that might sound like a lazy fix, but for the particuliar case where you might get an desync at the very last frame(happened in bushido blade)... I'll probably just make a checkbox in the control pannel that will automatically load the last frame of the movie. I think this will work just fine, that way. The reason for this, is once a movie is stopped, you can't reload a movie in lua and the only way to fix that would be manually read the pxm and press manually the input, which pretty much means to reinventing the wheels of the TAS tools :)
By the way, my current test for making an additional control pannel with the lua graphics libs was actually just a bad idea. It's gonna be a lot less of trouble to just make a quick and easy C# program and handle communication by XML. So yeah, next update should be *at least* a lot more convenient for the user.
Joined: 4/17/2010
Posts: 11475
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
I believe only the first non-movie frame is needed to be saved and checked, it's the state the game gets right after the movie, so it is going to be correct no matter what. No need to copy the last frame input, it will occur anyway.
Also, some basic Cx program that just presses buttons the way a human would for both instances will definitely save tons of time and labor. This way you won't need to invent insane techs to exchange data through lua addons, the program would store all already. .kkapture system may give some tips as it runs instances from the outside.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Also, some basic Cx program that just presses buttons the way a human would for both instances will definitely save tons of time and labor. This way you won't need to invent insane techs to exchange data through lua addons, the program would store all already. .kkapture system may give some tips as it runs instances from the outside.
There exist some library/program that allow you to do some UI automation with C#(like autohotkey), so I believe this kind of design is possible. I might try to look more at this once everything work.
That said, I don't think there is an easy way to externally grab some data from the lua script, since it would require a lot more of reverse-engineering overhead. The work around would be to get my hand directly with pcsxrr source to make some ugly hack, but I kinda don't feel like this is a good choice. So yeah, I still prefer to rely on something like XML for getting the job done.
So far, the pannel might look like something like this. Hopefully, I'll be able to work more on this weekend.
Joined: 4/17/2010
Posts: 11475
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
pcsx commandline allows to run it with a named script already loaded.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
pcsx commandline allows to run it with a named script already loaded.
Good point, also at some point I alway thought that we couldn't load lua script but I guess I was wrong this feature is there:
-lua <filename>
Automatically open and use the specified Lua script file. Example: pcsx.exe -lua "tools/Penny Racers.lua"
I tried to mess a bit with this and despite the movie.name() function that doesn't work(probably because pcsxrr will alway load the lua script before starting/update the current movie info), I'm sure we should be able to get some good result.
My new plan would be something like this:
-The user start the WorkflowPannel app
-The user enter all the setting you want with the path for .kkapture, pcsxrr and the movie
-Start a "dummy" instance of pcsxrr so the user can change the plugin to the eternal plugin
-.kkapture start
-The user enter all the info to start 1 instance(sync script) of pcsxrr in kkapture
-1 instance(sync script) of pcsxrr start, but *suspend* the process(pending for the detection script)
-Start a "dummy" instance of pcsxrr so the user can change the plugin to the TAS sound plugin
-1 instance(detection script) of pcsxrr start and begin to store the savestate while hashing the frame
-Once the first pass of the detection process end. Restart the pcsxrr(detectionScript) in "patroling mode" and keep looping with savestate.
-Resume the process of pcsxrr(sync script)
-The script try find all the desync point while the user can "monitor" with the WorkflowPannel app in case of problem.
-Once over, prompt the user
-Start a "dummy" instance of pcsxrr so the user can change the plugin to SPU eternal plugin
-The user restart kkapture with the sync script in "encode mode".
-When finished, close pcsxrr, reencode the .avi and upload@youtube
As you can see, the annoying part left of this is the fact that you still have to manualy enter the info to kkapture, but more important manually changing the plugin is a pretty bad issue.
I wonder if there is a file or something where pcsxrr keep the "profile information" about his current plugin. If we can change the profile without asking the user to click the menu, I'm sure this would make the workflow a bit less heavy.
Joined: 4/17/2010
Posts: 11475
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
This is the best thing to actually hack into the emulator - plugin switch through command line or so.
Also, I'm not sure kkapture is really needed to launch the emulator through it every time. This must be tested.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
