Post subject: copying memory card files outside of the game
RachelB
She/Her
Player (129)
Joined: 12/3/2011
Posts: 1579
Would this be allowed? The benefit is that when overwriting a save file the game gives an extra prompt asking if you want to overwrite it. Coping the memory card right after formatting it, and then copying it back after each save would avoid this. This is useful where save warps are used, and once a file is loaded it's never needed again. Since memory cards can be removed and files can be copied using freely available methods, this could even easily be done during a real time speedrun.
Post subject: Re: copying memory card files outside of the game
Banned User
Joined: 3/10/2004
Posts: 7698
Location: Finland
RachelB wrote:
The benefit is that when overwriting a save file the game gives an extra prompt asking if you want to overwrite it. Coping the memory card right after formatting it, and then copying it back after each save would avoid this. This is useful where save warps are used, and once a file is loaded it's never needed again. Since memory cards can be removed and files can be copied using freely available methods, this could even easily be done during a real time speedrun.
Could you explain what you are thinking here in a bit more detail? I'm not quite sure I understand what you are trying to achieve here.
Tub
Joined: 6/25/2005
Posts: 1377
So you want to a) load a savegame b) remove the memory card from the console c) restore the memory card to a previous version (with no save file) d) reinsert the memory card ? What happens to the real console when you remove the SD card while a game is running? Does the game or operating system notice? Is there some OS overhead for handling the changes that would need to be emulated? How is this going to affect recording and replaying? When loading a savestate, the emulator has to ensure that not only the SD card contents are in the correct state, but also that any backups are present and in the correct state as well. Can this be done in a safe and robust way without noticeably slowing down quicksaves/quickloads? And without introducing incompatibilities between emulator versions for savestate- or movie files? How is this going to affect movie verification? Allowing arbitrary writing of SD contents in a movie file is obviously a no-go, since you could just manipulate your savegames without anyone noticing. But even a simple API that only allows you to "save SD", "restore SD" or "format SD" is going to be problematic if someone calls them during saving or loading. Are there technical ways to prevent invalid movies from being submitted? Next up is console verification. NESBot's grand grand grandchildren are going to have trouble replaying a movie that does fancy things with a memory card. All in all, if it's just about skipping a confirmation dialog, I very much doubt that it's worth all the trouble. A hackish solution for a single game won't do, and a robust and accurate general-purpose implementation is a lot of effort, both on the emulator and the site's backend, and it'd need more valid use cases before I can consider it a good idea.
m00
Former player
Joined: 6/30/2010
Posts: 1107
Location: Zurich, Switzerland
Tub wrote:
Next up is console verification. NESBot's grand grand grandchildren are going to have trouble replaying a movie that does fancy things with a memory card.
I thought it won't be possible to emulate disc-based consoles anyway, because of the variable loading times. How do you emulate differences from a scratched disc?
Current project: Gex 3 any% Paused: Gex 64 any% There are no N64 emulators. Just SM64 emulators with hacky support for all the other games.
RachelB
She/Her
Player (129)
Joined: 12/3/2011
Posts: 1579
So you want to a) load a savegame b) remove the memory card from the console c) restore the memory card to a previous version (with no save file) d) reinsert the memory card ?
Yes, that's right.
What happens to the real console when you remove the SD card while a game is running?
Nothing, other than being impossible to save or load without one.
How is this going to affect recording and replaying? When loading a savestate, the emulator has to ensure that not only the SD card contents are in the correct state, but also that any backups are present and in the correct state as well. Can this be done in a safe and robust way without noticeably slowing down quicksaves/quickloads? And without introducing incompatibilities between emulator versions for savestate- or movie files?
Well it will obviously require a change to the movie file format, but otherwise settings the flags to copy/restore a memory card should be the only thing that needs to be done.
How is this going to affect movie verification? Allowing arbitrary writing of SD contents in a movie file is obviously a no-go, since you could just manipulate your savegames without anyone noticing. But even a simple API that only allows you to "save SD", "restore SD" or "format SD" is going to be problematic if someone calls them during saving or loading.
I don't see why it would. I don't plan to save the memory card to the movie file, just set a flag to copy it, and have the emulator handle the rest. Source code will of course be available, so it would be easy to prove it doesn't do anything more than that.
Are there technical ways to prevent invalid movies from being submitted?
No, but that's already the case. If someone doesn't bother to make sure their movie actually syncs, there's nothing we can do about that.
All in all, if it's just about skipping a confirmation dialog, I very much doubt that it's worth all the trouble.
Well it will add up to about 4 seconds in my case. I have spent more time than this would take to code to save far less time than that.
Warepire
He/Him
Editor
Joined: 3/2/2010
Posts: 2178
Location: A little to the left of nowhere (Sweden)
If the system allows hot-swapping memory cards, perhaps it should be allowed, but it should be considered a memory card swap and not a "clear memory card" operation. Which means you would need enough empty memory card files for each swap to be possible. Example: Let game save to Memory Card A, before another save occurs where you will be asked to overwrite etc, swap to Memory Card B, let the game save... and so on. If the system needs to be powered off to remove the memory card or has an internal hard drive which it stores saves on, this must not be possible, as it would violate the representation of the actual system.
Tub
Joined: 6/25/2005
Posts: 1377
RachelB wrote:
Are there technical ways to prevent invalid movies from being submitted?
No, but that's already the case. If someone doesn't bother to make sure their movie actually syncs, there's nothing we can do about that.
I'm sorry, I didn't mean "invalid", but "cheated". Imagine me submitting a movie that
  • plays the game normally
  • saves
  • uses memcard alterations in the middle of the saving process to create a broken savegame, like the chrono trigger run does with resetting
  • later reloads from that save under the pretence of save-warping
What I've been doing to the SD card is certainly cheating. Altering the SD card in between frames is impossible on a real console (no matter what kind of robot you use to pull and reinsert the cards), and I don't think the state of the SD card is clearly defined if it's pulled while written to. Assuming that the effects of the save manipulation are subtle enough that they're not immediately apparent to the casual viewer, how would you detect it?
m00
RachelB
She/Her
Player (129)
Joined: 12/3/2011
Posts: 1579
Warepire wrote:
If the system allows hot-swapping memory cards, perhaps it should be allowed, but it should be considered a memory card swap and not a "clear memory card" operation. Which means you would need enough empty memory card files for each swap to be possible. Example: Let game save to Memory Card A, before another save occurs where you will be asked to overwrite etc, swap to Memory Card B, let the game save... and so on. If the system needs to be powered off to remove the memory card or has an internal hard drive which it stores saves on, this must not be possible, as it would violate the representation of the actual system.
The system in question is gamecube. The memory cards can be removed at any time. The process to do this on a real console would look something like: 1) start the game with an unformatted memory card 2) format the memory card and create save data 2) remove the memory card 3) put the memory card into another console, and copy the (blank) save file to another memory card 4) continue playing, save, reset, and reload 5) remove the memory card, and put it back into the other console. 6) delete the save file, and copy back the blank save. 7) put the memory card back into the original console. 8) repeat step 4-7 as necessary
Altering the SD card in between frames is impossible on a real console (no matter what kind of robot you use to pull and reinsert the cards), and I don't think the state of the SD card is clearly defined if it's pulled while written to.
Well, it could be removed in the middle of a frame on a real console. What i wish to do would not allow that. I believe there is a ~2 second delay between inserting the memory card the game recognizing it anyway, so this would be impossible.
Warepire
He/Him
Editor
Joined: 3/2/2010
Posts: 2178
Location: A little to the left of nowhere (Sweden)
That's what I meant with "clear memory card", this is going to become a massive mess that no one will enjoy verifying. It's a lot more clever to use a swap memory card type of function that ejects the current memory card and puts in a new one. It would just become another input like disc changing. This method also becomes easily verifiable by playing back the movie, if it needs predefined saves that the game does not create during playback the movie will just desync for everyone.
Active player (309)
Joined: 8/21/2012
Posts: 429
Location: France
I'm ok with the idea (I don't think it's "cheating" or anything), but it can introduce some problems. The first one is for the viewers, and replaying the movie. Submitting a TAS that uses that kind of technique needs extra instructions to be able to watch it: "do this and that with the memory card at that time, and here too, etc...", unless of course if it's part of the movie file and the emulator code. The other "problem" can create some debates, depending on the use of it. Say, if the saves, and the "external" manipulations of the memory card (with another console) are done too fast. Sorry if I have a hard time explaining it, I'll just give an example ^^ You could potentially see comments on the run that look like this: "there is no way to remove a memory card and copy the save (or do anything specific with the memory cards) on another console that fast. Real hardware can't do that (as quickly)." I think that people would end up using it without thinking about the "real" minimum time it takes to do some memory card management with another console.
RachelB
She/Her
Player (129)
Joined: 12/3/2011
Posts: 1579
The first one is for the viewers, and replaying the movie. Submitting a TAS that uses that kind of technique needs extra instructions to be able to watch it: "do this and that with the memory card at that time, and here too, etc...", unless of course if it's part of the movie file and the emulator code.
I will code support for it into dolphin if it's allowed. Everything would be automatic.
You could potentially see comments on the run that look like this: "there is no way to remove a memory card and copy the save (or do anything specific with the memory cards) on another console that fast. Real hardware can't do that (as quickly)." I think that people would end up using it without thinking about the "real" minimum time it takes to do some memory card management with another console.
It only takes a few seconds on real hardware, and i believe all of the save warps i intend to do are 10+ minutes apart.
Warepire
He/Him
Editor
Joined: 3/2/2010
Posts: 2178
Location: A little to the left of nowhere (Sweden)
May I ask why my suggested solution goes ignored in the discussion?
RachelB
She/Her
Player (129)
Joined: 12/3/2011
Posts: 1579
Because a new memory card would need to be formatted, which takes far longer than just going through the extra prompt.
Warepire
He/Him
Editor
Joined: 3/2/2010
Posts: 2178
Location: A little to the left of nowhere (Sweden)
RachelB wrote:
Because a new memory card would need to be formatted, which takes far longer than just going through the extra prompt.
And it's not possible to pretend that they are all pre-formatted? I cannot say I am too keen on accepting altering the memory card's state through the movie. As others have said it will become a very big hassle to verify the validity of the run, also in my opinion it is making things really fuzzy regarding what should be considered input to the game. When input is starting to depend on hypothetical secondary consoles it doesn't really feel like valid input anymore to me.
Skilled player (1741)
Joined: 9/17/2009
Posts: 4981
Location: ̶C̶a̶n̶a̶d̶a̶ "Kanatah"
Is there any use of this trick other than skipping a minor confirmation box? Also, let me get this straight: You start recording in Dolphin just like how everyone else does for a TAS; with a blank memory card and from power on. You then TAS all the way to a save point. You then switch the memory card with another blank, new memory card, and save into that. This is then repeated everytime you want to save, as to prevent the "Are you sure you want to overwrite this file?" message from appearing. This ends up saving ~4 seconds in your TAS. Did I get everything correct? If so, I think that's fine, since in the TAS, you still start from a new card (as in, no save file), and in a TAS, it's also under the assumption that you can swap a card without the physical limitation of time. If that's not what you mean, then nevermind.
RachelB
She/Her
Player (129)
Joined: 12/3/2011
Posts: 1579
I cannot say I am too keen on accepting altering the memory card's state through the movie. As others have said it will become a very big hassle to verify the validity of the run, also in my opinion it is making things really fuzzy regarding what should be considered input to the game. When input is starting to depend on hypothetical secondary consoles it doesn't really feel like valid input anymore to me.
Why? It's not input to the game, but neither is swapping discs, or inserting/removing memory cards, or hard resets.
As others have said it will become a very big hassle to verify the validity of the run
All it will take to verify is looking over a <50 line patch, compiling it, and checking sync. It would be easy to see any potential for cheating in the source code. If no one is willing to do that, then anyone with commit rights for an emulator could easily get away all kinds of cheating, so i don't see how it's unreasonable to expect source code (especially such a small amount) be checked over. jlun2, not quite. The new memory card i would switch to will not be empty. It will contain a save file which was created at the start of the movie.
Skilled player (1741)
Joined: 9/17/2009
Posts: 4981
Location: ̶C̶a̶n̶a̶d̶a̶ "Kanatah"
RachelB wrote:
jlun2, not quite. The new memory card i would switch to will not be empty. It will contain a save file which was created at the start of the movie.
Does the movie input file include the time it took to create the new save file at the start of the movie?
RachelB
She/Her
Player (129)
Joined: 12/3/2011
Posts: 1579
jlun2 wrote:
RachelB wrote:
jlun2, not quite. The new memory card i would switch to will not be empty. It will contain a save file which was created at the start of the movie.
Does the movie input file include the time it took to create the new save file at the start of the movie?
Yes, of course, just not the time needed to copy the save file.
Skilled player (1741)
Joined: 9/17/2009
Posts: 4981
Location: ̶C̶a̶n̶a̶d̶a̶ "Kanatah"
RachelB wrote:
jlun2 wrote:
RachelB wrote:
jlun2, not quite. The new memory card i would switch to will not be empty. It will contain a save file which was created at the start of the movie.
Does the movie input file include the time it took to create the new save file at the start of the movie?
Yes, of course, just not the time needed to copy the save file.
Copy as in literally pressing "ctrl+c" and "ctrl+v" on the memory card file on your computer? If not, can you please youtube it to hopefully clear off some confusion?
RachelB
She/Her
Player (129)
Joined: 12/3/2011
Posts: 1579
jlun2 wrote:
Copy as in literally pressing "ctrl+c" and "ctrl+v" on the memory card file on your computer?
Basically, yes.
Warepire
He/Him
Editor
Joined: 3/2/2010
Posts: 2178
Location: A little to the left of nowhere (Sweden)
RachelB wrote:
I cannot say I am too keen on accepting altering the memory card's state through the movie. As others have said it will become a very big hassle to verify the validity of the run, also in my opinion it is making things really fuzzy regarding what should be considered input to the game. When input is starting to depend on hypothetical secondary consoles it doesn't really feel like valid input anymore to me.
Why? It's not input to the game, but neither is swapping discs, or inserting/removing memory cards, or hard resets.
Ok, I probably expressed myself a bit fuzzy and I am sorry for that, but my main point is in my last sentence, when "input" (as in commands / actions the console will accept) becomes dependent on theoretical secondary hardware like a second GameCube used to copy the data around on the memory cards as the first one is playing a game, this is where I think it goes a little too far in the lengths of how input shall be produced.
RachelB
She/Her
Player (129)
Joined: 12/3/2011
Posts: 1579
Warepire wrote:
RachelB wrote:
I cannot say I am too keen on accepting altering the memory card's state through the movie. As others have said it will become a very big hassle to verify the validity of the run, also in my opinion it is making things really fuzzy regarding what should be considered input to the game. When input is starting to depend on hypothetical secondary consoles it doesn't really feel like valid input anymore to me.
Why? It's not input to the game, but neither is swapping discs, or inserting/removing memory cards, or hard resets.
Ok, I probably expressed myself a bit fuzzy and I am sorry for that, but my main point is in my last sentence, when "input" (as in commands / actions the console will accept) becomes dependent on theoretical secondary hardware like a second GameCube used to copy the data around on the memory cards as the first one is playing a game, this is where I think it goes a little too far in the lengths of how input shall be produced.
What exactly is the destinction between this, and say, using a GBA to connect to a gamecube (for tingle tuner in zelda ww, etc)? Surely we would allow that?
Skilled player (1741)
Joined: 9/17/2009
Posts: 4981
Location: ̶C̶a̶n̶a̶d̶a̶ "Kanatah"
I guess that's alright. Let's wait for a judge to decide.
Warepire
He/Him
Editor
Joined: 3/2/2010
Posts: 2178
Location: A little to the left of nowhere (Sweden)
RachelB wrote:
Warepire wrote:
RachelB wrote:
I cannot say I am too keen on accepting altering the memory card's state through the movie. As others have said it will become a very big hassle to verify the validity of the run, also in my opinion it is making things really fuzzy regarding what should be considered input to the game. When input is starting to depend on hypothetical secondary consoles it doesn't really feel like valid input anymore to me.
Why? It's not input to the game, but neither is swapping discs, or inserting/removing memory cards, or hard resets.
Ok, I probably expressed myself a bit fuzzy and I am sorry for that, but my main point is in my last sentence, when "input" (as in commands / actions the console will accept) becomes dependent on theoretical secondary hardware like a second GameCube used to copy the data around on the memory cards as the first one is playing a game, this is where I think it goes a little too far in the lengths of how input shall be produced.
What exactly is the destinction between this, and say, using a GBA to connect to a gamecube (for tingle tuner in zelda ww, etc)? Surely we would allow that?
I will try again to make it clearer. Do note that I am distinguishing between hypothetical consoles and actual ones, I am not against connecting a GBA to the GameCube/Wii to use the tingle tuner in Wind Waker, or linking two GBCs to have a joint Oracle of Ages/Seasons TAS. Because here the GBA is another controller, and the linked up GBCs communicate between each other like 2 computers on a LAN. But we're discussing, as far as I understand it, that the movie will contain commands that behind the scenes will move around save data between memory cards. This is not a legal GameCube operation since it cannot be booted into it's memory card manager, and run the game at the same time. So we are left with a hypothetical second GameCube that exists only within the movie file. And that is what I think is a problem with the legitimacy of the movie. If I am understanding your idea entirely wrong, please make it clearer for me.
RachelB
She/Her
Player (129)
Joined: 12/3/2011
Posts: 1579
Hmm, i see. That is a fair argument.