Post subject: Editor for .dtm files?
Editor, Experienced player (570)
Joined: 11/8/2010
Posts: 4035
Days of work on my Paper Mario: The Thousand-Year Door movie may be lost because my movie desynced by one frame. Is there a program to edit .dtm files? TAS Movie Editor and Notepad++ don't seem to read the format and I don't know what else to try. I would really appreciate any help!
Skilled player (1737)
Joined: 9/17/2009
Posts: 4979
Location: ̶C̶a̶n̶a̶d̶a̶ "Kanatah"
Maybe a hex editor might work. I used HxD to edit my Syobon Action TAS many times. Edit: Link :)
Editor, Experienced player (570)
Joined: 11/8/2010
Posts: 4035
OK, I've been working with HxD, and I compared a backup copy of my .dtm before it desynced to my longer, desyncing movie file. There is a difference of about 7000 frames between the two, yet the backup copy of my movie file is a few lines longer than the desyncing movie file. How is that possible? EDIT: OK, I found it was some problem with the movie file. I tried loading several different savestates from before my movie ended and recording from there, but it would always either desync in one spot or randomly end the movie in another spot. I even tried making a new movie and loading a savestate from my current movie. Well, I'm guessing my .dtm is corrupted, so I'll just start all over with a new one. I do think we need an editor for .dtm files though (eventually).
Joined: 7/22/2009
Posts: 128
Location: Gatineau, Quebec, Canada
Hey there, So I was searching for DTM file editor, I fell on this thread and downloaded HxD editor. Also, the files seems to fit in HxD as well just fine. There is only one problem... The program decodes the file in hexadecimals and bites format instead of something readable for a human being like me. All the letters mixed with numbers are really confusing, and I can't really read what's going on, just by looking at the file. Is there a way to make it a bit more readable?
Taming Dolphin, one frame at a time
arflech
He/Him
Joined: 5/3/2008
Posts: 1120
HxD (as all hex editors) represents every byte as a pair of hexadecimal (base-16) digits: 0-9 and then A-F The ability to convert between decimal and hexadecimal, between hexadecimal and ISO-8859-1 or UCS-2, and between hexadecimal and binary is important for different purposes while using a hex editor, and for the first and last of these the Windows Calculator is useful; for the other one, Character Map may have some use... hex-dec: numerical values for things like lives, health, and score hex-text: text strings like a character's name hex-binary: "bit flags" in "bit vectors" that encode certain conditions, like whether certain levels were passed or certain special pieces of equipment obtained; these also are used in most input files, like in FCM and FM2 where each frame of input for one controller is represented by one byte, with each button corresponding to whether one of the bits is 1 or 0 Most hex editors will show you the hex value of each byte in one section and with the ISO-8859-1 value in the next section, with some default character chosen if the particular byte corresponds to a non-printable character. BTW I've heard good stuff about XVI32, a portable app that I think is included in the TASTools distribution.
i imgur com/QiCaaH8 png
Editor, Experienced player (570)
Joined: 11/8/2010
Posts: 4035
At least HxD is able to read it. It must be the .dtm format that makes it unreadable in Notepad, Notepad++, and DeHackEd's Microstorage. Do you think the .dtm format will eventually be updated with a more edit-friendly file format?
Joined: 7/22/2009
Posts: 128
Location: Gatineau, Quebec, Canada
I believe the dtm files for dolphin act quite different from the other movie files; they are hexadecimal files, instead of .txt Anyway, with dedication I am now able to edit .dtm files efficiently. But I have to say that it's still hard, because I have to work with numbers from 00 00 00 00 00 00 00 00 to FF FF FF FF FF FF FF FF as input, and the frame number also is an hexadecimal number that doesn't seem to represent the in-game number of frames. Also, if I am to edit a DTM file, I can't make a file any longer just by editing it with HxD. I have to go in-game and wait for the .dtm to get longer by itself, THEN I can add, let's say, 10 seconds of copy pasted input.
Taming Dolphin, one frame at a time
Editor, Experienced player (570)
Joined: 11/8/2010
Posts: 4035
Wak017 wrote:
I believe the dtm files for dolphin act quite different from the other movie files; they are hexadecimal files, instead of .txt Anyway, with dedication I am now able to edit .dtm files efficiently. But I have to say that it's still hard, because I have to work with numbers from 00 00 00 00 00 00 00 00 to FF FF FF FF FF FF FF FF as input, and the frame number also is an hexadecimal number that doesn't seem to represent the in-game number of frames. Also, if I am to edit a DTM file, I can't make a file any longer just by editing it with HxD. I have to go in-game and wait for the .dtm to get longer by itself, THEN I can add, let's say, 10 seconds of copy pasted input.
That's great! And thanks for posting this. I know it will help me if I ever have to edit my movie file.
Active player (335)
Joined: 1/19/2010
Posts: 383
Location: Texas
Wak017 wrote:
I believe the dtm files for dolphin act quite different from the other movie files; they are hexadecimal files, instead of .txt Anyway, with dedication I am now able to edit .dtm files efficiently. But I have to say that it's still hard, because I have to work with numbers from 00 00 00 00 00 00 00 00 to FF FF FF FF FF FF FF FF as input, and the frame number also is an hexadecimal number that doesn't seem to represent the in-game number of frames. Also, if I am to edit a DTM file, I can't make a file any longer just by editing it with HxD. I have to go in-game and wait for the .dtm to get longer by itself, THEN I can add, let's say, 10 seconds of copy pasted input.
I did some editing in HxD and it is really useful! I was able to redo ch1-2 in resident evil 4, and copy-paste ch1-3 into it for some of the parts. I did bump into that issue that you talked about Wak017 with modifying the dtm length. I'm searching to see if there are hex bytes somewhere that tell us how long the playback is (there should be). If I can modify those bytes directly, then editing it becomes 100x easier. Also, any suggestions to programming a hex editor for dtms? I have access to Microsoft Visual Studios, but idk what would be best to create an executable, if I decide to program this.
RachelB
She/Her
Player (129)
Joined: 12/3/2011
Posts: 1579
I did bump into that issue that you talked about Wak017 with modifying the dtm length. I'm searching to see if there are hex bytes somewhere that tell us how long the playback is (there should be).
It's at 0xd, and is backwards. So, looking at one of my .dtms, i have AF 25 01 00. This would be 000125af, or 75183 frames. The entire format is explained at http://code.google.com/p/dolphin-emu/wiki/TAS_Tasks
Active player (335)
Joined: 1/19/2010
Posts: 383
Location: Texas
rog wrote:
It's at 0xd, and is backwards. So, looking at one of my .dtms, i have AF 28 01 00. This would be 000125af, or 75183 frames. The entire format is explained at http://code.google.com/p/dolphin-emu/wiki/TAS_Tasks
Good to know. I am still kinda frustrated about read-only mode though. With the latest revisions of Dolphin (currently using 3.0-232), it seems as though when you load-state from a state that was created during the playback of a different dtm file, it it loads the dtm associated with that specific load-state, even if you were playing back a different dtm file before the load-state (confusing, I know). Example: I playback a file "A.dtm" and during that playback I savestate1. I then add onto "A.dtm" some additional TASing. I save the new file as "B.dtm". If I playback "B.dtm" with read-only mode check-marked, and then loadstate1, it seems to stop playing back "B.dtm", loads "A.dtm" from loadstate1 and plays it until the end of the file. Although A & B contains the same frame data at loadstate1, it still does this. If we could disassociate the dtm from the savestates, this would make read-only mode work (maybe?).
RachelB
She/Her
Player (129)
Joined: 12/3/2011
Posts: 1579
Use 235. Read only doesn't work prior to 234. It was just fixed the other day by Nitsuja.
Active player (335)
Joined: 1/19/2010
Posts: 383
Location: Texas
rog wrote:
Use 235. Read only doesn't work prior to 234. It was just fixed the other day by Nitsuja.
:3 The timing on his part could not be better, considering I went on the Dolphin website a few days ago to download the new revision.
Editor, Experienced player (570)
Joined: 11/8/2010
Posts: 4035
Also, nitsuja said that trying to load a savestate from another movie will give you an error message stating that the savestate you are attempting to load is not from the movie you are currently editing. This should help make sure you don't load another dtm's savestate into your current movie.
Active player (335)
Joined: 1/19/2010
Posts: 383
Location: Texas
CoolKirby wrote:
Also, nitsuja said that trying to load a savestate from another movie will give you an error message stating that the savestate you are attempting to load is not from the movie you are currently editing. This should help make sure you don't load another dtm's savestate into your current movie.
Yeah, I see that message now. How can I make it so I don't have to rewatch my TAS everytime I want to add something? I want to be able to just loadstate at a part of the game, and then it continues the playback from there. Is that not possible yet Nitsuja? I have done read-only mode on older revisions, but I have to create a savestate during playback of a dtm so it can continue playing from that point when I do loadstate, but that defeats the purpose since I have to watch my run anyway.
RachelB
She/Her
Player (129)
Joined: 12/3/2011
Posts: 1579
SoulCal wrote:
CoolKirby wrote:
Also, nitsuja said that trying to load a savestate from another movie will give you an error message stating that the savestate you are attempting to load is not from the movie you are currently editing. This should help make sure you don't load another dtm's savestate into your current movie.
Yeah, I see that message now. How can I make it so I don't have to rewatch my TAS everytime I want to add something? I want to be able to just loadstate at a part of the game, and then it continues the playback from there. Is that not possible yet Nitsuja? I have done read-only mode on older revisions, but I have to create a savestate during playback of a dtm so it can continue playing from that point when I do loadstate, but that defeats the purpose since I have to watch my run anyway.
Turn on read only if you want to play back, then load the savestate you want to start playing from. If you want to continue recording from the save state, turn off read only before loading it. Read only was broken before 234. That's not how it's supposed to (or does anymore) work. You can pause while recording, turn on read only, then load your last save state, and it'll play back from that save state up until the point you were at.
Active player (335)
Joined: 1/19/2010
Posts: 383
Location: Texas
rog wrote:
Turn on read only if you want to play back, then load the savestate you want to start playing from.
That won't work. If I do loadstate, it gives that error that the dtm and savestate don't match up. The emulator screen then says "End Movie." Let's say I did "savestate 1" while TASing at frame 100. I continue TASing till frame 200, then I close the emulator and save my dtm file. I want to be able to say "loadstate 1 and continue playback of new dtm at frame 100." However, since the savestate was created with the previous dtm, and not the new one, the new dtm won't do playback.
rog wrote:
If you want to continue recording from the save state, turn off read only before loading it.
This I know works. The problem is I want to Playback from this point; not just add more stuff from this point on.
rog wrote:
Read only was broken before 234. That's not how it's supposed to (or does anymore) work. You can pause while recording, turn on read only, then load your last save state, and it'll play back from that save state up until the point you were at.
If I understand this correctly, this won't do what I want it to either, since I will have to watch my entire run again to create those savestates to be associated with that specific dtm. I do see some benefit to doing this, but everytime I close the emulator (or more often, when Dolphin crashes) I will have to watch the entire run again. And btw, I apologize if I sound uptight. I find it difficult to put my thought process down.
RachelB
She/Her
Player (129)
Joined: 12/3/2011
Posts: 1579
Are you still using 3.0-232 or earlier? What you are describing is how read only was broken before. it should be completely fixed in 234 and 235.
Active player (335)
Joined: 1/19/2010
Posts: 383
Location: Texas
rog wrote:
Are you still using 3.0-232 or earlier? What you are describing is how read only was broken before. it should be completely fixed in 234 and 235.
I downloaded, and am now using v3.0-235.
RachelB
She/Her
Player (129)
Joined: 12/3/2011
Posts: 1579
And it still works as you described in your last post?
Active player (335)
Joined: 1/19/2010
Posts: 383
Location: Texas
rog wrote:
And it still works as you described in your last post?
Maybe we're having some miscommunication? I just don't see a difference yet in the 235 revision, except for the error messages saying that the dtm and savestate don't match up. I want to do exactly what I described with my frame 100/200 example above, but that doesn't seem to work.
RachelB
She/Her
Player (129)
Joined: 12/3/2011
Posts: 1579
You should be able to do exactly that with 235. After closing and reopening dolphin are you playing back the movie (the one you saved at frame 200)? or simply starting the game, and loading save state 1? If you start recording, make a save state right away, go through the menu at the beginning of the game, and make a second save state, then turn on read only mode, then load save state 1, what happens? Prior to 3.0-234, it would simply say movie ended, and it would not go any further. However after 3.0-234, it should play back until the point where you loaded the first save state. Does that not happen?
Active player (335)
Joined: 1/19/2010
Posts: 383
Location: Texas
rog wrote:
After closing and reopening dolphin are you playing back the movie (the one you saved at frame 200)? or simply starting the game, and loading save state 1?
I go to Emulation->play recording.
rog wrote:
If you start recording, make a save state right away, go through the menu at the beginning of the game, and make a second save state, then turn on read only mode, then load save state 1, what happens?
This works, however it is not what I want. I click the X to shut down Dolphin, it prompts me to save my dtm; which I do, and name it something like RE4.dtm and it is placed in the "Recordings" folder. If I play back RE4.dtm, then loadstate1, it will not play back from the point savestate1 is at, specifically because savestate1 was not created from RE4.dtm (which is that new error Nitsuja put in).
rog wrote:
Prior to 3.0-234, it would simply say movie ended, and it would not go any further.
This is partially untrue. Read-only mode sorta worked with older revisions. Say you have a dtm file(say 1000 frames long), open Dolphin, then play the dtm back and create a savestate at frame 500, then close Dolphin again. If you play back that same dtm and did loadstate, it would resume playback at frame 500. The "Movie End" message would appear only if you didn't create a savestate while playing it back. Don't think I'm crazy, cause I know this to be fact for older revisions. But like I said, this method forces the TASer to play back the dtm file regardless, so there's no point.
RachelB
She/Her
Player (129)
Joined: 12/3/2011
Posts: 1579
SoulCal wrote:
rog wrote:
If you start recording, make a save state right away, go through the menu at the beginning of the game, and make a second save state, then turn on read only mode, then load save state 1, what happens?
This works, however it is not what I want. I click the X to shut down Dolphin, it prompts me to save my dtm; which I do, and name it something like RE4.dtm and it is placed in the "Recordings" folder. If I play back RE4.dtm, then loadstate1, it will not play back from the point savestate1 is at, specifically because savestate1 was not created from RE4.dtm (which is that new error Nitsuja put in).
Try turning off panic handlers. It should work, regardless of the error. If not, can you try it in a different game? If it still doesn't work, you'll probably want to talk to nitsuja about it, because read only works perfectly for me in 235.
rog wrote:
Prior to 3.0-234, it would simply say movie ended, and it would not go any further.
This is partially untrue. Read-only mode sorta worked with older revisions. Say you have a dtm file(say 1000 frames long), open Dolphin, then play the dtm back and create a savestate at frame 500, then close Dolphin again. If you play back that same dtm and did loadstate, it would resume playback at frame 500. The "Movie End" message would appear only if you didn't create a savestate while playing it back. Don't think I'm crazy, cause I know this to be fact for older revisions. But like I said, this method forces the TASer to play back the dtm file regardless, so there's no point.
[/quote]That's correct, it would work when playing back a movie, but as you said, that is useless.
Active player (335)
Joined: 1/19/2010
Posts: 383
Location: Texas
Turning off Panic Handlers seemed to fix the issue, thanks. :)