Post subject: Movie file extension [flame]
Active player, Editor (296)
Joined: 3/8/2004
Posts: 7468
Location: Arzareth
<flame mode on> You named MSX rerecording movie files as .gz ???? What kind of idiotic decision is that? Did you know that .gz happens to be a standard file extension that stands for "gzip compressed"? Examples: -- something.tar.gz is a gzip compressed Tar Archive file -- dump.sql.gz is a gzip compressed SQL dump file -- patch-cromfs-writeaccess.gz is a gzip-compressed patch file (text format) -- mkcromfs.exe.gz is a gzip-compressed EXE file -- rockman2.fm2.gz is a gzip-compressed FCEUX movie file Now you are trying to imply that gz actually stands for MSX movie files? Did you not also realize that on TASVideos site, that movie files are actually identified by their filename extension, and that by committing the extension .gz exclusively to one movie file format, you are in effect depriving any other emulator the right to use .gz to signify that the movie file is compressed? It turns out that the MSX movie file "gz"s are, in fact, gzip compressed. I HASTILY SUGGEST that before any damage is done (that is, before MSX TASing becomes any more popular than it is now), an actual file name extension is assigned to MSX movie files (I suggest mxv), and IF the movie file is compressed, possibly an additional .gz is tacked to it; for example: terminator.mxv.gz. And on the TASVideos site code, if a .gz extension is encountered, an extra processing layer is simply added which gzip-decompresses the file before parsing it; without committing the extension .gz to any particular emulator. (Adding the .gz extension could also be optional. Such optional compression could also be added to FM2 parser etc.)
Emulator Coder, Player (31)
Joined: 4/24/2010
Posts: 73
Location: California
I agree
MSX Game specialist / openMSX team member
NitroGenesis
He/Him
Editor, Experienced player (550)
Joined: 12/24/2009
Posts: 1873
Vampier wrote:
I agree
since you are a coder of openMSX, why not try to fix this?
YoungJ1997lol wrote:
Normally i would say Yes, but thennI thought "its not the same hack" so ill stick with meh.
Emulator Coder, Player (31)
Joined: 4/24/2010
Posts: 73
Location: California
I'm not a coder, I do scripting and side support (romdb/trainers and other stuff)
MSX Game specialist / openMSX team member
Patashu
He/Him
Joined: 10/2/2005
Posts: 3999
In the name of consistency! *raises sword*
My Chiptune music, made in Famitracker: http://soundcloud.com/patashu My twitch. I stream mostly shmups & rhythm games http://twitch.tv/patashu My youtube, again shmups and rhythm games and misc stuff: http://youtube.com/user/patashu
Active player, Editor, Emulator Coder (262)
Joined: 10/17/2010
Posts: 124
Can someone explain to me what the relevancy is of file extensions for emulator replay files?
Patashu
He/Him
Joined: 10/2/2005
Posts: 3999
Quibus wrote:
Can someone explain to me what the relevancy is of file extensions for emulator replay files?
Bisqwit wrote:
Did you not also realize that on TASVideos site, that movie files are actually identified by their filename extension, and that by committing the extension .gz exclusively to one movie file format, you are in effect depriving any other emulator the right to use .gz to signify that the movie file is compressed?
My Chiptune music, made in Famitracker: http://soundcloud.com/patashu My twitch. I stream mostly shmups & rhythm games http://twitch.tv/patashu My youtube, again shmups and rhythm games and misc stuff: http://youtube.com/user/patashu
Active player, Editor, Emulator Coder (262)
Joined: 10/17/2010
Posts: 124
So, maybe that was just a bad idea? (Identifying movie files by their extension.) I'm sure Grunt is smart enough to make this a non-issue.
Post subject: Re: Movie file extension [flame]
Joined: 11/11/2006
Posts: 1235
Location: United Kingdom
Bisqwit wrote:
<flame> You named MSX rerecording movie files as .gz ???? What kind of idiotic decision is that?
I concur with Bisqwit
<adelikat> I am annoyed at my irc statements ending up in forums & sigs
Joined: 7/2/2007
Posts: 3960
Quibus wrote:
So, maybe that was just a bad idea? (Identifying movie files by their extension.) I'm sure Grunt is smart enough to make this a non-issue.
Identifying movies by their headers, which is the alternative, is much more difficult -- you have to open the file as binary and examine certain bits, then have a big ol' state machine to determine which bits match which movie formats. And you have to deal with the possibility that the file might be compressed, so if you can't match it against anything you have to try uncompressing it and then start all over. Matching by file extension is simple and straightforward; all it relies on is that someone not be silly enough to pick a file extension that is already being used by one of the most common programs in the world. I'd guess that the .gz extension ranks up there with .gif and .png in terms of commonality. There's really no excuse for re-using it.
Pyrel - an open-source rewrite of the Angband roguelike game in Python.
Editor, Player (68)
Joined: 1/18/2008
Posts: 663
Is the uncompressed movie file a standard encoding text file?
true on twitch - lsnes windows builds 20230425 - the date this site is buried
Active player, Editor, Emulator Coder (262)
Joined: 10/17/2010
Posts: 124
The uncompressed file is just an XML file.
Joined: 2/26/2007
Posts: 1360
Location: Minnesota
I hope Bisqwit knows how to flame off :(
adelikat wrote:
I very much agree with this post.
Bobmario511 wrote:
Forget party hats, Christmas tree hats all the way man.
Joined: 11/4/2007
Posts: 1772
Location: Australia, Victoria
Compressing a movie file is completely pointless if it's submitted to the site as a ZIP anyway.
Active player, Editor, Emulator Coder (262)
Joined: 10/17/2010
Posts: 124
The replay is compressed because it saves disk space for the user. And its compression ratio is very high, as it's plain XML.
Active player, Editor (296)
Joined: 3/8/2004
Posts: 7468
Location: Arzareth
Plain, very pessimal XML. With such traits as being indented (with spaces, causing an overhead of about 10 bytes per line) and double time tags (a total of eight occurrences of the word "time" for each hit of a button), with each input event increasing the uncompressed file size by some 250 bytes, or 500 bytes if you count press+release. (Contrast the text-format FM2 in which each frame takes 23 bytes, or the binary format SMV in which each frame takes 2 bytes (for a single-player movie).) But that is beyond the point of this this thread. The point is that OpenMSX's movie replay files should be assigned a proper file extension that is separate from the transmission aspect (i.e. the fact that they are gzip compressed).
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
If It's XML with lots of spaces, I recommend bz2 for compression.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Active player, Editor, Emulator Coder (262)
Joined: 10/17/2010
Posts: 124
The file can have any name, but if it is compressed, it needs to end with .gz. This limitation is currently under investigation to be removed. If successful (and currently it looks as if it's no big issue), you can rename it to whatever you like :P
Joined: 11/4/2007
Posts: 1772
Location: Australia, Victoria
I still honestly don't see the point of compressing it. FCEUX files are 100% .txt based and generally have a large filesize (In relative terms). But we're forgetting something, people generally have the disc space to hold multiple 1 megabyte files anyway. I'm sorry, but this is just one of those 'It bugs me' dealies.
Active player, Editor (296)
Joined: 3/8/2004
Posts: 7468
Location: Arzareth
Flygon wrote:
I still honestly don't see the point of compressing it. FCEUX files are 100% .txt based and generally have a large filesize (In relative terms).
The Rambo TAS is 1 minute and 39 seconds long. Its uncompressed movie file is 528597 bytes in size, of which about 200000 bytes are initialization. By extrapolation, a 30 minute movie would be 6 megabytes in size, assuming a similar input density.* In comparison, a FCEUX movie of 30 minutes is 2.5 megabytes in size. This calculation supports your point. *) This assumption may also fail. For a 30 minutes long movie that, by average, on each frame, presses and releases 10 buttons -- truly a button mashing fest indeed --, the movie file would be 540 megabytes in size. Actual movies are probably always smaller.
mth
Joined: 10/31/2010
Posts: 5
Location: Eindhoven, the Netherlands
I am one of the openMSX developers, although I did not work on the TAS mode, which was mostly done by Wouter. We discussed the file extension issue on IRC and the outcome is that the extension will be changed to ".omr" (short for "openMSX replay"). The reason for choosing gzip compression is that openMSX already includes support for gzip compression. This means fewer code is needed and no additional library dependencies. Also, it gives a good tradeoff between compression time and final size; for example bzip2 is more compact, but is significantly slower when compressing. The reason for compressing at all is that the uncompressed size, although manageable on modern PCs, is still big. One replay won't fill up your HD, but a few dozen do eat up free space. Also, openMSX runs on mobile devices (Dingoo A320, Nokia N900) as well, which have much smaller storage which is typically also slower. I don't know if people will do TAS-ing on mobile devices, but recording a walkthrough type of replay should be possible. The XML is generated from serialized C++ objects, which explains why its structure looks different from what a handcrafted format would look like.
openMSX developer
Active player, Editor (296)
Joined: 3/8/2004
Posts: 7468
Location: Arzareth