Post subject: TASing open source video games for PC!
MESHUGGAH
Other
Skilled player (1918)
Joined: 11/14/2009
Posts: 1353
Location: 𝔐𝔞𝔤𝑦𝔞𝔯
As a speedrun enthusiast and programmer, I'm about to start this project. Since we can't perfectly emulate the PC platform, the only way to TAS a PC game (atm) without desyncing (memory hacking, script writing...) is adding TAS functions into open source games. How does it works [the TASablegating]: - Downloading an open source video game's source code (for example: Quake) - Adding TAS related functions like printing out memory values (enemy HP), advance one frame, reading input from file, etc. - Build source code and release executable file (the game's main exe file) Open source video game means that the game's source code can be downloaded from the internet and you can modify it (of course the engine license is not permits to "do what you would like to"). For example, downloading Quake source code to make a new ability like double jumping from Unreal Tournament. There are many cons for releasing a game's source code. For further info, please read this page: http://en.wikipedia.org/wiki/Open_source_video_game Some open source video games that I recommend to make a TAS speedrun of it (in random order): NetHack, Wolfenstein 3D, Duke Nukem 3D, Doom, Heretic, Hexen I / II, Quake I / II, Abuse, Rise of the Triad, 0 A.D., Tyrian, Neverball, Tux Racer, Beneath a Steel Sky, Shadow Warrior, Alien versus Predator, MechCommander 2, Marathon 2: Durandal, Star Control 2, Blood 2, No Gravity, Chromium B.S.U. [bolds: SDA has runs of it] There were already TAS speedrun of Doom I and Doom II, so it's definetly works. For more open source video games and informations about the games I mentioned -> http://en.wikipedia.org/wiki/List_of_open-source_video_games http://www.classicdosgames.com/misc/source.html http://en.wikipedia.org/wiki/Video_games_notable_for_speedrunning http://en.wikipedia.org/wiki/Category:Commercial_video_games_with_freely_available_source_code http://www.hotud.org (google search) And how to make a TAS for these games: Every game have different source code, and needs different movie file. Here is an example: Quake from Id Software. After downloading the source code of Quake, I open it in the supported ide (for quake: vc++ 6.0) and find the most crucial program code parts like: hostframe (which runs in every frame), input handling, and then find tas related memory values (i mean boss hp and other things), and write frame advancing (for first: only forward.) and other options. Movie file example:
Quake has 18 actions and the mouse. I didn't read the whole quake source code, but as far as i saw it, the movie file will contains bytes like this:
0xFF 0x01 0x00 0x02 0x00 0x01 ... [more 16 bytes] and 0x00 then mouse coordinates (or whatever quake uses).

So what's this:
0xFF - new frame
Next 18 bytes: the corresponding actions (for example the first byte: +attack, next one: impulse 10 next one: +jump)
0x00 -> nothing
0x01 -> pressed
0x02 -> released
so 0xFF 0x01 0x00 0x02 0x00 and more 0x00 means that you pressed +attack and released +jump.
And I would write a simple replay function that would read these bytes from the file and giving these inputs in to the quake's original input handling function. And if the game wouldn't have a "startmovie" or "record" or other things, I could simply write a function to make a screenshot for each frame, and then you just simply run bmp2avi software or anything. That's all. Now what about TASing a modified (this way, to make it TASable) open source video game... Desync: Making the very same (frame by frame) input in to the original open source video game and in to the TASableted would have exactly the same outcome. Desync can't happen, because the input will be played back from frame to frame. I think only multi threading would be a headache. And of course you need a PC that mets the minimal system requirements to play the game. I don't think that a PC game would use such a complex random generator that would be changed cause there's other functions or because a hostframe code part makes more CPU cycles... Viewing the TAS movie: 3D games TASes must avoid motion-sickness. Of course, speed is more important. But I think you know what I'm trying to aim. Various possibilites by editing the source code of a video game: - Making a "buttons pressed" GUI like the one in one of the Super Mario World speedrun (that synchronizes buttons to the music) - Make comments on world brushes (just to mention a possibility, of course i wouldn't recommend this) - Make camera effects for the final movie (like quake speedrun) - Make internal subtitles or anything - Really anything. The only thing I need or anyone who is a programmer is the source code. I wrote Quake for examples because I already own it. Oh and two pros for open source video games [there's more actually]: -You can read nearly everything, how the game works (finding bugs, glitches, avoiding frictions, etc) -TASes of these games will works like that they are an emulator and they replay the submission movie just like someone would play it. So my only question would be this: Anyone interested in it, making a speedrun for an open source video game? Finally, sorry for my sluggish and lousy english. And I also tried to NOT make a long post. I failed. [edit: just text formatting + desyncinfo update + updating the games section after reading warp's post]
PhD in TASing 🎓 speedrun enthusiast ❤🚷🔥 white hat hacker ▓ black box tester ░ censorships and rules...
Post subject: Re: TASing open source video games for PC!
Banned User
Joined: 3/10/2004
Posts: 7698
Location: Finland
MESHUGGAH wrote:
I don't know why nobody came up with this idea earlier
Oh, the idea is really old, much older than this website. Tool-assisted speedrunning started in the Doom speedrunning community by doing exactly what you are suggesting: By patching the (then already open source) Doom source code with features which allowed what were effectively rerecording and savestates (and IIRC pausing). TAS-Doom was an entire subgenre of Doom speedrunning back then. The idea of using emulators to TAS old console games is actually newer than tool-assisted speedrunning itself.
MESHUGGAH
Other
Skilled player (1918)
Joined: 11/14/2009
Posts: 1353
Location: 𝔐𝔞𝔤𝑦𝔞𝔯
After reading your post, I searched for other TAS-es, but I only found Doom I [18:38] and Doom II [23:11]. I understand the "tool-assisted speedrunning", but why others try to TAS open source video games with harder techniques? For example, Quake (okay i won't use this for an example next time). Why others try to make a TAS of it by writing scripts? You can't do huge and precize mouse skip (like 286° turn and +29 Y...) And also fps drop... So my question still lives for those games that don't have a TAS speedrun (or even a normal speedrun).
PhD in TASing 🎓 speedrun enthusiast ❤🚷🔥 white hat hacker ▓ black box tester ░ censorships and rules...
Banned User
Joined: 3/10/2004
Posts: 7698
Location: Finland
I see one potential problem in adding TASing capabilities to the game itself, rather than the game being run unmodified inside an emulated environment: Namely, the game effectively becomes a hack. The attitude towards accepting TASes of hacked games has loosened a bit recently, but there's a difference: The hacked games are not even intended to pass as the original games: It's made clear that a TAS of a hacked game is certainly not the same thing as a TAS of the original game. The problem with what you are proposing is that you are basically trying to pass a TAS of a hacked game as a TAS of the original, unmodified one. The TAS would be completely non-problematic if the resulting recording file could be playable with the original, unmodified game (some games officially support user-recorded "demos", which would allow this). However, not many games support this. If the recording file is only playable with the modified game, the question raises whether it's equivalent to playing the original game. How can be know that the modifications didn't modify some elements of the gameplay itself? (It could be conceivable that such modifications could change some delays and such.)
MESHUGGAH
Other
Skilled player (1918)
Joined: 11/14/2009
Posts: 1353
Location: 𝔐𝔞𝔤𝑦𝔞𝔯
Warp wrote:
I see one potential problem in adding TASing capabilities to the game itself, rather than the game being run unmodified inside an emulated environment: Namely, the game effectively becomes a hack.
So running a game inside an emulated environment to advance one frame, which isn't supported by the game itself (there's no such function like "advance one frame" in 95% of the games) wouldn't be called as a "hack", while playing a game that has this function after adding it into the source code is a hack. I don't agree with that but it's only my opinion.
Warp wrote:
The TAS would be completely non-problematic if the resulting recording file could be playable with the original, unmodified game (some games officially support user-recorded "demos", which would allow this). However, not many games support this. If the recording file is only playable with the modified game, the question raises whether it's equivalent to playing the original game. How can be know that the modifications didn't modify some elements of the gameplay itself? (It could be conceivable that such modifications could change some delays and such.)
If the programmer made such an elaborate Host_Frame code that either corrupts memory or make a timer to time the execution time of the Host_Frame, I could simply time my own function, and delete it from the final time. Basicly, since I can read the "whole" cycle of one frame (Host_Frame's most of the times are fully released), I can make it to work 100% as I would like to. So delays are eliminated fully. For memory, I would either read the minimal system requirements of the game and make a huge memory buffer (sounds crazy, cause it's crazy) or do something similar (pagefiles or anything) so memory corruption would be still "emulated". And I would use custom memory buffer for my own functions. And how to check that the speedrun on the modified game is equivalent to playing the original game: memory dumps, code cycle watches, breakpoint analysis, value storing, etc. Depends on the game's source code, there's no global solution for this.
PhD in TASing 🎓 speedrun enthusiast ❤🚷🔥 white hat hacker ▓ black box tester ░ censorships and rules...
Editor, Active player (476)
Joined: 5/23/2006
Posts: 361
Location: Washington, United States
I think one major reason this isn't done very often is that you can (usually) only add TAS capabilities to one game at a time. As I understand it, even implementing savestates that don't desync would be a lot of work, something that most people aren't willing to put into just one game. If somebody is going to go to the effort of adding TAS capabilities to one particular game, why not put that effort into an emulator instead, which can potentially open up TASes for many games at once? I think that for this to happen, you either need somebody who's highly-motivated towards the project, or the backing of a group/community who are all working on the project at once. Now, this doesn't necessarily stop anybody from trying (I myself have experimented with external input/replay generation for certain PC games), but I do think the motivation will be lower, which is why we haven't really seen any projects like this so far.
Former player
Joined: 11/13/2005
Posts: 1587
MESHUGGAH wrote:
Warp wrote:
I see one potential problem in adding TASing capabilities to the game itself, rather than the game being run unmodified inside an emulated environment: Namely, the game effectively becomes a hack.
So running a game inside an emulated environment to advance one frame, which isn't supported by the game itself (there's no such function like "advance one frame" in 95% of the games) wouldn't be called as a "hack", while playing a game that has this function after adding it into the source code is a hack. I don't agree with that but it's only my opinion.
I agree with Warp. Using emulator tools doesn't alter the game in any way. It just allows to overcome human limitations of precision, reflex and perception.
nesrocks
He/Him
Player (246)
Joined: 5/1/2004
Posts: 4096
Location: Rio, Brazil
I guess that if the movie file can run on an official release of the game then there's no problem that it was recorded on a modified version (for acceptance on this website). But I'm no judge, mind you.
MESHUGGAH
Other
Skilled player (1918)
Joined: 11/14/2009
Posts: 1353
Location: 𝔐𝔞𝔤𝑦𝔞𝔯
If anybody would name a game, and I can download it's source code [and I know that language] then I would make a TAS version of it. And after time, the speedrun would be released [somewhere] and the next project would start with the same procedure again.
PhD in TASing 🎓 speedrun enthusiast ❤🚷🔥 white hat hacker ▓ black box tester ░ censorships and rules...
Lord_Tom
He/Him
Expert player (3143)
Joined: 5/25/2007
Posts: 399
Location: New England
I'd be perfectly fine with this sort of project as long as the source code for the modified version were available -- so that at least theoretically one could verify that the author has made a good-faith effort to avoid gameplay-altering mods. We TAS on emulators, not the original hardware, so I just don't see what difference it makes to add TAS capabilities into the source code of the original game.
Joined: 1/26/2009
Posts: 558
Location: Canada - Québec
MESHUGGAH wrote:
If anybody would name a game
Here a bunch of random game that I picked here and there: Armagetron Battle for Wesnoth Bombic C-evo Dark Oberon FreedroidRPG GL-117 Glest Globulation 2 Mtp Target Hedgewars (better than Wormux ?) ManiaDrive Micropolis (should be the original SimCity - free since 2008) NetHack (might or might not look better with a graphic port) Open Sonic Open Transport Tycoon Deluxe Pingus Ri-li Secret Maryo Chronicles Super Tux Super TuxKart Torcs Wolfenstein 3D X-moto (would be a nice one) Yo Frankie! Zelda fan-games : Oni Link Begins / Return of the Hylian / Time to Triumph ..... Hm.. of course there quite a lot of other game, but I can't even say wich one is enough "mature" for a complete playthrough. Creating an another thread about potential open-source game, might sound like a better idea. Patching manually your game could be hard and very long... If your going to add TASing tool on a simple windows game, maybe you should ask this guys, if his project already support it first, before your work is wasted.
MESHUGGAH
Other
Skilled player (1918)
Joined: 11/14/2009
Posts: 1353
Location: 𝔐𝔞𝔤𝑦𝔞𝔯
Lord Tom wrote:
I'd be perfectly fine with this sort of project as long as the source code for the modified version were available -- so that at least theoretically one could verify that the author has made a good-faith effort to avoid gameplay-altering mods. We TAS on emulators, not the original hardware, so I just don't see what difference it makes to add TAS capabilities into the source code of the original game.
Of course I thought that I would upload the original source code for the open source games (I think eula and license permits this) and also the modified version, so anyone can read the whole source code and rebuild it if he/she doesn't believes me :) And another thing regarding the "replaying the same input in the unmodified game". Since I'm about to make the TAS functions as perfect as I can (using external memory buffers, eliminating delays (if there's any)), there won't be impact on the outcome. But even if you would really like to play the TAS movie on the unmodified game -> just simply debug the original source code, make breakpoints on the user inputs, since the gameplay time and hostframe time and anything will also stop at those breakpoints. (I bolded this because this was a crucial threat about proving that the same input replayed on the unmodified game would be the same)
BadPotato wrote:
Patching manually your game could be hard and very long... If your going to add TASing tool on a simple windows game, maybe you should ask this guys, if his project already support it first, before your work is wasted.
It's not that long. I'm not a super special programmer expert guru (like John Carmack). I just simply have high IQ and generally understand the intent of the game's developers and also I'm a hacker :) Step-by-step: - find the main function that runs every frame (usually called host_frame) - add functions: frameadvance, readinput, readinputfromfile, frameback (later...), framepause, saveinput, loadstate (depends heavily on the game's code, it will be a very fast emulation), savestate (same), showgui (special TAS gui that shows frame number, TASer's favorite memory values like velocity speed, etc). I think I will make a documentation about the useful functions and I will use the same function name for all the other open source games to avoid confusion and to provide easy overview for others. And important thing about starting a TAS of an open source game: - Reading the full source code to find possible glitches and bugs (and writing down the cycle of a frame and other things.) I'm also interested in playing a game in various fps_max(maximum frame per sec) because it's alters game physics. For example in HL, playing on fps_max 20 you advance 160 unit by pressing +forward for 1 frame with 0 unit/sec initial speed, while in fps_max 100 you advance 32 unit.
PhD in TASing 🎓 speedrun enthusiast ❤🚷🔥 white hat hacker ▓ black box tester ░ censorships and rules...
Joined: 3/30/2007
Posts: 44
I just simply have high IQ and generally understand the intent of the game's developers and also I'm a hacker :)
Oh is that so? Then you don't need the source code at all. You just need to hook the game's API calls and trace the execution in a loader that tracks register states (hence the EIP register is most useful for that) via GetThreadContext(). >_>
Post subject: Re: TASing open source video games for PC!
Editor, Active player (297)
Joined: 3/8/2004
Posts: 7469
Location: Arzareth
Meshuggah, the problem with what you suggest is 1) verifiability and reliability 2) speciality and portability. 1) It's troublesome to verify that nothing else was done to the game other than adding TAS functions. After all, if you can hack it, you can do anything to it. It is possible, but troublesome. For the same reason, TASVideos cannot accept movies for which you need a specially modified emulator (i.e. with features that are specific to that particular game only) to play the movie. 2) Each game essentially becomes their own system. It becomes very troublesome to maintain movies for those games on this site, much more difficult than it is when exactly the same tools (including AVI recording, means of specifying movie file, etc.) work for every game.
Patashu
He/Him
Joined: 10/2/2005
Posts: 4043
Isn't there a special kind of file that describes the difference between one kind of a file? If your hack is expressed as one of these files, anyone could confirm that you do nothing except what you say you do. Even if it is still arbitrary.
My Chiptune music, made in Famitracker: http://soundcloud.com/patashu My twitch. I stream mostly shmups & rhythm games http://twitch.tv/patashu My youtube, again shmups and rhythm games and misc stuff: http://youtube.com/user/patashu
MESHUGGAH
Other
Skilled player (1918)
Joined: 11/14/2009
Posts: 1353
Location: 𝔐𝔞𝔤𝑦𝔞𝔯
mudlord :) Bisqwit: To verify? As I said, the source code of the "hacked" (i just use your words) game will be open to everyone. And the movie file also can be replayed by downloading the original quake source code, making breakpoints to the keyboard input and mouse input, and send exactly the same data like in the movie file. For the emulator, I thought there should be an external program that handles game memory and enables memory search and other things that needed to special things like game restart within a movie. But since nobody interested in this project, there's really nothing about to talk :(
PhD in TASing 🎓 speedrun enthusiast ❤🚷🔥 white hat hacker ▓ black box tester ░ censorships and rules...
Editor, Active player (297)
Joined: 3/8/2004
Posts: 7469
Location: Arzareth
MESHUGGAH wrote:
And the movie file also can be replayed by downloading the original quake source code
Well, that is different then. TASVideos does not care how you create the movie, as long as its playback does not depend on the game being changed somehow.
Joined: 11/22/2004
Posts: 1468
Location: Rotterdam, The Netherlands
MESHUGGAH wrote:
And the movie file also can be replayed by downloading the original quake source code
That's because Quake movie files work in a different manner. The way I understand it, they don't really contain input, but rather the output (in object coordinates) of the engine. This allows you to set an arbitrarily high virtual FPS, which modifies the way the engine calculates the results of your actions. That's how "cheated" replays are made in the first place. This goes against the way movie files are supposed to work to be acceptable for this site, since essentially you are feeding the engine vastly more information than would be possible under normal circumstances.
Player (146)
Joined: 7/16/2009
Posts: 686
As far as I understood from a Half-Life TASing thread (same idea as here, basically) on SDA a while ago, that game contains either coordinates or coordinate displacements per frame, both of which would allow you to complete any level with an "exit point" point in exactly 2 frames: a start frame, and one to "warp" you to it.
Active player (356)
Joined: 1/16/2008
Posts: 358
Location: The Netherlands
Dada wrote:
That's because Quake movie files work in a different manner. The way I understand it, they don't really contain input, but rather the output (in object coordinates) of the engine.
Or (under your description) you could use an altered version of the game that has a teleport function, and the original game would play back the resulting movie just fine (it doesn't 'emulate' the teleport, it's just told that it happened)? either way that hould be a no-go :) ps. I'd love to do runs on UT99... but it's not open source, making it harder to achieve TAS functionality edit:
Scepheo wrote:
As far as I understood from a Half-Life TASing thread (same idea as here, basically) on SDA a while ago, that game contains either coordinates or coordinate displacements per frame, both of which would allow you to complete any level with an "exit point" point in exactly 2 frames: a start frame, and one to "warp" you to it.
hmm same thing as Quake then? In the end it would up to the audience to determine whether the movie should be considered genuine? (e.g. dp we all believe that the modified game works withing original rules/physics/etc?)
TASes: [URL=http://tasvideos.org/Movies-298up-Obs.html]Mr. Nutz (SNES), Young Merlin 100% (SNES), Animaniacs 100% (SNES)[/URL]
Joined: 3/30/2007
Posts: 44
Scepheo wrote:
As far as I understood from a Half-Life TASing thread (same idea as here, basically) on SDA a while ago, that game contains either coordinates or coordinate displacements per frame, both of which would allow you to complete any level with an "exit point" point in exactly 2 frames: a start frame, and one to "warp" you to it.
All 3D games have variables for X, Y, and Z. Thus, that technique can be done in any game.
Player (146)
Joined: 7/16/2009
Posts: 686
mudlord wrote:
Scepheo wrote:
As far as I understood from a Half-Life TASing thread (same idea as here, basically) on SDA a while ago, that game contains either coordinates or coordinate displacements per frame, both of which would allow you to complete any level with an "exit point" point in exactly 2 frames: a start frame, and one to "warp" you to it.
All 3D games have variables for X, Y, and Z. Thus, that technique can be done in any game.
There is a great difference however between the movie files being plain input (which can only produce "genuine" movies) or consisting of the player's position per frame. The first would enable TASing as we know it, the latter would allow one to simply do with the coordinates whatever he wants. As much as you may be capable of altering the file, if the movie is input-based there is no way to move at over 9000 times the maximum speed (excluding glitches), but coordinate-based movie files easily allow this (heck, they allow traversing the map in 1 frame, and even further). In the end, if we want a way to check the legitimacy of movies that have been produced using a special rerecording version of the original game, this would both require the movies being playable in the original game (which limits our choices to games that have replay functionality) and the movie files being input-based (which limits our choices even further).
Editor, Reviewer, Experienced player (979)
Joined: 4/17/2004
Posts: 3109
Location: Sweden
I would love a TAS of Quake. It is such a big speedrunning game that I wonder why noone has gone and done it... Anyway, if the movie file is input-based like you seem to allude to in your description, noone should have any problems with the validity of the movie. Do it. We can TAS later. :)
Joined: 5/25/2010
Posts: 30
Location: California
A lot of people don't realize this, but the concept of tool-assisted speedrunning had actually started from the Doom community. However, the TAS'ing capabilities for that game are relatively primitive. While console emulators have all those fancy features, the TAS-enabled Doom ports (e.g., TasDoom, MBF and PrBoom) offer only basic ones, such as save states, rerecording and slowdown. I'd love to see more TAS features added to thouse source ports.
Editor, Skilled player (1439)
Joined: 3/31/2010
Posts: 2108
Well, with DOS TASing possible now, we've got to ask ourselves. Do we really need to, if he have one tool with all those fancy features already built in?