Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
I'm part of a team creating a Windows game (HourGlass!), and we had an interesting thought. We're going to make part of the game run on SQL, and intentionally allow for a way to do SQL injection in certain in-game interfaces. However, since we don't want the game utterly destroyed, it's going to be filtered to block DROPs and other really nasty things. Basically allow one to run arbitrary UPDATEs to do interesting things within the game engine itself.
Would a TAS which makes use of this intentional built-in exploit be considered executing arbitrary code? It's not so arbitrary, because it's filtered to an extent, and it's only the SQL code used for managing some data, not the primary game engine code. On the other hand, you can write practically any SQL statement consisting of approved functions.
We're also debating as to whether we should make it obvious to the exploiter that this is intentional or not. We thought perhaps a remark in the ending credits if you do hack in your own SQL like so:
Programmers:
Blah - Levels
Ring - Physics
John - Enemies
Dave - Graphics
Jim - Sound
Nach - System
You - SQL
Should we or shouldn't we alert the player at some point?
Thoughts?
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
I'm kinda torn on this issue.
On the one hand, ACE is simply running code that the game developers didn't put into the game. Using a memory exploit to jump to the endgame isn't ACE since the code is already there. Using a memory exploit to jump to a completely different game that wasn't there to begin with is ACE because the code isn't there. To that extent, it would be Arbitrary Code Execution, just one not requiring an unexpected exploit.
On the other hand, the ACE category seems to be reserved for making games do things they were never intended to do in the first place - making a game from 1998 play music from a cartoon from 2011, for a dramatic and divisive example. Even if you don't bring it to their attention, the fact that you're deliberately leaving this avenue of attack open just to see what sorts of inventive things people do with it, it's not in the spirit of ACE as we're familiar with it, and it becomes more of a playaround.
Either way, I'd love to see what I can do to it.
If you TAS Microsoft Excel to write some macros and execute them, is this ACE?
If you TAS Visual Studio to write a program, compile it and execute it, is this ACE?
(I would say 'yes' for both of the above, by the way - and for Nach's question, I would also say yes if I can make Pong, Snake, jump to the ending, jump to the final boss, etc etc. If I can't do any of those things, then it's not REALLY ACE.)
The key word here is arbitrary -- that implies that you must be able to make the program do anything (that the computer is physically capable of doing of course). Being able to perform SQL injection attacks is all well and good, and I think it's awesome that you're intentionally leaving in exploits like that, but unless the game is retrieving code from the SQL server and executing it, I don't think it really qualifies as ACE.
Pyrel - an open-source rewrite of the Angband roguelike game in Python.
In context, ACE as a term ought to be limited to situations and/or platforms where the user can't normally input and execute native machine code.
A PC with a C++ compiler might allow ACE technically speaking, but in that context the term makes little sense.
If a game internally uses some kind of scripting language, and you find an exploit to enter and run code in said scripting language, you might argue it's ACE, except for the fact that you are not completely free to run completely arbitrary code that the host machine is capable of (as you are bound to what the scripting language allows you to do), although this might be a matter of opinion and definition.
ACE makes most sense in the context of a closed platform where a regular user cannot normally input and execute any kind of machine code at all, under any circumstance (except through unintended exploits).
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Derakon wrote:
The key word here is arbitrary -- that implies that you must be able to make the program do anything (that the computer is physically capable of doing of course).
Anything?
Is not being able to alter the stats of all the enemies, how many spawn, items, etc... not enough? You can arbitrarily change those things as long as it doesn't fail any constraints.
Derakon wrote:
Being able to perform SQL injection attacks is all well and good, and I think it's awesome that you're intentionally leaving in exploits like that
We're not leaving in such an exploit, we're creating the exploit altogether. We have to go out of our way to have an SQL statement directly use user input.
Derakon wrote:
but unless the game is retrieving code from the SQL server and executing it, I don't think it really qualifies as ACE.
Embedded SQL.
Game maps, enemy data, items, and more is all in the database, with the game engine using all of it to do what it does. True you can't change the engine itself, but you can very much alter what it's doing.
Warp wrote:
ACE makes most sense in the context of a closed platform where a regular user cannot normally input and execute any kind of machine code at all, under any circumstance (except through unintended exploits).
Why does it have to be machine code? What if a game in an interpretive language, and you can inject code in said language?
Also, does it have to be an unintended exploit?
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Why does it have to be machine code? What if a game in an interpretive language, and you can inject code in said language?
I think "arbitrary" implies "anything you want, without limitation". A scripting language will always limit what you can do.
I do, however, understand if someone uses it with the meaning "something that wasn't intended by the developers".
Also, does it have to be an unintended exploit?
If I write a C++ program, compile and run it on this computer, would you consider it ACE? Technically speaking it is, but it's not what's usually meant by the term in the context of tool-assisted speedrunning.
The key word here is arbitrary -- that implies that you must be able to make the program do anything (that the computer is physically capable of doing of course).
Anything?
Is not being able to alter the stats of all the enemies, how many spawn, items, etc... not enough? You can arbitrarily change those things as long as it doesn't fail any constraints.
Wouldn't that be better called 'memory corruption' or 'memory editing'? ('data corruption' and 'data editing' if you prefer.) In pokemon RBY you can use the expanded inventory to meddle with things like the destination of the exit of the building, but that's not ACE unless you actually modify something that is later executed as code (or an equivalent effect like ROP/hijacking metaprogramming/self-rewriting code)
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Warp wrote:
Nach wrote:
Why does it have to be machine code? What if a game in an interpretive language, and you can inject code in said language?
I think "arbitrary" implies "anything you want, without limitation". A scripting language will always limit what you can do.
But the game itself would be limited in the exact same manner.
Warp wrote:
I do, however, understand if someone uses it with the meaning "something that wasn't intended by the developers".
There's a lot not intended by the developers that we exploit all the time.
I think the point here is "arbitrary code". Code that wasn't in the game to begin with, and players are adding it and getting it to run.
Warp wrote:
Also, does it have to be an unintended exploit?
If I write a C++ program, compile and run it on this computer, would you consider it ACE? Technically speaking it is, but it's not what's usually meant by the term in the context of tool-assisted speedrunning.
I don't see why it wouldn't be.
Where is our definition for TASs? I don't recall anyone ever precisely defining what it means, nor is it at Wiki: Glossary.
Edit:
Patashu wrote:
Wouldn't that be better called 'memory corruption' or 'memory editing'? ('data corruption' and 'data editing' if you prefer.) In pokemon RBY you can use the expanded inventory to meddle with things like the destination of the exit of the building, but that's not ACE unless you actually modify something that is later executed as code (or an equivalent effect like ROP/hijacking metaprogramming/self-rewriting code)
It is being executed as code. Just not the primary code.
Unlike just plain memory corruption, one is actually adding new code to the game, not just directly modifying some memory somewhere.
Also, in light of what you're saying, for my interpretative language game example I mentioned earlier, you would consider that "arbitrary code" if you get it to run your own?
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
I'm not familiar with that.
The example I was thinking of was a web browser game which has an input box somewhere, where the game is expecting a finite set of responses and runs it through eval(). Go beyond that finite set of responses the game is expecting, and you can execute anything, and even add your own code to the game, complete with a brand new debug console of your own creation to modify or add new things on the fly on a whim.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Is not being able to alter the stats of all the enemies, how many spawn, items, etc... not enough? You can arbitrarily change those things as long as it doesn't fail any constraints.
That's arbitrary memory manipulation, not arbitrary code execution. I'm not sure where "code execution" is unclear. If you cannot do turing complete computations, you haven't been executing code.
Nach wrote:
Why does it have to be machine code? What if a game in an interpretive language, and you can inject code in said language?
There's the grey area. We need to make a distinction between the application that's running, and the computing platform the game is running on. For the SNES, the platform is easy to determine: it's the hardware. For a PC, the platform could be the hardware. Or maybe just an OS. Or the browser. Flash. The Unreal engine. Whatever.
The existing ACE TASes gain full control over anything running on the platform, but do not change the platform (since it's hardware). I think the same distinction is useful for software platforms, too.
My personal criterion for "arbitrary" would be: can you use it to replace the game with SMB1, Tetris or any other game available on the platform? If you can't, it's not arbitrary enough.
To do so, you must exercise enough control over the platform to
stop the previously running game from executing
execute your own turing complete computations
load or create new game assets, including images and sounds
have full access to the platform's inputs (gamepad, keyboard, mouse, ...) to load new data
have full access to the plaform's outputs, usually video and sound
So for a HTML5 game, executing arbitrary JS should suffice: from there, you can replace the DOM, remove all existing code, listeners and timers, load new images and audio from data:// URLs and start a new game. You would not need to escape the javascript sandbox, the browser's sandbox, the OSs sandbox and then reboot into a hurd kernel. (Bonus points if you can though!)
Now you're asking: can your game engine qualify as a "computing platform" with the SQL accessible data being the "game"? I doubt it. From the way you describe it, the game inside the engine is modeled using declarative data, not code, so it's not a computing platform one can run applications on. If the "platform" is too weak to run any code, it's certainly unfit to run "arbitrary code".
Remember: if you're first looking at the things you can control, and then define the "platform" and "game" in such a way that the latter coincides with the things you control, then you might be stretching the terms.
Nach wrote:
Also, does it have to be an unintended exploit?
Finding and showcasing unintended exploits is a big part of the fascination of the category. Otherwise, it's no different than programming your own executable and running that, and I think we already have a demo scene for that.
I won't argue whether or not we could still call it ACE, I'm just saying that it wouldn't be publisheable here if it didn't feature a huge exploit. The movies we have all feature a central element: the ability to surprise the viewer. Typing something into visualbasic and then *gasp* running the code? Not very surprising, not interesting, nothing this site would care about.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
I agree with pretty much everything you said until:
Tub wrote:
Nach wrote:
Also, does it have to be an unintended exploit?
Finding and showcasing unintended exploits is a big part of the fascination of the category. Otherwise, it's no different than programming your own executable and running that, and I think we already have a demo scene for that.
I won't argue whether or not we could still call it ACE, I'm just saying that it wouldn't be publisheable here if it didn't feature a huge exploit. The movies we have all feature a central element: the ability to surprise the viewer. Typing something into visualbasic and then *gasp* running the code? Not very surprising, not interesting, nothing this site would care about.
If the application was being run was visual basic, then no, not very surprising. But if it was a typical game, and someone abused the "enter your name" screen to start typing in a program, even if it was intentionally designed that way, I think it'd still be *gasp*.
I think this point becomes stronger as the built-in exploit is placed somewhere further into the game and completely flies under the radar as an avenue for exploitation.
I also disagree with you that something has to be surprising to be published. I think most runs on this site aren't published for a single moment of shock value, but rather a continuous stream of impressive play.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
I'm fine with developers intentionally setting things up so that there are exploitable weird edge cases, especially if the exploits are "elegant" (here I'm specifically thinking about things like the Super Metroid mockball, which was probably unintentional but just looks so smooth when executed). Basically these then become "secret techniques" that skilled players of the game can take advantage of. Arguably they aren't glitches or exploits since they were intentionally coded into the game, but the concept is similar -- you have to be very dedicated to examining how the game works if you're to discover them and make use of them.
Pyrel - an open-source rewrite of the Angband roguelike game in Python.
Ok, I'll give you that running code inside a restricted environment can be a good challenge and may be fun to watch. And if the intended exploit is kept hidden enough, it may surprise a couple of viewers.
One of the questions every ACE run must answer is the following: does the run benefit from using the initial game as a vehicle?
Glitching a game with clever tricks can usually give a positive answer here: the breaking of the game is interesting to watch.
Watching frantic typing in Visual Basic before starting a nice demo is a negative; the parts before executing the payload aren't interesting; one should instead TAS the resulting executable directly.
Running code in a name input screen isn't really impressive when you know there's an intentional loophole in there. I can also run arbitrary code in Word by doing File -> Open -> navigate to Visual Basic -> right click the executable -> open -> enter code -> run. That'd certainly surprise some viewers ("Wait, you can right click?"), but when you know the trick, it's not impressive at all.
A great TAS will stay impressive even when the viewer knows how it's done. Finding out that a loophole was intentional usually diminishes that (not unlike finding out that an impressive run used cheat codes or easy mode), thus raising doubts whether the chosen vehicle adds enough to the run to be worthwhile. I concede that ACE runs using intentional glitches can be publishable, but they'd have to be held to higher standards to make up for that flaw.
/edit: y'all will have to agree with me, since this is my post number 1337 o_O
I don’t think you can call it ACE if the game itself lets you do that by intention. It’s the point of the game– the devs put it there– what are you calling it out for? (Ok I’m sure you don’t actually mean for it to be the point of the game, I’m just saying.)
If you define the term too loosely, it loses its meaning and becomes just an umbrella term for whatever you want it to mean, and that's not very useful.
Compare, for example, to how the term "luck abuse" was used prior to the overhaul of the tags. Since the term was extremely loosely defined, you could technically speaking apply it to almost every single run ever submitted. Which means that it was a rather useless term.
If you define "ACE" too loosely, you could argue for almost anything. The run jumps to the end of the game via glitching? ACE, if we loosen up the definition enough. The run causes items to appear in your inventory that shouldn't appear there? Same thing. And so on, and so forth.
In fact, if you loosen it up enough, it would simply become a synonym for "glitch".
Moderator, Senior Ambassador, Experienced player
(898)
Joined: 9/14/2008
Posts: 1007
Warepire wrote:
Correction: I am not thrilled at all about the name.
What name would you pick, then?
I ask because I was trying to figure out how to pitch the idea of a repeat of a Masterjun-style run for AGDQ 2015 and I finally had to settle on the phrase total control because the SDA community is not familiar with (and probably not interested in) the nuances of what Arbitrary Code Execution means. I'm not convinced that what we have here is total control (for that to happen, the TAS'er would need to, in my opinion, be able to issue the prohibited DROP command or something equally obviously demonstrating total control of the entire environment). What we have here is tantamount to a debug panel. Perhaps In-game Developer Environment Access (IDEA)? ...I might be bad at acronyms here. :)
For a "game" that incorporates the use of any kind of language as a deliberate in-game mechanic, ACE is not a meaningful category if used to denote the use of such a feature. The ACE category should be reserved for the situation where an algorithm that is input by the user is executed or interpreted by a Turing-equivalent machine in contradiction with the intentions and expectations of the software author. Preferably, the code that is being executed should be machine code.
I also think that any game that is deliberately designed specifically to blur the rules used in judging TASes would most likely be considered a bad game choice for TASing.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
To HHS and others, I think you're missing the point.
I'm not looking to make a certain language as something a user needs to use, and this is how a player plays a game.
I'm purposely adding a back door for those who find it, and it has nothing in the slightest to do with playing the game normally.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
That gets into what the definition of a bug is. Does it count if it is an intentionally implemented bug? What if the developer knows about it, are capable of fixing it, but don't? What if they know about it, would fix it if they could, but they lack the resources? The latter category is common in commercially-released software, as bugs can be deemed too hard to reproduce to be worth bothering with.
Pyrel - an open-source rewrite of the Angband roguelike game in Python.
For me ACE means
0) unintended - you can argue endlessly about intention
1) arbitrary - do you have some freedom here?
2) code - is it code? yes!
3) execution - is it executed? yes!
Then it's ACE.
Tub wrote:
If you cannot do turing complete computations, you haven't been executing code.
lol, I even doubt whether you know what is "turing complete computations".
You can run "turing complete computation" in SQL.
Tub wrote:
My personal criterion for "arbitrary" would be: can you use it to replace the game with SMB1, Tetris or any other game available on the platform? If you can't, it's not arbitrary enough.
Ha! Gotcha! "If you can't, it's not arbitrary enough." But, it's still arbitrary! Not enough, maybe, for something. But still, you can write whatever you want, but something won't run. Same thing in any language even in asm, you can write some bad opcodes, and they won't work. You can write some bad code in Javascript, and in certain conditions, it will stop. (exception/error occured).
Also, there are many of filters for user input to preventing injections, but still, hackers making exploits writing code with such restrictions: no spaces/no NOP byte. There are even special tools that generate machine code without restricted bytes/patterns.
Tub wrote:
To do so, you must exercise enough control over the platform to
stop the previously running game from executing
execute your own turing complete computations
load or create new game assets, including images and sounds
have full access to the platform's inputs (gamepad, keyboard, mouse, ...) to load new data
have full access to the plaform's outputs, usually video and sound
I would create tag "platform total control". :D
STOP! Who cares how to call it?
Is there run that needs tag? Nope, you talking about game.
Replay first -> tag after! :D
If this "hole" will make fun, then why not?
