Ram watch is an important tool when creating tas movies. But are the tools up to the task?
The dominant systems right now only allow individual variables at fixed locations. This was fine ten years ago. But games are more complex than that. Games use arrays of structures. They use fancy things built using points like linked lists and trees.
My observation is that the problem has already been solved. We just need to separate the definition of the tool from what we use it for.
As far as I can tell, ram watch is the same as using a debugger to inspect memory in a program. And debuggers have data formats that can cope with the complex reality that programs create. We have formats like PDB and DWARF that at least attempt to deal with the complexity.
Why aren't we using tools that can handle situations above the patently trivial ones?
Joined: 3/2/2010
Posts: 2178
Location: A little to the left of nowhere (Sweden)
Do you want to implement that, I was thinking of writing something like that for Hourglass and gave up pretty quickly when it came to finding a way to display everything in a decent manner, not even Visual Studio can display this in a decent manner.
RAM Watch itself is a bit obsolete anyhow, what should be done instead is to make LUA easier to use, and maybe allow a LUA console variant instead of only overlays like today.
Programming languages like LUA* is nice solution for more complicated data. But it shouldn't be needed for situations with just a little complexity. An array of structs is easy to deal with. Linked lists can usually be dealt with nicely too. People shouldn't need to be programmers to deal with just a little complexity.
And programming languages doesn't describe the data structures. It's not a format for exchanging data format definitions.
*LUA specifically is a horrible language in my opinion. Other languages are better.
Joined: 3/2/2010
Posts: 2178
Location: A little to the left of nowhere (Sweden)
You kinda missed my point with that we shall make LUA easier to use. The whole idea of that point was to device an auto-gen for LUA scripts that are merely RAM Watches.
If we find the data we're looking for, we just need a way to display it. And even Visual Studio fails to display anything nicely when you have such a common combined data structure as a linked list of structures, so doing it with a GUI like the current RAM Watch window, is going to be really hard once you go beyond a simple structure like a plain structure of 4-5 variables, of which none are pointers.
But if you have a good way to display such things, then by all means provide a design proposal.
I am not a big fan of LUA either. If you can come up with a better scripting language that's good for these kinds of things, I'd like to hear it. JavaScript isn't one of them. And Python is freaking huge to package, over 20 megabytes, I also don't know if there's any good Python into C/C++/C# integration.
In my opinion, in order to display something more complex data structures (or those larger than a few variables), some kind of draw-rendering should be done instead.
Because, really, a plain structure of just a few variables was starting to get obsolete already on the NES.
I know Tompa disagrees with the next part, and probably some others as well, but with the amount of data you need to track today, in my opinion one really wants it attached to the stuff it belongs to, and then you're writing a LUA script already anyway.
Linked lists can be converted into arrays for the purpose of display. Fancy things like iterators and what not can solve that problem.
But sometimes the game really is complex. You need trees or even webs to show how pieces are interconnected.
Linked lists can be converted into arrays for the purpose of display. Fancy things like iterators and what not can solve that problem.
But sometimes the game really is complex. You need trees or even webs to show how pieces are interconnected.
We understand we can simplify complex data structures. The problem is that the tools needs to be flexible enough to read whatever data structure we throw at it. Sure the idea seems simple but the GUI needs to be able to display every possible variable and the tool taught how to read everything. This is almost impossible to do and if it was, would require a ton of programming to cover every edge case. You will have to sacrifice several features to reduce the complexity to a manageable level (even Cheat Engine hasn't been able to pull off everything in your request without scripting).
I would much rather build my own cases into the emulator/tool than try a "one size fits all" tool. There is just too much to anticipate and I have gone as far as writing python scripts to analyze memory dumps of individual frames.
Can it do every case? No. But a few common ones? That's reasonable.
Go for the important parts first. Allow some sort of extensibility to let people cover rarer cases if they feel like.
Linked list: generic sequence container, commonly used for things like entity lists
Struct: Used for everything that needs more than a single number to describe it.
Array: Multiple identical items in a streak
Pointer to sub object: Item that should be considered a part of the main object, but for some reason isn't stored with it.
Array of struct: Multiple identical items in a streak that need more than one number each. The entity list often uses this setup.
Tree: Sorted lists, often used for various search indexes. Also used for hierarchical data, like the DOM in html.
Pointer to array: Commonly used when the array is variable sized and must be allocated separately.
Union: Multiple different variables sharing storage location, only one can be used at any time.
Tagged union: A combination of an union and a variable tracking which union field is currently valid. Frequently used to implement variable type variables.
Have you considered letting users define the data structure themselves, and then apply those user-defined structures to addresses in memory?
This doesn't address (no pun intended) all cases, but it does mean that when you figure out what one particular thing is, you can duplicate it by pointing the second one at a different memory address.
I am still the wizard that did it.
"On my business card, I am a corporate president. In my mind, I am a game developer. But in my heart, I am a gamer." -- Satoru Iwata
<scrimpy> at least I now know where every map, energy and save room in this game is
Joined: 4/17/2010
Posts: 11475
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Linked lists (and probably most of the rest, never tried) are a breeze in Lua, bonus points for displaying only what one needs, while all kinds of units can be tracked with no real speed loss. But lua isn't good at creating dialogs, while you might very well need to not spoil the game screen. So... what about Lua driven dialog for displaying all that crap in text and pictures?
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
