Oh, didn't think of the cheat table folder. Yeah, that'd break it. Perhaps I should look into making the require() paths more explicit for everything other than games/layouts. Thanks for finding the issue.
Fog: Can I see gamename.lua up to the point where the local dolphin is set? And what Cheat Engine version are you using?
I've read that require() can return true if the required file (dolphin.lua in this case) does not have a return statement at the end of the file, but if it's finding the repository's dolphin.lua then that should be covered. And I'm asking the CE version in case different Lua versions have different loader behavior.
https://github.com/yoshifan/ram-watch-cheat-engine
I've posted about this Lua script framework a few times in the past, but I figured I should make a proper thread for it instead of hijacking some other thread. That way I can field questions here, as well.
Anyway, I've given the framework a major update:
- New detailed tutorial
- You can now cleanly separate your own custom layouts from the framework code
- Reorganized a lot of things to enable writing shorter, more expressive code
Here's a video showing an example of a RAM display:
https://www.youtube.com/watch?v=Hri8f8Pgim8
Introduction based on the GitHub README:
RAM watch is a powerful tool for researching games - for modding, speedrunning, or just generally learning how the game works.
The tool-assisted speedrun community makes extensive use of RAM watch on emulated games. The most popular Gamecube/Wii emulator is Dolphin, and so far, Cheat Engine has been the most popular RAM watch solution for Dolphin.
In addition to supporting RAM viewing and scanning (and even modifying), Cheat Engine also includes a Lua scripting engine. This opens up many more possibilities for boosting research productivity, including:
More interactivity. With Lua scripting, you can create a separate Cheat Engine window with arbitrary GUI elements: buttons, text fields, and so on. You can record values to a .txt file as the game runs, and then paste results into a spreadsheet to make a graph (e.g. showing your character's speed over time).
More flexible RAM viewing compared to Cheat Engine's address list display. For example, you can set your font and font size, limit the number of decimal places in a float, and make the display update as often as once per frame.
It's easier to build upon previous results. Instead of entering the same pointer base for 10 different address list entries, you can save that pointer to a Lua variable and re-use that variable. You can run RAM values through formulas like sqrt(x^2 + y^2) and display the result.
This repository contains:
A Lua framework for writing custom RAM displays with the above features. Although the Lua framework can be used for any game, the main focus is on games running in Dolphin emulator.
A tutorial covering the basics of the Lua framework.
Advanced display layouts for a few games, most notably F-Zero GX and Super Mario Galaxy 1 and 2.
You can get a RAM watch script up and running without any prior knowledge of Lua coding. If you do know some Lua (or are willing to learn), this framework can help you write display scripts for any game you like.
石になったらほらムテキ。
上からおしてぶ[unreadable]ちやってほらステキ。
If you become a stone, see, you're invincible.
[Strike and] push from above, see, you're lovely.
* There's one character I can't make out so I'm not sure about the translation of the second sentence. I bracketed the bit that could be wrong.
Second sentence is:
上からおしつぶしちゃってほらステキ。
This is actually the same as JP Nightmare in Dream Land's pause-screen text, according to Kirby Wikia.
おしつぶす is to squash, crush, or flatten. English KatAM version includes 'crush' so we could go with that.
This is fantastic, thanks. I feel like I've pushed RAM search pretty hard in a few GC/Wii games by now, and have run into several situations where I couldn't quite get the research answers I wanted. I'll be sure to try this debugger sooner or later.
Ah, okay. It's been quite a while since I used Cheat Engine's address list for pointers, to be honest, so I'd forgotten just how inconvenient it was. Hopefully there will be an easier solution to deal with Dolphin pointers in the future.
Incidentally, if you want to try your hand at Lua scripting, I made a rough Lua framework for more customized Cheat Engine displays: https://github.com/yoshifan/ram-watch-cheat-engine
I wouldn't call it super easy to use, but if you can get things up and running, then it could save a lot of time with games that use pointers/dynamic memory.
Here is an example of a display I made with the framework.
It really worked with 7fff0000 for so much people that it really felt consistent enough. Even better, for Linux, I never had any problems for months and hundreds of reboot with always 2580000000. In fact, I can even confirm that I have another Windows laptop I did the 7fff0000 test and it worked. It just seems to be a place where dolphin always have its memory here. So tbh, it should work.....unless your dolphin version is old or 32 bit, then idk, but i can be sure that the static addresses I told does work.
From my Windows experience, Dolphin used to have changing game-RAM start addresses, but from the later 3.5 versions up to now (at least 4.0-8814), it seems like the valid start address(es) never change as long as you stick to the same version of Dolphin. And as far as I know, for any version in that same range, 7FFF0000 happens to be one of the valid start addresses.
Most of the Dolphin versions I've seen have most of the game RAM values duplicated in 4 addresses simultaneously. My test of Dolphin 4.0.2, for example, indicated that start addresses of 0B1A0000, 7FFF0000, FFFF0000, 13FFF0000 could all work just fine. I tend to use the 3rd address out of 4, since in my experience it was the only one that showed results on CE's "Find out what writes to this address" function. (Sometimes disabling those writing instructions is useful to figure out what an address is.)
Here's my Windows experience with various versions and their valid start addresses, using 64 bit when available:
3.5-0, 3.5-663, 3.5-1188: Changes if you quit your GC/Wii game and then open a game again. Addresses I've seen are 0B3F0000, 7FFF0000, FFFF0000, 13FFF0000, 180070000, 200070000, 240070000, 242070000, 2C2070000...
3.5-1696: 0C150000, 7FFF0000, FFFF0000, 13FFF0000. This version also might be subject to changing addresses, I don't remember exactly.
3.5-2302: 0B3B0000, 7FFF0000, FFFF0000, 13FFF0000
4.0.2: 0B1A0000, 7FFF0000, FFFF0000, 13FFF0000
4.0-1997: 08750000, 7FFF0000, FFFF0000, 13FFF0000
4.0-2826: 095F0000, 7FFF0000, FFFF0000, 13FFF0000
4.0-3599: 09940000, 7FFF0000, FFFF0000, 13FFF0000
4.0-4191: 7FFF0000, FFFF0000, 13FFF0000. Yes, only 3 in this case.
4.0-5702: 7FFF0000, 27FFF0000, 2FFFF0000, 33FFF0000
4.0-8814: 7FFF0000, 27FFF0000, 2FFFF0000, 33FFF0000
-----
Overall, I agree that pointers aren't needed for the start address, at least so far. However, often the GC/Wii game itself will use pointers for important addresses.
In F-Zero GX, a bunch of interesting values like position, orientation, in-race vehicle stats, etc. will move to a different location for different modes (Time Attack/Practice, Grand Prix, Story, Versus). In Super Mario Galaxy, position values move whenever you load a different area - not just for different galaxies, but even different levels of the same galaxy! When the values have over 50 different possible addresses depending on where you are in the game, it can get really annoying to create an address list for each case.
So in that case, it makes sense to find a pointer. A pointer, for our purposes, is just a RAM value whose purpose is to track the address of another RAM value (or a block of multiple RAM values). In general, the pointer's value isn't going to be exactly the same number as the address you're looking for. But whenever the address moves by X bytes, the pointer's value should change by that same amount, X. That's the principle we use to scan for pointers.
To find pointers, see this post from RachelB:
... find the address once, save state, then do whatever to make the address change, then find it again, and save state again. From there, search for 4 byte value > 0x80000000, load the other save state, then filter by changed by the difference between the two addresses.
To clarify, once you have those savestates:
1. Load your first state. Let's say you're looking for position, and position is at address 800 here.
2. Search for 4 Byte Big Endian in the address range you're interested in. If you already specify your RAM range in Memory Scan Options like aldelaro does, just scan for 'Unknown initial value'.
3. Load your second state. Let's say position is at address 8D4 here, which means the address increased by D4.
4. Scan for "Increased value by..." with D4 in the box (with the 'Hex' checkbox checked).
5. Now, any scan results that are changing constantly (when you don't expect the position address to change) probably aren't what you need. Use some Unchanged value scans to prune out those bad addresses. Eventually you should be left with a reasonably low amount of addresses. If not, you might want to add a third savestate to your procedure to increase your confidence in picking a pointer address.
6. Pick one of those results to use as your pointer address and test it. For example, let's say that when position was at address 800, this pointer had value 600; and when position was at address 8D4, this pointer had value 6D4. In this case, the position address is (pointer value + 200), so you'd use a pointer offset of 200.
-----
Other than that, I highly appreciate the detailed tutorial, aldelaro. I'll definitely be linking it to anyone interested in researching GC/Wii games.
Oh wait, seems like there might be two separate issues here: (1) hanging on frame advance, and (2) unpausing by itself. OP mentions both, and that linked issue covers only (2).
But to be clear, I've had (1) happen without (2). My experience is also without input recording; I'm just frame advancing to test stuff.
Known issue, it's been marked as a release blocker within the Dolphin issue tracker.
I was looking through the issue tracker but couldn't find this. Could someone provide a link?
Personally I found revision 5732 to be fairly stable with frame advances. Problems were reasonably uncommon, though if there was a problem it usually ended up being a crash (with the emulation window becoming completely unresponsive). 7365, however, has this much more frequent occurrence where I try to frame advance, and it gets stuck temporarily without crashing.
Sometimes waiting a few seconds and then advancing twice gets it unstuck. Other times, I must click the window close button on the emulation window and then select No to the prompt to fix it (at that point it may or may not unpause emulation by itself).
Awfully saddening how such talented and helpful people can be otherwise so unlucky in life.
Despite her situation, it seems that she made more of a positive difference in 2 or so years than many people do in a full lifetime. We should certainly look to her as an inspiration.
If you're familiar with a RAM watch program like MHS or Cheat Engine, you may be able to use that to determine whether the delay is the emulator or the game.
There should be some RAM values in Dolphin corresponding to the inputs you make. For example, the main stick's X direction could be in a signed byte (values of -128 to 127) which is negative when holding the stick left, and positive when holding the stick right.
So you could search for this kind of value (keep moving the stick and scan for increasing/decreasing values), and then browse the nearby memory addresses to find the rest of the stick directions and buttons. Then watch these memory values to see if they react to your TAS input / keyboard presses with the timing that you expect.
Are they doing YouTube now as well? The final part of xsHIMEsx's Pikmin TAS got this: "This video contains content from Nintendo, who has blocked it on copyright grounds." (Interestingly, the Nico upload of this is still up.)
https://www.youtube.com/watch?v=-XboAbx1Noc
For what it's worth, the following Nico TAS rankings are on my hard drive right now:
2013 Parts 1, 2, 3
2014 Parts 1, 3
2014 months 1, 2, 4, 5, 7, 8, 9, 10, 11, 12
2015 months 1, 2
So if further takedowns occur and the creator doesn't re-upload themselves, these videos can potentially be re-uploaded.
This account has been suspended indefinitely
Because this account has done something to violate the Niconico Douga terms of use, this account has been suspended indefinitely.
If the act in violation of the rules is repeated, or if the administration deems a need for it, this account may be deleted without advance notice.
ISMin also seemed to indicate (on Twitter) that a lot of their videos were deleted.
The 4 videos MUGG listed are still viewable, even the 2 by ISMin; so I'm not entirely sure what a suspension entails. I'm guessing they're unable to upload new videos though.
For those who want to save more videos, there's a Firefox addon called Nicofox that downloads Niconico videos along with the comments that you see during the video. (It doesn't save the video description though, so you have to save that yourself in a txt file or something.)
Nicofox doesn't seem to be working with Firefox 36 and up, but it does work with 35 or below, including FirefoxPortable 35.0.1.
This Nico user's article (linked in the NeoGAF thread) notes that there was a similar occurrence in 2008, where many Mario romhack / "automatic Mario" videos were taken down.
This year, romhack and TAS videos are being targeted; from the article's graphs it seems that the romhack videos have been steadily targeted starting around mid-February, whereas TAS takedowns have been mainly the past day or so.
Green Plant: Spiral in 1'57"997 (Max Speed settings) by E-Dragon:
Link to video
He says he's working on an improvement, since he learned about more shift boosts by the time he was at Lap 3. But yeah, it already beats the RTA record by ~50 seconds.
I've added some more structure to the Lua script. Now the utility functions and Dolphin config are in separate files from the game-specific addresses and stuff. This should make things easier if you often switch between multiple games.
Go here for the instructions and Lua code, including example scripts for F-Zero GX, Galaxy 1, Galaxy 2, and Metroid Prime:
https://github.com/yoshifan/ram-watch-cheat-engine
On a side note, is there an official timing method for individual level stars? For a TAS I've thought of using
(first frame of star celebration)-(first non-white frame after cutscene of flying in)/60
unless there is already an existing method.
Well, when ILs of this game were popular at the SDA forum, the rule was:
"Timing begins from the moment Mario or Luigi gains control and ends upon collecting the Power Star."
But that rule's start and end points are both a bit vague. Back then, I personally timed it from the first frame that I could notice Mario/Luigi's starting-to-run animation (still ambiguous), from the first frame of the star celebration cutscene.
One complication is that in an actual any% run, most stars (except Hidden stars) will require you to watch the beginning cutscenes because it'll be your first visit to that star. When you have to watch those cutscenes, Mario/Luigi goes through a long landing animation. But for ILs from a completed file, you can skip those cutscenes, and skipping the second cutscene means that you don't have to do the landing animation. Sometimes you'll even start at a slightly different spot if you skip the cutscene (see Dusty Dune 3 for an example)!
It's probably too hard to be completely fair between cutscene skips vs. no cutscene skips, so I'd recommend just doing whatever SMG2's in-game level timer does. This timer is not shown onscreen during the level, but you can see it (as a frame count) with RAM watch in Dolphin. I don't remember exactly, but I think SMG2's timer does roughly what you described, bihan.
The co-op jump is used extensively in the current any% WR by NEG_taro. Check it out in this Nico playlist.
It's been a while since I tried it myself, but I believe you're supposed to point the P2 Wiimote at Mario, and then press A on both Wiimotes at the same time (within a few frames of each other). As you'll see in his camera, when he needs to do co-op jumps, NEG_taro actually uses his foot for one Wiimote (the one he uses just for pressing A, not for aiming).
The co-op pointer is also able to stun enemies. See Good Egg 1's boulder planet, or Drip Drop's eels for good examples. It can also manipulate the bouncy plants like on final Bowser phase 2.
---
Also, since the any% route and starbit collection were asked about earlier (something along those lines at least):
My any% route's Hungry Luma stars are Sweet Sweet (Terrace), Sling Pod (Fountain), and Drip Drop (Kitchen). In my segmented Luigi any% run, these were around 33, 21, and 22 seconds respectively, from beginning of control to the star cutscene. These star times are easily some of the fastest in the game.
The other Hungry Lumas are Bigmouth, Sand Spiral, Boo's Boneyard, and Snow Cap. Sand Spiral and Bigmouth are each 45-60 seconds from what I recall, and can't be sped up much to my knowledge. Boo's Boneyard and Snow Cap are at least as long, and they also require getting the Grand Star from the Engine Room, which takes a while due to the really long cutscene of the Garden opening (and I didn't think the Garden stars were short enough to make up that time).
Some RTA players actually leave out Sling Pod and Drip Drop because they are hard to consistently get good times on, but they are excellent for segmented or TAS runs. (I'd love to see a Sling Pod TAS, by the way!) But even for an any% TAS, 3 Hungry Luma stars is still the maximum we'd probably see, for a couple of reasons. First, it takes a bit of time to get to the Hungry Lumas and feed them, offsetting the benefit you get from the short stars themselves. Second, every time you finish a level with star bits, you have the "You got some star bits!" dialog at the end of the level, which usually* wastes some time.
(*After you finish a star, even if "You got a star!" is the only dialog present, the game actually makes you wait a couple of seconds before you can proceed to the Save prompt. So a couple of seconds of dialogs won't hurt you at all. But if your dialogs last longer than that, you will be wasting time. So if you get other dialogs like unlocking a new galaxy, a new storybook chapter, etc., a starbit dialog will hurt you more. For the same reason, you should usually try to finish with 0 coins if you can.)
I might try to help fill out an (RTA) IL table for this game. Even if many of them would just be me cutting parts out of my segmented run, I guess that would help for convenient reference.
(Ignore the bottom right part, that's just my desktop)
Works up to 4.0-2826 now. The Lua script is in this pastebin.
I've set it up so that it's possible to change just one variable, DolphinVersion, in order to get the script working (as long as you have one of the exact Dolphin versions listed there). Otherwise, you will have to look for a few addresses like in Masterjun's original script. For convenience, I plan to add a version that has Wiimote + Nunchuk TAS Input, once RachelB has merged the Nunchuk input code.
Like Masterjun's script, all the addresses under the "function debugger_onBreakpoint()" part are for one game specifically (in this case, Super Mario Galaxy 2). The stuff above that it is applicable to any game. All in all, I made the script quite a bit longer, but that's largely because of the SMG2 stuff and my comments...
I got a little closer to making the script work on Dolphin 4.0-2826 + OpenGL:
I guess you were getting at this, solarplex, but I realized the Lua script's breakpoint just needed to be on an instruction that ran once every frame. There's already a decent amount of addresses in Dolphin.exe incremented once every frame (without updating to the commit you mentioned), so I used one of those addresses. Then I used Cheat Engine to see what instruction wrote to that address. Then I used that instruction for the breakpoint. But...
The display still isn't showing my text. There's a couple of possibilities: (1) I just made a silly mistake in the script somewhere (quite possible), or (2) The instruction I'm breakpointing on is run before the Dolphin code that normally updates the statistics, so my text keeps getting overwritten before it can be displayed. I tried a few different instructions, but I still can't say for sure whether (2) is being a problem. I don't know how to guarantee that my breakpoint is hit after the normal statistics are written. (At least, that problem makes sense in my head; maybe I'm just off the mark in general.)
Edit: I've just realized that Cheat Engine has more than enough GUI capabilities to make overwriting Dolphin's text unnecessary. So that's probably the route I'll take from here. Now (2) above is not a concern, and TexturesCreatedAddress is out of the picture. Just need to make sure this thing updates exactly once per frame, now.
Two other uses in the Any% route, off the top of my head:
- In Melty Monster S, The Chimp's Bowling Challenge, maybe you can go from platform to platform with the rock multi-jumps instead of the ramps. I haven't tried this, but if it works (the pins still spawn and everything), then you won't need to re-transform between each set of pins.
- In Boulder Bowl 1, Rock and Rollodillo, you can do rock multi-jumps to go directly from the comet medal to the planet with the chomps. (I've actually done this when experimenting with the rock jumps with frame advance.) Also, this isn't exactly about rock multi-jumps, but I had a couple of boss strat ideas that are good for TAS: see this video.
There's actually a bigger issue with the script in recent Dolphin versions - the scan to get the ArrayOfBytesAddress doesn't seem to work. (Works in 4.0.2, doesn't work in 4.0-1997-x64, doesn't work in 4.0-2826-x64.)
Here's the array of bytes that Masterjun's script says to scan for:
-- 48 63 D8 48 03 DF 48 83 7D C0 10
-- if you have 64 bit Dolphin
--
-- 83 C4 0C 83 7C 24 1C 10 8D 3C 06
-- if you have 32 bit Dolphin
In 4.0.2 and below, the bytes correspond to three particular compiled instructions in Dolphin, starting with the highlighted "movsxd" here:
But again, scanning for these bytes in more recent Dolphin versions turns up nothing. My guess is this part of the Dolphin code has changed since then, and now there may be a different array of bytes we can scan for to make the script work again. But what would that new array of bytes be? What instructions am I looking for here?
石になったらほらムテキ。 上からおしてぶ[unreadable]ちやってほらステキ。 If you become a stone, see, you're invincible. [Strike and] push from above, see, you're lovely. * There's one character I can't make out so I'm not sure about the translation of the second sentence. I bracketed the bit that could be wrong.
(Ignore the bottom right part, that's just my desktop)
Works up to 4.0-2826 now. The Lua script is in this pastebin.
I've set it up so that it's possible to change just one variable, DolphinVersion, in order to get the script working (as long as you have one of the exact Dolphin versions listed there). Otherwise, you will have to look for a few addresses like in Masterjun's original script. For convenience, I plan to add a version that has Wiimote + Nunchuk TAS Input, once RachelB has merged the Nunchuk input code.
Like Masterjun's script, all the addresses under the "function debugger_onBreakpoint()" part are for one game specifically (in this case, Super Mario Galaxy 2). The stuff above that it is applicable to any game. All in all, I made the script quite a bit longer, but that's largely because of the SMG2 stuff and my comments...
But again, scanning for these bytes in more recent Dolphin versions turns up nothing. My guess is this part of the Dolphin code has changed since then, and now there may be a different array of bytes we can scan for to make the script work again. But what would that new array of bytes be? What instructions am I looking for here?