Posts for Sand

1 2
8 9
Post subject: Route changes, virtual memory
Sand
He/Him
Experienced Forum User, Published Author, Player (147)
Joined: 6/26/2018
Posts: 209
I started experimenting with changes to the route and measuring the effect they have on completion time. After a few easy wins, I encountered an interesting situation: a faster way to do one segment that ends up being slower overall by the end of the run. After investigation, I suspect that the cause is the game's virtual memory system, which swaps content into memory from the floppy disk. Here's a summary of the effects of changes so far:
Change
Time difference
Comment
Baseline (Post #538557)
25139 frames
Change ulysse to odysse
−34 frames
?
Combine two-part commands (e.g. put solid;caseput solid in case)
−1479 frames
Use more conventional words (pealring, heapcoal)
+3 frames
?
Stack directional commands (w;s;e;uw.s.e.u)
−781 frames
temple rather than pray after getting coffin
+152 frames
???
The first change in the table, changing ulysse to odysse, I did for reasons of canonicity. (The cyclops is Greek, so he should know the Greek name, not the Latin name; also the acrostic in the black book spells out ODYSSEUS.) The strings are the same length, so it should be an inconsequential change—but it saves 34 frames. I am not sure, but the reason may have something to do with the virtual memory concerns I'll describe below. Joining two-part commands, as suggested in Post #537196, proves to be a big win on the Apple II implementation. Even thought it's more keystrokes, it saves a round trip through the parser and interpreter. It possibly also avoids some code paths that then don't have to be paged into memory. I thought the words peal and heap in the Gym Slow route were a bit weird and wanted to change them to the more normal ring and coal. This again should be an inconsequential change, but it results in a minuscule loss of frames. Why? I'm not sure. Stacking commands with . also turns out to be a big win. But, now that I've seen how it looks in a movie, I'm not so sure I like it and may decide to restrict myself to one command per input line. The last one is the interesting one. After getting to the gold coffin, you need to use a warp to return it to the trophy case, because it's too large to get out any other way. The Gym Slow route uses the pray command to warp back out to the forest, then re-enters the house from outside. I had the idea of instead using temple to warp to the thief's hideout, then take the secret passage back to the living room. This is how it breaks down:
pray frames
pray command
temple frames
temple command
Difference
4334
open it
4334
open it
0
4511
u.s
4511
u
0
4623
pray
4547
tEmPLE
−76
4721
e.s.e.w.w
4656
d.E.e
−65
4940
put solid in case
4814
put solid in case
−126
5146
open trap
4983
open trap
−163
By the time the two routes reconverge with put solid in case, the temple route is 126 frames ahead. But the lead doesn't remain constant: you can see that in the next command, it grows to 163 frames. The temple route's lead grows and shrinks until eventually it falls behind around frame 15000 and finishes over 150 frames slower. The left side of the graph below compares the long-term timing differences of the two slightly different routes. Two subgraphs. The left is a line graph with the title "Frames to reach each input line". The x-axis is "line" and the y-axis is "frames". There are two series: PRAY and TEMPLE. The two series are identical until about line 250/frame 5000, then TEMPLE starts to have slightly fewer frames per line. At about line 800/frame 15000, PRAY moves ahead and finishes at about line 1600 about 150 frames faster. The subgraph on the right is titled "Swaps". Its y-axis is "frame" and is aligned with the left subgraph. Two columns of a few hundred horizontal lines, colored to match the PRAY and TEMPLE lines in the other graph, show at what frames swaps occurred. They are identical up to about frame 5000, then they diverge. What's going on? I thought about it a lot. My best guess is that is has to do with the virtual memory system of the Z-machine. The game's story file is over 80 KB large, but the Apple II has only 64 KB of memory (and not all of that can be freely used). So what the game does is swap information from the disk into memory on demand. A page table keeps track of what pages in memory correspond to what pages on disk. Once the page table is full, every page that is swapped in from disk necessarily evicts a page that already existed; if that other page is needed again later, it will have to be swapped back in. I suspect that the praytemple route change, even though it uses fewer and faster commands, perturbs the contents of the page table in such a way that eventually requires doing more swaps and spending more time waiting for the disk. The right-hand side of the graph above shows the frames at which swaps occur in the two routes. They are identical up to the route change; then they diverge. There are some similar patterns, but one is not simply the shift of the other. In total, the pray route uses 655 swaps and the temple route 670. This kind of thing is hard to optimize, when a small change can have far-reaching and unpredictable effects. It does suggest a heuristic, which is to try to minimize the ranges of addresses that are accessed. I haven't checked, but that may be what's going on with the ulysseodysse change. In the source code, ODYSSEUS is the main syntax element, while ULYSSES is marked as a synonym. It may be that these two words are stored in different tables in memory, and that odysse requires only one memory access while ulysse requires two. That's just a guess, I haven't checked. I plan to try switching some commands to use the main word rather than synonyms, even when the synonym is shorter. It may explain why pealring had heapcoal an effect. In this case, though, RING and COAL are the main syntax elements. Maybe the lists of synonyms is near a page boundary, or something. I haven't checked. Differences in page tables is also the kind of thing where changes elsewhere in the route may end up making the temple variant faster after all.
Post subject: 10000
Sand
He/Him
Experienced Forum User, Published Author, Player (147)
Joined: 6/26/2018
Posts: 209
🥳 I was here! Yes vote too :)
Post subject: Automated RNG manipulation
Sand
He/Him
Experienced Forum User, Published Author, Player (147)
Joined: 6/26/2018
Posts: 209
Sand wrote:
I hope to automate this style of RNG manipulation to enable flexible experimentation with routing. As things stand now, if you wanted to try s.e instead of n.e to get behind the house at the beginning of the run, for example, it would desync all later random events. Automated RNG manipulation might also make it possible to play (what is effectively) the same run in either 40-column more or 80-column mode, or in either brief more or verbose mode. That would not normally be possible, because the timing, and hence the RNG, in the various modes would be different. But with a Lua script automatically adjusting the letter case of commands to make sure random rolls always come out the right way, it may be possible for the same route to work in either mode. One movie might have get eGg where the other has get EGg, but they would be effectively the same. Automated RNG manipulation may also make the run robust to timing changes in a future version of the emulator.
I've done the automation of RNG manipulation and used it to execute the same route as before under all 4 settings combinations of 40/80 columns and brief/superbrief. Here is how the times work out, from the fastest to the slowest settings:
ColumnsBrevityFramesTime
40superbrief187755:12.92
80superbrief198275:30.45
40brief234336:30.55
80brief251396:58.98
As you can see, 80 columns is about 6% slower than 40 columns, and brief mode is about 25% slower than superbrief mode. Even though it's the slowest combination, I plan to use "80 columns, brief" as the settings for the run, in a speed/entertainment tradeoff. Brief mode shows the room descriptions that are never shown in superbrief mode. 80 columns has distinct uppercase and lowercase letters, which is nicer to read and makes the mechanism of RNG manipulation visible. User movie #638973083207267679 is the movie file for "80 columns, brief". Here are encodes for all 4 combinations of settings: 40 columns, superbrief (5:12.92) Link to video 80 columns, superbrief (5:30.45) Link to video 40 columns, brief (6:30.55) Link to video 80 columns, brief (6:58.98) Link to video As before, the commands of the route are input by a Fennel script. The big enhancement in this revision of the script is automatic RNG manipulation by means of varying letter case. As I've discussed, the Apple II Z-machine interpreter's RNG is sensitive to timing at almost the cycle level, so any small change, such as from 40 to 80 columns or from brief to superbrief mode, requires redoing every manipulation. Luckily, the RNG state does not depend only on timing—it also mixes in the player's keyboard input. Whether letters are uppercase or lowercase doesn't matter to the game's text parser, but uppercase and lowercase letters do affect the game's RNG state differently. When the route needs a certain RNG outcome, it tries the command and checks if it worked; if not, then it backtracks, flips the case of an earlier letter, and tries again. (In 40-column mode, there's no visual difference between uppercase and lowercase, but this procedure is still happening.) For example, during the troll battle, we need a few random events to go our way:
  1. The troll must not attack us immediately on entering the room (68% chance)
  2. We must get a one-hit kill on the troll (2 in 9 chance)
  3. We must get the shortest combat result remark (1 in 3 chance)
Here are the case variations that lead to the desired outcome under each of the 4 combinations of settings. Case variation can reach back more than one command if necessary, as far as it needs to.
ColumnsBrevityCommand
40superbriefN;n;hit iT
80superbriefn;n;Hit it
40briefn;n;hit It
80briefn;n;hIT IT
Whenever you see capital letters, some kind of RNG manipulation is happening. If there's no obvious purpose to the manipulation, it's probably to prevent the thief from moving items on his circuit through the dungeon. Even though it's using the same route, this version of "40 columns, superbrief" is slightly faster than the one in Post #537196, because of the use of case variation rather than frame delays for RNG manipulation. How this looks in the code is, ranges of commands can be annotated with "must-not" and "must" events. As the script enters commands, it registers BizHawk events representing undesired or unnecessary outcomes. If, while entering the range of commands, any "must-not" events happen, or if, after entering the range of commands, not all of the "must" events have happened, the script backtracks and tries a different case variation. For example, below, we annotate the n command that enters the Troll Room with a "must-not" event for the VILLAIN-BLOW function being called. On the hit it command to attack the troll, we put a "must" event for the enemy being hit with a certain outcome (KILLED) and random remark (3).
(enter-commands
  [
   ; ...

   ;; Get the painting and go back to the cellar.
   "s"
   "e"
   "get"
   "w"
   "n"

   ;; Defeat the troll. The troll has a <PROB 33> chance of attacking as
   ;; soon as we enter the room, before we get a chance to act.
   [{:must-not [(make-check-exec-zaddr VILLAIN-BLOW-ZADDR)]}
    [
     "n"
     ]]
   ;; Require a one-hit kill with the shortest remark
   ;; "The troll takes a fatal blow and slumps to the floor dead."
   ;; https://github.com/historicalsource/zork1/blob/34cc828c4f/1actions.zil#L3574
   [{:must [(make-hero-blow-res-remark KILLED 3)]}
    [
     "hit it"
     ]]

   ; ...
   ])
The battle with the thief, near the end of the run, is the most RNG-intensive part of the run. Here, I relaxed the manipulation to require only certain combat outcomes, without also requiring the shortest random remark. This was just to speed up the manipulation. By my estimate, an optimal fight, including remarks, has about a 1 in 3000 chance. Ignoring remarks, it's about 1 in 60. It may be possible to do a more thorough optimization when the run is close to being finished. I also haven't checked to see if the slight difference in remark lengths actually makes a difference with respect to time. With automatic RNG manipulation in place, I now have freedom to experiment with changes to the route.
Post subject: An unsuccessful search for faster name payloads for game end glitch
Sand
He/Him
Experienced Forum User, Published Author, Player (147)
Joined: 6/26/2018
Posts: 209
I experimented with many different options for party member names in an attempt to find a faster credits warp than [4468] NES Final Fantasy "game end glitch" by AmaizumiUni, Spikestuff & DJ Incendration in 01:36.47. Long story short, I didn't find any. The names よるえき [0xaf 0xb2 0x8d 0x90] and ちごにむ [0x9a 0x4c 0x9f 0xaa] for party members 2 and 4 are the fastest of many alternatives I tried. Let's first walk through the exploit. Though 4468M is for the Japanese release, I'll use labels from the USA disassembly. The NES CPU's stack occupies 256 bytes from 0x0100 to 0x01ff. The ultimate cause of the game end glitch is the fact that the variable tmp_hi is located inside the stack area, at address 0x0110. tmp_hi is used during string formatting to store a pointer into a text buffer. Repeatedly climbing the stairs causes the stack to grow until it overlaps tmp_hi. When you press Start after filling the stack, the text formatting for the pause menu ends up storing the value 0x0302 in tmp_hi, overwriting the return address of an active stack frame. In this case, 0x0302 is a pointer into str_buf (which happens to overlap with the data structure ptygen that stores information about the party during party selection). When the stack frame returns, because of the overwritten return address, the CPU starts executing code at 0x0303. This address would be part of party member 1's name, except that the data for party member 1 data in ptygen has been overwritten, having been used as scratch space for string formatting. That scratch string data executes as harmless code and eventually reaches the part of ptygen reserved for party member 2 0x0310, which is where the first part of the exploit payload is stored. ptygen is not where party information is permanently stored. Once the game starts, that memory may be reused for other purposes, including string formatting, as we have seen. At the time of the exploit, the data for party member 1 has been overwritten, but the data for party members 2–4 is still intact. Some of the memory is controllable (party member classes and names), and some are not (e.g. screen coordinates). Here's what ptygen looks like at the time:
        00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f
0300             10 06 .. .. .. .. .. .. 7a 80 7a 80 00
0310    L2 00 N2 N2 N2 N2 15 .. .. .. .. .. .. .. .. ..
0320    .. .. .. .. .. .. .. .. .. .. .. .. 04 10 80 a0
0330    L4 00 N4 N4 N4 N4 15 18
  • .. bytes are omitted because they always get jumped over.
  • Ln and Nn bytes are the bytes we have control over:
    • Ln is a party member's class, 0x00 to 0x05,
    • Nn are the 4 bytes of a party member's name.
  • All other bytes are fixed.
The first few instructions starting at 0x0303 are some harmless branches and nops:
0303    BPL 0x030b
030B    NOP
030C    NOP #0x7a
030E    NOP #0x00
The first byte we have control over is L2 at 0x0310. The second byte is always 0x00 and cannot be changed, but then we get to execute the 4 bytes of party member 2's name:
0310    L2
0311    0x00
0312    N2
0313    N2
0314    N2
0315    N2
Starting at 0x0316, there are are some non-modifiable instructions that halt the CPU. No matter what alignment we start with, we will hit a 0x12 at address 0x031c which encodes a JAM instruction. Therefore, we are obligated to use the final N2 byte for a branch instruction (either BCC or BVC) to take the next byte 0x15 as an operand and jump ahead 21 bytes, to arrive at 0x032c. At 0x32c, there is a nop and a branch that doesn't get taken, and then we get to execute the class and name of player character 4:
032C    NOP 0x10
032E    BMI 0x02d0
0330    L4
0331    0x00
0332    N4
0333    N4
0334    N4
0335    N4
That's the extent of our control, the 10 bytes representing the class and name of party members 2 and 4. The L2 and L4 bytes are limited because they can only represent one of the six classes using byte values 0x00 to 0x05. The N2 and N4 bytes are more flexible, but even then we are limited to a palette of 90 bytes that represent name characters. Because the final byte of party member 2's name must be a branch instruction, we really have just about 7 bytes to work with.
The goal of the game end glitch is to jump to EnterEndingScene in memory bank 0x0d. For that, we want to jump to VictoryLoop in bank 0x0f. VictoryLoop loads bank 0x0d and jumps to EnterEndingScene.
C935    JSR EnterBattle_L   ; start the battle!
C938    BCC :+              ;  see if this battle was the end game battle

      @VictoryLoop:
C93A    JSR LoadEpilogueSceneGFX
C93D    LDA #BANK_ENDINGSCENE
C93F    JSR SwapPRG_L
C942    JSR EnterEndingScene
C945    JMP @VictoryLoop

      :
C948    JSR ReenterStandardMap  ; if this was just a normal battle, reenter the map
Because VictoryLoop is a loop, our jump target doesn't have to be exact. We can land at almost any address inside the loop and it will work. Even if we're misaligned with the intended instructions, it happens to still work out: the JMP at 0xc945 eventually runs and restarts the loop with proper alignment. And because the CPU's carry flag happens to be set at this point, we can even land on the BCC instruction before the loop. Therefore, we are looking to jump to any address between 0xc938 and 0xc945 inclusive.
We'd be happy with a jump target anywhere between 0xc938 and 0xc945. Unfortunately, we cannot just write a JMP instruction, because none of the bytes 0xc9 and 0x38–0x45 are available to us. The table below shows the naming bytes we have to work with, along with what instructions they encode:
0x8a あ0x8b い0x8c う0x8d え0x8e お0x48 が0x49 ぎ0x4a ぐ0x4b げ0x4c ご
TXA implANE #STY absSTA absSTX absPHA implEOR #LSR AALR #JMP abs
0x8f か0x90 き0x91 く0x92 け0x93 こ0x4d ざ0x4e じ0x4f ず0x50 ぜ0x51 ぞ
SAX absBCC relSTA ind,YJAMSHA ind,YEOR absLSR absSRE absBVC relEOR ind,Y
0x94 さ0x95 し0x96 す0x97 せ0x98 そ0x52 だ0x53 ぢ0x54 づ0x55 で0x56 ど
STY zpg,XSTA zpg,XSTX zpg,YSAX zpg,YTYA implJAMSRE ind,YNOP zpg,XEOR zpg,XLSR zpg,X
0x99 た0x9a ち0x9b つ0x9c て0x9d と0x57 ば0x58 び0x59 ぶ0x5a べ0x5b ぼ
STA abs,YTXS implTAS abs,YSHY abs,XSTA abs,XSRE zpg,XCLI implEOR abs,YNOP implSRE abs,Y
0x9e な0x9f に0xa0 ぬ0xa1 ね0xa2 の0x70 ぱ0x71 ぴ0x72 ぷ0x73 ぺ0x74 ぽ
SHX abs,YSHA abs,YLDY #LDA X,indLDX #BVS relADC ind,YJAMRRA ind,YNOP zpg,X
0xa3 は0xa4 ひ0xa5 ふ0xa6 へ0xa7 ほ0x7d ゃ0x7e ゅ0x7f ょ0x7c っ0xb9 。
LAX X,indLDY zpgLDA zpgLDX zpgLAX zpgADC abs,XROR abs,XRRA abs,XNOP abs,XLDA abs,Y
0xa8 ま0xa9 み0xaa む0xab め0xac も0x80 00x81 10x82 20x83 30x84 4
TAY implLDA #TAX implLXA #LDY absNOP #STA X,indNOP #SAX X,indSTY zpg
0xb0 ら0xb1 り0xb2 る0xb3 れ0xb4 ろ0x85 50x86 60x87 70x88 80x89 9
BCS relLDA ind,YJAMLAX ind,YLDY zpg,XSTA zpgSTX zpgSAX zpgDEY implNOP #
0xad や0xae ゆ0xaf よ0xb5 わ0xb6 ん0xc2 -0xc4 !0xc5 ?0xc3 ‥0xff  
LDA absLDX absLAX absLDA zpg,XLDX zpg,YNOP #CPY zpgCMP zpgDCP X,indISC abs,X
We do also have limited control over the L2 and L4 bytes, which represent party member classes and may take on any value from 0x00 to 0x05. The fastest option is to take the defaults, which are L2 = Thief = 0x01 = ORA X,ind and L4 = Red Mage = 0x03 = SLO X,ind. The Red Mage SLO can cause us problems in some circumstances, because it may change the value of the A register. We can work around the problem by setting and L4 = White Mage = 0x04 = NOP zpg, but that costs a few frames. The 0x00 BRK impl and 0x02 JAM instructions are not useful to us.
0x00 Fighter0x01 Thief0x02 Black Belt0x03 Red Mage0x04 White Mage0x05 Black Mage
BRK implORA implJAMSLO X,indNOP zpgORA zpg

The exploit payload of 4468M is clever and subtle. Because we cannot directly enter any of the jump destination address bytes we need, we must cobble together an address using bytes already in memory, on the stack and elsewhere. The general strategy is this:
  • Store a value in the A register that works as the low-order byte of a destination address in the range 0xc938 to 0xc945. Because we will eventually use an RTS instruction to do the jump, and RTS adds 1 to the address, A must actually be in the range 0x37 to 0x44.
  • Store a value in the X register that points to one byte before any of the multiple appearances of 0xc9 on the stack.
  • Transfer the value of the X register into the SP (stack pointer) register.
  • Push the value of the A register onto the stack using PHA. The stack pointer now points to an address 0xc9?? that works as a destination jump address.
  • Jump to the address on the stack with an RTS instruction.
It's a lot to accomplish in 7 bytes. Let's walk through it.
0310    01 00       ORA (0x00,X)    ; character class 0x01, (0x00,X) points to zero so there is no effect
0312    AF B2 8D    LAX 0x8db2      ; set A = X = 0x38, the value stored at 0x8db2
0313    90 15       BCC 0x032c      ; use the preexisting 0x15 operand, jump ahead to character 4 data
The 0x01 byte (representing the Thief class) encodes a two-byte ORA instruction, which has no effect given the current state of the registers. The exploit then spends 3 bytes on an instance of the undocumented/illegal instruction LAX, which assigns to the A register and the X register simultaneously. The final byte of party member 2's name must be used for a branch instruction (here BCC) to jump to party member 4's data—we have no choice. Assigning to A and X simultaneously is a good use of space, but it means we need a single value that works for both registers. A must be in the range 0x37 to 0x44, and X must point to one byte before a 0xc9 on the stack. Here's the relevant part of the stack:
        00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f
0130                            92 c9 01 18 00 0b 05 92
0140    c9 01 08 00 0b 05
There are only two values that satisfy both constraints: 0x38 and 0x3f. We can take either of these values from anywhere in memory. The payload takes it from address 0x8db2, which contains 0x38. (0x38, as a value for A and X, has a non-obvious advantage over 0x3f. The SLO (0x00,X) instruction that comes from from executing character class 0x03 (Red Mage) happens to leave the A register and processor flags unchanged when X = 0x38, but not when X = 0x3f. 0x3f can still work, but it requires changing party member 4's class to White Mage, which encodes a NOP instruction. This is the part that pirohiko marked "be careful" in Post #505719.)
0330    03 00       SLO (0x00,X)    ; character class 0x03, doesn't change A when X = 0x38
0332    9A          TXS             ; set SP = X, SP points to one byte before 0xc9
0333    4C 9F AA    JMP 0xaa9f      ; jump to code that will run PHA and RTS for us
The first byte of party member 4's name is spent on a TXS instruction to set the stack pointer. We have just 3 bytes of payload left, and we still need to do a PHA to push A on the stack and an RTS to do the jump. We cannot directly encode an RTS instruction (0x60 is not one of the bytes we have access to), but we can encode a JMP (0x4c). So what we can do is execute an absolute JMP to an RTS somewhere else in the code. But a JMP takes 3 bytes, which leaves no room for the PHA we also need. Therefore we need to jump to a place in memory that executes both a PHA and (eventually) an RTS. There's only one range of addresses that works, 0xaa93 to 0xaa9f. The payload uses 0xaa9f.
It's hard to imagine triggering the exploit any faster: start a new game, walk directly north to the stairs, walk up and down and open the menu a few times. Any improvement can only come from entering the party information faster. Is there a payload (a configuration of party member classes and names) that accomplishes the same exploit, but is faster to enter? That is what I tried and failed to find. My strategy was to keep the basic exploit technique, but generate a number of variations on the payload (that use different opcodes and addresses but have the same effect), then time each of those payloads using an automatic name entry script. These are some of the variations I tried:
  • LAX opcodes other than 0xaf, including 2-byte encodings.
  • Source addresses for LAX other than 0x8db2.
  • Destination addresses for JMP other than 0xaa9f.
  • BVC as an alternative to BCC.
  • Moving the PHA instruction into the payload, which gives more address freedom in the JMP to RTS.
  • Opcodes and placement of NOP instructions, in payloads that are short enough to leave room for a NOP.
4468M used a 3-byte form of the LAX instruction, opcode 0xaf, with an absolute addressing mode. The 2-byte operand is the address of a memory location that contains 0x38. Our repertoire of name characters gives us access to addressing modes of LAX whose encoding is only 2 bytes:
OpcodeInstructionBytes
0xa3LAX X,ind2 bytes
0xa7LAX zpg2 bytes
0xafLAX abs3 bytes
0xb3LAX ind,Y2 bytes
A search of the possible second operand bytes, using the values of the registers at the time of the exploit (X = 0x05, Y = 0x0e), finds just two possibilities for a 2-byte LAX instruction that places a 0x38 or 0x3f into both A and X. The first is LAX (0x59),Y; that is, the indirect, Y-indexed address mode with an operand of 0x59, whose encoding is 0xb3 0x59. The second possibility is only possible when you hold certain controller inputs at the time of the exploit, in order to stage a 0x38 value in the joy variable at address 0x0020. With this preparation you can used LAX (0x50,X), whose encoding is 0xa3 0x50. A 2-byte LAX gives us a lot of flexibility. Saving 1 byte means we can put the necessary PHA instruction directly in the payload, which means our JMP only has to hit an RTS, not PHA then RTS. Only needing to hit an RTS gives us a lot more possibilities for jump targets. Alternatively, we could continue jumping to PHA+RTS, and fill in the extra byte in the payload with a NOP instruction (or any 1-byte instruction that is effectively a nop). Or, as a special case in the final position, we can just repeat the previous name byte, because the JMP will happen before it gets executed. If we must jump to an address that executes PHA and then RTS, the only possibilities are 0xaa93–0xaa9f. I found this using a script—see gameend-search-jmp.fnl in the source code below. Of these possibilities, four are equally fast to enter, following a 0x4c byte: 0xaa95, 0xaa96, 0xaa9a, and 0xaa9f. 0xaa9f is the cleanest, as it points directly to the PHA instruction. Besides the variations listed above, I also tried things such as substituting 2-byte LDA followed by TAX for 3-byte LAX. See the file gameend-payloads.py in the source code for the full expression of variations:
Language: python

# 4468M payload, with all possibilities for branch instruction and PHA+RTS code address. # Also consider 2-byte LDA, ADC, EOR, SRE, or RRA followed by TAX as an alternative for 3-byte LAX. # Also consider 2-byte LDX followed by TXA as an alternative for 3-byte LAX. (LAX3 | (LDA2 | ADC2 | EOR2) + TAX | LDX2 + TXA) + BRANCH + TXS + JMP_PHA_RTS, # SRE and RRA can also serve to set A, though they may also change the value # of processor flags, so try more options for the branch (BVS and BCS in # addition to BVC and BCC). (SRE2 | RRA2) + TAX + ALL_BRANCH + TXS + JMP_PHA_RTS, # When we do 2-byte LDX first, we also have the option of doing TXS before TXA. (LDX2 + TXS) + BRANCH + TXA + JMP_PHA_RTS, # 2-byte LAX with jump to PHA+RTS, padding with 1-byte NOPs (or what are # effectively NOPs) in every possible position. What works as a NOP differs # in each position. # Anything that assigns to A or X is a NOP if it happens before LAX. # (As long as it doesn't interfere with the address mode of LAX.) # PHA and TXS work too, as we're not using the stack yet. Also include # TAY, even though it may interfere with the address mode of the LAX. (NOP | TXA | TYA | TAX | TAY | LSRA | PHA | TXS) + LAX2 + BRANCH + TXS + JMP_PHA_RTS, # After LAX, A and X are equal, so TXA and TAX are NOPs. We may now # clobber Y with TAY or DEY. PHA and TXS are still available to us too. LAX2 + (NOP | TXA | TAX | TAY | DEY | PHA | TXS) + BRANCH + TXS + JMP_PHA_RTS, # After TXS, we may still interchange A and X, or clobber Y. We can also # use a second TXS as a NOP. But we cannot use PHA anymore, because the # stack is now set up. LAX2 + TXS + BRANCH + (NOP | TXA | TAX | TAY | DEY | TXS) + JMP_PHA_RTS, # In the final position, don't use a NOP, just repeat the previous byte. LAX2 + TXS + BRANCH + JMP_PHA_RTS + REP, # 2-byte LAX with PHA and jump to RTS. LAX2 + TXS + BRANCH + PHA + JMP_RTS,
Sadly I could not find a use for the TAS instruction :(
In total I generated and timed 8168 candidate payloads. The list, sorted from fastest to slowest, is ff1-gameend-payloads-time.log. There are a few payloads that are equally fast (52 frames) as the one used in 4468M. These are basically the same, [LAX 0x8db2; BCC] [TXS; JMP ADDR], just using equivalent JMP addresses:
[0xaf 0xb2 0x8d 0x90]	22	[0x9a 0x4c 0x95 0xaa]	30	52
[0xaf 0xb2 0x8d 0x90]	22	[0x9a 0x4c 0x96 0xaa]	30	52
[0xaf 0xb2 0x8d 0x90]	22	[0x9a 0x4c 0x9a 0xaa]	30	52
[0xaf 0xb2 0x8d 0x90]	22	[0x9a 0x4c 0x9f 0xaa]	30	52
The LAX 0x8db2 instruction, despite needing 3 bytes to encode, uses characters that are all close to each other on the name entry screen. After that, 1 frame slower, there is a similar batch that uses address 0xc3b5 in place of 0x8db2 in the LAX instruction, and uses BVC in place of BCC: [LAX 0xc3b5; BCC] [TXS; JMP ADDR].
[0xaf 0xb5 0xc3 0x50]	23	[0x9a 0x4c 0x95 0xaa]	30	53
[0xaf 0xb5 0xc3 0x50]	23	[0x9a 0x4c 0x96 0xaa]	30	53
[0xaf 0xb5 0xc3 0x50]	23	[0x9a 0x4c 0x9a 0xaa]	30	53
[0xaf 0xb5 0xc3 0x50]	23	[0x9a 0x4c 0x9f 0xaa]	30	53
The first payloads that are notably different are 3 frames slower overall. I'll highlight just a few of them. One uses a 2-byte LAX and an explicit PHA followed by a JMP to RTS: [LAX (0x59),Y; TXS; BCC] [PHA; JMP 0xc34b].
[0xb3 0x59 0x9a 0x90]	32	[0x48 0x4c 0x4b 0xc3]	23	55
Another uses the space gained by a 2-byte LAX to store a TXA (which is effectively a nop) at the beginning of party member 4's name: [LAX (0x59),Y; TXS; BCC] [TXA; JMP 0xaa9f]. The TXA opcode 0x8a is in the upper left of the name entry palette and requires no cursor movement to enter.
[0xb3 0x59 0x9a 0x90]	32	[0x8a 0x4c 0x9f 0xaa]	23	55
This one uses an alternative 2-byte LAX, no explicit PHA, and a JMP to PHA+RTS. The final name character, which is not executed, can be a repeat of the previous character, which requires no cursor movement: [LAX (0x50,X); TXS; BCC] [JMP 0xaa9f].
[0xa3 0x50 0x9a 0x90]	32	[0x4c 0x9f 0xaa 0xaa]	23	55

In summary, I am impressed by the high degree of optimization achieved by [4468] NES Final Fantasy "game end glitch" by AmaizumiUni, Spikestuff & DJ Incendration in 01:36.47. Despite my best efforts, I was not able to improve it. My source code, data files, and development notes are available from git clone https://www.bamsoftware.com/git/ff1.git. The commit as of this writing is d5cd9d40f891729a441eb950da98e371b43167d2. A brief howto is: run gameend-payloads.py to generate a list of candidate payloads in gameend-payloads.fnl. Run gameend-payloads-time.fnl (in BizHawk) to time how the names take to enter. Run gameend-payloads-test.fnl to check whether they actually result in a game end glitch.
Post subject: Loud Room input illusion
Sand
He/Him
Experienced Forum User, Published Author, Player (147)
Joined: 6/26/2018
Posts: 209
Sand wrote:
Connecting commands with dots. It's possible to enter multiple commands at once, separated by periods. This capability is given in the manual. (c-square, your N,N,U,GET EGG,D in Post #476712 would have worked if you used periods instead of commas.) It was the main trick used to speed up the "fastest eaten by a grue" TAS, where it enabled ending input early. I didn't use this feature in this run, but I did a quick test, and to my dismay it appears that entering multiple commands on one line is slightly faster. N.N.U.GET EGG.D was 1690 frames in my test, compared to 1709 frames with the commands on separate lines. I say "to my dismay" because this would be a pretty extreme speed/entertainment tradeoff, a small gain in speed for a large decrease in entertainment. It's nicer to watch a run where the results of commands are close to the commands that caused them, not separated from them by pages of scrolling. I'm tempted not to make use of this trick, except perhaps when traversing long distances, such as through the mazes.
There's another place where joining multiple commands with dots doesn't work. The Loud Room, where—amazingly—the game sets up a simulation of the main game loop, complete with fake > prompt, in order to implement the echo puzzle. If there are any commands remaining when you enter the Loud Room (P-CONT is not empty), they get "lost in the noise".
The Troll Room There is a bloody axe here. >e.e.e.echo.get bar East-West Passage Round Room Loud Room On the ground is a large platinum bar. The rest of your commands have been lost in the noise. >
If you include additional commands after the command to solve the puzzle, it looks like they are ignored.
Loud Room On the ground is a large platinum bar. >echo.e The acoustics of the room change subtly. Loud Room On the ground is a large platinum bar. >
Post subject: RNG manipulation with ASCII codes
Sand
He/Him
Experienced Forum User, Published Author, Player (147)
Joined: 6/26/2018
Posts: 209
Sand wrote:
RNG. Speaking of RNG, the good and bad news is that the RNG in the Apple II version is very sensitive, changing with granularity smaller than a frame. I suspect (but have not confirmed) the Apple II port of Zork uses the built-in KEYIN random number generator, which simply rapidly increments a 16-bit number whenever waiting for keyboard input. This is good in the sense that, in principle, you can probably get any RNG output you need with less than a frame of waiting. It's bad in that it's hostile to console verification, and the Virtu core doesn't provide easy access to sub-frame inputs, as far as I know. In this run, I've done RNG manipulation by manually inserting frame delays. But I also experimented with toggling the Shift key while entering commands, and that alone seems to change the CPU cycles enough to affect RNG. I think it should be possible to eventually do such RNG manipulation automatically, using the script, but it will require more work to understand the game's memory layout, in order to read the results of commands.
The source code for many versions of Infocom's Z-Machine interpreters is online, so we can easily see how the RNG works in the Apple II version. As I suspected, the Apple II Z-Machine interpreter (ZIP) uses the KEYIN random number generator built into the Apple II firmware, which rapidly increments a 16-bit counter at addresses 0x4e and 0x4f (which the ZIP calls RNUM1 and RNUM2) while waiting for a key to be pressed. But there's a twist: after a key is typed, the ZIP mixes the ASCII code into the RNG state. This is good news, because it means you can manipulate RNG by introducing variations into the commands you enter. In particular, you can easily affect RNG state by swapping lowercase letters to uppercase, with no loss of time. The ZIP's GETKEY subroutine calls the Apple II RDKEY subroutine, which in turn calls KEYIN. KEYIN returns an ASCII code in the A register and leaves RNUM1 and RNUM2 in some hard-to-predict state. At the end of GETKEY, the ZIP further tweaks the random number state by adding the ASCII code into RNUM1 and xoring it into RNUM2: https://github.com/erkyrath/infocom-zcode-terps/blob/d5ac95a838/apple/zip/machine.g#L100
	ADC RNUM1		;FUTZ WITH RANDOM
	STA RNUM1
	EOR RNUM2
	STA RNUM2
Here's an example of how these observations could be applied. Certain rooms in the overworld are designated "forest rooms". Whenever you take a turn in a forest room, there's a 15% random chance that the game will print the message "You hear in the distance the chirping of a song bird." (Which is a hint for a later puzzle.) We want to avoid that random event, because every line of output takes time. Suppose you enter the command get egg while in the room UP-A-TREE, and, in addition to picking up the egg, you unluckily get the songbird message. Then you can backtrack and try the command get egG. If that doesn't work, try get eGg, then get eGG, and so on. If you exhaust all the letter case possibilities of the most recent command, you can continue with the next most recent, and so on, until you get the RNG you need. I hope to automate this style of RNG manipulation to enable flexible experimentation with routing. As things stand now, if you wanted to try s.e instead of n.e to get behind the house at the beginning of the run, for example, it would desync all later random events. Automated RNG manipulation might also make it possible to play (what is effectively) the same run in either 40-column more or 80-column mode, or in either brief more or verbose mode. That would not normally be possible, because the timing, and hence the RNG, in the various modes would be different. But with a Lua script automatically adjusting the letter case of commands to make sure random rolls always come out the right way, it may be possible for the same route to work in either mode. One movie might have get eGg where the other has get EGg, but they would be effectively the same. Automated RNG manipulation may also make the run robust to timing changes in a future version of the emulator. The main RNG events I can think of that will require manipulation are the songbird, troll and thief combat, candles being blown out in TINY-CAVE, and bat random drops. (I was surprised to learn that the movement of the thief is not random: rather he warps from room to room in order, skipping certain rooms.) The hardest one will probably be the thief combat, where we'll need a series of low-probability attack and defense rolls. But even that I think will be tractable.
Post subject: Baseline TAS for Apple II
Sand
He/Him
Experienced Forum User, Published Author, Player (147)
Joined: 6/26/2018
Posts: 209
Here's a first draft of a TAS of Zork I for Apple II. It uses revision 88 of the game, which, according to a survey by Data Driven Gamer, is "by far the most widely distributed version". The total time is 18871 frames or 5:14.52, in 258 moves. User movie #638893574504520990 Link to video (Since this TAS is for Apple II, I debated about whether to add it to this thread here in the MS-DOS Games forum or start a new thread in the Other Computers forum. I decided it's probably better to keep all the discussion for the game in one thread, even if it's a multi-platform game.) You may have seen that I've been working on scripting TASes with Lua and that I used the technique to improve the "fastest eaten by a grue" TAS. This is a straightforward extension to the full game. I didn't do independent route research for this run; instead I implemented Gym Slow's RTA route, unmodified except for a SUPER command at the beginning to activate superbrief mode. There are various small changes that would make the run faster taking the Apple II platform into account, as well as potential larger route changes taking advantage of the RNG manipulation that is possible in a TAS. I haven't done any of those. My idea was to let this run serve as a baseline to compare future improvements against. The run being scripted in Lua (actually Fennel) means that there is a script (100%.fnl) that contains a lot of code like this:
;; "80-COLUMN DISPLAY? (Y/N)"
(enter-text "n")

;; Skip over some loading. Get to somewhere near the first input prompt.
(savestate.next-frame 300)

(enter-commands ["super"])

(wait-for-keyboard-ready)
(savestate.next-frame 1) ;; RNG manipulation (avoid songbird).
; TODO use Shift or other keys for RNG manipulation.

;; Enter the schedule of commands.
;; TODO joining with "." can save time and alter RNG.
(enter-commands
  [
   ;; Climb the tree, get egg, and climb down again.
   "n"
   "n"
   "u"
   "get egg"
   "d"

   ;; Enter the house.
   "s"
   "e"
   "open"

   ;; ...
   ])
You can modify the run by modifying the script and re-running it in BizHawk. However, even small changes will require re-syncing RNG. I haven't done anything yet to automate RNG manipulation, which is needed to avoid unwanted random encounters, win combat quickly, and prevent certain other random events. I implemented RNG manipulation in this run by inserting 1-frame delays, though I think there are better ways to do it. You can see one instance of RNG manipulation in the snippet above, the (savestate.next-frame 1). More about RNG below. Other notes on this run and thoughts on possible improvements: 40-column mode vs. 80-column mode. The first thing the game does is offer a choice between 40-column mode and 80-column mode. 80-column mode looks nice: it has more text on the screen and uses both upper- and lowercase letters. I did a quick test of commands up to collecting the sword and lamp in the living room; it was 1279 frames in 40-column mode and 1341 frames in 80-column mode. So I guess 40-column mode is generally faster. But the choice of 40 or 80 columns could be considered a speed/entertainment tradeoff. Ultimately, I'd like to make the control script adaptively manipulate RNG, such that the run can be played back in either mode. Brief versus verbose. Besides the default verbose mode, the game offers a brief mode (which only shows you the long description of a room the first time you enter it, and a short description thereafter) and a superbrief mode (which always shows you the short description). Superbrief obviously saves time, as screen output on the Apple II is less than instant. Like the number of columns, the level of output verbosity could be considered a speed/entertainment tradeoff. (And similarly, any change will require re-synchronizing RNG.) RNG. Speaking of RNG, the good and bad news is that the RNG in the Apple II version is very sensitive, changing with granularity smaller than a frame. I suspect (but have not confirmed) the Apple II port of Zork uses the built-in KEYIN random number generator, which simply rapidly increments a 16-bit number whenever waiting for keyboard input. This is good in the sense that, in principle, you can probably get any RNG output you need with less than a frame of waiting. It's bad in that it's hostile to console verification, and the Virtu core doesn't provide easy access to sub-frame inputs, as far as I know. In this run, I've done RNG manipulation by manually inserting frame delays. But I also experimented with toggling the Shift key while entering commands, and that alone seems to change the CPU cycles enough to affect RNG. I think it should be possible to eventually do such RNG manipulation automatically, using the script, but it will require more work to understand the game's memory layout, in order to read the results of commands. You can see a listing of the part of the ROM that increments the 16-bit random number in the technical reference manual:
C83B:E6 4E          57 GETKEY  INC  RNDL      ;BUMP RANDOM SEED
C83D:D0 02   C841   58         BNE  GETK2
C83F:E6 4F          59         INC  RNDH
C841:AD 00 C0       60 GETK2   LDA  KBD       ;KEYPRESS?
C844:10 F4   C83B   61         BPL  GETKEY    ;=>NOPE
C846:8D 10 C0       62         STA  KBDSTRB   ;CLEAR STROBE
C849:60             63         RTS
Connecting commands with dots. It's possible to enter multiple commands at once, separated by periods. This capability is given in the manual. (c-square, your N,N,U,GET EGG,D in Post #476712 would have worked if you used periods instead of commas.) It was the main trick used to speed up the "fastest eaten by a grue" TAS, where it enabled ending input early. I didn't use this feature in this run, but I did a quick test, and to my dismay it appears that entering multiple commands on one line is slightly faster. N.N.U.GET EGG.D was 1690 frames in my test, compared to 1709 frames with the commands on separate lines. I say "to my dismay" because this would be a pretty extreme speed/entertainment tradeoff, a small gain in speed for a large decrease in entertainment. It's nicer to watch a run where the results of commands are close to the commands that caused them, not separated from them by pages of scrolling. I'm tempted not to make use of this trick, except perhaps when traversing long distances, such as through the mazes. Even if you were to enter every command of the run on one long line, it wouldn't allow ending input super early, because you would still have to press keys to scroll through all the [MORE] prompts and get to the ending. I tried joining just the last few commands, and buffering a Space press to clear the final [MORE] (as in Post #536873), but it doesn't work in this full-game run: after winning, the game seems to clear the keyboard buffer, or something, such that you have to wait for the final [MORE] to appear before you can dismiss it. Two-part commands. The Gym Slow route occasionally splits one command into two parts, in order to reduce keystrokes. This is when you enter a partial command, then the game asks for clarification, and you enter the word that's missing. For example, the route uses PUT SOLID and CASE to put the ("solid") gold coffin in the trophy case, because that saves having to enter the word IN:
>PUT SOLID
WHAT DO YOU WANT TO PUT THE SOLID IN?
>CASE
DONE.
This is good for an RTA run on a modern computer where the output is instantaneous, but on the Apple II it's probably better to spend some extra frames on input so the game doesn't have to emit so much output: PUT SOLID IN CASE. (Incidentally, these two-part commands don't work when entered as a single line separated by a period: you can't do PUT SOLID.CASE.) A similar case is in the dam maintenance room, where this route does PUSH ALL. There's only one thing in the room you need to push, but this command results in a screenful of output as you try to push every item (including the one you need to push). It may be faster to be more specific in the command so that there's less output. Early thief kill? This route leaves the battle with the thief until near the end. But before that, there are two occasions where you enter the thief's hideout in order to warp to the temple; each of these displays a long message about the thief entering the room and requires RNG manipulation to avoid getting killed or wounded (which would reduce carrying capacity). It may be possible, with RNG manipulation, to kill the thief at the first encounter. This would ease the rest of the run, as there would be no need to worry about random thief encounters or combat in the hideout. I haven't yet checked to see how combat works in the source code. It's conceivable that a route would call for getting robbed by the thief at a time when you're carrying lots of treasure far from the house, as that would effectively warp the treasure to be near the house, and free up your carrying capacity to pick up more items. I hope that's not required, because it would require careful manipulation. The thief's attack isn't a random probability per room; he actually physically moves around the cave at random. The route already uses the "raft of holding" bug (bug #27 in Graeme Cree's list) for the hardest part of inventory weight management. Vampire bat RNG manipulation? Normally, you're supposed to use the garlic to incapacitate the bat, which otherwise picks you up and drops you in a random place in the coal mine. It may be possible to instead allow the bat to grab you, and manipulate it to drop you deep in the mine, saving one of the traversals through the mine. (The current route traverses the mine a total of four times, twice heading in and twice heading out.) It's possible for the bat to drop you outside, in the squeaky room, so it wouldn't be necessary to have the garlic even on the way out. Bypass torch/dumbwaiter puzzle? Bug #41 in Nathan Simpson's list says that you don't need the torch in order to get light into the drafty room past the coal mine. There's a trick where you can store the candles in the sack and still fit through the small opening. I don't think it would save a ton of time, because you'd still need to get to the end of the mine and back to put the coal in the dumbwaiter, but it could free up routing possibilities with regard to when the torch gets deposited in the trophy case. The page says it only works if you extinguish and re-light the candles, so it would require getting the matchbook from the dam lobby, which the route currently doesn't do.
Post subject: Enter house with "w" instead of "in"
Sand
He/Him
Experienced Forum User, Published Author, Player (147)
Joined: 6/26/2018
Posts: 209
Here's a further 3 frames of improvement over Post #537069: User movie #638888257478152075 The difference is entering the house with w instead of in:
s.e.open.w.u.u
On its own, that would only save 1 frame, but it also happens to tweak the RNG such that the 2 frames of delay for RNG manipulation can be removed. I learned that w would work to enter the house from bug #25 at Graeme Cree's Zork I bug list. Source code is at https://archive.org/details/zork-appleii-tas-grue-20250722. The diff in grue.fnl is:
Language: diff

diff --git a/grue.fnl b/grue.fnl index 6e8c42e..48d5a34 100644 --- a/grue.fnl +++ b/grue.fnl @@ -81,11 +81,11 @@ ; (enter-commands ["go e" "go s" "go e" "open window" "go in" "go u" "go u"]) (wait-for-keyboard-ready) - (savestate.next-frame 2) ;; Delay for RNG manipulation. + (savestate.next-frame 0) ;; Delay for RNG manipulation. ;; Enter the schedule of commands. NB the "." inputs need to be manually ;; re-added to Input Log.txt in the .bk2 movie file, at least as of ;; BizHawk 2.10: https://github.com/TASEmulators/BizHawk/issues/4391. - (enter-commands ["s.e.open.in.u.u"]) + (enter-commands ["s.e.open.w.u.u"]) ;; Buffer a Space press to clear the "[MORE]". (savestate.next-frame) (savestate.joypad-press-release "Space")
Post subject: 1 frame shorter by clearing [MORE] with Space rather than Return
Sand
He/Him
Experienced Forum User, Published Author, Player (147)
Joined: 6/26/2018
Posts: 209
Sand wrote:
Even though this submission is a joke, there are ways to improve it. Here's a movie that reduces the time from 900 frames to 500 frames.
Here's a 1-frame improvement to Post #536873: User movie #638886542790751703 The difference is clearing the [MORE] prompt with the Space key, rather than Return. The previous run had to release Return for 1 frame in order to press it again; now that's not needed. Source code is at https://archive.org/details/zork-appleii-tas-grue-20250721. The diff in grue.fnl is just:
Language: diff

diff --git a/grue.fnl b/grue.fnl index 3811692..6e8c42e 100644 --- a/grue.fnl +++ b/grue.fnl @@ -86,11 +86,9 @@ ;; re-added to Input Log.txt in the .bk2 movie file, at least as of ;; BizHawk 2.10: https://github.com/TASEmulators/BizHawk/issues/4391. (enter-commands ["s.e.open.in.u.u"]) - ;; Buffer a Return press to clear the "[MORE]". + ;; Buffer a Space press to clear the "[MORE]". (savestate.next-frame) - ;; Add a blank frame because the previous frame was also Return. - (savestate.next-frame) - (savestate.joypad-press-release "Return") + (savestate.joypad-press-release "Space") ;; Sync the emulator to the state after entering all commands, for the ;; purpose of saving a movie.
Post subject: Application to Zork
Sand
He/Him
Experienced Forum User, Published Author, Player (147)
Joined: 6/26/2018
Posts: 209
Post #536873 has another application of the immutable savestate manipulation idea, this time with the text adventure game Zork. There's not a lot of deep game integration needed for this one, mainly just reading a certain memory location to know when the keyboard is ready to accept input. Here's the main Fennel script grue.fnl. You run this with EmuHawkMono.sh --lua=grue.lua Zork_I_The_Great_Underground_Empire_1980_Infocom_PASCAL.dsk. grue.lua is a loader for grue.fnl.
Language: fennel

;; Unload our own modules, to avoid old versions being cached in package.loaded ;; from a previous execution. (each [_ modname (ipairs [:bizhawk :savestate :util])] (tset package.loaded modname nil)) (local savestate (require :savestate)) ;; https://archive.org/details/Apple_IIe_Technical_Reference_Manual/page/n45/mode/1up ;; The least significant 7 bits of this memory location contains the most ;; recently typed character. The most significant bit (the strobe bit) is 1 ;; until the memory location 0xc010 (the clear-strobe switch) is read or ;; written. (local KEYBOARD-DATA-STROBE-ADDR 0xc000) (lambda wait-for-keyboard-ready [st] (let [(before-press _ _) (savestate.earliest-effective-frame st #(savestate.joypad-press-release $ "X") ;; Look ahead 1 frame to see if the strobe bit is 0. #(savestate.await-u8-satisfies (savestate.next-frame $) 1 KEYBOARD-DATA-STROBE-ADDR #(= 0 (band $ 0x80))))] before-press)) ;; Return the name of a joypad key that stands for the given character. (lambda key-for-character [c] (if ;; Digits and various other characters are named after themselves. (string.match c "^[%d%'%,%-%.%/%;%=%[%\\%]%`]$") c ;; Lowercase letters are named by the corresponding uppercase letter. (string.match c "^[%l]$") (string.upper c) ;; Some characters have textual names. (case (. {"\x7f" "Delete" "\x09" "Tab" "\x0a" "Return" "\x1b" "Escape" "\x20" "Space"} c) key key ;; We don't handle characters like uppercase letters, which would ;; require multiple keys (including Shift). _ (error (string.format "unknown how to enter %q" c))))) (lambda enter-character [st c ?prev-key] ;; Find the earliest frame at which the keyboard strobe bit will become 0 on ;; the following frame (indicating that the game read the input). (let [key (key-for-character c)] ;; Insert a blank frame if this key is the same as the previous one. (values (-> (if (not= key ?prev-key) st (savestate.next-frame st)) (wait-for-keyboard-ready) (savestate.joypad-press-release key)) key))) (lambda enter-text [st text] (var ?prev-key nil) (accumulate [s st c (string.gmatch text ".")] (let [(s key) (enter-character s c ?prev-key)] (set ?prev-key key) (savestate.next-frame s)))) (lambda enter-commands [st commands] (accumulate [s st i command (ipairs commands)] ;; Do a next-frame between commands, but not after the final command. (-> (if (= i 1) s (savestate.next-frame s)) (enter-text command) (enter-character "\n")))) (lambda main [] (-> (savestate.new-from-emulator!) ;; The emulator accepts an input on the very first frame, before the ;; game has begun. Skip that frame, and thereafter rely on ;; enter-character to find then next frame at which input is accepted ;; (after boot is finished and the game has presented a prompt). (savestate.next-frame) ;; Skip over most of the boot sequence, just to save time in looking for ;; the first frame that accepts keyboard input. (savestate.next-frame 450) ; ;; https://tasvideos.org/5891S inputs: ; (enter-commands ["go e" "go s" "go e" "open window" "go in" "go u" "go u"]) (wait-for-keyboard-ready) (savestate.next-frame 2) ;; Delay for RNG manipulation. ;; Enter the schedule of commands. NB the "." inputs need to be manually ;; re-added to Input Log.txt in the .bk2 movie file, at least as of ;; BizHawk 2.10: https://github.com/TASEmulators/BizHawk/issues/4391. (enter-commands ["s.e.open.in.u.u"]) ;; Buffer a Return press to clear the "[MORE]". (savestate.next-frame) ;; Add a blank frame because the previous frame was also Return. (savestate.next-frame) (savestate.joypad-press-release "Return") ;; Sync the emulator to the state after entering all commands, for the ;; purpose of saving a movie. (savestate.sync-emulator!))) (savestate.run main)
The main mechanical piece is the function wait-for-keyboard-ready, which finds the earliest frame at which keyboard input is accepted. It does this by repeatedly pressing a key, then looking forward 1 frame ((savestate.next-frame $)) to see if the most significant bit in KEYBOARD-DATA-STROBE-ADDR is clear, indicating that the program has read the input. The function is implemented in terms of earliest-effective-frame in the savestate module, which encapsulates this test-and-look-ahead procedure. Like all functions written in this style, wait-for-keyboard-ready takes a savestate object as input and produces a different savestate object as output. Savestate objects never change once created, so you could use the same input again for another purpose. Then enter-character is implemented using wait-for-keyboard-ready, also mapping string characters to Apple II key names. enter-text types in an entire string using enter-character. enter-commands enters a whole list of command strings. Originally I was expecting to have to enter multiple commands, as in #5891: adelikat's AppleII Zork I: The Great Underground Empire "Fastest eaten by a grue" in 00:15.00. That was before I learned you can join individual commands with . and thereby end input early. So the code ends up calling enter-commands with a table of length 1. I had to do a little low-level savestate.next-frame and savestate.joypad-press-release work after the commands themselves, to buffer an input to clear the [MORE] prompt that appears when too much input is produced at once. I was able to use the savestate.fnl module mostly unchanged from my experiments with Final Fantasy. But I had to add a couple of workarounds for bugs in the Apple II core (Virtu) in BizHawk. One is that joypad.getdoesn't work. For this, I added an auxiliary table inside the module that tracks the state of keyboard keys in the same way BizHawk would, and redefined joypad.get to consult this table instead of asking BizHawk directly. See JOYPAD-STATE in savestate.fnl. The other is that the keyboard state partially bleeds across savestate loads: the state of the emulator can be slightly different depending on what its state was when you loaded the savestate. This required a bit more elaborate of a workaround. I modified savestate objects to contain, in addition to a savestate ID and a schedule of inputs, the key that was registered as being pressed on their previous frame. Whenever a savestate is loaded, the code does a little procedure to sychronize the emulator's internal keyboard state with what it should be for the savestate being loaded. See ?virtu-current-key-pressed in savestate.fnl.
Post subject: Improvements
Sand
He/Him
Experienced Forum User, Published Author, Player (147)
Joined: 6/26/2018
Posts: 209
Even though this submission is a joke, there are ways to improve it. Here's a movie that reduces the time from 900 frames to 500 frames. User movie #638874688943283518 Link to video These are the inputs used in 5891S:
go e
go s
go e
open window
go in
go u
go u
These are the inputs used in this new run:
s.e.open.in.u.u
[Plus an extra Return to dismiss a [MORE] prompt.]
The improvements are:
  • Removing the first go e command, which steps into a blocked door.
  • Removing go from all commands. Just s works the same as go s, for example.
  • Shortening open window to open. This is one of those commands that will infer an object.
  • Entering the commands all in one line, separated by .. This doesn't make YOU HAVE DIED arrive any faster, but it allows ending input early. Because all the commands together produce more than a screen's worth of output, we have to spend an additional 2 frames to buffer a Return input to dismiss a [MORE] prompt.
The new combined command string hits the RNG differently, such that the final u command doesn't actually result in getting eaten by a grue, if you enter it as fast as possible. I had to insert 2 frames of delay at the beginning for RNG manipulation.
Modification
Change in frames
Remove go e
−46
Remove go everywhere
−25
Remove window
−22
Join commands with ., end input early
−309
RNG manipulation
+2
For fans of Lua integration, this TAS is another that was done using the data-oriented savestate manipulation technique I have been developing. That means the entire run is driven programmatically by a Lua script. (In this case, it's actually a Fennel script, which compiles to Lua.) See zork.zip or zork.bundle for source code. If you find any improvements, you may be able to implement them by editing the main script file grue.fnl and re-running it. A difficulty with the trick of concatenating commands with . is that currently released versions of BizHawk have a bug with the Apple II core where . inputs are not properly recorded in .bk2 files. So after recording the movie, you have to manually edit "Input Log.txt" inside the .bk2 file to restore the missing inputs. It will be fixed in a future release of BizHawk.
Sand
He/Him
Experienced Forum User, Published Author, Player (147)
Joined: 6/26/2018
Posts: 209
I'm impressed by the routing and inventory management in this one.
Sand
He/Him
Experienced Forum User, Published Author, Player (147)
Joined: 6/26/2018
Posts: 209
AmaizumiUni wrote:
I checked wram in mainmemory.read_u8 but the numbers are different, so it looks like I'm looking at ram.
I hope I understand your question. To read WRAM, you should use memory.read_u8 (not mainmemory.read_u8), and specify "WRAM" for the memory domain. Like this:
memory.read_u8(addr, "WRAM")
You can see the available memory domains with memory.getmemorydomainlist:
memory.getmemorydomainlist() --> {[0]="RAM", "System Bus", "PPU Bus", "CIRAM (nametables)", "PALRAM", "OAM", "Battery RAM", "PRG ROM", "VRAM", "WRAM"}
mainmemory.read_u8(addr) is like memory.read_u8(addr, mainmemory.getname()). The return value of mainmemory.getname() depends on the core. In NesHawk, it is "RAM". (This comes from the common implementation of IMemoryDomains and the ordering of memory domains in NesHawk.)
Sand
He/Him
Experienced Forum User, Published Author, Player (147)
Joined: 6/26/2018
Posts: 209
MUGG wrote:
Why didn't the first code work?
It's because you used the name val both as the name of the outer table and as one of the parameters in the anonymous callback function.
local val = {
function(addr, val, flags)
In the environment of the anonymous callback function, val refers to the function parameter, not the outer table you intend. BizHawk is calling the callback with an integer value in the val parameter, which is why the val.ball operation results in "attempt to index a number value". You could use a different name in one place or the other. Or you could omit the function parameters from the callback (since you aren't using them), so that values that are passed into the function are not bound to variable names.
Post subject: Lua/Fennel polyglot
Sand
He/Him
Experienced Forum User, Published Author, Player (147)
Joined: 6/26/2018
Posts: 209
As you may know from Post #535321, I've been experimenting with writing scripts for BizHawk in Fennel, a lisp programming language based on Lua. Fennel code has to be compiled to Lua before BizHawk can run it. To do that, I have been dividing programs into two files: a main ".fnl", written in Fennel, that has the actual code; and a stub ".lua" file that compiles and runs the .fnl file. You load the .lua file in BizHawk, then the .lua file runs the .fnl file. For example, I might have a "script.fnl" that contains the code of the script:
Language: fennel

(console.log "Hello Fennel") (console.log (string.format "ROM name: %q" (gameinfo.getromname))) (local fennel (require :fennel)) ;; for fennel.view (console.log "joypad" (fennel.view (joypad.get) {:one-line? true}))
And then a "script.lua" stub that executes the main Fennel file:
Language: lua

require("fennel").install().dofile("script.fnl")
The "fennel" module comes from fennel.lua, which you just have to install in the directory alongside your scripts. The two-file approach works fine. But I found a way to get the same effect with just one file. The file is simultaneously a Lua program and a Fennel program; i.e., a polyglot. It looks like this:
Language: lua

;; return require("fennel").install().eval([==[ (console.log "Hello Fennel") (console.log (string.format "ROM name: %q" (gameinfo.getromname))) (local fennel (require :fennel)) ;; for fennel.view (console.log "joypad" (fennel.view (joypad.get) {:one-line? true})) ;; ]==])
The ;; marks a comment in Fennel. So from the point of view of a Fennel interpreter, the first and last lines of the file disappear, leaving only the lisp code in between. In Lua, ; is an empty statement, a no-op separator. So the Lua interpreter sees and executes the require("fennel").install().eval(...). The [==[ marks the beginning of a long literal string that lasts until the ]==] on the last line—encompassing the entire text of the Fennel program. To the Lua interpreter, the Fennel part of the program looks like one long string, which gets passed to the fennel.eval function. It would be nice if only required adding something to the top of the file, not both top and bottom, but I couldn't think of a way to do that. I've found just one problem with the polyglot approach, which is that BizHawk won't let you load a file with a ".fnl" filename extension with the --lua option on the command line. I want to name the program file "script.fnl", so that when I edit it, I get syntax highlighting and automatic indentation appropriate for Fennel. But BizHawk's --lua option only allows ".lua" and ".txt" as filename extensions, and silently ignores files with any other extension. (Except for ".luases", which I don't know what that is.) You can load .fnl files from the "Open Script" UI in the Lua Console; it's just the --lua option that doesn't work. So I have to name the file "script.lua", which adds inconvenience when editing it as Fennel code. So I might stick with the two-file method anyway. If you want to see how the Fennel code compiles to Lua, you can paste it in here.
Post subject: Roguelike Celebration talk
Sand
He/Him
Experienced Forum User, Published Author, Player (147)
Joined: 6/26/2018
Posts: 209
Has the NetHack TAS team considered doing a talk at Roguelike Celebration about the design for a minimum-turns run? The 2025 call for presentations is up:
We can’t wait to welcome you all to this year’s Roguelike Celebration on Oct 25th and 26th - but first it is time to open the door to this year’s batch of speakers! Our conference is only as good as our speakers and presenters, and luckily every year we welcome a breadth of creative, passionate, and talented individuals to step onto the virtual conference stage. The call for presenters is now open, with a submission form here. Submissions will be open until June 30th. Please be sure to pass the link along to anyone you think would have something compelling to share. We welcome all types of presentations and performances, and encourage anyone with the hint of an idea to look at our past years to see just how wide an array of topics we accept and how many different backgrounds and life experiences can make up a Roguelike Celebration speaker!
I've watched this conference for years and I'm sure the topic would be well received. I don't want to create an obligation for anyone, but if you're looking to share in a venue that is more roguelike-oriented than speedrun-oriented, this would be a good one. Just walking through the post–turn 2000 actions à la t2000timings.txt (Post #475904) would be enough for a complete talk, I think. People will be interested to know about zero-time actions, how turns are structured, and the obscure features that come up in a TAS scenario. (It makes me think of the "NetHack: Tech Tourist Mode" talk from 2018.)
Sand
He/Him
Experienced Forum User, Published Author, Player (147)
Joined: 6/26/2018
Posts: 209
There's no memory.readword, but there are memory.read_s16_be, memory.read_s16_le, memory.read_u16_be, memory.read_u16_le: Wiki: BizHawk/LuaFunctions. You have to specify whether you want to read as signed (s) or unsigned (u), and big-endian (be) or little-endian (le).
Sand
He/Him
Experienced Forum User, Published Author, Player (147)
Joined: 6/26/2018
Posts: 209
MUGG wrote:
After I discovered my visualisation luascript doesn't run correctly due to syntax errors that have somehow slipped into that post… Here is the script with syntax errors "seemingly fixed".
Language: diff

-for n=1,table.getn(addressList) do +for n=1,#addressList do
I don't think it was a syntax error, exactly. It's just that the old script used the table.getn function, which was deprecated in Lua 5.1 and removed in Lua 5.2. BizHawk currently uses Lua 5.4. You fixed it correctly by changing it to use the # operator.
MUGG wrote:
I tried to fix the script but now it will output information that's different from before (when compared to the screenshot I had posted). When running the "seemingly fixed" script, I noticed the information output is different between v1.0 and v1.1 of the game. V1.0 (Every few frames of running right) V1.1 (Every few frames of running right)
If I understand you right, there are two problems:
  1. After you made changes to the script, it produces different output than it used to, for the same version of the game.
  2. The updated script produces different output for v1.0 and v1.1.
On point (1), I'm not sure what the cause could be. I looked at the differences between the old and new script in Post #462326 and https://pastebin.com/qjMjp6ZP. The only significant change seems to be that, in addition to an onmemoryexecute handler, the script installs onmemoryread and onmemorywrite handlers for each address in addressList. On point (2), the script is looking for the execution of specific addresses, and it would not be surprising for code addresses to be different in different versions of the game. You'll need a separate table of addresses for each different version. From a quick check, judging by the text labels in addressList, it looks like the current table is tuned for the 1.1 version. For example, the "READ FFEA" and "READ FFE9" entries:
Language: lua

[2]={0x2258,0xFFFFE000,"READ FFEA"}, [3]={0x225D,0xFFFFE000,"READ FFE9"},
match up with the listed addresses in the 1.1 ROM:
0x2258:	ld	a, [0xffea]
0x225a:	cp	0x01
0x225c:	ret	nZ
0x225d:	ld	a, [0xffe9]
But in the 1.0 ROM, those instructions appear a few bytes earlier, at addresses 0x224f and 0x2254:
0x224f:	ld	a, [0xffea]
0x2251:	cp	0x01
0x2253:	ret	nZ
0x2254:	ld	a, [0xffe9]
Post subject: Twitch archiving
Sand
He/Him
Experienced Forum User, Published Author, Player (147)
Joined: 6/26/2018
Posts: 209
KusogeMan wrote:
i just finished playing this game and i wanted to watch some speedruns and TASes, and the speedrun.com page says they might get removed by twitch!!! this is scary, also some of these links in the conversations don't work anymore, i hope we don't lose material for future TASing because of this inactivity!
There is a lot of Twitch video saved at the Internet Archive (archive.org). I'm sure it's a tiny fraction of all Twitch video ever broadcast, but it's hundreds of thousands of items. Here are two collections for example (not all Twitch stuff is in these collections): Archive Team has notes on archiving Twitch data (e.g. VODs, clips, chat), with suggestions of some software to use. The easiest automated way might actually be tubeup.py, which works with Twitch (and other video sites) in addition to YouTube: https://github.com/bibanon/tubeup. I've been disappointed in the quality of the metadata of tubeup.py items I've seen, but it's better than no archive at all. You can find existing tubeup.py items by searching for scanner:tubeup (currently 2,254,431), or just Twitch items with something like scanner:tubeup twitch (currently 284,589). The Archive Team IRC channel #burnthetwitch may have people who can give advice on archiving.
Post subject: Username highlighted on /Profile/UserFiles only(?)
Sand
He/Him
Experienced Forum User, Published Author, Player (147)
Joined: 6/26/2018
Posts: 209
When logged in, the username in the navbar is highlighted (has the CSS active class) at /Profile/UserFiles:
<a class="nav-link active" title="Manage" href="/Profile"><i class="fa fa-user d-inline"></i>&nbsp;Sand</a>
The username is not highlighted at other pages I looked at, for example /Forum, /GameResources, /Subs-List, and /UserFiles/Upload:
<a class="nav-link" title="Manage" href="/Profile"><i class="fa fa-user d-inline"></i>&nbsp;Sand</a>
I don't know if it's intended. I initially thought the highlighted username was a notification indicator, or something like that.
Post subject: Programmatic recreation
Sand
He/Him
Experienced Forum User, Published Author, Player (147)
Joined: 6/26/2018
Posts: 209
I used this movie as a target for a programmatic reimplementation (whole run as a Lua/Fennel script). The movie file is User movie #638789633114541718, and source code and notes are in Post #535321. I had the idea of using this implementation to help with looking for a faster set of names, but after reading pirohiko's glitch notes in Post #505719, I see that the current exploit is already subtle and sophisticated. It's not obvious if there might be a way to improve on it.
Post subject: Recreation of 4468M, implementation in Fennel
Sand
He/Him
Experienced Forum User, Published Author, Player (147)
Joined: 6/26/2018
Posts: 209
I've continued to experiment with scripting TAS in BizHawk in a data-oriented style. I've managed to write a script that functionally recreates [4468] NES Final Fantasy "game end glitch" by AmaizumiUni, Spikestuff & DJ Incendration in 01:36.47 (the inputs are not exactly the same, but it's equivalent in length). The movie file is User movie #638789633114541718. But more interesting than the movie itself is how it was made. Here is the source code in a Git repository and a zip snapshot: git clone https://www.bamsoftware.com/git/ff1.git (currently at commit c8b0cbad2c13b3a9c210b0daf9ae7777b4c00d06) https://www.bamsoftware.com/computers/tasvideos/ff1-20250330.zip The script to run is script/gameend.lua. It's using an evolution of the support code from Post #532299, now factored into modules. It's built on the same concept of immutable savestate objects and computation over values rather than dynamic control of the emulator state. It's also now written in Fennel, a Lisp language that compiles to Lua. The file gameend.lua is actually just a stub that compiles and run gameend.fnl, which is Fennel. I apologize slightly for that—I realize that presenting the savestate manipulation ideas in a less familiar language doesn't make them any more accessible. But I'm doing this for fun, and this was my project to learn Fennel. Writing Fennel for BizHawk turns out to be nice anyway. Here's the top-level gameend.fnl program and the main function:
Language: fennel

;; Unload our own modules, to avoid old versions being cached in package.loaded ;; from a previous execution. (each [_ modname (ipairs [:ff1 :ff1jp :game :savestate :ucs :util])] (tset package.loaded modname nil)) ;; Here we load a version-specific ff1 module, then additionally store it as ;; package.loaded.ff1. Other modules will require it simply as :ff1. (local ff1 (require :ff1jp)) (tset package.loaded :ff1 ff1) (local game (require :game)) (local {: run : restore-savestate!} (require :savestate)) (local {: Up : Down} game) ;; Call f iteratively n times, starting with the value init. (lambda times [init n f] (faccumulate [acc init _ 1 n] (f acc))) (lambda main [savestate] (-> savestate ;; Do the title menu and party selection. (game.title-menu 0) (game.party-select {:class ff1.CONST.CLS.FT :name [0x8b 0x90 0x91 0x96]} ; いきくす {:class ff1.CONST.CLS.TH :name [0xaf 0xb2 0x8d 0x90]} ; よるえき {:class ff1.CONST.CLS.BB :name [0x4c 0x51 0x50 0x55]} ; ごぞぜで {:class ff1.CONST.CLS.RM :name [0x9a 0x4c 0x9f 0xaa]}) ; ちごにむ ;; Enter the overworld. (game.await-screen-wipe) ;; Enter Coneria Castle. (times 6 #(game.move-on-map $ Up)) (game.await-screen-wipe-2) ;; Approach the stairs. (times 16 #(game.move-on-map $ Up)) ;; Start the repeated stair traversals. (times 18 #(-> $ (game.move-on-map Up) ; Step onto stairs. (game.await-screen-wipe-2) ; Screen transition. (game.move-on-map Down))) ; Step off of stairs. ;; Open and close the menu. (game.open-menu) (game.await-palette-cycle) (game.close-menu) (game.await-palette-cycle) ;; More stair traversals. (times 13 #(-> $ (game.move-on-map Up) ; Step onto stairs. (game.await-screen-wipe-2) ; Screen transition. (game.move-on-map Down))) ; Step off of stairs. ;; One last stair traversal. (game.move-on-map Up) (game.await-screen-wipe-2) ;; Open the menu. (game.open-menu) ;; Sync the emulator state to the final computed savestate, in order to ;; save a movie. (restore-savestate!))) (assert (run main))
gameend.fnl gives the overall plan of the TAS, fairly removed from the nitty-gritty of savestate manipulation. The code loads an ff1jp module for some memory address constants. The game module has subroutines specific to Final Fantasy, things like "open a menu" and "move on the map". The main function should be easy to follow. We start by advancing through the title menu and selecting a party with the given classes and names. Walk up 6 times to enter the castle (using an Up constant imported from the game module), then walk up another 16 times to reach the stairs. Walk up/down the stairs 18 times, open and close the menu, then walk up/down the stairs 13 more times. Finally, walk down the stairs a final time and open the menu to jump to the game ending sequence. How the glitch works is explained by pirohiko in Post #505719. The (restore-savestate!) at the end of main is only there for saving a movie file. As I've tried to explain, the whole point of programming in this way is not to have to think about the dynamic state of the emulator, but rather to think in terms of pure functions operating on immutable data. The emulator is just a tool that computes functions over savestates. But saving a movie from the BizHawk menu is one of the rare times we do care about the exact emulator state. The restore-savestate! function syncs the emulator with the final computed savestate and its input log, so you can "Stop Movie" in BizHawk and have a movie that ends exactly there. The function's name ends in an exclamation point to emphasize that it is not a pure function, but depends on or alters global state (which is a naming convention in Fennel). The next level of abstraction down is the game module, in game.fnl. The functions in this module describe how to do actions that are specific to the game of Final Fantasy, implemented on top of the lower-level code in the savestate module. The functions vary in complexity: title-menu, for example, accounts for input differences in the Japanese and USA releases of the game, and party-select computes an optimal sequence of inputs to enter the given names. Let's look at an easy function, await-screen-wipe, which waits for the horizontal screen wipe transition effect to finish:
Language: fennel

(lambda await-screen-wipe [savestate] (assert (await-exec savestate 128 ff1.CODE.ScreenWipe_Finalize)))
The implementation is simple: using the await-exec function from the savestate module, let the emulator run for up to 128 frames until the code address ScreenWipe_Finalize is executed, then return the resulting savestate. If the address is not executed within 128 frames, something is not as expected, so raise an error. await-screen-wipe is a good example of how Fennel compiles to Lua:
Language: lua

local function await_screen_wipe(savestate) return assert(await_exec(savestate, 128, ff1.CODE.ScreenWipe_Finalize)) end
There's also await-screen-wipe-2, which just waits for two screen wipes in sequence, such as happen when climbing or descending stairs:
Language: fennel

(lambda await-screen-wipe-2 [savestate] (-> savestate (await-screen-wipe) (await-screen-wipe)))
The -> macro is useful for sequences of operations like this, where the output of one function becomes the input to the next function. We pass the input savestate to await-screen-wipe to get a new savestate, then pass that new savestate into another invocation of await-screen-wipe. You'll see -> used also in the main function in gameend.fnl. This is how await-screen-wipe-2 compiles to Lua:
Language: lua

local function await_screen_wipe_2(savestate) return await_screen_wipe(await_screen_wipe(savestate)) end
The savestate module is the lowest level of abstraction, the code that interfaces directly with the running state of the emulator. It does all the event registration / coroutine yield / callback operations as were described in Post #532299. await-screen-wipe was defined in terms of await-exec from the savestate module, so let's look at await-exec:
Language: fennel

;; Return a new savestate representing the state of the emulator at the frame ;; boundary after the address addr is executed, or nil if ?frame-limit is ;; reached. (lambda await-exec [savestate ?frame-limit addr] (await-event savestate ?frame-limit #(bizhawk.event.on_bus_exec $ addr)))
As you can see, await-exec just passes the given savestate and ?frame-limit (name prefixed by a question mark to indicate it may be nil) to the await-event function, along with a function that describes how to register an event listener for the specific BizHawk event we're looking for (using the # and $ syntax for anonymous functions). await-event is the analog of what was called run_until_event in Post #532299, a core function whose purpose is to let the emulator run until an event happens, then return. The savestate module is what makes the data-oriented programming style possible. It's where the most complicated code is, but if you're interested in looking at it yourself, it's really not that bad: just 400 lines, plenty of which are comments.
Sand wrote:
The current version of the support code wouldn't work well for press-and-hold inputs (Kwirk only needs one-frame presses). In order to hold buttons in a convenient way, you may have to enrich the savestate object to support some kind of "sticky" inputs that don't become unpressed when the frame is advanced.
I implemented slightly enhanced joypad support in this version of the code. A savestate object stores, alongside a BizHawk-native savestate and the set of currently pressed joypad inputs, a schedule of inputs to happen in future frames. With functions like joypad-set and joypad-press-release, you can schedule button presses or releases to happen in the current frame or in a future frame. This is nice for press-and-hold inputs, as well as overlaying several actions that use different buttons.
Sand
He/Him
Experienced Forum User, Published Author, Player (147)
Joined: 6/26/2018
Posts: 209
YoshiRulz wrote:
CPP pushed a fix for pcall, so please try again with a new dev build.
That's great. A dev build after 0572266 works for me. Thanks for your help. It would have taken me much longer to find it on my own. Interesting that the cause was similar to JPC-RR builtin functions returning on the wrong thread. In that case, it affected every return of a builtin function, not just exceptions. So in one sense it was easier to notice. But also, JPC-RR only runs one Lua script at a time, rather than running each script in its own Lua thread, so it would only happen if you actually created new coroutines yourself, which was not the case here.
Post subject: Video of debugging and fix
Sand
He/Him
Experienced Forum User, Published Author, Player (147)
Joined: 6/26/2018
Posts: 209
I guess I never linked the video of fixing this bug. It's from 29:35 to 47:35 in the video below, which is part of my series of live TAS videos from [4345] DOS The Adventures of Captain Comic: Episode 1 - Planet of Death by Sand & Kabuto in 06:30.98. Link to video
Post subject: Name entry cursor movement
Sand
He/Him
Experienced Forum User, Published Author, Player (147)
Joined: 6/26/2018
Posts: 209
I haven't seen documentation of optimized cursor movement on the name entry menu, even though some published runs take advantage of it. I've posted new notes at Wiki: GameResources/NES/FinalFantasy1?revision=3#NameEntry. In short, the name entry of [2079] NES Final Fantasy by TheAxeMan in 1:09:57.70 could be done up to 5 frames faster; the name entry in [4468] NES Final Fantasy "game end glitch" by AmaizumiUni, Spikestuff & DJ Incendration in 01:36.47 is already optimized. Naively, to move the cursor from one point to another on the name entry screen, you would alternate between 1 frame of pressing a directional button and 1 frame of no input. When you need to do a diagonal movement, you save time by alternating the two directional buttons you need to press. For example, moving 2 spaces down and 3 spaces right (2D3R), a total distance of 5 spaces, can be done in 5 frames by alternating Right and Down inputs: ...R .D.. ...R .D.. ...R. But you can do even better than that. The name entry menu reacts to simultaneous directional inputs in unintuitive ways. For example, if you were pressing ...R on the previous frame, and press U..R on the current frame, the cursor moves 1 space down, despite the Down button never being pressed. Only U... and ..L. can move the cursor up or left, but there are many ways to move the cursor down or right, which means that you can often move at the maximum speed of 1 space per frame, even when moving straight horizontally or vertically.
A B C D E F G H I J
K L M N O P Q R S T
U V W X Y Z ’ , .  
0 1 2 3 4 5 6 7 8 9
a b c d e f g h i j
k l m n o p q r s t
u v w x y z - ‥ ! ?
[2079] NES Final Fantasy by TheAxeMan in 1:09:57.70 (and in turn namegenerator/inputgeneration.py from lightwarriorcode-v1.0.zip) applies the diagonal-movement optimization but not the simultaneous-buttons optimization. For example, the diagonal 2D2R movement from 'A' to 'W' in the name "Wedg" is already as fast as it can be:
|..|...R....| 'B'
|..|.D......| 'L'
|..|...R....| 'M'
|..|.D......| 'W'
|..|.......A|
But the 3R horizontal movement from 'd' to 'g' takes 5 frames when it could take only 3 frames:
|..|...R....| 'e'
|..|........|
|..|...R....| 'f'
|..|........|
|..|...R....| 'g'
|..|.......A|
You can move from 'd' to 'g' in only 3 frames by alternating ...R and ..LR:
|..|...R....| 'e'
|..|..LR....| 'f'
|..|...R....| 'g'
|..|.......A|
User movie #638766622812817201 is a demonstration of entering the "Wedg", "Bigs", "Axe ", "Viks" names from [2079] NES Final Fantasy by TheAxeMan in 1:09:57.70 5 frames faster. The inputs were computed using a shortest-path Lua script (a further development of the savestate manipulation technique from Post #532299) using the movement table from Wiki: GameResources/NES/FinalFantasy1#NameEntry. User movie #638766593655640386 is a partial resync of the original .fm2 to BizHawk 2.10, for easier comparison.
あいうえおがぎぐげご
かきくけこざじずぜぞ
さしすせそだぢづでど
たちつてとばびぶべぼ
なにぬねのぱぴぷぺぽ
はひふへほゃゅょっ。
まみむめも01234
らりるれろ56789
やゆよわん-!?‥ 
[4468] NES Final Fantasy "game end glitch" by AmaizumiUni, Spikestuff & DJ Incendration in 01:36.47 already applies both the diagonal-movement optimization and the simultaneous-buttons optimization. The only place the latter optimization is needed is in the name of the 4th party member, "ちごにむ". I believe the optimization was applied by Spikestuff in Post #505707 and User movie #71345684705421713. (The original submission in #7120: AmaizumiUni, Spikestuff & DJ Incendration's NES Final Fantasy "game end glitch" in 01:36.47 was replaced so I can't be sure.) To move 3D1R from 'あ' to 'ち', the movie uses a frame of U.L. to move the cursor down immediately after another down movement:
|..|.D......|........| 'か'
|..|...R....|........| 'き'
|..|.D......|........| 'し'
|..|U.L.....|........| 'ち'
|..|.......A|........|
The same trick is used to move 4D2R from 'ご' to 'に'. The 2D1R movement from 'に' to 'む' uses .D.. ...R U.L., but in this case the simultaneous buttons are unnecessary; .D.. ...R .D.. would also have worked.
1 2
8 9