TASVideos

Posted: 12 days ago
(Edited: 12 days ago)

Post subject: Investigation of page table dynamics

Sand

He/Him

Editor, Experienced Forum User, Published Author, Player (168)

Joined: 6/26/2018
Posts: 239

I spent time examining the Apple II emulator's simulated floppy disk drive, and the virtual memory implementation in the Z-code interpreter, in order to explain the "drifting" behavior I've spoken of: a small change in one command can cause later commands to run slower or faster, by up to dozens of frames, seemingly almost at random. I've determined, to my satisfaction, that the cause does not have anything to do with the simulated floppy drive. That part is consistent and predictable. The cause, rather, is the virtual memory paging code in the Apple II Z-code interpreter (6502 code), which behaves more like a random number generator than one would hope. The virtual memory page table is supposed to be an LRU cache; that is, when a page of memory is needed that is not already in the page table, the page table is supposed to evict the least recently used page in order to make room for the needed page to be swapped in. But because of a flaw in the implementation (or, perhaps, a deliberate design choice for the sake of simplicity), the page that gets evicted is not always the least recently used one–in fact, it is frequently one of the most recently used pages. When an evicted page is needed again later, it has to be swapped back in, which takes time. Sometimes, though, by chance, the wrongful eviction of a page happens to live up favorably with future disk access patterns. It is this, I believe, that gives rise to the "drift" in the timing of commands. Floppy disk drive In a past TAS project, I had the challenge of dealing with a simulated floppy drive (on a PC platform). There, the challenge was that accessing the disk incurred a delay of 1.2 s to allow the drive motor to spin up—but accessing the disk while the motor was already spinning incurred no additional delay. The general strategy was to avoid accessing the disk as much as possible—but then to batch many accesses together, while the disk was still spinning, when access was unavoidable. I thought something similar might be happening here. I also had a vague hypothesis that the emulator was simulating the rotation of the disk, or something, and that to read a sector one would have to wait until that sector spun around to arrive under the read head. Subtle changes of timing would affect the phase of the disk's rotation in a way that would make some disk accesses faster and some slower. Neither of the above is the case. The timing of different disk accesses throughout the run is indeed variable, but it breaks down into a few isolated components that are consistent and predictable. The simulated disk drive models which track is currently under the drive head, and seeking to different tracks take different amounts of time, but other than that, individual reads from disk are independent. Changes have only local effects: if you change one disk access in a sequence of accesses, it affects the timing of that one access and the one that follows it, at most. I believe it cannot result in timing drift many accesses into the future. Below I'll show evidence to support that claim. The directory experiments/getdsk in the TAS's source code repository contains raw data and analysis code. My notes on the subject start here. The BizHawk emulator uses the Virtu Apple II emulation core. Virtu includes a simulated Disk II floppy drive, implemented in the files DiskIIController.cs, DiskIIDrive.cs, and DiskDsk.cs. But the Disk II is software-controlled: most of the complexity of disk access lives in the program running on the Apple II, not in (simulated) hardware. The source of timing delays is, by and large, Infocom code (e.g. "ARM MOVEMENT DELAY"), not Virtu code. One memory page is 256 bytes, the same size as one disk sector. When the program needs to swap in a page, it calls the GETDSK subroutine, which converts a linear address to a track and sector, then calls the low-level subroutine DOS to read one sector from disk. I instrumented the call to DOS to measure the timing distribution of low-level disk reads. (In the 100% run, across all four combinations of game settings.)

A histogram, "cycles per call to DOS in GETDSK". The horizontal axis is "cycles" from 0 to 300,000 and "frames" from 0 to about 18. The vertical axis is "count" from 0 to 350. There is a prominent mode, with a count of about 350, at about 40,000 cycles. There are many sparse bars above that and a few below, with counts usually below 50. There's a prominent mode—about 20% of calls to DOS—that take around 40,000 CPU cycles, or 2 video frames, which is among the shortest times measured. But then there is a broad smear of times that extends up to 300,000 cycles, or 18 frames. Could this be the cause of the finicky timing of executing commands? No, I don't think so. To see why, understand that the DOS subroutine can be decomposed into a few blocks of code that take a fixed amount of time to execute, plus a call to SEEK and a GETADR loop that take variable amounts of time. The sums of these component timings is what gives rise to the distribution shown above. SEEK moves the read head to the requested track. Timing SEEK alone yields the following timing distribution:

A histogram, "cycles per call to SEEK in DOSEEK". The horizontal axis is "cycles" from 0 to 250,000 and "frames" from 0 to about 15. The vertical axis is "count" from 0 to 600. There are 15 non-zero bins. The largest, with a count of about 600, is at approximately 0 cycles. The next is at 50,000 cycles with a count of 280. Then the bins are spaced roughly evenly at intervals of about 20,000 cycles, for the most part smoothly decreasing in count. We see that only a few discrete timings are possible from SEEK. It turns out that the timing of SEEK depends entirely on how many tracks the read head has to travel over. Sometimes, the head is already over the right track; in that case the time required to seek is practically zero. The greater the absolute value of the difference between the current track (ctrack) and the desired track (ttrk), the more time it takes:

A scatterplot, "cycles per call to SEEK in DOSEEK, by distance". The horizontal axis is "ttrk − ctrk" from −30 to +30. The vertical axis is "cycles" from 0 to 250,000 and "frames" from 0 to about 15. The points are semitransparent but so heavily overplotted that all but the most extreme appear opaque. The plot is symmetrical about the middle. There is a point at (0, 0) at the bottom middle, and then the points increase smoothly on both sides. So the timing of SEEK depends only on the current track and the sought track. It cannot be the cause of long-term drift. What about GETADR, then? GETADR is a loop that repeatedly calls RDADDR until the disk drive reports that the sector currently being read is the right one. The timing distribution of the GETADR loop looks like this:

A histogram, "cycles per GETADR loop in DOS". The horizontal axis is "cycles" from 0 to 100,000 and "frames" from 0 to about 6. The vertical axis is "count" from 0 to 350. There are 16 main non-zero bins, evenly spaced at intervals of about 6,500 cycles starting from 0. The largest, with a count of about 350, is at about 20,000 cycles. The rest mostly have a count between 50 and 100. There are some shorter non-zero bins midway between some adjacent pairs of main bins. Again we see a distribution that is highly discrete. There are 16 main peaks: this makes sense as the disk format has 16 sectors per track. The peak with the highest count is the 4th one from the left, and I think there is a good reason for that. The sectors are stored on disk, not in numerical order, but in an "interleaved" order that means that logical sector n + 1 is usually stored 4 physical sectors after logical sector n. In the common case of reading sectors sequentially, the GETADR loop has to skip over 3 physical sectors between consecutive logical sectors. The Virtu emulator does not simulate continuous rotation of the disk. Instead, every time the track changes, it starts over with sector 0 of the track and serves the sectors to the application in order. That does it for the floppy disk simulation hypothesis. Disk access times are variable, but the timing of any given disk access depends only on the current track and sector and and track and sector to be read. Making a change in the middle of a sequence of accesses does not have long-term effects. The LRU page table that is not always LRU The cause of the timing variations, I am convinced, is the implementation of the virtual memory page table in the Z-code interpreter. The use of virtual memory by the Infocom interpreters is one of the most interesting facts about them, to me. The Zork I story file is about 83 KB, small enough to fit on a floppy disk but larger than the Apple II's 16-bit address space. (Even ignoring the space required for the interpreter itself.) So the interpreter inserts a virtualization layer between the program running in memory and the Z-code on disk. When a particular byte (at a particular address) is requested from the story file, the interpreter consults the page table to see if that page has been loaded into RAM; if it has, the interpreter looks up the address where the page is stored and reads the byte from there. If it has not, the interpreter reads the page from disk into RAM, updates the page table, then returns the requested byte. The page table has a finite size. The question is what to do when a page is requested that is not in the page table, but every slot in the page table is already occupied. The interpreter needs to evict some other page to make room for the page that is now needed. If that evicted page is needed again later, it will have to be paged back in from disk. This file apple/zip/npaging.h is a close match to the page table code that is present in the disk image I am using. (It's not an exact match: for example the code on the disk image lacks the EARLY2 subroutine and calls EARLY in its place; but it's closer than the files paging.asm and zpaging.asm in the same directory.) When the program fetches a byte of code (via NEXTPC) or data (via GETBYT), it calls the PAGE subroutine to find the necessary page in the page table, swapping it in if necessary. The program code intends to implement a least recently used (LRU) cache eviction policy. This is evidenced in comments like "TIME-STAMP PAGING ROUTINE" and labels like LRUMAP. But, in fact, when the time comes to evict a page from the cache, the page chosen is not always the least recently used one. Frequently, the page that is evicted is actually one that was used recently and will be needed again in the near future. This is because of an integer overflow in the incrementing counter that serves as a timestamp. (Call it a bug—there's nothing in the source code, in the form of comments or otherwise, that indicates it's supposed to act the way it does.) Here's how the page table is supposed to work. Each page table entry consists of two data fields: a 16-bit page ID and an 8-bit timestamp. There is a global 8-bit integer, STAMP, that increases by 1 every time any page is accessed. Whenever a page accessed, its timestamp is set to the current value of STAMP. The idea is that page table entries are brought up to date with the current STAMP whenever they are used, and unused pages' timestamps are left alone, so therefore you can find the least recently used page table entry by looking for the one with the oldest (lowest) timestamp. This is the ideal state of the page table: every entry's timestamp is less than or equal to STAMP, and the least recently used page has the lowest timestamp. While these conditions hold, the page table is a true LRU cache, but they almost never do. STAMP is an 8-bit counter—it can't keep increasing forever. What happens when STAMP overflows from 255 to 0? This is where things to go wrong. The page table code tries to do something to cope with the situation, but it does not really work, and STAMP very frequently ends up overflowing. When STAMP overflows, the page table does an operation I'll call "discounting". What it tries to do is shift the whole table back in time to make room for STAMP to continue increasing. Specifically: find the entry with the minimum timestamp, then subtract that minimum from every entry's timestamp, and from STAMP itself. If the minimum timestamp in the page table is 10, for example, then that entry's timestamp becomes 0, all other entries' timestamps get decreased by 10, and STAMP gets reset from 0 to 246, now with room to continue growing. The problem is, it's very very common for the minimum timestamp in the page table to be 0. When that happens, all timestamps and STAMP remain unchanged, and STAMP overflows from 255 to 0. To make matters worse, whatever page was just accessed gets marked with the new STAMP of 0—what should be the newest page in the table is instead tagged as being as old as it could possibly be! At the next page swap, the just-accessed page will be one of the leading candidates for eviction. (But it's not guaranteed to be the next page evicted: there can be ties between multiple entries with the same timestamp, and if the page continues to be accessed its timestamp will gradually increase as STAMP climbs back up from 0.) In the course of a run, STAMP overflows many times, and each time it overflows, some page gets tagged with an ultra-low timestamp that makes it prone to eviction. This haphazard eviction is sometimes favorable, sometimes unfavorable—it is hard to control and predict. Below is a graph that shows what real memory pages are resident at what times throughout the 100%-80-brief run. The directory experiments/page contains code to reproduce the graph, and my rough notes on the subject start here. A round dot indicates that a page was accessed: ideally, this should make it the most recently used page. The vertical colored strips show the value of the timestamp field of each live page table entry: darker is older; brighter is newer.

A large wide graph, "pages loaded into virtual memory and LRU timestamp". The horizontal axis is "page" from 0x50 to 0x150. The vertical axis is "cycle" from 0 to 315,000,000 and "frame" from 1 to 18,575 (increasing downward). A continuous color axis "stamp" ranges from 0 (black) to 255 (light blue). Strips of varying blue and black run vertically downward from starting dots, with dots punctuating most of the strips more than once. Some ranges of pages—from 0x50 to 0x70, around 0x90, and around 0x100—are frequently accessed and are mostly continuous from top to bottom. Other pages are only occasionally accessed and the strips are sparser and shorter-lived. When you see one of the vertical strips fade from bright blue to black, with no intervening dots, that's the discounting operation gradually decreasing that page table entry's timestamp. When you see a bright blue strip suddenly interrupted by a black dot, that's STAMP overflowing and a freshly accessed page being marked with a 0 timestamp. As you can see, STAMP overflows a lot—it's not at all an exceptional condition. Look at page 0x50 (at the far left) around cycle 240,000,000. The page, which had been bright blue, indicating a recent timestamp, is accessed as STAMP overflows, and becomes black. Then page 0x50 is evicted around cycle 244,000,000. If the page table were a proper LRU cache, that eviction would not have happened, because it's easy to see other pages (e.g. 0x5c, 0x78, 0x80) that were resident at the time and that had been accessed less recently. A demonstration of problems caused by anomalous page table eviction This STAMP overflow phenomenon is what makes the page table sensitively dependent on initial conditions—why small, local changes in page accesses can have accumulating, far-reaching effects. I tried an experiment with modifying just the final thief fight. I constrained the remarks after hitting the thief to be faster ones, which should only speed up the fight. And speed up the fight it does—but then unfavorable page table evictions make it slower by the end. Here's the before-and-after diff of the transcript of the thief fight:

Language: diff
 Your sword has begun to glow very brightly.
-You parry a lightning thrust, and the thief salutes you with a grim nod.
+The thief tries to sneak past your guard, but you twist away.
 
->Hit man
+>HIt mAn
 (with the sword)
-A savage blow on the thigh! The thief is stunned but can still fight!
-You parry a lightning thrust, and the thief salutes you with a grim nod.
+Slash! Your stroke connects! This could be serious!
+The thief tries to sneak past your guard, but you twist away.
 
->G
+>g
-A savage blow on the thigh! The thief is stunned but can still fight!
-You dodge as the thief comes in low.
+Slash! Your blow lands! That one hit an artery, it could be serious!
+You parry a lightning thrust, and the thief salutes you with a grim nod.
 
->g
+>G
 The thief takes a fatal blow and slumps to the floor dead.

At first, the modified fight is faster by 3 frames; but it ultimately works out to be 32 frames slower.

Before frames	Before command	After frames	After command	Cumulative advantage	Per-command advantage
15202	`s.u.w.w.u`	15202	`S.U.W.W.u`	0	+1
15661	`Hit man`	15660	`HIt mAn`	+1	+2
15874	`G`	15871	`g`	+3	0
15937	`g`	15934	`G`	+3	0
16100	`drop old`	16097	`drop old`	+3	0
16194	`get canary`	16191	`get canary`	+3	+7
16308	`temple`	16298	`temple`	+10	−5
16377	`d`	16372	`d`	+5	+11
16449	`get`	16433	`get`	+16	−26
16502	`u.s`	16512	`u.s`	−10	−4
16567	`pray`	16581	`Pray`	−14	+1
16653	`Wind canary`	16666	`wind canary`	−13	−16
16728	`get`	16757	`get`	−29	+12
16803	`e.s.e.w.w`	16820	`e.s.e.w.w`	−17	+1
17123	`drop all treasu in case`	17139	`drop all treasu in case`	−16	+12
17352	`w.w.u`	17356	`w.w.u`	−4	+11
17563	`drop all`	17556	`drop all`	+7	−39
17660	`Get all treasu`	17692	`get all treasu`	−32	−2
17821	`D`	17855	`d`	−34	+3
17887	`E`	17918	`E`	−31	−1
17935	`e`	17967	`e`	−32	0
18041	`drop all in case`	18073	`drop all in case`	−32	0
18280	`e.e.n.w.sw.w`	18312	`e.e.n.w.sw.w`	−32

The slightly changed fight results in drastic differences in the page table. The page table graph for the modified fight is here. But it's more instructive to look at an animation that flips between the two versions:

A 2-frame looping GIF animation, showing the lower left area of the page table graph: "cycles" from 265,000,000 to 315,000,000 and "pages" from 0x50 to 0xa4. The first frame is the same as in the graph above. The second frame shows the modified thief fight. There are places where the first frame has gaps and the second is connected, and vice versa. The dots (indicating page accesses) in the modified fight are shifted upward (indicating a time savings) until about cycle 280,000,000; but after that they are generally shifted downward, indicating a time loss. Let's look at a particular case of STAMP overflow to see its effects on the page table. The left group of columns is the original thief fight; the right group is the modified (faster) thief fight. This table doesn't show every page access, only ones that resulted in a swap.

cycle	frame	zpage	page	`STAMP`	cycle	frame	zpage	page	`STAMP`
272337867	15991	16	0x78	228	272287738	15988	16	0x78	224
272716133	16014	61	0xd7	254	272663464	16011	61	0xd7	254
272811368	16019	24	0xd2	255	272758699	16016	24	0xd2	255
273315572	16049	24	0x95	5	273262882	16046	12	0x95	0
275050196	16151	12	0x76	219	274999096	16148	12	0x76	213
275108188	16154	53	0x77	223	275057076	16151	1	0x77	217
275255591	16163	24	0x92	225	275204491	16160	61	0x92	219
276680776	16246	1	0x87	165	276629712	16243	60	0x87	161
276881986	16258	25	0xf1	168	276830922	16255	37	0xf1	164
277037566	16267	61	0x90	171	277153667	16274	43	0x54	188
277384683	16288	37	0x100	218	277981061	16323	45	0x8a	23
278212278	16336	43	0x8a	24	278140336	16332	22	0x93	25

On the left side, notice how page 0xd2 is swapped into page table entry 24 when STAMP is 255. Then, after STAMP overflows, page 0xd2, despite being recently used, is swapped out in favor of page 0x95. Something similar happens on the right side, but here page 0x95 does not evict page 0xd2 but instead goes into page table entry 12. But then the newly swapped-in page 0x95 is evicted to make way for page 0x76 in entry 12. The two page tables diverge further from there.

Posted: 9/10/2026 1:12 PM
(Edited: 9/10/2026 1:40 PM)

Post subject: A new route: dam, mine, maze, temple, falls

Sand

He/Him

Editor, Experienced Forum User, Published Author, Player (168)

Joined: 6/26/2018
Posts: 239

I've managed to save an additional 769 frames or over 12 seconds with a new route. If the previous route was "maze, temple, dam, falls, mine", this one is "dam, mine, maze, temple, falls". Doing the dam before the temple means not having a second source of light for the dumbwaiter puzzle, which requires traversing dark rooms and manipulating RNG to avoid grue attacks. Link to video

Here's a summary of the time saved through changes to the route so far, using the "80 columns, brief" run as the basis of comparison:

Route	Moves	Frames	Time
baseline	258	25139	6:58.98
baseline+tweaks	257	22848	6:20.80
thief-based route	236	20685	5:44.75
alternative commands	239	19759	5:29.32
`COFFIN` last	231	19263	5:21.05
this	222	18494	5:08.24

As before, the way the sequence of commands is actually entered into the emulator is with a Fennel script. The script takes care of things like finding the earliest moment at which an input prompt starts accepting keyboard input. But its most important function is manipulating RNG. Whenever a command has an undesired result (the songbird singing, taking damage in combat, the bat carrying the player to the wrong place, being eaten by a grue), the script backtracks and tries commands again with different letter case, which changes the results of random rolls. This RNG manipulation is what lets me execute the same route on four different combinations of game settings, even though each combination's internal RNG sequence follows a different trajectory.

Settings	Frames	Time
80 columns, brief	18494	5:08.24
40 columns, brief	17299	4:48.32
80 columns, superbrief	15096	4:11.60
40 columns, superbrief	14454	4:00.90

Doing the mine early in the route saves quite a few moves and adds a thrilling complication. There's a place in the mine where you're supposed to have two sources of light (i.e., one besides the LAMP you start with). The temple is the only place to get a second source of light (the CANDLES or TORCH). This means we have to do part of the mine sequence in the dark, under threat of grue attack. In the final return trip through the mine, the route goes through nine consecutive dark rooms:

>drop all in cage
huge diamond: Done.
brass lantern: Done.
screwdriver: Done.

>lift cage
The basket is raised to the top of the shaft.

>eaSt
You have moved into a dark place.
It is pitch black. You are likely to be eaten by a grue.

>EASt
You have moved into a dark place.
It is pitch black. You are likely to be eaten by a grue.

>gO up
You have moved into a dark place.
It is pitch black. You are likely to be eaten by a grue.

>go up
You have moved into a dark place.
It is pitch black. You are likely to be eaten by a grue.

>north
You have moved into a dark place.
It is pitch black. You are likely to be eaten by a grue.

>east
You have moved into a dark place.
It is pitch black. You are likely to be eaten by a grue.

>sOuth
You have moved into a dark place.
It is pitch black. You are likely to be eaten by a grue.

>NOrtH
You have moved into a dark place.
It is pitch black. You are likely to be eaten by a grue.

>gO up
You have moved into a dark place.
It is pitch black. You are likely to be eaten by a grue.

>south
Shaft Room
At the end of the chain is a basket.
The basket contains:
  A screwdriver
  A brass lantern (providing light)
  A huge diamond

Every time you step from one dark room to another, there is a 79% chance of grue attack. But that doesn't mean the odds of surviving this sequence of rooms are 0.21⁹ (≈ 1 in 1,259,000); that's because we have a chance to manipulate RNG between rooms. You see the case variation in the excerpt above. For this section I've used longer forms of movement commands—north instead of n—this is to give the execution script more fodder and speed up the manipulation. I think, in a final version, it will be possible to use the usual short movement commands here as well; it will just take more time to solve the manipulation. Besides the major routing change, this run has more tightly optimized inventory management. The thief helps make the run fast by collecting treasure that is awkward for us to pick up or deliver, but for treasure objects we are obligated to pick up anyway and which we can afford to carry, it's usally faster to keep them in the player inventory and deliver them to the trophy case directly. This route does more of that, and as a result has to make one fewer trek back to the TREASURE-ROOM to recover things collected by the thief. This new route features the fewest moves (222) of any I've shown so far. I found a minor modification (letting the thief take the PAINTING) that's only 221 moves, but it was not faster in real time. As things are, I don't have further ideas for route changes. There is still the nagging doubt of the page table, which can make seemingly inconsequential changes to the sequence of commands balloon into differences of hundreds of frames, for no obvious reason. It's possible that some of the minor route changes I've tested and found to be slower would actually be faster with the right page table massaging, but it would be too much work to investigate them all exhaustively. I'd like to do some more investigation and at least get an understanding of why it is so sensitive. I'll also need to do some work to implement the automatic testing of alternative commands I've mentioned.

Posted: 9/3/2026 5:03 AM
(Edited: 9/3/2026 2:12 PM)

Post subject: Combat remark timing

Sand

He/Him

Editor, Experienced Forum User, Published Author, Player (168)

Joined: 6/26/2018
Posts: 239

Another thing I did is time random combat remarks. When you or the enemy takes a turn in combat, there can be one of six results: MISSED, UNCONSCIOUS, KILLED, LIGHT-WOUND, SERIOUS-WOUND, STAGGER. These are looked up in probability tables depending on the strength of the attacker and defender. But for each combat result, there is also a table of flavor remarks. This gives the illusion of more variety in combat than there really is. For example, the player landing a SERIOUS-WOUND result can display any one of these remarks, uniformly at random:

"The enemy receives a deep gash in his side."
"A savage blow on the thigh! The enemy is stunned but can still fight!"
"Slash! Your blow lands! That one hit an artery, it could be serious!"
"Slash! Your stroke connects! This could be serious!"

I ran an experiment to time how long it takes to print all the relevant combat remarks, in order to see whether it matters to use RNG manipulation to control them. In short, it matters, but only by 1 or 2 frames per remark. When the player is the attacker and the result is KILLED, all three remarks actually take the same amount of time:

Remark #	Frames (80 columns)	Frames (40 columns)	Remark
1	125	119	"It's curtains for the troll as your sword removes his head."
2	125	119	"The fatal blow strikes the troll square in the heart: He dies."
3	125	119	"The troll takes a fatal blow and slumps to the floor dead."

When the player is the attacker and the result is SERIOUS-WOUND, remark 4 is faster than the others by 1 or 2 frames:

Remark #	Frames (80 columns)	Frames (40 columns)	Remark
1	42	42	"The thief receives a deep gash in his side."
2	42	42	"A savage blow on the thigh! The thief is stunned but can still fight!"
3	42	41	"Slash! Your blow lands! That one hit an artery, it could be serious!"
4	41	40	"Slash! Your stroke connects! This could be serious!"

When the thief is the attacker and the result is MISSED, remark 2 is faster than the others by 1 or 2 frames:

Remark #	Frames (80 columns)	Frames (40 columns)	Remark
1	45	44	"The thief stabs nonchalantly with his stiletto and misses."
2	44	42	"You dodge as the thief comes in low."
3	46	44	"You parry a lightning thrust, and the thief salutes you with a grim nod."
4	45	44	"The thief tries to sneak past your guard, but you twist away."

Despite knowing that some combat remarks are faster than others, I'm not currently constraining the remarks during the final thief fight. That's because the fight already requires manipulating a sequence of low-probability combat results, and manipulating the remarks as well would require even more RNG grinding.

Posted: 9/3/2026 4:14 AM

Post subject: Progress, COFFIN last

Sand

He/Him

Editor, Experienced Forum User, Published Author, Player (168)

Joined: 6/26/2018
Posts: 239

Here is a progress update. User file #639240050613301063 finishes the game in 19263 frames (5:21.05), about 8 seconds faster than the 19759 frames (5:29.32) of Post #544899. The transcript is here. You can see the changes to the orchestration program here. The biggest change is saving the COFFIN for last and hauling it outside in the same trip with the CANARY at the end. There are other, more minor optimizations, some of which I think are fun:

In LIVING-ROOM, in order to pick up the LAMP and SWORD, I used to do get all followed by light:

>get all
trophy case: The trophy case is securely fastened to the wall.
sword: Taken.
brass lantern: Taken.
carpet: The rug is extremely heavy and cannot be carried.

>light
(brass lantern)
The brass lantern is now on.

I changed it to light followed by get.

>light
(brass lantern)
(Taken)
The brass lantern is now on.

>get
(sword)
Taken.

The light command has an implicit take for whatever you're lighting. After that, the sword is the only thing left in the room with TAKEBIT set, so get suffices to take it.

There's a tricky new interaction with the bat in BAT-ROOM. We need to enter the BAT-ROOM carrying the GARLIC so the bat won't attack us, pick up the JADE, proceed to the SHAFT-ROOM, drop some things off in the dumbwaiter, then return to BAT-ROOM without the GARLIC so the bat does attack us. It used to work like this:

Bat Room
You are in a small room which has doors only to the east and south.
In the corner of the room on the ceiling is a large vampire bat who is
obviously deranged and holding his nose.
There is an exquisite jade figurine here.

>get
(jade figurine)
Taken.

>e
Shaft Room
This is a large room, in the middle of which is a small shaft descending
through the floor into darkness below. To the west and the north are exits from
this room. Constructed over the top of the shaft is a metal framework to which
a heavy iron chain is attached.
At the end of the chain is a basket.

>drop torch,screw in cage
torch: Done.
screwdriver: Done.

>eat
(clove of garlic)
What the heck! You won't make friends this way, but nobody around here is too
friendly anyhow. Gulp!

>w
Bat Room
You are in a small room which has doors only to the east and south.
A large vampire bat, hanging from the ceiling, swoops down at you!
    Fweep!
    Fweep!
    Fweep!


The bat grabs you by the scruff of your neck and lifts you away....

Ladder Top

I changed it to do this instead:

Bat Room
You are in a small room which has doors only to the east and south.
In the corner of the room on the ceiling is a large vampire bat who is
obviously deranged and holding his nose.
There is an exquisite jade figurine here.

Shaft Room
This is a large room, in the middle of which is a small shaft descending
through the floor into darkness below. To the west and the north are exits from
this room. Constructed over the top of the shaft is a metal framework to which
a heavy iron chain is attached.
At the end of the chain is a basket.

>drop torch,screw in cage
torch: Done.
screwdriver: Done.

>w
Bat Room
In the corner of the room on the ceiling is a large vampire bat who is
obviously deranged and holding his nose.
There is an exquisite jade figurine here.

>eat
(clove of garlic)
What the heck! You won't make friends this way, but nobody around here is too
friendly anyhow. Gulp!

>get jade,bat
jade figurine: Taken.
bat:     Fweep!
    Fweep!
    Fweep!


The bat grabs you by the scruff of your neck and lifts you away....

Ladder Top

How this works is we simply pass through BAT-ROOM the first time, without picking up the JADE. We stop at the dumbwaiter, then return to BAT-ROOM, still holding the GARLIC. We eat the GARLIC, which reactivates the bat, though it doesn't immediately attack us. We pick up the JADE and attempt to pick up the bat on the same turn, which makes the bat attack and carry us to where we want to be. Check out the text alignment bug on the "Fweep!" message from the game not planning for this interaction.

Next, I'm planning to test a significant change to the route. Currently, it goes: Temple, Dam, Falls, Mine. But I have a feeling Temple, Dam, Mine, Falls will be faster. Dam must come before Mine because you need the SCREWDRIVER in the mine. Dam must come before Falls because you need the boat for the river. Temple must come before Mine because you need a second source of light (the TORCH or the CANDLES). Temple must come before Falls because you need the SCEPTRE for the rainbow bridge.

Posted: 9/3/2026 3:58 AM

Sand

He/Him

Editor, Experienced Forum User, Published Author, Player (168)

Joined: 6/26/2018
Posts: 239

YoshiRulz wrote:

I think something like MCTS would be a good fit for that: start with your naive route, then make random perturbations and keep the fastest, moving towards a local minimum.

I think Monte Carlo tree search and the process you describe are two different things; but the process you describe is certainly a way to take a given route and wiggle it into a local minimum. One challenge is that testing a minor perturbation to the route is not exactly cheap, computationally. It's automated with the driver script, but it still takes at least tens of minutes to execute a complete run, or longer depending on how much work the RNG manipulation takes (the final battle with the thief is especially variable because a lot of low-probability rolls have to happen in sequence). You can cache the state of the game before the change using something like Post #544738, but everything that comes after has to be re-emulated. So it'll depend on one's appetite for letting it run and when you decide to call it done. The incremental greedy technique has the advantage that you're only timing each command in isolation, choosing the local fastest, and then moving on. So you can more quickly test a lot more variations. The disadvantage, of course, is that you're only looking at the timing of that one command, and not how it may effect other commands much later in the future. Really I'd like to get a better handle on what's going on with the page table (if indeed that is the source of the variation) and see if it can be tamed in some way.

Posted: 8/29/2026 5:59 AM

Post subject: A greedy strategy for choosing equivalent commands

Sand

He/Him

Editor, Experienced Forum User, Published Author, Player (168)

Joined: 6/26/2018
Posts: 239

On the topic of different but equivalent commands taking different amounts of time to execute, I did an experiment with a greedy strategy. I thought up a few equivalent alternatives for selected commands, such as get / get bell / get small / get all for picking up the BELL. Then, starting from the beginning, I tried all of the alternatives for the commands that had them, and kept the alternative that reached the next command prompt with the lowest frame count. And so on to the end of the run. Such a strategy is not guaranteed to find an optimal sequence of commands. As we've seen, a command that is faster in isolation may cause later commands to run slower. (I suspect it's because of differences in the virtual memory page table and resulting changes in access patterns to the emulated floppy disk drive.) Nevertheless, the greedy strategy is a decent heuristic, and in this case it cuts 688 frames, or over 11 seconds, from the previous record of 20447 frames. The new record is 19759 frames, or 5:29.32 at 60 FPS. That's not a bad improvement, considering it doesn't involve any macroscopic changes to the route. Two alternative commands saved a lot of time on their own. These were 205 frames from changing the two-part command put all in case;get clove,lamp,driver,torch to the one-part command put all but clove,lamp,screw,torch in case; and 96 frames from changing push all to push yellow in the dam maintenance room. The rest of the time savings came from just a lot of small cumulative improvements. The table of command alternatives and their timing is below. First, some general observations:

The choice between SYNONYM and ADJECTIVE, when referring to objects, doesn't matter much. The difference between in time sceptr (a SYNONYM) and sharp (an ADJECTIVE) is always small; sometimes the former is faster and sometimes the latter. The length of the word matters for speed of entry, so choose a short one. It also seems to matter where in the SYNONYM/ADJECTIVE list the word is. Words that are later in the lists seem to take slightly more time to access, on average. You can see this in the difference of book and page (positions 1 and 3, respectively, in the SYNONYM list of BOOK).
When a room contains a single gettable object, just plain get is usually the fastest way to pick it up, with get all close behind. get SYNONYM and get ADJECTIVE are usually slower by 10 or more frames. With other verbs, too, leaving the object to be inferred by the parser is usually faster. (See launch versus launch boat for example, where the former is faster even accounting for the different command lengths.)
When possible, identifying a group of objects with all SYNONYM or all ADJECTIVE, where SYNONYM or ADJECTIVE is something they have in common, is usually faster than alternative formulations like all but OTHERS. See at the end of the run, where drop all treasu in case is better than drop all but lamp in case.
drop X in Y and move X in Y both tend to be slightly faster ways to achieve the V-PUT effect than put X in Y, even though they are 1 character longer to type. There are other, exotic, ways to do a V-PUT, like hurl X in Y, squeeze X on Y, and apply X to Y, but they are not faster.
The order in which multiple items are listed (when separated by commas) can have a small effect on timing. See for example get huge,torch versus get torch,huge.
The word it is so much slower than other ways of referring to objects it's not worth considering.

I think I can automate greedy optimization like this in the Fennel script that coordinates the TAS. List alternatives for commands, try them all, and keep the fastest one. But it will increase the time needed for emulation and slow down testing and iteration, so I may leave it for a finishing touch after settling large-scale route changes. Here's the table of alternatives I tried. Indented lines following a command show alternatives and their cost in frames relative to the chosen command (higher numbers are slower).

"n.n.u"
"get egg"
     +2 "get encrus"
     +2 "get jewele"
     +4 "get treasu"
"d.s.e"
"open"
    +13 "open kitche"
    +14 "open small"
    +15 "open window"
"w.w"
"get old,lamp"
     +1 "get old,brass"
     +2 "get lamp,old"
     +3 "get brass,old"
     +4 "get all"
     +4 "get old and lamp"
     +4 "get sword,lamp"
     +5 "get sword,brass"
"tug rug"
     +2 "tug carpet"
    +11 "tug large"
    +13 "tug orient"
"open trap"
     =0 "open dusty"
     +1 "open cover"
     +3 "open trapdo"
"light"
"d.n"
"hit nasty"
     =0 "hit nasty with sword"
     +3 "attack nasty"
     +4 "hit troll"
     +5 "fight troll"
     +7 "attack troll"
    +11 "hit nasty with old"
    +19 "hit it"
    +20 "hit him"
"w.s.e.u"
"get bag"
     +2 "get all bag"
     +2 "get all old"
     +2 "get coins"
     +5 "get old"
"sw.e.s.se"
"odysse"
     +1 "ulysse"
"drop all but lamp"
     +1 "drop all old,egg"
    +24 "drop egg,all old"
    +38 "drop egg,bag,sword"
"u"
"treasu"
     =0 "temple"
"get"
    +12 "get all"
    +17 "get bell"
    +18 "get small"
    +22 "get all bell"
"d"
"get"
    +25 "get all"
    +32 "get solid"
    +33 "get casket"
    +34 "get coffin"
"open solid"
     +1 "open casket"
    +26 "open it"
"get sharp"
     +2 "get scepte"
     +2 "get sceptr"
"u"
"treasu"
     =0 "temple"
"drop solid"
"temple"
     =0 "treasu"
"n"
"get"
     +2 "get all"
    +10 "get flamin"
    +11 "get ivory"
    +11 "get torch"
"s.s"
"get all"
    +27 "get book,pair"
    +27 "get page,pair"
    +36 "get pair,book"
"d.d"
"drop pair"
     +2 "drop candles"
    +31 "drop flamin"
"ring bell"
     =0 "ring small"
"get"
    +10 "get pair"
"read"
    +14 "read book"
    +15 "read page"
"drop pair,book"
     =0 "drop book,pair"
     +1 "drop page,pair"
     +1 "drop pair,page"
"s.n.u.n.n.n.w.n.ne.e.n.n"
"push yellow"
    +17 "push yellow switch"
    +43 "push all switch"
    +96 "push all"
"get all tool"
    +22 "get screw,wrench"
    +23 "get wrench,screw"
    +27 "get wrench,screwd"
    +29 "get all"
"s.s"
"set bolt with wrench"
     +1 "set nut with wrench"
     +2 "turn bolt with wrench"
     +2 "turn nut with wrench"
"d"
"get"
     =0 "get all"
    +10 "get boat"
    +11 "get pile"
"drop wrench"
"z"
     +3 "wait"
    +91 "hi" "hi" "hi"
"z"
     +6 "wait"
    +13 "g"
"z"
"u.w.n.n"
"get"
     =0 "get all"
    +11 "get pump"
"n.s.s.s.se.d"
"echo"
"e.e.s"
"drop boat"
     +1 "drop pile"
     +3 "drop plasti"
"pump up boat"
     +4 "inflat boat with pump"
    +39 "pump up boat with pump"
"drop sceptr in boat"
     =0 "drop scepte in boat"
     +1 "move sceptr in boat"
     +3 "apply sharp to boat"
     +3 "drop sharp in boat"
     +3 "move sharp in boat"
     +6 "squeeze sharp on boat"
    +14 "put sharp in boat"
    +15 "apply sceptr to boat"
    +18 "squeeze sceptr on boat"
    +30 "put sharp in it"
    +32 "put scepte in it"
    +32 "put sceptr in it"
    +34 "put sharp in magic"
    +36 "put scepte in boat"
    +36 "put sceptr in boat"
"enter boat"
     +9 "board"
    +22 "get in"
    +23 "board boat"
    +28 "get in boat"
"launch"
    +14 "launch boat"
"get red"
     =0 "get all"
     +2 "get buoy"
"e"
"disemb"
     +8 "get out boat"
    +14 "get out"
    +15 "disemb boat"
    +16 "leave boat"
"open red"
     +2 "open buoy"
"get large"
     +3 "get emeral"
    +14 "get treasu"
    +28 "get it"
"n"
"get all"
     +1 "get"
    +14 "get shovel"
"ne"
"dig sand with shovel"
"g"
    +50 "dig sand with shovel"
"g"
"g"
"sw.s"
"get sceptr"
     =0 "get scepte"
     +2 "get sharp"
"s"
"wave sharp"
     +4 "wave sceptr"
     +5 "wave scepte"
"w.w.sw.u.u.nw.w.w"
"open brown"
     +1 "open bag"
     +4 "open sack"
"get clove"
     +1 "get garlic"
"w"
"open trophy"
     =0 "open case"
"put all but clove,lamp,screw,torch in case"
     +1 "drop all but clove,lamp,driver,torch in case"
     +1 "drop all but torch,driver,lamp,clove in case"
     +2 "drop all but clove,lamp,screwd,torch in case"
     +3 "put all but clove,lamp,driver,torch in case"
     +3 "put all but torch,driver,lamp,clove in case"
     +4 "put all but clove,lamp,screwd,torch in case"
     +6 "drop all but clove,lamp,screw,torch in case"
     +7 "move all but clove,lamp,screw,torch in case"
    +26 "drop sharp,shovel,large,red,pump in case"
    +28 "put sharp,shovel,large,red,pump in case"
   +205 "put all in case" "get clove,lamp,driver,torch"
"open trap"
     =0 "open dusty"
     +4 "open cover"
     +6 "open trapdo"
"w.w.u"
"temple"
     =0 "treasu"
"s.d.n"
"rub enormo"
     =0 "pat enormo"
     =0 "pat reflec"
     +1 "rub mirror"
     +1 "rub reflec"
     +6 "rub all enormo"
"n.w.n.w.n"
"get"
    +19 "get all jade"
    +21 "get figuri"
    +23 "get all"
"e"
"drop torch,screw in cage"
     +2 "put torch,screw in cage"
     +3 "drop tool,torch in cage"
     +3 "drop torch,screwd in cage"
     +3 "drop torch,tool in cage"
     +4 "put tool,torch in cage"
     +5 "put torch,screwd in cage"
     +5 "put torch,tool in cage"
"eat"
    +28 "eat clove"
    +29 "eat garlic"
"w"
"d.s"
"get"
    +20 "get all"
    +26 "get coal"
    +28 "get small"
"n.u.u.n.e.s.n"
"get"
    +25 "get all"
    +34 "get bracel"
    +34 "get jewel"
    +36 "get sapphi"
"u.s"
"drop coal in cage"
     +1 "move coal in cage"
     +2 "hurl coal in cage"
     +2 "put coal in cage"
     +3 "drop coal down cage"
     +3 "put pile in cage"
     +3 "put small in cage"
    +16 "apply coal to cage"
    +17 "put all coal in cage"
    +19 "squeeze coal on cage"
"lower cage"
     +2 "lower basket"
"w"
"drop all"
"w"
"w"
"get all from cage"
"s"
"open lid"
     +1 "open dryer"
     +2 "open machin"
"drop coal in lid"
     +1 "move coal in lid"
     +2 "put coal in lid"
"close lid"
"set switch"
"open lid"
"get"
     +6 "get huge"
    +11 "get diamon"
"n"
"put all in cage"
"e"
"e"
"get"
     +6 "get all"
    +10 "get lamp"
"u.u.n.e.s.n.u.s"
"lift cage"
     +1 "raise cage"
"get huge,ivory"
     +1 "get huge,torch"
     +1 "get ivory,huge"
     +1 "get torch,huge"
     +5 "get diamon,torch"
    +11 "get all from cage"
    +14 "get all treasu from cage"
"w"
"s.d.s.e"
"get"
     +1 "get all"
     +7 "get art"
    +11 "get painti"
"w.n.u"
"put all treasu in case"
    +11 "put all but lamp in case"
"w.w"
"get"
     +3 "get all"
     +7 "get old"
    +10 "get sword"
"u"
"hit man"
     +3 "hit thief"
"g"
    +53 "hit man"
"g"
"drop old"
     +2 "drop sword"
"get solid,pot,canary"
     =0 "get pot,canary,solid"
     =0 "get solid,canary,pot"
     +2 "get canary,pot,solid"
     +5 "get canary,pot,casket"
"temple"
     =0 "treasu"
"s"
"pray"
"e"
"wind canary"
     =0 "wind golden"
"get"
     =0 "get all"
    +10 "get bauble"
    +12 "get brass"
"s.e.w.w"
"drop all treasu in case"
     =0 "move all treasu in case"
     +2 "put all treasu in case"
    +12 "drop all but lamp in case"
    +14 "put all but lamp in case"
"w.w.u"
"get all treasu"
"d.e.e"
"drop all treasu in case"
"w.w.u"
"get all treasu"
"d.e.e"
"drop all treasu in case"
     +2 "put all treasu in case"
    +18 "drop all in case"
    +20 "put all in case"

Posted: 8/26/2026 5:58 AM
(Edited: 8/26/2026 6:19 AM)

Post subject: Experiments with equivalent ways of expressing commands

Sand

He/Him

Editor, Experienced Forum User, Published Author, Player (168)

Joined: 6/26/2018
Posts: 239

The Gym Slow route is optimized for reducing keystrokes. The idea is that if you're playing on a modern PC, the reaction to your commands is instant and the bottleneck is how fast you can type them in. On the emulated Apple II with its emulated floppy drive, however, fewer keystrokes is not necessarily faster. There can be many ways to type a command that have an equivalent effect. Expressing a command in a different way can make it run faster, even if sometimes means typing more characters. I saw an instance of this while I was preparing the new route of Post #544754. Near the end, there's a place where we are holding many pieces of treasure, and the lamp. We want to put all the treasure in the trophy case, but keep the lamp. I found that the straightforward command put all but lamp in case was about 51 frames slower than put all treasu in case, which has the same effect. (The Zork I command parser only looks at the first 6 letters of each word. So you can always abbreviate treasure to treasu. That's separate from the main point.) The game's command parser is complicated and I am far from fully understanding it, but you can get some insight into how it works by seeing how objects are defined. Every object has a set of SYNONYM words and a set of ADJECTIVE words. Take the definition of SCARAB for example:

<OBJECT SCARAB
	(IN SANDY-CAVE)
	(SYNONYM SCARAB BUG BEETLE TREASURE)
	(ADJECTIVE BEAUTI CARVED JEWELED)
	(DESC "beautiful jeweled scarab")
	(FLAGS TAKEBIT INVISIBLE)
	(SIZE 8)
	(VALUE 5)
	(TVALUE 5)>

We see the SYNONYMs of the object are scarab, bug, beetle, and treasure. Its ADJECTIVEs are beautiful, carved, and jeweled. This means that any of these commands work to pick up the scarab:

get scarab
get bug
get beetle
get jeweled scarab
get beautiful jeweled scarab

and even:

get treasure (if there's nothing else associated with the word "treasure" in sight)
get jeweled (if there's nothing else "jeweled" in sight)

and even:

get it (if the scarab is the most recent thing the game has mentioned)

That's the intuition behind the Gym Slow route using get solid instead of get coffin, and deflat it instead of deflat boat. But different options take different code paths through the parser and take different amounts of time to execute. (And modify the virtual memory page table in different ways, most likely.) You can see this in the Visible Zorker. Uncheck the "Collapse non-printing calls" and observe the differences in the call stack when you open the MAILBOX in different ways:

open mailbox (by SYNONYM, 52 calls)
open box (by SYNONYM, 52 calls)
open small (by ADJECTIVE, 51 calls)
open small mailbox (by ADJECTIVE and SYNONYM, 57 calls)
open it (by it, 89 calls)
open all mailbox (50 calls)

And different ways of getting the coffin:

get coffin (70 calls)
get solid (69 calls)
get solid coffin (78 calls)
get (63 calls)
get all (58 calls)

In general, it seems that it should be avoided, that it's slightly faster (fewer subroutine calls, anyway) to refer to objects by an ADJECTIVE rather than a SYNONYM, that letting the game infer an object can be faster than specifying it (both in typing and in computation time), and that adding all seems to save the parser a bit of work in searching for what object you're referring to. I did an experiment with changing most of the object references in the run of Post #544754 to change most of the object references to use a SYNONYM. The results are promising but mixed: the run was overall faster by about 1%, and the specific commands that were changed seemed to have a beneficial effect on balance, but there is a lot of noise such that it's hard to say for sure it's better. (The noise, I believe, comes from the Z-machine interpreter's virtual memory paging system, which is hard to predict and control; see Post #538861 and Post #544799.) Here's a table of the original and changed commands, as well as frame timestamps and the advantage of the SYNONYM commands (positive is better):

Chg?	Before frames	Before command	After frames	After command	Cumulative advantage	Per-command advantage
	521	`N.n.u`	521	`N.n.u`	0	0
	761	`get egg`	761	`get egg`	0	0
	920	`d.s.e`	920	`d.s.e`	0	0
	1016	`open`	1016	`open`	0	0
	1102	`w.w`	1102	`w.w`	0	0
	1331	`get all`	1331	`get all`	0	0
✓	1454	`tug it`	1454	`tug rug`	0	21
✓	1554	`open it`	1533	`open cover`	21	20
	1643	`light`	1602	`light`	41	0
	1711	`D.n`	1670	`d.n`	41	20
✓	1927	`HIT It`	1866	`HIT troll`	61	9
	2161	`w.s.e.u`	2091	`w.s.e.u`	70	69
✓	2487	`get old`	2348	`get bag`	139	14
	2645	`sw.e.s.se`	2492	`sw.e.s.se`	153	0
	2890	`odysse`	2737	`odysse`	153	15
	2979	`drop all but lamp`	2811	`DRoP aLl but lamp`	168	12
	3121	`u`	2941	`U`	180	2
	3366	`temple`	3184	`temple`	182	0
	3476	`get`	3294	`get`	182	7
	3600	`d`	3411	`D`	189	0
	3668	`get`	3479	`get`	189	2
✓	3727	`open it`	3536	`open casket`	191	38
✓	3854	`get sharp`	3625	`get sceptr`	229	0
	3937	`u`	3708	`u`	229	15
	3983	`TReasu`	3739	`TReasu`	244	5
✓	4103	`DrOp solid`	3854	`DRop casket`	249	7
	4216	`temple`	3960	`temple`	256	−8
	4263	`n`	4015	`n`	248	1
	4354	`get`	4105	`Get`	249	43
	4472	`s.s`	4180	`s.s`	292	31
	4612	`get all`	4289	`get all`	323	−86
	4701	`d.d`	4464	`d.d`	237	2
	4857	`drop pair`	4618	`drop pair`	239	−67
	4941	`ring bell`	4769	`ring bell`	172	−13
	5035	`get`	4876	`get`	159	−36
	5115	`read`	4992	`read`	123	0
	5179	`drop pair,book`	5056	`drop pair,book`	123	−11
	5270	`s.n.u.n.n.n.w.n.ne.e.n.n`	5158	`s.n.u.n.n.n.w.n.ne.e.n.n`	112	−16
	6009	`push all`	5913	`push all`	96	25
	6255	`get all tool`	6134	`get all tool`	121	−10
	6370	`s.s`	6259	`s.s`	111	−68
	6523	`set nut with wrench`	6480	`set nut with wrench`	43	−21
	6639	`d`	6617	`d`	22	−1
	6716	`get`	6695	`get`	21	27
	6818	`drop wrench`	6770	`drop wrench`	48	23
	6904	`wait`	6833	`wait`	71	0
	6939	`g`	6868	`g`	71	−6
	6974	`g`	6909	`g`	65	1
	7006	`u.w.n.n`	6940	`u.w.n.n`	66	−5
	7314	`get`	7253	`get`	61	−12
	7384	`N.s.s.s.se.d`	7335	`n.s.s.s.se.d`	49	13
	7813	`echo`	7751	`echo`	62	13
	7881	`E.e.s`	7806	`e.e.s`	75	5
	8096	`drop boat`	8016	`drop boat`	80	−16
✓	8184	`pump it up`	8120	`pump up boat`	64	37
✓	8354	`put sharp in it`	8253	`put sceptr in it`	101	−2
	8539	`Board`	8440	`board`	99	−4
	8610	`launch`	8515	`launch`	95	−15
✓	8760	`get red`	8680	`get buoy`	80	46
	8925	`e`	8799	`e`	126	15
	9006	`disemb`	8865	`disemb`	141	−5
✓	9066	`open red`	8930	`open buoy`	136	7
✓	9152	`get it`	9009	`get emeral`	143	20
	9242	`n`	9079	`n`	163	4
	9335	`get`	9168	`get`	167	1
	9408	`ne`	9240	`ne`	168	2
	9477	`dig sand with shovel`	9307	`dig sand with shovel`	170	0
	9619	`g`	9449	`g`	170	−11
	9701	`g`	9542	`g`	159	−12
	9752	`g`	9605	`g`	147	2
	9803	`sw.s`	9654	`sw.s`	149	18
✓	9914	`get sharp`	9747	`get sceptr`	167	−4
	10002	`s`	9839	`s`	163	14
✓	10070	`wave sharp`	9893	`wave sceptr`	177	−24
	10151	`W.w.sw.u.u.nw.w.w`	9998	`W.w.sw.u.u.nw.w.w`	153	12
	10644	`open bag`	10479	`open bag`	165	11
	10759	`Get clove`	10583	`gEt clove`	176	−26
	10864	`w`	10714	`w`	150	2
	10930	`open case`	10778	`open case`	152	−30
✓	11002	`put all in it`	10880	`put all in case`	122	21
✓	11272	`get clove,lamp,screw,torch`	11129	`get clove,lamp,driver,torch`	143	21
✓	11509	`opEn trap`	11345	`OpEn cover`	164	−4
	11607	`w.w.u`	11447	`W.w.u`	160	5
	12013	`temple`	11848	`temple`	165	−6
	12086	`s.d.n`	11927	`s.d.n`	159	20
✓	12212	`pat enormo`	12033	`rub enormo`	179	−45
	12300	`n.w.n.w.n`	12166	`n.w.n.w.n`	134	33
	12665	`get`	12498	`get`	167	−23
	12754	`e`	12610	`e`	144	3
	12848	`put torch,tool in cage`	12701	`put torch,tool in cage`	147	46
	13079	`eat clove`	12886	`EAT CLOve`	193	20
	13214	`w`	13001	`W`	213	0
	13339	`d.s`	13126	`d.s`	213	2
	13435	`get`	13220	`get`	215	15
	13507	`n.u.u.n.e.s.n`	13277	`n.u.u.n.e.s.n`	230	6
	13799	`get`	13563	`get`	236	−7
	13864	`u.s`	13635	`u.s`	229	9
	14010	`put coal in cage`	13772	`put coal in cage`	238	−1
	14140	`Lower cage`	13903	`Lower cage`	237	−6
	14222	`W`	13991	`W`	231	−12
	14314	`drop all`	14095	`drop all`	219	−3
	14405	`w`	14189	`w`	216	9
	14454	`w`	14229	`w`	225	12
	14560	`get all from cage`	14323	`get all from cage`	237	14
	14687	`s`	14436	`s`	251	5
	14779	`open lid`	14523	`open lid`	256	1
✓	14848	`put coal in it`	14591	`put coal in lid`	257	41
✓	14990	`close it`	14692	`close lid`	298	33
	15090	`set switch`	14759	`set switch`	331	−9
	15190	`open lid`	14868	`open lid`	322	9
	15268	`Get`	14937	`get`	331	1
	15336	`n`	15004	`n`	332	2
	15399	`put all in cage`	15065	`put all in cage`	334	1
	15515	`e`	15180	`e`	335	2
	15563	`e`	15226	`e`	337	0
	15626	`get`	15289	`get`	337	1
	15678	`u.U.n.e.s.n.u.s`	15340	`u.u.n.e.s.n.u.s`	338	11
	15982	`lift cage`	15633	`lift cage`	349	1
✓	16047	`Get huge,torch`	15697	`get diamon,torch`	350	−5
	16150	`w`	15805	`W`	345	24
	16269	`s.d.s.e`	15900	`s.d.s.e`	369	−9
	16473	`get`	16113	`get`	360	27
	16555	`w.n.u`	16168	`w.n.u`	387	−13
	16727	`pUt all treasu in case`	16353	`put all treasu in case`	374	−16
	16875	`w.w`	16517	`W.W`	358	−14
	16975	`Get`	16631	`get`	344	−16
	17050	`U`	16722	`u`	328	9
	17284	`hIt mAn`	16947	`HIt MAn`	337	−33
	17493	`g`	17189	`g`	304	−10
	17569	`G`	17275	`g`	294	0
	17761	`drop sword`	17467	`drop sword`	294	10
✓	17922	`get canary,pot,solid`	17618	`get canary,pot,casket`	304	−24
	18119	`temple`	17839	`temple`	280	8
	18179	`s`	17891	`s`	288	0
	18220	`pray`	17932	`Pray`	288	1
	18302	`E`	18013	`e`	289	0
✓	18333	`wind golden`	18044	`wind canary`	289	−56
	18410	`get`	18177	`get`	233	16
	18479	`s.e.w.w`	18230	`s.e.w.w`	249	−15
	18747	`put all treasu in case`	18513	`put all treasu in case`	234	7
	18953	`w.w.u`	18712	`w.w.u`	241	−19
	19174	`get all treasu`	18952	`get all treasu`	222	−44
	19402	`d.e.e`	19224	`d.e.e`	178	13
	19583	`put all treasu in case`	19392	`put all treasu in case`	191	4
	19817	`w.w.u`	19622	`w.w.u`	195	1
	19954	`get all treasu`	19758	`get all treasu`	196	14
	20066	`d.e.e`	19856	`d.e.e`	210	−2
	20258	`put all in case`	20050	`put all in case`	208	31
	20481	`e.e.n.w.sw.w`	20242	`e.e.n.w.sw.w`	239

You notice that even during stretches where the commands do not change, the advantage of the SYNONYM run fluctuates; this is the virtual memory thing I mentioned. If we isolate just the rows where the command changed, we see that, for the most part, the per-command advantage is positive. Not always, though: if there's a pattern, it's that changing sharp to sceptr did not show an advantage. (Keeping in mind the caveat that even identical commands had positive or negative advantage, so any effect may be illusory.) It is pretty consistently observable, however, that changing it to a specific synonym is beneficial.

Chg?	Before frames	Before command	After frames	After command	Cumulative advantage	Per-command advantage
✓	1454	`tug it`	1454	`tug rug`	0	21
✓	1554	`open it`	1533	`open cover`	21	20
✓	1927	`HIT It`	1866	`HIT troll`	61	9
✓	2487	`get old`	2348	`get bag`	139	14
✓	3727	`open it`	3536	`open casket`	191	38
✓	3854	`get sharp`	3625	`get sceptr`	229	0
✓	4103	`DrOp solid`	3854	`DRop casket`	249	7
✓	8184	`pump it up`	8120	`pump up boat`	64	37
✓	8354	`put sharp in it`	8253	`put sceptr in it`	101	−2
✓	8760	`get red`	8680	`get buoy`	80	46
✓	9066	`open red`	8930	`open buoy`	136	7
✓	9152	`get it`	9009	`get emeral`	143	20
✓	9914	`get sharp`	9747	`get sceptr`	167	−4
✓	10070	`wave sharp`	9893	`wave sceptr`	177	−24
✓	11002	`put all in it`	10880	`put all in case`	122	21
✓	11272	`get clove,lamp,screw,torch`	11129	`get clove,lamp,driver,torch`	143	21
✓	11509	`opEn trap`	11345	`OpEn cover`	164	−4
✓	12212	`pat enormo`	12033	`rub enormo`	179	−45
✓	14848	`put coal in it`	14591	`put coal in lid`	257	41
✓	14990	`close it`	14692	`close lid`	298	33
✓	16047	`Get huge,torch`	15697	`get diamon,torch`	350	−5
✓	17922	`get canary,pot,solid`	17618	`get canary,pot,casket`	304	−24
✓	18333	`wind golden`	18044	`wind canary`	289	−56

I did another, similar experiment with using ADJECTIVEs. The results were similar, only very slightly slower overall than the SYNONYM experiment, but not enough to show a conclusive difference.

Posted: 8/25/2026 4:11 PM

Post subject: An experiment with seeded runs to check for unseen random events

Sand

He/Him

Editor, Experienced Forum User, Published Author, Player (168)

Joined: 6/26/2018
Posts: 239

Sand wrote:

After a few easy wins, I encountered an interesting situation: a faster way to do one segment that ends up being slower overall by the end of the run. After investigation, I suspect that the cause is the game's virtual memory system, which swaps content into memory from the floppy disk.

I had a hypothesis: the reason why apparently inconsequential changes like this one result in differences in frame count has to do with an altered RNG state causing different random events to happen, invisibly offscreen. I did an experiment of executing the exact same run several times, "seeding" each one with a different no-op command to tweak the RNG. Then I recorded a trace of ZPC, the program counter in the Z-machine virtual machine, in order to see if anything different happened across the runs. The outcome was negative: differently seeded runs did not have differences in their ZPC traces (except for some minor explainable differences). However, the runs did show differences in their virtual memory swap traces. So as I suspected in Post #538861, virtual memory makes the difference, though I still don't understand why the same sequence of commands should sometimes cause virtual memory pages to be swapped in a different order. The code and data of the experiment are at https://repo.or.cz/zork1-appleii-tas.git/tree/56937043aa483435a6e7ad6021e3ee97f2a0dd48:/experiments/rng-tweak. Notes on the results are in notes.txt. I started with the 20685-frame run of Post #544754. I ran it again four times, each time starting the run with a different four-digit number command (which is a no-op in the game logic, but affects the RNG and virtual memory page table):

>1248
I don't know the word "1248".

>4567
I don't know the word "4567".

>0909
There was no verb in that sentence!

>5280
I don't know the word "5280".

It was an accident that the 0909 seed caused the interpreter to follow a different code path ("There was no verb in that sentence!"), but it turns out to be a useful point of comparison because it affects the page table differently. The runs finish with different frame counts. The difference between the slowest and the fastest is 197 frames, more than 3 seconds:

Seed	Frames
original	20685
`1248`	20488
`4567`	20522
`0909`	20559
`5280`	20488

There is an easily explainable difference in ZPC traces across the runs: more or fewer iterations through the REMARK routine (disassembly) that prints combat results. This is from the final thief battle. Though I constrain the result of each round of combat, I left the random remark that is printed for each result unconstrained, so as to require less RNG brute forcing. So in one seed the remark for a SERIOUS-WOUND result might be "A savage blow on the thigh! The thief is stunned but can still fight!" which has 3 elements because it refers to F-DEF (i.e., the word "thief"); while in another it might be "Slash! Your blow lands! That one hit an artery, it could be serious!" which has just 1 element. Other than the REMARK thing, though, and the minor differences in the first "seed" command, there's no difference in ZPC traces across the runs. So my guess that there might be some offscreen random events happening was not confirmed, at least for this set of seeds. Virtual memory has an interesting story, though. First, there are differences in the total number of page swaps:

Seed	Number of swaps
original	563
`1248`	554
`4567`	553
`0909`	584
`5280`	554

I compared traces of what virtual memory page gets swapped into what page table slot at what time. 3 of the 4 seeded runs (1248, 4567, 5280) are almost the same in this regard. 1248 and 5280 have the exact same sequence of swaps, in fact (though sometimes the swaps occur 1 frame earlier or later). 4567 is similar to those two, but it seems to sometimes swap the same pages into the same set of slots, in a different order. It has 1 fewer swap overall; despite that, it's not the fastest. The original, non-seeded run, and seed 0909 that results in a different response, diverge from these other three. They show behavior like Post #538861: the page table follows a different trajectory, and the run sometimes gains frames, sometimes loses frames. Starting with a different set of loaded pages at the first command has an effect that echoes until the end of the run. Inserting a useless command at the beginning of the run can actually result in reaching the end faster. Conclusion: random variations between two very similar runs are attributable to differences in virtual memory swapping behavior, which is hard to control. But more swaps is not always slower. At least I've ruled out the possibility that the differences are due to unseen random in-game events.

Posted: 8/23/2026 5:07 PM

Sand

He/Him

Editor, Experienced Forum User, Published Author, Player (168)

Joined: 6/26/2018
Posts: 239

Wow! The movement is amazing.

Posted: 8/23/2026 4:11 AM
(Edited: 8/23/2026 6:28 AM)

Post subject: A route that uses the thief to carry items

Sand

He/Him

Editor, Experienced Forum User, Published Author, Player (168)

Joined: 6/26/2018
Posts: 239

Link to video

I've prepared a draft of a new, faster route. It's about 18% faster than the baseline route of Post #538557 and 9% faster than the baseline+tweaks route of Post #538861.

Route	Moves	Frames	Time
baseline	258	25139	6:58.98
baseline+tweaks	257	22848	6:20.80
this	236	20685	5:44.75

The baseline runs used the Gym Slow route which is designed for RTA play. That route is built around the "boat of holding" glitch whereby you store items in the deflated boat (where they are weightless) to work around inventory weight limits. (The player character can carry only 95 units of weight.) This new route is built around a different trick: getting the thief to carry treasure for us. The thief is an NPC that wanders the dungeon along a set route and causes mischief such as engaging the player in combat or stealing items. Unlike the player, the thief has unlimited carrying capacity. He has a high chance of picking up treasure he sees, and with RNG manipulation we can make it 100%. He drops treasure off periodically at his lair, the TREASURE-ROOM, which is conveniently located just 3 moves away from the LIVING-ROOM and the trophy case where you need to eventually deposit the treasures. He'll also instantly warp to the TREASURE-ROOM when the player is there. The main constraint on the thief's ability to help with treasure collection is that he does not steal from rooms the player has not yet been to. A main theme of this route is efficiently "visiting" rooms that contain treasure, in order to make the treasure eligible for taking by the thief. (There are, however, several special cases that require special handling, such as a few rooms and items that are marked "sacred" and are off-limits to the thief.) The only treasure that must be held by the thief at some point is the EGG, because only the thief can open it to expose another treasure (the CANARY) that is inside. This is a schematic summary of the route. Here's a map to follow along.

Get the EGG, SWORD, and LAMP, defeat the troll, then go through the maze, getting the BAG-OF-COINS on the way. The cyclops is located at the entrance to the thief's TREASURE-ROOM. Scare away the cyclops, which opens a quick path back to the LIVING-ROOM. Drop the EGG and BAG-OF-COINS for the thief to pick up later; also drop the SWORD to free up 30 units of inventory space. (We won't need the sword again until the final battle with the thief.) Enter the TREASURE-ROOM and use a magic word to warp to NORTH-TEMPLE.
Get the COFFIN (which is a "sacred" item the thief cannot move for us), use a magic word to drop it in the TREASURE-ROOM, then warp right back. Get the TORCH and SCEPTRE, which are not only treasures but also tools we will need later. Take the BELL, BOOK, and CANDLES to the ENTRANCE-TO-HADES (manipulating RNG to keep the CANDLES from blowing out) and do a ritual to open the gates. Visit the crystal SKULL past the gates, leaving it for the thief to get. Then go to the dam area, passing through the EW-PASSAGE on the way to earn 5 points.
In the dam MAINTENANCE-ROOM, get the SCREWDRIVER and the WRENCH and use the WRENCH to drain the RESERVOIR and make it passable. Visit the TRUNK and TRIDENT past the reservoir, solve the LOUD-ROOM puzzle to make the BAR eligible for taking by the thief, then take the INFLATABLE-BOAT and PUMP to WHITE-CLIFFS-SOUTH, adjacent to the river.
Inflate the boat and use it to cross the river. On the river there is a BUOY that contains an EMERALD. The thief cannot enter water rooms, so we must collect the EMERALD, not just visit it. Take the SHOVEL and dig up the SCARAB, leaving it for the thief. Go to ARAGAIN-FALLS and wave the SCEPTRE to make a rainbow bridge. Take the bridge down to the base of the waterfall, visiting the POT-OF-GOLD, and return to the house through the overworld. Drop off the junk and treasure we're carrying, except for the LAMP, SCREWDRIVER, TORCH, and GARLIC, which we'll need in the coal mine.
Take the TREASURE-ROOM warp to NORTH-TEMPLE again, then warp from MIRROR-ROOM-2 to MIRROR-ROOM-1. Holding the GARLIC incapacitates the bat, which lets us get the JADE. (The BAT-ROOM is a sacred room, so we have to pick this one up.) Put the SCREWDRIVER and TORCH in the dumbwaiter. Dispose of the GARLIC in order to take advantage of the bat's attack. When the GARLIC is not present, the bat seizes you and takes you to a random room in the vicinity of the coal mine. By manipulating RNG, we force the bat to drop us at the far end of the mine, saving travel time. Take the COAL, walk out through the mine, place the COAL in the dumbwaiter, and lower the dumbwaiter. Use the bat to warp over the mine again, drop everything to get through the narrow passage, collect the items from the dumbwaiter, and convert the COAL into the DIAMOND using the machine. Raise the dumbwaiter, walk back out through the mine, recollect the TORCH and DIAMOND from the dumbwaiter, and take the slide back to the CELLAR below the LIVING-ROOM.
Take a short detour to the GALLERY to get the PAINTING. The thief could get this one, but we're now too close to the end of the run for him to have time to get there. Go to the LIVING-ROOM to drop off the treasure we're carrying. Go through the cyclops hole, pick up the SWORD, and enter the thief's lair.
Entering the TREASURE-ROOM causes the thief to warp there, along with the treasure he's carrying. We manipulate an ideal sequence of random combat outcomes: 3 MISSEDs from the thief, and 2 SERIOUS-WOUNDs and a KILLED from the player. There is now a pile of treasure here we have to carry to the nearby LIVING-ROOM. We start with the CANARY (plus as much else as we can carry): we must take it outside to the forest for one final piece of treasure. Use a magic word to warp to NORTH-TEMPLE, then another to warp from SOUTH-TEMPLE to FOREST-1. Wind the CANARY to make the songbird drop the BAUBLE. Re-enter the house and drop off the treasure. Make two round trips through the cyclops hole to fetch the rest of the treasure.

As before, the route is scripted with a Fennel program. You can see the commented code with individual commands here. The route is fun—the manipulation of the thief and the bat makes it different from RTA. I think there's potential to bring the time still lower. I've marked some potential changes with TODO comments in the source code. These are some ideas I have:

Save the COFFIN for last, rather than doing the awkward back-and-forth warp as soon as we get it. Collect the COFFIN on the final go-round with the CANARY, because we have to pass through that area anyway.
Maybe also leave the TORCH for last, or just visit it for the thief to pick up. The CANDLES work equally well as a light source in the dumbwaiter puzzle in the coal mine.
Consider collecting treasure as long as there's inventory space. I haven't audited which segments of the route have excess inventory capacity that might be used for carrying treasure. It'll be worth it if we can save one round trip during the final treasure haul.
Dropping the LAMP for the final treasure haul will leave room for slightly more treasure per trip. There are 2 dark rooms between the TREASURE-ROOM and the LIVING-ROOM, but entering the first dark room is safe, and the second one can be RNG-manipulated to prevent a grue attack.
I want to try a reordering where you do the coal mine before the river area. The slide after the coal mine returns you basically to the house, which is relatively close to the bottom of the waterfall. Waving the SCEPTRE from the bottom works to get up to the SCARAB and EMERALD, then return to the house through the overworld.

Posted: 8/22/2026 4:49 AM

Post subject: Caching / memoization for functions that operate on savestate objects

Sand

He/Him

Editor, Experienced Forum User, Published Author, Player (168)

Joined: 6/26/2018
Posts: 239

When you write programs in the data-oriented style, where savestates are represented by variables, you don't think in terms of "controlling the live emulator state" but rather "computing functions that take savestates as input and return savestates as output". In the Zork I TAS I'm working on, there's a basic function called enter-commands-with-checks that takes as input a savestate, a command string, and a set of "must-not" event registration functions; and returns either a new savestate or nil. Starting from a given savestate, enter-commands-with-checks types in the command, then waits until the game is ready to accept input again. As it waits, it watches to see if any of the "must-not" events occurs. If any occurs, it returns nil; otherwise it returns a new savestate at the point where the game is ready to accept the next command.

Language: fennel
;; Enter a single command and a newline, then wait for input ready with the
;; given must-not event set (as in wait-for-input-ready-with-checks).
(lambda enter-command-with-checks [st command must-not]
  (-> st
      (enter-command command)
      (savestate.next-frame)
      ;; The event checks run while we wait for input ready.
      (wait-for-input-ready-with-checks must-not)))

An example of how it's used is when we walk north into the Troll Room for the first time. The troll has about a ⅓ chance of attacking as soon as we enter the room, which costs time for no benefit. So what we do is pass "n" as the command and a function to check for the game's VILLAIN-BLOW routine is called as the must-not set:

Language: fennel
(enter-command-with-checks st "n" [CHECKS.VILLAIN-BLOW])

If the check fails (the troll attacks), the function returns nil, which signals the caller to go back and try altering the letter case of earlier commands to manipulate RNG before trying again. The Zork I TAS is scripted from start to finish. If I want to test a change to a command in the middle of the run, I can edit the command and run the script from the beginning. But this started to get time-consuming, as the script re-emulated all the other commands up to that point (and in particular re-rederived all the necessary RNG manipulation). To speed up the iteration cycle, I wrote a cache module to memoize the computation of a function like enter-command-with-checks by storing the return value of the function in a SQLite database. It works like this:

Language: fennel
(local cache (require :cache))
(local memoize (cache.open "zork-cache.db"))
(memoize enter-command-with-checks st command must-not)

The memoize function wraps a call to a given function. When called, it first computes a cryptographic hash of the function and its arguments. (In this case, Hash(enter-command-with-checks ∥ st ∥ command ∥ must-not).) It looks up the hash value in the database. If it's found, it returns the value cached in the database, without actually running the given function. If it's not found, it computes the function on the given arguments, stores the return value in the database, and passes the return value up to the caller. Now when I edit a command in the middle of the run, the script can quickly catch up to that point by looking up the results of executing commands in the cache, rather than by running the emulator, which is less efficient. Only after the returned savestates start to diverge from what has been seen already does the emulator machinery get involved again. I had to add a bunch of other support code to make the cache module possible. BizHawk has a built-in SQL module for interacting with an SQLite database, but its interface is not so nice to use, so I wrote an sqlite module that is described in Post #544597. There's a sha3 module for hashing; this is something you can get from third-party packages, but I was also curious about SHA-3 and wanted the experience of implementing it. The biggest new requirement for making the cache module work with savestate objects is being able to get the contents of a savestate file into memory. The savestate module internally represents savestates as a path to a file, but we want to store in the database not just a path, but the actual contents of the file. (As the files backing savestate objects are deleted when they are garbage collected or when BizHawk exits.) To get the contents of a savestate file into memory, I added the function savestate.to-bizhawk-format which calls BizHawk's own savestate.save on a temporary file and slurps the contents of the file; as well as savestate.new-from-bizhawk-format which writes a blob of bytes to a temporary file, then calls BizHawk's savestate.load on it. There's a little more work to do if you want to use the contents of a savestate file as a database key. That is because two different savestate files made from the exact same emulator state are not identical, in general. The reason is that a savestate file is a zip file, and as such contains timestamps that reflect the time of creation. I had to implement just enough of a zip parser to peek inside the zip file and hash just the file contents, not the zip metadata. The serialization of functions (for hashing or storing in the database) uses Lua's string.dump. This works okay, but it could be better. The main problem is that the string.dump serialization includes the line numbers at which the function is defined. That means that adding or removing lines before the function effectively invalidates the memoization cache for that function, even if the function itself is unchanged. That is annoying, but tolerable when the source code is reasonably stable. It would reasonably be possible to parse the string.dump format to overwrite the line numbers, but I judged the additional complexity not to be worth it; it would also tie the cache code to a particular version of Lua.

Posted: 8/13/2026 2:54 AM

Post subject: A nicer interface to SQLite from BizHawk

Sand

He/Him

Editor, Experienced Forum User, Published Author, Player (168)

Joined: 6/26/2018
Posts: 239

User file #639221830998247534 is a Lua module for BizHawk that wraps the built-in SQL module to make it easier to use. In particular, it makes the structure of result sets more natural, it converts various conditions and errors that the SQL module signals with magic strings into regular Lua values and errors, and it makes it possible to open multiple database files (even from multiple scripts) at the same time.

Language: lua
local sqlite = require("sqlite")
local db = sqlite.open("lag.db")
local lag_frames = {}
for _, row in ipairs(assert(db:execute("SELECT frame, islagged FROM lag"))) do
	if row.islagged ~= 0 then
		lag_frames[row.frame] = true
	end
end

If you're like me, you were excited when you saw that BizHawk has a built-in built-in SQL module, because easy access to SQLite is very handy. But then you were disappointed when you tried to use it, because the interface is awkward and confusing. Here are some things that make it hard to use:

The SQL.opendatabase function doesn't return anything like a database handle. The SQL module internally supports only one database connection at a time, stored in a singleton variable called _dbConnection. When you call SQL.opendatabase, it forgets the previous database connection and replaces it with a new one. Calls to SQL.readcommand and SQL.writecommand use whatever database was most recently opened—and the effect is global, across all running scripts. If the script A.lua opens the database A.db, then script B.lua opens the database B.db, all of a sudden A.lua will unexpectedly interacting with B.db, most likely resulting in errors. Consider these two scripts, for example: Download A.lua

Language: lua
SQL.opendatabase("A.db")
function assert_writecommand_ok(res) assert(res == "Command ran successfully", res) end
assert_writecommand_ok(SQL.writecommand(
[[CREATE TABLE IF NOT EXISTS lag (
	frame INTEGER NOT NULL,
	islagged INTEGER NOT NULL
) STRICT]]))
event.onframeend(function ()
	assert_writecommand_ok(SQL.writecommand(string.format(
		"INSERT INTO lag VALUES (%d, %d)",
		emu.framecount(),
		({[false] = 0, [true] = 1})[emu.islagged()]
	)))
end)

Download B.lua

Language: lua
SQL.opendatabase("B.db")
function assert_writecommand_ok(res) assert(res == "Command ran successfully", res) end
assert_writecommand_ok(SQL.writecommand(
[[CREATE TABLE IF NOT EXISTS inputpoll (
	cycles INTEGER NOT NULL
) STRICT]]))
event.oninputpoll(function ()
	assert_writecommand_ok(SQL.writecommand(string.format(
		"INSERT INTO inputpoll VALUES (%d)",
		emu.totalexecutedcycles()
	)))
end)

If you load these two scripts at the same time, one of them will crash with an error, depending on which was loaded first:

error running function attached by the event OnInputPoll
Error message: [string "main"]:2: SQLite Error 1: 'no such table: inputpoll'.

NLua.Exceptions.LuaScriptException: [string "main"]:2: SQLite Error 1: 'no such table: lag'.

Most of the functions in the SQL module return magic strings to indicate success or error conditions, like:
Proper use of the module requires checking for these strings and interpreting them appropriately.
The format in which results are returned is strange. Suppose you had this table:
```
CREATE TABLE tbl (x INTEGER, y STRING);
INSERT INTO tbl VALUES(123,'abc');
INSERT INTO tbl VALUES(999,'hello');
INSERT INTO tbl VALUES(456,NULL);
INSERT INTO tbl VALUES(NULL,'xyz');
```
x y
123 'abc'
999 'hello'
456 NULL
NULL 'xyz'
If you run the query SELECT x, y FROM tbl, you might reasonably expect to get a Lua table like this, with one entry for each row, and each row being represented by a key–value table:
```
{{x = 123, y = "abc"},
 {x = 999, y = "hello"},
 {x = 456},
 {y = "xyz"}}
```
Instead, results are returned in single flat key–value table, where keys encode both a column name and a row number. NULL values are not represented by a Lua nil, but by a particular kind of userdata:
```
{["x 0"] = 123, ["y 0"] = "abc",
 ["x 1"] = 999, ["y 1"] = "hello",
 ["x 2"] = 456, ["y 2"] = userdata,
 ["x 3"] = userdata, ["y 3"] = "xyz"}
```

`x`	`y`
`123`	`'abc'`
`999`	`'hello'`
`456`	`NULL`
`NULL`	`'xyz'`

The wrapper module provides the following advantages and conveniences over the built-in SQL module:

It simulates having handles to multiple databases open simultaneously. Calling sqlite.open gives you a handle that you can then use to execute statements on that database and no other. How this works is pretty simple. A database handle contains a filesystem path. Before executing a statement, the module re-opens the database at the given path so that the SQL module's global database connection points to the right place. That means you can run these two scripts at the same time, and they do not interfere with each other: Download A2.lua
```
Language: lua
local sqlite = require("sqlite")
local db = sqlite.open("A2.db")
assert(db:execute(
[[CREATE TABLE IF NOT EXISTS lag (
	frame INTEGER NOT NULL,
	islagged INTEGER NOT NULL
) STRICT]]))
event.onframeend(function ()
	assert(db:execute(string.format(
		"INSERT INTO lag VALUES (%s, %s)",
		sqlite.escape(emu.framecount(), emu.islagged())
	)))
end)
```
Download B2.lua
```
Language: lua
local sqlite = require("sqlite")
local db = sqlite.open("B2.db")
assert(db:execute(
[[CREATE TABLE IF NOT EXISTS inputpoll (
	cycles INTEGER NOT NULL
) STRICT]]))
event.oninputpoll(function ()
	assert(db:execute(string.format(
		"INSERT INTO inputpoll VALUES (%s)",
		sqlite.escape(emu.totalexecutedcycles())
	)))
end)
```
As an extra precaution, the module also restores the global database connection to what is was originally, after executing each statement. That means that scripts that use the wrapper module can run at the same time as up to one other script that uses the SQL module directly. (If two or more other scripts use the SQL module directly, they will still stomp on each other, and there's nothing we can do about that.)
It interprets the various magic strings that the SQL module returns and converts them to plain Lua values or error returns. A query that successfully returns 0 rows results in an empty table, not the string "No rows found".
It restructures result sets to use the more natural sequential row structure described above. You can iterate over the rows with ipairs and access the value of columns with normal table indexing.

Another way of working around the awkward interface of the SQL module is not to use it at all, and instead to install some third-party module such as LuaSQLite3. But this wrapper module can improve the experience of using SQLite from BizHawk without adding an external dependency.

Posted: 7/7/2026 8:21 PM

Sand

He/Him

Editor, Experienced Forum User, Published Author, Player (168)

Joined: 6/26/2018
Posts: 239

Great submission notes. It's interesting that the time for buying guests/house space between rounds, and opening the door for guests, is negligible, except for a few cases you noted (like the mermaid). Since RNG is not affected by timing, it might be technically possible to make a "relaxed" version of the TAS that buys guests and lets them in slowly, to better reveal the strategic choices. (Kind of like how some videos have alternative camhack encodes to show what's happening when the player character is way out of bounds.)

Posted: 6/12/2026 3:13 PM
(Edited: 6/12/2026 3:14 PM)

Sand

He/Him

Editor, Experienced Forum User, Published Author, Player (168)

Joined: 6/26/2018
Posts: 239

That's a "uses death to save time" run if there ever was one! The EXEC-0032 terminal code, I'm guessing that's just a faster way to get into the game than going through the usual game library menu?

Posted: 4/5/2026 5:18 AM

Sand

He/Him

Editor, Experienced Forum User, Published Author, Player (168)

Joined: 6/26/2018
Posts: 239

The strategy is more complicated than I would have guessed.

Posted: 3/17/2026 6:22 PM

Sand

He/Him

Editor, Experienced Forum User, Published Author, Player (168)

Joined: 6/26/2018
Posts: 239

Wow this is so cool :) I did not know about those Mortol glitches, the jump out of the parachute and the jump on spikes.

Posted: 3/7/2026 6:37 PM

Sand

He/Him

Editor, Experienced Forum User, Published Author, Player (168)

Joined: 6/26/2018
Posts: 239

The run looks like fun. faerie9987, can you say more about the game and the optimization? What is the goal in each level? What do the different NPCs do (security guards and secret agents)? It looks like you need to open certain doors to collect pages and then get to the bottom floor—how did you decide the route to visit the necessary doors? How do you decide when to take damage to save time? What do the bullet and grenade items do, and do you have to do anything to conserve items?

Posted: 3/7/2026 6:30 PM
(Edited: 3/7/2026 6:31 PM)

Sand

He/Him

Editor, Experienced Forum User, Published Author, Player (168)

Joined: 6/26/2018
Posts: 239

Walgrey wrote:

Apparently, the author submitted the same run twice, with the second one having the correct encode.

Yes, see Thread #27108: #10254: faerie9987's GBA Elevator Action: Old & New "New, Berry" in 37:57.594 for the replacement submission with the correct encode.

Posted: 3/6/2026 9:41 PM

Sand

He/Him

Editor, Experienced Forum User, Published Author, Player (168)

Joined: 6/26/2018
Posts: 239

It's possible to use third-party Lua modules (from, e.g., LuaRocks) from within the BizHawk Lua console. If you can find a module that plays sound in a non-blocking way, it might work for your purpose. (You may have to tinker with package.path / package.cpath, depending on where LuaRocks installs things.) I tried something with the Lua-SDL2 module (luarocks install lua-sdl2):

Language: lua
local SDL = require("SDL")
SDL.mixer = require("SDL.mixer")

assert(SDL.mixer.openAudio(48000, SDL.audioFormat.S16, 2, 2048))

local beep = assert(SDL.mixer.loadWAV("beep.wav"))
assert(beep:playChannel(-1, 0, 0))

However, it doesn't quite work, because BizHawk includes its own copy of libSDL2, which shadows the system libSDL2, and which isn't compiled with support for audio. It gives the error "SDL not built with audio support" in the Lua console. If you can find a module that doesn't conflict with one of BizHawk's included libraries, or if you can find a way to make the Lua script use the system libSDL2 rather than the BizHawk one, it may work. I tried the ao module and it does work, in the sense that it plays audio from within the BizHawk Lua console, but it has the same problem as your os.execute idea, namely that it blocks until the sound is finished playing.

Posted: 2/20/2026 5:44 PM
(Edited: 2/20/2026 5:44 PM)

Sand

He/Him

Editor, Experienced Forum User, Published Author, Player (168)

Joined: 6/26/2018
Posts: 239

synabler wrote:

Yesterday I experimented with a few games to see how RNG behaves, since CoffeeTools has a feature to display RNG information. Unfortunately, Party House RNG seems harder to manipulate than I had hoped for. Each day starts with the Rolodex pre-shuffled, and inviting, peeking, etc. simply extracts the next guest from the list. A saving grace though is Magician: trying to use his ability advances the RNG, even if it fails due to having no star guests. So I imagine scenario 5 would be easier to manipulate luck compared to other standard scenarios.

I see, how interesting. Still, even if the rolodex order is fixed, it should be possible to optimize any given day for cash and/or pop. Being able to see the order in advance, you would never draw one too many guests and overrun the trouble budget, slash you wouldn't have to spend resources on trouble mitigation when you know you won't need it. Conceivably, it could make sense in some cases to buy "chaff" guests, like old friend, purely for the effect they would have on shuffle (if you need a particular guest on the first turn of the next day, say). Since the game is so discrete, it's the kind of thing that might be amenable to offline optimization, reimplementing the RNG and shuffle algorithm and using computer search to find good seeds or good strategies per seed. I'm guessing that the RNG seed is somehow randomized at the start of a new game, even if is not changeable within or between rounds (excepting the magician).

Posted: 2/18/2026 6:46 PM

Sand

He/Him

Editor, Experienced Forum User, Published Author, Player (168)

Joined: 6/26/2018
Posts: 239

Wow! I'm excited about the possibility of UFO 50 TASing. I'm curious what RNG manipulation could accomplish in games like Planet Zoldath and Party House. Other games that come do mind that could have interesting TAS strategies are Onion Delivery, Rail Heist, Porgy, Overbold, and Mortol II.

ikuyo wrote:

1) To make a CoffeeTools TAS file readable to our site, we would need to have a parser file for it. The parser file would have to be able to count frames and framerate, and gives us the total length of the movie in seconds and miliseconds. This, once it gets deployed on our end, would be enough to allow all .CTAS files to be uploaded to The whole site is open source, and if you or anyone else in the community wants to make said parser, you are more than welcome!

Here are some past pull requests that add new parsers, to give you an idea of what's involved:

Posted: 2/6/2026 7:32 PM

Sand

He/Him

Editor, Experienced Forum User, Published Author, Player (168)

Joined: 6/26/2018
Posts: 239

feos wrote:

_{Would you like editor privs?}

Yes, if I can edit the page I know what to do with it.

Posted: 2/5/2026 8:46 PM

Sand

He/Him

Editor, Experienced Forum User, Published Author, Player (168)

Joined: 6/26/2018
Posts: 239

The TASVideos forum uses a markup format based on BBCode. Pandoc is a program that can convert between many different markup formats. This is a custom writer for Pandoc that lets you convert from any of Pandoc’s input formats to TASVideos forum markup.

Usage Install Pandoc and download tasvideos_forum.lua. Run pandoc with the -f option ("from") set to the format you are converting from, and the -t option ("to") set to the path to tasvideos_forum.lua. See the documentation on custom readers and writers. Say your input is in Markdown format in the file input.md. Then you would run:

pandoc -f markdown -t tasvideos_forum.lua input.md

If the input format is obvious from the filename extension, you can omit the -f option:

pandoc -t tasvideos_forum.lua input.md
pandoc -t tasvideos_forum.lua input.html
pandoc -t tasvideos_forum.lua input.docx

To save the output to a file, use the -o option:

pandoc -t tasvideos_forum.lua -o output.bbcode input.md

Example The custom writer can be useful when you have some input whose representation as BBCode is difficult (such as complicated tables), if you already have text written in some other format, or if you’re just more comfortable to writing in another format such as HTML or Markdown. Take this Markdown input:

![Small Mario](https://tasvideos.org/favicon.ico)

- *Lots* of lag reduction
- New shortcut

|World      |Frames saved|
|-----------|-----------:|
|Grass Land |          14|
|Desert Land|           2|

The command pandoc -t tasvideos_forum.lua input.md converts it to:

[img]https://tasvideos.org/favicon.ico[/img]
[b]Small Mario[/b]

[list]
[*][i]Lots[/i] of lag reduction
[*]New shortcut
[/list]
[table]
[tr]
[th]World[/th]
[th][right]Frames saved[/right][/th]
[/tr]
[tr]
[td]Grass Land[/td]
[td][right]14[/right][/td]
[/tr]
[tr]
[td]Desert Land[/td]
[td][right]2[/right][/td]
[/tr]
[/table]

Which renders on the forum like this:

Small Mario

Lots of lag reduction
New shortcut

World	Frames saved
Grass Land	14
Desert Land	2

Accessing TASVideos-specific markup Pandoc conversion doesn’t try to preserve anything about the input. You will get structural and semantic features like paragraphs, lists, tables, code blocks, links, bold, and italics, but you won’t get things like font sizes and colors. There are, however, ways to access TASVideos-specific markup, usually by setting classes or attributes. You can do this using either Markdown syntax (using Pandoc’s special extended Markdown):

::: {.spoiler}
Block attributes
:::

Inline [attributes]{.spoiler}

Or HTML syntax:

<div class="spoiler">
Block attributes
</div>

Inline <span class="spoiler">attributes</span>

For the [spoiler] tag, set the spoiler class. For the [highlight] tag, set the mark class. You can alternatively use the <mark> element in HTML input. For [note] and [warning], you can use the alerts syntax. Other tags, like [wiki], [post], and [movie], can be accessed with raw attributes and a format of tasvideos_forum:

`[wiki]ForumMarkup[/wiki]`{=tasvideos_forum}

This is `[frames]100[/frames]`{=tasvideos_forum} faster than `[movie]1234[/movie]`{=tasvideos_forum}

See the README for more features.

Posted: 2/5/2026 8:00 PM

Post subject: [wip] on ForumMarkup page should be [userfile]

Sand

He/Him

Editor, Experienced Forum User, Published Author, Player (168)

Joined: 6/26/2018
Posts: 239

Wiki: ForumMarkup?revision=28 mentions a [wip] tag, but there is no [wip]. It's actually called [userfile]. Wiki: ForumMarkup?revision=28#BrokenTags says [list=] and [hr] are broken, but they work fine as far as I can tell.

Posted: 2/5/2026 3:40 PM

Sand

He/Him

Editor, Experienced Forum User, Published Author, Player (168)

Joined: 6/26/2018
Posts: 239

gasuto4444 wrote:

However, I cannot download the original video from YouTube or Internet Archive. Can I directly record the footage from lsnes without including the first few seconds of description and information about TASVideos?

I see, it is because youtube.com and archive.org are blocked by the GFW. If you just need this one file, one option is that someone can download yoshisisland-tasv3-glitched-masterjun.mkv for you and put it in a place you can access. Recording the footage from lsnes means making your own encode. There is a guide here: Wiki: EncodingGuide/VideoDumping#Lsnes. In the existing encodes, the first few seconds of video with information about the side is called the encoder logo. For hosting on bilibili, I suppose it would make sense for the information to be written in Chinese. I don't know if there is a precedent for localizing the encoder logo, maybe someone else knows.

Posts for Sand