https://imgur.com/a/7Rxg7W9
Before we get into this, there are a couple things to preface. First, what I have drawn here is obviously very different from the interface now, and if implemented, it would be a major milestone release- which is why I wrote the version number as “v2.0”. 1.5.0 might be a round enough number, but I’ve seen screenshots of 1.3.x versions that look pretty similar. I don’t even know what the first versions looked like, because I couldn’t get them to install properly or find any images of them, so if anyone has any image or has gotten them to work, please share, because I’m very curious.
That said, these changes are only suggestions, and most of them only have to do with the UI. In other words, it would largely just be a rearrangement/reskinning of the existing functionality. Since this is a drawing, it’s difficult to really arrange things exactly how I want them if I change my mind (especially on a white board), so I will have a few things to elaborate on. But I think it does give a general idea of how the interface would feel. And it is the main window that would change the most, so let’s start there.
- - - MAIN MENU - - -
The first thing you may notice is that any distinct games would be displayed as icons at the top of the window, rather than their details. The way this would work is that you can add a new game using the “+” icon, and click the wrench to open the game’s details. There, you will find the game title field, FPS, executable path and command options, as well as the settings panel, which you can change per-game. It could also be stretched downward to show more games at once. The game’s settings and backup input LTMs will be stored in either a dedicated folder within the default backup location, or another path of your choice, like if you already keep games and their movie files organized and want to choose that existing folder.
This way, each game would have its own LibTAS settings, including hotkeys, input, runtime, etc. Checkboxes would go between not just true (checked) and false (empty), but also agnostic (dashed), meaning to use the default settings. Drop-down settings would do something similar by showing you “Use default” as an extra option. By the way, this could also be saved as a separate config file in the folder used for a particular game, including things like FPS, and have you bundle that config file into the archive before publishing it. This way, literally all settings in libTAS will be stored independently of the movie, minimizing the amount of instructions needed to troubleshoot unique user settings. So for instance, if you have different hotkeys for one game than another because they both use different keys, it can automatically switch them out without you having to think about all of this when switching between games.
How it all works currently is that when you change the executable, it automatically adds a new entry to a dropdown menu, which fills out the command options when selected, for which there is another dropdown. While that does help, it is a bit hard to follow what it’s doing, and if you change the command options, you get more new entries you have to sift through to find the one you want to use for your game.
Most of all, if you use Ruffle, you have to change only the Flash file path, which is in the command options. So if you’ve tried removing/adding “-g gl”, “—no-gui”, etc it automatically adds more and more options for the same game, making other games even less likely to stay in the drop-down’s recent options.
Adding this game profiles feature will make it easier to distinguish and juggle more game projects quickly by explicitly telling LibTAS details about what each game is, thereby easier to switch between current projects. By the way, in the second image I kept the same main window drawing but only added a message to ask if you want to set the screenshot you just took as the thumbnail shown for the game.
Speaking of tooltips, one of the major changes to the main window is that most of the settings are not drawn in text, but icons instead. While that would make even more space, it would probably also not work for everyone. So first, it could be optional, as drawn in the Interface panel (img 2). It wouldn’t even have to be default- the layout changes alone would save a good deal of space. Second, tooltips could be displayed in a similar way to Dolphin- in a footer below all the settings the moment you mouse over it, immediately explaining them as you explore. The other thing about the main window is that some of the settings currently displayed don’t need to be shown, imho (e.g. movie file, authors). Especially while the game is running, there are things you would want to see in the main menu before starting, like system time, FPS, and the entire games section, but not at runtime. These would turn the window into essentially a mini control panel.
Some things I have added, though, are the pause frame (which I find myself using quite a lot when having to restart my game, to stop it going past specific moments- so it’s annoying to have it buried), a speed percentage field (which could also replace fast forwarding), and most crucially, save states. Currently these don't appear very readily, only in the already-secluded "Configure mapping..." window under Input in case you happen to change it. And the default hotkeys for saving didn’t even work for me except for state 1, so attempting to load them would just give me a “no state in slot” message.
As such, I added 10 quick access options; a custom index field, just in case you would ever want to save more than 10 at once like you can in Dolphin; and options to load and save, as well as options to load movie branches with them. The field could even actually take names instead of just numbers, with a drop-down to list states saved. However, it may not have to show up until runtime, despite this being in the drawing.
If save states are eventually able to persist between game launches, they should be stored per-game- but under the hood, like in the main window but shows save states only if selected.
There could also be an option to turn off previous input verification, with a warning that you need to be sure it’s fine, otherwise it might mess up the sync.
By the way, the warning that save states may only work in GL would show up if your commands options do not include the -g gl flag.
- - - SETTINGS - - -
Above your games, the settings options would display “Interface - Runtime - Input - Audio/Video - Debug - Specific - Hotkeys”. Unlike the ones in the game panel, these would control the default settings in libTAS globally.
Speaking of settings, another thing that seems weird is that there’s a menu bar, and like a few of the settings are in the 5 items there, but then it seems like some of those could go into the settings window where everything else is. And the Input drop-down has only a couple of options, one of which is “Configure mapping”, which opens up a whole window with all the hotkeys separate from all the other settings, almost as if hotkeys aren’t even settings.
So it seems like maybe they could be consolidated a little. They probably should still be shown with the game running though, because some of them can be modified at runtime- and they can be displayed like the menu bar is currently, rather than beside the games.
And on that note, there seems to be quite a bit of empty space in libTAS. I’m not trying to make it too claustrophobic and I’m sure in practice it would feel different than I could draw in the pictures, but I noticed several text fields are way longer than they need to be for the amount of digits you could ever need to enter in them. And as for the remaining empty space, the windows could adaptively stretch downward as you open different panels to accommodate the settings in them. Just a suggestion though.
By the way, the Audio, Video, Input, and Movie categories are fairly sparse, especially compared to Debug. So I’ve joined those four things into two. Also, the word “movie” seems like it could sound as if it refers to video, when really it refers to a sequence of inputs, whereas Encode is for video, and could at least be elaborated as such. I know that TASVideos and other sources seem to conventionally use “movie” to refer to input, and maybe that is one meaning of the word. But I personally feel that “sequence” might actually convey that in more clarity. Perhaps a nitpick though. On the same point, the Record and Play options could just be Read/Write Inputs, represented by a pencil and book.
- - - INTERFACE - - -
Speaking of windows, the Interface panel could also have options for whether the settings open as separate windows, or just show in the main window as tabs at the top. Again, it would be optional, and it would also not apply to the Input Editor, Ram Search, etc you would obviously want as separate windows, so you can place them anywhere you want. But it would include the game and movie details panels, and an option to hide the States in the main window. I also added Dark Theme, to be more consistent with Linux, and a Colors dropdown, which would namely affect the Input Editor and Hotkey Editor. The interface actually might not necessarily have to use different vision modes- other design principles like contrast, boldness and visual separation can also work. There are a number of guidelines to follow here. But it might actually be helpful for displaying games, in case your OS doesn’t have the option (Linux distributions can vary wildly, and I couldn’t actually see any vision filter option in Ubuntu anyway).
- - - HOTKEYS - - -
Which brings me to the next proposal- the Hotkeys window drawing is really more of a general gist, because the keys should really be bigger, and there’s no numpad- ideally it should display every possible key, and a listen option in case you can’t see one of your keys. But it would help you to visualize which keys are available- pressing a key will automatically select it, and you can either listen for or select a key (this will remap in the game, because again that’s with a key already selected), or select an action (press/release, hold with duration and interval settings) a list of modifier combinations to assign additional bindings to (warn against using modifier keys alone), and bindings for each. This list, by the way, could include any setting in LibTAS that can be changed at runtime. For example, Tab press would be assigned to set the max speed to -1% (no limit), and release to 100% out of the box. "V press" and "V hold (past: 1 second)" would both frame advance. This would of course behave exactly as it does now, but with controls to make it far more flexible- for example you could set a speed cap, or make holding V start playing the game more quickly.
As for the look of this editor, the keys would display as either a dark blue or light yellow outline that changes to white or black when selected, depending on dark theme. When a key is assigned to press, the lower middle part would be a faint blue, with a gap between the outline. Release would be in the top middle part. Hold would make the outline dotted. When a modifier key is assigned to any key, it will show up as a more prominent red, and the symbols for those modifiers would show up on the key. Also, there could be a shortcut to save the current movie, and something pre-mapped to toggle input read/write, as I have found myself using since I assigned a key to it.
- - - INPUT EDITOR - - -
Similarly, the light yellow in the Input Editor would change to dark blue, the shades of green would be darker, and crucially, the current frame row would be slightly stretched and separated from all the other frames, and show up as a prominent red when recording, and either dark purple or light blue in playback. But then what should columns in Auto Fire look like? I think Orange might work, besides not all columns would always be set to fire most of the time.
On that note though, if you click on the top of a column, you may notice the colors change (or not), but still not know what it means. So a tooltip warning could be shown to say explicitly that auto-hold or -fire was enabled or disabled.
There could also be a list of sequences you can either set as your main active project file, or as repeatables with an automatic play check box, and select in the bind actions drop-down in the hotkey editor. This would be useful for if you want to repeat something like a short jump that’s too specific for auto-fire. It could also have options to type in an underscore or set as override, so that specific inputs can still be recorded while others can play.
For an even quicker approach, the Hotkeys window could have an option to add shortcuts to toggle the recording of a quick repeatable pattern, and then another key to play it.
With all of this being said about the Input Editor though, I should give credit where it’s due- this way of editing gives you a good visual reference for how your input will play out, because it is essentially just a big spreadsheet representation of your entire sequence. Plus, I found out that you can edit any pre-current-frame inputs past a saved game state and libTAS will attempt to load and seek to that point for you, making it invaluable for doing precise tests. As far as I remember experimenting with Dolphin before, it has no such workflow at all. It does have state saving of course, and a TAS input window to edit your controller inputs, but the closest thing you possibly have is to open up your sequence file in a text editor… maybe? And even if this is possible, you wouldn’t get to see them change in real time.
But speaking of that playback feature though, I found in my current project that there are several points where if you load a state and your current frame is past a certain frame from it, it would fail to load- which is really annoying. To clarify, this is probably the game’s fault, and sometimes this would make it hard to test certain cases regardless. But in other cases,
one thing that could help sometimes is having a chain of states saved for you every so often (every 10 frames or something for 5 states, let's say), as configured in the Input section. That would prevent it from ever having to load a state that is way too far back, just because you haven’t saved a new one. The tool tip for that interval value could even clarify that the lower you set that to, it could make crashes less likely. And if you want to combine elements from different versions instead of repeating stuff, the editor could be multi-window, and allow you to append frames from other files, or recover data from before loading a state.
Another thing I noticed is that if I try and drag to select frames of a level, they’re all highlighted blue. So I can’t see where the seekhead frame is which ones are green or light yellow). And really, you don’t need to highlight the whole frames because to select them, you click on just the left part where the frames numbers are anyway.
Sometimes it will automatically seek to the position of the state that just crashed the game when loaded next time you start it. But the game would have to be fully deterministic (which in my case it is), but also to work every time. For me, if I start the game unpaused, it somehow fails to sync and to pass the title screen- whereas starting paused and manually unpausing works. Instead, it could be done automatically- just wait a split second before starting to advance the game (at least optionally, maybe?).
Next, let's talk error prevention. Loading a state will sometimes truncate the TAS- which I really don't understand why it sometimes doesn't do that for me by the way, it seems fairly random- and the only way to get it back is to locate the folder it saves backups to. So first, I’ve added an option to open the backups in a file explorer, (as well as several more criteria) because those settings already allude to the backups feature,
and it could easily just tell you where it saves. More importantly, this could be prevented in the first place just by a simple popup warning, as it already does if you knowingly truncate.
Another thing I've had happen to me is when I load a state, forget to switch the mode back to play, unpause, and realize by the time it's too late that I now have a whole bunch of frames to re-record- which you can imagine would surely frustrate anyone. So an option to make that automatic in the Input Editor would probably help.
Another thing I think could be improved is Markers. They currently all behave as what I would call “time markers”- i.e. they don’t move up when deleting previous frames. That is obviously in many cases useful- when you want to test two or more routes in a game for speed- but not for marking specific events unless you don’t skip levels or sections prior to it.
So the suggestion is to call the current marker functionality “time markers”, and add another set of markers with the option “Add frame marker” (or flex marker, perhaps?). The markers could also be grouped, making it easier to organize markers for levels, specific moments, deterministic jump-off points, etc.
- - - MOUSE - - -
Speaking of inputs, there is currently no option to scale the position of your cursor on the window. There are width and height commands, but the issue is that
TASVideos.org wants your main submission to be at the default window size (which for Flash games is usually low quality). So if you want to make your own encode in HD, you have to redo it in that new size. So I proposed having multiplier values under Input. This would also warn if the scale you set is not a whole number (2, 3, etc. rather than 1.5, 1.2), to prevent potential desyncs due to rounding.
Also, the header of Ruffle varies in height at the top of the window, and mouse Y coordinate goes from the top down. That means if you do not use “no-gui”, someone else could get a desync if their display scale is different. So firstly, in the Virtual Resolution, there could be some option to specify scale.
Second off, there could be pre- and post-scale offset vectors added to the input section so you can subtract the width of Ruffle gui on your system, or compensate for any letterboxing that happens because of window size commands. This seems fairly basic and even though it is encouraged to use no-gui, people miss that sometimes. Another possibility is to warn if you are recording a sequence file when no-gui isn’t used, but it would have to detect Ruffle somehow if it doesn’t already.
Speaking of Ruffle header, many TASers might want to do not just frame-perfect tricks, but pixel-perfect ones. So if libTAS were to store and overlay the previous frame at a chosen opacity, and offer a magnifying bubble around the mouse with radius and mag. scale settings, that may help also. This could possibly be done in imGUI, as it seems that already scales the window without scaling the game resolution.
Also, I noticed when I load a state, the ImGUI arrangements on the game will revert back to when they were saved. Perhaps this could be helpful somehow, but for me it just is annoying. So I would leave this off by default.
So that’s pretty much it. If there is something you disagree with here, that is understandable because they are all merely ideas, some of which might seem fairly extravagant. What I want to drive home is some general clarity improvements to libTAS’s arrangement that I’m sure we can mostly unanimously agree on, as some of them are pretty basic- like the fact that most people use save states and libTAS doesn’t have UI options to do so; or the lack of popup questions when loading save states that your sequence will truncate, or when loading branches; or the lack of mouse scaling. I look forward to reading your thoughts, so we can discuss this.