Emulator Coder, Experienced Forum User, Published Author, Experienced player
(724)
Joined: 2/23/2006
Posts: 682
adelikat wrote:
That's an important point! For Vault rules, there must be some kind of distinction between homebrew and hack based on this logic.
What kind of distinction? Both should not go into Vault. Either a movie (made for romhack or homebrew game) is entertaining, then it goes to Moons, or it is not, and then it is rejected.
There may be an exception for notable romhacks and indie games, e.g. if Cave Story or Rockman 4 MI speedrun somehow would not be entertaining, then it could be published in the Vault. But speedruns for notable games/mods are usually entertaining/popular enough to be in Moons, so I'm sure these exceptions won't be needed.
Opening the Vault for all commercial games was already pretty risky. You don't know what the site will look like in a year because of this decision. So it's wise to wait some considerable time before attempting to open it even wider.
Emulator Coder, Experienced Forum User, Published Author, Experienced player
(724)
Joined: 2/23/2006
Posts: 682
adelikat wrote:
The logic for disallowing hacks is that it demands them to have entertainment value thus weeding out all the bad hacks out there (there's much more bad than good). Can we just lump homebrew and unlicensed into that logic as well?
Yes.
Copyrights aside, romhacking scene is as creative as homebrew scene, so I don't see why a homebrew game run can be published without being entertaining, while a romhack run cannot. Rules should be the same for those. As in, both have to be entertaining, period.
Emulator Coder, Experienced Forum User, Published Author, Experienced player
(724)
Joined: 2/23/2006
Posts: 682
Pretty entertaining run! I'm glad the tank skip in level 2 actually proved to be useful.
Heidman wrote:
Please note that this was all done on TAS Editor. TAS Editor likes to crash often, reset tweak counts every time it is reloaded, and just over all is not the best thing to use.
Please, change the "TAS Editor" there to "TAS Edit", because right now the text is misleading. What you used in FCEUX 2.1.5 was pre-alpha version of the first experimental prototype of TAS Editor's source of inspiration. The new TAS Editor is totally different (check it), so there's no crashes or tweak counts.
Heidman wrote:
My tweak count almost always loaded up at ~30k tweaks. I ended a bunch of sessions with that count well over ~200k tweaks, but reload and it is back to 28-30k. I don't get it but but it is not a big deal.
There's no reliable way to translate tweaks to rerecords. So actual TAS Editor counts rerecords.
Emulator Coder, Experienced Forum User, Published Author, Experienced player
(724)
Joined: 2/23/2006
Posts: 682
adelikat wrote:
The minimum that needs to happen is for something like that to be on the movie module itself. Clicking rate would open up a gui for selecting the values, rather than taking you to a new page.
This.
Ideally, people should be able to rate the movie while still watching it (maybe near the end of playing it).
Also. My dream is that this module would additionally pop up a tooltip-like list of "movies I rated", hinting the place this movie will take in this list if I confirm the current positions of sliders by clicking "Rate!". That list should only show 3 entries above and 3 entries below the movie, so I can quickly glance at the tooltip, remember the awesomeness of those old movies and compare the movie I just watched to those movies, then readjust my rating if needed, and after seeing its new place, confirm the rating.
I think this feature would greatly encourage ratings and discourage manipulations, since people will be thinking in terms of their personal collection, and not in terms of average statistics of the site.
Emulator Coder, Experienced Forum User, Published Author, Experienced player
(724)
Joined: 2/23/2006
Posts: 682
Button mnemonics say B_T_TAS, huh. And later it even counts as a valid Input to start the game.
I like it when TASes make some people think that there was cheating involved (e.g. that this is not original Battletoads, because of the music), while there actually wasn't any cheating.
Now let's see if Zlomus finds some other improvements!
Emulator Coder, Experienced Forum User, Published Author, Experienced player
(724)
Joined: 2/23/2006
Posts: 682
GMan wrote:
This gives rise to this decision procedure, correct?
I don't think the judging process is as simple. It can't be efficiently automated. I think only the division to popular and unpopular speedruns can be automated, everything else (even Stars) should be manual. So I wouldn't start drawing flowcharts for the process, especially with symmetry in mind! :)
SmashManiac wrote:
If you guys need to create 3D graphics and flowcharts to explain the tiers system, you can be sure nobody will understand it outside of the community.
But these charts are not meant for outsiders, they were made for site maintainers, to let them better understand the system they are employing. Also, I agree that multi-tiered site is more confusing for casual viewers, hence I was suggesting single-tiered system similar to old site system but with smart filters.
This and this are single-tiered system (+Stars, which are not really a tier, but a subset).
This and this are multi-tiered system with arbitrary division principles.
However, I'm not saying the tiers should be reverted. I say, let it be tested for a year or so, and when it becomes obvious the system failed, we can return to the thread.
SmashManiac wrote:
Personally I don't understand why we should separate movies into tiers in the first place, considering movies already have user ratings and can be sorted as such (at least for 8.0+ ratings). Why have a parallel system? Shouldn't stars and moons be automatically attributed based on that rating?
If Stars meaning has changed to "the most entertaining movies" then yes, they can be automatically attributed, but until their meaning is (was) "the showcase of the site content", they have to be manually picked.
Emulator Coder, Experienced Forum User, Published Author, Experienced player
(724)
Joined: 2/23/2006
Posts: 682
adelikat wrote:
Nice image. I think this precisely visualizes what I want.
OK, but, can you elaborate why there's this need to significantly increase the number of starred movies? Stars were good when they seemed like an unreachable height which motivated to use speed-entertainment tradeoffs, because people knew that pure speedrun is unlikely to make it into Stars.
But if there's more than 10% possibility to appear in Stars, every good run (i.e. rated above average) author becomes annoyed that it didn't become starred. Then he starts to compare his movie to starred movies and notices that there's not much difference. Like, this starred movie got even less entertainment rating than his movie, and this starred movie is completely suboptimal and poorly made, those three movies are both from one series, etc. This definitely wasn't the case when starred movies were obviously better than common movies.
adelikat wrote:
You seem to be concerned about size of the red slice in these proposed scheme. However, the size in relation to that tier is up to the TASers, no me. We don't get many of these types of this submissions, for reasons other than site rules.
We don't get many of these types exactly because we didn't form a league of playaround sports, like we did with tool-assisted speedruns.
Check this out, we also didn't get many of DOS/SMS/GG submissions, because people knew such submissions aren't gonna be published. But now the Vault is open, so people are motivated to TAS Atari/MSX/DOS/etc.
I am talking about the same kind of motivation applied to playarounds and esoteric goals. By making a dedicated tier for playarounds you inevitably loosen the rules of their acceptance. Right now the only way for a playaround to be published is to receive overwhelming support with little to no "No" votes. But once a site section appears, it asks to be filled with content. This will make judges accept questionable playarounds that only amuse specific audience, which in turn makes TASers (especially newbies) to think that experiments are encouraged. When you feel there's too much junk you can tighten the rules (and I'm sure you'll eventually tighten the rules of accepting "Unpopular speedruns" after there's more than 1000 movies in the Vault). But the initial incentive for including suboptimal antics into TASes will remain for some time even after the rules raise the bar again.
Do note that I'm not suggesting to lower the requirements artificially, because it's hard to define to what extent those can be lowered without damaging the site content. But I'm suggesting to rely on natural laws of supply and demand. You just have to create a market.
Also, if we had a dedicated storage for suboptimal movies, this movie would still be public, instead of being obsoleted by a speedrun. Yes, it has been starred, but you can't find it in the list of Stars.
adelikat wrote:
So far the system is clearly documented with a logical workflow that is mostly similar to what it is today. There's some one time labor happening to move to this set up (mostly in the re-judging of material that is now acceptable). But the workflow for this is easy to maintain and flexible for changes.
The very fact that the tier is manually defined makes the system inflexible. What if a judge promotes his favourite (but otherwise unpopular) game speedrun to Moons, where people couldn't even overthrow it by downvoting (especially if there's an artificial bonus for Moons/Stars)?
Plus the question of cross-tier obsoletions is just asking for trouble. We have yet to see if improvements for Vault movies are going to be judged independently from the "Vault curse" of the original movie. That really depends on judge, I guess, but people tend to go with the flow, and placing the improvement to the same shelf sounds very tempting.
adelikat wrote:
Also, at this time I am not prepared to rely solely on ratings for tier decisions. These are too easy to manipulate and lack enough accuracy to trust it in something automated.
The accuracy can be improved, the manipulation can be handled (I mean extreme cases of manipulations, as for minor manipulations, this is natural and cannot be avoided, since the voting itself is manipulation).
Anyway the automation vs. manual appointment is only the question of policy (democracy vs. authoritarianism), but the "5 checkboxes" idea can work with manual appointment too. BTW, what I'm calling checkboxes can as well be implemented as tiers, it's just that 5 tiers are too much, in my opinion. I also thought about a "slider" control, but this is still too esoteric. Oh well, I guess these are not the times for innovations. I'll remind you about the idea in a couple of years. Don't lock the thread please!
feos wrote:
AnS: if you pull Demos (14) and Playarounds (26) out of Vault (211) and Moons (738), niether Vault, nor Moons list would become significantly smaller or easier to observe. So the resulting view would be of the same measure of chaos, which I personally find unwatchable.
Haha, prepare to have >1000 Vault movies! >-] I don't know what to do with big numbers.
BTW, I wasn't trying to make Moons less overloaded. What I want is to have 140 Demos and 260 Playarounds. And when someone invents a way to handle 1000 Moons and 1000 Vaults then the same trick can be applied to 280 Demos and 520 Playarounds.
feos wrote:
Can you guys invent something that would fairly break those insane lists to something sensible?
I don't think anyone really browses the pages listing movies from all platforms, so maybe it's not such a problem. But if it is, then maybe pages should auto-append like in Facebook, e.g. only show 100 movies at first, and every time user scrolls too close to the bottom of the page it should unroll 50 more entries, etc.
Emulator Coder, Experienced Forum User, Published Author, Experienced player
(724)
Joined: 2/23/2006
Posts: 682
Indeed, with original proposal there's a lot of manual work, which inevitably makes the system inflexible and hard to maintain (and drama-inciting, since the Vault status is determined by a judge once and forever).
But it just saddens me that noone tried to think about GMan's proposal.
Here I'll try to talk in pictures.
First here's the legend of all current types of Superplay movies we have on site:
Of course the vast majority of TASVideos movies was always the "Entertaining speedruns". For example, here's how the site movies were just yesterday (very roughly):
Now, Adelikat wants to publish many Atari runs, so it's promptly decided to make a separate category for "Unpopular speedruns". Also the Stars are going to be inflated, for an unexplained reason.
Then people jump the bandwagon and suggest allotting a tier for "Demos", because they care about such movies.
Do you imagine how ugly this "Tiers 2.0" system is gonna be? That's because there will be no real system, just lots of manual labour and subjective selections. But what concerns me the most, such division also hurts "Playaround" movies.
But I couldn't think about better system, until GMan's proposal.
This system is both unrestrictive and systematic up to the point that it can be automated. So here's my proposal.
1. When a movie is submitted, people vote Yes/No/Meh, estimating the future entertainment value of the movie.
2. Judge decides whether to publish the movie. If the estimated value is low then this movie can be published only if it's the fastest playthrough in the world. If the estimated value is high then the movie is published, even if the goal is esoteric or the movie is intentionally suboptimal (not poorly made, but not aimed on speed).
3. All movies are published in one tier. Stars and Moons (or whatever) remain their original meaning (hand-picked, about 5% of all movies on the site).
4. Visitors can browse either the full archive or separate parts (Playarounds/Speedruns/Records/Demos/Stars). While browsing it's possible to uncheck/recheck any of 5 options (e.g. in the standard header of every page, right near the Megaman sprite).
5. The division to 5 categories is automatic. All movies with "Playaround" flag go to Playarounds, "Demonstrations" go to Demos, Starred go to Stars, speedruns with entertainment rating less than precalculated value (say, 5.0) go to Records, everything else goes to Speedruns. Thus no manual labour is needed, plus an unpopular speedrun is not doomed forever, as it has a chance to go up (for example, AVGN reviews the game and people start upvoting the run).
Yes, the "checkboxes" idea is knda meh, and it probably should be reconsidered into something more casual. But the general direction of his idea is much more solid than previous proposals.
Emulator Coder, Experienced Forum User, Published Author, Experienced player
(724)
Joined: 2/23/2006
Posts: 682
adelikat wrote:
Also, today I moved every existing publication with a rating under 6.0 to the vault. I picked this number because the result set looked to be a good case of border-line vault material movies. Was this the perfect list? No, but it got pretty close. I expect there will be outliers, so if you users disagree with some of those moves, please discuss in the thread for that publication.
Why are you using the general rating value instead of only entertainment rating? Isn't there supposed to be entertainment-based system in tiers? Right now there's even one Playaround and one Demonstration in the Vault! This is terribly incorrect. I thought you proposed the Vault as a place for speedruns of less popular games, but it appears you really meant it to be a dump.
Emulator Coder, Experienced Forum User, Published Author, Experienced player
(724)
Joined: 2/23/2006
Posts: 682
Yes, the author's naming isn't perfect. According to it, most of current movies are "Star" or at least "Superspeed". And yet all of them can be actually called Superplay, since it's so wide term. But semantics aside, the division shows that "Demos" cannot be a tier in a system that is simply sorted by entertainment value (original proposal by Adelikat).
Try looking at the diagram with conventional naming:
Star = Regular movies (called "Moons" in Adelikat's proposal) + Starred movies (which are just a hand-picked subset of regular movies)
Superplay = Runs with speed-entertainment tradeoffs
Superspeed = "Vault"
Moon = Demos, April1st, etc
Grue Food = Gruefood
Freeplay = Gruefood
Demo = Gruefood
Also replace the "Technical" axis with something like "using shortest route". Because all published runs are supposed to be technical (well polished and not poorly made), but not all of them are supposed to use the fastest method of achieving the goal.
Emulator Coder, Experienced Forum User, Published Author, Experienced player
(724)
Joined: 2/23/2006
Posts: 682
GMan wrote:
Now the point of these new tiers (I think) is to ensure these kinds of goals can be submitted to the site; they'll just be segmented off and clearly labeled "this requires you have a certain knack for this stuff" warning.
No, I don't think the point of these new tiers is to allow publishing esoteric goals, the original point is to allow publishing unentertaining speedruns that are still world records.
Anything can be submitted to the site, but if the goal is both too esoteric and too unentertaining, it doesn't get published.
So if the "Super Mario 64 minimal A presses" receives many No votes, the judge rejects it because it's both esoteric and unentertaining. But if it receives many Yes votes, the judge acceptes it and puts into Demos tier (octant).
So. the entertainment axis is still the main factor, which can even justify a Freerun (although I agree that there should not be such category called "Freerun" on the site).
Emulator Coder, Experienced Forum User, Published Author, Experienced player
(724)
Joined: 2/23/2006
Posts: 682
GMan wrote:
I've made visualizations of the resulting categories. Note: I made my own names for the categories that make sense to me, and I'll detail them below. Most important, I've reevaluated what I think a "Moon" movie should be, so don't let that throw you off.
Look, this is very solid. I like this, hopefully everyone else takes their time to read and appraise it.
I support both the categorization and suggested site changes (although these should be thought out more, because e.g. when a visitor browses a page of certain game, the checkboxes shouldn't matter, since he most likely wants to find everything about the game, no matter what general preferences he has). Anyway, that's just details, but overall the proposal is great.
However, as for workbench voting, I think there's no need to make it any more difficult (one experiment has already failed in the past). Once the question is changed back to "Do you find this entertaining?", it will serve its purpose. Because technical merit and goal objectivity of a submission are more obvious to a judge, so he only needs to ask the audience about entertainment (well, sometimes he also needs to consider technical details, but such matter cannot be answered by a simple yes/meh/no, for this matter there is forum duscussion).
Emulator Coder, Experienced Forum User, Published Author, Experienced player
(724)
Joined: 2/23/2006
Posts: 682
BrotherMojo wrote:
I mean that I'm getting different results when it doesn't seem like I should. In one branch, I am "switching weapons" while paused, but in a way that doesn't actually change what weapon I have equipped. (Specifically, switching weapons to the right when I already have the rightmost weapon equipped.) This is causing a lag frame to appear or disappear significantly later, and I'm trying to figure out why.
Ah, that's not quite desync, you're just dealing with unknown behavior of the game that needs to be investigated. Indeed, such cases need both RAM observation and probably even code debugging.
BrotherMojo wrote:
Though this helps me visibly compare the resulting frames, clicking back and forth between bookmarks doesn't seem to update RAM search, so that doesn't work. I'm trying to find what RAM values differ between those two branches, since there are few (purely cosmetic) differences until the lag frame occurs and disrupts the whole movie.
Wait, how is doesn't update RAM Search? I've just tested branching, and it updates it (and if it weren't, that would be a bug).
Are you sure you actually switch between branches? To do it, you have to double-click the icon, not just single-click.
Emulator Coder, Experienced Forum User, Published Author, Experienced player
(724)
Joined: 2/23/2006
Posts: 682
I agree with Derakon and feos, ratings should not depend on tier. If a (vault) movie suddenly gets many high votes then it simply means the judge made a mistake by not considering the movie entertaining. And I don't think the formula should minimize the audience impact.
rog wrote:
Movies with less than probably 10 votes or so should probably get a penalty. Above that though, i don't think anything needs to be done.
Defining limits by constant numbers is asking for trouble. The number should be calculated on the fly, to account for audience growing. Maybe it should be median of the number of votes for all published movies.
Emulator Coder, Experienced Forum User, Published Author, Experienced player
(724)
Joined: 2/23/2006
Posts: 682
CoolKirby wrote:
So if any% is defined as "fastest possible completion" and there are only any% and 100% categories, will a "glitched" run become the new any% run?
Then the glitched run should be the in the records (speedruns) (vault), and the old any% run (glitchless) should be either obsoleted or moved from Speedruns to Moons (Superplays), depending on its entertainment value.
Emulator Coder, Experienced Forum User, Published Author, Experienced player
(724)
Joined: 2/23/2006
Posts: 682
BrotherMojo wrote:
I've been working on a Blaster Master TAS, doing the whole thing by hand in TAS Edit...
Hey, I have great news for you. TAS Edit from FCEUX 2.1.5 was completely rewritten into TAS Editor for FCEUX 2.1.6. Download it here: http://fceux.com/zip
The issue about RAM Search is fixed there. Feel free to ask questions on usage, because it's very different from what old TAS Edit used to be.
BrotherMojo wrote:
I just find it a lot easier and more intuitive than the traditional frame advance + rerecord method.
That's damn right.
BrotherMojo wrote:
Also, there's a RAM Search feature that I would find extremely useful but I haven't seen it documented so I don't know if it exists. I'm getting some desync when changing the input sequence during pause frames, manifesting itself as minor graphical differences (the tank's tires are rotating slightly differently, but the tank's position and velocity are the same) followed by dis/appearance of lag frames at seemingly unrelated times. It would be really great if I could use RAM search to compare, say, frame 2600 of two movies which differ only by a minor change in the input on frame 2132. Is there a convenient way to do this?
Not sure what you mean by desync in this context.
To compare results of different movie branches you should use Bookmarks (movie savestates).
1. Make the first movie and stop the Playback cursor (light-blue line) at frame 2600. Save this to Bookmark 1 (either by pressing Shift+F1 or right-clicking on the first Bookmark). The "1" icon will appear near the frame in the Piano Roll.
2. Change Input of frame 2132/etc and replay this newly created branch of your project to the frame 2600. Save it to Bookmark 2 (Shift+F2). The "2" icon will appear near the frame in the Piano Roll, but the "1" is still there, under it.
3. Now you can compare two versions of the game state at frame 2600 by repeatedly switching between these two branches. To do it, either press F1 or F2 or F1 or F2... or double-click on respective Bookmark icons in the Branches Tree.
In your case the Tree will look as simple as that:
To switch to the Tree view, click on the Bookmarks/Branches caption.
Emulator Coder, Experienced Forum User, Published Author, Experienced player
(724)
Joined: 2/23/2006
Posts: 682
Ilari wrote:
Yes, there needs to be a better name for that...
Actually, the Vault also sounds strange. A bit derogatory, that is.
I think there should not be any emotionally colored difference between the two main tiers.
Since we all know what these tiers are about (in their essence), why not just call them literally instead of using allusions?
1st tier - Speedruns
2nd tier - Superplay
3rd category - Stars
Emulator Coder, Experienced Forum User, Published Author, Experienced player
(724)
Joined: 2/23/2006
Posts: 682
adelikat wrote:
By the current wording, entries in the vault say it must MATCH or beat all existing records. Perhaps I should reword that to say it must beat all known unassisted records, and at least match all known tool-assisted records?
Indeed, what would be the point of TASes that only match unassisted record?
Emulator Coder, Experienced Forum User, Published Author, Experienced player
(724)
Joined: 2/23/2006
Posts: 682
Yes, these would be two steps in the right direction:
1. Emphasize the Tool-Assisted Superplay aspect of the movies by making the entertainment value decide the tier.
2. Let the non-superplay TASes (which are still a record) be published in the Vault tier.
But there shouldn't be more than two tiers. The Starts should keep their meaning as a very limited selection of the best movies (the showcase of the site, which must be a single webpage that can be browsed in less than several minutes, of course not counting the time possibly spent for watching videos). Stars should not become a third tier, or else there will be much more severe drama about cool movies not making it into the highest tier.
Warepire wrote:
For standards I assume a good guideline would be that if it's impossible to complete the game faster by TASing it compared to playing it casually, it should not be allowed. Most (all?) of the WarioWare games fall into this category in my opinion.
But if it's really impossible then such TAS naturally cannot be considered a record, so it doesn't fit into the Vault. However, it may suddenly fit into Moons if TASer manages to make something ingenious while waiting/playing.
Emulator Coder, Experienced Forum User, Published Author, Experienced player
(724)
Joined: 2/23/2006
Posts: 682
cak wrote:
It's not the rom, it's FCEUX. Just keep reloading the rom until it shows the title screen, then make a save state. After that, you can just load the state and it'll work every time.
Of course the vast majority of TASVideos movies was always the "Entertaining speedruns". For example, here's how the site movies were just yesterday (very roughly):
Now, Adelikat wants to publish many Atari runs, so it's promptly decided to make a separate category for "Unpopular speedruns". Also the Stars are going to be inflated, for an unexplained reason.
Then people jump the bandwagon and suggest allotting a tier for "Demos", because they care about such movies.
Do you imagine how ugly this "Tiers 2.0" system is gonna be? That's because there will be no real system, just lots of manual labour and subjective selections. But what concerns me the most, such division also hurts "Playaround" movies.
But I couldn't think about better system, until GMan's proposal.
This system is both unrestrictive and systematic up to the point that it can be automated. So here's my proposal.
1. When a movie is submitted, people vote Yes/No/Meh, estimating the future entertainment value of the movie.
2. Judge decides whether to publish the movie. If the estimated value is low then this movie can be published only if it's the fastest playthrough in the world. If the estimated value is high then the movie is published, even if the goal is esoteric or the movie is intentionally suboptimal (not poorly made, but not aimed on speed).
3. All movies are published in one tier. Stars and Moons (or whatever) remain their original meaning (hand-picked, about 5% of all movies on the site).
4. Visitors can browse either the full archive or separate parts (Playarounds/Speedruns/Records/Demos/Stars). While browsing it's possible to uncheck/recheck any of 5 options (e.g. in the standard header of every page, right near the Megaman sprite).
5. The division to 5 categories is automatic. All movies with "Playaround" flag go to Playarounds, "Demonstrations" go to Demos, Starred go to Stars, speedruns with entertainment rating less than precalculated value (say, 5.0) go to Records, everything else goes to Speedruns. Thus no manual labour is needed, plus an unpopular speedrun is not doomed forever, as it has a chance to go up (for example, AVGN reviews the game and people start upvoting the run).
Yes, the "checkboxes" idea is knda meh, and it probably should be reconsidered into something more casual. But the general direction of his idea is much more solid than previous proposals.
To switch to the Tree view, click on the Bookmarks/Branches caption.