So... Mega Man 2 The Power Fighters have 3 distinct routes, each one with a different set of bosses. Since it's on the vault tier, it means only the fastest route will be published?
My first language is not English, so please excuse myself if I write something wrong. I'll do my best do write as cleary as I can, so cope with me here =)
(ﾉಥ益ಥ）ﾉ
Joined: 3/23/2012
Posts: 296
Location: In X position=50 y position=20
OmegaWatcher wrote:
So... Mega Man 2 The Power Fighters have 3 distinct routes, each one with a different set of bosses. Since it's on the vault tier, it means only the fastest route will be published?
all three routes should be seperately different branch
Jungon wrote:
if I was to have a Tool-Assisted real life ... I'd..
I could abuse death, just to see if it saves time ..
Emulator Coder, Site Developer, Site Owner, Expert player
(3573)
Joined: 11/3/2004
Posts: 4754
Location: Tennessee
subanark wrote:
From my understanding, any game that doesn't have "beats the game as fast as possible" as a goal can only be considered for moon and star, but not vault? While any run that beats the game using the clear and obvious way of "beating" the game can be put in the vault if it is optimized sufficiently? And there are still "games" that don't have any clear way of "beating it" that can be rejected for bad game choice?
You are correct on all your points. Except, the vault can also have "full game completion" type goals as well.
OmegaWatcher wrote:
So... Mega Man 2 The Power Fighters have 3 distinct routes, each one with a different set of bosses. Since it's on the vault tier, it means only the fastest route will be published?
The fastest of those routes can be easily published into the vault. The other 2, by rule, would have to be entertaining enough to warrant a moon publication. I am clarifying this because your question seems to imply that a vault publication means that game is doomed forever to be in the vault. That is not the case.
Also, today I moved every existing publication with a rating under 6.0 to the vault. I picked this number because the result set looked to be a good case of border-line vault material movies. Was this the perfect list? No, but it got pretty close. I expect there will be outliers, so if you users disagree with some of those moves, please discuss in the thread for that publication.
I guess I didn't notice this thread until the system got deployed...
I did't notice this thread either. I don't think any new change would be too late, though.
<klmz> it reminds me of that people used to keep quoting adelikat's IRC statements in the old good days
<adelikat> no doubt
<adelikat> klmz, they still do
Also, today I moved every existing publication with a rating under 6.0 to the vault.
This isn't about the content of the movie itself, but the GBC SpongeBob run is at exactly 6.0, not under, though it somehow ended up in the Vault. Was that a mistake?
This isn't about the content of the movie itself, but the GBC SpongeBob run is at exactly 6.0, not under, though it somehow ended up in the Vault.
Rounding. The true rating is 5.97 exact (both formulas, see below).
Discovered something interesting. There are two versions of the rating formula, one in the database (stored function) and one in the site code. These two do not agree.
E.g. On [1927] MSX Invasion of the Zombie Monsters by scrimpeh in 04:15.31, the former says: 7559/1260=5.99920634920634920634 but the latter says 43059/7000=6.15128571428571428571.
The mathematical reason why these two don't agree:
- The former formula computes average of votes weighted by product of vote category weight and voter weight.
- The latter computes averages of categories weighted by voter weight and then averages those according to category weight.
These two methods are not equivalent.
I like this proposal.
One question: how does this mesh with existing categories, in particular "not using a particular glitch" (e.g. Sonic's out-of-bounds level skipping)? I suppose the fastest with all glitches would qualify for the Vault, and a more visually pleasing one (that doesn't use the glitch) could be a Star, am I correct on this?
Joined: 11/20/2004
Posts: 236
Location: United States
Wow. This new site arrangement is... very um... Confusing. Very confusing indeed.
I skimmed the proposal and such, but even after doing that, it's still confusing for oldbies like me who "grew up on" starred entries being super awesome and moon entries being extraordinaire or really unique in some way. Now, it just looks like every video on the site is worth watching.
I guess I'll have to get used to it. Your site, your rules. I'll probably get the hang of it within a few days or so...
Also, today I moved every existing publication with a rating under 6.0 to the vault. I picked this number because the result set looked to be a good case of border-line vault material movies. Was this the perfect list? No, but it got pretty close. I expect there will be outliers, so if you users disagree with some of those moves, please discuss in the thread for that publication.
Why are you using the general rating value instead of only entertainment rating? Isn't there supposed to be entertainment-based system in tiers? Right now there's even one Playaround and one Demonstration in the Vault! This is terribly incorrect. I thought you proposed the Vault as a place for speedruns of less popular games, but it appears you really meant it to be a dump.
Emulator Coder, Site Developer, Site Owner, Expert player
(3573)
Joined: 11/3/2004
Posts: 4754
Location: Tennessee
Be patient, I tried to get all the playarounds and whatnot out of there, but I guess I missed a few? A more helpful thing would be to point out the ones I missed, thanks.
Also, today I moved every existing publication with a rating under 6.0 to the vault. I picked this number because the result set looked to be a good case of border-line vault material movies. Was this the perfect list? No, but it got pretty close. I expect there will be outliers, so if you users disagree with some of those moves, please discuss in the thread for that publication.
Why are you using the general rating value instead of only entertainment rating? Isn't there supposed to be entertainment-based system in tiers? Right now there's even one Playaround and one Demonstration in the Vault! This is terribly incorrect. I thought you proposed the Vault as a place for speedruns of less popular games, but it appears you really meant it to be a dump.
Give it some time... there is a lot to do and all TASes need to be classified.
The automated system isn't 100% accurate. Of lot of manual work is necessary. =)
Indeed, with original proposal there's a lot of manual work, which inevitably makes the system inflexible and hard to maintain (and drama-inciting, since the Vault status is determined by a judge once and forever).
But it just saddens me that noone tried to think about GMan's proposal.
Here I'll try to talk in pictures.
First here's the legend of all current types of Superplay movies we have on site:
Of course the vast majority of TASVideos movies was always the "Entertaining speedruns". For example, here's how the site movies were just yesterday (very roughly):
Now, Adelikat wants to publish many Atari runs, so it's promptly decided to make a separate category for "Unpopular speedruns". Also the Stars are going to be inflated, for an unexplained reason.
Then people jump the bandwagon and suggest allotting a tier for "Demos", because they care about such movies.
Do you imagine how ugly this "Tiers 2.0" system is gonna be? That's because there will be no real system, just lots of manual labour and subjective selections. But what concerns me the most, such division also hurts "Playaround" movies.
But I couldn't think about better system, until GMan's proposal.
This system is both unrestrictive and systematic up to the point that it can be automated. So here's my proposal.
1. When a movie is submitted, people vote Yes/No/Meh, estimating the future entertainment value of the movie.
2. Judge decides whether to publish the movie. If the estimated value is low then this movie can be published only if it's the fastest playthrough in the world. If the estimated value is high then the movie is published, even if the goal is esoteric or the movie is intentionally suboptimal (not poorly made, but not aimed on speed).
3. All movies are published in one tier. Stars and Moons (or whatever) remain their original meaning (hand-picked, about 5% of all movies on the site).
4. Visitors can browse either the full archive or separate parts (Playarounds/Speedruns/Records/Demos/Stars). While browsing it's possible to uncheck/recheck any of 5 options (e.g. in the standard header of every page, right near the Megaman sprite).
5. The division to 5 categories is automatic. All movies with "Playaround" flag go to Playarounds, "Demonstrations" go to Demos, Starred go to Stars, speedruns with entertainment rating less than precalculated value (say, 5.0) go to Records, everything else goes to Speedruns. Thus no manual labour is needed, plus an unpopular speedrun is not doomed forever, as it has a chance to go up (for example, AVGN reviews the game and people start upvoting the run).
Yes, the "checkboxes" idea is knda meh, and it probably should be reconsidered into something more casual. But the general direction of his idea is much more solid than previous proposals.
Joined: 4/17/2010
Posts: 11477
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
I need to admit that now, when every aspect of the above system was explained, it neither contradicts my own thoughts, nor it is hard to implement and maintain, since only the ENTERTAINMENT rating decides the Entertaining/Unpopular placement. Demo/Playaround are decided by the very goal of the run. Moons/Stars are manual.
I already stated a long ago that after creating statistics pages for each tier, we could decide whether the quota is fair.
Some ideas about the maintenance. There must be a thread for each tier, linking to:
tier descripton
statistics page (I suggest having all tiers stats on one separate page),
tier log page (separate for each tier, a full history, like the SVN changes pages, shows last 10 changes, but expandable to all).
In the thread people would not only discuss the placement of a certain run, but all related stuff.
EDIT: after checking the view options it looks good to me to still allow the platform branching we currently use. Brouse the content as usually, just exclude the stuff you're not interested in ATM. Remember the checkboxes for each user.
EDIT 2: the word Records sounds more promising than Speedruns. Replace the word Speedruns with Superplays (for their art value) and it starts working. An oportunity to advertise the word Superplay after all :D
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
I definitely think entertaining shouldn't be the only deciding factor: there are a lot of movies of excellent quality that don't get accepted because they're repetitive or dull, but they're still good examples of TAS.
Now, is this basically the tier sistem you're suggesting? I understand it this way:
-Star: best movies in the site, highlights.
-Moon: entertaining movies (though not of star quality).
-Vault: pure speed runs, regardless of everything else.
Antes de una TAS:
-Malo: ¡Voy a conquistar el mundo!
-Héroe: ¡Por poco tiempo!
I definitely think entertaining shouldn't be the only deciding factor: there are a lot of movies of excellent quality that don't get accepted because they're repetitive or dull, but they're still good examples of TAS.
Now, is this basically the tier sistem you're suggesting? I understand it this way:
-Star: best movies in the site, highlights.
-Moon: entertaining movies (though not of star quality).
-Vault: pure speed runs, regardless of everything else.
Yeah, pretty much. But it's based on ratings too from how I'm taking it (?), so I guess a run could technically hop in and out of a tier.
--
Also, with this new system it seems like every run is now "notable" in some way... I guess cause It'll take some time to get used to it. I noticed a little lightning bolt picture beside some TAS's (is that now the new determination of notable?)
Thank you for that! Definitely a clearer perspective on what was jumbling around in my mind, including for me. I think this way simply makes more sense, as more natural division of movies.
And you're right that entertainment should still be a primary factor. If a SM64 A-less run wants to get submitted, if needs to have a wider audience than just the SM64 thread if it's going to represent the site.
Well, since the workbench now contains about a hundred open entries, I went ahead and voted on about a dozen of them with a browser tabs... and this promptly triggered an automated ban on me. That was weird :)
Also, today I moved every existing publication with a rating under 6.0 to the vault. I picked this number because the result set looked to be a good case of border-line vault material movies.
Just something to watch out for, but if the movie's final rating ever gets changed to be based on tier (for example, Vault movies get a final rating based more heavily on Technical Quality than Entertainment) then using the final rating to place by tier is divergent.
Assume for a moment we have the Vault category, where movies go if their final rating is under 6 (otherwise they stay where they are). If we do a scheme like final score depends on category, then when the movie is moved to Vault (because its final score is less than 6) its final score calculation will change to be more Technical based. This could be enough to move its final score back above 6, so it gets moved (if this is automated)/should be moved(for consistency) back out of Vault. But now its out of Vault so its Entertainment factor is more important, pushing the final score below 6, etc.
I like both the idea of moving between tiers automatically and changing final rating based on tier. The former just needs to use a score independent of that, like the pure Entertainment rating (which in both of our proposals is the only thing that determines if something goes in the "vault" category, as all other categories require entertainment).
Emulator Coder, Site Developer, Site Owner, Expert player
(3573)
Joined: 11/3/2004
Posts: 4754
Location: Tennessee
Moving <6 was a one time deal. It was an arbitrary decision that got a decent result set of vault material. I do agree that using entertainment < some value is a better way to do it, and am in the process of transitioning to this.
For new publications, the decision will be based on the submission audience + judge. This was just a way to get a pretty good retroactive move. I just assume grant "amnesty" to any publications in moon that passed this cutoff but are arguable.
Emulator Coder, Site Developer, Site Owner, Expert player
(3573)
Joined: 11/3/2004
Posts: 4754
Location: Tennessee
AnS wrote:
Now, Adelikat wants to publish many Atari runs
This idea has been in the works for quite some time, I resent this statement.
Nice image. I think this precisely visualizes what I want. You seem to be concerned about size of the red slice in these proposed scheme. However, the size in relation to that tier is up to the TASers, no me. We don't get many of these types of this submissions, for reasons other than site rules.
Do you imagine how ugly this "Tiers 2.0" system is gonna be?
I don't see the logical jump from the images you presented and "an ugly mess" Nor do I see how any of this hurts playaround. Llike I said, people aren't making many of these, this will be true now matter the level if inclusionism we adopt.
That's because there will be no real system, just lots of manual labour and subjective selections. But what concerns me the most, such division also hurts "Playaround" movies.
So far the system is clearly documented with a logical workflow that is mostly similar to what it is today. There's some one time labor happening to move to this set up (mostly in the re-judging of material that is now acceptable). But the workflow for this is easy to maintain and flexible for changes.
Included in this flexibility is the fact that the tiers themselves can evolve over time, and we can easily add/remove them.
Also, at this time I am not prepared to rely solely on ratings for tier decisions. These are too easy to manipulate and lack enough accuracy to trust it in something automated.
Just a quick note here, I like the new system, but whatever is the most common (moons) shouldn't have a symbol, and/or you should be able to hover over a symbol to learn what it means. Just a minor thing! I don't go here very often and it was really difficult to find the info for what the new stuff is.
Designer of Copy Kitty, a game about giant robots and explosions
Joined: 4/17/2010
Posts: 11477
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
AnS: if you pull Demos (14) and Playarounds (26) out of Vault (211) and Moons (738), niether Vault, nor Moons list would become significantly smaller or easier to observe (even filling the Demo list with arbitrary goal runs we won't solve the global problem). So the resulting view would be of the same measure of chaos, which I personally find unwatchable.
Can you guys invent something that would fairly break those insane lists to something sensible?
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Nice image. I think this precisely visualizes what I want.
OK, but, can you elaborate why there's this need to significantly increase the number of starred movies? Stars were good when they seemed like an unreachable height which motivated to use speed-entertainment tradeoffs, because people knew that pure speedrun is unlikely to make it into Stars.
But if there's more than 10% possibility to appear in Stars, every good run (i.e. rated above average) author becomes annoyed that it didn't become starred. Then he starts to compare his movie to starred movies and notices that there's not much difference. Like, this starred movie got even less entertainment rating than his movie, and this starred movie is completely suboptimal and poorly made, those three movies are both from one series, etc. This definitely wasn't the case when starred movies were obviously better than common movies.
adelikat wrote:
You seem to be concerned about size of the red slice in these proposed scheme. However, the size in relation to that tier is up to the TASers, no me. We don't get many of these types of this submissions, for reasons other than site rules.
We don't get many of these types exactly because we didn't form a league of playaround sports, like we did with tool-assisted speedruns.
Check this out, we also didn't get many of DOS/SMS/GG submissions, because people knew such submissions aren't gonna be published. But now the Vault is open, so people are motivated to TAS Atari/MSX/DOS/etc.
I am talking about the same kind of motivation applied to playarounds and esoteric goals. By making a dedicated tier for playarounds you inevitably loosen the rules of their acceptance. Right now the only way for a playaround to be published is to receive overwhelming support with little to no "No" votes. But once a site section appears, it asks to be filled with content. This will make judges accept questionable playarounds that only amuse specific audience, which in turn makes TASers (especially newbies) to think that experiments are encouraged. When you feel there's too much junk you can tighten the rules (and I'm sure you'll eventually tighten the rules of accepting "Unpopular speedruns" after there's more than 1000 movies in the Vault). But the initial incentive for including suboptimal antics into TASes will remain for some time even after the rules raise the bar again.
Do note that I'm not suggesting to lower the requirements artificially, because it's hard to define to what extent those can be lowered without damaging the site content. But I'm suggesting to rely on natural laws of supply and demand. You just have to create a market.
Also, if we had a dedicated storage for suboptimal movies, this movie would still be public, instead of being obsoleted by a speedrun. Yes, it has been starred, but you can't find it in the list of Stars.
adelikat wrote:
So far the system is clearly documented with a logical workflow that is mostly similar to what it is today. There's some one time labor happening to move to this set up (mostly in the re-judging of material that is now acceptable). But the workflow for this is easy to maintain and flexible for changes.
The very fact that the tier is manually defined makes the system inflexible. What if a judge promotes his favourite (but otherwise unpopular) game speedrun to Moons, where people couldn't even overthrow it by downvoting (especially if there's an artificial bonus for Moons/Stars)?
Plus the question of cross-tier obsoletions is just asking for trouble. We have yet to see if improvements for Vault movies are going to be judged independently from the "Vault curse" of the original movie. That really depends on judge, I guess, but people tend to go with the flow, and placing the improvement to the same shelf sounds very tempting.
adelikat wrote:
Also, at this time I am not prepared to rely solely on ratings for tier decisions. These are too easy to manipulate and lack enough accuracy to trust it in something automated.
The accuracy can be improved, the manipulation can be handled (I mean extreme cases of manipulations, as for minor manipulations, this is natural and cannot be avoided, since the voting itself is manipulation).
Anyway the automation vs. manual appointment is only the question of policy (democracy vs. authoritarianism), but the "5 checkboxes" idea can work with manual appointment too. BTW, what I'm calling checkboxes can as well be implemented as tiers, it's just that 5 tiers are too much, in my opinion. I also thought about a "slider" control, but this is still too esoteric. Oh well, I guess these are not the times for innovations. I'll remind you about the idea in a couple of years. Don't lock the thread please!
feos wrote:
AnS: if you pull Demos (14) and Playarounds (26) out of Vault (211) and Moons (738), niether Vault, nor Moons list would become significantly smaller or easier to observe. So the resulting view would be of the same measure of chaos, which I personally find unwatchable.
Haha, prepare to have >1000 Vault movies! >-] I don't know what to do with big numbers.
BTW, I wasn't trying to make Moons less overloaded. What I want is to have 140 Demos and 260 Playarounds. And when someone invents a way to handle 1000 Moons and 1000 Vaults then the same trick can be applied to 280 Demos and 520 Playarounds.
feos wrote:
Can you guys invent something that would fairly break those insane lists to something sensible?
I don't think anyone really browses the pages listing movies from all platforms, so maybe it's not such a problem. But if it is, then maybe pages should auto-append like in Facebook, e.g. only show 100 movies at first, and every time user scrolls too close to the bottom of the page it should unroll 50 more entries, etc.
Perhaps we should start a thread on improving site navigation to avoid overloading this thread.
Navigation-wise it might be useful (again with the check box idea) to restrict movie navigation by movie attribute. (By attribute I mean "Best Ending", "Heavy Glitch Abuse".)
This would be a column on the right-most half of the screen (different from the tiers). This space is already wasted anyway:
Just toggle the attributes to easily filter. I think the overall point is that navigation should be interactive and easily filterable. (Not that implementing this is trivial to do.)
The list should either be sorted by use-count or split-count. That is, most popular attributes to least popular, or most dividing to least dividing. I can't think of which one is better.
Of course the vast majority of TASVideos movies was always the "Entertaining speedruns". For example, here's how the site movies were just yesterday (very roughly):
Now, Adelikat wants to publish many Atari runs, so it's promptly decided to make a separate category for "Unpopular speedruns". Also the Stars are going to be inflated, for an unexplained reason.
Then people jump the bandwagon and suggest allotting a tier for "Demos", because they care about such movies.
Do you imagine how ugly this "Tiers 2.0" system is gonna be? That's because there will be no real system, just lots of manual labour and subjective selections. But what concerns me the most, such division also hurts "Playaround" movies.
But I couldn't think about better system, until GMan's proposal.
This system is both unrestrictive and systematic up to the point that it can be automated. So here's my proposal.
1. When a movie is submitted, people vote Yes/No/Meh, estimating the future entertainment value of the movie.
2. Judge decides whether to publish the movie. If the estimated value is low then this movie can be published only if it's the fastest playthrough in the world. If the estimated value is high then the movie is published, even if the goal is esoteric or the movie is intentionally suboptimal (not poorly made, but not aimed on speed).
3. All movies are published in one tier. Stars and Moons (or whatever) remain their original meaning (hand-picked, about 5% of all movies on the site).
4. Visitors can browse either the full archive or separate parts (Playarounds/Speedruns/Records/Demos/Stars). While browsing it's possible to uncheck/recheck any of 5 options (e.g. in the standard header of every page, right near the Megaman sprite).
5. The division to 5 categories is automatic. All movies with "Playaround" flag go to Playarounds, "Demonstrations" go to Demos, Starred go to Stars, speedruns with entertainment rating less than precalculated value (say, 5.0) go to Records, everything else goes to Speedruns. Thus no manual labour is needed, plus an unpopular speedrun is not doomed forever, as it has a chance to go up (for example, AVGN reviews the game and people start upvoting the run).
Yes, the "checkboxes" idea is knda meh, and it probably should be reconsidered into something more casual. But the general direction of his idea is much more solid than previous proposals.
Just toggle the attributes to easily filter. I think the overall point is that navigation should be interactive and easily filterable. (Not that implementing this is trivial to do.)
The list should either be sorted by use-count or split-count. That is, most popular attributes to least popular, or most dividing to least dividing. I can't think of which one is better.
