Joined: 12/8/2012
Posts: 706
Location: Missouri, USA
Radiant wrote:
Well for starters, it's very easy to include vault tier when people list movies by platform. It's a URL that's part of our wiki (it currently goes NES-Star-Moons and if you change that to NES-Star-Moons-Vault it will include vault movies). Let's do that first.
Then, it strikes me that we could use some reorganizing. Regardless of what the original intent was, at present
Vault is for any% and 100% movies, except if they're entertaining
Moon is the default tier, since the majority of movies are in moon
Star is simply the highest-rated movies from moon (since to get added to star, it must have top ratings, and the lowest-rating movies are routinely removed from star)
...that probably wasn't the intent, but that's more-or-less how it turned out. It would help if we started clarifying what the intent actually is (and also, how the intent benefits from having "tiers").[/list]
I agree with the overall idea that it would be better to "tweak" (reorganize, update, etc.) a couple of things, here and there, concerning the Vault tier and the website.
Using an analogy, it would be like doing some handyman work/upgrading around a house vs. setting up a new foundation.
"But as it is written, Eye hath not seen, nor ear heard, neither have entered into the heart of man, the things which God hath prepared for them that love him." - 1 Corinthians 2:9
Warp is right. Star movies are hand-picked to represent certain genres, franchises and platforms, and to show off what a TAS can do (from what I understand).
I was browsing the Vault, and I'm not seeing a negative presentation - in fact, many of the runs are presented nicely with good descriptions and have over 10 votes (not at all ignored compared to the typical Moons movie). Howewer, there are a handful that are described with duller one-liners.
Regardless of changes or lack thereof to the tier system or front page presentation, that's something we can all edit to improve - a nice description makes any run that much more visible in the list, and makes a visitor more likely to watch it. I'm going to shamelessly plug [2119] DOS Avoid the Noid by turska & Ilari in 03:33.52 - the game might be obscure and on the wrong side of the entertainment fence, but the ridiculous premise of the game sells the movie even so, and it is not presented in a negative light at all.
Any user who's an Editor (ask your friendly neighborhood admin to become one) can edit movie descriptions. You can improve descriptions of your own movies that you think could be presented better if you're the author, too - I wrote most of the descriptions for TASes I've been involved in, for example, and you don't need to be staff to do this. Just ask to become an Editor and you can start making edits in the wiki.
Joined: 11/13/2006
Posts: 2793
Location: Northern California
I don't think the Vault itself presents the runs negatively. I think it's a step back from that: The site presents the Vault negatively. Several authors, myself included, see Vault as negative due to how the site presents it as the "lowest tier", and thus in some way we're discouraged from making runs for fear of having them end up there. I know that's not the intent, of course, but implications can have as much of an effect on people as something that's clearly stated and set in stone. Changing movie descriptions wouldn't help too much if people aren't looking at the runs in the first place. For example, how often do you think people look up Olympic bronze medalists out of genuine interest? They could have wonderfully written Wikipedia articles (there's an oxymoron) or insightful and inspiring articles/interviews, but they're still mostly going to be ignored since people generally want the best of the best.
EDIT: That being said, I'm all for giving movies insightful descriptions, especially if random runs are going to be featured on the front page.
TASvideos Admin and acting Senior Judge 💙 | CohostwarmCabin wrote:
You shouldn't need a degree in computer science to get into this hobby.
Ah, I always thought that not something I could do. I was thinking that Hydlide needs to be rewritten, so I will work on a text for that. I will also see if any of my other TASes need touching up, and I can also see if there other obscure games that I am familiar with that need touching up.
Samsara wrote:
I don't think the Vault itself presents the runs negatively. I think it's a step back from that: The site presents the Vault negatively.
For example, how often do you think people look up Olympic bronze medalists out of genuine interest? They could have wonderfully written Wikipedia articles (there's an oxymoron) or insightful and inspiring articles/interviews, but they're still mostly going to be ignored since people generally want the best of the best.
That's not going to change with any suggestion here barring "put them all together." It's just that instead of being put into X-tier, it'll be they won't get Y-tag, so people will still feel slighted and there'll still be segregation.
Star bellied sneetches and whantot.
Joined: 11/13/2006
Posts: 2793
Location: Northern California
The proposed new system would essentially be putting all the runs together, since it actually gives proper, set-in-stone criteria to Vault/Coins(?) and Moons and wouldn't leave anything up to opinion. An accepted run would be definitively Coins (by being specifically an any% or 100% run) or definitively Moons (by being a playaround or a "non-standard" category). The tiers are effectively equal, with awesome runs being contained in both, and more people being likely to check through both tiers instead of looking at Vault and thinking "Yeah, these runs are probably safe to ignore".
I'm not sure how anyone would feel slighted under the new system apart from not getting a Star, and even then I don't think anyone on the site is egotistical enough to demand a Star for their run.
TASvideos Admin and acting Senior Judge 💙 | CohostwarmCabin wrote:
You shouldn't need a degree in computer science to get into this hobby.
I'm going to put my vote in for going with feos' suggestion, basically not doing "tiers" but dividing movies based on the goals they have, with generic any%/100% getting one category and others, such as playaround/limitations/glitch aboidance/special goals getting the other. I didn't like tiers based on quality anyway, since that is subjective, and causes a lot of cases where the category is ambiguous.
I'm not sure how anyone would feel slighted under the new system apart from not getting a Star, and even then I don't think anyone on the site is egotistical enough to demand a Star for their run.
Oh come on, Hydlide totally deserves a star. :P
A few days ago, I was discussing on IRC with adelikat about the merits of the tier multipliers in player score calculations. That got me thinking about the feos/Warp plan, which would mess up the multipliers since it removes a tier level. So today I thought about it more and came up with an idea that will work with the plan, and I think is also an improvement.
I propose that this be added to the feos/Warp plan:
Change the calculation for the player score calculation to the following: O*max((r+15)/20*r^max(2.6-0.2*average_ratings/ratings,1)/sqrt[P],5)
The change is that M is changed to (r+15)/20. This changes the multiplier to be based on its rating rather than a tier so that the bonus or penalty multiplier is based on the TAS' merits rather than a placement by a judge, etc. The formula is a simplified version of 1+(r-5)/20 which means that 5, the middle, is neutral. Below 5 gets a penalty, and above a bonus, that gets larger as it goes further away from the middle. The maximum are 0.75 to 1.25 times the original value.
One advantage of this system is that it is fair to TASes that are star quality, but not chosen to be a star, since runs of the same rating give the same points. This eliminates tiers in that the star is now a tag for showcasing.
At the time of writing, the poll clearly indicates that the preferred option (by a large margin) is to keep the vault as-is. The second favorite option is to keep the concept of the vault intact, but maybe change its appearance a bit. Taking these two option together, there's a 70% majority that doesn't want the concept of the vault changed.
I'm getting the impression that this thread mostly consists of people strongly opposed to the vault posting repeatedly, while the majority of people are a-okay with the current situation, managed to express that in the poll, and abandoned the thread. My point is that it seems to me there is only a slight minority that sees the vault as something negative. Many authors (myself included) are happy our cookie-cutter game choices (as Nitrogenesis so eloquently put it) now have a place too.
I try to keep up with the workbench at all times, and from what I've seen, there's many authors that even aim for the vault. In fact, this should be true for all speed-TASes: aim for vault first (your run is accepted at all), moon or stars later (it looks good too).
I'd be okay with a name change (and I can't imagine the 13 people who voted "no change" are that attached to the current name), if only because vault doesn't really fit the theme. Other than that, if I were to recognize any stigma attached to the vault, my solution would be to stuff more into it. If it's the biggest tier, it's bound to be seen as the default tier.
- I don't like the name Vault.
- A safe logo for bad runs... really... #bringbacksmwpoll
- We have star movies that's dependent on votes and feos, moon movies depends on votes and the judge, and vault which depends on votes and judge. So what's the difference between vault and moon apart from "probably shittier" game?
I think you know my poll vote.
PhD in TASing 🎓 speedrun enthusiast ❤🚷🔥 white hat hacker ▓ black box tester ░ censorships and rules...
Alright, what do we have for now.
The most agreed option is that there should be 2 meta-categories, that can be definitely found out for any given run. Objectively. Then there should be a level above those that represents the best, cherry-picked runs. Depending on the size of those 3, we can leave them fit the current system code-wise: don't remove "tiers", maybe even keep the name, but rename some of them, and move runs around to settle forever.
Then, how it's judged.
Language: c
int JudgeMovie()
{
if (SubOptimal)
return Reject();
if (any% || 100%)
return Accept(Coins);
else
if (Boring)
return Reject();
else
return Accept(Moons);
}
How it's points are calculated: Whatever dunnius wrote, except that stars still need to give a higher score.
My question still remains:
- Should borderline + side-goal runs be accepted with hopes they are "entertaining enough'?
What will be good as well, if agreed about:
- Adding a NEW flag (shouting RATE ME on mouse hover) to attract more attention of raters.
- Option to hide away movies below a given rating. Say I hate runs that have 5-, someone hates all below 8, etc.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
I have one question, and it's related to the same issue as with the "official world record" tag proposed in the past:
If we indeed go with "any%/100% tier" and "everything-else tier", what kind of run exactly qualifies for which of those two tiers? And I'm thinking precisely about total control runs (and also reset abuse runs, but let's leave that one out of this, as it's irrelevant.)
Would the "normal" any% run be considered the default any% completion (that would go the first tier) and the total control run a "special" kind of run that goes to the new Moon tier? Or would it be the other way around?
In other words, what exactly is considered "any%" in terms of the new vault tier?
(I ask this because I'm assuming that only one "any%" run is eligible for the new "vault" tier, rather than a bunch of them. Anything else goes to the new "moon" tier as an alternative goal run, if it's well-received enough.)
Edit: I just realized that "only one any% run is eligible for vault, anything else goes to moon" almost makes it sound like the importance of the tiers has been reversed: Now it would be a privilege to get to the vault tier, while the moon tier is just a dumping ground for "everything else". Perhaps this is a good switcharound? :P
Adding a NEW flag (shouting RATE ME on mouse hover) to attract more attention of raters.
Without any statistics, people usually vote on submissions but publications. People who just want to enjoy videos won't register the site to make a vote. They watch the youtube encode and maybe leave a comment and/or rate the video.
I don't see what would change with a "flag".
PhD in TASing 🎓 speedrun enthusiast ❤🚷🔥 white hat hacker ▓ black box tester ░ censorships and rules...
Warp: the current way to do things won't change. It will still be "fastest possible!!!11 goes to any%, any entertainment tweaks that cost time go to side goals". But again, it doesn't mean we will make branch names depend on that, just because it'd mess everything up, since branches are way more complicated that it was believed some time ago.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Joined: 11/13/2006
Posts: 2793
Location: Northern California
Yes, let's continue trusting the poll results, where 14 people voted that the flawed system is absolutely perfect without even bothering to post their flawed reasoning. Let's just ignore the valid points brought up in the thread by the people who actually give enough of a damn to express an opinion on the matter.
TASvideos Admin and acting Senior Judge 💙 | CohostwarmCabin wrote:
You shouldn't need a degree in computer science to get into this hobby.
Yes, let's continue trusting the poll results, where 14 people voted that the flawed system is absolutely perfect without even bothering to post their flawed reasoning. Let's just ignore the valid points brought up in the thread by the people who actually give enough of a damn to express an opinion on the matter.
You sure like to say flawed, don't you?
I have no problem with the current system and I've seen nothing here that makes me reconsider that. The only reason to change I've seen so far is that some people feel like their feelings are hurt because their TASes of poor/uninteresting games don't get as much attention as more famous games or more entertaining TASes.
There's always going to be subjectivity in judging. There's always going to be changing standards and audience reception making one year's boring the next year's entertaining and vice versa. There's always going to be some kind of tagging or filtering separating runs people enjoy from ones people do not, ones for iconic games people know and love from ones that are little more than footnotes. Whether it's a tag that says "This is entertaining than other runs", or a little icon that's somewhat shinier than other icons, or what, that's not going to change and we'll just be right back here the next time someone's feelings are hurt because they didn't get what they wanted.
I have no problem with the current system and I've seen nothing here that makes me reconsider that. The only reason to change I've seen so far is that some people feel like their feelings are hurt because their TASes of poor/uninteresting games don't get as much attention as more famous games or more entertaining TASes.
The thing is: Is the current distinction between the two tiers useful. Could the distinction be improved and made more meaningful and useful?
There's always going to be subjectivity in judging.
Wouldn't it, thus, be an improvement to remove the subjectivity on the question of which tier a TAS belongs to?
Joined: 11/13/2006
Posts: 2793
Location: Northern California
You're probably not seeing any flaws with the current system because you've never actively looked for any flaws in it. Even if flaws don't come up often, the fact that there are still flaws is telling, especially when the flaws come up consistently. If there were odd, unrelated flaws popping up every now and then, this wouldn't be nearly as much of a problem. The flaws that do pop up, however, are almost always the same.
The only difference between Vault and Moons right now is subjectivity, since both tiers have arguably the exact same requirements apart from the entertainment of the game. This leads to borderline cases like Hydlide, where a judge's opinion may go against the public opinion for one reason or another (and props to turska for explaining his reasoning later on in that thread). The fact that tiering is purely subjective is inherently negative, since getting thrown in the Vault is basically being told that people don't like your run. Look back through any Vaulted run and count how many people specifically say things like "This wasn't entertaining, throw it in Vault" and then come back and tell me that Vault is positive. The site and the community actively paint the tier as low-quality and non-entertaining, which is... you guessed it, flawed.
If there's always going to be subjectivity, why not take actions to reduce it instead of leaving everything the way it is?
TASvideos Admin and acting Senior Judge 💙 | CohostwarmCabin wrote:
You shouldn't need a degree in computer science to get into this hobby.
I disagree with your (plural your) assertion that the existence of subjectivity is a flaw or that subjectivity is inherantly a negative that needs to be eliminated. I also find the remark that I'd agree with you if I'd just stop being willfully ignorant condescending.
I'll even go so far as to say that you're overblowing the amount of subjectivity there is. There are clearly common standards the community agrees on as to what is entertaining and what is not. It's not truly objective, but no judgement of anything in media is. Inter-subjective standards work just fine.
Warp,
I find the distinction broadly useful and I believe it will only become moreso as more runs are created. There are far more godawful games than good ones. And even many good ones can make for fairly hideous TASes. I don't personally agree with some of the choices, but I'm not the center of the world, so I just carry on regardless.
Joined: 11/13/2006
Posts: 2793
Location: Northern California
I just provided a possible reason why you might not have seen any issues with the system. I never implied that you'd agree with me if you did see any issues and I never implied you were ignorant for not seeing any.
TASvideos Admin and acting Senior Judge 💙 | CohostwarmCabin wrote:
You shouldn't need a degree in computer science to get into this hobby.
Joined: 11/13/2006
Posts: 2793
Location: Northern California
Scepheo wrote:
Samsara wrote:
The flaws that do pop up, however, are almost always the same.
Yes: people being overly butthurt about their run going to the vault.
The main problem is the consistent issue with borderline cases.. The issue with borderline cases is consistent enough of a problem that site staff are actually addressing it and coming up with ways to take care of it. As for the issue you brought up, even reading through this very thread will give you enough evidence that it's a legitimate issue, and another consistent one.
Again: Issues not appearing often doesn't mean the system is working just fine. If all the issues were completely unrelated and situational, then it wouldn't be as much of a problem. The fact that all the issues that arise are consistent and related is why the system needs to be re-examined like this.
Not everyone is affected by every problem, but every problem should be fixed.
EDIT: Reworded everything.
TASvideos Admin and acting Senior Judge 💙 | CohostwarmCabin wrote:
You shouldn't need a degree in computer science to get into this hobby.
Regarding poll results, if I understand correctly, it's long been site policy that poll results are only a small part of the decision making process, and the much more valuable data is the discussion connected to the polls.
