I thought my proposition is dedicated to minimize the infair use of rating powers. Let's say, I am a publisher and give everything 0-4 rating because of stupid things. But I have no tasing experience myself and my rates are insanely mood-dependent. My system would minimize my harm. Current can not.
Your system works if you assume that TASers (those with currently published movies in particular) are always fair, unbiased, and their votes never depend on mood, jealousy or other stupid things. Unfortunately, none of these assumptions are true. In all the significant cases of abuse on rating and voting systems thus far the culprits have been TASers. Yes, all of them.
In other words, your system solves one problem but introduces another.
feos wrote:
TAS experience IS a factor, and extremely important one.
I'm pretty sure I never argued otherwise; read the first post again.
Again, what I am against is stratification that would result from active players having significant weight advantage over the rest of the community. It is completely unneeded and only serves ego purposes. For instance, my own credibility in this community is pretty much entirely based on my forum posts. I know how to TAS, and some members here have seen the extent—but I'm easily bored/frustrated by the process and end up choosing not to spend my very limited time on it. I expect that there are many people in the same boat as myself, but if they'd been here long enough and seen enough movies (or made them long enough ago that they'd all gotten obsolete), why should they be put at such a disadvantage? Surely you can't say they can't fairly evaluate a movie?
It should be possible for everybody to eventually attain the same (or at least very similar) default rating weight experienced players have. It will be scaled back by personal vote distribution anyway.
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
Had rating been institutionalized the same way judging is, I would trust that Aglar is more qualified to evaluate a TAS. But that's why we have at most a dozen judges in a community of thousands, and judges have way more responsibility than normal users: they actually need to watch a lot of movies that other users can safely skip or postpone until much later, they need to properly explain their opinion when they judge, they are expected to be eloquent and moderate in their expressions. Granting benefits without such responsibility would lead to elitism and unfair stratification.
Right now anybody here, even a freshly registered user, can make a clever point and deliver it better than players with dozens of published submissions under their belt. It is important that it remains that way both on the forum and in the rating, but forum posting is a system based on free expression that makes it nearly impervious to abuse. In other words, you can't persuade people verbally without certain effort. Rating system, however, is impersonal and requires no conscious analysis on both ends. The intention of the changes I am proposing aren't to penalize some users and/or give advantage to others, but to make the calculation self-regulating and better protected from low participation issues as well as perfunctory/malevolent behavior.
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
I'm strictly against uncapped weighing, as it provides some people with unreasonable swaying power. In my opinion the difference between lowest unpenalized class's and the reference class's rating weight should never exceed 3:1 or so.
Also, I think post count or registration date shouldn't count towards weight upon reaching some sensible threshold (100 posts/1 year respectively, or something like that).
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
A TAS has clear entertainment potential when:
1) it's doing something unexpected, and decisions the player makes aren't obvious;
2) it looks significantly sharper than even the best examples of unassisted play of said game;
3) something interesting happens all the time with little downtime;
4) there is variety in gameplay;
5) you can relate to the gameplay somehow, or at least understand what is being done at a given moment.
That should pretty much cover all of the major reasons for rejection, btw. Rhythm games completely fail #1, #2 and #4, most puzzle games have trouble with #2 and #4, most autoscrollers commonly fail #1 and #3, RPGs' main problem is #3 and #5.
Platformers with complex movement (later Castlevania games, Metroid series, later Mario games, Sonic series, DKC series, Gimmick, Cave Story) or complex glitches (most of the Megaman series, SMB2, SMW, Sonic series again) are somewhat of a privileged genre because they rarely have problem with any of these criteria. Down to business, so to say.
TheNewTeddy wrote:
If I call them out, would they not be offended?
They won't, because they need to be accountable for their decisions if they are to remain judges in the first place.
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
Cutsceneless encodes have been posted numerous times for several games, including Super Metroid, Metroid Fusion, Kirby's Dream Course, Resident Evil, and probably others I don't remember. At no point they were official publication encodes. I don't see how this is going to be any different.
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
Actually, I still think scoring makes this game an order of magnitude more spectacular because, instead of going from A to B, more exciting route variation can be seen. But meh, beggars can't be choosers.
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
On another note, for a while I've been thinking that votes should be separated into votes of people that have played the game before versus people that haven't. I've seen videos on the site that are voted low, but I still have gotten a lot of enjoyment out of them since I'm familiar with the game. Also, It'd be nice to know what videos are entertaining to watch even if you haven't played the game.
I like this idea. Separation definitely won't happen for the reason goofydylan8 stated above, but just for the sake of interesting statistics I would like such a tick to be implemented someday. For many people this would require going over their rating though, so it should be at the very least accessible from the personal rating page, allowing to select multiple entries at a time.
goofydylan8 wrote:
I also agree with point 3, but less so than number 1. I disagree with this point because there are many members of this site who contribute greatly and have a good eye for quality but do not have a published TAS.
You misunderstood. Currently the staff members have a fixed value of 1 by default; in other words, they are the reference group for the rating system. What I propose here is shifting the reference from the staff to the experienced players, which won't hurt either as there is a significant overlap between the two. For those with no published movies this measure alone will make no difference whatsoever.
feos wrote:
I believe giving only high ratings has nothing to do with weight of the vote. Someone may really care only about awesome runs and rate them only. But have a good eye on everything. I mean, he doesn't want to give low rates to the movie he can't watch.
When combined with low participation, this general approach skews the rating of already popular movies towards the upper limit so much it makes it completely pointless, and that is the effect I would like to avoid in the first place. If it's only the already high-rated movies you care about, they can do just as well without the reinforcement that is your vote.
feos wrote:
What we reay need to do I think is improving our voting guidelines, reminding the important factors that help to give objective rates. See my page.
[...]
PS: The rating shall not ever depend on mood, only on real feeling about the amount of the work and on the resulting impression. And if someone votes by mood only, we shall work out a method to minimize that abuse. That;s why we shall set different weight for differently scored players.
The problem with this is that no matter what you think people "shall" be doing, they will still do as they please; they already do that with the current system anyway. Instead of trying to influence that, it's better to adapt the system in a way that accounts for that.
Warp wrote:
If this were implemented and public knowledge, it would be trivial to abuse: Just give 10/10 to a dozen of movies you like, and 0/0 to a dozen random movies. You have a couple of dozen rates (more than enough to get a full weight) and a full spread. In fact, this makes it only worse because now some innocent movies are being thrown some zeros they don't deserve, just to abuse the system.
Fair point, but don't forget that the overall number of votes also factors into weight distribution. As such one would have to rate a lot of movies with zeros to maximize their weight, and that shouldn't be hard to track. You're probably right that the exact formula shouldn't be public knowledge, though.
Warp wrote:
Could you be a bit more specific what exactly makes the current design so much of a "hassle"?
If you haven't noticed, the site has been awfully slow. Sometimes—more often than not when I actually need something—it may take several minutes to load the page over the countless "internal server errors".
Patashu wrote:
There has been a recent thread about movie rating that was about its visual representation and interface. This thread is about calculating the values.
As of today, a movie's rating is calculated as an average of the individual users' rating votes. Furthermore, each user's vote is algorithmically assigned with certain weight values based on the three-level scale:
— staff members' votes have a weight fixed to 1;
— banned users and users who have repeatedly abused the system have a weight fixed to a very low value;
— normal users (players, watchers, editors).
Normal users have additional cumulative criteria that factor into the final weight, driving it up to 1 as well:
— having a published movie (firsthand experience counts);
— number of posts (involvement with community counts too).
This system gets the job done, but doesn't cope very well with low vote amounts and heavy bias (in other words, fanboy and troll votes). So, let's gather improvement ideas that would make our movie rating less prone to abuse and more favorable towards seasoned critics, experienced players, and other TAS connoisseurs. Hopefully this will make somebody else than math nerds happier in some way. :D
I propose elaborating on the principle of weighing.
1. Ratings could factor in the mean values of each voter's personal rating history and the amount of movies watched (you can see these bits of statistics right now in our forum profiles under the "How this user rates movies"). User A who watches and rates a lot of movies and assigns to them the full spectrum of votes from 0 to 10 is expected to have a better idea of a movie's merits than user B who only votes on a few selected movies and tends towards giving out 10/10 indiscriminately. This doesn't mean that user B's 10/10 vote will drag the rating down as if it were, say, 5/5, but it does mean this vote will be more easily offset by a lower vote when calculating the average.
This measure alone will rather efficiently deal with lurkers registered to namevote a handful of familiar games or TASers, as well as diminish other kinds of abuse, since one would have to involve themself with the community and acquire themself with different kinds of movies in order to give more substance to their votes.
2. Authors' votes for their own movies, at least on the entertainment side, should have considerably lower weight due to the heavy bias involved. This bias is normal and expected; usually players have a lot of fun working on a TAS, which translates into the vote if they decide to cast it, but this entertainment is largely based on the process of TASing and not purely the result as would be expected from the rest of the viewers.
3. Instead of assigning the highest weight to staff members by default, it would seem fairer to do that to highly experienced players, as defined by player points (perhaps 1000 and above?). People with expertise in backend coding or publishing can't be expected to have the same, let alone higher, expertise in TASes as players actively making them. Besides, most of the high-rank players are judges anyway.
What are your ideas?
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
First of all, you have moderator privileges on this board so you can edit posts. Or you can tell adelikat to do that instead of vague hinting. But if all active developers know about this thread, have accounts here, and the forum's subscription system is still working, I don't know why you expect people to go out of their way and register someplace else just to ask a question or report an issue in the first place.
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
Now, now, let's keep it civil.
rog, you may want to familiarize yourself with nfq's style of posting which this conversation is a typical example of. In short, he's the resident wacko. :D
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
That's why it's called a "desktop replacement". You want to play catch with it or check your email in a field? Then this isn't the laptop for you.
Of course it isn't for me, although for completely different reasons (your examples are pretty funny though).
Still, that's not the question. The question is why "replace" a desktop with something that doesn't work quite as well and isn't as comfortable to use for things you do at home. You can assemble a better desktop for <1000$ easily.
Bobo the King wrote:
what don't you like about the screen?
Its matrix, which, going by numerous reviews, is a poor quality TN with horrible viewing angles. Resolution isn't everything. I would never get a TN screen for a computer I use most of the time.
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
Huge and heavy as fuck, low battery life, heat issues, rather bad screen even for a laptop. 5/10.
I never considered sacrificing comfort and proper ventilation for very limited portability to be worth it. This computer looks and sounds like it should have been a desktop.
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
An oxymoron, which becomes clear when you figure out what makes a rating reasonable in the first place.
rog wrote:
They have every right to criticize the run, but that's not what's going on here.
rog wrote:
his technical rating is appropriate
rog wrote:
no one would have
I thought I'd told you to stop deciding for other people and questioning their motives, yet you had decided to continue with twice the intensity. Clearly I have overestimated your intelligence. This discussion is indeed a waste of time.
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
beforehand or without knowledge, thought, or reason.
Are not true of this situation.
completely different pieces of work.
Completely different? Give me a break. It's the same game and same goal as a previous run. They are in fact quite similar.
You can't know how similar they are without watching them. There was an SM64 submission improving upon the previous one by three frames, but camera angles were so different (in a bad way) some people strongly felt against publishing it. Same game, same goal, pretty much the same time—apparently this knowledge isn't as relevant as you suggest, and no reason can help foresee certain author decisions that may or may not make an otherwise similar run either mindblowingly good or mindblowingly nauseating.
rog wrote:
You don't need to keep your ratings secret if they aren't entirely unreasonable.
I'm sure you're intelligent enough to understand the concept of privacy, and consequently the fact that people don't like sharing information about themselves and their preferences to various degrees. It's their absolute right not to, unless the otherwise is explicitly required. Most forms of invasion upon that right is considered a criminal offense in developed societies. You can speculate on the reasons and jump to conclusions all you want, but calling people out and bullying them on that basis will not be tolerated.
rog wrote:
And if anyone is a victim, it's the one getting multiple ratings 0 ratings based on things that have nothing to do with the run.
There you go again, deciding what things the ratings were based upon. This attitude is very annoying and tiresome, it won't get you or anybody else far on this forum.
maxx wrote:
i hope i'm entitled to my opinion, even if you disagree with it.
Yes, you absolutely are, and I will protect your right to be entitled to your opinion as long as the opinion stays politely expressed and on-topic. If you so wish, go to the off-topic section and create a new thread where you explain how much of an awful poster I am. You can knock yourself out there, but another such post here will be forcibly removed from the thread. I hope the message is clear.
Abahbob wrote:
It's not holding a prejudice. If you put your hand in a fire and see it's hot, you would assume other fires are hot. By what you are saying, people need to stick their hand into every fire they see to make sure it is hot.
You should consider thinking this point through; it's helpless to the point of being insulting to address.
Fire is hot enough to damage anyone and can't be cold due to its physics; the phenomenon is well-researched and burning temperature of any matter can be estimated easily and precisely. A Zelda run can be anywhere on the entertainment spectrum, will not appeal the same to different people, and there is no research that could tell you its entertaining value beforehand. Different Zelda runs may contain different events or tricks or route decisions that may affect entertainment value and can't precisely be known beforehand.
Warp wrote:
No. People are free to vote whatever they like, but the rating system should not be used to protest site politics and judging decisions. The forums should be used for that, not the rating system.
I believe that you understand the distinction but are nitpicking just because.
Yes, I understand the distinction. What I don't understand is the criteria you use to classify Saethori's or Cardboard's rating of this TAS as a protest against site policies or judging decisions. So far both have said they'd found the run extremely unentertaining but had to finish watching to be consistent with their votes. Both have given the courtesy of explaining their views publicly even though they were not obligated to, and the explanation had nothing to do with any kind of protest against the site.
(I haven't followed the submission voting and have not touched the subject; I've been only talking about rating votes here.)
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
Thank you, but the word was exactly "prejudice".
prej·u·dice [prej-uh-dis]
prej·u·diced, prej·u·dic·ing.
noun
1. an unfavorable opinion or feeling formed beforehand or without knowledge, thought, or reason.
2. any preconceived opinion or feeling, either favorable or unfavorable.rog wrote:
he must have known he'd hate any other run of the game.
In other words, by your suggestion he should have formed a preconceived unfavorable opinion based on completely different pieces of work.
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
People can vote each run the way they feel like, but in this particular case it is rather obvious, that most votes are not based on quality of the movie, but all the trouble surrounding it. The rating system should not be abused for some kind of protest, which clearly happened here.
So people can vote the way they feel like, but when they express extreme dissatisfaction it becomes abuse?
MrGrunz wrote:
You clearly stated earlier on, how much you hated every single OoT TAS, so why are you still watching them and even bother judging them with extrem low ratings? It somehow confuses me.
Let me help you there: he doesn't know in advance whether he'll like it or not.
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
OoT, SM64, SMB3, SMW and a handful other games have such a holy cow status that every time when somebody considers a TAS of any of them extremely unentertaining—and it's not exactly hard to see why—the fans feel the need to defend it and invalidate the "unacceptable" opinion. The act of which, for me, is a load of horseshit.
Zeupar wrote:
I think both Saethori and Cardboard used the entertainment part of the rating system to evaluate Swordless' personality instead of the run, which I didn't expect from any of them. One seems to try to hide this fact with rhetoric and the other didn't even bother explaining his clearly unfair rating.
Rhetoric such as yours would be expected from somebody who just can't see why this would not at all be an entertaining TAS, even though I'm well aware that you're not a mindless fanboy. Still, you shouldn't assume ulterior motives. People are different.
rog wrote:
You should only watch and rate runs that you think you'll rate higher than at least 1 or so, yes.
What other people should or shouldn't do is not up for you to decide. This "I know better" kind of attitude gets on my bad side all too easily these days; soon I'll stop asking nicely.
MrGrunz wrote:
Why don't you simply do the same, Cardboard?
Because he (and Saethori, for that matter) chooses not to. Some people like surprises, so they subject themselves to things that don't always promise entertainment. Some are completionists, some are just omnivores. It's not your nor anybody else's business how they choose to spend their time.
Saethori wrote:
...Which, in turn, means the ratings currently up do not reflect my honest view of the video. At this point, I would rather put up false ratings than get harassed further.
Which is upsetting at the very least. I try not to waddle through the mess that is OoT threads, but it seems I now have to devote more time to it to shoot down attempts to boss other people around; reading these are very similar in spirit to watching unentertaining movies rather than stopping midways or avoiding them outright.
I agree that Swordless had a moral right to know the reason for the low(er) tech rating, but the discussion shouldn't have escalated much further, definitely not to this point.
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
