Post subject: What is the point in submission polling?
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
As a rule judges completely ignore the votes if there's a robust discussion. For this submission, I really don't care what the votes say. I'm only going to be reading the posts, and those that seem valid (appears the poster actually watched the run etc...) I'll pay attention to. The rest are discarded.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Masterjun
He/Him
Site Developer, Skilled player (1987)
Joined: 10/12/2010
Posts: 1185
Location: Germany
HappyLee wrote:
None of where I slowed down looks like an oversight or mistake. Perhaps you're not paying attention to the details.
We don't require people to pay attention to the details to vote if they are entertained or not.
Warning: Might glitch to credits I will finish this ACE soon as possible (or will I?)
Player (42)
Joined: 12/27/2008
Posts: 873
Location: Germany
Essentially, if you spend months making a run and explaining how everything does, you should be open to people bashing it without knowing anything useful. But if you just spend 20 minutes watching a movie, then you are free to write and vote whatever you want for any reason, because it is not acceptable to say that their criticism is dumb. Youtube users that scream "Cheats!" at every TAS should find this policy very nice!
Site Admin, Skilled player (1251)
Joined: 4/17/2010
Posts: 11475
Location: Lake Char­gogg­a­gogg­man­chaugg­a­gogg­chau­bun­a­gung­a­maugg
p4wn3r wrote:
Essentially, if you spend months making a run and explaining how everything does, you should be open to people bashing it without knowing anything useful. But if you just spend 20 minutes watching a movie, then you are free to write and vote whatever you want for any reason, because it is not acceptable to say that their criticism is dumb. Youtube users that scream "Cheats!" at every TAS should find this policy very nice!
This post seems to be talking about policies. Which tasvideos policy does it criticize in particular, and what are the suggestions to fix or improve it?
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Player (42)
Joined: 12/27/2008
Posts: 873
Location: Germany
Now that you mention, an idea is to make the user names of all entertainment votes visible to all users in submission threads. It would certainly help mitigate abuses.
Site Admin, Skilled player (1251)
Joined: 4/17/2010
Posts: 11475
Location: Lake Char­gogg­a­gogg­man­chaugg­a­gogg­chau­bun­a­gung­a­maugg
To be honest, I don't know why we even still have polls. It's been mentioned a few times (not only in this thread), that when a movie is being judged, only posts that represent actual notion are considered. Silent votes are always disregarded. Because really, they don't prove anything at all, therefore we can't use them for anything when judging. What we care about is posts. And posts make it clear who dislikes or likes your movie. Also posts contain a whole lot of other potentially useful information, we share ideas and come up with solutions. Or argue, like ITT. I learned one important thing recently. Polls are for things that can be simplified to the point of stupidity. You don't have to invest any serious effort into brainstorming, looking for logical solutions, discussing, generating insights. You just voice your opinion in a simplified form. Since it's a simplified form, it's not helpful. Often we have complicated situations with tons of factors to consider, and it's not known beforehand if there is even a sensible solution. In order to resolve such open problems gracefully, a lot of mental effort is required. Sharing ideas, digging into non-obvious aspects, evaluating the potential outcomes, all sorts of crazy stuff. We are geeks, we love this, puzzle-solving is in our blood since the day we learned TASing. Do silent votes help with any of this? No way. Another problem, votes can be manipulated to provide desirable outcome. You just have to simplify the problem in people's eyes, and provide options that are based on that simplified problem. You can preset people's opinions if you're smart (and mean) enough. We don't want to rely on that kind of things. This is why we don't believe in silent votes officially. We only believe in helpful discussion. Submission votes are a legacy thing, maybe people want it to be there. I don't think removing its has ever been considered. But it's virtually non-existent when it comes to serious mental effort. tl;dr: don't mind silent votes, treat them as youtube comments. Because that's what judges do.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Skilled player (1671)
Joined: 7/1/2013
Posts: 447
p4wn3r wrote:
Now that you mention, an idea is to make the user names of all entertainment votes visible to all users in submission threads. It would certainly help mitigate abuses.
Agreed.
Player (42)
Joined: 12/27/2008
Posts: 873
Location: Germany
feos wrote:
Do silent votes help with any of this? No way. Another problem, votes can be manipulated to provide desirable outcome. You just have to simplify the problem in people's eyes, and provide options that are based on that simplified problem. You can preset people's opinions if you're smart (and mean) enough. We don't want to rely on that kind of things. This is why we don't believe in silent votes officially. We only believe in helpful discussion. Submission votes are a legacy thing, maybe people want it to be there. I don't think removing its has ever been considered. But it's virtually non-existent when it comes to serious mental effort. tl;dr: don't mind silent votes, treat them as youtube comments. Because that's what judges do.
I agree with that, but I still think the votes help a little bit. It's a fast way to check on the popularity of things. Realistically, not everybody will take their time to post. The votes are a heuristic more than anything. If something gathers lots of No votes, it might not be a reason to give it a lower tier, but at least it's reason to check out what's happening. There are many ways to manipulate votes and even what can constitute manipulation is debatable. I suspect many Yes votes in popular submissions come from the runner's friends or people who visit the site rarely and only post on their favorite game's threads. Can someone eventually see this as a group of people trying to manipulate the community's opinion? Perhaps, but we must also ask what's the problem with voting on submissions of a game you like or telling your friends "Hey, I made a run. Check it out!". In any case, all the problems could be solved by simply revealing the names behind the votes (Facebook reveals the name of people who liked something because it's so easy to manipulate). If that information is public, it is the community's problem if they fall for the manipulation. Also, in the case of a troll No/Meh vote, the thing a troll values most is anonymity. He just wants to click on a button and be done with it. However, if the name is public, he might have some explaining to do later, so I guess people would be inclined to vote No only if they truly believe it's important.
Personman
Other
Joined: 4/20/2008
Posts: 465
feos wrote:
To be honest, I don't know why we even still have polls.
I think this would be a great time to abolish voting altogether.
A warb degombs the brangy. Your gitch zanks and leils the warb.
Active player (312)
Joined: 2/28/2006
Posts: 2275
Location: Milky Way -> Earth -> Brazil
Yes, polls are necessary. "Meh" votes, on the other hand, have always been useless. I have a great idea though. You could make it so everyone who votes 'meh' gets a 1 day postblock, just so they learn that mediocrity never pays off. This should help them learn to be more enthusiastic about everything in life.
"Genuine self-esteem, however, consists not of causeless feelings, but of certain knowledge about yourself. It rests on the conviction that you — by your choices, effort and actions — have made yourself into the kind of person able to deal with reality. It is the conviction — based on the evidence of your own volitional functioning — that you are fundamentally able to succeed in life and, therefore, are deserving of that success." - Onkar Ghate
Bisqwit wrote:
Drama, too long, didn't read, lol.
Editor, Emulator Coder, Expert player (2156)
Joined: 5/22/2007
Posts: 1134
Location: Glitchvania
Kung Knut wrote:
For the record, my vote was "meh". ... And then it came, was watched with really high expectations, but did not quite live up to them.
It appears to me that you, maybe pirate_sephiroth and some others mistook what the "Meh" option is intended for in such a poll, which is, as I interpret, to allow voters to boycott the poll itself in a quick way, rather than to abbreviate the word "Mediocre". That is why the option is not put between "Yes" and "No", but on the last row. Perhaps we should change "Meh" to some clearer text like "I don't care" and provide a "Mediocre" option if we really desire it.
<klmz> it reminds me of that people used to keep quoting adelikat's IRC statements in the old good days <adelikat> no doubt <adelikat> klmz, they still do
Active player (312)
Joined: 2/28/2006
Posts: 2275
Location: Milky Way -> Earth -> Brazil
klmz wrote:
Kung Knut wrote:
For the record, my vote was "meh". ... And then it came, was watched with really high expectations, but did not quite live up to them.
It appears to me that you, maybe pirate_sephiroth and some others mistook what the "Meh" option is intended for in such a poll, which is, as I interpret, to allow voters to boycott the poll itself in a quick way, instead of abbreviating the word "Mediocre". That is why the option is not put between "Yes" and "No", but in the last row. Perhaps we should change "Meh" to some text like "I don't care" that is clearer, and provide a "Mediocre" option if we really desire it.
nope. the definition of boycot is "to refuse to buy a product or take part in an activity as a way of expressing strong disapproval" So if one wants to boycott a submission they would just comment without voting or simply ignore the thread completely. A 'meh' vote clearly displays undesirable behaviour. It's an obvious way to give mediocre people the illusion of making an important choice.
"Genuine self-esteem, however, consists not of causeless feelings, but of certain knowledge about yourself. It rests on the conviction that you — by your choices, effort and actions — have made yourself into the kind of person able to deal with reality. It is the conviction — based on the evidence of your own volitional functioning — that you are fundamentally able to succeed in life and, therefore, are deserving of that success." - Onkar Ghate
Bisqwit wrote:
Drama, too long, didn't read, lol.
Editor, Emulator Coder, Expert player (2156)
Joined: 5/22/2007
Posts: 1134
Location: Glitchvania
pirate_sephiroth wrote:
nope. the definition of boycot is "to refuse to buy a product or take part in an activity as a way of expressing strong disapproval" So if one wants to boycott a submission they would just comment without voting or simply ignore the thread completely.
"Meh" is not a logically sensible answer to the question "Did you find this movie entertaining?". By voting "Meh" the voter still abstains from the poll, just ironically using the same facility as the participants use.
<klmz> it reminds me of that people used to keep quoting adelikat's IRC statements in the old good days <adelikat> no doubt <adelikat> klmz, they still do
Active player (312)
Joined: 2/28/2006
Posts: 2275
Location: Milky Way -> Earth -> Brazil
klmz wrote:
pirate_sephiroth wrote:
nope. the definition of boycot is "to refuse to buy a product or take part in an activity as a way of expressing strong disapproval" So if one wants to boycott a submission they would just comment without voting or simply ignore the thread completely.
"Meh" is not a logically sensible answer to the question "Did you find this movie entertaining?". By voting "Meh" the voter still abstains from the poll, just ironically using the same facility as the participants use.
So if you don't like Iphones, you can buy one and smash it with a hammer. That's a valid form of boycotting, according to your logic.
"Genuine self-esteem, however, consists not of causeless feelings, but of certain knowledge about yourself. It rests on the conviction that you — by your choices, effort and actions — have made yourself into the kind of person able to deal with reality. It is the conviction — based on the evidence of your own volitional functioning — that you are fundamentally able to succeed in life and, therefore, are deserving of that success." - Onkar Ghate
Bisqwit wrote:
Drama, too long, didn't read, lol.
Memory
She/Her
Site Admin, Skilled player (1552)
Joined: 3/20/2014
Posts: 1765
Location: Dumpster
klmz wrote:
pirate_sephiroth wrote:
nope. the definition of boycot is "to refuse to buy a product or take part in an activity as a way of expressing strong disapproval" So if one wants to boycott a submission they would just comment without voting or simply ignore the thread completely.
"Meh" is not a logically sensible answer to the question "Did you find this movie entertaining?". By voting "Meh" the voter still abstains from the poll, just ironically using the same facility as the participants use.
I vote meh to say the movie was only somewhat entertaining. Not to say that I don't care. This is how most people I know use "meh" as well.
[16:36:31] <Mothrayas> I have to say this argument about robot drug usage is a lot more fun than whatever else we have been doing in the past two+ hours
[16:08:10] <BenLubar> a TAS is just the limit of a segmented speedrun as the segment length approaches zero
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3810)
Joined: 11/30/2014
Posts: 2828
Location: US
Personman wrote:
feos wrote:
To be honest, I don't know why we even still have polls.
I think this would be a great time to abolish voting altogether.
Personally I use the polls as a quick gauge of whether I should watch a run or not, so I guess it's not strictly a tool for judges. Although I could just as well use number of posts for the same purpose if the poll wasn't there. So I guess I agree. Also I don't like being asked a question only to be directly told that my response doesn't matter, might as well not even ask.
Editor, Skilled player (1438)
Joined: 3/31/2010
Posts: 2108
For what it's worth, I always viewed 'Meh' as an option between 'Yes' and 'No'. Maybe we should have a poll about it.
Editor, Emulator Coder, Expert player (2156)
Joined: 5/22/2007
Posts: 1134
Location: Glitchvania
pirate_sephiroth wrote:
So if you don't like Iphones, you can buy one and smash it with a hammer. That's a valid form of boycotting, according to your logic.
That is different. As soon as one buys an Iphone, the activity of the deal is done, and smashing the bought Iphone can't revert that. In the case of the polling here, nothing about the entertaiment aspect of the submission can be conclude from a "Meh" vote. The practical benefit in providing a "Meh" option in the polling is, in my opinion, to reduce the chance that some "too-lazy" people who want to demonstrate their willing for boycott may just misuse the meaningful options (most likely "No") for that. Unfortunately people seem to be misusing it as "Mediocre".
<klmz> it reminds me of that people used to keep quoting adelikat's IRC statements in the old good days <adelikat> no doubt <adelikat> klmz, they still do
Aran_Jaeger
He/Him
Banned User
Joined: 10/29/2014
Posts: 176
Location: Bavaria, Germany
There's a page that explains the voting ( http://tasvideos.org/VotingGuidelines.html ), and another page that explains submissions, so maybe some people in here might need to refresh on these again. From here ( http://tasvideos.org/Subs-521up.html ) you can find the meaning of the votes by TASVideos site's own definition/standards: [quote TASVideos] The voting data comes from the discussion forums. "65%" (6/1/3)" means 65% support, with 6 yes, 1 meh, and 3 no votes. Support is ( (yes + meh/2) / (yes+meh+no) ). To vote, click on the submission and then "Discuss this submission". [/quote] Even if you don't know or don't want it to mean that, this is what it effectively means. Feel free to look around and use the Article Index page link ( http://tasvideos.org/ArticleIndex.html ) to find information on what you want to know or what you need. Edit: In particular I want to point out the following: If we set the ''support'' function depending only anymore on the number of ''meh'' votes (with fixed number of ''yes'' and ''no'' votes) accordingly, we have f(m)=(y+(m/2))/(y+m+n). By L'Hopital's rule, this expression converges to a ''support'' value of 1/2 if the number of ''meh'' votes goes to +infinity. This means that ''meh'' votes decrease the ''support'' value and make it come closer to the value 1/2 in case that the situation started above, and in case that one started below 1/2, the ''meh'' votes increase the ''support'' value and make it come closer to 1/2. So for this matter, they aren't quite entirely without effect. In this context, 0 would be the worst possible ''support'' value and 1 the best, with 1/2 being perfectly in the middle, and I guess 1/2 would be corresponding to the most split situation to judge with respect to the votes and this ''support'' formula, if one would judge according to this. (Note that the situation is centered around 1/2 and not symmetrically around 0 with -1 and 1 as the extreme cases as one possibly could have expected.) 2nd edit: On a 2nd thought, if it was possible to apply a change to the formula or how it is expressed without too much effort, then I think I'd be for a change in it's representation, namely to center it around 0 instead (with limits -1/2 and +1/2, or -1 and 1 which though would double the function's range). And my reason for this would be that a layman or even someone that understands the formula (together with the meaning of the votes, how they play into it) but might not go ahead and analyse the formula, or people in general might be mislead to think that voting ''meh'' has a different (maybe positive) effect on the ''support'' value than what it actually does, going by the look of the formula. Or someone that sees the formula for the first time might not be aware of the effect of the ''meh'' votes to always make the ''support'' value move a step closer to the central value. Or maybe instead of changing the representation, one could put up an explanation next to the formula that goes into these aspects (if one wants). As an example: If the current situation is such that there is more ''yes'' votes than ''no'' votes, and someone wants to support a TAS submission (but maybe not by a lot with a ''yes'', and at most just a little), then one better refrains from voting ''meh'' instead of voting ''meh'', because it will lower the corresponding ''support'' value. However, if the current situation is such that there's more ''no'' votes than ''yes'' votes, then a ''meh'' vote will increase the ''support'' value.
collect, analyse, categorise. "Mathematics - When tool-assisted skills are just not enough" ;) Don't want to be taking up so much space adding to posts, but might be worth mentioning and letting others know for what games 1) already some TAS work has been done (ordered in decreasing amount, relative to a game completion) by me and 2) I am (in decreasing order) planning/considering to TAS them. Those would majorly be SNES games (if not, it will be indicated in the list) I'm focusing on. 1) Spanky's Quest; On the Ball/Cameltry; Musya; Super R-Type; Plok; Sutte Hakkun; The Wizard of Oz; Battletoads Doubledragon; Super Ghouls'n Ghosts; Firepower 2000; Brain Lord; Warios Woods; Super Turrican; The Humans. 2) Secret Command (SEGA); Star Force (NES); Hyperzone; Aladdin; R-Type 3; Power Blade 2 (NES); Super Turrican 2; First Samurai. (last updated: 18.03.2018)
Editor, Emulator Coder, Expert player (2156)
Joined: 5/22/2007
Posts: 1134
Location: Glitchvania
Aran Jaeger wrote:
Even if you don't know or don't want it to mean that, this is what it effectively means. Feel free to look around and use the Article Index page link ( http://tasvideos.org/ArticleIndex.html ) to find information on what you want to know or what you need.
FYI the source page is: Wiki: SystemSubmissionVotingHelp. The page only explains how the voting data are being used to calculate a "support" score, whose purpose hasn't earned consensus among the staff members. It doesn't explain what every voting option is intended for. It won't help with the fact that the term "Meh" means "I don't care" in daily language contradicts the usage for a "Mediocre" option, and I am pretty sure that there have been existing "Meh" votes used according to their literal meaning. Nonetheless, the method to gather "quick" feedbacks on submissions is never set in stone and some of us are talking about changing it, again, even right now.
<klmz> it reminds me of that people used to keep quoting adelikat's IRC statements in the old good days <adelikat> no doubt <adelikat> klmz, they still do
Memory
She/Her
Site Admin, Skilled player (1552)
Joined: 3/20/2014
Posts: 1765
Location: Dumpster
klmz wrote:
It won't help with the fact that the term "Meh" means "I don't care" in daily language contradicts the usage for a "Mediocre" option, and I am pretty sure that there have been existing "Meh" votes used according to their literal meaning.
A dictionary wrote:
adjective: not impressive : so-so a meh documentary
really not that out there of a definition The usefulness of votes as a whole is certainly not very much but there's nothing explicitly bad about the meh option in particular.
[16:36:31] <Mothrayas> I have to say this argument about robot drug usage is a lot more fun than whatever else we have been doing in the past two+ hours
[16:08:10] <BenLubar> a TAS is just the limit of a segmented speedrun as the segment length approaches zero
Aran_Jaeger
He/Him
Banned User
Joined: 10/29/2014
Posts: 176
Location: Bavaria, Germany
For anyone interested, here's the corresponding calculation to make this effect that I mentioned in my previous post visible and more understandable: Let's assume y > n, and we compare a (previous) situation with m ''meh'' votes, with its ''support'' value (y+m/2))/(y+m+n) to a (later) situation with m+1 ''meh'' votes that has the ''support'' value (y+(m+1)/2))/(y+m+1+n). Then the difference of the latter minus the former is given by (y+(m+1)/2))/(y+m+1+n) - (y+m/2))/(y+m+n) = ((y+m+n)*(y+(m+1)/2)-(y+m/2)*(y+m+1+n))/((y+m+1+n)*(y+m+n)). Here, the denominator is necessarily strictly positive (since the minimal number of ''no'' votes is 0 and we assumed y>n, so we have at least y=1 or larger; and if there's no ''yes'' aswell as no ''no'' votes at all, the whole calculation simplifies anyways). So the sign of this expression is determined by the numerator, so let's take out the denominator and check the numerator: (y+m+n)*(y+(m+1)/2) - (y+m/2)*(y+m+1+n) = (y+m+n)*(y+m/2) + (1/2)*(y+m+n) - (y+m/2)*(y+m+1+n) = (1/2)*(y+m+n) - (y+m/2) = (n-y)/2. Now since we assumed y>n, this expression is strictly negative which means the expression ''latter minus former'' is negative, so the previous ''support'' was larger than what the new resulting ''support'' value would be if one voted ''meh'' (instead of not voting), in the case that the number of ''yes'' votes is larger than the number of ''no'' votes (and if this is the other way around, the effect is flipped accordingly).
collect, analyse, categorise. "Mathematics - When tool-assisted skills are just not enough" ;) Don't want to be taking up so much space adding to posts, but might be worth mentioning and letting others know for what games 1) already some TAS work has been done (ordered in decreasing amount, relative to a game completion) by me and 2) I am (in decreasing order) planning/considering to TAS them. Those would majorly be SNES games (if not, it will be indicated in the list) I'm focusing on. 1) Spanky's Quest; On the Ball/Cameltry; Musya; Super R-Type; Plok; Sutte Hakkun; The Wizard of Oz; Battletoads Doubledragon; Super Ghouls'n Ghosts; Firepower 2000; Brain Lord; Warios Woods; Super Turrican; The Humans. 2) Secret Command (SEGA); Star Force (NES); Hyperzone; Aladdin; R-Type 3; Power Blade 2 (NES); Super Turrican 2; First Samurai. (last updated: 18.03.2018)
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
I would like to say that I've never intentionally1 voted "meh", as I'm not a wishy washy person. I know if I found something entertaining or not. I agree with pirate_sephiroth that one needs to approach everything in life with firmness and not mediocrity. As to why we still have a voting system, as I wrote originally: "As a rule judges completely ignore the votes if there's a robust discussion." Sometimes submissions don't have much of a discussion. In those cases I need insight into what people are thinking. Sometimes people vote without saying anything, I've even done that myself on occasion as I've felt other people in the thread mentioned anything I was thinking and I did not feel the need to be redundant. Even when there is discussion, when I browse submissions, I sometimes decide whether I want to watch or claim a run for judging based on how many votes I see were cast, as well as the direction they were leaning. So I won't say voting is useless. Just that for the judgement itself, a robust discussion supersedes them. ------------------------------ #1 - It's possible I've pressed the button by mistake at some point without realizing it.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Dimon12321
He/Him
Editor, Reviewer, Experienced player (595)
Joined: 4/5/2014
Posts: 1217
Location: Romania
Ignore everything! That discussion has gone far beyond the movie debrief. Just judge the movie and accept it if it fits all the criteria for being accepted. All I'm saying to you is, it can take very long time for the discussion to be concluded. You don't have to wait for it.
TASing is like making a film: only the best takes are shown in the final movie.
Player (146)
Joined: 7/16/2009
Posts: 686
Nach wrote:
I would like to say that I've never intentionally1 voted "meh", as I'm not a wishy washy person. I know if I found something entertaining or not.
That's just being silly. Entertainment isn't a binary thing. Sure, you could mentally rate everything 1-10, and then decide that everything below 5 is a "no", and everything else a "yes". But considering that "meh" means "so-so", is commonly used as "average" and is literally counted as half a "yes" vote, it's perfectly reasonable to say that some range around the middle constitutes a "meh" vote. That's not being wishy-washy or not knowing whether you found it entertaining: you can be perfectly decided on the entertainment being average.