Yeah, I'm familiar with this. Though hiding the voting results certainly won't help, since one could still conform to the posts that were made. And the votes are hidden unless you press "show votes" anyway.
I've considered 2 of the options
"No poll. But add "post <vote_type> post" buttons to where the poll was"
AND
"No poll, but enable rating for submissions like it is for publications"
I'm not really in favour of judges secretly knowing votes or ratings because it will no doubt lead to power abuse or information that could be shared with other members of the site who aren't judges, it's also an excuse not to read properly through a submission thread.
I voted for the second option I listed. Mainly because I'd be interested in seeing it trialled. I will admit that it could be abused with 0's because you don't like the game or the author and 10's for vice versa, or because you don't know any better. Hopefully most people on the forum will be mature enough and give sensible ratings.
Joined: 11/18/2006
Posts: 2426
Location: Back where I belong
AKA wrote:
I will admit that it could be abused with 0's because you don't like the game or the author and 10's for vice versa, or because you don't know any better. Hopefully most people on the forum will be mature enough and give sensible ratings.
Or people could give unusually low scores because the author didn't aim for fastest time, and instead aimed for entertainment.
I told Baxter I would make a post before I went to sleep, and even though i'm very tired and sick here it is.
First of all, people seem to think that I'm completely against all sort of voting type system. This isn't true. I just firmly believe that the former system was not doing it's job, and was in fact becoming a hindrance. I am more than open to discussion to get a better system, but I do agree with Bisqwit that nothing would have gotten accomplished if we had just started with discussion. Rocking the boat is sometimes the best way to get things changed.
For the people who argue that there is nothing wrong with the former system:
It was originally added to help judged sort through a list of over one hundred submissions to see which they should read/watch first. Since then it has grown into some sort of beast that people argue over, and think actually matters even though they are often reminded it doesn't. I think it's safe to say that at least half the posts in the submission forum now adays is either discussing/complaining about something related to the voting system OR a post just stating what someone voted. (Such as "I voted yes", or "First yes vote. I'll watch it now"). Neither of those types of posts are of any help at all. The posts stating how someone voted, are even less helpful because there is a poll in the thread already telling judges the totals. They just take up space.
Ok, some of the judges complain that they don't want to go through all the posts because there are so many of them. However, they don't seem to remember that if all the posts that were related to the voting system were removed from the submission forum, the total postcount would go down between 50-75%. There would be far less to read. This is a huge part of my desire to remove the voting system. It would remove pages of trash posts that come along with it.
On the topic of the judges are complaining about having too much to read. You have no room to complain. This is what you signed up for. YOU are the quality control, not the people on the forum. YOU are judges because you are suppose to have a good eye and watch the runs and take up questionable TASing with the author. YOU are suppose to sift through the posts to see if the there is something wrong with the run in the posts. YOU are the hidden face of the front page and what gets published. YOU are the 'voice' of Bisqwit. Why do you think there have been so few people to do it? It's a lot of work and Bisqwit puts a lot of trust in those people. If you don't like this, or think it's too much work, maybe you should reconsider your desire to be a judge.
For the voting systems that are currently being voted for:
Just adding "Yes/No/Meh" post buttons won't solve the garbage post problem. It will make it much, much worse. There will be floods of people who just post "I vote yes !!!!!!!" to get past length filters. The idea here is to make the forum more lean, not more spammy.
Just hiding the vote count until the submission is judged is just as bad of a solution. It will only increase the amount of junk posts that go in the submission forum, when everyone and their dog starts to feel it necessary to let everyone know how they voted. In a three word post.
Ratings, assuming they are just an average of everyone's ratings is a slightly better solution. It's already implemented in the site, and can be integrated in, which is very nice. However, it does suffer slightly from the same problem as above. I don't think as many people would make posts just to give their rating scores, but it still might happen. I think we would need a rule trying to forbid this. POSSIBLY don't allow people post their ratings at all, they can have them shown when the run is published.
This said, in my perfect world, as infeasible as it would be, there would only be posting. However, all posting would be directly about the movie. Basicaly asking questions and providing improvement suggestions. There would ideally be a bot similar to the ones in those chatrooms that ban people who say a sentence that has been said before. No repeat posts, only raw content. Praise and such can be done elsewhere mostly likely in the game related post where WIPs are posted. Granted, this wouldn't ever happen, and would upset a lot of people if it did. I'm just writing it here so people get some reference if they think I'm trying to be some sort of monster for suggesting removing the voting system.
My reaction to this thread:
Overall I'm glad there has been at least some helpful discussion and ideas to come out of it. However, it also reminds me why this site isn't a democracy. Certain people have been extremely childish, and quite frankly don't deserve a vote in this. There is a reason why companies don't give control of operations to everyone in the company. I would have liked to see someone submit something major so we could have seen a thread in action without the voting system, but from people's reactions, and someone suggesting to me that people are avoiding submitting for that reason, I unfortunately don't see that happening.
That's it for now, I'm sure I've forgotten a lot of what I was going to say because of sickness/tiredness, but I'll post more when I remember it. I greatly encourage people to read this post and discuss my points further with me.
Joined: 11/18/2006
Posts: 2426
Location: Back where I belong
BoltR wrote:
On the topic of the judges are complaining about having too much to read. You have no room to complain. This is what you signed up for. YOU are the quality control, not the people on the forum. YOU are judges because you are suppose to have a good eye and watch the runs and take up questionable TASing with the author. YOU are suppose to sift through the posts to see if the there is something wrong with the run in the posts. YOU are the hidden face of the front page and what gets published. YOU are the 'voice' of Bisqwit. Why do you think there have been so few people to do it? It's a lot of work and Bisqwit puts a lot of trust in those people. If you don't like this, or think it's too much work, maybe you should reconsider your desire to be a judge.
Just because I can live without something does not mean that I should want to live without it. I like having a dishwasher so that I don't have to do dishes by hand. I like having a car so that I don't have to walk everywhere. I like having votes to prioritize my time spent watching movies. Votes are a convenience. Votes are helpful to me. And that is why I like having some sort of poll around. It makes life easier for me.
I also dislike having you up on your soapbox tell me and others how to judge, when you were (and continue to be) very inactive in that roll. Since the Wiki based system was set up, you have 65 edits, very few (<5) of which happen to be judgements. Just because you have the title doesn't make you an active participant. You want to initiate change. I'm not against that. But please, if you're going to suggest a change in a system that you don't use, at least get some opinions from people who do use it before suggesting it be removed entirely.
I am glad that this has gotten more people talking and passionate about the state of the site. However, I think that it could have been implemented in a much better way.
First of all, people seem to think that I'm completely against all sort of voting type system. This isn't true. I just firmly believe that the former system was not doing it's job, and was in fact becoming a hindrance. I am more than open to discussion to get a better system, but I do agree with Bisqwit that nothing would have gotten accomplished if we had just started with discussion. Rocking the boat is sometimes the best way to get things changed.
Perhaps this is true... still don't know if I agree with it though, it seemed kinda sudden, but I guess it indeed was the best way to get things changed. I do hope whatever change is determined to be best, that it doesn't take a long time to be implemented... since I think the majority doesn't like the current state of having nothing.
BoltR wrote:
It was originally added to help judged sort through a list of over one hundred submissions to see which they should read/watch first.
Even if it lost its original purpose, it could still be something people got used to, and started to like for other reasons.
BoltR wrote:
Since then it has grown into some sort of beast that people argue over, and think actually matters even though they are often reminded it doesn't. I think it's safe to say that at least half the posts in the submission forum now adays is either discussing/complaining about something related to the voting system(1) OR a post just stating what someone voted(2). (Such as "I voted yes", or "First yes vote. I'll watch it now"). Neither of those types of posts are of any help at all.
(1) Posts debating the votes:
- I think posts wondering why people voted in a certain way are perfectly legit, and very much on topic of discussing the run. I agree that posts like "OMG, 10 yes votes already" or "a yes votes 2 minutes after the run is submitted" aren't useful at all, but I don't think the majority of the posts is like this. Discussing the reasons for what was voted, and if the votes should indicate publication or rejection seems perfectly fine to me.
Also:BoltR wrote:
The posts stating how someone voted, are even less helpful because there is a poll in the thread already telling judges the totals. They just take up space.
(2)Posts stating a personal vote:
- I don't see any problem with this. If it states a no-vote, than it's perfect, since especially those need posts attached to them. If it states a yes-vote, then it's fine too I think. People might know the author, and want him to know that they personally enjoyed the run, rather than being just one of the pile of ananymous yes votes. Most of the "I voted yes"-posts contain something like "great movie, yes vote", which gives the author of the run a feeling that his hard work is being appreciated, I think this is a good thing (ok... preferably the reasons why it is a great run should be stated... but I think even without it, it's fine).
I'm not gonna argue against your comments about judges... they should do that for themselves if they feel like it. But I will say that I can imagine if I were a judge that I would be interested in the voting results, even if I had watched the movie, and read all the forum posts.
BoltR wrote:
Just adding "Yes/No/Meh" post buttons won't solve the garbage post problem. It will make it much, much worse. There will be floods of people who just post "I vote yes !!!!!!!" to get past length filters. The idea here is to make the forum more lean, not more spammy.
Well, like I said, I disagree that it's spammy, for reasons stated above. I am however also against it, since some people just want to give their support for a run being either published or not, without being the 10th person in a row to post "great run, yes vote" (note that I'm not saying that would be spammy, I'm saying some people might not like to do it).
BoltR wrote:
Just hiding the vote count until the submission is judged is just as bad of a solution. It will only increase the amount of junk posts that go in the submission forum, when everyone and their dog starts to feel it necessary to let everyone know how they voted. In a three word post.
Well, without a voting system, people might just make such posts what they would have voted, had there been a voting system :P. Either way, I'm also very much against this option, for lots of reasons I stated in previous posts.
BoltR wrote:
Ratings, assuming they are just an average of everyone's ratings is a slightly better solution. It's already implemented in the site, and can be integrated in, which is very nice. However, it does suffer slightly from the same problem as above. I don't think as many people would make posts just to give their rating scores, but it still might happen. I think we would need a rule trying to forbid this. POSSIBLY don't allow people post their ratings at all, they can have them shown when the run is published.
Well, obviously I disagree with your statement that it's a bad thing if people would start posting their ratings (although obviously better with explanation of course). And you have to agree, someone just saying "8 entertainment, 5 technical", is more informative of someone just saying "yes vote". I also think this system encourages voting, which will be a good thing for the statistics (more ratings = better). I personally wouldn't mind these votes being public before publication, as it's nice to see what people voted (two 7's or a 5 and a 9 are quite different)... but I know this will never be done... so I'm just in favor of an average, and the amount of voters being public before publishing).
BoltR wrote:
This said, in my perfect world, as infeasible as it would be, there would only be posting. However, all posting would be directly about the movie. Basicaly asking questions and providing improvement suggestions. There would ideally be a bot similar to the ones in those chatrooms that ban people who say a sentence that has been said before. No repeat posts, only raw content. Praise and such can be done elsewhere mostly likely in the game related post where WIPs are posted.
I really, really disagree with this. This TAS authors spend lots and lots of their free time making these movies, and the only thing they can possibly get in return is appreciation from the people who watch it...
I know the argument that you should TAS because it's fun, and sometimes it indeed is... but sometimes, to get great results, endless and endless boring tests need to be done, which certainly isn't always fun. Knowing that it will produce a great result, and that people will appreciate it is a big motivation to pull through.
BoltR wrote:
Certain people have been extremely childish, and quite frankly don't deserve a vote in this.
Well, I hope that, just like the publication of movies, the votes are just an indication, and that the decision will be made on who is giving the best arguments, and not what has the most votes.
The 'soapboxing' was partially a way to see some of the newer judges reactions. I'm glad you have enough passion to stand up to it, even if my being inactive doesn't change the fact that it's true. That said, I don't try to hide the fact that I haven't been active in quite some time. Between finishing university, moving multiple times, friends, and now a job that I sometimes work in excess of 55 hours a week my free time has gone down greatly. I rarely have large chunks of time where I can watch, read, capture, and encode movies. That said, I also haven't hid the fact that I plan on doing it again once I get more free time. I've been rewriting my automated scripts for encoding as an example.
Also, while I may not post a lot or edit many pages, that doesn't mean I don't keep up with the site. I haven't ever really posted often.
As for the changing things without discussion. As I said in IRC, I was surprised Bisqwit acted on it so quickly. I bounced the idea off of him to see what he though about it, and it was done. Though as with Bisqwit, I don't regret or apologise that it happened. I don't personally think that anything would have gotten accomplished if he hadn't done it.
For the "I shouldn't have to deal with things that make it easier for me." I don't disagree. Believe it or not the goal wasn't to make it harder for judges. My original goal was to try and clean up all the garbage posting that was going on and try and make the submission forum more useful/helpful. Technically something like banning all discussion directly related to the voting process would have done this, but that is laughably infeasible.
Note: As I've told you in IRC Baxter, i'll put up a reply to your post tommorow, I really need to get to sleep.
I'm still a bit confused as to what was so wrong with the original system. As long as there are forums, people will bicker about something, votes or no votes. How exactly does removing the voting system solve the problem?
People will still make posts saying "I liked this video, yay."
People will still make posts saying they didn't like it without saying why.
People will still argue over whether a movie should or shouldn't be liked, or published, etc.
I don't see how the voting system's removal avoids any of these problems... you've only taken away the helpfulness it provides to judges and the sense of meaningful contribution it provides to the viewers who felt like their opinion on a movie had some worth or merit here. It seems like you're basically saying that whether or not I enjoyed a video isn't really relevant if I can't write out an essay describing it.
BoltR wrote:
The posts stating how someone voted, are even less helpful because there is a poll in the thread already telling judges the totals.
So if there's a voting system, the judges will ignore our posts and only look at the numbers? Forgive my ignorance, but where has this been a problem?
---
Eh, I don't think it's bad to try new things. It's basic human nature that it's always easier to convince people to keep doing the same thing than it is to convince them to try something different, so I see no harm in pulling it first and apologizing later. The TASes aren't going anywhere, after all.
And I'm not a judge or a runner, so maybe if it's the belief of the powers that be that my vote shouldn't be worth anything, I can respect that, I'm just lucky to be here. Still, I haven't seen any real rebuttal answering my above concerns. I'm all ears if anyone thinks there's good reasons, though.
Voted "Add a text urging one to post", although "vote buttons" seems to be much more realistic option for our lazy community.
Honestly, do judges really need any statistics other than number of posts in submission topic?
This "number" is enough to form priorities. As for final decision, neither votes nor ratings can be trusted alone (without proper watching and analysing comments).
I like current time at Workbench, when viewers are forced to put their feelings into words. It's the best motivation for TASing!
Here's how I see the site mission. For me, TASvideos is something between demoscene and Youtube. When the site was young, it was more closer to demoscene. But to spread TASing among common public it was necessary to add such features as ratings/polls/tags and so on. Thus it became closer to generic Web2.0 portal with user-generated content.
Sure, it was needed when TASing wasn't well-known (or even accused on cheating/etc), but today IMHO it's time for old values to make comeback. At least in terms of reappraising channels of feedback - comments should be considered much more valuable than statistics.
I hope TASvideos can find its healthy balance between pop art and alternative art.
Joined: 11/18/2006
Posts: 2426
Location: Back where I belong
After talking things over with some people, I've come up with an idea that would be my ideal method for fixing this dilemma. Firstly, I voted for the last option: "No poll, but ratings, but individual ratings are kept secret until judging."
I think that implementing ratings are a very neat way of tying two parts of the site together, and increasing usability of both systems. However:
1) I don't like the idea of having only an average score being displayed, because I do not think that a simple average is a good enough measure for all the information we'd be getting
and
2) Nor do I like the idea of having everyone's rating be displayed anonymously (similar to how we used to with yes/no/meh votes). Displaying everyone's ratings anonymously would continue the witch-hunts, and would become very cluttered in large/popular submissions.
So, my suggestion is this (Warning: Statistics 101 revisisted): Use a Box Plot (also known as a box and whiskers plot). For those of you who do not know and don't care to click the handy link, a box plot is a simple graphical representation of a set of data, that makes it very clear what the lowest vote is, the 1st quartile, the median, the 3rd quartile, and what the highest vote is. Example
0 1 2 3 4 5 6 7 8 9 10
|-------[====|====]--|
The line furthest to the left is the lowest score. The first "[" is the 1st quartile, the middle "|" is the median, the "]" is the third quartile, and the farthest right "|" is the highest vote.
Three of these graphs could be used, one for entertainment scores, one for tech scores, and one for overall scores. It would allow everyone, judges and viewers alike, to see if a run has received high entertainment, but low technical scores, or high technical scores, but lower entertainment scores (with the overall score still being used as a good indicator as to whether the public thinks the submission warrants publication.)
If the quartile system seems to be superfluous information, a similar type plot showing the low score, the mean (average), and high score would work as equally well, in my opinion.
So, the points I'm trying to make with this suggestion
1) It incorporates a system already in place on the site, which brings more continuity between the published movies and the forums
2) It provides an anonymous vote system where the lowest/highest vote for a specific category is displayed, but could possibly lead to a decrease in witch-hunts (the person who voted 1 in entertainment could also have voted 9 in tech, after all).
3) It provides everyone an equal representation of the data available, without giving judges extra knowledge.
4) It provides a much greater wealth of information that yes/no/meh votes, in that a run with a high entertainment score and low technical score will most likely need to be redone for publication, whereas a run with a high technical score but a low entertainment score will likely not be accepted due to poor game choice.
5) The viewers will still be able to voice their opinion of their run before, with less of a temptation to simply post what they vote ("I voted 6/7" without giving a reason seems to be more of an incomplete post than "I voted yes" without giving a reason). This will hopefully increase the signal:noise ratio.
Added mmbossman's suggestion.
Votes reset.
They were:
4 for restore, 14 for judges-only vote results, 8 for post buttons, 2 for text, 2 for nothing, 3 for ratings, 8 for secret ratings.
Emulator Coder, Site Developer, Site Owner, Expert player
(3599)
Joined: 11/3/2004
Posts: 4739
Location: Tennessee
I think mmbossman's suggestion is a great one, and a no brainer here. The previous votes indicate that people are clearly in favor of having less information displayed. This addresses that issue, while avoiding a "You guys can't behave so I will remove this feature" kind of approach, AND it increases the amount of information available for judging. I also think that rating the movies will increase the amount of involvement a voter feels when watching submissions. In addition, it improves another aspect of the site (movie ratings). A win/win/win/win if you ask me.
(Obviously I voted for this option).
