The question cannot be "should this movie be published".
The person who votes in the poll is not a judge. They are usually able to say how they liked the movie, but they are not able to determine what is good for the site and what is not.
I knew you were going to say this.
Fabian wrote:
This makes no sense. Just because a person is not actually a judge doesn't mean he can't have an opinion, which is probably a good one anyway. Your statement would be very valid if the real judges just went with whatever the poll results indicated, but that's obviously not the case.
Very true. It is intended that people write down why they voted something anyway, and the real judges will be able to make their own decisions with the given information.
I think this goes off on a slight tangent, but is relevant to the current discussion.
I would really like to see all votes (whether it would be yes, no, meh, w/e) justified. Perhaps some mandatory comment box (that can be edited) to go along with each person who votes might suffice. This may discourage voters, but that way, it seems we can get rid of a lot of people who don't take the poll very seriously (provide a poll still exists after this discussion has reached some sort of conclusion).
Alternatively (or additionally), you could put in some sort of anonymous option, where no comment is necessary, and this could be indicated in the poll results.
Joined: 3/17/2007
Posts: 97
Location: Berkeley, CA
Abstaining from poll since I'm hardly an active member. Here's my opinion FWIW.
Bisqwit wrote:
The question cannot be "should this movie be published".
The person who votes in the poll is not a judge. They are usually able to say how they liked the movie, but they are not able to determine what is good for the site and what is not.
I disagree, because I've always considered the ideal poll question to be mmbossman's proposal: "Do you think this movie would make a quality addition to the site?"
Of course viewers can't determine what is good for the site and what is not, but they clearly can determine what is good for their enjoyment of the site. And, after all, isn't that what we want to gauge in the first place? Imagine a submitted movie which matches the high-quality entertainment of its predecessor, but is a little worse technically (e.g. slower due to fadeout lag or door transition problems). The honest answer to the current poll question would be Yes, but I think most people would agree the "correct" vote would be No.
I also think that the question of obsolescence should be polled separately (to stop it from influencing the "main" poll question, or maybe to help in situations where it's not clear which older run(s) might have to go). This change should coincide with the move to Tub's layout of the movies page, which will tip the balance between the benefits and drawbacks of separating these questions.
Finally, I'd really like to see the judges weigh in on what they want to learn from the polls which are supposed to help them gauge support for a movie.
Here's the easy solution: Remove the poll.
-It was added back when the submission queue was WAY out of control.
-It was added to help judges pick out what movies should be published first out of a ton of games which they may not have known anything about.
-It was added to relieve the complaining when a movie would go a long time without publishing because none of the judges knew anything about the game. Bisqwit & Co. are busy people and didn't have time to go through the 60+ submissions which were in queue and read the discussion every time they had time to publish something. Presented with a large list like that it's easier to stick with what you know.
-It was this here which is what it made it useful.
-It WAS NOT added to give the site a more democratic feel to it. People seem to completely ignore that fact. If a movie has 25 yes votes a judge could reject it just because they wanted to. Even though that wouldn't likely happen.
The queue has since shrunk, and it's not needed anymore, so remove it. It's become extra fluff. All it does now is cause arguments, or cause problems such as people voting multiple times on your own movie. Drama is not a good thing.
Removing it just means that people would actually need to actually voice their opinion of why a movie should be published, rather than arbitrarily voting. This is a good thing.[/url]
Sometimes I abstain from voting, because I'm torn between a run being done very well, but the game being a boring one, or one that just isnt very interesting for a TAS (e.g. xipo's recent Mighty Final Fight, and Twisted Eye's Archon)
I think the question should be "Do you think this movie should be published?"
True, we shouldn't have a standard of "80% and you're in", but it should be at the judge's discretion. However, it would still be a "democracy" in the sense that most first-world countries are democracies. In most "democratic" countries, including Finland, Canada, America and Japan, the polls sway the politicians, but do not directly influence them. However, the voters choose the politicians. However, a vote for judges on tasvideos would be a popularity contest, so I say:
When it comes to making judges, benevolent dictatorship.
When it comes to publishing movies, Canadian/Finnish/whatever-style democracy.
Here's the easy solution: Remove the poll.
...
-It was added back when the submission queue was WAY out of control.
...
The queue has since shrunk, and it's not needed anymore, so remove it. It's become extra fluff.
...
I politely disagree with your argument, but I do agree that the submission queue has shrunk. The thing is, what is stopping the queue from growing in size again? If the voting helps the judges get things done quicker and more efficiently (as in not letting the movies pile up) I would argue for keeping the voting.
More on topic: I voted to keep the question the same. Although I could be influenced into changing it, I personally never take the question as an exact meaning, basically if I liked many different things about the movie, as well as noticing few flaws, I vote "Yes", not "Yes, I enjoyed watching this movie"
adelikat wrote:
I very much agree with this post.
Bobmario511 wrote:
Forget party hats, Christmas tree hats all the way man.
I have become very annoyed by how people think that getting 80% yes votes is an instant sign of accepting the movie. This is NOT a democracy and the voting system is there only to help judges in cases they're not sure about. In this sense I'm tempted to support BoltR suggestion of removing the poll altogether. Still, even though the site now has a lot more users and judges than when the question was instated, the poll is needed in some cases. It makes the judges' workload a bit lighter when they have some kind of idea of the public opinion visible to them right away.
As for the question, I've never ever answered solely the "did you like this movie" question. To me that's not the point of voting. There are standards at this website and even if I like a movie that's full of flaws, I won't (and I don't think anyone should) vote yes. The question I have always answered is "do you think this movie should be published", because that's what the judges want to know. The form of the question, to me, is irrelevant. I think with my brains, not my eyes.
The new SM movie that inspired this thread is of course a great example. Of course I liked the movie, it's a quality SM TAS! Still, I don't think it should obsolete hero's movie and thus I voted no. If everyone just voted yes because they liked the movie, the help that the poll is supposed to give to the judges goes down the drain because they still have to make the hardest decision alone: should this obsolete hero's movie?
I think that the basic idea of the poll is that the actual judges, those who actually decide if the submission is accepted or not, can get second opinions and points of view from other people. This can be very helpful because they can miss something important. 30 eyes see more than 2. If a submission gets a considerable amount of "no" votes then the judges will look more closely at what could be the problem people are seeing with the submission. This probably helps increasing the average quality of publications.
I have the opposite suggestion (only half-seriously presented, but something to think about): Not only change the question, but also offer more than 3 choices.
I have seen several times people commenting that they are not voting at all (ie. not even "meh") even though they watched the submission, for a multitude of reasons. Perhaps these people should be given more options to vote.
Perhaps one of the options could be something which means about I'm not completely happy with it, although if I would be forced to decide I would have to opt for the 'yes' vote", and the opposite. Or whatever.
Joined: 3/25/2006
Posts: 850
Location: stuck in Pandora's box HELLPP!!!
Kyrsimys wrote:
I don't think it should obsolete hero's movie and thus I voted no.
That's not a reason to vote no, you can protest to hero's movie being obsoleted in the discussion, it's not as if the judges ignore the discussion. A movie gets published if it has a superior quality/entertainment. That is not to say that the former movie get's removed off of the database, or that it has not been ever considered published.
Just because you like the former video more, doesn't mean you can't watch it anymore if the new one gets published. Whether something gets published or not usually gets discussed in the thread anyway. And it get's a lot more attention than the poll anyway
Kyrsimys wrote:
If everyone just voted yes because they liked the movie, the help that the poll is supposed to give to the judges goes down the drain because they still have to make the hardest decision alone: should this obsolete hero's movie?
It wouldn't go down the drain, if everyone voted yes on the grounds that they liked the movie, then that forefills exactly what the poll asks. The poll doesn't ask whether the movie should be published, nor does it mean that. That's is what the discussion about the movie is for.
And the hardest decision is what the judges do, it's what they are for.
We should really get a judge to comment in this thread
It wouldn't go down the drain, if everyone voted yes on the grounds that they liked the movie, then that forefills exactly what the poll asks. The poll doesn't ask whether the movie should be published, nor does it mean that. That's is what the discussion about the movie is for.
The point of the poll is that judges get a little bit of help when deciding the fate of the submission, no? If I just vote yes on all submissions simply because I like watching a TAS, any TAS, it's not really very helpful. Of course the movie needs to be compared to the existing movie, otherwise the poll won't mean a thing. If everyone truly voted yes for everything they enjoyed watching, we would have hardly any no votes at all.
P.JBoy wrote:
And the hardest decision is what the judges do, it's what they are for.
And that's what the poll is for, too. To help the judges make that decision.
Joined: 3/25/2006
Posts: 850
Location: stuck in Pandora's box HELLPP!!!
Kyrsimys wrote:
the movie needs to be compared to the existing movie, otherwise the poll won't mean a thing.
Yeah, you're right.
The point of the poll is that judges get a little bit of help when deciding the fate of the submission, no?
But it needs to help by comparing how many people like the movie, if it is used for "if it should be published" then that can take away 1 area of help as well. It would just be a count for what is discussed in the movie's thread. Then we wouldn't a decent knowledge of the percentage of people who liked the movie. Whether it should be published, or not.
You are absolutely right. My own purpose for it was originally to have a look at that list and see if there's something interesting in the list of 100 submissions.
Now I have much less used it so.. but I have much less judged movies, too.
People can post comments on a submission regardless of whether they can vote about it or not. However, the vote is an easy and quick way of seeing if there are negative opinions about the submission. This may be especially relevant if the submission thread is very long. The judge can see at a quick glance if there are any negative opinions or not, and if there are, then he can try looking for them in the thread. (Not that he shouldn't read the thread anyways, but a quick&short summary of opinions at the beginning shouldn't hurt.)
Personally I think the question must change but to what exactly I've yet to determine. The only thing about rating submissions is will people be reponsible in terms of how they rate them e.g. Will they give it an automatic 10 just becuase they liked it, or 1 or 0 just becuase they didn't (looking at a certain Mr X). I think ratings would be good, but will they be seperate or have we got to balance up multiple elements in order to determine a final score.
EDIT:
I think the key advantage of the rating system would be to determine what runs to prioritise and what runs to quickly reject.
I politely disagree with your argument, but I do agree that the submission queue has shrunk. The thing is, what is stopping the queue from growing in size again? If the voting helps the judges get things done quicker and more efficiently (as in not letting the movies pile up) I would argue for keeping the voting.
More judges have been added, and adelikat does a lot of work. There are also A LOT more people who are willing to help in this department.
Warp wrote:
I think that the basic idea of the poll is that the actual judges, those who actually decide if the submission is accepted or not, can get second opinions and points of view from other people. This can be very helpful because they can miss something important. 30 eyes see more than 2. If a submission gets a considerable amount of "no" votes then the judges will look more closely at what could be the problem people are seeing with the submission.
This is what the idea was, but that was back when there were over 100 movies in the queue. We didn't have time to read through 100 topics, some of which could have double digit page counts, to see which movie should be posted first. It was easy to fall behind. Now there are less than 20, so it's not as hard to read them.
AKA wrote:
I think the key advantage of the rating system would be to determine what runs to prioritise and what runs to quickly reject.
That is all I used to use it for, and I think is all Bisqwit did too. Go into the submission queue, pick a movie with one of the highest confidence factor, read through the thread, and then take ownership of the publication.
PJBoy, you have touched on one of the huge problems with the poll in the last few posts of yours. Everyone votes on their own terms. It doesn't matter what the text says, they vote yes or no based on their own criteria. They might vote no because the game kicked their dog when they were young, and not say anything about it. The game might (not) use a glitch that some people don't like. Really it's not very helpful.
Also you have a few judges and former judges comments in the thread. I'm not sure if you would count me or not, as i've been in limbo for the past long while. Working 12+ hour days is killing me, and in the short term it's only going to get worse. However, I was doing it back in the days of the 100 movies queues, and it's something I look forward to getting back to once I have free time again. Hopefully soon...
Remember, deleting the poll will if anything only encourage more discussion in the thread itself. In the end comments are way more important to judges than the stupid poll. A bunch of no votes doesn't change anything, but ONE good comment could make the difference between a publication or not.
That gives me an idea.
How about abolishing the poll and instead, rating the submission as publications are rated now?
Ratings would transfer/extend to the published movie once it's published.
To better help judges, it could also represent the ratings graphically to show the distribution of votes.
That gives me an idea.
How about abolishing the poll and instead, rating the submission as publications are rated now?
Then instead of "Voted Yes!" there would be bunch of "Rated 5/6!" posts. Or even no posts at all, because such doublepoll encourages people to express their feeling in terms of numbers, not in words. E.g. if I voted Y/N, I still haven't released all my emotions from watched movie. But if I took some time and effort to rate submission, then I become sure I fully expressed my feelings about that movie, therefore no need to write additional posts and stuff.
I'd like current polls to be untouched. Of course when I read "Did you like" I mostly interpret it as "Do you think this should be published?", being selfish enough to think that I know what's good for the site.
But IMHO, this little misinterpretaions is for good. It makes members of the community feel more involved/important.
That gives me an idea.
How about abolishing the poll and instead, rating the submission as publications are rated now?
Ratings would transfer/extend to the published movie once it's published.
To better help judges, it could also represent the ratings graphically to show the distribution of votes.
I like that idea, also. (Possibly because I may or may not have suggested it myself some months ago.)
2. "The question cannot be "should this movie be published".
The person who votes in the poll is not a judge. They are usually able to say how they liked the movie, but they are not able to determine what is good for the site and what is not."
This makes no sense. Just because a person is not actually a judge doesn't mean he can't have an opinion, which is probably a good one anyway. Your statement would be very valid if the real judges just went with whatever the poll results indicated, but that's obviously not the case.
Edit: Makes no sense is stupid and harsh. To clarify, while it's true Joe doesn't know what's best for the site, he will generally have a pretty good idea. I find the voting polls to be pretty accurate in general. When/if they're not, the judge should be able to see that I guess.
Not to mention, but voting here usally requires some arbitrary number of posts before you can vote, so it should be expected that you have some idea of how the site functions.
Then again, KRocketNeo is right there proving that point completely false, so whatever.
To me, it doesn't matter anymore. My faith in judges to actually lead the site has dipped and the publish matter of submissions is something out of the community's control other than mysterious "influence", so even if a high percentage of "us" enjoyed a movie it can still be sent to the trash and vice-versa.
I don't even see why we have polls, considering it's stated all over the site that the votes of which have basically 0 effect on judging.
stickyman05> The thing is, what is stopping the queue from growing in size again?
Me.
And adelikat.
I would not like to see the poll question removed, because the poll results offer good oversight on if a submission is popular or not, without reading 30 posts.
I would not like to have a comment to everyone's vote being made obligatory, because 90% of them would be "voted yes". If people have something they want to say about a movie, they will post anyway.
About the poll question, I realize it isn't completely correct, but the cases where the answers to "Did you find this movie entertaining" differs from "Do you think this movie should be published" are quite rare. People will always post their opinions in that case.
So I wouldn't mind it staying, or being changed to "Do you think this movie should be published" or even "Did you find this movie entertaining and think it should be published?"
About the poll question, I realize it isn't completely correct, but the cases where the answers to "Did you find this movie entertaining" differs from "Do you think this movie should be published" are quite rare. People will always post their opinions in that case.
And then someone else will post, "Uh oh, you didn't give an honest answer to the correct question! You are bad!"
put yourself in my rocketpack if that poochie is one outrageous dude
I kind of like giving my input even knowing that it's not a democracy. I'm sure in some cases it's helpful.
With that recent Turok: Dinosaur Hunter submission by nfq, I voted yes on the current question because I did find it entertaining. Then in my post with my comment, I noted that if the question were, "do I think this movie should be published", I would vote no because I didn't think it was up to the site's standards of quality.
Before reading through the posts here my suggestion was going to be that it would be nice to have two questions for submissions: "in your opinion, is this movie worthy of being published?" alongisde "did you find this movie entertaining?". I find frequently that when I watch a new submission I have different answers to these two questions, and that both would be relevant to a judge's gauging of the audience's response.
Having read other people's suggestions, using the rating system would probably have the same result and be easier to implement.
Joined: 5/2/2006
Posts: 1020
Location: Boulder, CO
The problem presents itself in polls like the one for turok. 20 yes, 0 meh and 1 no, and because the one person who voted no was a judge, it gets rejected.
I know his reasoning for doing it, but it still feels wrong for a movie that was enjoyed by 100% of the other people who voted to rot.
