Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Memory wrote:
if amount of work is not a criteria then why is it listed on voting guidelines as such
I don't know who put it there.
feos makes a good point that there's no way a viewer can honestly know this information. Therefore with his good argument, we should remove it from the voting guidelines.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
If the primary concern is entertainment value, why do we care about perceived technical prowess in the first place?
If we're concerned about entertainment both of the following hold equal place:
If it's entertaining, who cares how sophisticated the techniques were?
If it's not entertaining, who cares how sophisticated the techniques were?
To me, when entertainment is the primary concern, a technical rating alone is nigh meaningless separated from the entertainment rating of the same video.
Thus, if technical rating is regarded as holding value separately from entertainment rating, a movie's technical rating shouldn't affect the player points that are intended to indicate who's entertaining the audience.
If people are rating on a global scale, and counting amount of techniques used in the run and how expertly it was made, then you end up with a score that becomes informative. If people just make up numbers that have no global bearing, then yes, in that case it wouldn't help.
Unfortunately there's no way to know which of these methods raters are using.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
DrD2k9 wrote:
If the primary concern is entertainment value, why do we care about perceived technical prowess in the first place?
We have more than an average amount of users who are autistic or have similar issues where they cannot appreciate what typical people finding entertaining. They typically only understand things they can measure in some way. We cater to these people too with a technical rating.
Thankfully, entertainment and technical properties are often linked together, so if you do a good job on one, you usually do a good job on the other as well. This way everyone is included and can understand what is going on, make something that every kind of person can find appealing.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
DrD2k9 wrote:
Nach wrote:
If people are rating on a global scale, and counting amount of techniques used in the run and how expertly it was made, then you end up with a score that becomes informative. If people just make up numbers that have no global bearing, then yes, in that case it wouldn't help.
Unfortunately there's no way to know which of these methods raters are using.
Indeed. And some people also rate based on whether the author is their spouse or their ex, and such. It's not perfect.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
I don't know what you're getting at here. I said people may want to watch something technical so we have that information. You say you found those movies with low entertainment rating boring, so didn't. What are you trying to prove by having different objectives than the one being discussed?
I'm saying that tech rating is impossible to properly give on the scale of the whole community, and it's impossible to sensibly use, because
1) the site is not designed for that,
2) it's inherently misleading due to being mostly subjective, as you already said, and also
3) a user will most likely just pick a high tech run that's also entertaining, and disregard the one that's only high tech.
Nach wrote:
The list may or may not be the same.
I can go and check once again, for any group of movies you want. It is mostly the same.
Nach wrote:
feos wrote:
The question. Why would anyone even want to care about technical top if entertaining top is already all they need to see, and is guaranteed to also be enjoyable?
How is it guaranteed?
You questioned the probability of being entertained by the most entertaining movies on our site, but you skipped the question about what a user is more likely to watch.
Nach wrote:
feos wrote:
Exactly because technical rating is nearly entirely subjective, it's nearly entirely useless.
Why is subjectively informing people what other viewers find entertaining or technical useless?
I never said that entertainment rating is useless. And the tech rating's uselessness is addressed at the top of this post.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
feos wrote:
I'm saying that tech rating is impossible to properly give on the scale of the whole community, and it's impossible to sensibly use, because
1) the site is not designed for that,
2) it's inherently misleading due to being mostly subjective, as you already said, and also
3) a user will most likely just pick a high tech run that's also entertaining, and disregard the one that's only high tech.
The only thing I've really been hearing from you that it's impossible to give with the criteria you defined or use in the limited way you consider personally using it. This indicates to me that you should give ratings using better criteria and be more considerate of others.
As for your three points, I don't know what 1 means. I don't find how it's misleading just because it's nearly entirely subjective. All ratings anywhere provided by human beings is subjective at least to some extent. Whether people rate properly or not, be that tech or entertainment is also a general issue that applies everywhere. Humans aren't perfect. So what?
feos wrote:
Nach wrote:
The list may or may not be the same.
I can go and check once again, for any group of movies you want. It is mostly the same.
I'm not arguing that. Nothing enforces that they be the same. They may, or they may not.
feos wrote:
You questioned the probability of being entertained by the most entertaining movies on our site, but you skipped the question about what a user is more likely to watch.
I don't know what you mean by this. At this point in the exchange, I'm not even sure if we're speaking about the same topic.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
DrD2k9 wrote:
If we can't agree on criteria for determining a rating (of any type, not just technical), what value does the rating hold?
When you reach this point in your thinking, you ask yourself what's the point in ever voting for anything or anyone.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
The only thing I've really been hearing from you that it's impossible to give with the criteria you defined or use in the limited way you consider personally using it.
I asked you how to use it, and I'm still waiting for the answer. As for giving, I'll look at other criteria mentioned on the page and examine them in my next post.
Nach wrote:
As for your three points, I don't know what 1 means. I don't find how it's misleading just because it's nearly entirely subjective. All ratings anywhere provided by human beings is subjective at least to some extent. Whether people rate properly or not, be that tech or entertainment is also a general issue that applies everywhere. Humans aren't perfect. So what?
1 means the site doesn't let you easily obtain the information about top tech movies exclusively. This info is your argument for having tech ratings in the first place.
And it's misleading, because people rate based on their idea of the questions asked, not based on the actual questions. I'll get to that.
Nach wrote:
feos wrote:
You questioned the probability of being entertained by the most entertaining movies on our site, but you skipped the question about what a user is more likely to watch.
I don't know what you mean by this. At this point in the exchange, I'm not even sure if we're speaking about the same topic.
I already repeated this point several times. When someone's goal is learning TAS-only techniques, they would rather watch movies that are entertaining and technical, rather than movies that are only technical. Therefore, only having a list of the most entertaining movies is enough for that purpose. Because all the top entertaining movies are also top technical.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
feos wrote:
I asked you how to use it, and I'm still waiting for the answer.
I don't understand how you're waiting for an answer when I already provided.
feos wrote:
1 means the site doesn't let you easily obtain the information about top tech movies exclusively.
It doesn't? If there's a usability problem there, we should fix it.
feos wrote:
Because all the top entertaining movies are also top technical.
I've mentioned this a few times already. This statement is false.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
I don't understand how you're waiting for an answer when I already provided.
Quote? How exactly do I make use of the information about technical top?
Nach wrote:
feos wrote:
Because all the top entertaining movies are also top technical.
I've mentioned this a few times already. This statement is false.
OK, so you want me to come up with actual lists and compare them. Which movie groups would make you agree with my point?
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
feos wrote:
Nach wrote:
I don't understand how you're waiting for an answer when I already provided.
Quote? How exactly do I make use of the information about technical top?
You yourself quoted me saying how one might use it. I don't understand why you're playing these games now.
feos wrote:
Nach wrote:
feos wrote:
Because all the top entertaining movies are also top technical.
I've mentioned this a few times already. This statement is false.
OK, so you want me to come up with actual lists and compare them. Which movie groups would make you agree with my point?
Why would I want your lists?
Nothing agrees with your point because you have two unrelated things which are constantly shifting.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Yes, I know that we can sort by rating and create such links. But to make this option usable for tech top, one has to find movie lists that are already small enough. Using huge movie lists in order to check the tech top takes too much time and site resources. Using smaller lists implies that the user already looks for something very specific, so sorting by tech rating doesn't change the result too much.
You see, the site uses the tier system that's based on entertainment, we promote Moons over Vault, Stars over Moons, Newcomer-rec over Stars. These serve as means to shrink the movie lists, so they can be observed with little hassle.
There's nothing similar on the site to make using the tech top nearly as easy. But even if there is, we return to my point that the tops are almost the same anyway.
Nach wrote:
Nothing agrees with your point because you have two unrelated things which are constantly shifting.
I posted a statement that you declared false. We can only check if it's true or false using actual data it was talking about: movie lists sorted by tech and by entertainment. If they are mostly the same, my statement is correct. If they are significantly different, my statement is wrong. I asked you which data should I pick for an actual experiment. You simply declared that it's impossible for my point to be correct. Therefore you don't seem to want reality to be checked. In that sense, your claim about my statement is unrefutable. Do I need to explain what that means for this talk?
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
feos wrote:
Yes, I know that we can sort by rating and create such links. But to make this option usable for tech top, one has to find movie lists that are already small enough. Using huge movie lists in order to check the tech top takes too much time and site resources. Using smaller lists implies that the user already looks for something very specific, so sorting by tech rating doesn't change the result too much.
I don't understand.
If a user wants to find top tech for NES movies, they can. I don't see what's missing.
feos wrote:
I posted a statement that you declared false. We can only check if it's true or false using actual data it was talking about: movie lists sorted by tech and by entertainment.
Your suggested method for looking into this only serves to perpetuate the falsehood.
feos wrote:
If they are mostly the same, my statement is correct. If they are significantly different, my statement is wrong.
No, in either case it's wrong because they are not linked. What you see today changes tomorrow.
feos wrote:
I asked you which data should I pick for an actual experiment.
Picking data for the experiment means you misunderstand the issue at hand. It's not about data, it's about the system itself.
feos wrote:
You simply declared that it's impossible for my point to be correct. Therefore you don't seem to want reality to be checked. In that sense, your claim about my statement is unrefutable. Do I need to explain what that means for this talk?
You can argue against me by stating why my system isn't good. Showing me data for a different system is not relevant and therefore the point you are making is impossible to be correct.
When I'm talking about dogs, and you want to discuss Saturn's moons, any relationship you want to make about Saturn's moons to the dogs isn't a real relationship and therefore incorrect.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
If we can't agree on criteria for determining a rating (of any type, not just technical), what value does the rating hold?
When you reach this point in your thinking, you ask yourself what's the point in ever voting for anything or anyone.
Nice strategy to avoid actually answering the question.
I repeat, without criteria what value does the rating hold?
In my opinion, there is little to no value.
If the site's only criteria for technical rating is based on the whim of what the watcher perceives to be technical at that point in their life, fine; at least that's a criteria. But if no specific criteria are established, no one has the right to discount anyone else's ratings, reasoning for said ratings, or definition of what is technically good/bad.
For the record, I don't rate published movies, because I don't care about the ratings. I watch what movies I want to watch regardless of what the general audience has previously decided about them. I don't use ratings or tier placement to help me decide what to watch. I'd be amazed if I'm the only one who make this independent decision on what to watch.
Also, voting is different than rating. To me, rating something is tying a degree of value to a comparison and thus needs some established criteria (or baseline) on which to assess that value. Voting is simply making a choice between two (or more) selections, not grading their degree of value. Polls (when worded appropriately) can take the purpose of either voting or rating. (When worded poorly, the purpose can't be deciphered between voting or rating and the results are of little use.)
So at least for the purposes of this site, I personally don't find much if any value in ratings or polls.
And now that I've thought more in depth on this topic, I will probably refrain from participating in future forum polls if I cannot decipher which purpose the poll has (choosing a specific option vs. rating). Unfortunately, I find the poll regarding workbench submissions to be poorly worded and will thus refrain from utilizing that feature.
I don't understand.
If a user wants to find top tech for NES movies, they can. I don't see what's missing.
It is too hard to use, this is what I'm saying. I went as far as to say that it's "impossible to sensibly use", partially because of how hard it is.
Nach wrote:
Your suggested method for looking into this only serves to perpetuate the falsehood.
What you see today changes tomorrow.
Picking data for the experiment means you misunderstand the issue at hand. It's not about data, it's about the system itself.
My point was about how the system is being used, and how usable tech rating is. Saying that tech top list will in future significantly diverge from top entertainment is yet another claim that can't be checked. I say that these top lists will remain very similar forever. Because of how people use tech rating (I'll still get to this).
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
DrD2k9 wrote:
I repeat, without criteria what value does the rating hold?
In my opinion, there is little to no value.
That is indeed the existential question.
DrD2k9 wrote:
But if no specific criteria are established, no one has the right to discount anyone else's ratings, reasoning for said ratings, or definition of what is technically good/bad.
We have a page with guidelines (thanks Warp) linked from the rating's page describing how to rate. The criteria is established. Whether the majority follow the criteria for anything is anyone's guess. We can be certain that not everyone is for both entertainment and technical.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
I don't know who put it there.
feos makes a good point that there's no way a viewer can honestly know this information. Therefore with his good argument, we should remove it from the voting guidelines.
I was asked to write that section.
With "amount of work" I didn't mean "hours put into it" (at least not solely), but more like how much effort was done/needed to do the run.
Of course basically the only way that a viewer can get any sort of idea how much effort was put into the run is by reading the submission description (which the author ought to also put effort into). I was thinking that if the author demonstrates a great deal of work put into making the run (and its submission text, I suppose), that could be a good reason to raise the technical rating by a notch or two.
Like the other bullet points in that list, it wasn't intended as a hard rule. As in "does the author demonstrate a great amount of effort? If yes, raise the score, else lower it." Instead, it's more like a suggestion or idea of something to look for and consider, when deciding on a technical score. I hope I have succeeded in describing what I was thinking.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
feos wrote:
It is too hard to use, this is what I'm saying. I went as far as to say that it's "impossible to sensibly use", partially because of how hard it is.
Okay, how would you like to improve it?
feos wrote:
Saying that tech top list will in future significantly diverge from top entertainment is yet another claim that can't be checked.
This continues to miss the point.
My statement on the matter is that while there is some actual link in what occurs between entertaining and technical because one often fuels the other, the actual link in results is of a temporal nature.
You can show me all the lists you want, that does absolutely nothing to debunk my claim of the rating link being of a temporal nature. If you want to debunk my claim, you must show that there is a causal link between the two which cannot be broken. Anything else is arguing in the wrong arena.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Warp wrote:
I was asked to write that section.
Thank you for writing that section.
Warp wrote:
Of course basically the only way that a viewer can get any sort of idea how much effort was put into the run is by reading the submission description (which the author ought to also put effort into). I was thinking that if the author demonstrates a great deal of work put into making the run (and its submission text, I suppose), that could be a good reason to raise the technical rating by a notch or two.
Also they can look at how many rerecords are used and so on.
But this depends on the author being truthful. I could make a submission I spent one hour on, change the rerecording count into the billions, and write a long text about how I spent years researching the game and post a disassembly I generated with some tool, and make it seem like I really did go nuts with it.
I think a person should pay more attention to what it actually looks like is going on, than to anything they claim.
Warp wrote:
Like the other bullet points in that list, it wasn't intended as a hard rule. As in "does the author demonstrate a great amount of effort? If yes, raise the score, else lower it." Instead, it's more like a suggestion or idea of something to look for and consider, when deciding on a technical score. I hope I have succeeded in describing what I was thinking.
I believe I got the gist of what you're saying. I think it may have some ambiguity though as some people earlier seem to think they need to understand the game inside out, instead of seeing how the the actual results speaks to them, and rate it based on that.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
If we can't agree on criteria for determining a rating (of any type, not just technical), what value does the rating hold?
I see this as a 'make of it what you will' type of thing. It might have little/no value, but it is also of little/no consequence. If someone gets something out of it that's fine, otherwise it's safely ignored.
If it were a situation of 'you must have this player rank to do this thing on the site,' then figuring out the best way to calculate it would obviously be more important, but it's not, it's just a number.
If it were a situation of 'you must have this player rank to do this thing on the site,' then figuring out the best way to calculate it would obviously be more important, but it's not, it's just a number.
My statement on the matter is that while there is some actual link in what occurs between entertaining and technical because one often fuels the other, the actual link in results is of a temporal nature.
You can show me all the lists you want, that does absolutely nothing to debunk my claim of the rating link being of a temporal nature. If you want to debunk my claim, you must show that there is a causal link between the two which cannot be broken. Anything else is arguing in the wrong arena.
My plan is to look at how users actually use the system, so we could talk about real problems instead of our idea of them.
But please give me an example of a statement that'd debunk your claim if it was true. Without such an example I'd have to rely on my own fantasies about this "causal link". I mean, I'm not even sure what kind of information I need to provide here.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
