Editor, Experienced Forum User, Judge, Published Author, Expert player
(2057)
Joined: 8/21/2016
Posts: 1011
Location: US
feos wrote:
So I think for this game, a movie should complete 3 loops of mode B, so difficulty goes 2, 3, 4, and then it freezes gameplay-wise, even though the value of the address keeps increasing after new loops.
...
It checks if difficulty-1 is above 3, which indeed means difficulty caps out at 4.
NOTE:
If all the content of the difficulty 4 appears earlier, then it'd instead be the point to stop the movie after. I just haven't tested every difficulty thoroughly. But this video gives some idea, even though it's for mode A.
So does the above make this game an example of one where a future submission that completes all the proposed loops would obsolete the current publication even though the future run will be a longer time than the current?
Editor, Experienced Forum User, Judge, Published Author, Expert player
(2057)
Joined: 8/21/2016
Posts: 1011
Location: US
Alyosha wrote:
If it were a situation of 'you must have this player rank to do this thing on the site,' then figuring out the best way to calculate it would obviously be more important, but it's not, it's just a number.
Editor, Experienced Forum User, Judge, Published Author, Expert player
(2057)
Joined: 8/21/2016
Posts: 1011
Location: US
Nach wrote:
DrD2k9 wrote:
If we can't agree on criteria for determining a rating (of any type, not just technical), what value does the rating hold?
When you reach this point in your thinking, you ask yourself what's the point in ever voting for anything or anyone.
Nice strategy to avoid actually answering the question.
I repeat, without criteria what value does the rating hold?
In my opinion, there is little to no value.
If the site's only criteria for technical rating is based on the whim of what the watcher perceives to be technical at that point in their life, fine; at least that's a criteria. But if no specific criteria are established, no one has the right to discount anyone else's ratings, reasoning for said ratings, or definition of what is technically good/bad.
For the record, I don't rate published movies, because I don't care about the ratings. I watch what movies I want to watch regardless of what the general audience has previously decided about them. I don't use ratings or tier placement to help me decide what to watch. I'd be amazed if I'm the only one who make this independent decision on what to watch.
Also, voting is different than rating. To me, rating something is tying a degree of value to a comparison and thus needs some established criteria (or baseline) on which to assess that value. Voting is simply making a choice between two (or more) selections, not grading their degree of value. Polls (when worded appropriately) can take the purpose of either voting or rating. (When worded poorly, the purpose can't be deciphered between voting or rating and the results are of little use.)
So at least for the purposes of this site, I personally don't find much if any value in ratings or polls.
And now that I've thought more in depth on this topic, I will probably refrain from participating in future forum polls if I cannot decipher which purpose the poll has (choosing a specific option vs. rating). Unfortunately, I find the poll regarding workbench submissions to be poorly worded and will thus refrain from utilizing that feature.
Editor, Experienced Forum User, Judge, Published Author, Expert player
(2057)
Joined: 8/21/2016
Posts: 1011
Location: US
No. I don't think (as a group) humans are ever entirely objective on anything....polls included.
Certain individuals may have mastered the discipline of being more objective and less emotionally driven than others most of the time, but I don't believe anyone always takes the perfectly objective perspective on all issues.
I sure don't....and frankly, I'm thankful for some of the emotional decisions I've made in life that have led to what I believe have been much better outcomes than would have resulted if I had instead taken a strictly objective approach to the situation.
Editor, Experienced Forum User, Judge, Published Author, Expert player
(2057)
Joined: 8/21/2016
Posts: 1011
Location: US
Nach wrote:
If people are rating on a global scale, and counting amount of techniques used in the run and how expertly it was made, then you end up with a score that becomes informative. If people just make up numbers that have no global bearing, then yes, in that case it wouldn't help.
Unfortunately there's no way to know which of these methods raters are using.
Editor, Experienced Forum User, Judge, Published Author, Expert player
(2057)
Joined: 8/21/2016
Posts: 1011
Location: US
If the primary concern is entertainment value, why do we care about perceived technical prowess in the first place?
If we're concerned about entertainment both of the following hold equal place:
If it's entertaining, who cares how sophisticated the techniques were?
If it's not entertaining, who cares how sophisticated the techniques were?
To me, when entertainment is the primary concern, a technical rating alone is nigh meaningless separated from the entertainment rating of the same video.
Thus, if technical rating is regarded as holding value separately from entertainment rating, a movie's technical rating shouldn't affect the player points that are intended to indicate who's entertaining the audience.
Editor, Experienced Forum User, Judge, Published Author, Expert player
(2057)
Joined: 8/21/2016
Posts: 1011
Location: US
TASeditor wrote:
DrD2k9 wrote:
2) If the run is obsoleted by a different author, the author of the obsoleted run gets the point value of the shorter run which in theory should always be equal to or lower than the published run (some obsoleted runs may actually be shorter due to publication prior to site clarification of game endpoints--in these cases, the author of the longer published run deserves the higher point value than the author of the shorter obsoleted run).
Aren't obsoleted runs usually longer or do you mean how long since the movie has been published?
Yes, obsoleted runs are usually longer. But with now clarified rules on when a game ends, it's possible that the obsoleted run could be shorter. It's not going to be a common thing though.
But all my recommendations (which are geared toward indicating general TASing experience) are moot since Nach clarified the purpose of the player points to be an indicator of who's doing the most entertaining of the audience.
Editor, Experienced Forum User, Judge, Published Author, Expert player
(2057)
Joined: 8/21/2016
Posts: 1011
Location: US
Nach wrote:
Player points tells you who is doing a good job of keeping the audience satisfied.
Thank you for the clarification on what the site wants/uses the number to represent.
That being the case, I now feel similar to Alyosha with player points being a mostly useless number for me.
I don't say this to be offensive, but I personally don't care who is or isn't entertaining/satisfying the audience (others may).
I started TASing for my own enjoyment and continue to do so for that reason. If what I accomplish entertains others, great! If it doesn't entertain others, great!
Editor, Experienced Forum User, Judge, Published Author, Expert player
(2057)
Joined: 8/21/2016
Posts: 1011
Location: US
Nach wrote:
ViGadeomes wrote:
It's why I never rated a movie because I never played a game, so I'm not able to say what is good or not in the movie. I'll rate movies from games I know but that's it.
That doesn't make any sense. Your rating is how you feel something compared to other things you've seen on our site. You're not rating the movie's use of the game's inherent qualities. You don't need to know about a game to determine how a movie of it makes you feel versus watching other movies.
(underline mine)
So essentially, a large portion of player points is based on the subjective opinion of others, not on objective factors.
Again, if we want this as a 'who's who' or popularity ranking system that varies over time, that's perfectly acceptable. But if we want the player points to be more of an experience system/indicator, the subjective opinions of others shouldn't be included in the calculation.
I can use myself as an example: my own points increased recently because EZGames69 rated a few of my publications, then they dropped back down because one of my runs was obsoleted. Neither of these changes happened due to my own actions, yet still affected the score associated with my name. I realize that using my earlier recommendation for calculating the points would still result in the decrease due to the obsoleted run, but the score wouldn't have increased due to the ratings. And while that means I would have even fewer points now, I feel it would be a better indicator of my experience.
As it stands, utilizing ratings automatically rewards popularity of a game because more popular games are more likely to be watched and therefore more likely to be given ratings. This popularity bias gives extra reward to TASers just for choosing a popular game. Again, if we want this score/rating as a 'who's the current best or most popular' approach, no problem. If we want the score as an experience indicator this bias should be removed.
TL;DR: The big question that needs to be answered is what do we want the player points to indicate?
Disclaimer: I don't put much thought to my own score because I had already recognized the subjective nature of the player points long before this topic was started. If it were switched to a more experience based model, I might be more interested in my own score. This may also encourage me to work a bit more on increasing it; and if others shared that perspective, it may increase the overall production of TASes.
Editor, Experienced Forum User, Judge, Published Author, Expert player
(2057)
Joined: 8/21/2016
Posts: 1011
Location: US
Alyosha wrote:
Personally I don't think player points have much if any value. They do seem to do a poor job representing a TASer's body of work for the reasons TASeditor mentioned. I'd rather just have a direct link to a person's published works in that space instead.
The primary benefit I see in having the points visible is as a quick indicator of TASing experience.
For example, if I saw two conflicting opinions on some TASing topic, I'd be more apt to accept the opinion of someone with a higher point total than that of a lurker.
It doesn't mean the more experienced person is always right, or that lurkers are wrong. It's just that the high point TASer has more evidence of experience to make whatever claim they are.
To put it simply. I see it as akin to experience points (or gamerscore), not as a true ranking mechanism. It's also for that reason I don't like the idea of losing a ton of points when a run is obsoleted by someone else.
If we wanted the score to simply show experience, we could simplify the point calculation to something along the lines of the following:
For current publications: every game would start with a base value regardless of game length and then the game length (while potentially exploitable) would still factor into the score Z=(length of publication in frames)/3600*(# of players)+100
Then a member's score would be (the sum of all Z's for currently published runs) + (the value of obsoleted games see below)
Regarding obsoleted videos. As much as I don't like the idea of losing drastic amounts of experience points for an obsoleted run, I also don't think a longer run should be worth more points than a shorter run of the same game. So in the case of a run being obsoleted, I'd recommend a couple caveats.
1) If a run is obsoleted by the same author, the author only gets the points of the published run for the given branch.
2) If the run is obsoleted by a different author, the author of the obsoleted run gets the point value of the shorter run which in theory should always be equal to or lower than the published run (some obsoleted runs may actually be shorter due to publication prior to site clarification of game endpoints--in these cases, the author of the longer published run deserves the higher point value than the author of the shorter obsoleted run).
This calculation would eliminate any voting/rating/popularity impact on the player's experience point calculation for a given game (which also means perceived entertainment value would not be considered for indicating player experience). This would eliminate poor scores due to low quantity of rating/voting or inflated scores due to game popularity. This would also eliminate player's experience level from changing due to a game being moved up/down a tier. Highly experienced TASers can still make boring videos, and conversely inexperienced TASers may still be able to produce something entertaining. How well a run is perceived by the community from an entertainment perspective is not an indicator of how experienced a TASer is or isn't.
EDIT: This approach would also eliminate the "Former Player" rank and give credit to those who have accomplished things before even if they have no currently published runs.
Editor, Experienced Forum User, Judge, Published Author, Expert player
(2057)
Joined: 8/21/2016
Posts: 1011
Location: US
TASeditor wrote:
DrD2k9 wrote:
As far as using the rank calculation system to try to encourage authors to update older runs, I don't agree. I don't think anyone should be penalized for a long lasting published TAS.
I don't want to penalize people for long lasting runs, I want them to make new movies. Inactivity should be penalized. Maybe the factor could be set back to 1 after 20 years, to account for "perfect" (not all old runs are perfect, of course) runs.
(bold is my highlight)
This brings up the question of whether or not we, as a community, want to reward(penalize) activity(inactivity) as a whole. So, are you suggesting that currently active members are worthy of increased player points over members who are no longer active? (I'm not suggesting this opinion would be wrong; it's just a possible perspective and you're welcome to it if that's your opinion.)
To me, though, that would be suggesting that those who have come before--but are now no longer active--are somehow less worthy of a given score just because they aren't active anymore. Essentially, I feel it suggests that a given achievement is less worthy of recognition simply because the person who accomplished it is no longer active. This doesn't make sense to me; it'd be like saying an athlete that holds a world record is due more praise if they are still competing than if they were no longer competing. Whether or not they are still active doesn't change the fact that they hold a world record. To me, we shouldn't say that someone's work is worth more just because they are active as opposed to inactive.
Take Mothrayas, for example, who has recently chosen to take a step down in his level of activity. I don't think he should be penalized for that decision, because I don't know the life circumstances that led to him making that decision. Even if he were to never be involved again, I believe he's worthy of the full recognition for all his contributions/achievements.
A further problem, in my opinion, with altering the factor based on time since publication is that it can still penalize active members, not just inactive ones. If any penalty to a single publication's rank took place simply due to the passing of an arbitrary amount of time, the author(s) would be penalized regardless of how active they have or haven't been on the site. That member may have gotten 300 more published TASes of other games in that time frame, but would still be penalized because they didn't update that one game. So their personal player score would still suffer even though they've been plenty active.
Granted, all of my thoughts here are based on my perspective that our player rank/score here is more akin to an Xbox's gamerscore than it is a system of actually ranking our members in how good they are at TASing. If others actually view them as a way of ranking who's currently the 'best' or at the 'top of the leaderboards' then my thoughts are moot, and it would make sense to penalize inactivity.
Editor, Experienced Forum User, Judge, Published Author, Expert player
(2057)
Joined: 8/21/2016
Posts: 1011
Location: US
feos wrote:
DrD2k9 wrote:
Consistently submitting runs with lots of blank frames may get you limited in your ability to submit.
Wrong. We only disable submission privilege when the author consistently breaks basic movie rules. Ending your movie at the right time is easy to learn when you are already aware of other rules.
Sorry, I was trying to suggest that if it was a continued problem with the same submitter where that person knew it could be fixed, but didn't show the desire/attempt to fix this problem. Perhaps even then I'm incorrect.
This is not an uncommon mistake for new TASers, one most correct very quickly. I wasn't trying to scare anyone.
Editor, Experienced Forum User, Judge, Published Author, Expert player
(2057)
Joined: 8/21/2016
Posts: 1011
Location: US
Rewstarr wrote:
I'm not really sure what a light Spike is or how the attached file looks different in the video in comparison, probably due to my inexperience.
First, welcome to the community.
Second, a "light spike" is Spikestuff's way of updating your submission movie to cut all the frames of blank input at the end of your run. Basically this will not alter your work in any way, but will more accurately indicate the time of the run.
Consistently submitting runs with lots of blank frames may get you limited in your ability to submit.
If you need help knowing how to avoid/remove these ending blank frames, just ask here or in discord and we'll be glad to show you how.
Editor, Experienced Forum User, Judge, Published Author, Expert player
(2057)
Joined: 8/21/2016
Posts: 1011
Location: US
I partially agree with one of these problems but disagree with the other.
While I partially agree that factoring in worktime would be ideal, I think it's something that isn't well defined enough to adequately implement into the calculation; even considering your suggestion using re-record counts.
There are plenty of reasons that re-record counts may not be accurate:
1) You've already mentioned bot use and rerecording behavior of the author. It would be difficult to vary a value/multiplier based on each member's rerecord behavior (which doesn't even consider the subjective nature of this type of factor).
2) Situations of copy/pasting of inputs (at least for games that can use TAStudio). Some older runs are improved/updated through copy/pasting of inputs.
3) Situations where an older movie is updated to a newer/more accurate emulator. (I'm not sure either way if converting between emulators--i.e. FCEUX to BizHawk--preserves rerecord count, so this situation may not be a huge issue).
4) Direct editing of input within movie files (as can be done in a text editor for DOS games run on JPC-rr).
5) Any other reasons that didn't quickly pop into my head.
As far as using the rank calculation system to try to encourage authors to update older runs, I don't agree. I don't think anyone should be penalized for a long lasting published TAS.
Let's take the newest SMB warpless run as an example. Most of us would probably agree that SMB is becoming near impossible to improve further (if it's not already there). So if the current publication stands for the next 20 years without anyone obsoleting it, why should its rank value be diminished? HappyLee and Mars608 shouldn't have their rank penalized just because no one (including themselves) was able to beat their current run over any given length of time.
As for my own beef with the current rank calculation, I don't like how obsoleted runs are so harshly diminished in their importance. But I don't have a recommendation on how to improve that...so it's just me whining on that point, I guess.
Editor, Experienced Forum User, Judge, Published Author, Expert player
(2057)
Joined: 8/21/2016
Posts: 1011
Location: US
I think this is one of those games that you have to understand play from a casual perspective to fully appreciate the TAS.
Unfortunately I've never played it casually and thus find it only mediocre entertainment.
It is obviously superhuman, and the submission notes helped with understanding some of the actions. But this knowledge just didn't add enough for me to give this more than a 'meh' vote.
Editor, Experienced Forum User, Judge, Published Author, Expert player
(2057)
Joined: 8/21/2016
Posts: 1011
Location: US
EZgames69 beat me to the encode by 10 minutes.
Looks good though. Yes vote from me. No reason I can see that this run shouldn't be published as an improvement to my run.
Editor, Experienced Forum User, Judge, Published Author, Expert player
(2057)
Joined: 8/21/2016
Posts: 1011
Location: US
Soooo...getting back to the actual title of this topic.
Is there enough perceived confusion about polls (specifically, but not limited to the 'meh' option) for the staff to consider a change of any type? Or is the general consensus that what we have is good enough and doesn't need changing?
Personally, I think the available answers unfortunately cause the poll to be utilized by voters from both a perspective of entertainment (as it's written/intended) but also as meaning "should this be published?" (which is not up to the community, but the judge).
I think (as I suggested before) that a poll regarding the degree of entertainment may be better than the yes/no/meh options for "did you find this entertaining?"
Editor, Experienced Forum User, Judge, Published Author, Expert player
(2057)
Joined: 8/21/2016
Posts: 1011
Location: US
Almost 1:20 improvement! Well done! I'm glad the bot searching helped you get the Gold bug stuff figured out. Sorry I couldn't help any more than I did on this run (real life kinda got in the way).
I'm looking forward to watching the run as soon as I have a few spare minutes. I'll also make a temp encode if there isn't one by the time I watch it.
A side by side comparison would be interesting to me....but I'm not sure how to make one.
Editor, Experienced Forum User, Judge, Published Author, Expert player
(2057)
Joined: 8/21/2016
Posts: 1011
Location: US
HappyLee wrote:
Criticisms are OK, as long as they are fair and rational. But this time, having to argue over a run that shouldn't be controversial at all really upsets me.
Lee, continually emphasizing your perspective that all this "shouldn't be controversial" only furthers the idea that you think there's a major problem simply because what has happened didn't line up with your personal expectations.
It obviously has been controversial...which only serves to prove your assertion that it "shouldn't be controversial" is incorrect.
As far as your original expectations on the run: There's nothing wrong anticipating a particular response. The problem exists when you staunchly and publicly assert that things are awry simply BECAUSE they are different than your expectations.
As it stands, your run is sitting at 81% positive response. That's nothing to be upset about; its a fantastic positive response (and in my opinion not low enough for a judge to consider dropping the run to a lower tier than the current publication). You're only complaining because it's not MORE positive and closer to what some of your previous runs have attained. While it makes sense to base one's expectations on what previous submissions have attained, those results on previous runs don't guarantee ANYTHING about a current/future submission.
You've got to get to the point where you either:
1) Consider that your original expectations weren't as accurate as you anticipated.
2) Accept that the voting results are what they are (regardless of whether votes were cast for submission based or personal reasons); but the results as they are aren't like to endanger the run of being dropped a tier.
If all the arguing (that the run shouldn't be controversial and that the negative/neutral votes aren't valid) isn't likely to change the resulting publication tier; there is no reason to continue griping about the vote counts. Doing so only makes it appear that you are concerned with the vote percentages for personal accreditation/acclaim (in other words, your ego).
By all means be proud of your work, you should be; you've accomplished what many considered impossible! But don't try to force others to give you acclaim by complaining and insinuating fraud when others don't agree with you (regardless of their reasons why).
Editor, Experienced Forum User, Judge, Published Author, Expert player
(2057)
Joined: 8/21/2016
Posts: 1011
Location: US
p4wn3r wrote:
In practice, it goes like this: if someone requests the investigation, and you deny it, and eventually another party determines that there was indeed a problem, that makes you automatically an accomplice. So, usually you tend to investigate most things and ask people to not take it personally. Of course, in this forum we are not dealing with lots of money, so this is more of an inconvenience.
I understand your concerns about voting abuse; we should all be concerned (to a practical degree) about such abuse.
It appears to me that the site staff has taken necessary and appropriate actions in the past when such abuse has been suspected and confirmed.
I disagree with your assertion that a lack of an investigation makes them accomplice though. The staff may simply have not seen validity in the claim. Just because a 3rd party does find fraudulent activity later doesn't make the staff complicit in the abuse.
The problem is that, you are asking for more investigations into voting abuse without addressing how those investigations should be requested.
The suggestion of voting abuse in the SMB run stemmed from an author who couldn't accept that the community response to his submission didn't turn out exactly as he suspected it (no one said he couldn't believe that). BUT, he then publicly tried to blame that differing result on previous drama within the community--which is unprovable even if an investigation were carried out. Further, he never requested an investigation...just kept publicly complaining that the results couldn't be accurate even when presented with reasons that it could be. Thus drama exploded into about 8 different topics.
Regarding inivestigations: There's nothing stopping site staff from investigating anything they perceive as odd. But placing the impetus on the staff to investigate any/all random sumbission's voting is backwards.
If an author (or anyone else) suspects abuse for any given submission, the impetus is on them to request an investigation; and that call to investigate should come in the form of a private request to the staff, not a public outcry within the submission discussion. As can be seen with the submission that stemmed this topic, making a public accusation of abuse leads to a whole mess of off-topic forum posts (at least regarding the submission in question) that dilutes the discussion of the submitted run.
tl;dr:
The impetus for initiating an investigation of a specific vote tally is not on the staff, it's the responsibility of the community members to privately request investigations of potential abuse. THEN it becomes the responsibility of the staff to determine if the claim is valid enough to warrant an investigation. A decision not to investigate does not equate to complicity in any (later discovered) abuse, simply a decision that investigation did not appear warranted at the time of the request.
EDIT: To the staff: Even though this topic itself was branched from the submission topic....It appears we've already gotten off-topic regarding "the point of submission polling." Should all the discussion here regarding investigation of voting abuse be branched off again into its own topic?
