Currently I see two problems with the current player points calculation, these are:
•Calculations do not account for active worktime spend on a project;
•Not encouraging authors to improve their own published movies
First issue:
From all of my movies, [3226] NES Kirby's Adventure by TASeditor in 34:22.22 is the one I spent most time on, ~3 years in fact and all I get is 155 player points for it. In the same time as making the TAS for Kirby's Adventure, I also made some TASes only with a fraction of time spent on and got almost as much as from Kirby, not a small fraction. I selected some of those (4 from Moons, 4 from Vault) which have an equal amount of voters and compiled a list of how much player points I would get if I didn't do Kirby's Adventure but made TASes with equal worktime from the movies from the list, I call this Kirby Equivalent.
From that list it is noticable that if I didn't work 3 years on Kirby's Adventure, but made lots of quick projects I would have gotten at least 2000 points, even for movies that would ended up in vault. Another thing to point out is that the longer a project took to make the lower the Kirby Equivalent.
So, why would anyone be stupid enough to spend 3 years on a project to just get ~150 points, if the same person could do a project that takes a week and gives just as much points?
My idea is to account worktime of a project into player point calculations.
It wouldn't work if just the date from first WIP to submission date was taken, because projects might have breaks in it and it would mean more work for judges/publishers finding evidence for the date of first WIP and breaks. Movie lenght also wouldn't work, because not always takes a longer movie more time to make than a shorter one.
I suggest using rerecords. It gives a good estimate for active time spent on a project, the numbers wont go higher if the author decides to take a break from the project. Of course it is needed to adjust the value according to tools used (a bot makes much more rerecords than a user with TAS Editor/TAStudio and users without these tools often make less, but not always) and probably account for rerecording behaviour of author as well. And if an author decides to submit a movie with a very high faked rerecord count, the judges is encouraged to clear that out and the author would not get a bonus factor to player points. Same should also apply if someone makes a small readjustment (I wont call names out) to a movie which had a long wortime and doesn't update the rerecord count to represent his/her work. The bonus factor should only only apply to movies which are in Moons or Stars. And there should be no penalty, meaning the bonus factor should at least be 1.
The new formula could look like this: Z_movie=O*max(max(round(M-0.75+0.5)*rerecords/(tool_usage*rerecording_behaviour),1)*M*r^max(2.6-0.2*average_ratings/ratings,1)/sqrt(p), 5)
I used a tool_usage*rerecording_behaviour of 100000 for my movies which gives me 1912 player points for Kirby's Adventure.
Second issue:
The higher the amount of voters on a movie the higher the amount of earned player points. Usually older movies tend to have more a higher amount of voters.
The player points earned from an older movie should be reduced, by accounting how old the movie is. I suggest using 6 to 7 years without lowering, then start lowering but stopping at ten years at a factor of 0.5 or 0.3. By doing this the author will be more encouraged to try to update their old movies and stay engaged with the site by making new movies.
I don't reach a goal to have the most important player point but I understand and agree that it's not a very good representation of the work behind movies.
It took me 6 months to do [3649] GBC Astérix: Search for Dogmatix by ViGadeomes in 15:35.05 (TAS that learned me everything about main TASing things).
And something like a week-end for [3709] A2600 Boxing by ViGadeomes in 00:12.80 and [3660] A2600 Grand Prix by ViGadeomes in 01:34.67 .
But the idea of using rerecord count could be a problem too because we can't if the author increased his rerecord count value. Sometimes, I just copy-paste a big part of my TAS from an older release of BizHawk with ~4000 to a newer version of TAStudio's BizHawk and just resync it. Other time, I can take an other *.tasproj and delete a lot of his content to do a complete other movie and the rerecord count keeps rerecords of the other movie.
And I'm not sure judges will be happy to spend more time for judging a movie with the big work they have to do without it just to choose how many player points the author have to gain from a movie.
I totally agree with your second issue, however I have problems with the first one :
TASeditor wrote:
Of course it is needed to adjust the value according to tools used (a bot makes much more rerecords than a user with TAS Editor/TAStudio and users without these tools often make less, but not always) and probably account for rerecording behaviour of author as well.
I don't agree with the fact that a user without tools will do less rerecords, and I'd say it's the opposite : you cannot simply edit inputs without tools like you would do with TAStudio, because it shows your inputs and a simple click change the input, when you have to load a savestate and change all the inputs you have to manually.
About taking the rerecords count in consideration, it's a good idea, but with several issues :
As you said, faked rerecord counts shouldn't be taken in consideration (obviously).
I think the bonus factor should apply for Vault too, because a TAS, even in Vault, still costs time and implies a few reserches. And vault TASes doesn't often have a high RR counts value, so their bonus will be small, reflecting their tier.
Another problem is TASes like Zook man ZX4 would be impacted, since it's hard to keep a track on such TASes.
Also, could you give the details on what are the values for tool_usage and rerecording_behaviour, like could you give us the whole detail on how you would calculate your Kirby TAS ?
About my opinion, I think we should take the movie length in consideration, with a small ratio but still be present, it is also time spent working on the TAS and doing reserches. Also, we should take an external thing to calculate, like the YouTube likes and/or views. However it also have its own problems and have to be discussed.
[17:37:00]<TheCoreyBurton> It's N64 - it's ALWAYS bad news.
I partially agree with one of these problems but disagree with the other.
While I partially agree that factoring in worktime would be ideal, I think it's something that isn't well defined enough to adequately implement into the calculation; even considering your suggestion using re-record counts.
There are plenty of reasons that re-record counts may not be accurate:
1) You've already mentioned bot use and rerecording behavior of the author. It would be difficult to vary a value/multiplier based on each member's rerecord behavior (which doesn't even consider the subjective nature of this type of factor).
2) Situations of copy/pasting of inputs (at least for games that can use TAStudio). Some older runs are improved/updated through copy/pasting of inputs.
3) Situations where an older movie is updated to a newer/more accurate emulator. (I'm not sure either way if converting between emulators--i.e. FCEUX to BizHawk--preserves rerecord count, so this situation may not be a huge issue).
4) Direct editing of input within movie files (as can be done in a text editor for DOS games run on JPC-rr).
5) Any other reasons that didn't quickly pop into my head.
As far as using the rank calculation system to try to encourage authors to update older runs, I don't agree. I don't think anyone should be penalized for a long lasting published TAS.
Let's take the newest SMB warpless run as an example. Most of us would probably agree that SMB is becoming near impossible to improve further (if it's not already there). So if the current publication stands for the next 20 years without anyone obsoleting it, why should its rank value be diminished? HappyLee and Mars608 shouldn't have their rank penalized just because no one (including themselves) was able to beat their current run over any given length of time.
As for my own beef with the current rank calculation, I don't like how obsoleted runs are so harshly diminished in their importance. But I don't have a recommendation on how to improve that...so it's just me whining on that point, I guess.
But the idea of using rerecord count could be a problem too because we can't if the author increased his rerecord count value. Sometimes, I just copy-paste a big part of my TAS from an older release of BizHawk with ~4000 to a newer version of TAStudio's BizHawk and just resync it. Other time, I can take an other *.tasproj and delete a lot of his content to do a complete other movie and the rerecord count keeps rerecords of the other movie.
And I'm not sure judges will be happy to spend more time for judging a movie with the big work they have to do without it just to choose how many player points the author have to gain from a movie.
Updating to emulator version, resyncing the project and typing the rerecord number from the old file into the new file isn't an issue. And deleting lots of content from a project and not lowering the rerecord counter isn't and issue as well, the number should represent how much work you put into a project, even if you decide to restart.
It's only a little bit of work for judges. Most work would be current published movies, evaluating the tools used (pre 2012 is all non-TAS Editor/TAStudio, lua/botting started ~2009)
Stovent wrote:
I don't agree with the fact that a user without tools will do less rerecords, and I'd say it's the opposite : you cannot simply edit inputs without tools like you would do with TAStudio, because it shows your inputs and a simple click change the input, when you have to load a savestate and change all the inputs you have to manually.
It depends on user. I know many users who use TAStudio and have an rerecord count lower than non-TAStudio users, but I also know people who do the opposite. You can always edit a single frame with one rerecord, no matter which tools are used. You are stating that you can't remember your inputs after the frame you loaded; just recording input alone doesn't increase the rerecord counter.
Stovent wrote:
Also, could you give the details on what are the values for tool_usage and rerecording_behaviour, like could you give us the whole detail on how you would calculate your Kirby TAS ?
For tool_usage the idea is to not give botted TASes with >10M rerecords automatically an higher player point bonus compared to non-botted, the tool_usage value should be higher for botted runs in order to divide by a bigger number. rerecording_behaviour is more complicated to evaluate, people make different amount of rerecords in the same time span, using averages doens't work. For Kirby I'd like at least 2000 points, so I adjusted tool_usage*rerecording_behaviour to a value (100000, my average rerecord count for all my Moon movies is ~200000, but I didn't start with this value) to give me those 2000 points.
DrD2k9: See the reply to ViGadeomes.
DrD2k9 wrote:
As far as using the rank calculation system to try to encourage authors to update older runs, I don't agree. I don't think anyone should be penalized for a long lasting published TAS.
I don't want to penalize people for long lasting runs, I want them to make new movies. Inactivity should be penalized. Maybe the factor could be set back to 1 after 20 years, to account for "perfect" (not all old runs are perfect, of course) runs.
feos: I agree, and have nothing against getting rid of the concept of player points althogether.
As far as using the rank calculation system to try to encourage authors to update older runs, I don't agree. I don't think anyone should be penalized for a long lasting published TAS.
I don't want to penalize people for long lasting runs, I want them to make new movies. Inactivity should be penalized. Maybe the factor could be set back to 1 after 20 years, to account for "perfect" (not all old runs are perfect, of course) runs.
(bold is my highlight)
This brings up the question of whether or not we, as a community, want to reward(penalize) activity(inactivity) as a whole. So, are you suggesting that currently active members are worthy of increased player points over members who are no longer active? (I'm not suggesting this opinion would be wrong; it's just a possible perspective and you're welcome to it if that's your opinion.)
To me, though, that would be suggesting that those who have come before--but are now no longer active--are somehow less worthy of a given score just because they aren't active anymore. Essentially, I feel it suggests that a given achievement is less worthy of recognition simply because the person who accomplished it is no longer active. This doesn't make sense to me; it'd be like saying an athlete that holds a world record is due more praise if they are still competing than if they were no longer competing. Whether or not they are still active doesn't change the fact that they hold a world record. To me, we shouldn't say that someone's work is worth more just because they are active as opposed to inactive.
Take Mothrayas, for example, who has recently chosen to take a step down in his level of activity. I don't think he should be penalized for that decision, because I don't know the life circumstances that led to him making that decision. Even if he were to never be involved again, I believe he's worthy of the full recognition for all his contributions/achievements.
A further problem, in my opinion, with altering the factor based on time since publication is that it can still penalize active members, not just inactive ones. If any penalty to a single publication's rank took place simply due to the passing of an arbitrary amount of time, the author(s) would be penalized regardless of how active they have or haven't been on the site. That member may have gotten 300 more published TASes of other games in that time frame, but would still be penalized because they didn't update that one game. So their personal player score would still suffer even though they've been plenty active.
Granted, all of my thoughts here are based on my perspective that our player rank/score here is more akin to an Xbox's gamerscore than it is a system of actually ranking our members in how good they are at TASing. If others actually view them as a way of ranking who's currently the 'best' or at the 'top of the leaderboards' then my thoughts are moot, and it would make sense to penalize inactivity.
Personally I don't think player points have much if any value. They do seem to do a poor job representing a TASer's body of work for the reasons TASeditor mentioned. I'd rather just have a direct link to a person's published works in that space instead.
Joined: 8/14/2009
Posts: 4083
Location: The Netherlands
TASeditor wrote:
So, why would anyone be stupid enough to spend 3 years on a project to just get ~150 points, if the same person could do a project that takes a week and gives just as much points?
I hope nobody on this site makes any decisions on TASing projects based on the amount of player points they would receive from them.
TASeditor wrote:
I don't want to penalize people for long lasting runs, I want them to make new movies. Inactivity should be penalized. Maybe the factor could be set back to 1 after 20 years, to account for "perfect" (not all old runs are perfect, of course) runs.
DrD2k9 wrote:
If others actually view them as a way of ranking who's currently the 'best' or at the 'top of the leaderboards' then my thoughts are moot, and it would make sense to penalize inactivity.
For anyone who's interested in player points as a more active ranking measure, a few years ago I set up a leaderboard of sorts that only takes into account movies from within a certain timespan (two years). It does still use the same player point formula, with the same flaws and benefits.
That being said, I agree that the player points formula is far from perfect, but I also don't know if there is any sane way to quantify effort put into a TAS that would not be exploitable in some way. Movie length, rerecord count, creation timespan etc. are all very flawed indicators that would provide a misleading picture, and would be easy to exploit.
http://www.youtube.com/Noxxa
<dwangoAC> This is a TAS (...). Not suitable for all audiences. May cause undesirable side-effects. May contain emulator abuse. Emulator may be abusive. This product contains glitches known to the state of California to cause egg defects.
<Masterjun> I'm just a guy arranging bits in a sequence which could potentially amuse other people looking at these bits
<adelikat> In Oregon Trail, I sacrificed my own family to save time. In Star trek, I killed helpless comrades in escape pods to save time. Here, I kill my allies to save time. I think I need help.
Personally I don't think player points have much if any value. They do seem to do a poor job representing a TASer's body of work for the reasons TASeditor mentioned. I'd rather just have a direct link to a person's published works in that space instead.
The primary benefit I see in having the points visible is as a quick indicator of TASing experience.
For example, if I saw two conflicting opinions on some TASing topic, I'd be more apt to accept the opinion of someone with a higher point total than that of a lurker.
It doesn't mean the more experienced person is always right, or that lurkers are wrong. It's just that the high point TASer has more evidence of experience to make whatever claim they are.
To put it simply. I see it as akin to experience points (or gamerscore), not as a true ranking mechanism. It's also for that reason I don't like the idea of losing a ton of points when a run is obsoleted by someone else.
If we wanted the score to simply show experience, we could simplify the point calculation to something along the lines of the following:
For current publications: every game would start with a base value regardless of game length and then the game length (while potentially exploitable) would still factor into the score Z=(length of publication in frames)/3600*(# of players)+100
Then a member's score would be (the sum of all Z's for currently published runs) + (the value of obsoleted games see below)
Regarding obsoleted videos. As much as I don't like the idea of losing drastic amounts of experience points for an obsoleted run, I also don't think a longer run should be worth more points than a shorter run of the same game. So in the case of a run being obsoleted, I'd recommend a couple caveats.
1) If a run is obsoleted by the same author, the author only gets the points of the published run for the given branch.
2) If the run is obsoleted by a different author, the author of the obsoleted run gets the point value of the shorter run which in theory should always be equal to or lower than the published run (some obsoleted runs may actually be shorter due to publication prior to site clarification of game endpoints--in these cases, the author of the longer published run deserves the higher point value than the author of the shorter obsoleted run).
This calculation would eliminate any voting/rating/popularity impact on the player's experience point calculation for a given game (which also means perceived entertainment value would not be considered for indicating player experience). This would eliminate poor scores due to low quantity of rating/voting or inflated scores due to game popularity. This would also eliminate player's experience level from changing due to a game being moved up/down a tier. Highly experienced TASers can still make boring videos, and conversely inexperienced TASers may still be able to produce something entertaining. How well a run is perceived by the community from an entertainment perspective is not an indicator of how experienced a TASer is or isn't.
EDIT: This approach would also eliminate the "Former Player" rank and give credit to those who have accomplished things before even if they have no currently published runs.
A further problem, in my opinion, with altering the factor based on time since publication is that it can still penalize active members, not just inactive ones. If any penalty to a single publication's rank took place simply due to the passing of an arbitrary amount of time, the author(s) would be penalized regardless of how active they have or haven't been on the site. That member may have gotten 300 more published TASes of other games in that time frame, but would still be penalized because they didn't update that one game. So their personal player score would still suffer even though they've been plenty active.
With the player point degrading I want do archieve exactly that, encouraging people to improve their old TASes. How long do people need to take a break in order to call the inactive? I used 6 - 7 years for time without penatly, because in that time span I would see people as active.
DrD2k9 wrote:
For current publications: every game would start with a base value regardless of game length and then the game length (while potentially exploitable) would still factor into the score Z=(length of publication in frames)/3600*(# of players)+100
I don't like using like using
movie length as a factor into the score. I have movies that are shorter and worked much longer on than movies that are longer.
DrD2k9 wrote:
Regarding obsoleted videos. As much as I don't like the idea of losing drastic amounts of experience points for an obsoleted run, I also don't think a longer run should be worth more points than a shorter run of the same game. So in the case of a run being obsoleted, I'd recommend a couple caveats.
1) If a run is obsoleted by the same author, the author only gets the points of the published run for the given branch.
2) If the run is obsoleted by a different author, the author of the obsoleted run gets the point value of the shorter run which in theory should always be equal to or lower than the published run (some obsoleted runs may actually be shorter due to publication prior to site clarification of game endpoints--in these cases, the author of the longer published run deserves the higher point value than the author of the shorter obsoleted run).
Aren't obsoleted runs usually longer or do you mean how long since the movie has been published?
To 1): It should only count if the author is current author for the branch and only the movies he/she made him-/herself are chained in a list example:
Author A 20 pts
Author A 40 pts
Author B 30 pts
Author C 30 pts
Author A 40 pts
Author A 50 pts
Author A would get 90 points, he doesn't get the additional 60 points, because his obsoletion chain was interrupted by different authors. Author B and C get 0 points.
To 2): I don't agree with that if an author gets obsoleted by a different author the author of the obsoleted movie should get player points aswell.
the only thing i'd like to contribute is obsoleted movies should pass their points to the new movie somehow, and yeah i agree with the distortion suggested by the thread creator but i don't think anybody cares about the points or that having a way of measure effort is possible
TAS i'm interested:
megaman series: mmbn1 all chips, mmx3 any% psx glitched
fighting games with speed goals in general
2) If the run is obsoleted by a different author, the author of the obsoleted run gets the point value of the shorter run which in theory should always be equal to or lower than the published run (some obsoleted runs may actually be shorter due to publication prior to site clarification of game endpoints--in these cases, the author of the longer published run deserves the higher point value than the author of the shorter obsoleted run).
Aren't obsoleted runs usually longer or do you mean how long since the movie has been published?
Yes, obsoleted runs are usually longer. But with now clarified rules on when a game ends, it's possible that the obsoleted run could be shorter. It's not going to be a common thing though.
But all my recommendations (which are geared toward indicating general TASing experience) are moot since Nach clarified the purpose of the player points to be an indicator of who's doing the most entertaining of the audience.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
DrD2k9 wrote:
But all my recommendations (which are geared toward indicating general TASing experience) are moot since Nach clarified the purpose of the player points to be an indicator of who's doing the most entertaining of the audience.
I would like to add if you also want to know who is doing the best job recently, Mothrayas put together an excellent leaderboard: Wiki: Mothrayas/Rankings.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
