Forum:Planning A Change To The Way User Reputation/Rankings Are Computed. Comments/Feedback Sought.

Question

[Istvan Albert]

We are planning to introduce a change to user reputation/rankings. This is to reward new and active users and to better reflect the ongoing activity levels for each user. The underlying problem that we are trying to fix is that even inactive users can maintain their rankings over long periods of time whereas newcomers face the nearly impossible task of catching up with someone that has been around for a lot longer.

The change is to move the current reputation points to a so called lifetime reputation that is to be computed the same way as it is today.

This lifetime reputation will be displayed on each user's profile page.

Then there will be an active reputation that is displayed in the user link and is used to rank users. This active reputation will be computed for the votes received over the last six months. Moderation rights will be automatically granted based on active reputation.

We're also considering moving away from purely numerical displays and instead display a little 'flair' icon next to the user's name to indicate both reputation but also other characteristics for that user: basically someone very active in the last would get a small icon of one kind, someone that has not been on the site for a long time would get another etc. With that we'd like to communicate more information on the user.

Feedback/comments are sought.

[SES]

I like the proposed changes, but I'm not sure I understand what the generic 'flair' adds. Seeing the reputation score is already an indicator of activity, and the badges indicate that someone has asked questions, etc. I think it would be more descriptive to see that someone has a gold badge for 'samtools' or 'perl' rather than just a generic badge for activity. I have read about people getting jobs from SO, and it seems like the tag badges would be very helpful for people trying to identify job candidates, and of course it would give users another way to reflect their expertise (e.g., for job applications).

[Istvan Albert]

yes, that is in the same spirit that I am thinking, to provide some extra information that otherwise would be difficult to assert

[Damian Kao]

Is not being able to catch up to high scoring users a significant reason why people are not active though? I guess I could imagine users being intimidated by huge numbers next to some of the long-time users. Designing a flair to indicate tiers to score is a good solution I think.

Answer 1

[Chris Miller]

I don't really understand the motivation. If someone has been a valuable contributor for 3 years and then takes a 6-month hiatus, that doesn't tarnish their reputation or make them less of an expert.

If you want to highlight up-and-coming members, add some options to the "users" page so that you can sort by "all time reputation", "reputation over last 6 months", etc. Flair is fine, I guess, as long as it's small. The pages are beginning to get really cluttered as is.

[Neilfws]

Completely agree with Chris's post. I don't see the need to distinguish users based on how long they've been around or help them "catch up".

[Rm]

@chris: i like this

[Istvan Albert]

this is a valid point - the motivation is really to have ways to encourage people to contribute more and not feel that it would be impossible to get ranked highly on the top user page.

but perhaps as you and Eric Normandeau says this is really a matter of sorting/displaying users.

as to flair, I was hoping to remove the numbers and this simplify it, instead of number like 9340 it would be replaced by a flair that corresponds to reps say between 8K and 10K.

Answer 2

[Dan D]

The way you describe it, it seems like you want to represent two separate concepts: reputation and ranking. But the plan as described mingles the two as different takes on the same "reputation" idea.

Here's how I think of it: Joe Frazier's reputation is that of a Hall of Fame boxer who substantially reduced the IQ of anyone who stepped in the ring with him. Joe Frazier's ranking is something totally different because he is not an active professional boxer. They're two distinct things.

An "active reputation" is just a more confusing way to say "ranking."

It seems like you're trying to overcome the power law effect, and I understand your motivation if that's the case, though I personally don't agree with it. Adding a scoring metric based on recent activity provides a gamification angle that encourages participation and healthy competition. I think the reputation score is a quick and easy way to tell, well, reputation. It's a quick indicator of how much a person has contributed to the site. I don't really care how active a particular person is, and I'm not going to consider one answer to be better than another based on someone's reputation. But it's your site. I think the key question is which of these you want to emphasize, and why.

[Istvan Albert]

that is a good point and helps clarify the concepts and what I am after

[matted]

So is the actual goal getting some new blood as moderators, which is currently hard because long-term users have a reputation advantage? Or is the goal providing encouragement for new users by letting them rise in the rankings, without waiting (possibly) years?

The first goal seems okay, though I don't know how much of a change (or improvement) it would make to the site. The second goal doesn't seem like it matters too much, at least to me. I find that most of the time I judge answers/comments by their quality, not the reputation of the user. Most good answers are long enough where their quality (or accuracy) stands alone, rather than only believing it because it's from a high-reputation user. The few exceptions are names where I can attach the user to their publication record, but that's because of the name, not the reputation score.

Or maybe I'm missing other possible downstream goals of this change...

[Istvan Albert]

In the a mixture of both, the rankings seem to be "frozen" and I can see that effect solidifying in time. having a way to rank users by their recent amount and quality of contributions would be a good way to show a different aspect of the site.

but as Deedee pointed out it would be best to untangle the reputation from the ranking - and I think I am going to go with that option where the default sort is by recent contributions rather than overall rep

[Chris Miller]

As an alternate solution, how about a little piece of flair that designates a user as up-and-coming. Maybe a little flame, designating that a specific user is "hot"?

Answer 3

[Eric Normandeau]

Using pretty much what you suggest, you could use an approach that is similar to the one the 'top' command implements to display the load average: Last6MonthsRep : allTimeRep, or Last6MonthsFlair : allTimeRep.

Example:

###########
# Userpic #
###########
created X minutes ago
by Whomever
540/4200

However, I am not sure what this will add to the user experience. I think that the lifetime Rep is in fact a good estimator of the value of a contributor, while I am not sure what the Last6Months Rep tell. 6 months can be long if the person has contributed 2 months and disapeared or short for users that have been contributing for 3 years.

You could also multiply the rep by 1 if it was in the last 6 months, 0.5 for the 6 previous ones, 0.25 for the 6 previous ones, 0.125 ... This would give a decreasing value to older rep, while still making it having some weight.