Lack of statistical normalization (in tag badges) of merit system penalizes effort in favor of popularity

I don't think this is a wholly bad idea, as long as the relationship is sub-linear.

If a tag has 1/1000 the traffic of Java, perhaps needing only 1/10 or 1/20 the score is reasonable. (Perhaps, low traffic tags need 1/10 the score that is currently needed, and high traffic tags are changed to require twice what they now need, with medium traffic tags being somewhere between 50-100% of the existing requirement).

What can't ever work is doing it in linear proportion to tag popularity. That would make it far too easy to make up a new tag and immediately gain a badge in it.

I have pondered this - it's not just Stack Overflow though. Some of the quieter Stack Exchange sites can take a long time to get there too.

The idea is that with a good track record of contribution, privileges are unlocked. In some cases, that's reputation points based (for example, delete and vote to close). The gold/silver/bronze tag badges are similar, but they are geared to per-tag score instead. (Which is almost the same, given each upvote is worth 10).

So the idea is: You get a gold badge when you've a good track record of solid answers and expertise in a topic. You get more moderator tools as your contribution to the site as a whole progresses.

But I've also wondered if there's scope for reward scaling. I mean, part of the point of gamification is to steer behaviours. But as it stands, answering an easy question, quickly, in a popular tag will get you more recognition than a well-thought-out answer on an old question in a quiet tag.

I notice the trend - the answers that I personally feel are my best contributions... don't do so well. It's been mentioned a few times that 'fastest gun in the west' is the way to 'win' Stack Overflow.

So the suggestion I'd offer is to perhaps reward a reputation points bonus that's triggered based on:

• Question score vs. median score. (Question must beat 'median' question score).
• Answers that beat 'median' score for questions that this applies to, get a bonus.

I'm not sure how this'd work on multi-tagged questions though (the obvious would be 'take worst case', and accept that means if you tag 'java' along with 'veryobscurething' it'll suffer a bit, but more eyes will see it, so it probably balances out). Nor do I know if there's a solution in terms of score-based badges. I assume they're score based rather than reputation points based for a reason.

Or perhaps it's as simple as an adjustment based on average-time-to-gold, given earning rates of the active users in the tag.

I think any normalization would have to be done very carefully with a solid statistical basis.

More popular tags have greater competition. In theory multiple great answers might all get upvotes, but in practice many people answer with similar answers but only a couple get votes, pushed to the top, and the rest largely ignored.

A tag with more questions doesn't necessarily enjoy the same proportion of more votes per question. A user is still only going to visit and vote on a certain number of questions per day on average. If there are more users in a tag, there probably are more questions and answers among which to divide their time spent voting. Unless it's a question that comes up in google alot for common issues, then it's not likely to get a disproportional amount of attention. I've seen plenty of great answers in popular tags that don't have more than a couple upvotes, the asker's upvote, and maybe one or two other people who were involved in the question at the time it was answered.

Lastly, older popular tags tend to have more questions that deal with troubleshooting some strange issue. It's harder to find questions that are asking about a feature that is core to the technology, that someone who is simply knowledgeable of the field can quickly answer. Instead, you get questions where answerers must spend more time intimately familiarizing themselves with the asker's specific scenario.

If anything, IMO it's easier to do well in a younger less popular tag. Assuming you are well versed in the technology, you'll have an opportunity to get in there and answer those core questions that future beginners will revisit many times and feed you upvotes. Granted it will take awhile.

I don't dismiss your ideas in whole though. I just want to emphasize that there's a great deal of factors in play. It's a bit like economic theories. We can model it all day based on our assumptions, but in practice we are probably way off.

It shouldn't be too easy to get a Gold badge in a currently unpopular tag. It's a bit unfair to late comers who might, in the long run, have much more to offer. It could also be gamed to give people more power than they perhaps should have.

I would not only look at current statistics, but take a retrospective look at what now popular tags looked like when they were younger. Consider if the new rules were in place at that time, take individuals who would have gotten a gold badge as-per the new rules at that time. How many of them would have gotten the gold badge then, but do not have the badge today, because they have never met today's standards. Would they have deserved that badge? Did their early contributions warrant them getting the badge then, even though their current contributions don't meet today's standards? Just doing that analysis on a case by case basis might reveal some insights that might lead one to adjust the criteria further, abandon wholesale, or raise more questions that need further study.

Additionally, it could be like chasing an ever moving goal. Depending on the formula you use, you might have to contribute at an ever increasing rate which is greater than the growth of the tag's popularity.

I certainly like the idea of having shared knowledge be rewarded for its own sake, without regard to how popular the subject is. Academics was fun! From this perspective, a Ph.D, no matter how arcane the mathematical field, should be worth a great reputation.

Another view is more market oriented: you can work hard and be brilliant at a failed startup, and you get no street cred ("you worked for who?) or reward (options value = $0.00). From this perspective, how much you can help others (how applicable your answer is to the current market) matters - and street cred comes from knowing a lot about a topic people care about.

So I guess the question is - is this a market oriented site, where reputation is a measure of your worth to others, or is it an academic site, where reputation should be based on a less measured representation of value.