Niche quality control

Quick note about niche approvals in general.

It may be that the prerogative was initially to get as many approved niches as fast as possible.

I certainly get that impression with niches that the Tribunal is approving in appeals, and the way Team members are voting.

I think it would be healthy to strongly question this prerogative.

We now have 2128 approved niches.  These were achieved from a user base of less than 6000 members.  Niches are still being suggested at a steady rate.  I don't think we have to be afraid there will be a lack of niches suggested.  The niche verse will be discovered and made manifest - there should be no concern of this.  I think the platform could launch its Beta successfully with half as many niches.

These considerations should allow us to be a lot more conscious of niche quality, only approving niches that are good, as opposed to passable.  Narrative is supposed to be synonymous with quality.

In the community, we have some people who are really very good at suggesting good niches.  We should aim to use their strength.  Why settle for less?  We're not desperate for niches, and there is no upside to rewarding users for contributions that don't make the grade.  If we reject a substandard niche and explain why, the good niche writers can jump in and submit an improved version.  

The game theory the @Narrative Network Team has put into action is good: they've motivated people to submit niches that are good enough to get approved, and they get a rep bonus for this.  Ironically, we're failing to set the bar at a desirable level for what constitutes a good niche.  @MOLLY O described very well what a good niche needs, in her Medium blog about guidelines for niche suggestion.  But we're accepting niches that fall short of those guidelines, and the Tribunal is sometimes letting them through even when they are appealed.

I think we need to wake up.  The clarity, uniqueness and general viability of niches is possibly our greatest vulnerability when we launch.  With a new paradigm, new users will be disenchanted if the basics that other existing platforms have nailed, don't work on Narrative.  Do we want people to be distracted from the awesomeness of Narrative's new offerings, by our failure to get this new paradigm to deliver coherent categories for content, a characteristic we take for granted?  Do we want them to wince at spelling and grammar mistakes in descriptions?  It isn't easy to ask people to look beyond that sort of unprofessional impression.  It runs deep in us to trust platforms that are well organised and use correct spelling and grammar.  We have distrust and dislike of those that don't.

I remain dubious that cleanup efforts at the extremely busy time of Beta launch, would be a good idea.  A good system doesn't make mess that needs cleaning up later.

This is a call to arms to tighten the ship.  When the storm of Beta Launch hits, efforts to shore up these issues of basic organisation will strongly detract from what the Team should be doing after Beta Launch: rolling out technical fixes, tweaking the rep system, and developing the all important future functionalities like advertising, staking, etc.  Anything that detracts from those huge tasks will pull the network down.

Original Post

To be completely honest, I've allowed the way @Narrative Network Team members have voted to sway my vote, on occasion, even when I believed the niche was sub par.  Not often, but on occasion.

Why?  Because of reputation.  We see the Tribunal members voting on niches, and we know they are going to be the ones fielding any appeal of the community decision to approve or reject the niche.  So their votes become strong predictors of what is going to be considered the 'right' decision.

I'm speaking up about this because we need to correct this flaw in the system.

Either the Tribunal members should not be voting on approvals, AND/OR the Tribunal should decide to strictly adhere to the guidelines @MOLLY O drew up in her excellent blog post.  The second option is the best, IMO, but both could be helpful.

If Molly's guidelines are strictly enforced, it creates a communally shared definition of the criteria for a niche to pass muster, and our results will improve significantly.  As long as they are not enforced, the decisions of the Tribunal will appear arbitrary, springing from a fount of wisdom us community members do not understand the mechanics of.  Worse, from a fount of wisdom that seems to ignore or negate @MOLLY O's article.  How do we expect users to follow these guidelines, if the Tribunal does not enforce them?

The guidelines article is a great resource, and as long as we're not letting it guide our efforts, we're squandering that resource needlessly.

This change has to happen from the top down.  If the company does not make this decision, communicate it clearly, and hold the Tribunal to enforcing it, nobody should expect anything to change.  The Tribunal holds complete power over the final decisions made on niche approvals.  If they do not follow the criteria we've all been given, not only will nothing change, but the power of reputation will cause some of us to vote as they do, compounding the problem.  Confusion over how to vote will only grow, the more the value of reputation is realized by the community.

We're beginning to see that the power of reputation is significant, and it can have a dark side unless the final arbiters of what is positive for the network make a bigger effort towards consistency.  The first step towards that is to enforce the guidelines.

I'm with you, @Malkazoid. I'm the only naysayer on some niches up for approval, and it makes me wary of voting against the way Tribunal members do because I don't know if voting with the majority in any way affects my own reputation. As such, I've changed a few of my votes, wondering if I've been too harsh. But I've done content moderation for over a decade, and user moderation for a while before that, and about 25% of the approved niches up for auction make me want to cringe. (Maybe that percentage is hyperbole, maybe not.) 

Malkazoid posted:

 

Either the Tribunal members should not be voting on approvals, AND/OR the Tribunal should decide to strictly adhere to the guidelines @MOLLY O drew up in her excellent blog post.  The second option is the best, IMO, but both could be helpful.

If Molly's guidelines are strictly enforced, it creates a communally shared definition of the criteria for a niche to pass muster, and our results will improve significantly.  As long as they are not enforced, the decisions of the Tribunal will appear arbitrary, springing from a fount of wisdom us community members do not understand the mechanics of.  Worse, from a fount of wisdom that seems to ignore or negate @MOLLY O's article.  How do we expect users to follow these guidelines, if the Tribunal does not enforce them?

 

I agree with everything @Malkazoid states her. There is no excuse for the tribunal to not be adhering to the very guidelines for quality and redundancy that they put out. If this format does not adhere to quality, then there will be no advertisers. If there are no advertisers there will be no content. Community is here to get paid for their content, and quickly via crypto. It is very simple.

Further supporting what Malkazoid has stated, I go a step further and suggest that now that we have reputation established, I see no reasons why the original tribunal should still be in place. They should be focused on meeting the deadlines of the roadmap.

How hard would it be to duplicate the voting section of Moderator and make it active for the Tribunal? I cannot imagine it taking more than a day or two, yet would free up time for each tribunal team member to focus on the important aspect of their job, that cannot be done by community members.

Time to elect non-narrative team members to the positions of Tribunal. This should become a immediate priority in my opinion.

Christina Gleason posted:

I'm with you, @Malkazoid. I'm the only naysayer on some niches up for approval, and it makes me wary of voting against the way Tribunal members do because I don't know if voting with the majority in any way affects my own reputation. As such, I've changed a few of my votes, wondering if I've been too harsh. But I've done content moderation for over a decade, and user moderation for a while before that, and about 25% of the approved niches up for auction make me want to cringe. (Maybe that percentage is hyperbole, maybe not.) 

This is exactly why the reputation, and how it is added and subtracted needs to be transparent!! It affects behavior that compromises the quality of the platform. A big lose for Narrative if it persists in cloaking a rather benign aspect. It does no harm for people to have knowledge on how to improve their reputation.

I am not in favour of the platform launching its Beta before this quality problem is resolved, both from a systemic viewpoint, and from a clean up of existing problems standpoint.

Systemically, all it takes is for the Tribunal to adhere to @MOLLY O's guidelines, and some messaging to the community that our votes should do so too.

A clean up isn't hard after that, either.  If the Tribunal promises to uphold the guidelines, we can appeal all the problem niches.

Correct spelling and grammar should be present too, or the niches should be rejected.  Launching with niches with these problems risks undermining ALL the other hard work the @Narrative Network Team and the Community have done, are doing, and will do.

The general public won't care how revolutionary Narrative may become - all they will see is our content areas fall far short of the quality we promised this place was about.  FAR short.  It becomes difficult to believe in the ambitious goals of Narrative, if the basics are found lacking.  Coherent categories, and correct spelling and grammar are the basics.  We are failing those right now, unnecessarily.

I hope the team acknowledges this when they return from their well-deserved holidays.  I hope they communicate their intentions on how to deal with it.  The jeopardy is real, and this is something the Community cannot fix without the Team taking heed and taking action.

 

Time to elect non-narrative team members to the positions of Tribunal. This should become a immediate priority in my opinion.

That would certainly solve the problem too.  I am pretty confident the community will be sensitive to an election campaign based on upholding the guidelines in pursuit of lifting the quality to the levels the platform set out to achieve.  

So the new Tribunal would be crewed by people who are committed to this goal.  

I agree that the current Tribunal members are probably all stretched very thin - which could partially explain their voting patterns.  Their time is certainly too valuable to be diverted to only partially doing justice to the Tribunal's role.

Holding Tribunal elections and the resulting community governance would be an incredible PR move too, as an added bonus.  We've fallen behind schedule for Beta Launch, but we could be ahead of schedule for Community Governance.

This may also solve the potential legal jeopardy for the company, of approving niche descriptions (product descriptions) that may later be deemed unfit for use on the network as described, AFTER a user has purchased the product.  Company members making these decisions is what creates the potential for liability.  But community members making the decisions may go a long way to removing any company liability.  

Anyway, either way, this quality problem is critical and must be solved.  There are several ways to achieve the quality standards - the Team should pick one rapidly and communicate it.

Part of the problem of low quality niches sure is caused by democratic quality control, which is an unfortunate side effect of something that’s good by nature. That’s where appeals should come in, and the Tribunal should only be satisfied with perfect niches. 

Personally, I believe Narrative users, including myself, will notice and appeal niche errors more often once content drafting starts. It’ll make errors way more apparent.

Emily Barnett posted:
Christina Gleason posted:

I'm with you, @Malkazoid. I'm the only naysayer on some niches up for approval, and it makes me wary of voting against the way Tribunal members do because I don't know if voting with the majority in any way affects my own reputation. As such, I've changed a few of my votes, wondering if I've been too harsh. But I've done content moderation for over a decade, and user moderation for a while before that, and about 25% of the approved niches up for auction make me want to cringe. (Maybe that percentage is hyperbole, maybe not.) 

This is exactly why the reputation, and how it is added and subtracted needs to be transparent!! It affects behavior that compromises the quality of the platform. A big lose for Narrative if it persists in cloaking a rather benign aspect. It does no harm for people to have knowledge on how to improve their reputation.

Hey Emily—wanted to provide you with a quick update on this. As Ted shared this morning, we do have plans to publish the entire Reputation spec, including the formulas once we feel that it is stabilized and working as intended. Likely within the first year post beta launch. 

Add Reply

×
×
×
×