Karma-Based Mailing Lists (or: how to automate a meritocracy)

Standard

Defining the Problem

The problem with mailing lists is that they are a free for all, it doesn’t matter who posts, everybody at every level gets to see the post.

In the real world, communications pass through a hierarchy of people, escalating as necessary, passing from person to person up the chain.

This means that, given enough time, any mailing list starts to have a large noise:signal ratio, at least for any given person’s take on the list;
they want to read what they want to read, and don’t need to be distracted ignoring the stuff they don’t want to read.

Solving the Problem

There is an unspoken — some what hap-hazard — hierarchy among the community, which with some thought, I believe, could be defined, refined
and utilized to our advantage. As an example:

  - Active internals contributors with access to internals CVS (contributions of code and useful discussions)
  
   - Active internals contributors without access to internals CVS (patch submitters, useful discussions)
   - Inactive internals contributors with access to internals CVS (previous contributions of code and useful discussions)
      
      - Active non-internals, PHP contributions (docs, phpweb, PEAR)
      - Active community leaders
      - Active project leaders
      - Active linux distro maintainers
        
        - General Users with a high understanding
          
           - Genernal users with little understanding (newbies)

If we take each of these, and assign them a number:

L1  - Active internals contributors with access to internals CVS (contributions of code and useful discussions)
    
L2   - Active internals contributiors without access to internals CVS (patch submitters, useful discussions)
L2   - Inactive internals contributors with access to internals CVS (previous contributions of code and useful discussions)
        
L3      - Active non-internals, PHP contributions (docs, phpweb, PEAR)
L3      - Active community leaders
L3      - Active project leaders
L3      - Active linux distro maintainers
          
L4        - General Users with a high understand
            
L5           - Genernal users with little understanding (newbies)

Now, what if, at any level, you could only see (by default) 1 level below you (and all levels above you). For example: L1 can see L2, L2 can see L3 etc.

This immediately means that you only see stuff that might be relevant to you; however, as a community, we then lose the ability for newcomers to contribute good ideas; because they would start out with zero karma. To help solve this issue, we adjust karma based on responses:

Scenario

  • A L3 user posts something of interest
  • A L2 user see’s the post and replies, the reply is L2
    • This bumps the original post up to L2 as well
  • A L1 see’s the post now, and can then participate in the discussion if they choose

In this case, only the thread in question is bumped up, however given enough L2/L1 (weighted) direct responses over different threads, a L3 user can gain karma and eventually become a L2 user (and obviously this applies to anyone moving up the chain)

In this way, threads (and by this mechanism, users also) can organically make their way up the tree as they gain traction, are discussed at each level and moved up.

I believe it would be possible to have a ”’single”’ mailing list that could span everything from internals right down to php-general, but this is probably not desired! It would however allow the users of any list to, regardless of their experience in the tree, contribute without weighing down the list.

Features

  • Karma tree seeded by current “social” climate
    • based on CVS access level, activity in a sliding timescale, community contributions etc
    • Some manual work will be needed on this
  • Weighted responses, a L2 responding to a L3, will move it to L2, but 3 L2’s or 1 L2 and 1 L1 responding would move it to L1, for example.
  • Adjustable threshold. Perhaps some magnanimous internals contributor likes to help out newbies, he can choose to see the whole tree, or perhaps just 3 levels down
  • Championing — it should be possible for a user to champion someone to quickly move them up the ladder, for example a L2 can bring a L5 up to L3, so their peers can see stuff, this is bringing the user up, not the thread
  • Continued tweaking of karma based on CVS access and contributions
  • Personal filters, you can add users (of any level) from whom you would like to see threads, helping the movement of stuff up the tree, skipping levels

In this way, we can, in some ways, automate the karma, and in someways advance it through our own choices; creating a hierarchy based on merit and trust.

Final Thoughts

  • Some smart filtering, so that “You’re an idiot” responses don’t elevate a thread would be good
    • That, or just handle it biologically — Don’t respond to people in negative ways
  • Implementing this as a ML (in terms of interaction) is likely the only way to get some of the higher internals folks using it, a web interface [for the messages] just won’t fly
  • Personal filters would be handled at master.php.net or maybe a new web interface for this

Now, obviously, this is a huge undertaking; certainly not one any single person could complete on their own… but it’s food for thought.

– Davey

4 thoughts on “Karma-Based Mailing Lists (or: how to automate a meritocracy)

  1. I think the filtering has to happen on the receiver side and shouldn’t be done by the server. Every receiver has his reasons for reading the list and his interests. Deepending on that other comments might be relvant.

    For somebody with a “maintainer” role it might be important to get all reports about possible problems – independently who the sender is. Some distributor might only be interested in bigger decisions and focus on “the big names”, some lurker might be interested in some of the big stuff, that is not too technical, and in some beginner questions where he can (finally) answer. Just to name a few ones.

    Of course there are uninteresting things, but every modern mail client should be able to create filters to ignore specific threads and/or persons and highlight others and on that list it’s not like there was such a fluctuation on the list that you can’t recognize names and decide yourself.

    Oh and with your approach there’s the problem that internals handles quite some “special interest” stuff – like a patch to some “less important” extension, that might be send by a “L5″ and not be interesting for anybody but the “L1″ who is maintainer of that code.

    I certainly don’t like anybody else to decide which mails I get to read. :-)

  2. I think this idea of yours is anathema to the spirit of open source development. You should read “The Cathedral and the Bazaar” by Eric Raymond. You are attempting to codify elitism, and change the bazaar into a cathedral.

  3. Johannes summed it up nicely, I think the chance that a L5 posting ends up at L1, although interesting, would be near zero – at least given the current state of php mailing lists.

    Putting the threshold to -2 instead of -1 would make more sense, but I still don’t think it would be a good idea. Not from a social view, and also technically it would be.. say, interesting, to implement that given the frequent switches between your L2 and L3.

    New people would be highly discouraged as well if they can’t make the jump from L4 to L3/L2 – and I’m not only talking about internals (what you maybe had in mind as well) – I’m only a reader there, sitting on L3 myself, but I have the feeling the “bogus” +1/-1 votes are easily discarded by the decision makers and I don’t think there’s much noise to begin with.

  4. I’d suggest that this is a people problem, and a very decent solution to this exists already.

    If it’s a stupid post, which is more or less just a troll/flamebait? Delete key!

    Uninformed, passionate post which is wrong? Calm responses, show that everyone on each side of the conversation is human; and… then the Delete key!

    I do agree that you need a way to show an outsider that they are arguing with the creator of PHP about how feature X works, or other similar scenarios. But I don’t think baking that into a tool in a manner similar to a spam filter would be healthy.

    Case in point:
    Shelley Powers vs HTML5 / Ian Hickson – Microformats and data.

    Ian is a huge controlling aspect to what gets put into the HTML5 spec, as he’s the one with his fingers on the keyboard.

    Shelley is an advocate for RDF / RDFa. RDFa is a standard for microformats in XHTML, so not unreasonably, she (and many, many others) suggested its use in HTML5.

    This situation devolves pretty quickly. Ian appears closed on the idea of RDFa; but after much back and forth a new section appears in the spec. It’s not microformats, it’s not RDFa. All efforts to suggest that this is wheel reinvention seem to be ignored.

    If Shelley’s ability to provide criticism in the same venue as others was restricted by karma as you described above, much of the community around HTML5 would be unaware of this issue.

    Because of the tension between her and some of the core contributors, she’d be out on her behind in an instant. The issue would never have escalated, and I’d be unaware of it; I’d have no chance to contribute to HTML5 / use cases for RDFa.

    In these situations, I don’t think you need tools to manage noise.

    You need people to manage people (sooth, clarify, cajole), clear rules (don’t X, Y, and show a little respect!), and a way to get rid of the annoyance that is less costly that creating the annoyance (delete key).

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>