Goodness is Not Enough

There are lots of rating systems out there. You have seen them. They show you a little set of radio buttons, 1 to 5 or bad to good. (They rather remind me, for all the choice and quality of expression they offer, of those terribly serious multiple choice quizes about a third of us bother to take every 4 years or so.) Anyway, these rating systems don't all seem to know it but they are implicity using the criterion goodness. Now, goodness is fine and all, maybe its even great. But goodness is not enough.

So we've had an Aha! experience. But the moment we start trying to figure out which criteria ARE enough, that is; which are the RIGHT criteria then we start to realize that it takes a lot of hubris to attempt to pick the definitive set of dimensions along which other people OUGHT to express their evaluations of things in general. And even if you are among the very wise; those so wise that they can lay out the lines for others thoughts, then it still might take a fair amount of time to accumulate all these brilliant criteria.

To we who are not so wise, it is becoming clear that it might make a bit more sense to just let the community itself take care of contributing the criteria. They can add new criteria as the fancy strikes. They can go on evaluation binges with their new criteria. They can really get involved. Others might see these new criteria and begin to appreciate their utility too. Some criteria might get to be quite popular indeed. Possibly because they get at the heart of some crucial distinction in some discipline. Or maybe because they treat with the very essence of say, versions with respect to one another. Examples of these inter-version include:

• clarity of expression
• accuracy
• brevity
• true to original intent

"Just a second." you might say. "Things are getting a bit out of hand. We used to have simple little boxes with 5 radio buttons in them and yes that was limiting (often verging on useless at times) but at least it didn't take up the whole screen with countless user contributed criteria! I could perhaps tolerate the half dozen criteria that I use most often being presented to me. And yes, when I was exploring an area new to me then maybe the most popular criteria in that discipline would be handy instead. And certainly, when I was seriously working on things of deep interest to me I would want to be able to use a broader range of criteria. I suppose I could rate criteria themselves, as a way of sorting them for display in various contexts or prioritizing their filtering impact. That sure is the snake eating its tail, isn't it! In fact, come to think of it, how I use criteria (or how I rate them) really says something about my way of seeing the world. I could even caricature myself by prioritizing the criteria I use and whose evaluations I heed in each area. It would even be handy to create caricatures of such easily characterized perspectives as Banker, Gandhi, SingleMom or other 'world views'. Whoa! Donning one of these WorldViews would be like looking through their glasses. I could go exploring familiar terrain with a new perspective or even go exploring the perspectives themselves."

Soon you'll have me thinking that this isn't such a bad idea! We might be able to piggy-back such a self-organizing peer review engine on the existing web by making a proxy for example. Or a virtual proxy like Crit.org. One sort of success would be incorporating it into Google or the Open Directory Project. Another path would be the creation of a standalone groupware package (such as the currenlty alpha software Nooron) which would manage richly typed content as well as providing an environment in which it improves.